2023-11-24T05:50:24.635 INFO:root:teuthology version: 0.0.1.dev205+g2442239 2023-11-24T05:50:24.635 DEBUG:teuthology.run:Teuthology command: teuthology --archive /home/teuthworker/archive/rishabh-2023-11-24_05:41:33-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7465546 --description fs:volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/basic}} --owner scheduled_rishabh@teuthology --name rishabh-2023-11-24_05:41:33-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi --verbose -- /home/teuthworker/archive/rishabh-2023-11-24_05:41:33-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7465546/orig.config.yaml 2023-11-24T05:50:24.692 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-24T05:50:24.849 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/rishabh-2023-11-24_05:41:33-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7465546 branch: rishabh-mgr-vol-misc description: fs:volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/basic}} email: null first_in_suite: false job_id: '7465546' kernel: flavor: default kdb: true sha1: 8247bbb5b470f7e2a1f3eefcc16e24cf1338f606 last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: rishabh-2023-11-24_05:41:33-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: rishabh-mgr-vol-misc ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug finisher: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - \(POOL_APP_NOT_ENABLED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - OSD full dropping all updates - OSD near full - pausewr flag - failsafe engaged, dropping updates - failsafe disengaged, no longer dropping - is full \(reached quota - POOL_FULL - POOL_BACKFILLFULL sha1: a13ea5ff00cb4df7a9cc941166ddeb358881144c ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: a13ea5ff00cb4df7a9cc941166ddeb358881144c thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: rishabh-mgr-vol-misc sha1: a13ea5ff00cb4df7a9cc941166ddeb358881144c owner: scheduled_rishabh@teuthology priority: 50 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 3305 sha1: a13ea5ff00cb4df7a9cc941166ddeb358881144c sleep_before_teardown: 0 suite: fs:volumes suite_branch: rishabh-mgr-vol-misc suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_a13ea5ff00cb4df7a9cc941166ddeb358881144c/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: a13ea5ff00cb4df7a9cc941166ddeb358881144c syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi092.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBM/6AgknJyX69rVjCUaIgHxhqMmdlXcpfMRywOSR//a3BdC53SvAjaxmI9LSZboTEgD8EzutTJWZfDUDZG21qko= smithi118.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDtNpRTYzJZfvzdUULaqo1CEoH8cnZH6jMFDwICFAvv6nLJJyaEH9q71cRsl+4OgPQO1/cgIH3Ch5W2Ttk4w7N8= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - ceph-fuse: null - cephfs_test_runner: fail_on_skip: false modules: - tasks.cephfs.test_volumes.TestVolumes - tasks.cephfs.test_volumes.TestSubvolumeGroups - tasks.cephfs.test_volumes.TestSubvolumes - tasks.cephfs.test_subvolume.TestSubvolume teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: 2442239d2653456406c25ae0c71b689c8f2657b6 timestamp: 2023-11-24_05:41:33 tube: smithi user: rishabh verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.232157 2023-11-24T05:50:24.849 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_a13ea5ff00cb4df7a9cc941166ddeb358881144c/qa; will attempt to use it 2023-11-24T05:50:24.850 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_a13ea5ff00cb4df7a9cc941166ddeb358881144c/qa/tasks 2023-11-24T05:50:24.850 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-11-24T05:50:24.852 INFO:teuthology.task.internal:Checking packages... 2023-11-24T05:50:24.884 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'a13ea5ff00cb4df7a9cc941166ddeb358881144c' 2023-11-24T05:50:24.884 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-11-24T05:50:24.884 INFO:teuthology.packaging:ref: None 2023-11-24T05:50:24.884 INFO:teuthology.packaging:tag: None 2023-11-24T05:50:24.884 INFO:teuthology.packaging:branch: rishabh-mgr-vol-misc 2023-11-24T05:50:24.884 INFO:teuthology.packaging:sha1: a13ea5ff00cb4df7a9cc941166ddeb358881144c 2023-11-24T05:50:24.885 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=rishabh-mgr-vol-misc 2023-11-24T05:50:25.109 INFO:teuthology.task.internal:Found packages for ceph version 18.0.0-7484.ga13ea5ff 2023-11-24T05:50:25.109 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-11-24T05:50:25.115 INFO:teuthology.task.internal:no buildpackages task found 2023-11-24T05:50:25.115 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-11-24T05:50:25.121 INFO:teuthology.task.internal:Saving configuration 2023-11-24T05:50:25.140 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-11-24T05:50:25.146 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-11-24T05:50:25.170 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi092.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-11-24_05:41:33-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7465546', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-24 05:45:04.588550', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBM/6AgknJyX69rVjCUaIgHxhqMmdlXcpfMRywOSR//a3BdC53SvAjaxmI9LSZboTEgD8EzutTJWZfDUDZG21qko='} 2023-11-24T05:50:25.197 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi118.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-11-24_05:41:33-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7465546', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-24 05:45:04.590485', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDtNpRTYzJZfvzdUULaqo1CEoH8cnZH6jMFDwICFAvv6nLJJyaEH9q71cRsl+4OgPQO1/cgIH3Ch5W2Ttk4w7N8='} 2023-11-24T05:50:25.197 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-11-24T05:50:25.203 INFO:teuthology.task.internal:roles: ubuntu@smithi092.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2023-11-24T05:50:25.203 INFO:teuthology.task.internal:roles: ubuntu@smithi118.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2023-11-24T05:50:25.203 INFO:teuthology.run_tasks:Running task console_log... 2023-11-24T05:50:25.401 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f1c8e6bdb80>, signals=[15]) 2023-11-24T05:50:25.402 INFO:teuthology.run_tasks:Running task internal.connect... 2023-11-24T05:50:25.408 INFO:teuthology.task.internal:Opening connections... 2023-11-24T05:50:25.408 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi092.front.sepia.ceph.com 2023-11-24T05:50:25.411 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi092.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-24T05:50:25.538 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi118.front.sepia.ceph.com 2023-11-24T05:50:25.539 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi118.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-24T05:50:25.627 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-11-24T05:50:25.635 DEBUG:teuthology.orchestra.run.smithi092:> uname -m 2023-11-24T05:50:25.655 INFO:teuthology.orchestra.run.smithi092.stdout:x86_64 2023-11-24T05:50:25.655 DEBUG:teuthology.orchestra.run.smithi092:> cat /etc/os-release 2023-11-24T05:50:25.711 INFO:teuthology.orchestra.run.smithi092.stdout:NAME="CentOS Stream" 2023-11-24T05:50:25.711 INFO:teuthology.orchestra.run.smithi092.stdout:VERSION="9" 2023-11-24T05:50:25.711 INFO:teuthology.orchestra.run.smithi092.stdout:ID="centos" 2023-11-24T05:50:25.711 INFO:teuthology.orchestra.run.smithi092.stdout:ID_LIKE="rhel fedora" 2023-11-24T05:50:25.711 INFO:teuthology.orchestra.run.smithi092.stdout:VERSION_ID="9" 2023-11-24T05:50:25.711 INFO:teuthology.orchestra.run.smithi092.stdout:PLATFORM_ID="platform:el9" 2023-11-24T05:50:25.711 INFO:teuthology.orchestra.run.smithi092.stdout:PRETTY_NAME="CentOS Stream 9" 2023-11-24T05:50:25.712 INFO:teuthology.orchestra.run.smithi092.stdout:ANSI_COLOR="0;31" 2023-11-24T05:50:25.712 INFO:teuthology.orchestra.run.smithi092.stdout:LOGO="fedora-logo-icon" 2023-11-24T05:50:25.712 INFO:teuthology.orchestra.run.smithi092.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2023-11-24T05:50:25.712 INFO:teuthology.orchestra.run.smithi092.stdout:HOME_URL="https://centos.org/" 2023-11-24T05:50:25.712 INFO:teuthology.orchestra.run.smithi092.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-11-24T05:50:25.712 INFO:teuthology.orchestra.run.smithi092.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2023-11-24T05:50:25.712 INFO:teuthology.orchestra.run.smithi092.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-11-24T05:50:25.713 INFO:teuthology.lock.ops:Updating smithi092.front.sepia.ceph.com on lock server 2023-11-24T05:50:25.741 DEBUG:teuthology.orchestra.run.smithi118:> uname -m 2023-11-24T05:50:25.758 INFO:teuthology.orchestra.run.smithi118.stdout:x86_64 2023-11-24T05:50:25.759 DEBUG:teuthology.orchestra.run.smithi118:> cat /etc/os-release 2023-11-24T05:50:25.816 INFO:teuthology.orchestra.run.smithi118.stdout:NAME="CentOS Stream" 2023-11-24T05:50:25.816 INFO:teuthology.orchestra.run.smithi118.stdout:VERSION="9" 2023-11-24T05:50:25.816 INFO:teuthology.orchestra.run.smithi118.stdout:ID="centos" 2023-11-24T05:50:25.816 INFO:teuthology.orchestra.run.smithi118.stdout:ID_LIKE="rhel fedora" 2023-11-24T05:50:25.816 INFO:teuthology.orchestra.run.smithi118.stdout:VERSION_ID="9" 2023-11-24T05:50:25.816 INFO:teuthology.orchestra.run.smithi118.stdout:PLATFORM_ID="platform:el9" 2023-11-24T05:50:25.816 INFO:teuthology.orchestra.run.smithi118.stdout:PRETTY_NAME="CentOS Stream 9" 2023-11-24T05:50:25.816 INFO:teuthology.orchestra.run.smithi118.stdout:ANSI_COLOR="0;31" 2023-11-24T05:50:25.816 INFO:teuthology.orchestra.run.smithi118.stdout:LOGO="fedora-logo-icon" 2023-11-24T05:50:25.817 INFO:teuthology.orchestra.run.smithi118.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2023-11-24T05:50:25.817 INFO:teuthology.orchestra.run.smithi118.stdout:HOME_URL="https://centos.org/" 2023-11-24T05:50:25.817 INFO:teuthology.orchestra.run.smithi118.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-11-24T05:50:25.817 INFO:teuthology.orchestra.run.smithi118.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2023-11-24T05:50:25.817 INFO:teuthology.orchestra.run.smithi118.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-11-24T05:50:25.817 INFO:teuthology.lock.ops:Updating smithi118.front.sepia.ceph.com on lock server 2023-11-24T05:50:25.840 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-11-24T05:50:25.850 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-11-24T05:50:25.860 INFO:teuthology.task.internal:Checking for old test directory... 2023-11-24T05:50:25.861 DEBUG:teuthology.orchestra.run.smithi092:> test '!' -e /home/ubuntu/cephtest 2023-11-24T05:50:25.866 DEBUG:teuthology.orchestra.run.smithi118:> test '!' -e /home/ubuntu/cephtest 2023-11-24T05:50:25.883 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-11-24T05:50:25.890 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-11-24T05:50:25.890 DEBUG:teuthology.orchestra.run.smithi092:> test -z $(ls -A /var/lib/ceph) 2023-11-24T05:50:25.922 DEBUG:teuthology.orchestra.run.smithi118:> test -z $(ls -A /var/lib/ceph) 2023-11-24T05:50:25.949 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-11-24T05:50:26.007 INFO:teuthology.run_tasks:Running task kernel... 2023-11-24T05:50:26.029 INFO:teuthology.task.kernel:normalize config orig: {'flavor': 'default', 'kdb': True, 'sha1': '8247bbb5b470f7e2a1f3eefcc16e24cf1338f606'} 2023-11-24T05:50:26.030 INFO:teuthology.task.kernel:config {'mon.a': {'flavor': 'default', 'kdb': True, 'sha1': '8247bbb5b470f7e2a1f3eefcc16e24cf1338f606'}, 'mon.b': {'flavor': 'default', 'kdb': True, 'sha1': '8247bbb5b470f7e2a1f3eefcc16e24cf1338f606'}}, timeout 300 2023-11-24T05:50:26.030 DEBUG:teuthology.orchestra.run.smithi092:> test -f /run/.containerenv -o -f /.dockerenv 2023-11-24T05:50:26.030 DEBUG:teuthology.orchestra.run.smithi118:> test -f /run/.containerenv -o -f /.dockerenv 2023-11-24T05:50:26.046 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-24T05:50:26.047 DEBUG:teuthology.task.kernel:sha1 for mon.a is 8247bbb5b470f7e2a1f3eefcc16e24cf1338f606 2023-11-24T05:50:26.047 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "8247bbb5b470f7e2a1f3eefcc16e24cf1338f606"... 2023-11-24T05:50:26.047 DEBUG:teuthology.orchestra.run.smithi092:> uname -r 2023-11-24T05:50:26.049 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-24T05:50:26.049 DEBUG:teuthology.task.kernel:sha1 for mon.b is 8247bbb5b470f7e2a1f3eefcc16e24cf1338f606 2023-11-24T05:50:26.049 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "8247bbb5b470f7e2a1f3eefcc16e24cf1338f606"... 2023-11-24T05:50:26.049 DEBUG:teuthology.orchestra.run.smithi118:> uname -r 2023-11-24T05:50:26.103 INFO:teuthology.orchestra.run.smithi092.stdout:5.14.0-361.el9.x86_64 2023-11-24T05:50:26.104 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-361.el9.x86_64 vs 8247bbb5b470f7e2a1f3eefcc16e24cf1338f606 2023-11-24T05:50:26.104 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2023-11-24T05:50:26.105 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8247bbb5b470f7e2a1f3eefcc16e24cf1338f606 2023-11-24T05:50:26.108 INFO:teuthology.orchestra.run.smithi118.stdout:5.14.0-361.el9.x86_64 2023-11-24T05:50:26.109 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-361.el9.x86_64 vs 8247bbb5b470f7e2a1f3eefcc16e24cf1338f606 2023-11-24T05:50:26.109 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2023-11-24T05:50:26.109 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8247bbb5b470f7e2a1f3eefcc16e24cf1338f606 2023-11-24T05:50:26.329 DEBUG:teuthology.orchestra.run.smithi118:> sudo yum upgrade -y linux-firmware 2023-11-24T05:50:26.331 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum upgrade -y linux-firmware 2023-11-24T05:50:28.438 INFO:teuthology.orchestra.run.smithi092.stdout:CentOS-9 - AppStream 12 MB/s | 6.4 MB 00:00 2023-11-24T05:50:29.553 INFO:teuthology.orchestra.run.smithi118.stdout:CentOS-9 - AppStream 13 MB/s | 6.4 MB 00:00 2023-11-24T05:50:30.072 INFO:teuthology.orchestra.run.smithi092.stdout:CentOS-9 - BaseOS 3.4 MB/s | 2.0 MB 00:00 2023-11-24T05:50:30.954 INFO:teuthology.orchestra.run.smithi118.stdout:CentOS-9 - BaseOS 6.1 MB/s | 2.0 MB 00:00 2023-11-24T05:50:31.270 INFO:teuthology.orchestra.run.smithi092.stdout:CentOS Stream 9 - BaseOS 9.0 MB/s | 7.9 MB 00:00 2023-11-24T05:50:33.119 INFO:teuthology.orchestra.run.smithi118.stdout:CentOS Stream 9 - BaseOS 4.4 MB/s | 7.9 MB 00:01 2023-11-24T05:50:34.829 INFO:teuthology.orchestra.run.smithi118.stdout:CentOS Stream 9 - AppStream 29 MB/s | 18 MB 00:00 2023-11-24T05:50:35.371 INFO:teuthology.orchestra.run.smithi092.stdout:CentOS Stream 9 - AppStream 6.0 MB/s | 18 MB 00:02 2023-11-24T05:50:38.821 INFO:teuthology.orchestra.run.smithi118.stdout:CentOS Stream 9 - CRB 8.9 MB/s | 5.8 MB 00:00 2023-11-24T05:50:39.104 INFO:teuthology.orchestra.run.smithi092.stdout:CentOS Stream 9 - CRB 15 MB/s | 5.8 MB 00:00 2023-11-24T05:50:40.423 INFO:teuthology.orchestra.run.smithi118.stdout:CentOS Stream 9 - Extras packages 30 kB/s | 15 kB 00:00 2023-11-24T05:50:40.580 INFO:teuthology.orchestra.run.smithi092.stdout:CentOS Stream 9 - Extras packages 40 kB/s | 15 kB 00:00 2023-11-24T05:50:41.458 INFO:teuthology.orchestra.run.smithi092.stdout:Extra Packages for Enterprise Linux 27 MB/s | 20 MB 00:00 2023-11-24T05:50:42.617 INFO:teuthology.orchestra.run.smithi118.stdout:Extra Packages for Enterprise Linux 9.6 MB/s | 20 MB 00:02 2023-11-24T05:50:46.289 INFO:teuthology.orchestra.run.smithi092.stdout:lab-extras 15 kB/s | 1.7 kB 00:00 2023-11-24T05:50:47.372 INFO:teuthology.orchestra.run.smithi118.stdout:lab-extras 41 kB/s | 1.7 kB 00:00 2023-11-24T05:50:48.739 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2023-11-24T05:50:48.751 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2023-11-24T05:50:48.751 INFO:teuthology.orchestra.run.smithi092.stdout: Package Arch Version Repository Size 2023-11-24T05:50:48.751 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2023-11-24T05:50:48.752 INFO:teuthology.orchestra.run.smithi092.stdout:Upgrading: 2023-11-24T05:50:48.752 INFO:teuthology.orchestra.run.smithi092.stdout: linux-firmware noarch 20231030-141.el9 CentOS-BaseOS 328 M 2023-11-24T05:50:48.752 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-24T05:50:48.752 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction Summary 2023-11-24T05:50:48.752 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2023-11-24T05:50:48.752 INFO:teuthology.orchestra.run.smithi092.stdout:Upgrade 1 Package 2023-11-24T05:50:48.752 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-24T05:50:48.753 INFO:teuthology.orchestra.run.smithi092.stdout:Total download size: 328 M 2023-11-24T05:50:48.753 INFO:teuthology.orchestra.run.smithi092.stdout:Downloading Packages: 2023-11-24T05:50:49.794 INFO:teuthology.orchestra.run.smithi118.stdout:Dependencies resolved. 2023-11-24T05:50:49.806 INFO:teuthology.orchestra.run.smithi118.stdout:================================================================================ 2023-11-24T05:50:49.807 INFO:teuthology.orchestra.run.smithi118.stdout: Package Arch Version Repository Size 2023-11-24T05:50:49.807 INFO:teuthology.orchestra.run.smithi118.stdout:================================================================================ 2023-11-24T05:50:49.807 INFO:teuthology.orchestra.run.smithi118.stdout:Upgrading: 2023-11-24T05:50:49.807 INFO:teuthology.orchestra.run.smithi118.stdout: linux-firmware noarch 20231030-141.el9 CentOS-BaseOS 328 M 2023-11-24T05:50:49.807 INFO:teuthology.orchestra.run.smithi118.stdout: 2023-11-24T05:50:49.807 INFO:teuthology.orchestra.run.smithi118.stdout:Transaction Summary 2023-11-24T05:50:49.807 INFO:teuthology.orchestra.run.smithi118.stdout:================================================================================ 2023-11-24T05:50:49.807 INFO:teuthology.orchestra.run.smithi118.stdout:Upgrade 1 Package 2023-11-24T05:50:49.807 INFO:teuthology.orchestra.run.smithi118.stdout: 2023-11-24T05:50:49.807 INFO:teuthology.orchestra.run.smithi118.stdout:Total download size: 328 M 2023-11-24T05:50:49.808 INFO:teuthology.orchestra.run.smithi118.stdout:Downloading Packages: 2023-11-24T05:50:53.781 INFO:teuthology.orchestra.run.smithi092.stdout:linux-firmware-20231030-141.el9.noarch.rpm 65 MB/s | 328 MB 00:05 2023-11-24T05:50:53.782 INFO:teuthology.orchestra.run.smithi092.stdout:-------------------------------------------------------------------------------- 2023-11-24T05:50:53.782 INFO:teuthology.orchestra.run.smithi092.stdout:Total 65 MB/s | 328 MB 00:05 2023-11-24T05:50:53.783 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction check 2023-11-24T05:50:53.862 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction check succeeded. 2023-11-24T05:50:53.862 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction test 2023-11-24T05:50:56.025 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction test succeeded. 2023-11-24T05:50:56.025 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction 2023-11-24T05:50:56.398 INFO:teuthology.orchestra.run.smithi118.stdout:linux-firmware-20231030-141.el9.noarch.rpm 50 MB/s | 328 MB 00:06 2023-11-24T05:50:56.399 INFO:teuthology.orchestra.run.smithi118.stdout:-------------------------------------------------------------------------------- 2023-11-24T05:50:56.400 INFO:teuthology.orchestra.run.smithi118.stdout:Total 50 MB/s | 328 MB 00:06 2023-11-24T05:50:56.400 INFO:teuthology.orchestra.run.smithi118.stdout:Running transaction check 2023-11-24T05:50:56.485 INFO:teuthology.orchestra.run.smithi118.stdout:Transaction check succeeded. 2023-11-24T05:50:56.485 INFO:teuthology.orchestra.run.smithi118.stdout:Running transaction test 2023-11-24T05:50:58.570 INFO:teuthology.orchestra.run.smithi118.stdout:Transaction test succeeded. 2023-11-24T05:50:58.571 INFO:teuthology.orchestra.run.smithi118.stdout:Running transaction 2023-11-24T05:51:01.546 INFO:teuthology.orchestra.run.smithi092.stdout: Preparing : 1/1 2023-11-24T05:51:02.055 INFO:teuthology.orchestra.run.smithi092.stdout: Upgrading : linux-firmware-20231030-141.el9.noarch 1/2 2023-11-24T05:51:02.161 INFO:teuthology.orchestra.run.smithi092.stdout: Cleanup : linux-firmware-20230814-139.el9.noarch 2/2 2023-11-24T05:51:03.270 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: linux-firmware-20230814-139.el9.noarch 2/2 2023-11-24T05:51:03.271 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : linux-firmware-20231030-141.el9.noarch 1/2 2023-11-24T05:51:03.666 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : linux-firmware-20230814-139.el9.noarch 2/2 2023-11-24T05:51:03.667 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-24T05:51:03.667 INFO:teuthology.orchestra.run.smithi092.stdout:Upgraded: 2023-11-24T05:51:03.667 INFO:teuthology.orchestra.run.smithi092.stdout: linux-firmware-20231030-141.el9.noarch 2023-11-24T05:51:03.667 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-24T05:51:03.667 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2023-11-24T05:51:03.958 INFO:teuthology.task.kernel:Downloading kernel 8247bbb5b470f7e2a1f3eefcc16e24cf1338f606 on mon.a... 2023-11-24T05:51:03.959 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8247bbb5b470f7e2a1f3eefcc16e24cf1338f606 2023-11-24T05:51:04.187 INFO:teuthology.task.kernel:fetching, builder baseurl is https://3.chacra.ceph.com/r/kernel/testing/8247bbb5b470f7e2a1f3eefcc16e24cf1338f606/centos/9/flavors/default/x86_64/ 2023-11-24T05:51:04.187 DEBUG:teuthology.orchestra.run.smithi092:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.6.0_g8247bbb5b470-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://3.chacra.ceph.com/r/kernel/testing/8247bbb5b470f7e2a1f3eefcc16e24cf1338f606/centos/9/flavors/default/x86_64/ --input-file=- 2023-11-24T05:51:04.191 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi092.front.sepia.ceph.com to complete... 2023-11-24T05:51:04.248 INFO:teuthology.orchestra.run.smithi118.stdout: Preparing : 1/1 2023-11-24T05:51:04.675 INFO:teuthology.orchestra.run.smithi118.stdout: Upgrading : linux-firmware-20231030-141.el9.noarch 1/2 2023-11-24T05:51:04.771 INFO:teuthology.orchestra.run.smithi118.stdout: Cleanup : linux-firmware-20230814-139.el9.noarch 2/2 2023-11-24T05:51:05.758 INFO:teuthology.orchestra.run.smithi118.stdout: Running scriptlet: linux-firmware-20230814-139.el9.noarch 2/2 2023-11-24T05:51:05.758 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : linux-firmware-20231030-141.el9.noarch 1/2 2023-11-24T05:51:06.106 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : linux-firmware-20230814-139.el9.noarch 2/2 2023-11-24T05:51:06.107 INFO:teuthology.orchestra.run.smithi118.stdout: 2023-11-24T05:51:06.107 INFO:teuthology.orchestra.run.smithi118.stdout:Upgraded: 2023-11-24T05:51:06.107 INFO:teuthology.orchestra.run.smithi118.stdout: linux-firmware-20231030-141.el9.noarch 2023-11-24T05:51:06.107 INFO:teuthology.orchestra.run.smithi118.stdout: 2023-11-24T05:51:06.107 INFO:teuthology.orchestra.run.smithi118.stdout:Complete! 2023-11-24T05:51:06.419 INFO:teuthology.task.kernel:Downloading kernel 8247bbb5b470f7e2a1f3eefcc16e24cf1338f606 on mon.b... 2023-11-24T05:51:06.419 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8247bbb5b470f7e2a1f3eefcc16e24cf1338f606 2023-11-24T05:51:06.565 INFO:teuthology.task.kernel:fetching, builder baseurl is https://3.chacra.ceph.com/r/kernel/testing/8247bbb5b470f7e2a1f3eefcc16e24cf1338f606/centos/9/flavors/default/x86_64/ 2023-11-24T05:51:06.566 DEBUG:teuthology.orchestra.run.smithi118:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.6.0_g8247bbb5b470-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://3.chacra.ceph.com/r/kernel/testing/8247bbb5b470f7e2a1f3eefcc16e24cf1338f606/centos/9/flavors/default/x86_64/ --input-file=- 2023-11-24T05:51:06.568 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi118.front.sepia.ceph.com to complete... 2023-11-24T05:51:20.843 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24 05:51:20 URL:https://3.chacra.ceph.com/r/kernel/testing/8247bbb5b470f7e2a1f3eefcc16e24cf1338f606/centos/9/flavors/default/x86_64/kernel-6.6.0_g8247bbb5b470-1.x86_64.rpm [320493731/320493731] -> "/tmp/kernel.x86_64.rpm" [1] 2023-11-24T05:51:20.843 INFO:teuthology.orchestra.run.smithi118.stderr:FINISHED --2023-11-24 05:51:20-- 2023-11-24T05:51:20.844 INFO:teuthology.orchestra.run.smithi118.stderr:Total wall clock time: 14s 2023-11-24T05:51:20.844 INFO:teuthology.orchestra.run.smithi118.stderr:Downloaded: 1 files, 306M in 14s (22.0 MB/s) 2023-11-24T05:51:20.845 INFO:teuthology.task.kernel:Installing kernel 8247bbb5b470f7e2a1f3eefcc16e24cf1338f606 on mon.b... 2023-11-24T05:51:20.846 DEBUG:teuthology.orchestra.run.smithi118:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2023-11-24T05:51:22.685 INFO:teuthology.orchestra.run.smithi118.stdout:Verifying... ######################################## 2023-11-24T05:51:22.733 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing... ######################################## 2023-11-24T05:51:22.813 INFO:teuthology.orchestra.run.smithi118.stdout:Updating / installing... 2023-11-24T05:51:29.824 INFO:teuthology.orchestra.run.smithi118.stdout:kernel-6.6.0_g8247bbb5b470-1 ######################################## 2023-11-24T05:51:29.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24 05:51:29 URL:https://3.chacra.ceph.com/r/kernel/testing/8247bbb5b470f7e2a1f3eefcc16e24cf1338f606/centos/9/flavors/default/x86_64/kernel-6.6.0_g8247bbb5b470-1.x86_64.rpm [320493731/320493731] -> "/tmp/kernel.x86_64.rpm" [1] 2023-11-24T05:51:29.957 INFO:teuthology.orchestra.run.smithi092.stderr:FINISHED --2023-11-24 05:51:29-- 2023-11-24T05:51:29.958 INFO:teuthology.orchestra.run.smithi092.stderr:Total wall clock time: 26s 2023-11-24T05:51:29.958 INFO:teuthology.orchestra.run.smithi092.stderr:Downloaded: 1 files, 306M in 25s (12.1 MB/s) 2023-11-24T05:51:29.959 INFO:teuthology.task.kernel:Installing kernel 8247bbb5b470f7e2a1f3eefcc16e24cf1338f606 on mon.a... 2023-11-24T05:51:29.960 DEBUG:teuthology.orchestra.run.smithi092:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2023-11-24T05:51:31.801 INFO:teuthology.orchestra.run.smithi092.stdout:Verifying... ######################################## 2023-11-24T05:51:31.898 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing... ######################################## 2023-11-24T05:51:31.978 INFO:teuthology.orchestra.run.smithi092.stdout:Updating / installing... 2023-11-24T05:51:39.063 INFO:teuthology.orchestra.run.smithi092.stdout:kernel-6.6.0_g8247bbb5b470-1 ######################################## 2023-11-24T05:51:46.610 INFO:teuthology.orchestra.run.smithi118.stderr:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2023-11-24T05:51:53.906 INFO:teuthology.orchestra.run.smithi092.stderr:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2023-11-24T05:52:07.635 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi118.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2023-11-24T05:52:07.635 DEBUG:teuthology.orchestra.run.smithi118:> rpm -qlp /tmp/kernel.x86_64.rpm 2023-11-24T05:52:07.704 INFO:teuthology.orchestra.run.smithi118.stdout:/boot/System.map-6.6.0-g8247bbb5b470 2023-11-24T05:52:07.705 INFO:teuthology.orchestra.run.smithi118.stdout:/boot/config-6.6.0-g8247bbb5b470 2023-11-24T05:52:07.705 INFO:teuthology.orchestra.run.smithi118.stdout:/boot/vmlinuz-6.6.0-g8247bbb5b470 2023-11-24T05:52:07.705 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470 2023-11-24T05:52:07.705 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel 2023-11-24T05:52:07.705 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch 2023-11-24T05:52:07.705 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86 2023-11-24T05:52:07.706 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto 2023-11-24T05:52:07.706 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/blowfish-x86_64.ko 2023-11-24T05:52:07.706 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2023-11-24T05:52:07.706 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2023-11-24T05:52:07.706 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/camellia-x86_64.ko 2023-11-24T05:52:07.707 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2023-11-24T05:52:07.707 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2023-11-24T05:52:07.707 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/crc32-pclmul.ko 2023-11-24T05:52:07.707 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/crc32c-intel.ko 2023-11-24T05:52:07.708 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2023-11-24T05:52:07.708 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2023-11-24T05:52:07.708 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2023-11-24T05:52:07.708 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2023-11-24T05:52:07.709 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/serpent-avx2.ko 2023-11-24T05:52:07.709 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2023-11-24T05:52:07.709 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/sha1-ssse3.ko 2023-11-24T05:52:07.709 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/sha256-ssse3.ko 2023-11-24T05:52:07.710 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/sha512-ssse3.ko 2023-11-24T05:52:07.710 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2023-11-24T05:52:07.710 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2023-11-24T05:52:07.710 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/twofish-x86_64.ko 2023-11-24T05:52:07.711 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kernel 2023-11-24T05:52:07.711 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kernel/cpu 2023-11-24T05:52:07.711 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kernel/cpu/mce 2023-11-24T05:52:07.711 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2023-11-24T05:52:07.712 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kvm 2023-11-24T05:52:07.712 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kvm/kvm-amd.ko 2023-11-24T05:52:07.712 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kvm/kvm-intel.ko 2023-11-24T05:52:07.712 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kvm/kvm.ko 2023-11-24T05:52:07.712 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto 2023-11-24T05:52:07.713 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/ansi_cprng.ko 2023-11-24T05:52:07.713 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/anubis.ko 2023-11-24T05:52:07.713 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/arc4.ko 2023-11-24T05:52:07.713 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/async_tx 2023-11-24T05:52:07.714 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/async_tx/async_memcpy.ko 2023-11-24T05:52:07.715 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/async_tx/async_pq.ko 2023-11-24T05:52:07.715 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/async_tx/async_raid6_recov.ko 2023-11-24T05:52:07.715 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/async_tx/async_tx.ko 2023-11-24T05:52:07.716 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/async_tx/async_xor.ko 2023-11-24T05:52:07.716 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/async_tx/raid6test.ko 2023-11-24T05:52:07.716 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/authenc.ko 2023-11-24T05:52:07.716 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/authencesn.ko 2023-11-24T05:52:07.717 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/blake2b_generic.ko 2023-11-24T05:52:07.717 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/blowfish_common.ko 2023-11-24T05:52:07.717 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/blowfish_generic.ko 2023-11-24T05:52:07.717 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/camellia_generic.ko 2023-11-24T05:52:07.718 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/cast5_generic.ko 2023-11-24T05:52:07.718 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/cast6_generic.ko 2023-11-24T05:52:07.718 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/cast_common.ko 2023-11-24T05:52:07.718 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/ccm.ko 2023-11-24T05:52:07.718 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/cmac.ko 2023-11-24T05:52:07.718 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/crc32_generic.ko 2023-11-24T05:52:07.718 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/crypto_engine.ko 2023-11-24T05:52:07.718 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/crypto_user.ko 2023-11-24T05:52:07.718 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/cts.ko 2023-11-24T05:52:07.718 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/des_generic.ko 2023-11-24T05:52:07.718 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/dh_generic.ko 2023-11-24T05:52:07.719 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/echainiv.ko 2023-11-24T05:52:07.719 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/essiv.ko 2023-11-24T05:52:07.719 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/fcrypt.ko 2023-11-24T05:52:07.719 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/khazad.ko 2023-11-24T05:52:07.719 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/lz4.ko 2023-11-24T05:52:07.719 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/lz4hc.ko 2023-11-24T05:52:07.719 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/md4.ko 2023-11-24T05:52:07.719 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/michael_mic.ko 2023-11-24T05:52:07.719 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/pcbc.ko 2023-11-24T05:52:07.719 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/pcrypt.ko 2023-11-24T05:52:07.719 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/rmd160.ko 2023-11-24T05:52:07.720 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/seed.ko 2023-11-24T05:52:07.720 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/serpent_generic.ko 2023-11-24T05:52:07.720 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/tcrypt.ko 2023-11-24T05:52:07.720 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/tea.ko 2023-11-24T05:52:07.720 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/twofish_common.ko 2023-11-24T05:52:07.720 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/twofish_generic.ko 2023-11-24T05:52:07.720 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/vmac.ko 2023-11-24T05:52:07.720 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/wp512.ko 2023-11-24T05:52:07.720 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/xcbc.ko 2023-11-24T05:52:07.720 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/xor.ko 2023-11-24T05:52:07.720 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/xxhash_generic.ko 2023-11-24T05:52:07.720 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers 2023-11-24T05:52:07.721 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi 2023-11-24T05:52:07.721 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/acpi_ipmi.ko 2023-11-24T05:52:07.721 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/acpi_pad.ko 2023-11-24T05:52:07.721 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/custom_method.ko 2023-11-24T05:52:07.721 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/ec_sys.ko 2023-11-24T05:52:07.721 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/platform_profile.ko 2023-11-24T05:52:07.721 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/sbs.ko 2023-11-24T05:52:07.721 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/sbshc.ko 2023-11-24T05:52:07.721 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/video.ko 2023-11-24T05:52:07.721 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata 2023-11-24T05:52:07.721 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/acard-ahci.ko 2023-11-24T05:52:07.721 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/ahci_platform.ko 2023-11-24T05:52:07.722 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/ata_generic.ko 2023-11-24T05:52:07.722 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/libahci_platform.ko 2023-11-24T05:52:07.722 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_acpi.ko 2023-11-24T05:52:07.722 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_ali.ko 2023-11-24T05:52:07.722 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_amd.ko 2023-11-24T05:52:07.722 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_artop.ko 2023-11-24T05:52:07.722 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_atiixp.ko 2023-11-24T05:52:07.722 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_atp867x.ko 2023-11-24T05:52:07.722 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_cmd640.ko 2023-11-24T05:52:07.722 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_cmd64x.ko 2023-11-24T05:52:07.722 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_cypress.ko 2023-11-24T05:52:07.722 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_efar.ko 2023-11-24T05:52:07.723 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_hpt366.ko 2023-11-24T05:52:07.723 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_hpt37x.ko 2023-11-24T05:52:07.723 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_hpt3x2n.ko 2023-11-24T05:52:07.723 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_hpt3x3.ko 2023-11-24T05:52:07.723 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_it8213.ko 2023-11-24T05:52:07.723 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_it821x.ko 2023-11-24T05:52:07.723 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_jmicron.ko 2023-11-24T05:52:07.723 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_marvell.ko 2023-11-24T05:52:07.723 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_mpiix.ko 2023-11-24T05:52:07.723 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_netcell.ko 2023-11-24T05:52:07.723 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_ninja32.ko 2023-11-24T05:52:07.724 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_ns87410.ko 2023-11-24T05:52:07.724 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_ns87415.ko 2023-11-24T05:52:07.724 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_oldpiix.ko 2023-11-24T05:52:07.724 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_opti.ko 2023-11-24T05:52:07.724 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_optidma.ko 2023-11-24T05:52:07.724 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_pcmcia.ko 2023-11-24T05:52:07.724 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_pdc2027x.ko 2023-11-24T05:52:07.724 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_pdc202xx_old.ko 2023-11-24T05:52:07.724 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_piccolo.ko 2023-11-24T05:52:07.724 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_rdc.ko 2023-11-24T05:52:07.724 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_sch.ko 2023-11-24T05:52:07.725 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_serverworks.ko 2023-11-24T05:52:07.725 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_sil680.ko 2023-11-24T05:52:07.725 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_sis.ko 2023-11-24T05:52:07.725 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_sl82c105.ko 2023-11-24T05:52:07.725 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_triflex.ko 2023-11-24T05:52:07.725 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_via.ko 2023-11-24T05:52:07.725 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pdc_adma.ko 2023-11-24T05:52:07.725 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_inic162x.ko 2023-11-24T05:52:07.726 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_mv.ko 2023-11-24T05:52:07.726 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_nv.ko 2023-11-24T05:52:07.726 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_promise.ko 2023-11-24T05:52:07.726 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_qstor.ko 2023-11-24T05:52:07.726 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_sil.ko 2023-11-24T05:52:07.726 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_sil24.ko 2023-11-24T05:52:07.726 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_sis.ko 2023-11-24T05:52:07.726 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_svw.ko 2023-11-24T05:52:07.726 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_sx4.ko 2023-11-24T05:52:07.726 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_uli.ko 2023-11-24T05:52:07.726 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_via.ko 2023-11-24T05:52:07.726 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_vsc.ko 2023-11-24T05:52:07.727 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/auxdisplay 2023-11-24T05:52:07.727 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/auxdisplay/ks0108.ko 2023-11-24T05:52:07.727 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/base 2023-11-24T05:52:07.727 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/base/regmap 2023-11-24T05:52:07.727 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/base/regmap/regmap-i2c.ko 2023-11-24T05:52:07.727 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/base/regmap/regmap-mmio.ko 2023-11-24T05:52:07.727 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/bcma 2023-11-24T05:52:07.727 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/bcma/bcma.ko 2023-11-24T05:52:07.727 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block 2023-11-24T05:52:07.727 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/aoe 2023-11-24T05:52:07.727 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/aoe/aoe.ko 2023-11-24T05:52:07.728 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/brd.ko 2023-11-24T05:52:07.728 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/drbd 2023-11-24T05:52:07.728 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/drbd/drbd.ko 2023-11-24T05:52:07.728 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/floppy.ko 2023-11-24T05:52:07.728 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/loop.ko 2023-11-24T05:52:07.728 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/mtip32xx 2023-11-24T05:52:07.730 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/mtip32xx/mtip32xx.ko 2023-11-24T05:52:07.730 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/nbd.ko 2023-11-24T05:52:07.731 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/null_blk 2023-11-24T05:52:07.731 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/null_blk/null_blk.ko 2023-11-24T05:52:07.731 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/pktcdvd.ko 2023-11-24T05:52:07.731 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/rbd.ko 2023-11-24T05:52:07.732 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/virtio_blk.ko 2023-11-24T05:52:07.732 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/xen-blkback 2023-11-24T05:52:07.732 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/xen-blkback/xen-blkback.ko 2023-11-24T05:52:07.732 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/xen-blkfront.ko 2023-11-24T05:52:07.733 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/zram 2023-11-24T05:52:07.733 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/zram/zram.ko 2023-11-24T05:52:07.733 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char 2023-11-24T05:52:07.733 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/hangcheck-timer.ko 2023-11-24T05:52:07.734 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/hw_random 2023-11-24T05:52:07.734 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/hw_random/amd-rng.ko 2023-11-24T05:52:07.734 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/hw_random/intel-rng.ko 2023-11-24T05:52:07.734 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/hw_random/timeriomem-rng.ko 2023-11-24T05:52:07.734 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/hw_random/via-rng.ko 2023-11-24T05:52:07.735 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/hw_random/virtio-rng.ko 2023-11-24T05:52:07.735 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ipmi 2023-11-24T05:52:07.736 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ipmi/ipmi_devintf.ko 2023-11-24T05:52:07.736 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2023-11-24T05:52:07.736 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2023-11-24T05:52:07.736 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ipmi/ipmi_si.ko 2023-11-24T05:52:07.736 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ipmi/ipmi_ssif.ko 2023-11-24T05:52:07.736 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2023-11-24T05:52:07.736 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/lp.ko 2023-11-24T05:52:07.736 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/mwave 2023-11-24T05:52:07.736 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/mwave/mwave.ko 2023-11-24T05:52:07.736 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ppdev.ko 2023-11-24T05:52:07.737 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tlclk.ko 2023-11-24T05:52:07.737 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tpm 2023-11-24T05:52:07.737 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tpm/tpm.ko 2023-11-24T05:52:07.737 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tpm/tpm_atmel.ko 2023-11-24T05:52:07.737 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tpm/tpm_infineon.ko 2023-11-24T05:52:07.737 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tpm/tpm_nsc.ko 2023-11-24T05:52:07.737 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tpm/tpm_tis.ko 2023-11-24T05:52:07.737 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tpm/tpm_tis_core.ko 2023-11-24T05:52:07.738 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/uv_mmtimer.ko 2023-11-24T05:52:07.738 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/virtio_console.ko 2023-11-24T05:52:07.738 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/cpufreq 2023-11-24T05:52:07.738 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/cpufreq/acpi-cpufreq.ko 2023-11-24T05:52:07.738 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2023-11-24T05:52:07.738 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/cpufreq/p4-clockmod.ko 2023-11-24T05:52:07.738 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/cpufreq/pcc-cpufreq.ko 2023-11-24T05:52:07.738 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/cpufreq/powernow-k8.ko 2023-11-24T05:52:07.738 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/cpufreq/speedstep-lib.ko 2023-11-24T05:52:07.738 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto 2023-11-24T05:52:07.739 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/ccp 2023-11-24T05:52:07.739 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/ccp/ccp-crypto.ko 2023-11-24T05:52:07.739 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/ccp/ccp.ko 2023-11-24T05:52:07.739 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/intel 2023-11-24T05:52:07.739 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/intel/qat 2023-11-24T05:52:07.739 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/intel/qat/qat_common 2023-11-24T05:52:07.739 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2023-11-24T05:52:07.739 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2023-11-24T05:52:07.739 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2023-11-24T05:52:07.739 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/padlock-aes.ko 2023-11-24T05:52:07.740 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/padlock-sha.ko 2023-11-24T05:52:07.740 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/virtio 2023-11-24T05:52:07.740 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/virtio/virtio_crypto.ko 2023-11-24T05:52:07.740 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/dca 2023-11-24T05:52:07.740 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/dca/dca.ko 2023-11-24T05:52:07.740 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/devfreq 2023-11-24T05:52:07.740 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/devfreq/governor_simpleondemand.ko 2023-11-24T05:52:07.740 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/dma 2023-11-24T05:52:07.740 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/dma/dw 2023-11-24T05:52:07.740 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/dma/dw/dw_dmac.ko 2023-11-24T05:52:07.740 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/dma/ioat 2023-11-24T05:52:07.740 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/dma/ioat/ioatdma.ko 2023-11-24T05:52:07.740 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac 2023-11-24T05:52:07.740 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/amd64_edac.ko 2023-11-24T05:52:07.741 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/e752x_edac.ko 2023-11-24T05:52:07.741 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/edac_mce_amd.ko 2023-11-24T05:52:07.741 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/i3000_edac.ko 2023-11-24T05:52:07.741 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/i3200_edac.ko 2023-11-24T05:52:07.741 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/i5100_edac.ko 2023-11-24T05:52:07.741 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/i5400_edac.ko 2023-11-24T05:52:07.741 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/i7300_edac.ko 2023-11-24T05:52:07.741 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/i7core_edac.ko 2023-11-24T05:52:07.741 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/i82975x_edac.ko 2023-11-24T05:52:07.741 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/ie31200_edac.ko 2023-11-24T05:52:07.742 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/sb_edac.ko 2023-11-24T05:52:07.742 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/x38_edac.ko 2023-11-24T05:52:07.742 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/firmware 2023-11-24T05:52:07.742 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/firmware/edd.ko 2023-11-24T05:52:07.743 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/firmware/iscsi_ibft.ko 2023-11-24T05:52:07.743 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio 2023-11-24T05:52:07.743 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-ich.ko 2023-11-24T05:52:07.743 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-viperboard.ko 2023-11-24T05:52:07.743 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid 2023-11-24T05:52:07.743 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-appleir.ko 2023-11-24T05:52:07.743 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-aureal.ko 2023-11-24T05:52:07.743 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-axff.ko 2023-11-24T05:52:07.743 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-dr.ko 2023-11-24T05:52:07.743 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-elecom.ko 2023-11-24T05:52:07.743 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-elo.ko 2023-11-24T05:52:07.743 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-emsff.ko 2023-11-24T05:52:07.744 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-gaff.ko 2023-11-24T05:52:07.744 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-gt683r.ko 2023-11-24T05:52:07.744 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-gyration.ko 2023-11-24T05:52:07.744 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-holtek-kbd.ko 2023-11-24T05:52:07.744 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-holtek-mouse.ko 2023-11-24T05:52:07.744 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-holtekff.ko 2023-11-24T05:52:07.744 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-hyperv.ko 2023-11-24T05:52:07.745 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-icade.ko 2023-11-24T05:52:07.745 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-keytouch.ko 2023-11-24T05:52:07.745 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-kye.ko 2023-11-24T05:52:07.745 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-lcpower.ko 2023-11-24T05:52:07.745 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-led.ko 2023-11-24T05:52:07.745 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-lenovo.ko 2023-11-24T05:52:07.745 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-logitech-dj.ko 2023-11-24T05:52:07.745 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-logitech-hidpp.ko 2023-11-24T05:52:07.746 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-multitouch.ko 2023-11-24T05:52:07.746 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-ortek.ko 2023-11-24T05:52:07.746 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-penmount.ko 2023-11-24T05:52:07.746 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-petalynx.ko 2023-11-24T05:52:07.746 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-picolcd.ko 2023-11-24T05:52:07.746 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-pl.ko 2023-11-24T05:52:07.746 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-plantronics.ko 2023-11-24T05:52:07.746 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-primax.ko 2023-11-24T05:52:07.746 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-rmi.ko 2023-11-24T05:52:07.746 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-arvo.ko 2023-11-24T05:52:07.746 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-common.ko 2023-11-24T05:52:07.747 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-isku.ko 2023-11-24T05:52:07.747 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-kone.ko 2023-11-24T05:52:07.747 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-koneplus.ko 2023-11-24T05:52:07.747 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-konepure.ko 2023-11-24T05:52:07.747 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-kovaplus.ko 2023-11-24T05:52:07.747 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-lua.ko 2023-11-24T05:52:07.747 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-pyra.ko 2023-11-24T05:52:07.747 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-ryos.ko 2023-11-24T05:52:07.747 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-savu.ko 2023-11-24T05:52:07.747 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat.ko 2023-11-24T05:52:07.747 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-saitek.ko 2023-11-24T05:52:07.748 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-samsung.ko 2023-11-24T05:52:07.748 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-sensor-hub.ko 2023-11-24T05:52:07.748 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-sjoy.ko 2023-11-24T05:52:07.748 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-sony.ko 2023-11-24T05:52:07.748 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-speedlink.ko 2023-11-24T05:52:07.748 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-steelseries.ko 2023-11-24T05:52:07.748 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-sunplus.ko 2023-11-24T05:52:07.748 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-thrustmaster.ko 2023-11-24T05:52:07.748 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-tivo.ko 2023-11-24T05:52:07.748 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-tmff.ko 2023-11-24T05:52:07.748 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-topseed.ko 2023-11-24T05:52:07.748 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-twinhan.ko 2023-11-24T05:52:07.748 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-uclogic.ko 2023-11-24T05:52:07.749 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-waltop.ko 2023-11-24T05:52:07.749 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-wiimote.ko 2023-11-24T05:52:07.749 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-xinmo.ko 2023-11-24T05:52:07.749 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-zpff.ko 2023-11-24T05:52:07.749 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-zydacron.ko 2023-11-24T05:52:07.749 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/uhid.ko 2023-11-24T05:52:07.749 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/wacom.ko 2023-11-24T05:52:07.749 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hv 2023-11-24T05:52:07.749 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hv/hv_balloon.ko 2023-11-24T05:52:07.749 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hv/hv_utils.ko 2023-11-24T05:52:07.749 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hv/hv_vmbus.ko 2023-11-24T05:52:07.749 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon 2023-11-24T05:52:07.750 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/abituguru.ko 2023-11-24T05:52:07.750 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/abituguru3.ko 2023-11-24T05:52:07.750 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/acpi_power_meter.ko 2023-11-24T05:52:07.750 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ad7414.ko 2023-11-24T05:52:07.750 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ad7418.ko 2023-11-24T05:52:07.750 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adc128d818.ko 2023-11-24T05:52:07.750 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adm1025.ko 2023-11-24T05:52:07.750 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adm1026.ko 2023-11-24T05:52:07.750 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adm1029.ko 2023-11-24T05:52:07.750 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adm1031.ko 2023-11-24T05:52:07.750 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adm9240.ko 2023-11-24T05:52:07.751 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ads7828.ko 2023-11-24T05:52:07.751 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adt7410.ko 2023-11-24T05:52:07.751 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adt7411.ko 2023-11-24T05:52:07.751 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adt7462.ko 2023-11-24T05:52:07.751 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adt7470.ko 2023-11-24T05:52:07.751 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adt7475.ko 2023-11-24T05:52:07.751 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adt7x10.ko 2023-11-24T05:52:07.751 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/amc6821.ko 2023-11-24T05:52:07.751 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/applesmc.ko 2023-11-24T05:52:07.751 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/asb100.ko 2023-11-24T05:52:07.751 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/asc7621.ko 2023-11-24T05:52:07.751 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/asus_atk0110.ko 2023-11-24T05:52:07.751 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/atxp1.ko 2023-11-24T05:52:07.752 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/coretemp.ko 2023-11-24T05:52:07.752 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/dell-smm-hwmon.ko 2023-11-24T05:52:07.752 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/dme1737.ko 2023-11-24T05:52:07.752 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ds1621.ko 2023-11-24T05:52:07.752 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ds620.ko 2023-11-24T05:52:07.752 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/emc1403.ko 2023-11-24T05:52:07.752 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/emc6w201.ko 2023-11-24T05:52:07.752 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/f71805f.ko 2023-11-24T05:52:07.752 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/f71882fg.ko 2023-11-24T05:52:07.752 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/f75375s.ko 2023-11-24T05:52:07.752 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/fam15h_power.ko 2023-11-24T05:52:07.752 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/fschmd.ko 2023-11-24T05:52:07.752 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/g760a.ko 2023-11-24T05:52:07.753 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/g762.ko 2023-11-24T05:52:07.753 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/gl518sm.ko 2023-11-24T05:52:07.753 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/gl520sm.ko 2023-11-24T05:52:07.753 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/hwmon-vid.ko 2023-11-24T05:52:07.753 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/i5500_temp.ko 2023-11-24T05:52:07.753 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/i5k_amb.ko 2023-11-24T05:52:07.753 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ibmaem.ko 2023-11-24T05:52:07.753 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ibmpex.ko 2023-11-24T05:52:07.753 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ina209.ko 2023-11-24T05:52:07.753 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ina2xx.ko 2023-11-24T05:52:07.753 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/it87.ko 2023-11-24T05:52:07.753 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/k10temp.ko 2023-11-24T05:52:07.754 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/k8temp.ko 2023-11-24T05:52:07.754 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lineage-pem.ko 2023-11-24T05:52:07.754 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm63.ko 2023-11-24T05:52:07.754 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm73.ko 2023-11-24T05:52:07.754 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm75.ko 2023-11-24T05:52:07.754 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm77.ko 2023-11-24T05:52:07.754 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm78.ko 2023-11-24T05:52:07.754 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm80.ko 2023-11-24T05:52:07.754 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm83.ko 2023-11-24T05:52:07.755 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm85.ko 2023-11-24T05:52:07.755 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm87.ko 2023-11-24T05:52:07.755 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm90.ko 2023-11-24T05:52:07.755 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm92.ko 2023-11-24T05:52:07.755 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm93.ko 2023-11-24T05:52:07.755 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm95234.ko 2023-11-24T05:52:07.755 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm95241.ko 2023-11-24T05:52:07.755 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm95245.ko 2023-11-24T05:52:07.755 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ltc2945.ko 2023-11-24T05:52:07.756 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ltc4151.ko 2023-11-24T05:52:07.756 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ltc4215.ko 2023-11-24T05:52:07.756 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ltc4222.ko 2023-11-24T05:52:07.756 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ltc4245.ko 2023-11-24T05:52:07.756 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ltc4260.ko 2023-11-24T05:52:07.756 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ltc4261.ko 2023-11-24T05:52:07.756 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/max16065.ko 2023-11-24T05:52:07.756 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/max1619.ko 2023-11-24T05:52:07.757 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/max1668.ko 2023-11-24T05:52:07.757 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/max197.ko 2023-11-24T05:52:07.757 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/max6639.ko 2023-11-24T05:52:07.757 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/max6650.ko 2023-11-24T05:52:07.757 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/max6697.ko 2023-11-24T05:52:07.757 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/mcp3021.ko 2023-11-24T05:52:07.757 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/nct6683.ko 2023-11-24T05:52:07.757 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/nct6775-core.ko 2023-11-24T05:52:07.757 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/nct6775.ko 2023-11-24T05:52:07.757 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/nct7802.ko 2023-11-24T05:52:07.757 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pc87360.ko 2023-11-24T05:52:07.757 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pc87427.ko 2023-11-24T05:52:07.758 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pcf8591.ko 2023-11-24T05:52:07.758 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus 2023-11-24T05:52:07.758 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/adm1275.ko 2023-11-24T05:52:07.758 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/lm25066.ko 2023-11-24T05:52:07.758 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/ltc2978.ko 2023-11-24T05:52:07.758 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/max16064.ko 2023-11-24T05:52:07.758 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/max34440.ko 2023-11-24T05:52:07.758 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/max8688.ko 2023-11-24T05:52:07.758 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/pmbus.ko 2023-11-24T05:52:07.759 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2023-11-24T05:52:07.759 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/tps40422.ko 2023-11-24T05:52:07.759 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/ucd9000.ko 2023-11-24T05:52:07.759 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/ucd9200.ko 2023-11-24T05:52:07.759 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/zl6100.ko 2023-11-24T05:52:07.759 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/powr1220.ko 2023-11-24T05:52:07.759 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/sch5627.ko 2023-11-24T05:52:07.759 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/sch5636.ko 2023-11-24T05:52:07.759 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/sch56xx-common.ko 2023-11-24T05:52:07.759 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/sht15.ko 2023-11-24T05:52:07.759 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/sht21.ko 2023-11-24T05:52:07.760 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/shtc1.ko 2023-11-24T05:52:07.760 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/sis5595.ko 2023-11-24T05:52:07.760 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/smsc47b397.ko 2023-11-24T05:52:07.760 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/smsc47m1.ko 2023-11-24T05:52:07.760 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/smsc47m192.ko 2023-11-24T05:52:07.760 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/thmc50.ko 2023-11-24T05:52:07.761 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/tmp102.ko 2023-11-24T05:52:07.761 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/tmp103.ko 2023-11-24T05:52:07.761 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/tmp401.ko 2023-11-24T05:52:07.761 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/tmp421.ko 2023-11-24T05:52:07.761 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/via-cputemp.ko 2023-11-24T05:52:07.761 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/via686a.ko 2023-11-24T05:52:07.761 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/vt1211.ko 2023-11-24T05:52:07.761 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/vt8231.ko 2023-11-24T05:52:07.761 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83627ehf.ko 2023-11-24T05:52:07.761 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83627hf.ko 2023-11-24T05:52:07.762 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83781d.ko 2023-11-24T05:52:07.762 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83791d.ko 2023-11-24T05:52:07.762 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83792d.ko 2023-11-24T05:52:07.762 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83793.ko 2023-11-24T05:52:07.763 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83795.ko 2023-11-24T05:52:07.763 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83l785ts.ko 2023-11-24T05:52:07.763 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83l786ng.ko 2023-11-24T05:52:07.763 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c 2023-11-24T05:52:07.763 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/algos 2023-11-24T05:52:07.763 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2023-11-24T05:52:07.763 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2023-11-24T05:52:07.763 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses 2023-11-24T05:52:07.763 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2023-11-24T05:52:07.763 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-amd756.ko 2023-11-24T05:52:07.763 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-amd8111.ko 2023-11-24T05:52:07.764 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2023-11-24T05:52:07.764 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-designware-core.ko 2023-11-24T05:52:07.764 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2023-11-24T05:52:07.764 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2023-11-24T05:52:07.764 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2023-11-24T05:52:07.764 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-i801.ko 2023-11-24T05:52:07.764 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-isch.ko 2023-11-24T05:52:07.764 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-ismt.ko 2023-11-24T05:52:07.764 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2023-11-24T05:52:07.764 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-nforce2.ko 2023-11-24T05:52:07.764 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-parport.ko 2023-11-24T05:52:07.764 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2023-11-24T05:52:07.764 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-piix4.ko 2023-11-24T05:52:07.765 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-scmi.ko 2023-11-24T05:52:07.765 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-simtec.ko 2023-11-24T05:52:07.765 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-sis96x.ko 2023-11-24T05:52:07.765 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2023-11-24T05:52:07.765 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-via.ko 2023-11-24T05:52:07.765 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-viapro.ko 2023-11-24T05:52:07.765 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-viperboard.ko 2023-11-24T05:52:07.765 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/i2c-dev.ko 2023-11-24T05:52:07.765 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/i2c-mux.ko 2023-11-24T05:52:07.765 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/i2c-slave-eeprom.ko 2023-11-24T05:52:07.765 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/i2c-smbus.ko 2023-11-24T05:52:07.765 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/i2c-stub.ko 2023-11-24T05:52:07.766 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband 2023-11-24T05:52:07.766 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core 2023-11-24T05:52:07.766 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core/ib_cm.ko 2023-11-24T05:52:07.766 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core/ib_core.ko 2023-11-24T05:52:07.766 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core/ib_umad.ko 2023-11-24T05:52:07.766 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core/ib_uverbs.ko 2023-11-24T05:52:07.767 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core/iw_cm.ko 2023-11-24T05:52:07.767 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core/rdma_cm.ko 2023-11-24T05:52:07.767 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core/rdma_ucm.ko 2023-11-24T05:52:07.767 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw 2023-11-24T05:52:07.767 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/cxgb4 2023-11-24T05:52:07.767 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2023-11-24T05:52:07.767 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/mlx4 2023-11-24T05:52:07.767 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2023-11-24T05:52:07.767 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/mlx5 2023-11-24T05:52:07.767 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2023-11-24T05:52:07.767 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/mthca 2023-11-24T05:52:07.767 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2023-11-24T05:52:07.767 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp 2023-11-24T05:52:07.768 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/ipoib 2023-11-24T05:52:07.768 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2023-11-24T05:52:07.768 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/iser 2023-11-24T05:52:07.768 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2023-11-24T05:52:07.768 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/isert 2023-11-24T05:52:07.768 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2023-11-24T05:52:07.768 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/srp 2023-11-24T05:52:07.768 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2023-11-24T05:52:07.768 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/srpt 2023-11-24T05:52:07.768 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2023-11-24T05:52:07.768 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input 2023-11-24T05:52:07.768 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/gameport 2023-11-24T05:52:07.769 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/gameport/emu10k1-gp.ko 2023-11-24T05:52:07.769 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/gameport/fm801-gp.ko 2023-11-24T05:52:07.769 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/gameport/gameport.ko 2023-11-24T05:52:07.769 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/joydev.ko 2023-11-24T05:52:07.769 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard 2023-11-24T05:52:07.769 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard/gpio_keys.ko 2023-11-24T05:52:07.769 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse 2023-11-24T05:52:07.769 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/appletouch.ko 2023-11-24T05:52:07.769 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/bcm5974.ko 2023-11-24T05:52:07.769 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/cyapatp.ko 2023-11-24T05:52:07.769 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/elan_i2c.ko 2023-11-24T05:52:07.769 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/sermouse.ko 2023-11-24T05:52:07.769 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/synaptics_i2c.ko 2023-11-24T05:52:07.770 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/synaptics_usb.ko 2023-11-24T05:52:07.770 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/vsxxxaa.ko 2023-11-24T05:52:07.770 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/rmi4 2023-11-24T05:52:07.770 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/rmi4/rmi_core.ko 2023-11-24T05:52:07.770 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/serio 2023-11-24T05:52:07.770 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/serio/altera_ps2.ko 2023-11-24T05:52:07.770 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/serio/arc_ps2.ko 2023-11-24T05:52:07.770 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/serio/hyperv-keyboard.ko 2023-11-24T05:52:07.770 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/serio/serio_raw.ko 2023-11-24T05:52:07.770 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/sparse-keymap.ko 2023-11-24T05:52:07.770 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/iommu 2023-11-24T05:52:07.770 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/iommu/amd 2023-11-24T05:52:07.771 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/iommu/amd/iommu_v2.ko 2023-11-24T05:52:07.771 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds 2023-11-24T05:52:07.771 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-blinkm.ko 2023-11-24T05:52:07.771 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-lm3530.ko 2023-11-24T05:52:07.771 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-lp3944.ko 2023-11-24T05:52:07.771 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-ss4200.ko 2023-11-24T05:52:07.771 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger 2023-11-24T05:52:07.771 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-audio.ko 2023-11-24T05:52:07.771 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2023-11-24T05:52:07.772 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-camera.ko 2023-11-24T05:52:07.772 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2023-11-24T05:52:07.772 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2023-11-24T05:52:07.772 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2023-11-24T05:52:07.772 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-timer.ko 2023-11-24T05:52:07.772 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-transient.ko 2023-11-24T05:52:07.772 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md 2023-11-24T05:52:07.772 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/bcache 2023-11-24T05:52:07.773 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/bcache/bcache.ko 2023-11-24T05:52:07.773 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-bio-prison.ko 2023-11-24T05:52:07.773 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-cache-smq.ko 2023-11-24T05:52:07.773 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-cache.ko 2023-11-24T05:52:07.773 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-crypt.ko 2023-11-24T05:52:07.773 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-delay.ko 2023-11-24T05:52:07.773 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-flakey.ko 2023-11-24T05:52:07.773 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-log-userspace.ko 2023-11-24T05:52:07.773 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-multipath.ko 2023-11-24T05:52:07.774 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-queue-length.ko 2023-11-24T05:52:07.774 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-raid.ko 2023-11-24T05:52:07.774 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-round-robin.ko 2023-11-24T05:52:07.774 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-service-time.ko 2023-11-24T05:52:07.774 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-switch.ko 2023-11-24T05:52:07.774 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-thin-pool.ko 2023-11-24T05:52:07.774 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-verity.ko 2023-11-24T05:52:07.774 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/faulty.ko 2023-11-24T05:52:07.774 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/linear.ko 2023-11-24T05:52:07.774 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/multipath.ko 2023-11-24T05:52:07.774 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/persistent-data 2023-11-24T05:52:07.774 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2023-11-24T05:52:07.774 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/raid0.ko 2023-11-24T05:52:07.775 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/raid1.ko 2023-11-24T05:52:07.775 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/raid10.ko 2023-11-24T05:52:07.775 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/raid456.ko 2023-11-24T05:52:07.775 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message 2023-11-24T05:52:07.775 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion 2023-11-24T05:52:07.775 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion/mptbase.ko 2023-11-24T05:52:07.775 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion/mptctl.ko 2023-11-24T05:52:07.775 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion/mptfc.ko 2023-11-24T05:52:07.775 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion/mptlan.ko 2023-11-24T05:52:07.775 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion/mptsas.ko 2023-11-24T05:52:07.775 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion/mptscsih.ko 2023-11-24T05:52:07.775 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion/mptspi.ko 2023-11-24T05:52:07.775 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd 2023-11-24T05:52:07.776 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/lpc_ich.ko 2023-11-24T05:52:07.776 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/lpc_sch.ko 2023-11-24T05:52:07.776 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/mfd-core.ko 2023-11-24T05:52:07.776 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/sm501.ko 2023-11-24T05:52:07.776 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/viperboard.ko 2023-11-24T05:52:07.776 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/vx855.ko 2023-11-24T05:52:07.776 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/wl1273-core.ko 2023-11-24T05:52:07.776 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc 2023-11-24T05:52:07.776 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/altera-stapl 2023-11-24T05:52:07.776 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2023-11-24T05:52:07.776 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/apds9802als.ko 2023-11-24T05:52:07.776 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/apds990x.ko 2023-11-24T05:52:07.776 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/bh1770glc.ko 2023-11-24T05:52:07.777 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/cb710 2023-11-24T05:52:07.777 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/cb710/cb710.ko 2023-11-24T05:52:07.777 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/eeprom 2023-11-24T05:52:07.777 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/eeprom/at24.ko 2023-11-24T05:52:07.777 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/eeprom/eeprom.ko 2023-11-24T05:52:07.777 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2023-11-24T05:52:07.777 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/eeprom/max6875.ko 2023-11-24T05:52:07.777 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/enclosure.ko 2023-11-24T05:52:07.777 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/hpilo.ko 2023-11-24T05:52:07.777 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/isl29003.ko 2023-11-24T05:52:07.777 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/isl29020.ko 2023-11-24T05:52:07.777 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/lis3lv02d 2023-11-24T05:52:07.777 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2023-11-24T05:52:07.778 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2023-11-24T05:52:07.778 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/mei 2023-11-24T05:52:07.778 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/mei/mei-me.ko 2023-11-24T05:52:07.778 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/mei/mei-txe.ko 2023-11-24T05:52:07.778 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/mei/mei.ko 2023-11-24T05:52:07.778 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/sgi-gru 2023-11-24T05:52:07.778 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/sgi-gru/gru.ko 2023-11-24T05:52:07.778 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/sgi-xp 2023-11-24T05:52:07.778 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/sgi-xp/xp.ko 2023-11-24T05:52:07.778 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/sgi-xp/xpc.ko 2023-11-24T05:52:07.778 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/sgi-xp/xpnet.ko 2023-11-24T05:52:07.779 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/tifm_7xx1.ko 2023-11-24T05:52:07.779 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/tifm_core.ko 2023-11-24T05:52:07.779 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/tsl2550.ko 2023-11-24T05:52:07.779 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/vmw_balloon.ko 2023-11-24T05:52:07.779 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/vmw_vmci 2023-11-24T05:52:07.779 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2023-11-24T05:52:07.779 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net 2023-11-24T05:52:07.779 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/bonding 2023-11-24T05:52:07.780 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/bonding/bonding.ko 2023-11-24T05:52:07.780 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dsa 2023-11-24T05:52:07.780 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dsa/b53 2023-11-24T05:52:07.780 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dsa/b53/b53_common.ko 2023-11-24T05:52:07.780 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dsa/bcm-sf2.ko 2023-11-24T05:52:07.780 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dsa/mv88e6060.ko 2023-11-24T05:52:07.780 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dsa/mv88e6xxx 2023-11-24T05:52:07.780 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2023-11-24T05:52:07.780 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dummy.ko 2023-11-24T05:52:07.780 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/eql.ko 2023-11-24T05:52:07.780 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet 2023-11-24T05:52:07.780 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/3com 2023-11-24T05:52:07.781 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2023-11-24T05:52:07.781 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2023-11-24T05:52:07.781 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/3com/3c59x.ko 2023-11-24T05:52:07.781 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/3com/typhoon.ko 2023-11-24T05:52:07.781 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/8390 2023-11-24T05:52:07.781 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/8390/8390.ko 2023-11-24T05:52:07.781 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2023-11-24T05:52:07.781 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2023-11-24T05:52:07.781 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2023-11-24T05:52:07.781 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/adaptec 2023-11-24T05:52:07.781 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/adaptec/starfire.ko 2023-11-24T05:52:07.781 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/agere 2023-11-24T05:52:07.782 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/agere/et131x.ko 2023-11-24T05:52:07.782 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/alteon 2023-11-24T05:52:07.782 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/alteon/acenic.ko 2023-11-24T05:52:07.782 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/altera 2023-11-24T05:52:07.782 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/altera/altera_tse.ko 2023-11-24T05:52:07.782 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/amd 2023-11-24T05:52:07.782 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/amd/amd8111e.ko 2023-11-24T05:52:07.782 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2023-11-24T05:52:07.782 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/amd/pcnet32.ko 2023-11-24T05:52:07.782 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros 2023-11-24T05:52:07.782 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/alx 2023-11-24T05:52:07.782 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2023-11-24T05:52:07.782 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/atl1c 2023-11-24T05:52:07.783 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2023-11-24T05:52:07.783 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/atl1e 2023-11-24T05:52:07.783 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2023-11-24T05:52:07.783 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/atlx 2023-11-24T05:52:07.783 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2023-11-24T05:52:07.783 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2023-11-24T05:52:07.783 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom 2023-11-24T05:52:07.783 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/b44.ko 2023-11-24T05:52:07.783 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2023-11-24T05:52:07.783 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/bnx2x 2023-11-24T05:52:07.783 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2023-11-24T05:52:07.783 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/cnic.ko 2023-11-24T05:52:07.783 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/genet 2023-11-24T05:52:07.783 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2023-11-24T05:52:07.784 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/tg3.ko 2023-11-24T05:52:07.784 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/brocade 2023-11-24T05:52:07.784 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/brocade/bna 2023-11-24T05:52:07.784 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2023-11-24T05:52:07.784 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio 2023-11-24T05:52:07.784 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb 2023-11-24T05:52:07.784 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2023-11-24T05:52:07.784 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb3 2023-11-24T05:52:07.784 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2023-11-24T05:52:07.784 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb4 2023-11-24T05:52:07.785 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2023-11-24T05:52:07.785 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2023-11-24T05:52:07.785 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2023-11-24T05:52:07.785 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/libcxgb 2023-11-24T05:52:07.785 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2023-11-24T05:52:07.785 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/cisco 2023-11-24T05:52:07.785 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/cisco/enic 2023-11-24T05:52:07.785 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2023-11-24T05:52:07.785 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec 2023-11-24T05:52:07.785 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec/tulip 2023-11-24T05:52:07.786 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2023-11-24T05:52:07.786 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2023-11-24T05:52:07.786 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2023-11-24T05:52:07.786 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2023-11-24T05:52:07.786 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2023-11-24T05:52:07.786 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2023-11-24T05:52:07.786 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dlink 2023-11-24T05:52:07.786 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dlink/dl2k.ko 2023-11-24T05:52:07.786 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dlink/sundance.ko 2023-11-24T05:52:07.786 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dnet.ko 2023-11-24T05:52:07.786 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/emulex 2023-11-24T05:52:07.786 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/emulex/benet 2023-11-24T05:52:07.787 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2023-11-24T05:52:07.789 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/ethoc.ko 2023-11-24T05:52:07.789 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/fealnx.ko 2023-11-24T05:52:07.789 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel 2023-11-24T05:52:07.789 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/e100.ko 2023-11-24T05:52:07.789 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/e1000 2023-11-24T05:52:07.790 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2023-11-24T05:52:07.790 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/e1000e 2023-11-24T05:52:07.790 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2023-11-24T05:52:07.790 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/fm10k 2023-11-24T05:52:07.790 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2023-11-24T05:52:07.790 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/i40e 2023-11-24T05:52:07.790 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2023-11-24T05:52:07.790 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/iavf 2023-11-24T05:52:07.790 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2023-11-24T05:52:07.791 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/igb 2023-11-24T05:52:07.791 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/igb/igb.ko 2023-11-24T05:52:07.791 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/igbvf 2023-11-24T05:52:07.791 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2023-11-24T05:52:07.791 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/ixgbe 2023-11-24T05:52:07.791 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2023-11-24T05:52:07.791 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/ixgbevf 2023-11-24T05:52:07.791 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2023-11-24T05:52:07.792 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/jme.ko 2023-11-24T05:52:07.792 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/marvell 2023-11-24T05:52:07.792 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2023-11-24T05:52:07.792 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/marvell/skge.ko 2023-11-24T05:52:07.792 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/marvell/sky2.ko 2023-11-24T05:52:07.792 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/mellanox 2023-11-24T05:52:07.792 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/mellanox/mlx4 2023-11-24T05:52:07.792 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2023-11-24T05:52:07.792 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2023-11-24T05:52:07.792 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/mellanox/mlx5 2023-11-24T05:52:07.792 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/mellanox/mlx5/core 2023-11-24T05:52:07.792 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2023-11-24T05:52:07.793 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/micrel 2023-11-24T05:52:07.793 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2023-11-24T05:52:07.793 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/myricom 2023-11-24T05:52:07.793 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/myricom/myri10ge 2023-11-24T05:52:07.793 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2023-11-24T05:52:07.793 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/natsemi 2023-11-24T05:52:07.793 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2023-11-24T05:52:07.793 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2023-11-24T05:52:07.793 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/neterion 2023-11-24T05:52:07.793 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/neterion/s2io.ko 2023-11-24T05:52:07.793 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/nvidia 2023-11-24T05:52:07.794 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2023-11-24T05:52:07.794 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/packetengines 2023-11-24T05:52:07.794 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2023-11-24T05:52:07.794 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2023-11-24T05:52:07.794 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/qlogic 2023-11-24T05:52:07.794 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/qlogic/netxen 2023-11-24T05:52:07.794 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2023-11-24T05:52:07.794 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2023-11-24T05:52:07.794 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/qlogic/qlcnic 2023-11-24T05:52:07.794 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2023-11-24T05:52:07.794 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/rdc 2023-11-24T05:52:07.794 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/rdc/r6040.ko 2023-11-24T05:52:07.794 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/realtek 2023-11-24T05:52:07.795 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/realtek/8139cp.ko 2023-11-24T05:52:07.795 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/realtek/8139too.ko 2023-11-24T05:52:07.795 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/realtek/atp.ko 2023-11-24T05:52:07.795 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/realtek/r8169.ko 2023-11-24T05:52:07.795 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/rocker 2023-11-24T05:52:07.795 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/rocker/rocker.ko 2023-11-24T05:52:07.795 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sfc 2023-11-24T05:52:07.795 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sfc/sfc.ko 2023-11-24T05:52:07.795 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/silan 2023-11-24T05:52:07.795 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/silan/sc92031.ko 2023-11-24T05:52:07.795 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sis 2023-11-24T05:52:07.795 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sis/sis190.ko 2023-11-24T05:52:07.796 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sis/sis900.ko 2023-11-24T05:52:07.796 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/smsc 2023-11-24T05:52:07.796 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/smsc/epic100.ko 2023-11-24T05:52:07.796 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2023-11-24T05:52:07.796 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2023-11-24T05:52:07.796 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2023-11-24T05:52:07.796 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/stmicro 2023-11-24T05:52:07.796 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/stmicro/stmmac 2023-11-24T05:52:07.796 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2023-11-24T05:52:07.796 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2023-11-24T05:52:07.796 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sun 2023-11-24T05:52:07.796 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sun/cassini.ko 2023-11-24T05:52:07.796 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sun/niu.ko 2023-11-24T05:52:07.797 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sun/sungem.ko 2023-11-24T05:52:07.797 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sun/sunhme.ko 2023-11-24T05:52:07.797 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/tehuti 2023-11-24T05:52:07.797 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2023-11-24T05:52:07.797 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/ti 2023-11-24T05:52:07.797 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/ti/tlan.ko 2023-11-24T05:52:07.797 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/via 2023-11-24T05:52:07.797 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/via/via-rhine.ko 2023-11-24T05:52:07.797 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/via/via-velocity.ko 2023-11-24T05:52:07.797 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/wiznet 2023-11-24T05:52:07.797 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/wiznet/w5100.ko 2023-11-24T05:52:07.797 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/wiznet/w5300.ko 2023-11-24T05:52:07.798 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/xircom 2023-11-24T05:52:07.798 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2023-11-24T05:52:07.798 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/geneve.ko 2023-11-24T05:52:07.798 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/hyperv 2023-11-24T05:52:07.798 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/hyperv/hv_netvsc.ko 2023-11-24T05:52:07.798 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ieee802154 2023-11-24T05:52:07.798 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ieee802154/fakelb.ko 2023-11-24T05:52:07.798 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ifb.ko 2023-11-24T05:52:07.798 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ipvlan 2023-11-24T05:52:07.798 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ipvlan/ipvlan.ko 2023-11-24T05:52:07.798 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/macvlan.ko 2023-11-24T05:52:07.798 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/macvtap.ko 2023-11-24T05:52:07.798 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/mdio 2023-11-24T05:52:07.799 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/mdio.ko 2023-11-24T05:52:07.799 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2023-11-24T05:52:07.799 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/mdio/mdio-bitbang.ko 2023-11-24T05:52:07.799 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/mdio/mdio-regmap.ko 2023-11-24T05:52:07.799 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/mii.ko 2023-11-24T05:52:07.799 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/net_failover.ko 2023-11-24T05:52:07.799 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/netconsole.ko 2023-11-24T05:52:07.799 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/nlmon.ko 2023-11-24T05:52:07.799 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/pcs 2023-11-24T05:52:07.799 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/pcs/pcs-lynx.ko 2023-11-24T05:52:07.799 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/pcs/pcs_xpcs.ko 2023-11-24T05:52:07.799 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy 2023-11-24T05:52:07.800 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/amd.ko 2023-11-24T05:52:07.800 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/bcm-phy-lib.ko 2023-11-24T05:52:07.800 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/bcm-phy-ptp.ko 2023-11-24T05:52:07.800 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/bcm7xxx.ko 2023-11-24T05:52:07.800 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/bcm87xx.ko 2023-11-24T05:52:07.800 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/broadcom.ko 2023-11-24T05:52:07.800 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/cicada.ko 2023-11-24T05:52:07.800 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/davicom.ko 2023-11-24T05:52:07.800 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/dp83640.ko 2023-11-24T05:52:07.800 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/et1011c.ko 2023-11-24T05:52:07.800 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/icplus.ko 2023-11-24T05:52:07.800 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/lxt.ko 2023-11-24T05:52:07.801 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/marvell.ko 2023-11-24T05:52:07.801 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/micrel.ko 2023-11-24T05:52:07.801 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/national.ko 2023-11-24T05:52:07.801 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/phylink.ko 2023-11-24T05:52:07.801 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/qsemi.ko 2023-11-24T05:52:07.801 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/realtek.ko 2023-11-24T05:52:07.801 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/smsc.ko 2023-11-24T05:52:07.801 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/ste10Xp.ko 2023-11-24T05:52:07.801 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/vitesse.ko 2023-11-24T05:52:07.802 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp 2023-11-24T05:52:07.802 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/bsd_comp.ko 2023-11-24T05:52:07.802 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/ppp_async.ko 2023-11-24T05:52:07.802 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/ppp_deflate.ko 2023-11-24T05:52:07.802 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/ppp_generic.ko 2023-11-24T05:52:07.802 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/ppp_mppe.ko 2023-11-24T05:52:07.802 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/ppp_synctty.ko 2023-11-24T05:52:07.802 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/pppoe.ko 2023-11-24T05:52:07.802 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/pppox.ko 2023-11-24T05:52:07.802 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/pptp.ko 2023-11-24T05:52:07.802 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/slip 2023-11-24T05:52:07.803 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/slip/slhc.ko 2023-11-24T05:52:07.803 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/slip/slip.ko 2023-11-24T05:52:07.803 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/sungem_phy.ko 2023-11-24T05:52:07.803 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/tap.ko 2023-11-24T05:52:07.803 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/team 2023-11-24T05:52:07.803 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/team/team.ko 2023-11-24T05:52:07.803 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/team/team_mode_activebackup.ko 2023-11-24T05:52:07.803 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/team/team_mode_broadcast.ko 2023-11-24T05:52:07.803 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/team/team_mode_loadbalance.ko 2023-11-24T05:52:07.803 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/team/team_mode_random.ko 2023-11-24T05:52:07.803 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/team/team_mode_roundrobin.ko 2023-11-24T05:52:07.803 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/tun.ko 2023-11-24T05:52:07.804 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/veth.ko 2023-11-24T05:52:07.804 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/virtio_net.ko 2023-11-24T05:52:07.804 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/vmxnet3 2023-11-24T05:52:07.804 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/vmxnet3/vmxnet3.ko 2023-11-24T05:52:07.804 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/vxlan 2023-11-24T05:52:07.804 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/vxlan/vxlan.ko 2023-11-24T05:52:07.804 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/xen-netback 2023-11-24T05:52:07.804 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/xen-netback/xen-netback.ko 2023-11-24T05:52:07.804 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/xen-netfront.ko 2023-11-24T05:52:07.804 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ntb 2023-11-24T05:52:07.804 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ntb/ntb.ko 2023-11-24T05:52:07.804 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/nvme 2023-11-24T05:52:07.804 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/nvme/host 2023-11-24T05:52:07.805 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/nvme/host/nvme-core.ko 2023-11-24T05:52:07.805 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/nvme/host/nvme.ko 2023-11-24T05:52:07.805 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/parport 2023-11-24T05:52:07.805 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/parport/parport.ko 2023-11-24T05:52:07.805 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/parport/parport_cs.ko 2023-11-24T05:52:07.805 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/parport/parport_pc.ko 2023-11-24T05:52:07.805 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/parport/parport_serial.ko 2023-11-24T05:52:07.805 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pci 2023-11-24T05:52:07.805 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pci/hotplug 2023-11-24T05:52:07.805 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2023-11-24T05:52:07.805 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pci/pcie 2023-11-24T05:52:07.805 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pci/pcie/aer_inject.ko 2023-11-24T05:52:07.806 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pci/xen-pcifront.ko 2023-11-24T05:52:07.806 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pcmcia 2023-11-24T05:52:07.806 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pcmcia/i82092.ko 2023-11-24T05:52:07.806 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pcmcia/pd6729.ko 2023-11-24T05:52:07.806 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pcmcia/yenta_socket.ko 2023-11-24T05:52:07.806 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pinctrl 2023-11-24T05:52:07.806 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pinctrl/intel 2023-11-24T05:52:07.806 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2023-11-24T05:52:07.806 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform 2023-11-24T05:52:07.806 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86 2023-11-24T05:52:07.806 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/acer-wmi.ko 2023-11-24T05:52:07.806 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/acerhdf.ko 2023-11-24T05:52:07.806 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/apple-gmux.ko 2023-11-24T05:52:07.807 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/asus-laptop.ko 2023-11-24T05:52:07.807 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/asus-nb-wmi.ko 2023-11-24T05:52:07.807 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/asus-wmi.ko 2023-11-24T05:52:07.807 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/classmate-laptop.ko 2023-11-24T05:52:07.807 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/eeepc-laptop.ko 2023-11-24T05:52:07.807 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/eeepc-wmi.ko 2023-11-24T05:52:07.807 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/fujitsu-laptop.ko 2023-11-24T05:52:07.807 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/fujitsu-tablet.ko 2023-11-24T05:52:07.807 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/hdaps.ko 2023-11-24T05:52:07.807 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/intel 2023-11-24T05:52:07.807 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/intel/intel-rst.ko 2023-11-24T05:52:07.807 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/intel_ips.ko 2023-11-24T05:52:07.808 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/msi-wmi.ko 2023-11-24T05:52:07.808 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/mxm-wmi.ko 2023-11-24T05:52:07.808 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/panasonic-laptop.ko 2023-11-24T05:52:07.808 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/samsung-laptop.ko 2023-11-24T05:52:07.808 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/samsung-q10.ko 2023-11-24T05:52:07.808 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/topstar-laptop.ko 2023-11-24T05:52:07.808 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2023-11-24T05:52:07.808 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/toshiba_haps.ko 2023-11-24T05:52:07.808 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/wmi-bmof.ko 2023-11-24T05:52:07.808 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/wmi.ko 2023-11-24T05:52:07.808 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/powercap 2023-11-24T05:52:07.808 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/powercap/intel_rapl_common.ko 2023-11-24T05:52:07.808 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/powercap/intel_rapl_msr.ko 2023-11-24T05:52:07.808 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pps 2023-11-24T05:52:07.809 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pps/clients 2023-11-24T05:52:07.809 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pps/clients/pps-gpio.ko 2023-11-24T05:52:07.809 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pps/clients/pps-ldisc.ko 2023-11-24T05:52:07.809 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pps/clients/pps_parport.ko 2023-11-24T05:52:07.809 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pps/pps_core.ko 2023-11-24T05:52:07.809 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ptp 2023-11-24T05:52:07.809 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ptp/ptp.ko 2023-11-24T05:52:07.809 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ptp/ptp_kvm.ko 2023-11-24T05:52:07.809 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc 2023-11-24T05:52:07.809 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-bq32k.ko 2023-11-24T05:52:07.809 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds1286.ko 2023-11-24T05:52:07.809 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds1307.ko 2023-11-24T05:52:07.810 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds1374.ko 2023-11-24T05:52:07.810 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds1511.ko 2023-11-24T05:52:07.810 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds1553.ko 2023-11-24T05:52:07.810 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds1672.ko 2023-11-24T05:52:07.810 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds1742.ko 2023-11-24T05:52:07.810 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds2404.ko 2023-11-24T05:52:07.810 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds3232.ko 2023-11-24T05:52:07.810 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-em3027.ko 2023-11-24T05:52:07.810 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-fm3130.ko 2023-11-24T05:52:07.810 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-isl12022.ko 2023-11-24T05:52:07.810 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-isl1208.ko 2023-11-24T05:52:07.810 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-m41t80.ko 2023-11-24T05:52:07.811 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-m48t35.ko 2023-11-24T05:52:07.811 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-m48t59.ko 2023-11-24T05:52:07.811 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-max6900.ko 2023-11-24T05:52:07.811 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-msm6242.ko 2023-11-24T05:52:07.811 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-pcf2127.ko 2023-11-24T05:52:07.811 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-pcf85063.ko 2023-11-24T05:52:07.811 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-pcf8523.ko 2023-11-24T05:52:07.811 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-pcf8563.ko 2023-11-24T05:52:07.811 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-pcf8583.ko 2023-11-24T05:52:07.811 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-rp5c01.ko 2023-11-24T05:52:07.811 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-rs5c372.ko 2023-11-24T05:52:07.811 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-rv3029c2.ko 2023-11-24T05:52:07.812 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-rx8025.ko 2023-11-24T05:52:07.812 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-rx8581.ko 2023-11-24T05:52:07.812 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-stk17ta8.ko 2023-11-24T05:52:07.812 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-x1205.ko 2023-11-24T05:52:07.812 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi 2023-11-24T05:52:07.812 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/3w-9xxx.ko 2023-11-24T05:52:07.812 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/3w-sas.ko 2023-11-24T05:52:07.812 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/3w-xxxx.ko 2023-11-24T05:52:07.812 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/BusLogic.ko 2023-11-24T05:52:07.812 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/a100u2w.ko 2023-11-24T05:52:07.812 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/aacraid 2023-11-24T05:52:07.812 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/aacraid/aacraid.ko 2023-11-24T05:52:07.813 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/advansys.ko 2023-11-24T05:52:07.813 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/aic7xxx 2023-11-24T05:52:07.813 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2023-11-24T05:52:07.813 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2023-11-24T05:52:07.813 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/am53c974.ko 2023-11-24T05:52:07.813 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/arcmsr 2023-11-24T05:52:07.813 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/arcmsr/arcmsr.ko 2023-11-24T05:52:07.813 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/atp870u.ko 2023-11-24T05:52:07.813 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/be2iscsi 2023-11-24T05:52:07.813 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2023-11-24T05:52:07.813 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/bfa 2023-11-24T05:52:07.813 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/bfa/bfa.ko 2023-11-24T05:52:07.813 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/bnx2fc 2023-11-24T05:52:07.814 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2023-11-24T05:52:07.814 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/bnx2i 2023-11-24T05:52:07.814 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/bnx2i/bnx2i.ko 2023-11-24T05:52:07.814 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/ch.ko 2023-11-24T05:52:07.814 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/csiostor 2023-11-24T05:52:07.814 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/csiostor/csiostor.ko 2023-11-24T05:52:07.814 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/cxgbi 2023-11-24T05:52:07.814 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/cxgbi/cxgb3i 2023-11-24T05:52:07.814 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2023-11-24T05:52:07.814 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/cxgbi/cxgb4i 2023-11-24T05:52:07.815 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2023-11-24T05:52:07.815 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2023-11-24T05:52:07.815 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/dc395x.ko 2023-11-24T05:52:07.815 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/device_handler 2023-11-24T05:52:07.815 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2023-11-24T05:52:07.815 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2023-11-24T05:52:07.815 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2023-11-24T05:52:07.815 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2023-11-24T05:52:07.815 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/dmx3191d.ko 2023-11-24T05:52:07.815 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/esas2r 2023-11-24T05:52:07.815 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/esas2r/esas2r.ko 2023-11-24T05:52:07.815 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/esp_scsi.ko 2023-11-24T05:52:07.815 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/fcoe 2023-11-24T05:52:07.815 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/fcoe/fcoe.ko 2023-11-24T05:52:07.816 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/fcoe/libfcoe.ko 2023-11-24T05:52:07.816 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/fnic 2023-11-24T05:52:07.816 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/fnic/fnic.ko 2023-11-24T05:52:07.816 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/hpsa.ko 2023-11-24T05:52:07.816 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/hptiop.ko 2023-11-24T05:52:07.816 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/hv_storvsc.ko 2023-11-24T05:52:07.816 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/initio.ko 2023-11-24T05:52:07.816 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/ipr.ko 2023-11-24T05:52:07.816 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/ips.ko 2023-11-24T05:52:07.816 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/isci 2023-11-24T05:52:07.816 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/isci/isci.ko 2023-11-24T05:52:07.816 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2023-11-24T05:52:07.816 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/iscsi_tcp.ko 2023-11-24T05:52:07.817 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/libfc 2023-11-24T05:52:07.817 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/libfc/libfc.ko 2023-11-24T05:52:07.817 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/libiscsi.ko 2023-11-24T05:52:07.817 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/libiscsi_tcp.ko 2023-11-24T05:52:07.817 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/libsas 2023-11-24T05:52:07.817 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/libsas/libsas.ko 2023-11-24T05:52:07.817 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/lpfc 2023-11-24T05:52:07.817 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/lpfc/lpfc.ko 2023-11-24T05:52:07.817 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/megaraid 2023-11-24T05:52:07.817 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/megaraid.ko 2023-11-24T05:52:07.817 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2023-11-24T05:52:07.817 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2023-11-24T05:52:07.817 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2023-11-24T05:52:07.818 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/mpt3sas 2023-11-24T05:52:07.818 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2023-11-24T05:52:07.818 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/mvsas 2023-11-24T05:52:07.818 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/mvsas/mvsas.ko 2023-11-24T05:52:07.818 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/mvumi.ko 2023-11-24T05:52:07.818 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/pm8001 2023-11-24T05:52:07.818 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/pm8001/pm80xx.ko 2023-11-24T05:52:07.818 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/pmcraid.ko 2023-11-24T05:52:07.818 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/qla1280.ko 2023-11-24T05:52:07.818 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/qla2xxx 2023-11-24T05:52:07.818 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2023-11-24T05:52:07.818 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2023-11-24T05:52:07.818 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/qla4xxx 2023-11-24T05:52:07.819 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2023-11-24T05:52:07.819 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/raid_class.ko 2023-11-24T05:52:07.819 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/scsi_debug.ko 2023-11-24T05:52:07.819 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/scsi_transport_fc.ko 2023-11-24T05:52:07.819 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/scsi_transport_iscsi.ko 2023-11-24T05:52:07.819 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/scsi_transport_sas.ko 2023-11-24T05:52:07.819 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/scsi_transport_spi.ko 2023-11-24T05:52:07.819 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/scsi_transport_srp.ko 2023-11-24T05:52:07.819 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/ses.ko 2023-11-24T05:52:07.819 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/st.ko 2023-11-24T05:52:07.819 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/stex.ko 2023-11-24T05:52:07.819 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/sym53c8xx_2 2023-11-24T05:52:07.819 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2023-11-24T05:52:07.820 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/virtio_scsi.ko 2023-11-24T05:52:07.820 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/vmw_pvscsi.ko 2023-11-24T05:52:07.820 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/wd719x.ko 2023-11-24T05:52:07.820 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ssb 2023-11-24T05:52:07.820 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ssb/ssb.ko 2023-11-24T05:52:07.820 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target 2023-11-24T05:52:07.820 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/iscsi 2023-11-24T05:52:07.820 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2023-11-24T05:52:07.820 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/loopback 2023-11-24T05:52:07.820 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/loopback/tcm_loop.ko 2023-11-24T05:52:07.820 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/target_core_file.ko 2023-11-24T05:52:07.820 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/target_core_iblock.ko 2023-11-24T05:52:07.820 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/target_core_mod.ko 2023-11-24T05:52:07.821 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/target_core_pscsi.ko 2023-11-24T05:52:07.821 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/target_core_user.ko 2023-11-24T05:52:07.821 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/tcm_fc 2023-11-24T05:52:07.821 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/tcm_fc/tcm_fc.ko 2023-11-24T05:52:07.821 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal 2023-11-24T05:52:07.821 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel 2023-11-24T05:52:07.821 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal 2023-11-24T05:52:07.821 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2023-11-24T05:52:07.821 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2023-11-24T05:52:07.821 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2023-11-24T05:52:07.821 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2023-11-24T05:52:07.821 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2023-11-24T05:52:07.821 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2023-11-24T05:52:07.821 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2023-11-24T05:52:07.822 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2023-11-24T05:52:07.822 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2023-11-24T05:52:07.822 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2023-11-24T05:52:07.822 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2023-11-24T05:52:07.822 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2023-11-24T05:52:07.822 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2023-11-24T05:52:07.822 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2023-11-24T05:52:07.822 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2023-11-24T05:52:07.822 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty 2023-11-24T05:52:07.823 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/ipwireless 2023-11-24T05:52:07.823 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/ipwireless/ipwireless.ko 2023-11-24T05:52:07.823 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/n_gsm.ko 2023-11-24T05:52:07.823 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/n_hdlc.ko 2023-11-24T05:52:07.823 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/nozomi.ko 2023-11-24T05:52:07.823 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial 2023-11-24T05:52:07.823 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial/8250 2023-11-24T05:52:07.823 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial/8250/serial_cs.ko 2023-11-24T05:52:07.823 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial/arc_uart.ko 2023-11-24T05:52:07.823 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial/jsm 2023-11-24T05:52:07.823 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial/jsm/jsm.ko 2023-11-24T05:52:07.823 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ufs 2023-11-24T05:52:07.823 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ufs/core 2023-11-24T05:52:07.823 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ufs/core/ufshcd-core.ko 2023-11-24T05:52:07.824 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ufs/host 2023-11-24T05:52:07.824 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ufs/host/ufshcd-pci.ko 2023-11-24T05:52:07.824 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/uio 2023-11-24T05:52:07.824 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/uio/uio.ko 2023-11-24T05:52:07.824 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/uio/uio_aec.ko 2023-11-24T05:52:07.824 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/uio/uio_cif.ko 2023-11-24T05:52:07.824 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/uio/uio_pci_generic.ko 2023-11-24T05:52:07.824 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/uio/uio_sercos3.ko 2023-11-24T05:52:07.824 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb 2023-11-24T05:52:07.824 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/class 2023-11-24T05:52:07.824 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/class/cdc-acm.ko 2023-11-24T05:52:07.824 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/class/cdc-wdm.ko 2023-11-24T05:52:07.824 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/class/usblp.ko 2023-11-24T05:52:07.825 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/class/usbtmc.ko 2023-11-24T05:52:07.825 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/host 2023-11-24T05:52:07.825 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/host/sl811-hcd.ko 2023-11-24T05:52:07.825 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/image 2023-11-24T05:52:07.825 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/image/mdc800.ko 2023-11-24T05:52:07.825 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/image/microtek.ko 2023-11-24T05:52:07.825 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc 2023-11-24T05:52:07.825 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/adutux.ko 2023-11-24T05:52:07.825 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/appledisplay.ko 2023-11-24T05:52:07.825 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/emi26.ko 2023-11-24T05:52:07.825 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/emi62.ko 2023-11-24T05:52:07.825 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/ezusb.ko 2023-11-24T05:52:07.826 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/idmouse.ko 2023-11-24T05:52:07.826 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/iowarrior.ko 2023-11-24T05:52:07.826 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/isight_firmware.ko 2023-11-24T05:52:07.826 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/ldusb.ko 2023-11-24T05:52:07.826 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/legousbtower.ko 2023-11-24T05:52:07.826 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/sisusbvga 2023-11-24T05:52:07.826 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2023-11-24T05:52:07.826 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/trancevibrator.ko 2023-11-24T05:52:07.826 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/usb3503.ko 2023-11-24T05:52:07.826 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/usblcd.ko 2023-11-24T05:52:07.826 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/usbsevseg.ko 2023-11-24T05:52:07.826 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/uss720.ko 2023-11-24T05:52:07.826 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/yurex.ko 2023-11-24T05:52:07.827 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/phy 2023-11-24T05:52:07.827 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/phy/phy-generic.ko 2023-11-24T05:52:07.827 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial 2023-11-24T05:52:07.827 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/aircable.ko 2023-11-24T05:52:07.827 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ark3116.ko 2023-11-24T05:52:07.827 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/belkin_sa.ko 2023-11-24T05:52:07.827 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ch341.ko 2023-11-24T05:52:07.827 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/cp210x.ko 2023-11-24T05:52:07.827 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/cyberjack.ko 2023-11-24T05:52:07.827 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/cypress_m8.ko 2023-11-24T05:52:07.828 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/digi_acceleport.ko 2023-11-24T05:52:07.828 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/empeg.ko 2023-11-24T05:52:07.828 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ftdi_sio.ko 2023-11-24T05:52:07.828 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/garmin_gps.ko 2023-11-24T05:52:07.828 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/io_edgeport.ko 2023-11-24T05:52:07.828 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/io_ti.ko 2023-11-24T05:52:07.828 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ipaq.ko 2023-11-24T05:52:07.828 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ipw.ko 2023-11-24T05:52:07.828 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ir-usb.ko 2023-11-24T05:52:07.828 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/iuu_phoenix.ko 2023-11-24T05:52:07.828 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/keyspan.ko 2023-11-24T05:52:07.828 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/keyspan_pda.ko 2023-11-24T05:52:07.829 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/kl5kusb105.ko 2023-11-24T05:52:07.829 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/kobil_sct.ko 2023-11-24T05:52:07.829 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/mct_u232.ko 2023-11-24T05:52:07.829 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/mos7720.ko 2023-11-24T05:52:07.829 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/mos7840.ko 2023-11-24T05:52:07.829 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/navman.ko 2023-11-24T05:52:07.829 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/omninet.ko 2023-11-24T05:52:07.829 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/opticon.ko 2023-11-24T05:52:07.829 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/option.ko 2023-11-24T05:52:07.829 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/oti6858.ko 2023-11-24T05:52:07.829 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/pl2303.ko 2023-11-24T05:52:07.829 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/qcaux.ko 2023-11-24T05:52:07.829 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/qcserial.ko 2023-11-24T05:52:07.830 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/quatech2.ko 2023-11-24T05:52:07.830 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/safe_serial.ko 2023-11-24T05:52:07.830 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/sierra.ko 2023-11-24T05:52:07.830 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/spcp8x5.ko 2023-11-24T05:52:07.830 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ssu100.ko 2023-11-24T05:52:07.830 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/symbolserial.ko 2023-11-24T05:52:07.830 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2023-11-24T05:52:07.830 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/usb-serial-simple.ko 2023-11-24T05:52:07.830 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/usb_debug.ko 2023-11-24T05:52:07.830 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/usb_wwan.ko 2023-11-24T05:52:07.830 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/visor.ko 2023-11-24T05:52:07.830 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/whiteheat.ko 2023-11-24T05:52:07.831 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/xsens_mt.ko 2023-11-24T05:52:07.831 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage 2023-11-24T05:52:07.831 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/uas.ko 2023-11-24T05:52:07.831 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-alauda.ko 2023-11-24T05:52:07.831 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-cypress.ko 2023-11-24T05:52:07.831 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-datafab.ko 2023-11-24T05:52:07.831 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-eneub6250.ko 2023-11-24T05:52:07.831 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-freecom.ko 2023-11-24T05:52:07.831 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-isd200.ko 2023-11-24T05:52:07.831 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-jumpshot.ko 2023-11-24T05:52:07.831 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-karma.ko 2023-11-24T05:52:07.831 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-onetouch.ko 2023-11-24T05:52:07.831 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-realtek.ko 2023-11-24T05:52:07.832 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-sddr09.ko 2023-11-24T05:52:07.832 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-sddr55.ko 2023-11-24T05:52:07.832 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-usbat.ko 2023-11-24T05:52:07.832 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/usb-storage.ko 2023-11-24T05:52:07.832 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/usbip 2023-11-24T05:52:07.832 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/usbip/usbip-core.ko 2023-11-24T05:52:07.832 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/usbip/usbip-host.ko 2023-11-24T05:52:07.832 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/usbip/vhci-hcd.ko 2023-11-24T05:52:07.832 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vfio 2023-11-24T05:52:07.832 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vfio/pci 2023-11-24T05:52:07.832 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vfio/pci/vfio-pci-core.ko 2023-11-24T05:52:07.832 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vfio/pci/vfio-pci.ko 2023-11-24T05:52:07.833 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vfio/vfio.ko 2023-11-24T05:52:07.833 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vfio/vfio_iommu_type1.ko 2023-11-24T05:52:07.833 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vhost 2023-11-24T05:52:07.833 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vhost/vhost.ko 2023-11-24T05:52:07.833 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vhost/vhost_iotlb.ko 2023-11-24T05:52:07.833 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vhost/vhost_net.ko 2023-11-24T05:52:07.833 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vhost/vhost_scsi.ko 2023-11-24T05:52:07.833 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video 2023-11-24T05:52:07.833 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight 2023-11-24T05:52:07.833 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/apple_bl.ko 2023-11-24T05:52:07.833 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/lcd.ko 2023-11-24T05:52:07.833 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/platform_lcd.ko 2023-11-24T05:52:07.833 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/virtio 2023-11-24T05:52:07.834 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/virtio/virtio.ko 2023-11-24T05:52:07.834 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/virtio/virtio_balloon.ko 2023-11-24T05:52:07.834 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/virtio/virtio_mmio.ko 2023-11-24T05:52:07.834 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/virtio/virtio_pci.ko 2023-11-24T05:52:07.834 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2023-11-24T05:52:07.834 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2023-11-24T05:52:07.834 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/virtio/virtio_ring.ko 2023-11-24T05:52:07.834 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1 2023-11-24T05:52:07.834 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/masters 2023-11-24T05:52:07.834 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/masters/ds2482.ko 2023-11-24T05:52:07.834 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/masters/ds2490.ko 2023-11-24T05:52:07.834 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves 2023-11-24T05:52:07.835 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2406.ko 2023-11-24T05:52:07.835 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2408.ko 2023-11-24T05:52:07.835 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2413.ko 2023-11-24T05:52:07.835 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2423.ko 2023-11-24T05:52:07.835 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2431.ko 2023-11-24T05:52:07.835 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2433.ko 2023-11-24T05:52:07.835 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2780.ko 2023-11-24T05:52:07.835 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2781.ko 2023-11-24T05:52:07.835 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds28e04.ko 2023-11-24T05:52:07.835 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_smem.ko 2023-11-24T05:52:07.835 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_therm.ko 2023-11-24T05:52:07.835 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/wire.ko 2023-11-24T05:52:07.835 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog 2023-11-24T05:52:07.836 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/alim1535_wdt.ko 2023-11-24T05:52:07.836 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/alim7101_wdt.ko 2023-11-24T05:52:07.836 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/f71808e_wdt.ko 2023-11-24T05:52:07.836 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/hpwdt.ko 2023-11-24T05:52:07.836 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/i6300esb.ko 2023-11-24T05:52:07.836 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/iTCO_vendor_support.ko 2023-11-24T05:52:07.836 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/iTCO_wdt.ko 2023-11-24T05:52:07.836 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/ib700wdt.ko 2023-11-24T05:52:07.836 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/ibmasr.ko 2023-11-24T05:52:07.836 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/ie6xx_wdt.ko 2023-11-24T05:52:07.836 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/it8712f_wdt.ko 2023-11-24T05:52:07.836 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/it87_wdt.ko 2023-11-24T05:52:07.836 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/machzwd.ko 2023-11-24T05:52:07.837 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/nv_tco.ko 2023-11-24T05:52:07.837 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/pcwd_pci.ko 2023-11-24T05:52:07.837 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/pcwd_usb.ko 2023-11-24T05:52:07.837 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2023-11-24T05:52:07.837 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/sch311x_wdt.ko 2023-11-24T05:52:07.837 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/softdog.ko 2023-11-24T05:52:07.837 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/sp5100_tco.ko 2023-11-24T05:52:07.837 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/via_wdt.ko 2023-11-24T05:52:07.837 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/w83627hf_wdt.ko 2023-11-24T05:52:07.837 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/w83877f_wdt.ko 2023-11-24T05:52:07.837 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/w83977f_wdt.ko 2023-11-24T05:52:07.837 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/wdt_pci.ko 2023-11-24T05:52:07.837 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/xen_wdt.ko 2023-11-24T05:52:07.838 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen 2023-11-24T05:52:07.838 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xen-acpi-processor.ko 2023-11-24T05:52:07.838 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xen-evtchn.ko 2023-11-24T05:52:07.838 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xen-gntalloc.ko 2023-11-24T05:52:07.838 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xen-gntdev.ko 2023-11-24T05:52:07.838 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xen-pciback 2023-11-24T05:52:07.838 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2023-11-24T05:52:07.838 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xen-privcmd.ko 2023-11-24T05:52:07.838 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xenfs 2023-11-24T05:52:07.838 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xenfs/xenfs.ko 2023-11-24T05:52:07.838 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs 2023-11-24T05:52:07.838 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/9p 2023-11-24T05:52:07.838 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/9p/9p.ko 2023-11-24T05:52:07.839 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/affs 2023-11-24T05:52:07.839 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/affs/affs.ko 2023-11-24T05:52:07.839 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/befs 2023-11-24T05:52:07.839 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/befs/befs.ko 2023-11-24T05:52:07.839 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/binfmt_misc.ko 2023-11-24T05:52:07.839 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/btrfs 2023-11-24T05:52:07.839 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/btrfs/btrfs.ko 2023-11-24T05:52:07.839 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/cachefiles 2023-11-24T05:52:07.839 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/cachefiles/cachefiles.ko 2023-11-24T05:52:07.839 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ceph 2023-11-24T05:52:07.839 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ceph/ceph.ko 2023-11-24T05:52:07.839 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/coda 2023-11-24T05:52:07.839 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/coda/coda.ko 2023-11-24T05:52:07.840 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/cramfs 2023-11-24T05:52:07.840 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/cramfs/cramfs.ko 2023-11-24T05:52:07.840 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/dlm 2023-11-24T05:52:07.840 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/dlm/dlm.ko 2023-11-24T05:52:07.840 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ecryptfs 2023-11-24T05:52:07.840 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ecryptfs/ecryptfs.ko 2023-11-24T05:52:07.840 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/f2fs 2023-11-24T05:52:07.840 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/f2fs/f2fs.ko 2023-11-24T05:52:07.840 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fat 2023-11-24T05:52:07.840 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fat/fat.ko 2023-11-24T05:52:07.840 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fat/msdos.ko 2023-11-24T05:52:07.840 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fat/vfat.ko 2023-11-24T05:52:07.840 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fscache 2023-11-24T05:52:07.840 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fscache/fscache.ko 2023-11-24T05:52:07.841 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fuse 2023-11-24T05:52:07.841 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fuse/cuse.ko 2023-11-24T05:52:07.841 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fuse/fuse.ko 2023-11-24T05:52:07.841 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/gfs2 2023-11-24T05:52:07.841 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/gfs2/gfs2.ko 2023-11-24T05:52:07.841 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/hfs 2023-11-24T05:52:07.841 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/hfs/hfs.ko 2023-11-24T05:52:07.841 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/hfsplus 2023-11-24T05:52:07.841 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/hfsplus/hfsplus.ko 2023-11-24T05:52:07.841 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/isofs 2023-11-24T05:52:07.841 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/isofs/isofs.ko 2023-11-24T05:52:07.841 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/jfs 2023-11-24T05:52:07.841 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/jfs/jfs.ko 2023-11-24T05:52:07.842 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/lockd 2023-11-24T05:52:07.842 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/lockd/lockd.ko 2023-11-24T05:52:07.842 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/minix 2023-11-24T05:52:07.842 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/minix/minix.ko 2023-11-24T05:52:07.842 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/netfs 2023-11-24T05:52:07.842 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/netfs/netfs.ko 2023-11-24T05:52:07.842 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs 2023-11-24T05:52:07.842 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/blocklayout 2023-11-24T05:52:07.842 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2023-11-24T05:52:07.842 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/filelayout 2023-11-24T05:52:07.842 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2023-11-24T05:52:07.842 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/flexfilelayout 2023-11-24T05:52:07.843 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2023-11-24T05:52:07.843 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/nfs.ko 2023-11-24T05:52:07.843 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/nfsv3.ko 2023-11-24T05:52:07.843 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/nfsv4.ko 2023-11-24T05:52:07.843 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs_common 2023-11-24T05:52:07.843 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs_common/grace.ko 2023-11-24T05:52:07.843 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs_common/nfs_acl.ko 2023-11-24T05:52:07.843 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfsd 2023-11-24T05:52:07.843 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfsd/nfsd.ko 2023-11-24T05:52:07.843 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nilfs2 2023-11-24T05:52:07.843 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nilfs2/nilfs2.ko 2023-11-24T05:52:07.843 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls 2023-11-24T05:52:07.844 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-celtic.ko 2023-11-24T05:52:07.844 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-centeuro.ko 2023-11-24T05:52:07.844 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-croatian.ko 2023-11-24T05:52:07.844 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-cyrillic.ko 2023-11-24T05:52:07.844 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-gaelic.ko 2023-11-24T05:52:07.844 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-greek.ko 2023-11-24T05:52:07.844 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-iceland.ko 2023-11-24T05:52:07.844 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-inuit.ko 2023-11-24T05:52:07.844 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-roman.ko 2023-11-24T05:52:07.844 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-romanian.ko 2023-11-24T05:52:07.844 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-turkish.ko 2023-11-24T05:52:07.844 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp1250.ko 2023-11-24T05:52:07.844 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp1251.ko 2023-11-24T05:52:07.844 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp1255.ko 2023-11-24T05:52:07.845 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp737.ko 2023-11-24T05:52:07.845 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp775.ko 2023-11-24T05:52:07.845 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp850.ko 2023-11-24T05:52:07.845 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp852.ko 2023-11-24T05:52:07.845 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp855.ko 2023-11-24T05:52:07.845 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp857.ko 2023-11-24T05:52:07.845 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp860.ko 2023-11-24T05:52:07.845 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp861.ko 2023-11-24T05:52:07.845 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp862.ko 2023-11-24T05:52:07.845 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp863.ko 2023-11-24T05:52:07.845 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp864.ko 2023-11-24T05:52:07.845 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp865.ko 2023-11-24T05:52:07.845 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp866.ko 2023-11-24T05:52:07.845 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp869.ko 2023-11-24T05:52:07.846 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp874.ko 2023-11-24T05:52:07.846 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp932.ko 2023-11-24T05:52:07.846 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp936.ko 2023-11-24T05:52:07.846 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp949.ko 2023-11-24T05:52:07.846 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp950.ko 2023-11-24T05:52:07.846 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_euc-jp.ko 2023-11-24T05:52:07.846 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-1.ko 2023-11-24T05:52:07.846 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-13.ko 2023-11-24T05:52:07.846 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-14.ko 2023-11-24T05:52:07.846 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-15.ko 2023-11-24T05:52:07.846 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-2.ko 2023-11-24T05:52:07.846 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-3.ko 2023-11-24T05:52:07.846 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-4.ko 2023-11-24T05:52:07.847 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-5.ko 2023-11-24T05:52:07.847 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-6.ko 2023-11-24T05:52:07.847 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-7.ko 2023-11-24T05:52:07.847 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-9.ko 2023-11-24T05:52:07.847 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_koi8-r.ko 2023-11-24T05:52:07.847 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_koi8-ru.ko 2023-11-24T05:52:07.847 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_koi8-u.ko 2023-11-24T05:52:07.847 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_ucs2_utils.ko 2023-11-24T05:52:07.847 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_utf8.ko 2023-11-24T05:52:07.847 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2 2023-11-24T05:52:07.847 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/cluster 2023-11-24T05:52:07.847 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2023-11-24T05:52:07.847 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/dlm 2023-11-24T05:52:07.847 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2023-11-24T05:52:07.848 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/dlmfs 2023-11-24T05:52:07.848 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2023-11-24T05:52:07.848 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/ocfs2.ko 2023-11-24T05:52:07.848 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2023-11-24T05:52:07.848 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/ocfs2_stack_user.ko 2023-11-24T05:52:07.848 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/ocfs2_stackglue.ko 2023-11-24T05:52:07.848 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/overlayfs 2023-11-24T05:52:07.848 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/overlayfs/overlay.ko 2023-11-24T05:52:07.848 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/pstore 2023-11-24T05:52:07.848 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/pstore/ramoops.ko 2023-11-24T05:52:07.848 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/reiserfs 2023-11-24T05:52:07.848 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/reiserfs/reiserfs.ko 2023-11-24T05:52:07.848 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/romfs 2023-11-24T05:52:07.849 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/romfs/romfs.ko 2023-11-24T05:52:07.849 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/smb 2023-11-24T05:52:07.849 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/smb/client 2023-11-24T05:52:07.849 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/smb/client/cifs.ko 2023-11-24T05:52:07.849 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/smb/common 2023-11-24T05:52:07.849 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/smb/common/cifs_arc4.ko 2023-11-24T05:52:07.849 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/smb/common/cifs_md4.ko 2023-11-24T05:52:07.849 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/squashfs 2023-11-24T05:52:07.849 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/squashfs/squashfs.ko 2023-11-24T05:52:07.849 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/sysv 2023-11-24T05:52:07.849 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/sysv/sysv.ko 2023-11-24T05:52:07.849 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/udf 2023-11-24T05:52:07.849 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/udf/udf.ko 2023-11-24T05:52:07.850 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ufs 2023-11-24T05:52:07.850 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ufs/ufs.ko 2023-11-24T05:52:07.850 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/xfs 2023-11-24T05:52:07.850 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/xfs/xfs.ko 2023-11-24T05:52:07.850 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/kernel 2023-11-24T05:52:07.850 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/kernel/rcu 2023-11-24T05:52:07.850 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/kernel/rcu/rcutorture.ko 2023-11-24T05:52:07.850 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/kernel/torture.ko 2023-11-24T05:52:07.850 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/kernel/trace 2023-11-24T05:52:07.850 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/kernel/trace/ring_buffer_benchmark.ko 2023-11-24T05:52:07.850 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib 2023-11-24T05:52:07.850 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/asn1_encoder.ko 2023-11-24T05:52:07.850 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/crc-itu-t.ko 2023-11-24T05:52:07.851 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/crc8.ko 2023-11-24T05:52:07.851 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/crypto 2023-11-24T05:52:07.851 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/crypto/libarc4.ko 2023-11-24T05:52:07.851 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/crypto/libdes.ko 2023-11-24T05:52:07.851 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/libcrc32c.ko 2023-11-24T05:52:07.851 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/lru_cache.ko 2023-11-24T05:52:07.851 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/lz4 2023-11-24T05:52:07.851 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/lz4/lz4_compress.ko 2023-11-24T05:52:07.851 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/lz4/lz4hc_compress.ko 2023-11-24T05:52:07.851 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/math 2023-11-24T05:52:07.851 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/math/cordic.ko 2023-11-24T05:52:07.851 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/raid6 2023-11-24T05:52:07.851 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/raid6/raid6_pq.ko 2023-11-24T05:52:07.852 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/reed_solomon 2023-11-24T05:52:07.852 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/reed_solomon/reed_solomon.ko 2023-11-24T05:52:07.852 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/ts_bm.ko 2023-11-24T05:52:07.852 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/ts_fsm.ko 2023-11-24T05:52:07.852 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/ts_kmp.ko 2023-11-24T05:52:07.852 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/zstd 2023-11-24T05:52:07.852 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/zstd/zstd_compress.ko 2023-11-24T05:52:07.852 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/mm 2023-11-24T05:52:07.852 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/mm/hwpoison-inject.ko 2023-11-24T05:52:07.852 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net 2023-11-24T05:52:07.852 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan 2023-11-24T05:52:07.852 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/6lowpan.ko 2023-11-24T05:52:07.852 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/nhc_dest.ko 2023-11-24T05:52:07.853 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/nhc_fragment.ko 2023-11-24T05:52:07.853 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/nhc_hop.ko 2023-11-24T05:52:07.853 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/nhc_ipv6.ko 2023-11-24T05:52:07.853 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/nhc_mobility.ko 2023-11-24T05:52:07.853 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/nhc_routing.ko 2023-11-24T05:52:07.853 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/nhc_udp.ko 2023-11-24T05:52:07.853 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/802 2023-11-24T05:52:07.853 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/802/garp.ko 2023-11-24T05:52:07.853 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/802/mrp.ko 2023-11-24T05:52:07.853 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/802/p8022.ko 2023-11-24T05:52:07.853 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/802/psnap.ko 2023-11-24T05:52:07.853 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/802/stp.ko 2023-11-24T05:52:07.853 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/8021q 2023-11-24T05:52:07.854 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/8021q/8021q.ko 2023-11-24T05:52:07.854 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/9p 2023-11-24T05:52:07.854 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/9p/9pnet.ko 2023-11-24T05:52:07.854 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/9p/9pnet_fd.ko 2023-11-24T05:52:07.854 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/9p/9pnet_rdma.ko 2023-11-24T05:52:07.854 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/9p/9pnet_virtio.ko 2023-11-24T05:52:07.854 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/batman-adv 2023-11-24T05:52:07.854 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/batman-adv/batman-adv.ko 2023-11-24T05:52:07.854 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge 2023-11-24T05:52:07.854 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/br_netfilter.ko 2023-11-24T05:52:07.854 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/bridge.ko 2023-11-24T05:52:07.854 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter 2023-11-24T05:52:07.854 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_802_3.ko 2023-11-24T05:52:07.854 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_among.ko 2023-11-24T05:52:07.855 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_arp.ko 2023-11-24T05:52:07.855 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_arpreply.ko 2023-11-24T05:52:07.855 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_dnat.ko 2023-11-24T05:52:07.855 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_ip.ko 2023-11-24T05:52:07.855 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_ip6.ko 2023-11-24T05:52:07.855 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_limit.ko 2023-11-24T05:52:07.855 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_log.ko 2023-11-24T05:52:07.855 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_mark.ko 2023-11-24T05:52:07.855 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_mark_m.ko 2023-11-24T05:52:07.855 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_nflog.ko 2023-11-24T05:52:07.855 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_pkttype.ko 2023-11-24T05:52:07.855 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_redirect.ko 2023-11-24T05:52:07.855 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_snat.ko 2023-11-24T05:52:07.856 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_stp.ko 2023-11-24T05:52:07.856 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_vlan.ko 2023-11-24T05:52:07.856 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebtable_broute.ko 2023-11-24T05:52:07.856 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebtable_filter.ko 2023-11-24T05:52:07.856 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebtable_nat.ko 2023-11-24T05:52:07.856 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebtables.ko 2023-11-24T05:52:07.856 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2023-11-24T05:52:07.856 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2023-11-24T05:52:07.856 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2023-11-24T05:52:07.856 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ceph 2023-11-24T05:52:07.856 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ceph/libceph.ko 2023-11-24T05:52:07.856 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/core 2023-11-24T05:52:07.856 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/core/failover.ko 2023-11-24T05:52:07.856 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/core/pktgen.ko 2023-11-24T05:52:07.857 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dccp 2023-11-24T05:52:07.857 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dccp/dccp.ko 2023-11-24T05:52:07.857 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dccp/dccp_diag.ko 2023-11-24T05:52:07.857 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dccp/dccp_ipv4.ko 2023-11-24T05:52:07.857 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dccp/dccp_ipv6.ko 2023-11-24T05:52:07.857 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dns_resolver 2023-11-24T05:52:07.857 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dns_resolver/dns_resolver.ko 2023-11-24T05:52:07.857 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dsa 2023-11-24T05:52:07.857 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dsa/dsa_core.ko 2023-11-24T05:52:07.857 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dsa/tag_brcm.ko 2023-11-24T05:52:07.857 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dsa/tag_dsa.ko 2023-11-24T05:52:07.857 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dsa/tag_none.ko 2023-11-24T05:52:07.857 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dsa/tag_trailer.ko 2023-11-24T05:52:07.857 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ieee802154 2023-11-24T05:52:07.858 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ieee802154/6lowpan 2023-11-24T05:52:07.858 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2023-11-24T05:52:07.858 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ieee802154/ieee802154.ko 2023-11-24T05:52:07.858 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ieee802154/ieee802154_socket.ko 2023-11-24T05:52:07.858 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4 2023-11-24T05:52:07.858 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/ah4.ko 2023-11-24T05:52:07.858 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/esp4.ko 2023-11-24T05:52:07.858 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/fou.ko 2023-11-24T05:52:07.858 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/gre.ko 2023-11-24T05:52:07.858 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/inet_diag.ko 2023-11-24T05:52:07.858 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/ip_gre.ko 2023-11-24T05:52:07.859 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/ip_tunnel.ko 2023-11-24T05:52:07.859 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/ip_vti.ko 2023-11-24T05:52:07.859 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/ipcomp.ko 2023-11-24T05:52:07.859 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/ipip.ko 2023-11-24T05:52:07.859 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter 2023-11-24T05:52:07.859 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/arp_tables.ko 2023-11-24T05:52:07.859 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/arpt_mangle.ko 2023-11-24T05:52:07.859 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/arptable_filter.ko 2023-11-24T05:52:07.859 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/ipt_ECN.ko 2023-11-24T05:52:07.859 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2023-11-24T05:52:07.859 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/ipt_ah.ko 2023-11-24T05:52:07.859 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2023-11-24T05:52:07.859 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/iptable_mangle.ko 2023-11-24T05:52:07.859 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/iptable_nat.ko 2023-11-24T05:52:07.860 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/iptable_raw.ko 2023-11-24T05:52:07.860 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/iptable_security.ko 2023-11-24T05:52:07.860 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2023-11-24T05:52:07.860 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2023-11-24T05:52:07.860 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2023-11-24T05:52:07.860 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2023-11-24T05:52:07.860 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2023-11-24T05:52:07.860 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2023-11-24T05:52:07.860 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2023-11-24T05:52:07.860 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2023-11-24T05:52:07.860 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2023-11-24T05:52:07.860 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2023-11-24T05:52:07.861 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_bic.ko 2023-11-24T05:52:07.861 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_dctcp.ko 2023-11-24T05:52:07.861 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_diag.ko 2023-11-24T05:52:07.861 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_highspeed.ko 2023-11-24T05:52:07.861 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_htcp.ko 2023-11-24T05:52:07.861 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_hybla.ko 2023-11-24T05:52:07.861 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_illinois.ko 2023-11-24T05:52:07.861 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_lp.ko 2023-11-24T05:52:07.861 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_scalable.ko 2023-11-24T05:52:07.861 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_vegas.ko 2023-11-24T05:52:07.861 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_veno.ko 2023-11-24T05:52:07.861 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_westwood.ko 2023-11-24T05:52:07.861 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_yeah.ko 2023-11-24T05:52:07.862 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tunnel4.ko 2023-11-24T05:52:07.862 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/udp_diag.ko 2023-11-24T05:52:07.862 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/udp_tunnel.ko 2023-11-24T05:52:07.862 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/xfrm4_tunnel.ko 2023-11-24T05:52:07.862 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6 2023-11-24T05:52:07.862 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/ah6.ko 2023-11-24T05:52:07.862 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/esp6.ko 2023-11-24T05:52:07.862 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/fou6.ko 2023-11-24T05:52:07.862 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/ip6_tunnel.ko 2023-11-24T05:52:07.862 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/ip6_udp_tunnel.ko 2023-11-24T05:52:07.862 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/ip6_vti.ko 2023-11-24T05:52:07.863 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/ipcomp6.ko 2023-11-24T05:52:07.863 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter 2023-11-24T05:52:07.863 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6_tables.ko 2023-11-24T05:52:07.863 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2023-11-24T05:52:07.863 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2023-11-24T05:52:07.863 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_ah.ko 2023-11-24T05:52:07.863 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2023-11-24T05:52:07.863 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_frag.ko 2023-11-24T05:52:07.863 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2023-11-24T05:52:07.863 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2023-11-24T05:52:07.863 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_mh.ko 2023-11-24T05:52:07.863 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2023-11-24T05:52:07.863 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_rt.ko 2023-11-24T05:52:07.864 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6table_filter.ko 2023-11-24T05:52:07.864 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2023-11-24T05:52:07.864 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6table_nat.ko 2023-11-24T05:52:07.864 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6table_raw.ko 2023-11-24T05:52:07.864 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6table_security.ko 2023-11-24T05:52:07.864 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2023-11-24T05:52:07.864 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2023-11-24T05:52:07.864 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2023-11-24T05:52:07.864 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2023-11-24T05:52:07.864 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2023-11-24T05:52:07.864 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2023-11-24T05:52:07.864 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2023-11-24T05:52:07.864 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2023-11-24T05:52:07.865 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/sit.ko 2023-11-24T05:52:07.865 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/tunnel6.ko 2023-11-24T05:52:07.865 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/xfrm6_tunnel.ko 2023-11-24T05:52:07.865 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/key 2023-11-24T05:52:07.865 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/key/af_key.ko 2023-11-24T05:52:07.865 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp 2023-11-24T05:52:07.865 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp/l2tp_core.ko 2023-11-24T05:52:07.865 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp/l2tp_debugfs.ko 2023-11-24T05:52:07.865 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp/l2tp_eth.ko 2023-11-24T05:52:07.865 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp/l2tp_ip.ko 2023-11-24T05:52:07.865 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp/l2tp_ip6.ko 2023-11-24T05:52:07.865 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp/l2tp_netlink.ko 2023-11-24T05:52:07.865 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp/l2tp_ppp.ko 2023-11-24T05:52:07.866 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/llc 2023-11-24T05:52:07.866 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/llc/llc.ko 2023-11-24T05:52:07.866 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/mac802154 2023-11-24T05:52:07.866 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/mac802154/mac802154.ko 2023-11-24T05:52:07.866 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/mpls 2023-11-24T05:52:07.866 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/mpls/mpls_gso.ko 2023-11-24T05:52:07.867 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter 2023-11-24T05:52:07.867 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset 2023-11-24T05:52:07.867 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set.ko 2023-11-24T05:52:07.867 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2023-11-24T05:52:07.867 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2023-11-24T05:52:07.867 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2023-11-24T05:52:07.867 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2023-11-24T05:52:07.867 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2023-11-24T05:52:07.867 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2023-11-24T05:52:07.867 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2023-11-24T05:52:07.867 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2023-11-24T05:52:07.867 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2023-11-24T05:52:07.867 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2023-11-24T05:52:07.868 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2023-11-24T05:52:07.868 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2023-11-24T05:52:07.868 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2023-11-24T05:52:07.868 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2023-11-24T05:52:07.868 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_list_set.ko 2023-11-24T05:52:07.868 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs 2023-11-24T05:52:07.868 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs.ko 2023-11-24T05:52:07.868 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2023-11-24T05:52:07.868 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2023-11-24T05:52:07.868 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2023-11-24T05:52:07.868 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2023-11-24T05:52:07.868 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2023-11-24T05:52:07.868 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2023-11-24T05:52:07.869 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2023-11-24T05:52:07.869 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2023-11-24T05:52:07.869 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2023-11-24T05:52:07.869 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2023-11-24T05:52:07.869 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2023-11-24T05:52:07.869 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2023-11-24T05:52:07.869 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2023-11-24T05:52:07.869 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conncount.ko 2023-11-24T05:52:07.869 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack.ko 2023-11-24T05:52:07.869 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_amanda.ko 2023-11-24T05:52:07.869 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_broadcast.ko 2023-11-24T05:52:07.869 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_ftp.ko 2023-11-24T05:52:07.870 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_h323.ko 2023-11-24T05:52:07.870 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_irc.ko 2023-11-24T05:52:07.870 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2023-11-24T05:52:07.870 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_netlink.ko 2023-11-24T05:52:07.870 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_pptp.ko 2023-11-24T05:52:07.870 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_sane.ko 2023-11-24T05:52:07.870 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_sip.ko 2023-11-24T05:52:07.870 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_snmp.ko 2023-11-24T05:52:07.870 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_tftp.ko 2023-11-24T05:52:07.870 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_dup_netdev.ko 2023-11-24T05:52:07.870 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_flow_table.ko 2023-11-24T05:52:07.870 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_flow_table_inet.ko 2023-11-24T05:52:07.870 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_log_syslog.ko 2023-11-24T05:52:07.871 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_nat.ko 2023-11-24T05:52:07.871 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_nat_amanda.ko 2023-11-24T05:52:07.871 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_nat_ftp.ko 2023-11-24T05:52:07.871 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_nat_irc.ko 2023-11-24T05:52:07.871 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_nat_sip.ko 2023-11-24T05:52:07.871 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_nat_tftp.ko 2023-11-24T05:52:07.871 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_synproxy_core.ko 2023-11-24T05:52:07.871 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_tables.ko 2023-11-24T05:52:07.871 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nfnetlink.ko 2023-11-24T05:52:07.872 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nfnetlink_acct.ko 2023-11-24T05:52:07.872 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nfnetlink_log.ko 2023-11-24T05:52:07.872 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nfnetlink_osf.ko 2023-11-24T05:52:07.872 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nfnetlink_queue.ko 2023-11-24T05:52:07.872 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_chain_nat.ko 2023-11-24T05:52:07.872 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_compat.ko 2023-11-24T05:52:07.872 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_ct.ko 2023-11-24T05:52:07.872 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_dup_netdev.ko 2023-11-24T05:52:07.872 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_fib.ko 2023-11-24T05:52:07.872 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_fwd_netdev.ko 2023-11-24T05:52:07.872 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_hash.ko 2023-11-24T05:52:07.872 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_limit.ko 2023-11-24T05:52:07.872 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_log.ko 2023-11-24T05:52:07.873 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_masq.ko 2023-11-24T05:52:07.873 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_nat.ko 2023-11-24T05:52:07.873 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_numgen.ko 2023-11-24T05:52:07.873 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_queue.ko 2023-11-24T05:52:07.873 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_quota.ko 2023-11-24T05:52:07.873 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_redir.ko 2023-11-24T05:52:07.873 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_reject.ko 2023-11-24T05:52:07.873 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_reject_inet.ko 2023-11-24T05:52:07.873 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_socket.ko 2023-11-24T05:52:07.873 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_tunnel.ko 2023-11-24T05:52:07.873 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_AUDIT.ko 2023-11-24T05:52:07.873 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_CHECKSUM.ko 2023-11-24T05:52:07.873 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_CLASSIFY.ko 2023-11-24T05:52:07.873 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_CONNSECMARK.ko 2023-11-24T05:52:07.874 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_CT.ko 2023-11-24T05:52:07.874 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_DSCP.ko 2023-11-24T05:52:07.874 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_HL.ko 2023-11-24T05:52:07.874 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_HMARK.ko 2023-11-24T05:52:07.874 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_IDLETIMER.ko 2023-11-24T05:52:07.874 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_LED.ko 2023-11-24T05:52:07.874 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_LOG.ko 2023-11-24T05:52:07.874 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_MASQUERADE.ko 2023-11-24T05:52:07.874 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_NETMAP.ko 2023-11-24T05:52:07.874 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_NFLOG.ko 2023-11-24T05:52:07.874 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_NFQUEUE.ko 2023-11-24T05:52:07.874 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_RATEEST.ko 2023-11-24T05:52:07.875 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_REDIRECT.ko 2023-11-24T05:52:07.875 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_SECMARK.ko 2023-11-24T05:52:07.875 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_TCPMSS.ko 2023-11-24T05:52:07.875 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2023-11-24T05:52:07.875 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_TEE.ko 2023-11-24T05:52:07.875 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_TPROXY.ko 2023-11-24T05:52:07.875 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_TRACE.ko 2023-11-24T05:52:07.875 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_addrtype.ko 2023-11-24T05:52:07.875 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_bpf.ko 2023-11-24T05:52:07.875 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_cgroup.ko 2023-11-24T05:52:07.875 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_cluster.ko 2023-11-24T05:52:07.875 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_comment.ko 2023-11-24T05:52:07.875 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_connbytes.ko 2023-11-24T05:52:07.876 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_connlabel.ko 2023-11-24T05:52:07.876 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_connlimit.ko 2023-11-24T05:52:07.876 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_connmark.ko 2023-11-24T05:52:07.876 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_conntrack.ko 2023-11-24T05:52:07.876 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_cpu.ko 2023-11-24T05:52:07.876 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_dccp.ko 2023-11-24T05:52:07.876 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_devgroup.ko 2023-11-24T05:52:07.876 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_dscp.ko 2023-11-24T05:52:07.876 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_ecn.ko 2023-11-24T05:52:07.877 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_esp.ko 2023-11-24T05:52:07.877 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_hashlimit.ko 2023-11-24T05:52:07.877 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_helper.ko 2023-11-24T05:52:07.877 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_hl.ko 2023-11-24T05:52:07.877 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_ipcomp.ko 2023-11-24T05:52:07.877 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_iprange.ko 2023-11-24T05:52:07.877 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_ipvs.ko 2023-11-24T05:52:07.877 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_l2tp.ko 2023-11-24T05:52:07.877 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_length.ko 2023-11-24T05:52:07.877 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_limit.ko 2023-11-24T05:52:07.877 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_mac.ko 2023-11-24T05:52:07.878 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_mark.ko 2023-11-24T05:52:07.878 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_multiport.ko 2023-11-24T05:52:07.878 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_nat.ko 2023-11-24T05:52:07.878 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_nfacct.ko 2023-11-24T05:52:07.878 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_osf.ko 2023-11-24T05:52:07.878 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_owner.ko 2023-11-24T05:52:07.878 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_physdev.ko 2023-11-24T05:52:07.878 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_pkttype.ko 2023-11-24T05:52:07.878 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_policy.ko 2023-11-24T05:52:07.878 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_quota.ko 2023-11-24T05:52:07.878 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_rateest.ko 2023-11-24T05:52:07.878 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_realm.ko 2023-11-24T05:52:07.878 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_recent.ko 2023-11-24T05:52:07.879 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_sctp.ko 2023-11-24T05:52:07.879 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_set.ko 2023-11-24T05:52:07.879 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_socket.ko 2023-11-24T05:52:07.879 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_state.ko 2023-11-24T05:52:07.879 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_statistic.ko 2023-11-24T05:52:07.879 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_string.ko 2023-11-24T05:52:07.879 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_tcpmss.ko 2023-11-24T05:52:07.879 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_time.ko 2023-11-24T05:52:07.879 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_u32.ko 2023-11-24T05:52:07.880 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netlink 2023-11-24T05:52:07.880 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netlink/netlink_diag.ko 2023-11-24T05:52:07.880 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/nsh 2023-11-24T05:52:07.880 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/nsh/nsh.ko 2023-11-24T05:52:07.880 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/openvswitch 2023-11-24T05:52:07.880 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/openvswitch/openvswitch.ko 2023-11-24T05:52:07.880 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/openvswitch/vport-geneve.ko 2023-11-24T05:52:07.880 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/openvswitch/vport-gre.ko 2023-11-24T05:52:07.880 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/openvswitch/vport-vxlan.ko 2023-11-24T05:52:07.880 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/packet 2023-11-24T05:52:07.880 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/packet/af_packet_diag.ko 2023-11-24T05:52:07.880 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/rds 2023-11-24T05:52:07.880 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/rds/rds.ko 2023-11-24T05:52:07.881 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/rds/rds_rdma.ko 2023-11-24T05:52:07.881 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/rds/rds_tcp.ko 2023-11-24T05:52:07.881 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched 2023-11-24T05:52:07.881 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_csum.ko 2023-11-24T05:52:07.881 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_gact.ko 2023-11-24T05:52:07.881 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_ipt.ko 2023-11-24T05:52:07.881 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_mirred.ko 2023-11-24T05:52:07.881 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_nat.ko 2023-11-24T05:52:07.881 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_pedit.ko 2023-11-24T05:52:07.881 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_police.ko 2023-11-24T05:52:07.881 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_simple.ko 2023-11-24T05:52:07.881 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_skbedit.ko 2023-11-24T05:52:07.881 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_vlan.ko 2023-11-24T05:52:07.882 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/cls_basic.ko 2023-11-24T05:52:07.882 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/cls_bpf.ko 2023-11-24T05:52:07.882 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/cls_flow.ko 2023-11-24T05:52:07.882 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/cls_fw.ko 2023-11-24T05:52:07.882 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/cls_route.ko 2023-11-24T05:52:07.882 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/cls_u32.ko 2023-11-24T05:52:07.882 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/em_cmp.ko 2023-11-24T05:52:07.882 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/em_ipset.ko 2023-11-24T05:52:07.882 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/em_meta.ko 2023-11-24T05:52:07.882 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/em_nbyte.ko 2023-11-24T05:52:07.882 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/em_text.ko 2023-11-24T05:52:07.882 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/em_u32.ko 2023-11-24T05:52:07.882 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_choke.ko 2023-11-24T05:52:07.883 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_codel.ko 2023-11-24T05:52:07.883 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_drr.ko 2023-11-24T05:52:07.883 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_fq.ko 2023-11-24T05:52:07.883 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_gred.ko 2023-11-24T05:52:07.883 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_hfsc.ko 2023-11-24T05:52:07.883 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_hhf.ko 2023-11-24T05:52:07.883 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_htb.ko 2023-11-24T05:52:07.883 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_ingress.ko 2023-11-24T05:52:07.883 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_mqprio.ko 2023-11-24T05:52:07.883 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_mqprio_lib.ko 2023-11-24T05:52:07.883 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_multiq.ko 2023-11-24T05:52:07.883 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_netem.ko 2023-11-24T05:52:07.883 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_pie.ko 2023-11-24T05:52:07.884 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_plug.ko 2023-11-24T05:52:07.884 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_prio.ko 2023-11-24T05:52:07.884 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_qfq.ko 2023-11-24T05:52:07.884 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_red.ko 2023-11-24T05:52:07.884 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_sfb.ko 2023-11-24T05:52:07.884 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_sfq.ko 2023-11-24T05:52:07.884 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_tbf.ko 2023-11-24T05:52:07.884 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_teql.ko 2023-11-24T05:52:07.884 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sctp 2023-11-24T05:52:07.884 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sctp/sctp.ko 2023-11-24T05:52:07.884 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sctp/sctp_diag.ko 2023-11-24T05:52:07.884 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sunrpc 2023-11-24T05:52:07.884 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sunrpc/auth_gss 2023-11-24T05:52:07.885 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2023-11-24T05:52:07.885 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2023-11-24T05:52:07.885 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sunrpc/sunrpc.ko 2023-11-24T05:52:07.885 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sunrpc/xprtrdma 2023-11-24T05:52:07.885 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2023-11-24T05:52:07.885 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/tipc 2023-11-24T05:52:07.885 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/tipc/diag.ko 2023-11-24T05:52:07.885 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/tipc/tipc.ko 2023-11-24T05:52:07.885 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/unix 2023-11-24T05:52:07.885 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/unix/unix_diag.ko 2023-11-24T05:52:07.885 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/vmw_vsock 2023-11-24T05:52:07.885 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2023-11-24T05:52:07.886 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2023-11-24T05:52:07.886 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/vmw_vsock/vsock.ko 2023-11-24T05:52:07.886 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/vmw_vsock/vsock_diag.ko 2023-11-24T05:52:07.886 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/vmw_vsock/vsock_loopback.ko 2023-11-24T05:52:07.886 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/xfrm 2023-11-24T05:52:07.886 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/xfrm/xfrm_ipcomp.ko 2023-11-24T05:52:07.886 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/security 2023-11-24T05:52:07.886 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/security/keys 2023-11-24T05:52:07.886 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/security/keys/encrypted-keys 2023-11-24T05:52:07.886 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2023-11-24T05:52:07.886 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/security/keys/trusted-keys 2023-11-24T05:52:07.886 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/security/keys/trusted-keys/trusted.ko 2023-11-24T05:52:07.886 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/virt 2023-11-24T05:52:07.886 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/virt/lib 2023-11-24T05:52:07.887 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/virt/lib/irqbypass.ko 2023-11-24T05:52:07.887 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.alias 2023-11-24T05:52:07.887 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.alias.bin 2023-11-24T05:52:07.887 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.builtin 2023-11-24T05:52:07.887 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.builtin.alias.bin 2023-11-24T05:52:07.887 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.builtin.bin 2023-11-24T05:52:07.887 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.builtin.modinfo 2023-11-24T05:52:07.888 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.dep 2023-11-24T05:52:07.888 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.dep.bin 2023-11-24T05:52:07.888 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.devname 2023-11-24T05:52:07.888 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.order 2023-11-24T05:52:07.888 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.softdep 2023-11-24T05:52:07.888 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.symbols 2023-11-24T05:52:07.888 INFO:teuthology.orchestra.run.smithi118.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.symbols.bin 2023-11-24T05:52:07.889 DEBUG:teuthology.task.kernel:get_image_version: 6.6.0-g8247bbb5b470 2023-11-24T05:52:07.889 DEBUG:teuthology.orchestra.run.smithi118:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2023-11-24T05:52:07.912 INFO:teuthology.orchestra.run.smithi118.stdout:postinstall scriptlet (using /bin/sh): 2023-11-24T05:52:07.913 INFO:teuthology.orchestra.run.smithi118.stdout:if [ -x /sbin/installkernel -a -r /boot/vmlinuz-6.6.0-g8247bbb5b470 -a -r /boot/System.map-6.6.0-g8247bbb5b470 ]; then 2023-11-24T05:52:07.913 INFO:teuthology.orchestra.run.smithi118.stdout:cp /boot/vmlinuz-6.6.0-g8247bbb5b470 /boot/.vmlinuz-6.6.0-g8247bbb5b470-rpm 2023-11-24T05:52:07.913 INFO:teuthology.orchestra.run.smithi118.stdout:cp /boot/System.map-6.6.0-g8247bbb5b470 /boot/.System.map-6.6.0-g8247bbb5b470-rpm 2023-11-24T05:52:07.913 INFO:teuthology.orchestra.run.smithi118.stdout:rm -f /boot/vmlinuz-6.6.0-g8247bbb5b470 /boot/System.map-6.6.0-g8247bbb5b470 2023-11-24T05:52:07.913 INFO:teuthology.orchestra.run.smithi118.stdout:/sbin/installkernel 6.6.0-g8247bbb5b470 /boot/.vmlinuz-6.6.0-g8247bbb5b470-rpm /boot/.System.map-6.6.0-g8247bbb5b470-rpm 2023-11-24T05:52:07.913 INFO:teuthology.orchestra.run.smithi118.stdout:rm -f /boot/.vmlinuz-6.6.0-g8247bbb5b470-rpm /boot/.System.map-6.6.0-g8247bbb5b470-rpm 2023-11-24T05:52:07.913 INFO:teuthology.orchestra.run.smithi118.stdout:fi 2023-11-24T05:52:07.913 INFO:teuthology.orchestra.run.smithi118.stdout:preuninstall scriptlet (using /bin/sh): 2023-11-24T05:52:07.914 INFO:teuthology.orchestra.run.smithi118.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2023-11-24T05:52:07.914 INFO:teuthology.orchestra.run.smithi118.stdout:new-kernel-pkg --remove 6.6.0-g8247bbb5b470 --rminitrd --initrdfile=/boot/initramfs-6.6.0-g8247bbb5b470.img 2023-11-24T05:52:07.914 INFO:teuthology.orchestra.run.smithi118.stdout:elif [ -x /usr/bin/kernel-install ]; then 2023-11-24T05:52:07.914 INFO:teuthology.orchestra.run.smithi118.stdout:kernel-install remove 6.6.0-g8247bbb5b470 2023-11-24T05:52:07.914 INFO:teuthology.orchestra.run.smithi118.stdout:fi 2023-11-24T05:52:07.914 INFO:teuthology.orchestra.run.smithi118.stdout:postuninstall scriptlet (using /bin/sh): 2023-11-24T05:52:07.914 INFO:teuthology.orchestra.run.smithi118.stdout:if [ -x /sbin/update-bootloader ]; then 2023-11-24T05:52:07.915 INFO:teuthology.orchestra.run.smithi118.stdout:/sbin/update-bootloader --remove 6.6.0-g8247bbb5b470 2023-11-24T05:52:07.915 INFO:teuthology.orchestra.run.smithi118.stdout:fi 2023-11-24T05:52:07.915 DEBUG:teuthology.orchestra.run.smithi118:> sudo rpm -qi grub2-tools 2023-11-24T05:52:07.987 INFO:teuthology.orchestra.run.smithi118.stdout:Name : grub2-tools 2023-11-24T05:52:07.988 INFO:teuthology.orchestra.run.smithi118.stdout:Epoch : 1 2023-11-24T05:52:07.988 INFO:teuthology.orchestra.run.smithi118.stdout:Version : 2.06 2023-11-24T05:52:07.988 INFO:teuthology.orchestra.run.smithi118.stdout:Release : 68.el9 2023-11-24T05:52:07.988 INFO:teuthology.orchestra.run.smithi118.stdout:Architecture: x86_64 2023-11-24T05:52:07.988 INFO:teuthology.orchestra.run.smithi118.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2023-11-24T05:52:07.988 INFO:teuthology.orchestra.run.smithi118.stdout:Group : Unspecified 2023-11-24T05:52:07.988 INFO:teuthology.orchestra.run.smithi118.stdout:Size : 8264254 2023-11-24T05:52:07.988 INFO:teuthology.orchestra.run.smithi118.stdout:License : GPLv3+ 2023-11-24T05:52:07.988 INFO:teuthology.orchestra.run.smithi118.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2023-11-24T05:52:07.988 INFO:teuthology.orchestra.run.smithi118.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2023-11-24T05:52:07.988 INFO:teuthology.orchestra.run.smithi118.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2023-11-24T05:52:07.989 INFO:teuthology.orchestra.run.smithi118.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2023-11-24T05:52:07.989 INFO:teuthology.orchestra.run.smithi118.stdout:Packager : builder@centos.org 2023-11-24T05:52:07.989 INFO:teuthology.orchestra.run.smithi118.stdout:Vendor : CentOS 2023-11-24T05:52:07.989 INFO:teuthology.orchestra.run.smithi118.stdout:URL : http://www.gnu.org/software/grub/ 2023-11-24T05:52:07.989 INFO:teuthology.orchestra.run.smithi118.stdout:Summary : Support tools for GRUB. 2023-11-24T05:52:07.989 INFO:teuthology.orchestra.run.smithi118.stdout:Description : 2023-11-24T05:52:07.989 INFO:teuthology.orchestra.run.smithi118.stdout: 2023-11-24T05:52:07.989 INFO:teuthology.orchestra.run.smithi118.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-11-24T05:52:07.989 INFO:teuthology.orchestra.run.smithi118.stdout:customizable bootloader with modular architecture. It supports a rich 2023-11-24T05:52:07.989 INFO:teuthology.orchestra.run.smithi118.stdout:variety of kernel formats, file systems, computer architectures and 2023-11-24T05:52:07.989 INFO:teuthology.orchestra.run.smithi118.stdout:hardware devices. 2023-11-24T05:52:07.989 INFO:teuthology.orchestra.run.smithi118.stdout: 2023-11-24T05:52:07.989 INFO:teuthology.orchestra.run.smithi118.stdout:This subpackage provides tools for support of all platforms. 2023-11-24T05:52:07.998 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-11-24T05:52:07.998 INFO:teuthology.task.kernel:Updating grub on smithi118 to boot 6.6.0-g8247bbb5b470 2023-11-24T05:52:07.998 DEBUG:teuthology.orchestra.run.smithi118:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-11-24T05:52:08.556 INFO:teuthology.orchestra.run.smithi118.stderr:Generating grub configuration file ... 2023-11-24T05:52:10.018 INFO:teuthology.orchestra.run.smithi118.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2023-11-24T05:52:10.050 INFO:teuthology.orchestra.run.smithi118.stderr:done 2023-11-24T05:52:10.053 DEBUG:teuthology.orchestra.run.smithi118:> mktemp 2023-11-24T05:52:10.068 INFO:teuthology.orchestra.run.smithi118.stdout:/tmp/tmp.WFMrSdn3RA 2023-11-24T05:52:10.068 DEBUG:teuthology.orchestra.run.smithi118:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.WFMrSdn3RA 2023-11-24T05:52:10.142 DEBUG:teuthology.orchestra.run.smithi118:> sudo chmod 0666 /tmp/tmp.WFMrSdn3RA 2023-11-24T05:52:10.292 DEBUG:teuthology.orchestra.remote:smithi118:/tmp/tmp.WFMrSdn3RA is 6KB 2023-11-24T05:52:10.344 DEBUG:teuthology.orchestra.run.smithi118:> rm -fr /tmp/tmp.WFMrSdn3RA 2023-11-24T05:52:10.360 DEBUG:teuthology.orchestra.run.smithi118:> sudo /bin/ls /boot/loader/entries || true 2023-11-24T05:52:10.424 INFO:teuthology.orchestra.run.smithi118.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2023-11-24T05:52:10.424 INFO:teuthology.orchestra.run.smithi118.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2023-11-24T05:52:10.425 INFO:teuthology.orchestra.run.smithi118.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2023-11-24T05:52:10.425 INFO:teuthology.orchestra.run.smithi118.stdout:15949e3c88704300882482eddfd7baae-6.6.0-g8247bbb5b470.conf 2023-11-24T05:52:10.426 DEBUG:teuthology.orchestra.run.smithi118:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-6.6.0-g8247bbb5b470 2023-11-24T05:52:10.575 DEBUG:teuthology.orchestra.run.smithi118:> sudo shutdown -r now 2023-11-24T05:52:15.082 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi092.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2023-11-24T05:52:15.083 DEBUG:teuthology.orchestra.run.smithi092:> rpm -qlp /tmp/kernel.x86_64.rpm 2023-11-24T05:52:15.214 INFO:teuthology.orchestra.run.smithi092.stdout:/boot/System.map-6.6.0-g8247bbb5b470 2023-11-24T05:52:15.214 INFO:teuthology.orchestra.run.smithi092.stdout:/boot/config-6.6.0-g8247bbb5b470 2023-11-24T05:52:15.214 INFO:teuthology.orchestra.run.smithi092.stdout:/boot/vmlinuz-6.6.0-g8247bbb5b470 2023-11-24T05:52:15.214 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470 2023-11-24T05:52:15.215 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel 2023-11-24T05:52:15.215 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch 2023-11-24T05:52:15.215 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86 2023-11-24T05:52:15.215 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto 2023-11-24T05:52:15.215 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/blowfish-x86_64.ko 2023-11-24T05:52:15.215 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2023-11-24T05:52:15.216 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2023-11-24T05:52:15.216 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/camellia-x86_64.ko 2023-11-24T05:52:15.216 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2023-11-24T05:52:15.216 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2023-11-24T05:52:15.216 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/crc32-pclmul.ko 2023-11-24T05:52:15.216 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/crc32c-intel.ko 2023-11-24T05:52:15.216 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2023-11-24T05:52:15.216 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2023-11-24T05:52:15.216 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2023-11-24T05:52:15.216 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2023-11-24T05:52:15.216 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/serpent-avx2.ko 2023-11-24T05:52:15.217 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2023-11-24T05:52:15.217 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/sha1-ssse3.ko 2023-11-24T05:52:15.217 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/sha256-ssse3.ko 2023-11-24T05:52:15.217 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/sha512-ssse3.ko 2023-11-24T05:52:15.217 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2023-11-24T05:52:15.217 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2023-11-24T05:52:15.217 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/twofish-x86_64.ko 2023-11-24T05:52:15.217 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kernel 2023-11-24T05:52:15.217 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kernel/cpu 2023-11-24T05:52:15.217 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kernel/cpu/mce 2023-11-24T05:52:15.218 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2023-11-24T05:52:15.218 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kvm 2023-11-24T05:52:15.218 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kvm/kvm-amd.ko 2023-11-24T05:52:15.218 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kvm/kvm-intel.ko 2023-11-24T05:52:15.218 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kvm/kvm.ko 2023-11-24T05:52:15.218 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto 2023-11-24T05:52:15.218 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/ansi_cprng.ko 2023-11-24T05:52:15.218 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/anubis.ko 2023-11-24T05:52:15.218 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/arc4.ko 2023-11-24T05:52:15.218 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/async_tx 2023-11-24T05:52:15.218 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/async_tx/async_memcpy.ko 2023-11-24T05:52:15.219 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/async_tx/async_pq.ko 2023-11-24T05:52:15.219 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/async_tx/async_raid6_recov.ko 2023-11-24T05:52:15.219 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/async_tx/async_tx.ko 2023-11-24T05:52:15.219 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/async_tx/async_xor.ko 2023-11-24T05:52:15.219 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/async_tx/raid6test.ko 2023-11-24T05:52:15.219 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/authenc.ko 2023-11-24T05:52:15.219 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/authencesn.ko 2023-11-24T05:52:15.219 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/blake2b_generic.ko 2023-11-24T05:52:15.219 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/blowfish_common.ko 2023-11-24T05:52:15.219 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/blowfish_generic.ko 2023-11-24T05:52:15.220 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/camellia_generic.ko 2023-11-24T05:52:15.220 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/cast5_generic.ko 2023-11-24T05:52:15.220 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/cast6_generic.ko 2023-11-24T05:52:15.220 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/cast_common.ko 2023-11-24T05:52:15.220 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/ccm.ko 2023-11-24T05:52:15.220 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/cmac.ko 2023-11-24T05:52:15.220 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/crc32_generic.ko 2023-11-24T05:52:15.220 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/crypto_engine.ko 2023-11-24T05:52:15.220 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/crypto_user.ko 2023-11-24T05:52:15.220 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/cts.ko 2023-11-24T05:52:15.220 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/des_generic.ko 2023-11-24T05:52:15.221 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/dh_generic.ko 2023-11-24T05:52:15.221 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/echainiv.ko 2023-11-24T05:52:15.221 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/essiv.ko 2023-11-24T05:52:15.221 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/fcrypt.ko 2023-11-24T05:52:15.221 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/khazad.ko 2023-11-24T05:52:15.221 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/lz4.ko 2023-11-24T05:52:15.221 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/lz4hc.ko 2023-11-24T05:52:15.221 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/md4.ko 2023-11-24T05:52:15.221 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/michael_mic.ko 2023-11-24T05:52:15.221 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/pcbc.ko 2023-11-24T05:52:15.222 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/pcrypt.ko 2023-11-24T05:52:15.222 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/rmd160.ko 2023-11-24T05:52:15.222 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/seed.ko 2023-11-24T05:52:15.222 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/serpent_generic.ko 2023-11-24T05:52:15.222 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/tcrypt.ko 2023-11-24T05:52:15.222 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/tea.ko 2023-11-24T05:52:15.222 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/twofish_common.ko 2023-11-24T05:52:15.222 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/twofish_generic.ko 2023-11-24T05:52:15.222 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/vmac.ko 2023-11-24T05:52:15.222 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/wp512.ko 2023-11-24T05:52:15.222 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/xcbc.ko 2023-11-24T05:52:15.223 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/xor.ko 2023-11-24T05:52:15.223 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/xxhash_generic.ko 2023-11-24T05:52:15.223 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers 2023-11-24T05:52:15.223 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi 2023-11-24T05:52:15.223 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/acpi_ipmi.ko 2023-11-24T05:52:15.223 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/acpi_pad.ko 2023-11-24T05:52:15.223 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/custom_method.ko 2023-11-24T05:52:15.223 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/ec_sys.ko 2023-11-24T05:52:15.223 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/platform_profile.ko 2023-11-24T05:52:15.223 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/sbs.ko 2023-11-24T05:52:15.223 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/sbshc.ko 2023-11-24T05:52:15.224 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/video.ko 2023-11-24T05:52:15.224 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata 2023-11-24T05:52:15.224 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/acard-ahci.ko 2023-11-24T05:52:15.224 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/ahci_platform.ko 2023-11-24T05:52:15.224 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/ata_generic.ko 2023-11-24T05:52:15.224 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/libahci_platform.ko 2023-11-24T05:52:15.224 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_acpi.ko 2023-11-24T05:52:15.225 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_ali.ko 2023-11-24T05:52:15.225 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_amd.ko 2023-11-24T05:52:15.225 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_artop.ko 2023-11-24T05:52:15.225 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_atiixp.ko 2023-11-24T05:52:15.225 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_atp867x.ko 2023-11-24T05:52:15.225 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_cmd640.ko 2023-11-24T05:52:15.225 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_cmd64x.ko 2023-11-24T05:52:15.225 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_cypress.ko 2023-11-24T05:52:15.225 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_efar.ko 2023-11-24T05:52:15.225 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_hpt366.ko 2023-11-24T05:52:15.226 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_hpt37x.ko 2023-11-24T05:52:15.226 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_hpt3x2n.ko 2023-11-24T05:52:15.226 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_hpt3x3.ko 2023-11-24T05:52:15.226 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_it8213.ko 2023-11-24T05:52:15.226 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_it821x.ko 2023-11-24T05:52:15.226 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_jmicron.ko 2023-11-24T05:52:15.226 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_marvell.ko 2023-11-24T05:52:15.226 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_mpiix.ko 2023-11-24T05:52:15.226 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_netcell.ko 2023-11-24T05:52:15.226 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_ninja32.ko 2023-11-24T05:52:15.227 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_ns87410.ko 2023-11-24T05:52:15.227 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_ns87415.ko 2023-11-24T05:52:15.227 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_oldpiix.ko 2023-11-24T05:52:15.227 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_opti.ko 2023-11-24T05:52:15.227 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_optidma.ko 2023-11-24T05:52:15.227 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_pcmcia.ko 2023-11-24T05:52:15.227 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_pdc2027x.ko 2023-11-24T05:52:15.227 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_pdc202xx_old.ko 2023-11-24T05:52:15.227 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_piccolo.ko 2023-11-24T05:52:15.227 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_rdc.ko 2023-11-24T05:52:15.227 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_sch.ko 2023-11-24T05:52:15.228 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_serverworks.ko 2023-11-24T05:52:15.228 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_sil680.ko 2023-11-24T05:52:15.228 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_sis.ko 2023-11-24T05:52:15.228 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_sl82c105.ko 2023-11-24T05:52:15.228 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_triflex.ko 2023-11-24T05:52:15.228 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_via.ko 2023-11-24T05:52:15.228 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pdc_adma.ko 2023-11-24T05:52:15.228 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_inic162x.ko 2023-11-24T05:52:15.228 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_mv.ko 2023-11-24T05:52:15.228 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_nv.ko 2023-11-24T05:52:15.228 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_promise.ko 2023-11-24T05:52:15.229 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_qstor.ko 2023-11-24T05:52:15.229 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_sil.ko 2023-11-24T05:52:15.229 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_sil24.ko 2023-11-24T05:52:15.229 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_sis.ko 2023-11-24T05:52:15.229 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_svw.ko 2023-11-24T05:52:15.229 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_sx4.ko 2023-11-24T05:52:15.229 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_uli.ko 2023-11-24T05:52:15.229 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_via.ko 2023-11-24T05:52:15.229 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_vsc.ko 2023-11-24T05:52:15.229 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/auxdisplay 2023-11-24T05:52:15.229 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/auxdisplay/ks0108.ko 2023-11-24T05:52:15.230 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/base 2023-11-24T05:52:15.230 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/base/regmap 2023-11-24T05:52:15.230 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/base/regmap/regmap-i2c.ko 2023-11-24T05:52:15.230 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/base/regmap/regmap-mmio.ko 2023-11-24T05:52:15.230 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/bcma 2023-11-24T05:52:15.230 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/bcma/bcma.ko 2023-11-24T05:52:15.230 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block 2023-11-24T05:52:15.230 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/aoe 2023-11-24T05:52:15.230 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/aoe/aoe.ko 2023-11-24T05:52:15.230 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/brd.ko 2023-11-24T05:52:15.231 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/drbd 2023-11-24T05:52:15.231 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/drbd/drbd.ko 2023-11-24T05:52:15.231 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/floppy.ko 2023-11-24T05:52:15.231 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/loop.ko 2023-11-24T05:52:15.231 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/mtip32xx 2023-11-24T05:52:15.231 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/mtip32xx/mtip32xx.ko 2023-11-24T05:52:15.231 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/nbd.ko 2023-11-24T05:52:15.231 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/null_blk 2023-11-24T05:52:15.231 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/null_blk/null_blk.ko 2023-11-24T05:52:15.231 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/pktcdvd.ko 2023-11-24T05:52:15.231 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/rbd.ko 2023-11-24T05:52:15.232 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/virtio_blk.ko 2023-11-24T05:52:15.232 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/xen-blkback 2023-11-24T05:52:15.232 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/xen-blkback/xen-blkback.ko 2023-11-24T05:52:15.232 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/xen-blkfront.ko 2023-11-24T05:52:15.232 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/zram 2023-11-24T05:52:15.232 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/zram/zram.ko 2023-11-24T05:52:15.232 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char 2023-11-24T05:52:15.232 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/hangcheck-timer.ko 2023-11-24T05:52:15.232 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/hw_random 2023-11-24T05:52:15.232 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/hw_random/amd-rng.ko 2023-11-24T05:52:15.232 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/hw_random/intel-rng.ko 2023-11-24T05:52:15.233 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/hw_random/timeriomem-rng.ko 2023-11-24T05:52:15.233 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/hw_random/via-rng.ko 2023-11-24T05:52:15.233 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/hw_random/virtio-rng.ko 2023-11-24T05:52:15.233 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ipmi 2023-11-24T05:52:15.233 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ipmi/ipmi_devintf.ko 2023-11-24T05:52:15.233 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2023-11-24T05:52:15.233 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2023-11-24T05:52:15.233 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ipmi/ipmi_si.ko 2023-11-24T05:52:15.233 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ipmi/ipmi_ssif.ko 2023-11-24T05:52:15.233 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2023-11-24T05:52:15.233 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/lp.ko 2023-11-24T05:52:15.233 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/mwave 2023-11-24T05:52:15.234 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/mwave/mwave.ko 2023-11-24T05:52:15.234 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ppdev.ko 2023-11-24T05:52:15.234 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tlclk.ko 2023-11-24T05:52:15.234 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tpm 2023-11-24T05:52:15.234 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tpm/tpm.ko 2023-11-24T05:52:15.234 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tpm/tpm_atmel.ko 2023-11-24T05:52:15.234 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tpm/tpm_infineon.ko 2023-11-24T05:52:15.234 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tpm/tpm_nsc.ko 2023-11-24T05:52:15.234 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tpm/tpm_tis.ko 2023-11-24T05:52:15.234 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tpm/tpm_tis_core.ko 2023-11-24T05:52:15.235 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/uv_mmtimer.ko 2023-11-24T05:52:15.235 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/virtio_console.ko 2023-11-24T05:52:15.235 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/cpufreq 2023-11-24T05:52:15.235 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/cpufreq/acpi-cpufreq.ko 2023-11-24T05:52:15.235 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2023-11-24T05:52:15.235 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/cpufreq/p4-clockmod.ko 2023-11-24T05:52:15.235 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/cpufreq/pcc-cpufreq.ko 2023-11-24T05:52:15.235 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/cpufreq/powernow-k8.ko 2023-11-24T05:52:15.235 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/cpufreq/speedstep-lib.ko 2023-11-24T05:52:15.235 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto 2023-11-24T05:52:15.236 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/ccp 2023-11-24T05:52:15.236 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/ccp/ccp-crypto.ko 2023-11-24T05:52:15.236 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/ccp/ccp.ko 2023-11-24T05:52:15.236 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/intel 2023-11-24T05:52:15.236 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/intel/qat 2023-11-24T05:52:15.236 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/intel/qat/qat_common 2023-11-24T05:52:15.236 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2023-11-24T05:52:15.236 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2023-11-24T05:52:15.236 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2023-11-24T05:52:15.236 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/padlock-aes.ko 2023-11-24T05:52:15.237 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/padlock-sha.ko 2023-11-24T05:52:15.237 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/virtio 2023-11-24T05:52:15.237 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/virtio/virtio_crypto.ko 2023-11-24T05:52:15.237 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/dca 2023-11-24T05:52:15.237 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/dca/dca.ko 2023-11-24T05:52:15.237 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/devfreq 2023-11-24T05:52:15.237 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/devfreq/governor_simpleondemand.ko 2023-11-24T05:52:15.237 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/dma 2023-11-24T05:52:15.237 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/dma/dw 2023-11-24T05:52:15.238 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/dma/dw/dw_dmac.ko 2023-11-24T05:52:15.238 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/dma/ioat 2023-11-24T05:52:15.239 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/dma/ioat/ioatdma.ko 2023-11-24T05:52:15.239 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac 2023-11-24T05:52:15.239 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/amd64_edac.ko 2023-11-24T05:52:15.239 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/e752x_edac.ko 2023-11-24T05:52:15.239 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/edac_mce_amd.ko 2023-11-24T05:52:15.239 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/i3000_edac.ko 2023-11-24T05:52:15.239 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/i3200_edac.ko 2023-11-24T05:52:15.239 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/i5100_edac.ko 2023-11-24T05:52:15.239 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/i5400_edac.ko 2023-11-24T05:52:15.239 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/i7300_edac.ko 2023-11-24T05:52:15.239 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/i7core_edac.ko 2023-11-24T05:52:15.240 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/i82975x_edac.ko 2023-11-24T05:52:15.240 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/ie31200_edac.ko 2023-11-24T05:52:15.240 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/sb_edac.ko 2023-11-24T05:52:15.240 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/x38_edac.ko 2023-11-24T05:52:15.240 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/firmware 2023-11-24T05:52:15.240 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/firmware/edd.ko 2023-11-24T05:52:15.240 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/firmware/iscsi_ibft.ko 2023-11-24T05:52:15.240 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio 2023-11-24T05:52:15.240 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-ich.ko 2023-11-24T05:52:15.240 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-viperboard.ko 2023-11-24T05:52:15.241 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid 2023-11-24T05:52:15.241 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-appleir.ko 2023-11-24T05:52:15.241 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-aureal.ko 2023-11-24T05:52:15.241 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-axff.ko 2023-11-24T05:52:15.241 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-dr.ko 2023-11-24T05:52:15.241 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-elecom.ko 2023-11-24T05:52:15.241 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-elo.ko 2023-11-24T05:52:15.241 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-emsff.ko 2023-11-24T05:52:15.241 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-gaff.ko 2023-11-24T05:52:15.241 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-gt683r.ko 2023-11-24T05:52:15.242 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-gyration.ko 2023-11-24T05:52:15.242 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-holtek-kbd.ko 2023-11-24T05:52:15.242 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-holtek-mouse.ko 2023-11-24T05:52:15.242 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-holtekff.ko 2023-11-24T05:52:15.242 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-hyperv.ko 2023-11-24T05:52:15.242 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-icade.ko 2023-11-24T05:52:15.242 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-keytouch.ko 2023-11-24T05:52:15.242 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-kye.ko 2023-11-24T05:52:15.242 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-lcpower.ko 2023-11-24T05:52:15.243 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-led.ko 2023-11-24T05:52:15.243 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-lenovo.ko 2023-11-24T05:52:15.243 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-logitech-dj.ko 2023-11-24T05:52:15.243 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-logitech-hidpp.ko 2023-11-24T05:52:15.243 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-multitouch.ko 2023-11-24T05:52:15.243 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-ortek.ko 2023-11-24T05:52:15.243 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-penmount.ko 2023-11-24T05:52:15.243 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-petalynx.ko 2023-11-24T05:52:15.243 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-picolcd.ko 2023-11-24T05:52:15.244 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-pl.ko 2023-11-24T05:52:15.244 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-plantronics.ko 2023-11-24T05:52:15.244 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-primax.ko 2023-11-24T05:52:15.244 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-rmi.ko 2023-11-24T05:52:15.244 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-arvo.ko 2023-11-24T05:52:15.244 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-common.ko 2023-11-24T05:52:15.244 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-isku.ko 2023-11-24T05:52:15.244 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-kone.ko 2023-11-24T05:52:15.244 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-koneplus.ko 2023-11-24T05:52:15.244 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-konepure.ko 2023-11-24T05:52:15.244 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-kovaplus.ko 2023-11-24T05:52:15.245 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-lua.ko 2023-11-24T05:52:15.245 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-pyra.ko 2023-11-24T05:52:15.245 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-ryos.ko 2023-11-24T05:52:15.245 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-savu.ko 2023-11-24T05:52:15.245 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat.ko 2023-11-24T05:52:15.245 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-saitek.ko 2023-11-24T05:52:15.245 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-samsung.ko 2023-11-24T05:52:15.245 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-sensor-hub.ko 2023-11-24T05:52:15.245 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-sjoy.ko 2023-11-24T05:52:15.245 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-sony.ko 2023-11-24T05:52:15.245 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-speedlink.ko 2023-11-24T05:52:15.246 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-steelseries.ko 2023-11-24T05:52:15.246 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-sunplus.ko 2023-11-24T05:52:15.246 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-thrustmaster.ko 2023-11-24T05:52:15.246 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-tivo.ko 2023-11-24T05:52:15.246 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-tmff.ko 2023-11-24T05:52:15.246 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-topseed.ko 2023-11-24T05:52:15.246 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-twinhan.ko 2023-11-24T05:52:15.247 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-uclogic.ko 2023-11-24T05:52:15.247 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-waltop.ko 2023-11-24T05:52:15.247 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-wiimote.ko 2023-11-24T05:52:15.247 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-xinmo.ko 2023-11-24T05:52:15.247 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-zpff.ko 2023-11-24T05:52:15.247 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-zydacron.ko 2023-11-24T05:52:15.247 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/uhid.ko 2023-11-24T05:52:15.247 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/wacom.ko 2023-11-24T05:52:15.247 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hv 2023-11-24T05:52:15.247 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hv/hv_balloon.ko 2023-11-24T05:52:15.247 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hv/hv_utils.ko 2023-11-24T05:52:15.248 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hv/hv_vmbus.ko 2023-11-24T05:52:15.248 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon 2023-11-24T05:52:15.248 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/abituguru.ko 2023-11-24T05:52:15.248 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/abituguru3.ko 2023-11-24T05:52:15.248 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/acpi_power_meter.ko 2023-11-24T05:52:15.248 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ad7414.ko 2023-11-24T05:52:15.248 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ad7418.ko 2023-11-24T05:52:15.248 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adc128d818.ko 2023-11-24T05:52:15.248 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adm1025.ko 2023-11-24T05:52:15.248 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adm1026.ko 2023-11-24T05:52:15.248 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adm1029.ko 2023-11-24T05:52:15.249 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adm1031.ko 2023-11-24T05:52:15.249 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adm9240.ko 2023-11-24T05:52:15.249 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ads7828.ko 2023-11-24T05:52:15.249 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adt7410.ko 2023-11-24T05:52:15.249 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adt7411.ko 2023-11-24T05:52:15.249 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adt7462.ko 2023-11-24T05:52:15.249 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adt7470.ko 2023-11-24T05:52:15.249 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adt7475.ko 2023-11-24T05:52:15.249 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adt7x10.ko 2023-11-24T05:52:15.249 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/amc6821.ko 2023-11-24T05:52:15.250 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/applesmc.ko 2023-11-24T05:52:15.250 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/asb100.ko 2023-11-24T05:52:15.250 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/asc7621.ko 2023-11-24T05:52:15.250 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/asus_atk0110.ko 2023-11-24T05:52:15.250 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/atxp1.ko 2023-11-24T05:52:15.250 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/coretemp.ko 2023-11-24T05:52:15.250 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/dell-smm-hwmon.ko 2023-11-24T05:52:15.250 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/dme1737.ko 2023-11-24T05:52:15.251 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ds1621.ko 2023-11-24T05:52:15.251 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ds620.ko 2023-11-24T05:52:15.251 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/emc1403.ko 2023-11-24T05:52:15.251 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/emc6w201.ko 2023-11-24T05:52:15.251 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/f71805f.ko 2023-11-24T05:52:15.251 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/f71882fg.ko 2023-11-24T05:52:15.251 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/f75375s.ko 2023-11-24T05:52:15.251 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/fam15h_power.ko 2023-11-24T05:52:15.251 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/fschmd.ko 2023-11-24T05:52:15.251 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/g760a.ko 2023-11-24T05:52:15.251 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/g762.ko 2023-11-24T05:52:15.252 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/gl518sm.ko 2023-11-24T05:52:15.252 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/gl520sm.ko 2023-11-24T05:52:15.252 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/hwmon-vid.ko 2023-11-24T05:52:15.252 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/i5500_temp.ko 2023-11-24T05:52:15.252 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/i5k_amb.ko 2023-11-24T05:52:15.252 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ibmaem.ko 2023-11-24T05:52:15.252 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ibmpex.ko 2023-11-24T05:52:15.252 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ina209.ko 2023-11-24T05:52:15.252 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ina2xx.ko 2023-11-24T05:52:15.252 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/it87.ko 2023-11-24T05:52:15.253 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/k10temp.ko 2023-11-24T05:52:15.253 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/k8temp.ko 2023-11-24T05:52:15.253 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lineage-pem.ko 2023-11-24T05:52:15.253 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm63.ko 2023-11-24T05:52:15.253 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm73.ko 2023-11-24T05:52:15.253 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm75.ko 2023-11-24T05:52:15.253 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm77.ko 2023-11-24T05:52:15.253 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm78.ko 2023-11-24T05:52:15.253 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm80.ko 2023-11-24T05:52:15.253 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm83.ko 2023-11-24T05:52:15.253 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm85.ko 2023-11-24T05:52:15.254 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm87.ko 2023-11-24T05:52:15.254 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm90.ko 2023-11-24T05:52:15.254 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm92.ko 2023-11-24T05:52:15.254 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm93.ko 2023-11-24T05:52:15.254 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm95234.ko 2023-11-24T05:52:15.254 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm95241.ko 2023-11-24T05:52:15.254 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm95245.ko 2023-11-24T05:52:15.254 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ltc2945.ko 2023-11-24T05:52:15.254 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ltc4151.ko 2023-11-24T05:52:15.254 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ltc4215.ko 2023-11-24T05:52:15.255 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ltc4222.ko 2023-11-24T05:52:15.255 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ltc4245.ko 2023-11-24T05:52:15.255 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ltc4260.ko 2023-11-24T05:52:15.255 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ltc4261.ko 2023-11-24T05:52:15.255 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/max16065.ko 2023-11-24T05:52:15.255 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/max1619.ko 2023-11-24T05:52:15.255 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/max1668.ko 2023-11-24T05:52:15.255 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/max197.ko 2023-11-24T05:52:15.255 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/max6639.ko 2023-11-24T05:52:15.255 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/max6650.ko 2023-11-24T05:52:15.256 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/max6697.ko 2023-11-24T05:52:15.256 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/mcp3021.ko 2023-11-24T05:52:15.256 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/nct6683.ko 2023-11-24T05:52:15.256 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/nct6775-core.ko 2023-11-24T05:52:15.256 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/nct6775.ko 2023-11-24T05:52:15.256 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/nct7802.ko 2023-11-24T05:52:15.256 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pc87360.ko 2023-11-24T05:52:15.256 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pc87427.ko 2023-11-24T05:52:15.256 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pcf8591.ko 2023-11-24T05:52:15.257 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus 2023-11-24T05:52:15.257 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/adm1275.ko 2023-11-24T05:52:15.257 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/lm25066.ko 2023-11-24T05:52:15.257 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/ltc2978.ko 2023-11-24T05:52:15.257 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/max16064.ko 2023-11-24T05:52:15.257 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/max34440.ko 2023-11-24T05:52:15.257 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/max8688.ko 2023-11-24T05:52:15.257 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/pmbus.ko 2023-11-24T05:52:15.257 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2023-11-24T05:52:15.257 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/tps40422.ko 2023-11-24T05:52:15.257 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/ucd9000.ko 2023-11-24T05:52:15.258 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/ucd9200.ko 2023-11-24T05:52:15.258 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/zl6100.ko 2023-11-24T05:52:15.258 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/powr1220.ko 2023-11-24T05:52:15.258 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/sch5627.ko 2023-11-24T05:52:15.258 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/sch5636.ko 2023-11-24T05:52:15.258 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/sch56xx-common.ko 2023-11-24T05:52:15.258 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/sht15.ko 2023-11-24T05:52:15.258 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/sht21.ko 2023-11-24T05:52:15.258 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/shtc1.ko 2023-11-24T05:52:15.258 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/sis5595.ko 2023-11-24T05:52:15.259 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/smsc47b397.ko 2023-11-24T05:52:15.259 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/smsc47m1.ko 2023-11-24T05:52:15.259 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/smsc47m192.ko 2023-11-24T05:52:15.259 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/thmc50.ko 2023-11-24T05:52:15.259 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/tmp102.ko 2023-11-24T05:52:15.259 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/tmp103.ko 2023-11-24T05:52:15.259 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/tmp401.ko 2023-11-24T05:52:15.259 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/tmp421.ko 2023-11-24T05:52:15.259 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/via-cputemp.ko 2023-11-24T05:52:15.259 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/via686a.ko 2023-11-24T05:52:15.259 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/vt1211.ko 2023-11-24T05:52:15.260 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/vt8231.ko 2023-11-24T05:52:15.260 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83627ehf.ko 2023-11-24T05:52:15.260 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83627hf.ko 2023-11-24T05:52:15.260 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83781d.ko 2023-11-24T05:52:15.260 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83791d.ko 2023-11-24T05:52:15.260 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83792d.ko 2023-11-24T05:52:15.260 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83793.ko 2023-11-24T05:52:15.260 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83795.ko 2023-11-24T05:52:15.260 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83l785ts.ko 2023-11-24T05:52:15.260 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83l786ng.ko 2023-11-24T05:52:15.261 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c 2023-11-24T05:52:15.261 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/algos 2023-11-24T05:52:15.261 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2023-11-24T05:52:15.261 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2023-11-24T05:52:15.261 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses 2023-11-24T05:52:15.261 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2023-11-24T05:52:15.261 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-amd756.ko 2023-11-24T05:52:15.261 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-amd8111.ko 2023-11-24T05:52:15.261 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2023-11-24T05:52:15.261 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-designware-core.ko 2023-11-24T05:52:15.261 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2023-11-24T05:52:15.262 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2023-11-24T05:52:15.262 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2023-11-24T05:52:15.262 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-i801.ko 2023-11-24T05:52:15.262 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-isch.ko 2023-11-24T05:52:15.262 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-ismt.ko 2023-11-24T05:52:15.262 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2023-11-24T05:52:15.262 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-nforce2.ko 2023-11-24T05:52:15.262 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-parport.ko 2023-11-24T05:52:15.262 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2023-11-24T05:52:15.262 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-piix4.ko 2023-11-24T05:52:15.263 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-scmi.ko 2023-11-24T05:52:15.263 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-simtec.ko 2023-11-24T05:52:15.263 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-sis96x.ko 2023-11-24T05:52:15.263 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2023-11-24T05:52:15.263 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-via.ko 2023-11-24T05:52:15.263 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-viapro.ko 2023-11-24T05:52:15.263 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-viperboard.ko 2023-11-24T05:52:15.263 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/i2c-dev.ko 2023-11-24T05:52:15.263 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/i2c-mux.ko 2023-11-24T05:52:15.263 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/i2c-slave-eeprom.ko 2023-11-24T05:52:15.263 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/i2c-smbus.ko 2023-11-24T05:52:15.264 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/i2c-stub.ko 2023-11-24T05:52:15.264 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband 2023-11-24T05:52:15.264 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core 2023-11-24T05:52:15.264 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core/ib_cm.ko 2023-11-24T05:52:15.264 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core/ib_core.ko 2023-11-24T05:52:15.264 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core/ib_umad.ko 2023-11-24T05:52:15.264 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core/ib_uverbs.ko 2023-11-24T05:52:15.265 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core/iw_cm.ko 2023-11-24T05:52:15.265 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core/rdma_cm.ko 2023-11-24T05:52:15.265 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core/rdma_ucm.ko 2023-11-24T05:52:15.265 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw 2023-11-24T05:52:15.265 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/cxgb4 2023-11-24T05:52:15.265 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2023-11-24T05:52:15.265 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/mlx4 2023-11-24T05:52:15.265 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2023-11-24T05:52:15.265 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/mlx5 2023-11-24T05:52:15.265 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2023-11-24T05:52:15.266 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/mthca 2023-11-24T05:52:15.266 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2023-11-24T05:52:15.266 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp 2023-11-24T05:52:15.266 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/ipoib 2023-11-24T05:52:15.266 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2023-11-24T05:52:15.266 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/iser 2023-11-24T05:52:15.266 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2023-11-24T05:52:15.266 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/isert 2023-11-24T05:52:15.266 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2023-11-24T05:52:15.266 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/srp 2023-11-24T05:52:15.267 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2023-11-24T05:52:15.267 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/srpt 2023-11-24T05:52:15.267 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2023-11-24T05:52:15.267 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input 2023-11-24T05:52:15.267 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/gameport 2023-11-24T05:52:15.267 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/gameport/emu10k1-gp.ko 2023-11-24T05:52:15.267 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/gameport/fm801-gp.ko 2023-11-24T05:52:15.267 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/gameport/gameport.ko 2023-11-24T05:52:15.267 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/joydev.ko 2023-11-24T05:52:15.267 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard 2023-11-24T05:52:15.268 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard/gpio_keys.ko 2023-11-24T05:52:15.268 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse 2023-11-24T05:52:15.268 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/appletouch.ko 2023-11-24T05:52:15.268 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/bcm5974.ko 2023-11-24T05:52:15.268 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/cyapatp.ko 2023-11-24T05:52:15.268 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/elan_i2c.ko 2023-11-24T05:52:15.268 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/sermouse.ko 2023-11-24T05:52:15.268 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/synaptics_i2c.ko 2023-11-24T05:52:15.268 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/synaptics_usb.ko 2023-11-24T05:52:15.268 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/vsxxxaa.ko 2023-11-24T05:52:15.268 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/rmi4 2023-11-24T05:52:15.269 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/rmi4/rmi_core.ko 2023-11-24T05:52:15.269 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/serio 2023-11-24T05:52:15.269 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/serio/altera_ps2.ko 2023-11-24T05:52:15.269 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/serio/arc_ps2.ko 2023-11-24T05:52:15.269 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/serio/hyperv-keyboard.ko 2023-11-24T05:52:15.269 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/serio/serio_raw.ko 2023-11-24T05:52:15.269 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/sparse-keymap.ko 2023-11-24T05:52:15.269 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/iommu 2023-11-24T05:52:15.269 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/iommu/amd 2023-11-24T05:52:15.269 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/iommu/amd/iommu_v2.ko 2023-11-24T05:52:15.270 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds 2023-11-24T05:52:15.270 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-blinkm.ko 2023-11-24T05:52:15.270 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-lm3530.ko 2023-11-24T05:52:15.270 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-lp3944.ko 2023-11-24T05:52:15.270 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-ss4200.ko 2023-11-24T05:52:15.270 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger 2023-11-24T05:52:15.270 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-audio.ko 2023-11-24T05:52:15.270 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2023-11-24T05:52:15.270 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-camera.ko 2023-11-24T05:52:15.270 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2023-11-24T05:52:15.271 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2023-11-24T05:52:15.271 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2023-11-24T05:52:15.271 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-timer.ko 2023-11-24T05:52:15.272 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-transient.ko 2023-11-24T05:52:15.272 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md 2023-11-24T05:52:15.272 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/bcache 2023-11-24T05:52:15.272 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/bcache/bcache.ko 2023-11-24T05:52:15.272 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-bio-prison.ko 2023-11-24T05:52:15.272 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-cache-smq.ko 2023-11-24T05:52:15.272 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-cache.ko 2023-11-24T05:52:15.272 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-crypt.ko 2023-11-24T05:52:15.272 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-delay.ko 2023-11-24T05:52:15.272 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-flakey.ko 2023-11-24T05:52:15.272 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-log-userspace.ko 2023-11-24T05:52:15.273 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-multipath.ko 2023-11-24T05:52:15.273 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-queue-length.ko 2023-11-24T05:52:15.273 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-raid.ko 2023-11-24T05:52:15.273 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-round-robin.ko 2023-11-24T05:52:15.273 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-service-time.ko 2023-11-24T05:52:15.273 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-switch.ko 2023-11-24T05:52:15.273 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-thin-pool.ko 2023-11-24T05:52:15.273 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-verity.ko 2023-11-24T05:52:15.274 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/faulty.ko 2023-11-24T05:52:15.274 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/linear.ko 2023-11-24T05:52:15.274 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/multipath.ko 2023-11-24T05:52:15.274 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/persistent-data 2023-11-24T05:52:15.274 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2023-11-24T05:52:15.274 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/raid0.ko 2023-11-24T05:52:15.274 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/raid1.ko 2023-11-24T05:52:15.274 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/raid10.ko 2023-11-24T05:52:15.274 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/raid456.ko 2023-11-24T05:52:15.274 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message 2023-11-24T05:52:15.274 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion 2023-11-24T05:52:15.274 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion/mptbase.ko 2023-11-24T05:52:15.275 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion/mptctl.ko 2023-11-24T05:52:15.275 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion/mptfc.ko 2023-11-24T05:52:15.275 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion/mptlan.ko 2023-11-24T05:52:15.275 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion/mptsas.ko 2023-11-24T05:52:15.275 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion/mptscsih.ko 2023-11-24T05:52:15.275 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion/mptspi.ko 2023-11-24T05:52:15.275 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd 2023-11-24T05:52:15.275 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/lpc_ich.ko 2023-11-24T05:52:15.275 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/lpc_sch.ko 2023-11-24T05:52:15.275 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/mfd-core.ko 2023-11-24T05:52:15.275 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/sm501.ko 2023-11-24T05:52:15.275 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/viperboard.ko 2023-11-24T05:52:15.275 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/vx855.ko 2023-11-24T05:52:15.276 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/wl1273-core.ko 2023-11-24T05:52:15.276 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc 2023-11-24T05:52:15.276 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/altera-stapl 2023-11-24T05:52:15.276 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2023-11-24T05:52:15.276 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/apds9802als.ko 2023-11-24T05:52:15.277 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/apds990x.ko 2023-11-24T05:52:15.277 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/bh1770glc.ko 2023-11-24T05:52:15.277 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/cb710 2023-11-24T05:52:15.277 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/cb710/cb710.ko 2023-11-24T05:52:15.277 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/eeprom 2023-11-24T05:52:15.277 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/eeprom/at24.ko 2023-11-24T05:52:15.277 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/eeprom/eeprom.ko 2023-11-24T05:52:15.278 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2023-11-24T05:52:15.278 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/eeprom/max6875.ko 2023-11-24T05:52:15.278 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/enclosure.ko 2023-11-24T05:52:15.278 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/hpilo.ko 2023-11-24T05:52:15.278 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/isl29003.ko 2023-11-24T05:52:15.278 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/isl29020.ko 2023-11-24T05:52:15.278 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/lis3lv02d 2023-11-24T05:52:15.278 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2023-11-24T05:52:15.278 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2023-11-24T05:52:15.278 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/mei 2023-11-24T05:52:15.278 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/mei/mei-me.ko 2023-11-24T05:52:15.279 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/mei/mei-txe.ko 2023-11-24T05:52:15.279 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/mei/mei.ko 2023-11-24T05:52:15.279 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/sgi-gru 2023-11-24T05:52:15.279 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/sgi-gru/gru.ko 2023-11-24T05:52:15.279 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/sgi-xp 2023-11-24T05:52:15.279 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/sgi-xp/xp.ko 2023-11-24T05:52:15.279 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/sgi-xp/xpc.ko 2023-11-24T05:52:15.279 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/sgi-xp/xpnet.ko 2023-11-24T05:52:15.279 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/tifm_7xx1.ko 2023-11-24T05:52:15.280 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/tifm_core.ko 2023-11-24T05:52:15.280 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/tsl2550.ko 2023-11-24T05:52:15.280 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/vmw_balloon.ko 2023-11-24T05:52:15.280 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/vmw_vmci 2023-11-24T05:52:15.280 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2023-11-24T05:52:15.280 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net 2023-11-24T05:52:15.280 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/bonding 2023-11-24T05:52:15.280 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/bonding/bonding.ko 2023-11-24T05:52:15.280 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dsa 2023-11-24T05:52:15.280 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dsa/b53 2023-11-24T05:52:15.280 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dsa/b53/b53_common.ko 2023-11-24T05:52:15.280 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dsa/bcm-sf2.ko 2023-11-24T05:52:15.281 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dsa/mv88e6060.ko 2023-11-24T05:52:15.281 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dsa/mv88e6xxx 2023-11-24T05:52:15.281 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2023-11-24T05:52:15.281 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dummy.ko 2023-11-24T05:52:15.281 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/eql.ko 2023-11-24T05:52:15.281 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet 2023-11-24T05:52:15.281 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/3com 2023-11-24T05:52:15.281 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2023-11-24T05:52:15.281 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2023-11-24T05:52:15.281 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/3com/3c59x.ko 2023-11-24T05:52:15.281 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/3com/typhoon.ko 2023-11-24T05:52:15.281 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/8390 2023-11-24T05:52:15.281 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/8390/8390.ko 2023-11-24T05:52:15.282 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2023-11-24T05:52:15.282 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2023-11-24T05:52:15.282 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2023-11-24T05:52:15.282 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/adaptec 2023-11-24T05:52:15.282 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/adaptec/starfire.ko 2023-11-24T05:52:15.282 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/agere 2023-11-24T05:52:15.283 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/agere/et131x.ko 2023-11-24T05:52:15.283 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/alteon 2023-11-24T05:52:15.283 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/alteon/acenic.ko 2023-11-24T05:52:15.283 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/altera 2023-11-24T05:52:15.283 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/altera/altera_tse.ko 2023-11-24T05:52:15.283 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/amd 2023-11-24T05:52:15.283 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/amd/amd8111e.ko 2023-11-24T05:52:15.283 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2023-11-24T05:52:15.284 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/amd/pcnet32.ko 2023-11-24T05:52:15.284 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros 2023-11-24T05:52:15.284 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/alx 2023-11-24T05:52:15.284 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2023-11-24T05:52:15.284 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/atl1c 2023-11-24T05:52:15.284 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2023-11-24T05:52:15.284 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/atl1e 2023-11-24T05:52:15.284 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2023-11-24T05:52:15.284 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/atlx 2023-11-24T05:52:15.284 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2023-11-24T05:52:15.285 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2023-11-24T05:52:15.285 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom 2023-11-24T05:52:15.285 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/b44.ko 2023-11-24T05:52:15.285 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2023-11-24T05:52:15.285 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/bnx2x 2023-11-24T05:52:15.285 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2023-11-24T05:52:15.285 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/cnic.ko 2023-11-24T05:52:15.285 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/genet 2023-11-24T05:52:15.285 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2023-11-24T05:52:15.285 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/tg3.ko 2023-11-24T05:52:15.285 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/brocade 2023-11-24T05:52:15.286 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/brocade/bna 2023-11-24T05:52:15.286 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2023-11-24T05:52:15.286 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio 2023-11-24T05:52:15.286 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb 2023-11-24T05:52:15.286 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2023-11-24T05:52:15.286 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb3 2023-11-24T05:52:15.286 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2023-11-24T05:52:15.287 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb4 2023-11-24T05:52:15.287 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2023-11-24T05:52:15.287 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2023-11-24T05:52:15.287 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2023-11-24T05:52:15.287 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/libcxgb 2023-11-24T05:52:15.287 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2023-11-24T05:52:15.287 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/cisco 2023-11-24T05:52:15.288 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/cisco/enic 2023-11-24T05:52:15.288 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2023-11-24T05:52:15.288 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec 2023-11-24T05:52:15.288 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec/tulip 2023-11-24T05:52:15.288 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2023-11-24T05:52:15.288 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2023-11-24T05:52:15.288 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2023-11-24T05:52:15.288 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2023-11-24T05:52:15.288 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2023-11-24T05:52:15.289 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2023-11-24T05:52:15.289 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dlink 2023-11-24T05:52:15.289 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dlink/dl2k.ko 2023-11-24T05:52:15.289 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dlink/sundance.ko 2023-11-24T05:52:15.289 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dnet.ko 2023-11-24T05:52:15.290 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/emulex 2023-11-24T05:52:15.290 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/emulex/benet 2023-11-24T05:52:15.290 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2023-11-24T05:52:15.290 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/ethoc.ko 2023-11-24T05:52:15.290 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/fealnx.ko 2023-11-24T05:52:15.290 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel 2023-11-24T05:52:15.290 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/e100.ko 2023-11-24T05:52:15.290 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/e1000 2023-11-24T05:52:15.290 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2023-11-24T05:52:15.290 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/e1000e 2023-11-24T05:52:15.290 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2023-11-24T05:52:15.290 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/fm10k 2023-11-24T05:52:15.291 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2023-11-24T05:52:15.291 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/i40e 2023-11-24T05:52:15.291 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2023-11-24T05:52:15.291 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/iavf 2023-11-24T05:52:15.291 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2023-11-24T05:52:15.291 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/igb 2023-11-24T05:52:15.291 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/igb/igb.ko 2023-11-24T05:52:15.291 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/igbvf 2023-11-24T05:52:15.291 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2023-11-24T05:52:15.292 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/ixgbe 2023-11-24T05:52:15.292 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2023-11-24T05:52:15.292 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/ixgbevf 2023-11-24T05:52:15.292 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2023-11-24T05:52:15.292 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/jme.ko 2023-11-24T05:52:15.292 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/marvell 2023-11-24T05:52:15.292 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2023-11-24T05:52:15.292 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/marvell/skge.ko 2023-11-24T05:52:15.292 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/marvell/sky2.ko 2023-11-24T05:52:15.292 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/mellanox 2023-11-24T05:52:15.292 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/mellanox/mlx4 2023-11-24T05:52:15.292 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2023-11-24T05:52:15.292 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2023-11-24T05:52:15.293 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/mellanox/mlx5 2023-11-24T05:52:15.293 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/mellanox/mlx5/core 2023-11-24T05:52:15.293 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2023-11-24T05:52:15.293 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/micrel 2023-11-24T05:52:15.293 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2023-11-24T05:52:15.293 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/myricom 2023-11-24T05:52:15.293 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/myricom/myri10ge 2023-11-24T05:52:15.293 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2023-11-24T05:52:15.293 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/natsemi 2023-11-24T05:52:15.293 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2023-11-24T05:52:15.293 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2023-11-24T05:52:15.293 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/neterion 2023-11-24T05:52:15.293 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/neterion/s2io.ko 2023-11-24T05:52:15.294 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/nvidia 2023-11-24T05:52:15.294 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2023-11-24T05:52:15.294 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/packetengines 2023-11-24T05:52:15.294 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2023-11-24T05:52:15.294 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2023-11-24T05:52:15.294 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/qlogic 2023-11-24T05:52:15.294 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/qlogic/netxen 2023-11-24T05:52:15.294 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2023-11-24T05:52:15.294 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2023-11-24T05:52:15.294 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/qlogic/qlcnic 2023-11-24T05:52:15.294 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2023-11-24T05:52:15.294 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/rdc 2023-11-24T05:52:15.295 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/rdc/r6040.ko 2023-11-24T05:52:15.295 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/realtek 2023-11-24T05:52:15.295 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/realtek/8139cp.ko 2023-11-24T05:52:15.295 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/realtek/8139too.ko 2023-11-24T05:52:15.295 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/realtek/atp.ko 2023-11-24T05:52:15.295 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/realtek/r8169.ko 2023-11-24T05:52:15.295 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/rocker 2023-11-24T05:52:15.295 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/rocker/rocker.ko 2023-11-24T05:52:15.295 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sfc 2023-11-24T05:52:15.295 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sfc/sfc.ko 2023-11-24T05:52:15.295 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/silan 2023-11-24T05:52:15.295 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/silan/sc92031.ko 2023-11-24T05:52:15.295 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sis 2023-11-24T05:52:15.296 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sis/sis190.ko 2023-11-24T05:52:15.296 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sis/sis900.ko 2023-11-24T05:52:15.296 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/smsc 2023-11-24T05:52:15.296 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/smsc/epic100.ko 2023-11-24T05:52:15.296 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2023-11-24T05:52:15.296 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2023-11-24T05:52:15.296 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2023-11-24T05:52:15.296 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/stmicro 2023-11-24T05:52:15.296 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/stmicro/stmmac 2023-11-24T05:52:15.296 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2023-11-24T05:52:15.296 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2023-11-24T05:52:15.296 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sun 2023-11-24T05:52:15.296 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sun/cassini.ko 2023-11-24T05:52:15.297 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sun/niu.ko 2023-11-24T05:52:15.297 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sun/sungem.ko 2023-11-24T05:52:15.297 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sun/sunhme.ko 2023-11-24T05:52:15.298 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/tehuti 2023-11-24T05:52:15.298 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2023-11-24T05:52:15.298 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/ti 2023-11-24T05:52:15.298 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/ti/tlan.ko 2023-11-24T05:52:15.298 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/via 2023-11-24T05:52:15.298 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/via/via-rhine.ko 2023-11-24T05:52:15.298 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/via/via-velocity.ko 2023-11-24T05:52:15.298 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/wiznet 2023-11-24T05:52:15.298 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/wiznet/w5100.ko 2023-11-24T05:52:15.298 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/wiznet/w5300.ko 2023-11-24T05:52:15.298 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/xircom 2023-11-24T05:52:15.298 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2023-11-24T05:52:15.298 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/geneve.ko 2023-11-24T05:52:15.299 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/hyperv 2023-11-24T05:52:15.299 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/hyperv/hv_netvsc.ko 2023-11-24T05:52:15.299 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ieee802154 2023-11-24T05:52:15.299 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ieee802154/fakelb.ko 2023-11-24T05:52:15.299 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ifb.ko 2023-11-24T05:52:15.299 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ipvlan 2023-11-24T05:52:15.299 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ipvlan/ipvlan.ko 2023-11-24T05:52:15.299 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/macvlan.ko 2023-11-24T05:52:15.299 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/macvtap.ko 2023-11-24T05:52:15.299 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/mdio 2023-11-24T05:52:15.299 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/mdio.ko 2023-11-24T05:52:15.299 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2023-11-24T05:52:15.300 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/mdio/mdio-bitbang.ko 2023-11-24T05:52:15.300 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/mdio/mdio-regmap.ko 2023-11-24T05:52:15.300 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/mii.ko 2023-11-24T05:52:15.300 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/net_failover.ko 2023-11-24T05:52:15.300 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/netconsole.ko 2023-11-24T05:52:15.300 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/nlmon.ko 2023-11-24T05:52:15.300 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/pcs 2023-11-24T05:52:15.300 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/pcs/pcs-lynx.ko 2023-11-24T05:52:15.300 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/pcs/pcs_xpcs.ko 2023-11-24T05:52:15.300 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy 2023-11-24T05:52:15.300 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/amd.ko 2023-11-24T05:52:15.300 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/bcm-phy-lib.ko 2023-11-24T05:52:15.301 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/bcm-phy-ptp.ko 2023-11-24T05:52:15.301 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/bcm7xxx.ko 2023-11-24T05:52:15.301 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/bcm87xx.ko 2023-11-24T05:52:15.301 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/broadcom.ko 2023-11-24T05:52:15.301 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/cicada.ko 2023-11-24T05:52:15.301 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/davicom.ko 2023-11-24T05:52:15.301 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/dp83640.ko 2023-11-24T05:52:15.301 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/et1011c.ko 2023-11-24T05:52:15.301 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/icplus.ko 2023-11-24T05:52:15.301 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/lxt.ko 2023-11-24T05:52:15.301 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/marvell.ko 2023-11-24T05:52:15.301 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/micrel.ko 2023-11-24T05:52:15.302 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/national.ko 2023-11-24T05:52:15.302 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/phylink.ko 2023-11-24T05:52:15.302 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/qsemi.ko 2023-11-24T05:52:15.302 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/realtek.ko 2023-11-24T05:52:15.302 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/smsc.ko 2023-11-24T05:52:15.302 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/ste10Xp.ko 2023-11-24T05:52:15.302 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/vitesse.ko 2023-11-24T05:52:15.302 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp 2023-11-24T05:52:15.303 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/bsd_comp.ko 2023-11-24T05:52:15.303 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/ppp_async.ko 2023-11-24T05:52:15.303 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/ppp_deflate.ko 2023-11-24T05:52:15.303 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/ppp_generic.ko 2023-11-24T05:52:15.303 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/ppp_mppe.ko 2023-11-24T05:52:15.303 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/ppp_synctty.ko 2023-11-24T05:52:15.303 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/pppoe.ko 2023-11-24T05:52:15.303 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/pppox.ko 2023-11-24T05:52:15.303 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/pptp.ko 2023-11-24T05:52:15.303 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/slip 2023-11-24T05:52:15.303 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/slip/slhc.ko 2023-11-24T05:52:15.303 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/slip/slip.ko 2023-11-24T05:52:15.303 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/sungem_phy.ko 2023-11-24T05:52:15.304 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/tap.ko 2023-11-24T05:52:15.304 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/team 2023-11-24T05:52:15.304 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/team/team.ko 2023-11-24T05:52:15.304 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/team/team_mode_activebackup.ko 2023-11-24T05:52:15.304 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/team/team_mode_broadcast.ko 2023-11-24T05:52:15.304 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/team/team_mode_loadbalance.ko 2023-11-24T05:52:15.304 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/team/team_mode_random.ko 2023-11-24T05:52:15.305 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/team/team_mode_roundrobin.ko 2023-11-24T05:52:15.305 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/tun.ko 2023-11-24T05:52:15.305 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/veth.ko 2023-11-24T05:52:15.305 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/virtio_net.ko 2023-11-24T05:52:15.305 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/vmxnet3 2023-11-24T05:52:15.305 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/vmxnet3/vmxnet3.ko 2023-11-24T05:52:15.305 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/vxlan 2023-11-24T05:52:15.305 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/vxlan/vxlan.ko 2023-11-24T05:52:15.305 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/xen-netback 2023-11-24T05:52:15.305 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/xen-netback/xen-netback.ko 2023-11-24T05:52:15.305 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/xen-netfront.ko 2023-11-24T05:52:15.305 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ntb 2023-11-24T05:52:15.305 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ntb/ntb.ko 2023-11-24T05:52:15.306 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/nvme 2023-11-24T05:52:15.306 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/nvme/host 2023-11-24T05:52:15.306 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/nvme/host/nvme-core.ko 2023-11-24T05:52:15.306 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/nvme/host/nvme.ko 2023-11-24T05:52:15.306 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/parport 2023-11-24T05:52:15.306 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/parport/parport.ko 2023-11-24T05:52:15.306 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/parport/parport_cs.ko 2023-11-24T05:52:15.306 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/parport/parport_pc.ko 2023-11-24T05:52:15.306 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/parport/parport_serial.ko 2023-11-24T05:52:15.306 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pci 2023-11-24T05:52:15.306 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pci/hotplug 2023-11-24T05:52:15.306 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2023-11-24T05:52:15.307 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pci/pcie 2023-11-24T05:52:15.308 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pci/pcie/aer_inject.ko 2023-11-24T05:52:15.308 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pci/xen-pcifront.ko 2023-11-24T05:52:15.308 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pcmcia 2023-11-24T05:52:15.308 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pcmcia/i82092.ko 2023-11-24T05:52:15.308 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pcmcia/pd6729.ko 2023-11-24T05:52:15.308 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pcmcia/yenta_socket.ko 2023-11-24T05:52:15.308 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pinctrl 2023-11-24T05:52:15.308 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pinctrl/intel 2023-11-24T05:52:15.308 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2023-11-24T05:52:15.308 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform 2023-11-24T05:52:15.308 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86 2023-11-24T05:52:15.309 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/acer-wmi.ko 2023-11-24T05:52:15.309 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/acerhdf.ko 2023-11-24T05:52:15.309 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/apple-gmux.ko 2023-11-24T05:52:15.309 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/asus-laptop.ko 2023-11-24T05:52:15.309 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/asus-nb-wmi.ko 2023-11-24T05:52:15.309 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/asus-wmi.ko 2023-11-24T05:52:15.309 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/classmate-laptop.ko 2023-11-24T05:52:15.309 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/eeepc-laptop.ko 2023-11-24T05:52:15.310 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/eeepc-wmi.ko 2023-11-24T05:52:15.310 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/fujitsu-laptop.ko 2023-11-24T05:52:15.310 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/fujitsu-tablet.ko 2023-11-24T05:52:15.310 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/hdaps.ko 2023-11-24T05:52:15.310 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/intel 2023-11-24T05:52:15.310 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/intel/intel-rst.ko 2023-11-24T05:52:15.310 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/intel_ips.ko 2023-11-24T05:52:15.310 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/msi-wmi.ko 2023-11-24T05:52:15.310 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/mxm-wmi.ko 2023-11-24T05:52:15.311 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/panasonic-laptop.ko 2023-11-24T05:52:15.311 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/samsung-laptop.ko 2023-11-24T05:52:15.311 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/samsung-q10.ko 2023-11-24T05:52:15.311 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/topstar-laptop.ko 2023-11-24T05:52:15.311 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2023-11-24T05:52:15.311 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/toshiba_haps.ko 2023-11-24T05:52:15.311 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/wmi-bmof.ko 2023-11-24T05:52:15.312 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/wmi.ko 2023-11-24T05:52:15.312 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/powercap 2023-11-24T05:52:15.312 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/powercap/intel_rapl_common.ko 2023-11-24T05:52:15.312 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/powercap/intel_rapl_msr.ko 2023-11-24T05:52:15.312 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pps 2023-11-24T05:52:15.312 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pps/clients 2023-11-24T05:52:15.312 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pps/clients/pps-gpio.ko 2023-11-24T05:52:15.312 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pps/clients/pps-ldisc.ko 2023-11-24T05:52:15.312 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pps/clients/pps_parport.ko 2023-11-24T05:52:15.312 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pps/pps_core.ko 2023-11-24T05:52:15.312 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ptp 2023-11-24T05:52:15.312 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ptp/ptp.ko 2023-11-24T05:52:15.312 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ptp/ptp_kvm.ko 2023-11-24T05:52:15.313 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc 2023-11-24T05:52:15.313 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-bq32k.ko 2023-11-24T05:52:15.313 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds1286.ko 2023-11-24T05:52:15.313 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds1307.ko 2023-11-24T05:52:15.313 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds1374.ko 2023-11-24T05:52:15.313 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds1511.ko 2023-11-24T05:52:15.313 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds1553.ko 2023-11-24T05:52:15.313 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds1672.ko 2023-11-24T05:52:15.313 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds1742.ko 2023-11-24T05:52:15.313 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds2404.ko 2023-11-24T05:52:15.313 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds3232.ko 2023-11-24T05:52:15.313 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-em3027.ko 2023-11-24T05:52:15.313 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-fm3130.ko 2023-11-24T05:52:15.314 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-isl12022.ko 2023-11-24T05:52:15.314 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-isl1208.ko 2023-11-24T05:52:15.314 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-m41t80.ko 2023-11-24T05:52:15.314 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-m48t35.ko 2023-11-24T05:52:15.315 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-m48t59.ko 2023-11-24T05:52:15.315 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-max6900.ko 2023-11-24T05:52:15.315 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-msm6242.ko 2023-11-24T05:52:15.315 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-pcf2127.ko 2023-11-24T05:52:15.315 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-pcf85063.ko 2023-11-24T05:52:15.315 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-pcf8523.ko 2023-11-24T05:52:15.315 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-pcf8563.ko 2023-11-24T05:52:15.315 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-pcf8583.ko 2023-11-24T05:52:15.315 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-rp5c01.ko 2023-11-24T05:52:15.315 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-rs5c372.ko 2023-11-24T05:52:15.315 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-rv3029c2.ko 2023-11-24T05:52:15.315 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-rx8025.ko 2023-11-24T05:52:15.315 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-rx8581.ko 2023-11-24T05:52:15.316 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-stk17ta8.ko 2023-11-24T05:52:15.316 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-x1205.ko 2023-11-24T05:52:15.316 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi 2023-11-24T05:52:15.316 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/3w-9xxx.ko 2023-11-24T05:52:15.316 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/3w-sas.ko 2023-11-24T05:52:15.316 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/3w-xxxx.ko 2023-11-24T05:52:15.316 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/BusLogic.ko 2023-11-24T05:52:15.316 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/a100u2w.ko 2023-11-24T05:52:15.316 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/aacraid 2023-11-24T05:52:15.316 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/aacraid/aacraid.ko 2023-11-24T05:52:15.316 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/advansys.ko 2023-11-24T05:52:15.316 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/aic7xxx 2023-11-24T05:52:15.316 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2023-11-24T05:52:15.317 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2023-11-24T05:52:15.317 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/am53c974.ko 2023-11-24T05:52:15.317 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/arcmsr 2023-11-24T05:52:15.317 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/arcmsr/arcmsr.ko 2023-11-24T05:52:15.317 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/atp870u.ko 2023-11-24T05:52:15.317 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/be2iscsi 2023-11-24T05:52:15.317 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2023-11-24T05:52:15.317 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/bfa 2023-11-24T05:52:15.317 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/bfa/bfa.ko 2023-11-24T05:52:15.317 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/bnx2fc 2023-11-24T05:52:15.317 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2023-11-24T05:52:15.317 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/bnx2i 2023-11-24T05:52:15.317 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/bnx2i/bnx2i.ko 2023-11-24T05:52:15.318 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/ch.ko 2023-11-24T05:52:15.318 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/csiostor 2023-11-24T05:52:15.318 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/csiostor/csiostor.ko 2023-11-24T05:52:15.318 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/cxgbi 2023-11-24T05:52:15.318 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/cxgbi/cxgb3i 2023-11-24T05:52:15.318 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2023-11-24T05:52:15.318 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/cxgbi/cxgb4i 2023-11-24T05:52:15.318 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2023-11-24T05:52:15.318 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2023-11-24T05:52:15.318 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/dc395x.ko 2023-11-24T05:52:15.318 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/device_handler 2023-11-24T05:52:15.318 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2023-11-24T05:52:15.318 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2023-11-24T05:52:15.319 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2023-11-24T05:52:15.319 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2023-11-24T05:52:15.319 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/dmx3191d.ko 2023-11-24T05:52:15.319 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/esas2r 2023-11-24T05:52:15.319 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/esas2r/esas2r.ko 2023-11-24T05:52:15.319 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/esp_scsi.ko 2023-11-24T05:52:15.319 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/fcoe 2023-11-24T05:52:15.319 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/fcoe/fcoe.ko 2023-11-24T05:52:15.319 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/fcoe/libfcoe.ko 2023-11-24T05:52:15.319 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/fnic 2023-11-24T05:52:15.319 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/fnic/fnic.ko 2023-11-24T05:52:15.319 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/hpsa.ko 2023-11-24T05:52:15.319 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/hptiop.ko 2023-11-24T05:52:15.320 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/hv_storvsc.ko 2023-11-24T05:52:15.320 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/initio.ko 2023-11-24T05:52:15.320 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/ipr.ko 2023-11-24T05:52:15.320 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/ips.ko 2023-11-24T05:52:15.320 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/isci 2023-11-24T05:52:15.320 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/isci/isci.ko 2023-11-24T05:52:15.320 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2023-11-24T05:52:15.320 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/iscsi_tcp.ko 2023-11-24T05:52:15.320 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/libfc 2023-11-24T05:52:15.320 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/libfc/libfc.ko 2023-11-24T05:52:15.320 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/libiscsi.ko 2023-11-24T05:52:15.320 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/libiscsi_tcp.ko 2023-11-24T05:52:15.320 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/libsas 2023-11-24T05:52:15.321 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/libsas/libsas.ko 2023-11-24T05:52:15.321 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/lpfc 2023-11-24T05:52:15.321 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/lpfc/lpfc.ko 2023-11-24T05:52:15.321 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/megaraid 2023-11-24T05:52:15.321 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/megaraid.ko 2023-11-24T05:52:15.321 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2023-11-24T05:52:15.321 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2023-11-24T05:52:15.321 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2023-11-24T05:52:15.321 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/mpt3sas 2023-11-24T05:52:15.321 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2023-11-24T05:52:15.321 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/mvsas 2023-11-24T05:52:15.321 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/mvsas/mvsas.ko 2023-11-24T05:52:15.321 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/mvumi.ko 2023-11-24T05:52:15.321 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/pm8001 2023-11-24T05:52:15.322 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/pm8001/pm80xx.ko 2023-11-24T05:52:15.322 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/pmcraid.ko 2023-11-24T05:52:15.322 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/qla1280.ko 2023-11-24T05:52:15.322 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/qla2xxx 2023-11-24T05:52:15.322 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2023-11-24T05:52:15.322 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2023-11-24T05:52:15.322 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/qla4xxx 2023-11-24T05:52:15.322 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2023-11-24T05:52:15.322 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/raid_class.ko 2023-11-24T05:52:15.322 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/scsi_debug.ko 2023-11-24T05:52:15.322 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/scsi_transport_fc.ko 2023-11-24T05:52:15.322 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/scsi_transport_iscsi.ko 2023-11-24T05:52:15.322 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/scsi_transport_sas.ko 2023-11-24T05:52:15.323 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/scsi_transport_spi.ko 2023-11-24T05:52:15.323 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/scsi_transport_srp.ko 2023-11-24T05:52:15.323 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/ses.ko 2023-11-24T05:52:15.323 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/st.ko 2023-11-24T05:52:15.323 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/stex.ko 2023-11-24T05:52:15.323 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/sym53c8xx_2 2023-11-24T05:52:15.323 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2023-11-24T05:52:15.323 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/virtio_scsi.ko 2023-11-24T05:52:15.323 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/vmw_pvscsi.ko 2023-11-24T05:52:15.323 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/wd719x.ko 2023-11-24T05:52:15.323 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ssb 2023-11-24T05:52:15.323 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ssb/ssb.ko 2023-11-24T05:52:15.323 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target 2023-11-24T05:52:15.323 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/iscsi 2023-11-24T05:52:15.324 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2023-11-24T05:52:15.324 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/loopback 2023-11-24T05:52:15.324 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/loopback/tcm_loop.ko 2023-11-24T05:52:15.324 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/target_core_file.ko 2023-11-24T05:52:15.324 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/target_core_iblock.ko 2023-11-24T05:52:15.324 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/target_core_mod.ko 2023-11-24T05:52:15.324 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/target_core_pscsi.ko 2023-11-24T05:52:15.324 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/target_core_user.ko 2023-11-24T05:52:15.324 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/tcm_fc 2023-11-24T05:52:15.324 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/tcm_fc/tcm_fc.ko 2023-11-24T05:52:15.324 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal 2023-11-24T05:52:15.325 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel 2023-11-24T05:52:15.325 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal 2023-11-24T05:52:15.325 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2023-11-24T05:52:15.325 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2023-11-24T05:52:15.325 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2023-11-24T05:52:15.325 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2023-11-24T05:52:15.326 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2023-11-24T05:52:15.326 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2023-11-24T05:52:15.326 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2023-11-24T05:52:15.326 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2023-11-24T05:52:15.326 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2023-11-24T05:52:15.326 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2023-11-24T05:52:15.326 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2023-11-24T05:52:15.326 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2023-11-24T05:52:15.326 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2023-11-24T05:52:15.326 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2023-11-24T05:52:15.326 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2023-11-24T05:52:15.326 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty 2023-11-24T05:52:15.326 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/ipwireless 2023-11-24T05:52:15.327 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/ipwireless/ipwireless.ko 2023-11-24T05:52:15.327 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/n_gsm.ko 2023-11-24T05:52:15.327 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/n_hdlc.ko 2023-11-24T05:52:15.327 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/nozomi.ko 2023-11-24T05:52:15.327 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial 2023-11-24T05:52:15.327 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial/8250 2023-11-24T05:52:15.327 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial/8250/serial_cs.ko 2023-11-24T05:52:15.327 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial/arc_uart.ko 2023-11-24T05:52:15.327 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial/jsm 2023-11-24T05:52:15.327 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial/jsm/jsm.ko 2023-11-24T05:52:15.327 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ufs 2023-11-24T05:52:15.327 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ufs/core 2023-11-24T05:52:15.327 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ufs/core/ufshcd-core.ko 2023-11-24T05:52:15.327 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ufs/host 2023-11-24T05:52:15.328 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ufs/host/ufshcd-pci.ko 2023-11-24T05:52:15.328 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/uio 2023-11-24T05:52:15.328 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/uio/uio.ko 2023-11-24T05:52:15.328 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/uio/uio_aec.ko 2023-11-24T05:52:15.328 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/uio/uio_cif.ko 2023-11-24T05:52:15.328 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/uio/uio_pci_generic.ko 2023-11-24T05:52:15.328 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/uio/uio_sercos3.ko 2023-11-24T05:52:15.328 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb 2023-11-24T05:52:15.328 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/class 2023-11-24T05:52:15.328 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/class/cdc-acm.ko 2023-11-24T05:52:15.328 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/class/cdc-wdm.ko 2023-11-24T05:52:15.328 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/class/usblp.ko 2023-11-24T05:52:15.328 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/class/usbtmc.ko 2023-11-24T05:52:15.328 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/host 2023-11-24T05:52:15.329 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/host/sl811-hcd.ko 2023-11-24T05:52:15.329 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/image 2023-11-24T05:52:15.329 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/image/mdc800.ko 2023-11-24T05:52:15.329 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/image/microtek.ko 2023-11-24T05:52:15.329 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc 2023-11-24T05:52:15.329 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/adutux.ko 2023-11-24T05:52:15.329 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/appledisplay.ko 2023-11-24T05:52:15.329 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/emi26.ko 2023-11-24T05:52:15.329 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/emi62.ko 2023-11-24T05:52:15.329 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/ezusb.ko 2023-11-24T05:52:15.329 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/idmouse.ko 2023-11-24T05:52:15.329 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/iowarrior.ko 2023-11-24T05:52:15.329 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/isight_firmware.ko 2023-11-24T05:52:15.329 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/ldusb.ko 2023-11-24T05:52:15.330 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/legousbtower.ko 2023-11-24T05:52:15.330 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/sisusbvga 2023-11-24T05:52:15.330 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2023-11-24T05:52:15.330 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/trancevibrator.ko 2023-11-24T05:52:15.330 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/usb3503.ko 2023-11-24T05:52:15.330 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/usblcd.ko 2023-11-24T05:52:15.330 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/usbsevseg.ko 2023-11-24T05:52:15.330 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/uss720.ko 2023-11-24T05:52:15.330 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/yurex.ko 2023-11-24T05:52:15.330 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/phy 2023-11-24T05:52:15.330 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/phy/phy-generic.ko 2023-11-24T05:52:15.330 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial 2023-11-24T05:52:15.330 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/aircable.ko 2023-11-24T05:52:15.330 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ark3116.ko 2023-11-24T05:52:15.331 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/belkin_sa.ko 2023-11-24T05:52:15.331 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ch341.ko 2023-11-24T05:52:15.331 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/cp210x.ko 2023-11-24T05:52:15.331 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/cyberjack.ko 2023-11-24T05:52:15.331 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/cypress_m8.ko 2023-11-24T05:52:15.331 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/digi_acceleport.ko 2023-11-24T05:52:15.331 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/empeg.ko 2023-11-24T05:52:15.331 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ftdi_sio.ko 2023-11-24T05:52:15.331 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/garmin_gps.ko 2023-11-24T05:52:15.331 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/io_edgeport.ko 2023-11-24T05:52:15.331 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/io_ti.ko 2023-11-24T05:52:15.331 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ipaq.ko 2023-11-24T05:52:15.331 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ipw.ko 2023-11-24T05:52:15.332 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ir-usb.ko 2023-11-24T05:52:15.332 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/iuu_phoenix.ko 2023-11-24T05:52:15.332 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/keyspan.ko 2023-11-24T05:52:15.332 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/keyspan_pda.ko 2023-11-24T05:52:15.332 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/kl5kusb105.ko 2023-11-24T05:52:15.332 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/kobil_sct.ko 2023-11-24T05:52:15.332 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/mct_u232.ko 2023-11-24T05:52:15.332 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/mos7720.ko 2023-11-24T05:52:15.332 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/mos7840.ko 2023-11-24T05:52:15.332 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/navman.ko 2023-11-24T05:52:15.332 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/omninet.ko 2023-11-24T05:52:15.332 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/opticon.ko 2023-11-24T05:52:15.332 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/option.ko 2023-11-24T05:52:15.332 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/oti6858.ko 2023-11-24T05:52:15.332 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/pl2303.ko 2023-11-24T05:52:15.333 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/qcaux.ko 2023-11-24T05:52:15.333 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/qcserial.ko 2023-11-24T05:52:15.333 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/quatech2.ko 2023-11-24T05:52:15.333 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/safe_serial.ko 2023-11-24T05:52:15.333 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/sierra.ko 2023-11-24T05:52:15.333 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/spcp8x5.ko 2023-11-24T05:52:15.333 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ssu100.ko 2023-11-24T05:52:15.334 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/symbolserial.ko 2023-11-24T05:52:15.334 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2023-11-24T05:52:15.334 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/usb-serial-simple.ko 2023-11-24T05:52:15.334 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/usb_debug.ko 2023-11-24T05:52:15.334 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/usb_wwan.ko 2023-11-24T05:52:15.334 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/visor.ko 2023-11-24T05:52:15.334 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/whiteheat.ko 2023-11-24T05:52:15.334 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/xsens_mt.ko 2023-11-24T05:52:15.334 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage 2023-11-24T05:52:15.335 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/uas.ko 2023-11-24T05:52:15.335 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-alauda.ko 2023-11-24T05:52:15.335 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-cypress.ko 2023-11-24T05:52:15.335 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-datafab.ko 2023-11-24T05:52:15.335 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-eneub6250.ko 2023-11-24T05:52:15.335 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-freecom.ko 2023-11-24T05:52:15.335 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-isd200.ko 2023-11-24T05:52:15.336 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-jumpshot.ko 2023-11-24T05:52:15.336 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-karma.ko 2023-11-24T05:52:15.336 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-onetouch.ko 2023-11-24T05:52:15.336 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-realtek.ko 2023-11-24T05:52:15.336 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-sddr09.ko 2023-11-24T05:52:15.336 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-sddr55.ko 2023-11-24T05:52:15.336 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-usbat.ko 2023-11-24T05:52:15.336 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/usb-storage.ko 2023-11-24T05:52:15.336 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/usbip 2023-11-24T05:52:15.336 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/usbip/usbip-core.ko 2023-11-24T05:52:15.336 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/usbip/usbip-host.ko 2023-11-24T05:52:15.336 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/usbip/vhci-hcd.ko 2023-11-24T05:52:15.337 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vfio 2023-11-24T05:52:15.337 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vfio/pci 2023-11-24T05:52:15.337 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vfio/pci/vfio-pci-core.ko 2023-11-24T05:52:15.337 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vfio/pci/vfio-pci.ko 2023-11-24T05:52:15.337 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vfio/vfio.ko 2023-11-24T05:52:15.337 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vfio/vfio_iommu_type1.ko 2023-11-24T05:52:15.337 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vhost 2023-11-24T05:52:15.337 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vhost/vhost.ko 2023-11-24T05:52:15.337 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vhost/vhost_iotlb.ko 2023-11-24T05:52:15.337 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vhost/vhost_net.ko 2023-11-24T05:52:15.337 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vhost/vhost_scsi.ko 2023-11-24T05:52:15.337 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video 2023-11-24T05:52:15.337 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight 2023-11-24T05:52:15.337 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/apple_bl.ko 2023-11-24T05:52:15.338 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/lcd.ko 2023-11-24T05:52:15.338 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/platform_lcd.ko 2023-11-24T05:52:15.338 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/virtio 2023-11-24T05:52:15.338 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/virtio/virtio.ko 2023-11-24T05:52:15.338 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/virtio/virtio_balloon.ko 2023-11-24T05:52:15.338 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/virtio/virtio_mmio.ko 2023-11-24T05:52:15.338 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/virtio/virtio_pci.ko 2023-11-24T05:52:15.338 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2023-11-24T05:52:15.338 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2023-11-24T05:52:15.338 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/virtio/virtio_ring.ko 2023-11-24T05:52:15.338 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1 2023-11-24T05:52:15.338 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/masters 2023-11-24T05:52:15.338 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/masters/ds2482.ko 2023-11-24T05:52:15.338 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/masters/ds2490.ko 2023-11-24T05:52:15.339 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves 2023-11-24T05:52:15.339 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2406.ko 2023-11-24T05:52:15.339 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2408.ko 2023-11-24T05:52:15.339 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2413.ko 2023-11-24T05:52:15.339 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2423.ko 2023-11-24T05:52:15.339 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2431.ko 2023-11-24T05:52:15.339 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2433.ko 2023-11-24T05:52:15.339 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2780.ko 2023-11-24T05:52:15.339 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2781.ko 2023-11-24T05:52:15.339 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds28e04.ko 2023-11-24T05:52:15.339 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_smem.ko 2023-11-24T05:52:15.339 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_therm.ko 2023-11-24T05:52:15.339 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/wire.ko 2023-11-24T05:52:15.339 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog 2023-11-24T05:52:15.340 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/alim1535_wdt.ko 2023-11-24T05:52:15.340 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/alim7101_wdt.ko 2023-11-24T05:52:15.340 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/f71808e_wdt.ko 2023-11-24T05:52:15.340 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/hpwdt.ko 2023-11-24T05:52:15.340 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/i6300esb.ko 2023-11-24T05:52:15.340 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/iTCO_vendor_support.ko 2023-11-24T05:52:15.340 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/iTCO_wdt.ko 2023-11-24T05:52:15.340 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/ib700wdt.ko 2023-11-24T05:52:15.340 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/ibmasr.ko 2023-11-24T05:52:15.340 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/ie6xx_wdt.ko 2023-11-24T05:52:15.340 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/it8712f_wdt.ko 2023-11-24T05:52:15.340 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/it87_wdt.ko 2023-11-24T05:52:15.340 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/machzwd.ko 2023-11-24T05:52:15.340 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/nv_tco.ko 2023-11-24T05:52:15.341 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/pcwd_pci.ko 2023-11-24T05:52:15.341 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/pcwd_usb.ko 2023-11-24T05:52:15.341 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2023-11-24T05:52:15.341 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/sch311x_wdt.ko 2023-11-24T05:52:15.341 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/softdog.ko 2023-11-24T05:52:15.341 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/sp5100_tco.ko 2023-11-24T05:52:15.341 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/via_wdt.ko 2023-11-24T05:52:15.341 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/w83627hf_wdt.ko 2023-11-24T05:52:15.341 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/w83877f_wdt.ko 2023-11-24T05:52:15.341 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/w83977f_wdt.ko 2023-11-24T05:52:15.341 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/wdt_pci.ko 2023-11-24T05:52:15.341 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/xen_wdt.ko 2023-11-24T05:52:15.341 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen 2023-11-24T05:52:15.341 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xen-acpi-processor.ko 2023-11-24T05:52:15.342 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xen-evtchn.ko 2023-11-24T05:52:15.342 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xen-gntalloc.ko 2023-11-24T05:52:15.342 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xen-gntdev.ko 2023-11-24T05:52:15.342 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xen-pciback 2023-11-24T05:52:15.342 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2023-11-24T05:52:15.342 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xen-privcmd.ko 2023-11-24T05:52:15.342 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xenfs 2023-11-24T05:52:15.342 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xenfs/xenfs.ko 2023-11-24T05:52:15.342 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs 2023-11-24T05:52:15.342 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/9p 2023-11-24T05:52:15.342 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/9p/9p.ko 2023-11-24T05:52:15.342 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/affs 2023-11-24T05:52:15.342 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/affs/affs.ko 2023-11-24T05:52:15.342 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/befs 2023-11-24T05:52:15.343 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/befs/befs.ko 2023-11-24T05:52:15.343 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/binfmt_misc.ko 2023-11-24T05:52:15.343 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/btrfs 2023-11-24T05:52:15.343 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/btrfs/btrfs.ko 2023-11-24T05:52:15.343 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/cachefiles 2023-11-24T05:52:15.343 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/cachefiles/cachefiles.ko 2023-11-24T05:52:15.343 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ceph 2023-11-24T05:52:15.343 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ceph/ceph.ko 2023-11-24T05:52:15.343 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/coda 2023-11-24T05:52:15.343 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/coda/coda.ko 2023-11-24T05:52:15.343 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/cramfs 2023-11-24T05:52:15.343 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/cramfs/cramfs.ko 2023-11-24T05:52:15.343 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/dlm 2023-11-24T05:52:15.343 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/dlm/dlm.ko 2023-11-24T05:52:15.344 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ecryptfs 2023-11-24T05:52:15.344 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ecryptfs/ecryptfs.ko 2023-11-24T05:52:15.344 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/f2fs 2023-11-24T05:52:15.344 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/f2fs/f2fs.ko 2023-11-24T05:52:15.344 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fat 2023-11-24T05:52:15.344 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fat/fat.ko 2023-11-24T05:52:15.345 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fat/msdos.ko 2023-11-24T05:52:15.345 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fat/vfat.ko 2023-11-24T05:52:15.345 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fscache 2023-11-24T05:52:15.345 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fscache/fscache.ko 2023-11-24T05:52:15.345 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fuse 2023-11-24T05:52:15.345 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fuse/cuse.ko 2023-11-24T05:52:15.345 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fuse/fuse.ko 2023-11-24T05:52:15.345 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/gfs2 2023-11-24T05:52:15.345 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/gfs2/gfs2.ko 2023-11-24T05:52:15.345 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/hfs 2023-11-24T05:52:15.345 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/hfs/hfs.ko 2023-11-24T05:52:15.345 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/hfsplus 2023-11-24T05:52:15.345 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/hfsplus/hfsplus.ko 2023-11-24T05:52:15.345 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/isofs 2023-11-24T05:52:15.345 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/isofs/isofs.ko 2023-11-24T05:52:15.346 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/jfs 2023-11-24T05:52:15.346 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/jfs/jfs.ko 2023-11-24T05:52:15.346 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/lockd 2023-11-24T05:52:15.346 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/lockd/lockd.ko 2023-11-24T05:52:15.346 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/minix 2023-11-24T05:52:15.346 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/minix/minix.ko 2023-11-24T05:52:15.346 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/netfs 2023-11-24T05:52:15.346 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/netfs/netfs.ko 2023-11-24T05:52:15.346 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs 2023-11-24T05:52:15.346 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/blocklayout 2023-11-24T05:52:15.346 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2023-11-24T05:52:15.347 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/filelayout 2023-11-24T05:52:15.347 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2023-11-24T05:52:15.347 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/flexfilelayout 2023-11-24T05:52:15.347 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2023-11-24T05:52:15.347 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/nfs.ko 2023-11-24T05:52:15.347 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/nfsv3.ko 2023-11-24T05:52:15.347 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/nfsv4.ko 2023-11-24T05:52:15.347 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs_common 2023-11-24T05:52:15.347 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs_common/grace.ko 2023-11-24T05:52:15.347 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs_common/nfs_acl.ko 2023-11-24T05:52:15.347 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfsd 2023-11-24T05:52:15.347 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfsd/nfsd.ko 2023-11-24T05:52:15.347 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nilfs2 2023-11-24T05:52:15.347 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nilfs2/nilfs2.ko 2023-11-24T05:52:15.348 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls 2023-11-24T05:52:15.348 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-celtic.ko 2023-11-24T05:52:15.348 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-centeuro.ko 2023-11-24T05:52:15.348 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-croatian.ko 2023-11-24T05:52:15.348 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-cyrillic.ko 2023-11-24T05:52:15.348 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-gaelic.ko 2023-11-24T05:52:15.348 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-greek.ko 2023-11-24T05:52:15.348 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-iceland.ko 2023-11-24T05:52:15.348 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-inuit.ko 2023-11-24T05:52:15.348 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-roman.ko 2023-11-24T05:52:15.348 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-romanian.ko 2023-11-24T05:52:15.348 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-turkish.ko 2023-11-24T05:52:15.348 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp1250.ko 2023-11-24T05:52:15.348 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp1251.ko 2023-11-24T05:52:15.348 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp1255.ko 2023-11-24T05:52:15.349 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp737.ko 2023-11-24T05:52:15.349 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp775.ko 2023-11-24T05:52:15.349 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp850.ko 2023-11-24T05:52:15.349 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp852.ko 2023-11-24T05:52:15.349 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp855.ko 2023-11-24T05:52:15.349 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp857.ko 2023-11-24T05:52:15.349 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp860.ko 2023-11-24T05:52:15.349 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp861.ko 2023-11-24T05:52:15.349 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp862.ko 2023-11-24T05:52:15.349 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp863.ko 2023-11-24T05:52:15.349 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp864.ko 2023-11-24T05:52:15.349 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp865.ko 2023-11-24T05:52:15.349 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp866.ko 2023-11-24T05:52:15.349 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp869.ko 2023-11-24T05:52:15.350 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp874.ko 2023-11-24T05:52:15.350 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp932.ko 2023-11-24T05:52:15.350 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp936.ko 2023-11-24T05:52:15.350 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp949.ko 2023-11-24T05:52:15.350 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp950.ko 2023-11-24T05:52:15.350 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_euc-jp.ko 2023-11-24T05:52:15.350 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-1.ko 2023-11-24T05:52:15.350 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-13.ko 2023-11-24T05:52:15.350 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-14.ko 2023-11-24T05:52:15.350 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-15.ko 2023-11-24T05:52:15.350 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-2.ko 2023-11-24T05:52:15.350 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-3.ko 2023-11-24T05:52:15.350 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-4.ko 2023-11-24T05:52:15.351 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-5.ko 2023-11-24T05:52:15.351 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-6.ko 2023-11-24T05:52:15.351 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-7.ko 2023-11-24T05:52:15.351 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-9.ko 2023-11-24T05:52:15.351 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_koi8-r.ko 2023-11-24T05:52:15.351 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_koi8-ru.ko 2023-11-24T05:52:15.351 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_koi8-u.ko 2023-11-24T05:52:15.351 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_ucs2_utils.ko 2023-11-24T05:52:15.351 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_utf8.ko 2023-11-24T05:52:15.351 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2 2023-11-24T05:52:15.351 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/cluster 2023-11-24T05:52:15.351 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2023-11-24T05:52:15.351 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/dlm 2023-11-24T05:52:15.351 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2023-11-24T05:52:15.351 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/dlmfs 2023-11-24T05:52:15.352 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2023-11-24T05:52:15.352 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/ocfs2.ko 2023-11-24T05:52:15.352 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2023-11-24T05:52:15.352 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/ocfs2_stack_user.ko 2023-11-24T05:52:15.352 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/ocfs2_stackglue.ko 2023-11-24T05:52:15.352 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/overlayfs 2023-11-24T05:52:15.352 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/overlayfs/overlay.ko 2023-11-24T05:52:15.352 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/pstore 2023-11-24T05:52:15.352 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/pstore/ramoops.ko 2023-11-24T05:52:15.352 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/reiserfs 2023-11-24T05:52:15.352 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/reiserfs/reiserfs.ko 2023-11-24T05:52:15.352 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/romfs 2023-11-24T05:52:15.352 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/romfs/romfs.ko 2023-11-24T05:52:15.353 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/smb 2023-11-24T05:52:15.353 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/smb/client 2023-11-24T05:52:15.353 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/smb/client/cifs.ko 2023-11-24T05:52:15.353 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/smb/common 2023-11-24T05:52:15.353 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/smb/common/cifs_arc4.ko 2023-11-24T05:52:15.353 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/smb/common/cifs_md4.ko 2023-11-24T05:52:15.353 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/squashfs 2023-11-24T05:52:15.353 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/squashfs/squashfs.ko 2023-11-24T05:52:15.353 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/sysv 2023-11-24T05:52:15.353 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/sysv/sysv.ko 2023-11-24T05:52:15.353 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/udf 2023-11-24T05:52:15.353 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/udf/udf.ko 2023-11-24T05:52:15.353 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ufs 2023-11-24T05:52:15.353 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ufs/ufs.ko 2023-11-24T05:52:15.354 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/xfs 2023-11-24T05:52:15.354 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/xfs/xfs.ko 2023-11-24T05:52:15.354 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/kernel 2023-11-24T05:52:15.354 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/kernel/rcu 2023-11-24T05:52:15.354 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/kernel/rcu/rcutorture.ko 2023-11-24T05:52:15.354 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/kernel/torture.ko 2023-11-24T05:52:15.354 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/kernel/trace 2023-11-24T05:52:15.354 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/kernel/trace/ring_buffer_benchmark.ko 2023-11-24T05:52:15.354 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib 2023-11-24T05:52:15.354 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/asn1_encoder.ko 2023-11-24T05:52:15.354 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/crc-itu-t.ko 2023-11-24T05:52:15.354 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/crc8.ko 2023-11-24T05:52:15.354 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/crypto 2023-11-24T05:52:15.354 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/crypto/libarc4.ko 2023-11-24T05:52:15.355 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/crypto/libdes.ko 2023-11-24T05:52:15.355 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/libcrc32c.ko 2023-11-24T05:52:15.355 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/lru_cache.ko 2023-11-24T05:52:15.355 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/lz4 2023-11-24T05:52:15.355 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/lz4/lz4_compress.ko 2023-11-24T05:52:15.355 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/lz4/lz4hc_compress.ko 2023-11-24T05:52:15.355 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/math 2023-11-24T05:52:15.355 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/math/cordic.ko 2023-11-24T05:52:15.355 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/raid6 2023-11-24T05:52:15.355 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/raid6/raid6_pq.ko 2023-11-24T05:52:15.355 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/reed_solomon 2023-11-24T05:52:15.355 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/reed_solomon/reed_solomon.ko 2023-11-24T05:52:15.355 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/ts_bm.ko 2023-11-24T05:52:15.355 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/ts_fsm.ko 2023-11-24T05:52:15.356 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/ts_kmp.ko 2023-11-24T05:52:15.356 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/zstd 2023-11-24T05:52:15.356 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/zstd/zstd_compress.ko 2023-11-24T05:52:15.356 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/mm 2023-11-24T05:52:15.356 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/mm/hwpoison-inject.ko 2023-11-24T05:52:15.356 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net 2023-11-24T05:52:15.356 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan 2023-11-24T05:52:15.356 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/6lowpan.ko 2023-11-24T05:52:15.356 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/nhc_dest.ko 2023-11-24T05:52:15.356 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/nhc_fragment.ko 2023-11-24T05:52:15.356 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/nhc_hop.ko 2023-11-24T05:52:15.356 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/nhc_ipv6.ko 2023-11-24T05:52:15.356 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/nhc_mobility.ko 2023-11-24T05:52:15.356 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/nhc_routing.ko 2023-11-24T05:52:15.357 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/nhc_udp.ko 2023-11-24T05:52:15.357 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/802 2023-11-24T05:52:15.357 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/802/garp.ko 2023-11-24T05:52:15.357 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/802/mrp.ko 2023-11-24T05:52:15.357 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/802/p8022.ko 2023-11-24T05:52:15.357 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/802/psnap.ko 2023-11-24T05:52:15.357 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/802/stp.ko 2023-11-24T05:52:15.357 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/8021q 2023-11-24T05:52:15.357 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/8021q/8021q.ko 2023-11-24T05:52:15.357 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/9p 2023-11-24T05:52:15.357 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/9p/9pnet.ko 2023-11-24T05:52:15.357 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/9p/9pnet_fd.ko 2023-11-24T05:52:15.357 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/9p/9pnet_rdma.ko 2023-11-24T05:52:15.357 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/9p/9pnet_virtio.ko 2023-11-24T05:52:15.358 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/batman-adv 2023-11-24T05:52:15.358 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/batman-adv/batman-adv.ko 2023-11-24T05:52:15.358 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge 2023-11-24T05:52:15.358 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/br_netfilter.ko 2023-11-24T05:52:15.358 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/bridge.ko 2023-11-24T05:52:15.358 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter 2023-11-24T05:52:15.358 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_802_3.ko 2023-11-24T05:52:15.358 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_among.ko 2023-11-24T05:52:15.358 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_arp.ko 2023-11-24T05:52:15.358 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_arpreply.ko 2023-11-24T05:52:15.358 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_dnat.ko 2023-11-24T05:52:15.358 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_ip.ko 2023-11-24T05:52:15.358 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_ip6.ko 2023-11-24T05:52:15.359 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_limit.ko 2023-11-24T05:52:15.359 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_log.ko 2023-11-24T05:52:15.359 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_mark.ko 2023-11-24T05:52:15.359 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_mark_m.ko 2023-11-24T05:52:15.359 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_nflog.ko 2023-11-24T05:52:15.359 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_pkttype.ko 2023-11-24T05:52:15.359 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_redirect.ko 2023-11-24T05:52:15.359 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_snat.ko 2023-11-24T05:52:15.359 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_stp.ko 2023-11-24T05:52:15.359 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_vlan.ko 2023-11-24T05:52:15.359 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebtable_broute.ko 2023-11-24T05:52:15.359 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebtable_filter.ko 2023-11-24T05:52:15.360 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebtable_nat.ko 2023-11-24T05:52:15.360 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebtables.ko 2023-11-24T05:52:15.360 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2023-11-24T05:52:15.360 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2023-11-24T05:52:15.360 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2023-11-24T05:52:15.360 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ceph 2023-11-24T05:52:15.360 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ceph/libceph.ko 2023-11-24T05:52:15.360 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/core 2023-11-24T05:52:15.360 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/core/failover.ko 2023-11-24T05:52:15.360 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/core/pktgen.ko 2023-11-24T05:52:15.360 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dccp 2023-11-24T05:52:15.360 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dccp/dccp.ko 2023-11-24T05:52:15.360 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dccp/dccp_diag.ko 2023-11-24T05:52:15.361 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dccp/dccp_ipv4.ko 2023-11-24T05:52:15.361 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dccp/dccp_ipv6.ko 2023-11-24T05:52:15.361 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dns_resolver 2023-11-24T05:52:15.361 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dns_resolver/dns_resolver.ko 2023-11-24T05:52:15.361 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dsa 2023-11-24T05:52:15.361 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dsa/dsa_core.ko 2023-11-24T05:52:15.361 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dsa/tag_brcm.ko 2023-11-24T05:52:15.361 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dsa/tag_dsa.ko 2023-11-24T05:52:15.361 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dsa/tag_none.ko 2023-11-24T05:52:15.361 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dsa/tag_trailer.ko 2023-11-24T05:52:15.361 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ieee802154 2023-11-24T05:52:15.361 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ieee802154/6lowpan 2023-11-24T05:52:15.361 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2023-11-24T05:52:15.361 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ieee802154/ieee802154.ko 2023-11-24T05:52:15.362 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ieee802154/ieee802154_socket.ko 2023-11-24T05:52:15.362 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4 2023-11-24T05:52:15.362 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/ah4.ko 2023-11-24T05:52:15.362 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/esp4.ko 2023-11-24T05:52:15.362 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/fou.ko 2023-11-24T05:52:15.362 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/gre.ko 2023-11-24T05:52:15.362 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/inet_diag.ko 2023-11-24T05:52:15.362 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/ip_gre.ko 2023-11-24T05:52:15.362 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/ip_tunnel.ko 2023-11-24T05:52:15.362 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/ip_vti.ko 2023-11-24T05:52:15.362 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/ipcomp.ko 2023-11-24T05:52:15.362 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/ipip.ko 2023-11-24T05:52:15.362 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter 2023-11-24T05:52:15.363 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/arp_tables.ko 2023-11-24T05:52:15.363 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/arpt_mangle.ko 2023-11-24T05:52:15.363 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/arptable_filter.ko 2023-11-24T05:52:15.363 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/ipt_ECN.ko 2023-11-24T05:52:15.363 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2023-11-24T05:52:15.363 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/ipt_ah.ko 2023-11-24T05:52:15.363 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2023-11-24T05:52:15.363 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/iptable_mangle.ko 2023-11-24T05:52:15.363 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/iptable_nat.ko 2023-11-24T05:52:15.363 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/iptable_raw.ko 2023-11-24T05:52:15.363 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/iptable_security.ko 2023-11-24T05:52:15.363 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2023-11-24T05:52:15.363 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2023-11-24T05:52:15.363 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2023-11-24T05:52:15.364 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2023-11-24T05:52:15.364 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2023-11-24T05:52:15.364 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2023-11-24T05:52:15.364 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2023-11-24T05:52:15.364 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2023-11-24T05:52:15.364 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2023-11-24T05:52:15.364 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2023-11-24T05:52:15.364 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_bic.ko 2023-11-24T05:52:15.364 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_dctcp.ko 2023-11-24T05:52:15.364 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_diag.ko 2023-11-24T05:52:15.364 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_highspeed.ko 2023-11-24T05:52:15.364 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_htcp.ko 2023-11-24T05:52:15.364 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_hybla.ko 2023-11-24T05:52:15.364 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_illinois.ko 2023-11-24T05:52:15.365 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_lp.ko 2023-11-24T05:52:15.365 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_scalable.ko 2023-11-24T05:52:15.365 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_vegas.ko 2023-11-24T05:52:15.365 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_veno.ko 2023-11-24T05:52:15.365 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_westwood.ko 2023-11-24T05:52:15.365 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_yeah.ko 2023-11-24T05:52:15.365 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tunnel4.ko 2023-11-24T05:52:15.365 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/udp_diag.ko 2023-11-24T05:52:15.365 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/udp_tunnel.ko 2023-11-24T05:52:15.365 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/xfrm4_tunnel.ko 2023-11-24T05:52:15.365 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6 2023-11-24T05:52:15.365 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/ah6.ko 2023-11-24T05:52:15.365 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/esp6.ko 2023-11-24T05:52:15.365 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/fou6.ko 2023-11-24T05:52:15.366 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/ip6_tunnel.ko 2023-11-24T05:52:15.366 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/ip6_udp_tunnel.ko 2023-11-24T05:52:15.366 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/ip6_vti.ko 2023-11-24T05:52:15.366 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/ipcomp6.ko 2023-11-24T05:52:15.366 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter 2023-11-24T05:52:15.366 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6_tables.ko 2023-11-24T05:52:15.366 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2023-11-24T05:52:15.366 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2023-11-24T05:52:15.366 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_ah.ko 2023-11-24T05:52:15.366 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2023-11-24T05:52:15.366 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_frag.ko 2023-11-24T05:52:15.366 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2023-11-24T05:52:15.366 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2023-11-24T05:52:15.366 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_mh.ko 2023-11-24T05:52:15.367 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2023-11-24T05:52:15.367 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_rt.ko 2023-11-24T05:52:15.367 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6table_filter.ko 2023-11-24T05:52:15.367 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2023-11-24T05:52:15.367 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6table_nat.ko 2023-11-24T05:52:15.367 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6table_raw.ko 2023-11-24T05:52:15.367 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6table_security.ko 2023-11-24T05:52:15.367 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2023-11-24T05:52:15.367 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2023-11-24T05:52:15.367 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2023-11-24T05:52:15.368 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2023-11-24T05:52:15.368 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2023-11-24T05:52:15.368 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2023-11-24T05:52:15.368 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2023-11-24T05:52:15.368 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2023-11-24T05:52:15.368 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/sit.ko 2023-11-24T05:52:15.368 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/tunnel6.ko 2023-11-24T05:52:15.368 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/xfrm6_tunnel.ko 2023-11-24T05:52:15.368 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/key 2023-11-24T05:52:15.368 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/key/af_key.ko 2023-11-24T05:52:15.368 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp 2023-11-24T05:52:15.368 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp/l2tp_core.ko 2023-11-24T05:52:15.369 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp/l2tp_debugfs.ko 2023-11-24T05:52:15.369 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp/l2tp_eth.ko 2023-11-24T05:52:15.369 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp/l2tp_ip.ko 2023-11-24T05:52:15.369 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp/l2tp_ip6.ko 2023-11-24T05:52:15.369 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp/l2tp_netlink.ko 2023-11-24T05:52:15.369 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp/l2tp_ppp.ko 2023-11-24T05:52:15.369 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/llc 2023-11-24T05:52:15.369 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/llc/llc.ko 2023-11-24T05:52:15.369 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/mac802154 2023-11-24T05:52:15.369 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/mac802154/mac802154.ko 2023-11-24T05:52:15.369 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/mpls 2023-11-24T05:52:15.369 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/mpls/mpls_gso.ko 2023-11-24T05:52:15.369 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter 2023-11-24T05:52:15.370 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset 2023-11-24T05:52:15.370 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set.ko 2023-11-24T05:52:15.370 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2023-11-24T05:52:15.370 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2023-11-24T05:52:15.370 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2023-11-24T05:52:15.370 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2023-11-24T05:52:15.370 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2023-11-24T05:52:15.370 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2023-11-24T05:52:15.370 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2023-11-24T05:52:15.370 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2023-11-24T05:52:15.370 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2023-11-24T05:52:15.370 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2023-11-24T05:52:15.370 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2023-11-24T05:52:15.370 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2023-11-24T05:52:15.371 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2023-11-24T05:52:15.371 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2023-11-24T05:52:15.371 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_list_set.ko 2023-11-24T05:52:15.371 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs 2023-11-24T05:52:15.371 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs.ko 2023-11-24T05:52:15.371 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2023-11-24T05:52:15.371 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2023-11-24T05:52:15.371 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2023-11-24T05:52:15.371 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2023-11-24T05:52:15.371 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2023-11-24T05:52:15.371 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2023-11-24T05:52:15.371 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2023-11-24T05:52:15.371 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2023-11-24T05:52:15.372 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2023-11-24T05:52:15.372 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2023-11-24T05:52:15.372 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2023-11-24T05:52:15.372 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2023-11-24T05:52:15.372 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2023-11-24T05:52:15.372 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conncount.ko 2023-11-24T05:52:15.372 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack.ko 2023-11-24T05:52:15.372 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_amanda.ko 2023-11-24T05:52:15.372 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_broadcast.ko 2023-11-24T05:52:15.372 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_ftp.ko 2023-11-24T05:52:15.372 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_h323.ko 2023-11-24T05:52:15.372 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_irc.ko 2023-11-24T05:52:15.372 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2023-11-24T05:52:15.372 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_netlink.ko 2023-11-24T05:52:15.373 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_pptp.ko 2023-11-24T05:52:15.373 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_sane.ko 2023-11-24T05:52:15.373 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_sip.ko 2023-11-24T05:52:15.373 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_snmp.ko 2023-11-24T05:52:15.373 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_tftp.ko 2023-11-24T05:52:15.373 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_dup_netdev.ko 2023-11-24T05:52:15.373 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_flow_table.ko 2023-11-24T05:52:15.373 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_flow_table_inet.ko 2023-11-24T05:52:15.373 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_log_syslog.ko 2023-11-24T05:52:15.373 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_nat.ko 2023-11-24T05:52:15.373 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_nat_amanda.ko 2023-11-24T05:52:15.373 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_nat_ftp.ko 2023-11-24T05:52:15.373 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_nat_irc.ko 2023-11-24T05:52:15.373 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_nat_sip.ko 2023-11-24T05:52:15.373 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_nat_tftp.ko 2023-11-24T05:52:15.374 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_synproxy_core.ko 2023-11-24T05:52:15.374 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_tables.ko 2023-11-24T05:52:15.374 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nfnetlink.ko 2023-11-24T05:52:15.374 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nfnetlink_acct.ko 2023-11-24T05:52:15.374 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nfnetlink_log.ko 2023-11-24T05:52:15.374 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nfnetlink_osf.ko 2023-11-24T05:52:15.374 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nfnetlink_queue.ko 2023-11-24T05:52:15.374 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_chain_nat.ko 2023-11-24T05:52:15.374 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_compat.ko 2023-11-24T05:52:15.374 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_ct.ko 2023-11-24T05:52:15.374 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_dup_netdev.ko 2023-11-24T05:52:15.374 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_fib.ko 2023-11-24T05:52:15.374 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_fwd_netdev.ko 2023-11-24T05:52:15.375 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_hash.ko 2023-11-24T05:52:15.375 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_limit.ko 2023-11-24T05:52:15.375 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_log.ko 2023-11-24T05:52:15.375 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_masq.ko 2023-11-24T05:52:15.375 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_nat.ko 2023-11-24T05:52:15.375 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_numgen.ko 2023-11-24T05:52:15.375 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_queue.ko 2023-11-24T05:52:15.375 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_quota.ko 2023-11-24T05:52:15.375 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_redir.ko 2023-11-24T05:52:15.375 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_reject.ko 2023-11-24T05:52:15.375 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_reject_inet.ko 2023-11-24T05:52:15.375 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_socket.ko 2023-11-24T05:52:15.375 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_tunnel.ko 2023-11-24T05:52:15.375 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_AUDIT.ko 2023-11-24T05:52:15.376 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_CHECKSUM.ko 2023-11-24T05:52:15.376 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_CLASSIFY.ko 2023-11-24T05:52:15.376 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_CONNSECMARK.ko 2023-11-24T05:52:15.376 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_CT.ko 2023-11-24T05:52:15.376 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_DSCP.ko 2023-11-24T05:52:15.376 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_HL.ko 2023-11-24T05:52:15.376 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_HMARK.ko 2023-11-24T05:52:15.376 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_IDLETIMER.ko 2023-11-24T05:52:15.376 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_LED.ko 2023-11-24T05:52:15.376 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_LOG.ko 2023-11-24T05:52:15.376 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_MASQUERADE.ko 2023-11-24T05:52:15.376 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_NETMAP.ko 2023-11-24T05:52:15.376 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_NFLOG.ko 2023-11-24T05:52:15.376 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_NFQUEUE.ko 2023-11-24T05:52:15.377 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_RATEEST.ko 2023-11-24T05:52:15.377 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_REDIRECT.ko 2023-11-24T05:52:15.377 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_SECMARK.ko 2023-11-24T05:52:15.377 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_TCPMSS.ko 2023-11-24T05:52:15.377 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2023-11-24T05:52:15.377 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_TEE.ko 2023-11-24T05:52:15.377 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_TPROXY.ko 2023-11-24T05:52:15.377 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_TRACE.ko 2023-11-24T05:52:15.377 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_addrtype.ko 2023-11-24T05:52:15.377 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_bpf.ko 2023-11-24T05:52:15.377 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_cgroup.ko 2023-11-24T05:52:15.377 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_cluster.ko 2023-11-24T05:52:15.377 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_comment.ko 2023-11-24T05:52:15.378 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_connbytes.ko 2023-11-24T05:52:15.378 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_connlabel.ko 2023-11-24T05:52:15.378 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_connlimit.ko 2023-11-24T05:52:15.378 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_connmark.ko 2023-11-24T05:52:15.378 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_conntrack.ko 2023-11-24T05:52:15.378 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_cpu.ko 2023-11-24T05:52:15.378 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_dccp.ko 2023-11-24T05:52:15.379 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_devgroup.ko 2023-11-24T05:52:15.379 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_dscp.ko 2023-11-24T05:52:15.379 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_ecn.ko 2023-11-24T05:52:15.379 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_esp.ko 2023-11-24T05:52:15.379 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_hashlimit.ko 2023-11-24T05:52:15.379 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_helper.ko 2023-11-24T05:52:15.379 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_hl.ko 2023-11-24T05:52:15.379 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_ipcomp.ko 2023-11-24T05:52:15.379 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_iprange.ko 2023-11-24T05:52:15.379 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_ipvs.ko 2023-11-24T05:52:15.379 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_l2tp.ko 2023-11-24T05:52:15.379 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_length.ko 2023-11-24T05:52:15.379 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_limit.ko 2023-11-24T05:52:15.379 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_mac.ko 2023-11-24T05:52:15.380 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_mark.ko 2023-11-24T05:52:15.380 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_multiport.ko 2023-11-24T05:52:15.380 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_nat.ko 2023-11-24T05:52:15.380 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_nfacct.ko 2023-11-24T05:52:15.380 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_osf.ko 2023-11-24T05:52:15.380 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_owner.ko 2023-11-24T05:52:15.380 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_physdev.ko 2023-11-24T05:52:15.380 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_pkttype.ko 2023-11-24T05:52:15.380 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_policy.ko 2023-11-24T05:52:15.380 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_quota.ko 2023-11-24T05:52:15.380 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_rateest.ko 2023-11-24T05:52:15.380 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_realm.ko 2023-11-24T05:52:15.380 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_recent.ko 2023-11-24T05:52:15.381 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_sctp.ko 2023-11-24T05:52:15.381 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_set.ko 2023-11-24T05:52:15.381 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_socket.ko 2023-11-24T05:52:15.381 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_state.ko 2023-11-24T05:52:15.381 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_statistic.ko 2023-11-24T05:52:15.381 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_string.ko 2023-11-24T05:52:15.381 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_tcpmss.ko 2023-11-24T05:52:15.381 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_time.ko 2023-11-24T05:52:15.381 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_u32.ko 2023-11-24T05:52:15.381 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netlink 2023-11-24T05:52:15.381 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netlink/netlink_diag.ko 2023-11-24T05:52:15.381 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/nsh 2023-11-24T05:52:15.381 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/nsh/nsh.ko 2023-11-24T05:52:15.381 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/openvswitch 2023-11-24T05:52:15.382 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/openvswitch/openvswitch.ko 2023-11-24T05:52:15.382 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/openvswitch/vport-geneve.ko 2023-11-24T05:52:15.382 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/openvswitch/vport-gre.ko 2023-11-24T05:52:15.382 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/openvswitch/vport-vxlan.ko 2023-11-24T05:52:15.382 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/packet 2023-11-24T05:52:15.382 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/packet/af_packet_diag.ko 2023-11-24T05:52:15.382 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/rds 2023-11-24T05:52:15.382 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/rds/rds.ko 2023-11-24T05:52:15.383 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/rds/rds_rdma.ko 2023-11-24T05:52:15.383 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/rds/rds_tcp.ko 2023-11-24T05:52:15.383 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched 2023-11-24T05:52:15.383 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_csum.ko 2023-11-24T05:52:15.383 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_gact.ko 2023-11-24T05:52:15.383 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_ipt.ko 2023-11-24T05:52:15.383 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_mirred.ko 2023-11-24T05:52:15.383 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_nat.ko 2023-11-24T05:52:15.383 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_pedit.ko 2023-11-24T05:52:15.384 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_police.ko 2023-11-24T05:52:15.384 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_simple.ko 2023-11-24T05:52:15.384 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_skbedit.ko 2023-11-24T05:52:15.384 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_vlan.ko 2023-11-24T05:52:15.384 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/cls_basic.ko 2023-11-24T05:52:15.384 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/cls_bpf.ko 2023-11-24T05:52:15.384 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/cls_flow.ko 2023-11-24T05:52:15.384 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/cls_fw.ko 2023-11-24T05:52:15.384 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/cls_route.ko 2023-11-24T05:52:15.384 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/cls_u32.ko 2023-11-24T05:52:15.384 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/em_cmp.ko 2023-11-24T05:52:15.384 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/em_ipset.ko 2023-11-24T05:52:15.384 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/em_meta.ko 2023-11-24T05:52:15.384 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/em_nbyte.ko 2023-11-24T05:52:15.385 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/em_text.ko 2023-11-24T05:52:15.385 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/em_u32.ko 2023-11-24T05:52:15.385 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_choke.ko 2023-11-24T05:52:15.385 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_codel.ko 2023-11-24T05:52:15.385 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_drr.ko 2023-11-24T05:52:15.385 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_fq.ko 2023-11-24T05:52:15.385 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_gred.ko 2023-11-24T05:52:15.385 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_hfsc.ko 2023-11-24T05:52:15.385 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_hhf.ko 2023-11-24T05:52:15.385 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_htb.ko 2023-11-24T05:52:15.385 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_ingress.ko 2023-11-24T05:52:15.385 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_mqprio.ko 2023-11-24T05:52:15.385 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_mqprio_lib.ko 2023-11-24T05:52:15.385 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_multiq.ko 2023-11-24T05:52:15.386 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_netem.ko 2023-11-24T05:52:15.386 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_pie.ko 2023-11-24T05:52:15.386 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_plug.ko 2023-11-24T05:52:15.386 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_prio.ko 2023-11-24T05:52:15.386 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_qfq.ko 2023-11-24T05:52:15.386 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_red.ko 2023-11-24T05:52:15.386 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_sfb.ko 2023-11-24T05:52:15.386 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_sfq.ko 2023-11-24T05:52:15.386 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_tbf.ko 2023-11-24T05:52:15.386 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_teql.ko 2023-11-24T05:52:15.386 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sctp 2023-11-24T05:52:15.386 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sctp/sctp.ko 2023-11-24T05:52:15.386 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sctp/sctp_diag.ko 2023-11-24T05:52:15.386 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sunrpc 2023-11-24T05:52:15.387 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sunrpc/auth_gss 2023-11-24T05:52:15.387 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2023-11-24T05:52:15.387 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2023-11-24T05:52:15.387 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sunrpc/sunrpc.ko 2023-11-24T05:52:15.387 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sunrpc/xprtrdma 2023-11-24T05:52:15.387 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2023-11-24T05:52:15.387 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/tipc 2023-11-24T05:52:15.387 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/tipc/diag.ko 2023-11-24T05:52:15.387 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/tipc/tipc.ko 2023-11-24T05:52:15.387 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/unix 2023-11-24T05:52:15.387 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/unix/unix_diag.ko 2023-11-24T05:52:15.387 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/vmw_vsock 2023-11-24T05:52:15.387 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2023-11-24T05:52:15.387 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2023-11-24T05:52:15.388 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/vmw_vsock/vsock.ko 2023-11-24T05:52:15.388 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/vmw_vsock/vsock_diag.ko 2023-11-24T05:52:15.388 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/vmw_vsock/vsock_loopback.ko 2023-11-24T05:52:15.388 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/xfrm 2023-11-24T05:52:15.388 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/xfrm/xfrm_ipcomp.ko 2023-11-24T05:52:15.388 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/security 2023-11-24T05:52:15.388 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/security/keys 2023-11-24T05:52:15.388 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/security/keys/encrypted-keys 2023-11-24T05:52:15.388 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2023-11-24T05:52:15.388 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/security/keys/trusted-keys 2023-11-24T05:52:15.388 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/security/keys/trusted-keys/trusted.ko 2023-11-24T05:52:15.388 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/virt 2023-11-24T05:52:15.388 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/virt/lib 2023-11-24T05:52:15.388 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/virt/lib/irqbypass.ko 2023-11-24T05:52:15.389 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.alias 2023-11-24T05:52:15.389 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.alias.bin 2023-11-24T05:52:15.389 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.builtin 2023-11-24T05:52:15.389 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.builtin.alias.bin 2023-11-24T05:52:15.389 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.builtin.bin 2023-11-24T05:52:15.389 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.builtin.modinfo 2023-11-24T05:52:15.389 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.dep 2023-11-24T05:52:15.389 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.dep.bin 2023-11-24T05:52:15.389 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.devname 2023-11-24T05:52:15.389 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.order 2023-11-24T05:52:15.389 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.softdep 2023-11-24T05:52:15.389 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.symbols 2023-11-24T05:52:15.389 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.symbols.bin 2023-11-24T05:52:15.390 DEBUG:teuthology.task.kernel:get_image_version: 6.6.0-g8247bbb5b470 2023-11-24T05:52:15.390 DEBUG:teuthology.orchestra.run.smithi092:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2023-11-24T05:52:15.417 INFO:teuthology.orchestra.run.smithi092.stdout:postinstall scriptlet (using /bin/sh): 2023-11-24T05:52:15.417 INFO:teuthology.orchestra.run.smithi092.stdout:if [ -x /sbin/installkernel -a -r /boot/vmlinuz-6.6.0-g8247bbb5b470 -a -r /boot/System.map-6.6.0-g8247bbb5b470 ]; then 2023-11-24T05:52:15.417 INFO:teuthology.orchestra.run.smithi092.stdout:cp /boot/vmlinuz-6.6.0-g8247bbb5b470 /boot/.vmlinuz-6.6.0-g8247bbb5b470-rpm 2023-11-24T05:52:15.417 INFO:teuthology.orchestra.run.smithi092.stdout:cp /boot/System.map-6.6.0-g8247bbb5b470 /boot/.System.map-6.6.0-g8247bbb5b470-rpm 2023-11-24T05:52:15.417 INFO:teuthology.orchestra.run.smithi092.stdout:rm -f /boot/vmlinuz-6.6.0-g8247bbb5b470 /boot/System.map-6.6.0-g8247bbb5b470 2023-11-24T05:52:15.417 INFO:teuthology.orchestra.run.smithi092.stdout:/sbin/installkernel 6.6.0-g8247bbb5b470 /boot/.vmlinuz-6.6.0-g8247bbb5b470-rpm /boot/.System.map-6.6.0-g8247bbb5b470-rpm 2023-11-24T05:52:15.417 INFO:teuthology.orchestra.run.smithi092.stdout:rm -f /boot/.vmlinuz-6.6.0-g8247bbb5b470-rpm /boot/.System.map-6.6.0-g8247bbb5b470-rpm 2023-11-24T05:52:15.417 INFO:teuthology.orchestra.run.smithi092.stdout:fi 2023-11-24T05:52:15.418 INFO:teuthology.orchestra.run.smithi092.stdout:preuninstall scriptlet (using /bin/sh): 2023-11-24T05:52:15.418 INFO:teuthology.orchestra.run.smithi092.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2023-11-24T05:52:15.418 INFO:teuthology.orchestra.run.smithi092.stdout:new-kernel-pkg --remove 6.6.0-g8247bbb5b470 --rminitrd --initrdfile=/boot/initramfs-6.6.0-g8247bbb5b470.img 2023-11-24T05:52:15.418 INFO:teuthology.orchestra.run.smithi092.stdout:elif [ -x /usr/bin/kernel-install ]; then 2023-11-24T05:52:15.418 INFO:teuthology.orchestra.run.smithi092.stdout:kernel-install remove 6.6.0-g8247bbb5b470 2023-11-24T05:52:15.418 INFO:teuthology.orchestra.run.smithi092.stdout:fi 2023-11-24T05:52:15.418 INFO:teuthology.orchestra.run.smithi092.stdout:postuninstall scriptlet (using /bin/sh): 2023-11-24T05:52:15.418 INFO:teuthology.orchestra.run.smithi092.stdout:if [ -x /sbin/update-bootloader ]; then 2023-11-24T05:52:15.418 INFO:teuthology.orchestra.run.smithi092.stdout:/sbin/update-bootloader --remove 6.6.0-g8247bbb5b470 2023-11-24T05:52:15.418 INFO:teuthology.orchestra.run.smithi092.stdout:fi 2023-11-24T05:52:15.419 DEBUG:teuthology.orchestra.run.smithi092:> sudo rpm -qi grub2-tools 2023-11-24T05:52:15.496 INFO:teuthology.orchestra.run.smithi092.stdout:Name : grub2-tools 2023-11-24T05:52:15.496 INFO:teuthology.orchestra.run.smithi092.stdout:Epoch : 1 2023-11-24T05:52:15.496 INFO:teuthology.orchestra.run.smithi092.stdout:Version : 2.06 2023-11-24T05:52:15.496 INFO:teuthology.orchestra.run.smithi092.stdout:Release : 68.el9 2023-11-24T05:52:15.496 INFO:teuthology.orchestra.run.smithi092.stdout:Architecture: x86_64 2023-11-24T05:52:15.496 INFO:teuthology.orchestra.run.smithi092.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2023-11-24T05:52:15.496 INFO:teuthology.orchestra.run.smithi092.stdout:Group : Unspecified 2023-11-24T05:52:15.496 INFO:teuthology.orchestra.run.smithi092.stdout:Size : 8264254 2023-11-24T05:52:15.496 INFO:teuthology.orchestra.run.smithi092.stdout:License : GPLv3+ 2023-11-24T05:52:15.496 INFO:teuthology.orchestra.run.smithi092.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2023-11-24T05:52:15.497 INFO:teuthology.orchestra.run.smithi092.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2023-11-24T05:52:15.497 INFO:teuthology.orchestra.run.smithi092.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2023-11-24T05:52:15.497 INFO:teuthology.orchestra.run.smithi092.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2023-11-24T05:52:15.497 INFO:teuthology.orchestra.run.smithi092.stdout:Packager : builder@centos.org 2023-11-24T05:52:15.497 INFO:teuthology.orchestra.run.smithi092.stdout:Vendor : CentOS 2023-11-24T05:52:15.497 INFO:teuthology.orchestra.run.smithi092.stdout:URL : http://www.gnu.org/software/grub/ 2023-11-24T05:52:15.497 INFO:teuthology.orchestra.run.smithi092.stdout:Summary : Support tools for GRUB. 2023-11-24T05:52:15.497 INFO:teuthology.orchestra.run.smithi092.stdout:Description : 2023-11-24T05:52:15.497 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-24T05:52:15.497 INFO:teuthology.orchestra.run.smithi092.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-11-24T05:52:15.497 INFO:teuthology.orchestra.run.smithi092.stdout:customizable bootloader with modular architecture. It supports a rich 2023-11-24T05:52:15.497 INFO:teuthology.orchestra.run.smithi092.stdout:variety of kernel formats, file systems, computer architectures and 2023-11-24T05:52:15.497 INFO:teuthology.orchestra.run.smithi092.stdout:hardware devices. 2023-11-24T05:52:15.498 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-24T05:52:15.498 INFO:teuthology.orchestra.run.smithi092.stdout:This subpackage provides tools for support of all platforms. 2023-11-24T05:52:15.499 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-11-24T05:52:15.499 INFO:teuthology.task.kernel:Updating grub on smithi092 to boot 6.6.0-g8247bbb5b470 2023-11-24T05:52:15.499 DEBUG:teuthology.orchestra.run.smithi092:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-11-24T05:52:16.036 INFO:teuthology.orchestra.run.smithi092.stderr:Generating grub configuration file ... 2023-11-24T05:52:17.366 INFO:teuthology.orchestra.run.smithi092.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2023-11-24T05:52:17.405 INFO:teuthology.orchestra.run.smithi092.stderr:done 2023-11-24T05:52:17.410 DEBUG:teuthology.orchestra.run.smithi092:> mktemp 2023-11-24T05:52:17.427 INFO:teuthology.orchestra.run.smithi092.stdout:/tmp/tmp.05oBJRL5SQ 2023-11-24T05:52:17.427 DEBUG:teuthology.orchestra.run.smithi092:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.05oBJRL5SQ 2023-11-24T05:52:17.496 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod 0666 /tmp/tmp.05oBJRL5SQ 2023-11-24T05:52:17.636 DEBUG:teuthology.orchestra.remote:smithi092:/tmp/tmp.05oBJRL5SQ is 6KB 2023-11-24T05:52:17.695 DEBUG:teuthology.orchestra.run.smithi092:> rm -fr /tmp/tmp.05oBJRL5SQ 2023-11-24T05:52:17.710 DEBUG:teuthology.orchestra.run.smithi092:> sudo /bin/ls /boot/loader/entries || true 2023-11-24T05:52:17.775 INFO:teuthology.orchestra.run.smithi092.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2023-11-24T05:52:17.775 INFO:teuthology.orchestra.run.smithi092.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2023-11-24T05:52:17.775 INFO:teuthology.orchestra.run.smithi092.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2023-11-24T05:52:17.775 INFO:teuthology.orchestra.run.smithi092.stdout:15949e3c88704300882482eddfd7baae-6.6.0-g8247bbb5b470.conf 2023-11-24T05:52:17.776 DEBUG:teuthology.orchestra.run.smithi092:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-6.6.0-g8247bbb5b470 2023-11-24T05:52:17.920 DEBUG:teuthology.orchestra.run.smithi092:> sudo shutdown -r now 2023-11-24T05:52:40.599 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2023-11-24T05:52:40.599 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi118.front.sepia.ceph.com' 2023-11-24T05:52:40.600 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi118.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-24T05:52:47.930 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2023-11-24T05:52:47.930 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi092.front.sepia.ceph.com' 2023-11-24T05:52:47.931 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi092.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-24T05:52:58.978 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.118 2023-11-24T05:53:06.405 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.92 2023-11-24T05:53:07.980 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi118.front.sepia.ceph.com' 2023-11-24T05:53:07.981 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi118.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-24T05:53:11.042 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.118 2023-11-24T05:53:15.410 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi092.front.sepia.ceph.com' 2023-11-24T05:53:15.411 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi092.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-24T05:53:18.466 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.92 2023-11-24T05:53:23.060 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi118.front.sepia.ceph.com' 2023-11-24T05:53:23.061 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi118.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-24T05:53:30.474 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi092.front.sepia.ceph.com' 2023-11-24T05:53:30.475 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi092.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-24T05:54:02.326 DEBUG:teuthology.orchestra.run.smithi092:> true 2023-11-24T05:54:02.942 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi092.front.sepia.ceph.com' 2023-11-24T05:54:02.942 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "6.6.0-g8247bbb5b470"... 2023-11-24T05:54:02.942 DEBUG:teuthology.orchestra.run.smithi092:> uname -r 2023-11-24T05:54:02.985 INFO:teuthology.orchestra.run.smithi092.stdout:6.6.0-g8247bbb5b470 2023-11-24T05:54:02.985 DEBUG:teuthology.task.kernel:current kernel version is 6.6.0-g8247bbb5b470 vs 6.6.0-g8247bbb5b470 2023-11-24T05:54:02.985 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-11-24T05:54:02.985 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2023-11-24T05:54:03.986 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2023-11-24T05:54:03.987 DEBUG:teuthology.orchestra.run.smithi092:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-11-24T05:54:04.147 INFO:teuthology.orchestra.run.smithi092.stdout:ttyS1 2023-11-24T05:54:04.212 DEBUG:teuthology.parallel:result is None 2023-11-24T05:54:23.065 DEBUG:teuthology.orchestra.remote:timed out 2023-11-24T05:54:38.066 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi118.front.sepia.ceph.com' 2023-11-24T05:54:38.068 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi118.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-24T05:54:38.564 DEBUG:teuthology.orchestra.run.smithi118:> true 2023-11-24T05:54:39.273 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi118.front.sepia.ceph.com' 2023-11-24T05:54:39.273 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "6.6.0-g8247bbb5b470"... 2023-11-24T05:54:39.273 DEBUG:teuthology.orchestra.run.smithi118:> uname -r 2023-11-24T05:54:39.326 INFO:teuthology.orchestra.run.smithi118.stdout:6.6.0-g8247bbb5b470 2023-11-24T05:54:39.327 DEBUG:teuthology.task.kernel:current kernel version is 6.6.0-g8247bbb5b470 vs 6.6.0-g8247bbb5b470 2023-11-24T05:54:39.327 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-11-24T05:54:39.327 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2023-11-24T05:54:40.328 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2023-11-24T05:54:40.328 DEBUG:teuthology.orchestra.run.smithi118:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-11-24T05:54:40.480 INFO:teuthology.orchestra.run.smithi118.stdout:ttyS1 2023-11-24T05:54:40.519 DEBUG:teuthology.parallel:result is None 2023-11-24T05:54:40.519 INFO:teuthology.run_tasks:Running task internal.base... 2023-11-24T05:54:40.528 INFO:teuthology.task.internal:Creating test directory... 2023-11-24T05:54:40.528 DEBUG:teuthology.orchestra.run.smithi092:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-11-24T05:54:40.539 DEBUG:teuthology.orchestra.run.smithi118:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-11-24T05:54:40.604 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-11-24T05:54:40.628 INFO:teuthology.run_tasks:Running task internal.archive... 2023-11-24T05:54:40.642 INFO:teuthology.task.internal:Creating archive directory... 2023-11-24T05:54:40.643 DEBUG:teuthology.orchestra.run.smithi092:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-11-24T05:54:40.652 DEBUG:teuthology.orchestra.run.smithi118:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-11-24T05:54:40.722 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-11-24T05:54:40.733 INFO:teuthology.task.internal:Enabling coredump saving... 2023-11-24T05:54:40.734 DEBUG:teuthology.orchestra.run.smithi092:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-11-24T05:54:40.740 DEBUG:teuthology.orchestra.run.smithi118:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-11-24T05:54:40.867 INFO:teuthology.orchestra.run.smithi118.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-24T05:54:40.868 INFO:teuthology.orchestra.run.smithi092.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-24T05:54:40.913 INFO:teuthology.orchestra.run.smithi118.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-24T05:54:40.914 INFO:teuthology.orchestra.run.smithi092.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-24T05:54:40.919 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-11-24T05:54:40.930 INFO:teuthology.task.internal:Configuring sudo... 2023-11-24T05:54:40.930 DEBUG:teuthology.orchestra.run.smithi092:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-11-24T05:54:40.935 DEBUG:teuthology.orchestra.run.smithi118:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-11-24T05:54:41.036 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-11-24T05:54:41.049 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-11-24T05:54:41.050 DEBUG:teuthology.orchestra.run.smithi092:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-11-24T05:54:41.058 DEBUG:teuthology.orchestra.run.smithi118:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-11-24T05:54:41.120 DEBUG:teuthology.orchestra.run.smithi092:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-11-24T05:54:41.210 DEBUG:teuthology.orchestra.run.smithi092:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-11-24T05:54:41.322 DEBUG:teuthology.orchestra.run.smithi092:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-11-24T05:54:41.409 DEBUG:teuthology.orchestra.run.smithi092:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-11-24T05:54:41.517 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-11-24T05:54:41.517 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-11-24T05:54:41.615 DEBUG:teuthology.orchestra.run.smithi118:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-11-24T05:54:41.705 DEBUG:teuthology.orchestra.run.smithi118:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-11-24T05:54:41.818 DEBUG:teuthology.orchestra.run.smithi118:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-11-24T05:54:41.909 DEBUG:teuthology.orchestra.run.smithi118:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-11-24T05:54:42.010 DEBUG:teuthology.orchestra.run.smithi118:> set -ex 2023-11-24T05:54:42.010 DEBUG:teuthology.orchestra.run.smithi118:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-11-24T05:54:42.107 DEBUG:teuthology.orchestra.run.smithi092:> sudo service rsyslog restart 2023-11-24T05:54:42.117 DEBUG:teuthology.orchestra.run.smithi118:> sudo service rsyslog restart 2023-11-24T05:54:42.250 INFO:teuthology.orchestra.run.smithi092.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-11-24T05:54:42.275 INFO:teuthology.orchestra.run.smithi118.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-11-24T05:54:42.619 INFO:teuthology.run_tasks:Running task internal.timer... 2023-11-24T05:54:42.631 INFO:teuthology.task.internal:Starting timer... 2023-11-24T05:54:42.632 INFO:teuthology.run_tasks:Running task pcp... 2023-11-24T05:54:42.649 INFO:teuthology.run_tasks:Running task selinux... 2023-11-24T05:54:42.667 DEBUG:teuthology.orchestra.run.smithi092:> sudo service auditd rotate 2023-11-24T05:54:42.793 INFO:teuthology.orchestra.run.smithi092.stdout:Rotating logs: 2023-11-24T05:54:42.798 DEBUG:teuthology.orchestra.run.smithi118:> sudo service auditd rotate 2023-11-24T05:54:42.934 INFO:teuthology.orchestra.run.smithi118.stdout:Rotating logs: 2023-11-24T05:54:42.938 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-11-24T05:54:42.938 DEBUG:teuthology.orchestra.run.smithi092:> /usr/sbin/getenforce 2023-11-24T05:54:43.009 INFO:teuthology.orchestra.run.smithi092.stdout:Permissive 2023-11-24T05:54:43.010 DEBUG:teuthology.orchestra.run.smithi118:> /usr/sbin/getenforce 2023-11-24T05:54:43.071 INFO:teuthology.orchestra.run.smithi118.stdout:Permissive 2023-11-24T05:54:43.071 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi092.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi118.front.sepia.ceph.com': 'permissive'} 2023-11-24T05:54:43.071 DEBUG:teuthology.orchestra.run.smithi092:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-11-24T05:54:43.170 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-24T05:54:43.171 DEBUG:teuthology.orchestra.run.smithi118:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-11-24T05:54:43.270 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-24T05:54:43.271 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-11-24T05:54:43.271 DEBUG:teuthology.orchestra.run.smithi092:> sudo /usr/sbin/setenforce permissive 2023-11-24T05:54:43.379 DEBUG:teuthology.orchestra.run.smithi118:> sudo /usr/sbin/setenforce permissive 2023-11-24T05:54:43.484 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-11-24T05:54:43.498 INFO:teuthology.repo_utils:/home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main was just updated or references a specific commit; assuming it is current 2023-11-24T05:54:43.498 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-11-24T05:54:43.576 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-11-24T05:54:43.579 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi092.front.sepia.ceph.com,smithi118.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-11-24T06:06:49.357 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi092.front.sepia.ceph.com'), Remote(name='ubuntu@smithi118.front.sepia.ceph.com')] 2023-11-24T06:06:49.358 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi092.front.sepia.ceph.com' 2023-11-24T06:06:49.359 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi092.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-24T06:06:49.494 DEBUG:teuthology.orchestra.run.smithi092:> true 2023-11-24T06:06:49.645 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi092.front.sepia.ceph.com' 2023-11-24T06:06:49.646 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi118.front.sepia.ceph.com' 2023-11-24T06:06:49.646 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi118.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-24T06:06:49.785 DEBUG:teuthology.orchestra.run.smithi118:> true 2023-11-24T06:06:49.940 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi118.front.sepia.ceph.com' 2023-11-24T06:06:49.941 INFO:teuthology.run_tasks:Running task clock... 2023-11-24T06:06:49.952 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-11-24T06:06:49.952 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-11-24T06:06:49.953 DEBUG:teuthology.orchestra.run.smithi092:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-11-24T06:06:49.962 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-11-24T06:06:49.962 DEBUG:teuthology.orchestra.run.smithi118:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-11-24T06:06:50.059 INFO:teuthology.orchestra.run.smithi118.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-11-24T06:06:50.065 INFO:teuthology.orchestra.run.smithi092.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-11-24T06:06:50.115 INFO:teuthology.orchestra.run.smithi118.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-11-24T06:06:50.118 INFO:teuthology.orchestra.run.smithi092.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-11-24T06:06:50.209 INFO:teuthology.orchestra.run.smithi118.stderr:sudo: ntpd: command not found 2023-11-24T06:06:50.211 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: ntpd: command not found 2023-11-24T06:06:50.249 INFO:teuthology.orchestra.run.smithi118.stdout:506 Cannot talk to daemon 2023-11-24T06:06:50.254 INFO:teuthology.orchestra.run.smithi092.stdout:506 Cannot talk to daemon 2023-11-24T06:06:50.300 INFO:teuthology.orchestra.run.smithi118.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-11-24T06:06:50.309 INFO:teuthology.orchestra.run.smithi092.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-11-24T06:06:50.353 INFO:teuthology.orchestra.run.smithi118.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-11-24T06:06:50.360 INFO:teuthology.orchestra.run.smithi092.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-11-24T06:06:50.473 INFO:teuthology.orchestra.run.smithi118.stderr:bash: line 1: ntpq: command not found 2023-11-24T06:06:50.480 INFO:teuthology.orchestra.run.smithi118.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-11-24T06:06:50.481 INFO:teuthology.orchestra.run.smithi118.stdout:=============================================================================== 2023-11-24T06:06:50.481 INFO:teuthology.orchestra.run.smithi118.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-24T06:06:50.481 INFO:teuthology.orchestra.run.smithi118.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-24T06:06:50.481 INFO:teuthology.orchestra.run.smithi118.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-24T06:06:50.481 INFO:teuthology.orchestra.run.smithi118.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-24T06:06:50.487 INFO:teuthology.orchestra.run.smithi092.stderr:bash: line 1: ntpq: command not found 2023-11-24T06:06:50.496 INFO:teuthology.orchestra.run.smithi092.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-11-24T06:06:50.496 INFO:teuthology.orchestra.run.smithi092.stdout:=============================================================================== 2023-11-24T06:06:50.496 INFO:teuthology.orchestra.run.smithi092.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-24T06:06:50.496 INFO:teuthology.orchestra.run.smithi092.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-24T06:06:50.496 INFO:teuthology.orchestra.run.smithi092.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-24T06:06:50.497 INFO:teuthology.orchestra.run.smithi092.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-24T06:06:50.498 INFO:teuthology.run_tasks:Running task install... 2023-11-24T06:06:50.505 DEBUG:teuthology.task.install:project ceph 2023-11-24T06:06:50.506 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'a13ea5ff00cb4df7a9cc941166ddeb358881144c'}} 2023-11-24T06:06:50.506 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'a13ea5ff00cb4df7a9cc941166ddeb358881144c'} 2023-11-24T06:06:50.506 INFO:teuthology.task.install:Using flavor: default 2023-11-24T06:06:50.511 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-11-24T06:06:50.512 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2023-11-24T06:06:50.512 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'a13ea5ff00cb4df7a9cc941166ddeb358881144c', 'tag': None, 'wait_for_package': False} 2023-11-24T06:06:50.512 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a13ea5ff00cb4df7a9cc941166ddeb358881144c 2023-11-24T06:06:50.514 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'a13ea5ff00cb4df7a9cc941166ddeb358881144c', 'tag': None, 'wait_for_package': False} 2023-11-24T06:06:50.514 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a13ea5ff00cb4df7a9cc941166ddeb358881144c 2023-11-24T06:06:50.657 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/a13ea5ff00cb4df7a9cc941166ddeb358881144c/centos/9/flavors/default/ 2023-11-24T06:06:50.657 INFO:teuthology.task.install.rpm:Package version is 18.0.0-7484.ga13ea5ff 2023-11-24T06:06:50.763 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/a13ea5ff00cb4df7a9cc941166ddeb358881144c/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/a13ea5ff00cb4df7a9cc941166ddeb358881144c/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/a13ea5ff00cb4df7a9cc941166ddeb358881144c/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-11-24T06:06:50.764 DEBUG:teuthology.orchestra.run.smithi118:> set -ex 2023-11-24T06:06:50.764 DEBUG:teuthology.orchestra.run.smithi118:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-11-24T06:06:50.784 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/a13ea5ff00cb4df7a9cc941166ddeb358881144c/centos/9/flavors/default/ 2023-11-24T06:06:50.784 INFO:teuthology.task.install.rpm:Package version is 18.0.0-7484.ga13ea5ff 2023-11-24T06:06:50.868 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-11-24T06:06:50.868 DEBUG:teuthology.orchestra.run.smithi118:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/a13ea5ff00cb4df7a9cc941166ddeb358881144c/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-11-24T06:06:50.894 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/a13ea5ff00cb4df7a9cc941166ddeb358881144c/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/a13ea5ff00cb4df7a9cc941166ddeb358881144c/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/a13ea5ff00cb4df7a9cc941166ddeb358881144c/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-11-24T06:06:50.894 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-11-24T06:06:50.894 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-11-24T06:06:50.995 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-11-24T06:06:50.995 DEBUG:teuthology.orchestra.run.smithi092:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/a13ea5ff00cb4df7a9cc941166ddeb358881144c/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-11-24T06:06:51.010 DEBUG:teuthology.orchestra.run.smithi118:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-11-24T06:06:51.138 DEBUG:teuthology.orchestra.run.smithi092:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-11-24T06:06:51.142 DEBUG:teuthology.orchestra.run.smithi118:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-11-24T06:06:51.228 INFO:teuthology.orchestra.run.smithi118.stdout:check_obsoletes = 1 2023-11-24T06:06:51.232 DEBUG:teuthology.orchestra.run.smithi118:> sudo yum clean all 2023-11-24T06:06:51.269 DEBUG:teuthology.orchestra.run.smithi092:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-11-24T06:06:51.356 INFO:teuthology.orchestra.run.smithi092.stdout:check_obsoletes = 1 2023-11-24T06:06:51.361 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum clean all 2023-11-24T06:06:51.595 INFO:teuthology.orchestra.run.smithi118.stdout:73 files removed 2023-11-24T06:06:51.633 DEBUG:teuthology.orchestra.run.smithi118:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-11-24T06:06:51.724 INFO:teuthology.orchestra.run.smithi092.stdout:73 files removed 2023-11-24T06:06:51.766 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-11-24T06:06:52.538 INFO:teuthology.orchestra.run.smithi118.stdout:ceph packages for x86_64 193 kB/s | 78 kB 00:00 2023-11-24T06:06:52.838 INFO:teuthology.orchestra.run.smithi092.stdout:ceph packages for x86_64 138 kB/s | 78 kB 00:00 2023-11-24T06:06:53.413 INFO:teuthology.orchestra.run.smithi118.stdout:ceph noarch packages 12 kB/s | 9.5 kB 00:00 2023-11-24T06:06:53.697 INFO:teuthology.orchestra.run.smithi092.stdout:ceph noarch packages 12 kB/s | 9.5 kB 00:00 2023-11-24T06:06:54.030 INFO:teuthology.orchestra.run.smithi118.stdout:ceph source packages 3.5 kB/s | 1.9 kB 00:00 2023-11-24T06:06:54.523 INFO:teuthology.orchestra.run.smithi092.stdout:ceph source packages 2.5 kB/s | 1.9 kB 00:00 2023-11-24T06:06:54.717 INFO:teuthology.orchestra.run.smithi118.stdout:CentOS-9 - AppStream 10 MB/s | 6.4 MB 00:00 2023-11-24T06:06:55.526 INFO:teuthology.orchestra.run.smithi092.stdout:CentOS-9 - AppStream 6.9 MB/s | 6.4 MB 00:00 2023-11-24T06:06:57.276 INFO:teuthology.orchestra.run.smithi118.stdout:CentOS-9 - BaseOS 1.6 MB/s | 2.0 MB 00:01 2023-11-24T06:06:57.327 INFO:teuthology.orchestra.run.smithi092.stdout:CentOS-9 - BaseOS 3.6 MB/s | 2.0 MB 00:00 2023-11-24T06:06:58.823 INFO:teuthology.orchestra.run.smithi092.stdout:CentOS Stream 9 - BaseOS 7.2 MB/s | 7.9 MB 00:01 2023-11-24T06:07:01.419 INFO:teuthology.orchestra.run.smithi118.stdout:CentOS Stream 9 - BaseOS 2.1 MB/s | 7.9 MB 00:03 2023-11-24T06:07:06.972 INFO:teuthology.orchestra.run.smithi118.stdout:CentOS Stream 9 - AppStream 4.2 MB/s | 18 MB 00:04 2023-11-24T06:07:08.823 INFO:teuthology.orchestra.run.smithi092.stdout:CentOS Stream 9 - AppStream 2.1 MB/s | 18 MB 00:08 2023-11-24T06:07:11.548 INFO:teuthology.orchestra.run.smithi118.stdout:CentOS Stream 9 - CRB 7.2 MB/s | 5.8 MB 00:00 2023-11-24T06:07:14.549 INFO:teuthology.orchestra.run.smithi118.stdout:CentOS Stream 9 - Extras packages 8.9 kB/s | 15 kB 00:01 2023-11-24T06:07:16.959 INFO:teuthology.orchestra.run.smithi118.stdout:Extra Packages for Enterprise Linux 8.9 MB/s | 20 MB 00:02 2023-11-24T06:07:21.716 INFO:teuthology.orchestra.run.smithi092.stdout:CentOS Stream 9 - CRB 645 kB/s | 5.8 MB 00:09 2023-11-24T06:07:22.575 INFO:teuthology.orchestra.run.smithi118.stdout:lab-extras 28 kB/s | 1.7 kB 00:00 2023-11-24T06:07:23.301 INFO:teuthology.orchestra.run.smithi092.stdout:CentOS Stream 9 - Extras packages 53 kB/s | 15 kB 00:00 2023-11-24T06:07:25.388 INFO:teuthology.orchestra.run.smithi118.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2023-11-24T06:07:25.389 INFO:teuthology.orchestra.run.smithi118.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2023-11-24T06:07:25.409 INFO:teuthology.orchestra.run.smithi118.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2023-11-24T06:07:25.411 INFO:teuthology.orchestra.run.smithi118.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2023-11-24T06:07:25.412 INFO:teuthology.orchestra.run.smithi118.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2023-11-24T06:07:25.413 INFO:teuthology.orchestra.run.smithi118.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2023-11-24T06:07:25.416 INFO:teuthology.orchestra.run.smithi118.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2023-11-24T06:07:25.420 INFO:teuthology.orchestra.run.smithi118.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2023-11-24T06:07:25.421 INFO:teuthology.orchestra.run.smithi118.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2023-11-24T06:07:25.499 INFO:teuthology.orchestra.run.smithi118.stdout:Dependencies resolved. 2023-11-24T06:07:25.518 INFO:teuthology.orchestra.run.smithi118.stdout:============================================================================================ 2023-11-24T06:07:25.518 INFO:teuthology.orchestra.run.smithi118.stdout: Package Arch Version Repository Size 2023-11-24T06:07:25.518 INFO:teuthology.orchestra.run.smithi118.stdout:============================================================================================ 2023-11-24T06:07:25.518 INFO:teuthology.orchestra.run.smithi118.stdout:Installing: 2023-11-24T06:07:25.518 INFO:teuthology.orchestra.run.smithi118.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2023-11-24T06:07:25.518 INFO:teuthology.orchestra.run.smithi118.stdout: ceph x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 6.5 k 2023-11-24T06:07:25.518 INFO:teuthology.orchestra.run.smithi118.stdout: ceph-base x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 5.2 M 2023-11-24T06:07:25.518 INFO:teuthology.orchestra.run.smithi118.stdout: ceph-fuse x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 859 k 2023-11-24T06:07:25.519 INFO:teuthology.orchestra.run.smithi118.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 141 k 2023-11-24T06:07:25.519 INFO:teuthology.orchestra.run.smithi118.stdout: ceph-mgr x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 1.5 M 2023-11-24T06:07:25.519 INFO:teuthology.orchestra.run.smithi118.stdout: ceph-mgr-cephadm noarch 2:18.0.0-7484.ga13ea5ff.el9 ceph-noarch 133 k 2023-11-24T06:07:25.519 INFO:teuthology.orchestra.run.smithi118.stdout: ceph-mgr-dashboard noarch 2:18.0.0-7484.ga13ea5ff.el9 ceph-noarch 1.8 M 2023-11-24T06:07:25.519 INFO:teuthology.orchestra.run.smithi118.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-7484.ga13ea5ff.el9 ceph-noarch 7.4 M 2023-11-24T06:07:25.519 INFO:teuthology.orchestra.run.smithi118.stdout: ceph-mgr-rook noarch 2:18.0.0-7484.ga13ea5ff.el9 ceph-noarch 50 k 2023-11-24T06:07:25.519 INFO:teuthology.orchestra.run.smithi118.stdout: ceph-radosgw x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 8.6 M 2023-11-24T06:07:25.519 INFO:teuthology.orchestra.run.smithi118.stdout: ceph-test x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 41 M 2023-11-24T06:07:25.519 INFO:teuthology.orchestra.run.smithi118.stdout: ceph-volume noarch 2:18.0.0-7484.ga13ea5ff.el9 ceph-noarch 262 k 2023-11-24T06:07:25.519 INFO:teuthology.orchestra.run.smithi118.stdout: cephadm noarch 2:18.0.0-7484.ga13ea5ff.el9 ceph-noarch 772 k 2023-11-24T06:07:25.519 INFO:teuthology.orchestra.run.smithi118.stdout: cephfs-mirror x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 217 k 2023-11-24T06:07:25.519 INFO:teuthology.orchestra.run.smithi118.stdout: cephfs-top noarch 2:18.0.0-7484.ga13ea5ff.el9 ceph-noarch 18 k 2023-11-24T06:07:25.520 INFO:teuthology.orchestra.run.smithi118.stdout: elfutils-libelf-devel x86_64 0.190-1.el9 CentOS-AppStream 21 k 2023-11-24T06:07:25.520 INFO:teuthology.orchestra.run.smithi118.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2023-11-24T06:07:25.520 INFO:teuthology.orchestra.run.smithi118.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2023-11-24T06:07:25.520 INFO:teuthology.orchestra.run.smithi118.stdout: libcephfs-devel x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 31 k 2023-11-24T06:07:25.520 INFO:teuthology.orchestra.run.smithi118.stdout: libcephfs2 x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 719 k 2023-11-24T06:07:25.520 INFO:teuthology.orchestra.run.smithi118.stdout: librados-devel x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 126 k 2023-11-24T06:07:25.520 INFO:teuthology.orchestra.run.smithi118.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2023-11-24T06:07:25.520 INFO:teuthology.orchestra.run.smithi118.stdout: libuuid-devel x86_64 2.37.4-15.el9 CentOS-AppStream 24 k 2023-11-24T06:07:25.520 INFO:teuthology.orchestra.run.smithi118.stdout: openssl-devel x86_64 1:3.0.7-25.el9 appstream 4.1 M 2023-11-24T06:07:25.520 INFO:teuthology.orchestra.run.smithi118.stdout: postgresql x86_64 13.11-1.el9 CentOS-AppStream 1.5 M 2023-11-24T06:07:25.520 INFO:teuthology.orchestra.run.smithi118.stdout: postgresql-contrib x86_64 13.11-1.el9 CentOS-AppStream 811 k 2023-11-24T06:07:25.520 INFO:teuthology.orchestra.run.smithi118.stdout: postgresql-server x86_64 13.11-1.el9 CentOS-AppStream 5.7 M 2023-11-24T06:07:25.521 INFO:teuthology.orchestra.run.smithi118.stdout: python3-cephfs x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 162 k 2023-11-24T06:07:25.521 INFO:teuthology.orchestra.run.smithi118.stdout: python3-rados x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 319 k 2023-11-24T06:07:25.521 INFO:teuthology.orchestra.run.smithi118.stdout: python3-rbd x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 298 k 2023-11-24T06:07:25.521 INFO:teuthology.orchestra.run.smithi118.stdout: python3-rgw x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 99 k 2023-11-24T06:07:25.521 INFO:teuthology.orchestra.run.smithi118.stdout: rbd-fuse x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 85 k 2023-11-24T06:07:25.521 INFO:teuthology.orchestra.run.smithi118.stdout: rbd-mirror x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 3.0 M 2023-11-24T06:07:25.521 INFO:teuthology.orchestra.run.smithi118.stdout: rbd-nbd x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 170 k 2023-11-24T06:07:25.521 INFO:teuthology.orchestra.run.smithi118.stdout: xfsprogs-devel x86_64 5.19.0-4.el9 CentOS-AppStream 85 k 2023-11-24T06:07:25.521 INFO:teuthology.orchestra.run.smithi118.stdout:Upgrading: 2023-11-24T06:07:25.521 INFO:teuthology.orchestra.run.smithi118.stdout: NetworkManager x86_64 1:1.45.6-1.el9 CentOS-BaseOS 2.2 M 2023-11-24T06:07:25.521 INFO:teuthology.orchestra.run.smithi118.stdout: NetworkManager-libnm x86_64 1:1.45.6-1.el9 CentOS-BaseOS 1.8 M 2023-11-24T06:07:25.521 INFO:teuthology.orchestra.run.smithi118.stdout: NetworkManager-team x86_64 1:1.45.6-1.el9 CentOS-BaseOS 42 k 2023-11-24T06:07:25.522 INFO:teuthology.orchestra.run.smithi118.stdout: NetworkManager-tui x86_64 1:1.45.6-1.el9 CentOS-BaseOS 245 k 2023-11-24T06:07:25.522 INFO:teuthology.orchestra.run.smithi118.stdout: acl x86_64 2.3.1-4.el9 CentOS-BaseOS 69 k 2023-11-24T06:07:25.522 INFO:teuthology.orchestra.run.smithi118.stdout: elfutils-debuginfod-client x86_64 0.190-1.el9 CentOS-BaseOS 36 k 2023-11-24T06:07:25.522 INFO:teuthology.orchestra.run.smithi118.stdout: elfutils-libelf x86_64 0.190-1.el9 CentOS-BaseOS 192 k 2023-11-24T06:07:25.522 INFO:teuthology.orchestra.run.smithi118.stdout: elfutils-libs x86_64 0.190-1.el9 CentOS-BaseOS 255 k 2023-11-24T06:07:25.522 INFO:teuthology.orchestra.run.smithi118.stdout: libacl x86_64 2.3.1-4.el9 CentOS-BaseOS 22 k 2023-11-24T06:07:25.522 INFO:teuthology.orchestra.run.smithi118.stdout: libacl-devel x86_64 2.3.1-4.el9 CentOS-AppStream 68 k 2023-11-24T06:07:25.522 INFO:teuthology.orchestra.run.smithi118.stdout: librados2 x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 3.3 M 2023-11-24T06:07:25.522 INFO:teuthology.orchestra.run.smithi118.stdout: librbd1 x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 3.0 M 2023-11-24T06:07:25.522 INFO:teuthology.orchestra.run.smithi118.stdout: openssl x86_64 1:3.0.7-25.el9 baseos 1.2 M 2023-11-24T06:07:25.523 INFO:teuthology.orchestra.run.smithi118.stdout: openssl-libs x86_64 1:3.0.7-25.el9 baseos 2.2 M 2023-11-24T06:07:25.523 INFO:teuthology.orchestra.run.smithi118.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2023-11-24T06:07:25.523 INFO:teuthology.orchestra.run.smithi118.stdout:Installing dependencies: 2023-11-24T06:07:25.523 INFO:teuthology.orchestra.run.smithi118.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2023-11-24T06:07:25.523 INFO:teuthology.orchestra.run.smithi118.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2023-11-24T06:07:25.523 INFO:teuthology.orchestra.run.smithi118.stdout: ceph-common x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 19 M 2023-11-24T06:07:25.523 INFO:teuthology.orchestra.run.smithi118.stdout: ceph-grafana-dashboards noarch 2:18.0.0-7484.ga13ea5ff.el9 ceph-noarch 28 k 2023-11-24T06:07:25.523 INFO:teuthology.orchestra.run.smithi118.stdout: ceph-mds x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 2.1 M 2023-11-24T06:07:25.523 INFO:teuthology.orchestra.run.smithi118.stdout: ceph-mgr-modules-core noarch 2:18.0.0-7484.ga13ea5ff.el9 ceph-noarch 243 k 2023-11-24T06:07:25.523 INFO:teuthology.orchestra.run.smithi118.stdout: ceph-mon x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 4.4 M 2023-11-24T06:07:25.523 INFO:teuthology.orchestra.run.smithi118.stdout: ceph-osd x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 18 M 2023-11-24T06:07:25.524 INFO:teuthology.orchestra.run.smithi118.stdout: ceph-prometheus-alerts noarch 2:18.0.0-7484.ga13ea5ff.el9 ceph-noarch 15 k 2023-11-24T06:07:25.524 INFO:teuthology.orchestra.run.smithi118.stdout: ceph-selinux x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 24 k 2023-11-24T06:07:25.524 INFO:teuthology.orchestra.run.smithi118.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2023-11-24T06:07:25.524 INFO:teuthology.orchestra.run.smithi118.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2023-11-24T06:07:25.524 INFO:teuthology.orchestra.run.smithi118.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2023-11-24T06:07:25.524 INFO:teuthology.orchestra.run.smithi118.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2023-11-24T06:07:25.524 INFO:teuthology.orchestra.run.smithi118.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2023-11-24T06:07:25.524 INFO:teuthology.orchestra.run.smithi118.stdout: jq x86_64 1.6-16.el9 CentOS-AppStream 186 k 2023-11-24T06:07:25.524 INFO:teuthology.orchestra.run.smithi118.stdout: libarrow x86_64 9.0.0-9.el9 epel 4.4 M 2023-11-24T06:07:25.524 INFO:teuthology.orchestra.run.smithi118.stdout: libarrow-doc noarch 9.0.0-9.el9 epel 24 k 2023-11-24T06:07:25.524 INFO:teuthology.orchestra.run.smithi118.stdout: libcephsqlite x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 163 k 2023-11-24T06:07:25.525 INFO:teuthology.orchestra.run.smithi118.stdout: libgfortran x86_64 11.4.1-2.3.el9 CentOS-BaseOS 803 k 2023-11-24T06:07:25.525 INFO:teuthology.orchestra.run.smithi118.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2023-11-24T06:07:25.525 INFO:teuthology.orchestra.run.smithi118.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2023-11-24T06:07:25.525 INFO:teuthology.orchestra.run.smithi118.stdout: libquadmath x86_64 11.4.1-2.3.el9 CentOS-BaseOS 195 k 2023-11-24T06:07:25.525 INFO:teuthology.orchestra.run.smithi118.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2023-11-24T06:07:25.525 INFO:teuthology.orchestra.run.smithi118.stdout: libradosstriper1 x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 476 k 2023-11-24T06:07:25.525 INFO:teuthology.orchestra.run.smithi118.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2023-11-24T06:07:25.525 INFO:teuthology.orchestra.run.smithi118.stdout: librgw2 x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 4.8 M 2023-11-24T06:07:25.525 INFO:teuthology.orchestra.run.smithi118.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2023-11-24T06:07:25.525 INFO:teuthology.orchestra.run.smithi118.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2023-11-24T06:07:25.525 INFO:teuthology.orchestra.run.smithi118.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2023-11-24T06:07:25.526 INFO:teuthology.orchestra.run.smithi118.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2023-11-24T06:07:25.526 INFO:teuthology.orchestra.run.smithi118.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2023-11-24T06:07:25.526 INFO:teuthology.orchestra.run.smithi118.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2023-11-24T06:07:25.526 INFO:teuthology.orchestra.run.smithi118.stdout: parquet-libs x86_64 9.0.0-9.el9 epel 837 k 2023-11-24T06:07:25.526 INFO:teuthology.orchestra.run.smithi118.stdout: postgresql-private-libs x86_64 13.11-1.el9 CentOS-AppStream 134 k 2023-11-24T06:07:25.526 INFO:teuthology.orchestra.run.smithi118.stdout: python3-asyncssh noarch 2.13.2-3.el9 epel 548 k 2023-11-24T06:07:25.526 INFO:teuthology.orchestra.run.smithi118.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2023-11-24T06:07:25.526 INFO:teuthology.orchestra.run.smithi118.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2023-11-24T06:07:25.526 INFO:teuthology.orchestra.run.smithi118.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2023-11-24T06:07:25.526 INFO:teuthology.orchestra.run.smithi118.stdout: python3-ceph-argparse x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 45 k 2023-11-24T06:07:25.526 INFO:teuthology.orchestra.run.smithi118.stdout: python3-ceph-common x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 129 k 2023-11-24T06:07:25.526 INFO:teuthology.orchestra.run.smithi118.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2023-11-24T06:07:25.527 INFO:teuthology.orchestra.run.smithi118.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2023-11-24T06:07:25.527 INFO:teuthology.orchestra.run.smithi118.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2023-11-24T06:07:25.527 INFO:teuthology.orchestra.run.smithi118.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2023-11-24T06:07:25.527 INFO:teuthology.orchestra.run.smithi118.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2023-11-24T06:07:25.527 INFO:teuthology.orchestra.run.smithi118.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2023-11-24T06:07:25.527 INFO:teuthology.orchestra.run.smithi118.stdout: python3-google-auth noarch 1:2.23.4-1.el9 epel 197 k 2023-11-24T06:07:25.527 INFO:teuthology.orchestra.run.smithi118.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2023-11-24T06:07:25.527 INFO:teuthology.orchestra.run.smithi118.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2023-11-24T06:07:25.527 INFO:teuthology.orchestra.run.smithi118.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2023-11-24T06:07:25.527 INFO:teuthology.orchestra.run.smithi118.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2023-11-24T06:07:25.527 INFO:teuthology.orchestra.run.smithi118.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2023-11-24T06:07:25.528 INFO:teuthology.orchestra.run.smithi118.stdout: python3-jinja2 noarch 2.11.3-4.el9 CentOS-AppStream 228 k 2023-11-24T06:07:25.528 INFO:teuthology.orchestra.run.smithi118.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2023-11-24T06:07:25.528 INFO:teuthology.orchestra.run.smithi118.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2023-11-24T06:07:25.528 INFO:teuthology.orchestra.run.smithi118.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2023-11-24T06:07:25.528 INFO:teuthology.orchestra.run.smithi118.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2023-11-24T06:07:25.528 INFO:teuthology.orchestra.run.smithi118.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2023-11-24T06:07:25.528 INFO:teuthology.orchestra.run.smithi118.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2023-11-24T06:07:25.528 INFO:teuthology.orchestra.run.smithi118.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2023-11-24T06:07:25.528 INFO:teuthology.orchestra.run.smithi118.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2023-11-24T06:07:25.528 INFO:teuthology.orchestra.run.smithi118.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2023-11-24T06:07:25.528 INFO:teuthology.orchestra.run.smithi118.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2023-11-24T06:07:25.529 INFO:teuthology.orchestra.run.smithi118.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2023-11-24T06:07:25.529 INFO:teuthology.orchestra.run.smithi118.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2023-11-24T06:07:25.529 INFO:teuthology.orchestra.run.smithi118.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2023-11-24T06:07:25.529 INFO:teuthology.orchestra.run.smithi118.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2023-11-24T06:07:25.529 INFO:teuthology.orchestra.run.smithi118.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2023-11-24T06:07:25.529 INFO:teuthology.orchestra.run.smithi118.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2023-11-24T06:07:25.529 INFO:teuthology.orchestra.run.smithi118.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2023-11-24T06:07:25.529 INFO:teuthology.orchestra.run.smithi118.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2023-11-24T06:07:25.529 INFO:teuthology.orchestra.run.smithi118.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2023-11-24T06:07:25.529 INFO:teuthology.orchestra.run.smithi118.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2023-11-24T06:07:25.529 INFO:teuthology.orchestra.run.smithi118.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2023-11-24T06:07:25.529 INFO:teuthology.orchestra.run.smithi118.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2023-11-24T06:07:25.530 INFO:teuthology.orchestra.run.smithi118.stdout: python3-requests noarch 2.25.1-7.el9 CentOS-BaseOS 113 k 2023-11-24T06:07:25.530 INFO:teuthology.orchestra.run.smithi118.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2023-11-24T06:07:25.530 INFO:teuthology.orchestra.run.smithi118.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2023-11-24T06:07:25.530 INFO:teuthology.orchestra.run.smithi118.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2023-11-24T06:07:25.530 INFO:teuthology.orchestra.run.smithi118.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2023-11-24T06:07:25.530 INFO:teuthology.orchestra.run.smithi118.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2023-11-24T06:07:25.530 INFO:teuthology.orchestra.run.smithi118.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2023-11-24T06:07:25.530 INFO:teuthology.orchestra.run.smithi118.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2023-11-24T06:07:25.530 INFO:teuthology.orchestra.run.smithi118.stdout: python3-urllib3 noarch 1.26.5-4.el9 CentOS-BaseOS 187 k 2023-11-24T06:07:25.530 INFO:teuthology.orchestra.run.smithi118.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2023-11-24T06:07:25.530 INFO:teuthology.orchestra.run.smithi118.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2023-11-24T06:07:25.531 INFO:teuthology.orchestra.run.smithi118.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2023-11-24T06:07:25.531 INFO:teuthology.orchestra.run.smithi118.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2023-11-24T06:07:25.531 INFO:teuthology.orchestra.run.smithi118.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2023-11-24T06:07:25.531 INFO:teuthology.orchestra.run.smithi118.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2023-11-24T06:07:25.531 INFO:teuthology.orchestra.run.smithi118.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2023-11-24T06:07:25.531 INFO:teuthology.orchestra.run.smithi118.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2023-11-24T06:07:25.531 INFO:teuthology.orchestra.run.smithi118.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2023-11-24T06:07:25.531 INFO:teuthology.orchestra.run.smithi118.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2023-11-24T06:07:25.531 INFO:teuthology.orchestra.run.smithi118.stdout:Installing weak dependencies: 2023-11-24T06:07:25.531 INFO:teuthology.orchestra.run.smithi118.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2023-11-24T06:07:25.531 INFO:teuthology.orchestra.run.smithi118.stdout: 2023-11-24T06:07:25.531 INFO:teuthology.orchestra.run.smithi118.stdout:Transaction Summary 2023-11-24T06:07:25.532 INFO:teuthology.orchestra.run.smithi118.stdout:============================================================================================ 2023-11-24T06:07:25.532 INFO:teuthology.orchestra.run.smithi118.stdout:Install 132 Packages 2023-11-24T06:07:25.532 INFO:teuthology.orchestra.run.smithi118.stdout:Upgrade 15 Packages 2023-11-24T06:07:25.532 INFO:teuthology.orchestra.run.smithi118.stdout: 2023-11-24T06:07:25.532 INFO:teuthology.orchestra.run.smithi118.stdout:Total download size: 204 M 2023-11-24T06:07:25.532 INFO:teuthology.orchestra.run.smithi118.stdout:Downloading Packages: 2023-11-24T06:07:25.799 INFO:teuthology.orchestra.run.smithi092.stdout:Extra Packages for Enterprise Linux 8.6 MB/s | 20 MB 00:02 2023-11-24T06:07:26.702 INFO:teuthology.orchestra.run.smithi118.stdout:(1/147): ceph-18.0.0-7484.ga13ea5ff.el9.x86_64. 12 kB/s | 6.5 kB 00:00 2023-11-24T06:07:29.787 INFO:teuthology.orchestra.run.smithi118.stdout:(2/147): ceph-fuse-18.0.0-7484.ga13ea5ff.el9.x8 279 kB/s | 859 kB 00:03 2023-11-24T06:07:30.127 INFO:teuthology.orchestra.run.smithi118.stdout:(3/147): ceph-immutable-object-cache-18.0.0-748 413 kB/s | 141 kB 00:00 2023-11-24T06:07:30.461 INFO:teuthology.orchestra.run.smithi118.stdout:(4/147): ceph-base-18.0.0-7484.ga13ea5ff.el9.x8 1.2 MB/s | 5.2 MB 00:04 2023-11-24T06:07:30.869 INFO:teuthology.orchestra.run.smithi118.stdout:(5/147): ceph-mgr-18.0.0-7484.ga13ea5ff.el9.x86 3.6 MB/s | 1.5 MB 00:00 2023-11-24T06:07:31.377 INFO:teuthology.orchestra.run.smithi092.stdout:lab-extras 18 kB/s | 1.7 kB 00:00 2023-11-24T06:07:31.686 INFO:teuthology.orchestra.run.smithi118.stdout:(6/147): ceph-mon-18.0.0-7484.ga13ea5ff.el9.x86 5.4 MB/s | 4.4 MB 00:00 2023-11-24T06:07:33.270 INFO:teuthology.orchestra.run.smithi118.stdout:(7/147): ceph-osd-18.0.0-7484.ga13ea5ff.el9.x86 11 MB/s | 18 MB 00:01 2023-11-24T06:07:34.054 INFO:teuthology.orchestra.run.smithi118.stdout:(8/147): ceph-radosgw-18.0.0-7484.ga13ea5ff.el9 11 MB/s | 8.6 MB 00:00 2023-11-24T06:07:34.170 INFO:teuthology.orchestra.run.smithi118.stdout:(9/147): ceph-selinux-18.0.0-7484.ga13ea5ff.el9 207 kB/s | 24 kB 00:00 2023-11-24T06:07:34.255 INFO:teuthology.orchestra.run.smithi092.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2023-11-24T06:07:34.257 INFO:teuthology.orchestra.run.smithi092.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2023-11-24T06:07:34.276 INFO:teuthology.orchestra.run.smithi092.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2023-11-24T06:07:34.278 INFO:teuthology.orchestra.run.smithi092.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2023-11-24T06:07:34.279 INFO:teuthology.orchestra.run.smithi092.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2023-11-24T06:07:34.280 INFO:teuthology.orchestra.run.smithi092.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2023-11-24T06:07:34.283 INFO:teuthology.orchestra.run.smithi092.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2023-11-24T06:07:34.286 INFO:teuthology.orchestra.run.smithi092.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2023-11-24T06:07:34.288 INFO:teuthology.orchestra.run.smithi092.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2023-11-24T06:07:34.368 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2023-11-24T06:07:34.396 INFO:teuthology.orchestra.run.smithi092.stdout:============================================================================================ 2023-11-24T06:07:34.396 INFO:teuthology.orchestra.run.smithi092.stdout: Package Arch Version Repository Size 2023-11-24T06:07:34.396 INFO:teuthology.orchestra.run.smithi092.stdout:============================================================================================ 2023-11-24T06:07:34.396 INFO:teuthology.orchestra.run.smithi092.stdout:Installing: 2023-11-24T06:07:34.397 INFO:teuthology.orchestra.run.smithi092.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2023-11-24T06:07:34.397 INFO:teuthology.orchestra.run.smithi092.stdout: ceph x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 6.5 k 2023-11-24T06:07:34.397 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-base x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 5.2 M 2023-11-24T06:07:34.397 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-fuse x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 859 k 2023-11-24T06:07:34.397 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 141 k 2023-11-24T06:07:34.397 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 1.5 M 2023-11-24T06:07:34.397 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-cephadm noarch 2:18.0.0-7484.ga13ea5ff.el9 ceph-noarch 133 k 2023-11-24T06:07:34.397 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-dashboard noarch 2:18.0.0-7484.ga13ea5ff.el9 ceph-noarch 1.8 M 2023-11-24T06:07:34.397 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-7484.ga13ea5ff.el9 ceph-noarch 7.4 M 2023-11-24T06:07:34.397 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-rook noarch 2:18.0.0-7484.ga13ea5ff.el9 ceph-noarch 50 k 2023-11-24T06:07:34.397 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-radosgw x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 8.6 M 2023-11-24T06:07:34.398 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-test x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 41 M 2023-11-24T06:07:34.398 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-volume noarch 2:18.0.0-7484.ga13ea5ff.el9 ceph-noarch 262 k 2023-11-24T06:07:34.398 INFO:teuthology.orchestra.run.smithi092.stdout: cephadm noarch 2:18.0.0-7484.ga13ea5ff.el9 ceph-noarch 772 k 2023-11-24T06:07:34.398 INFO:teuthology.orchestra.run.smithi092.stdout: cephfs-mirror x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 217 k 2023-11-24T06:07:34.398 INFO:teuthology.orchestra.run.smithi092.stdout: cephfs-top noarch 2:18.0.0-7484.ga13ea5ff.el9 ceph-noarch 18 k 2023-11-24T06:07:34.398 INFO:teuthology.orchestra.run.smithi092.stdout: elfutils-libelf-devel x86_64 0.190-1.el9 CentOS-AppStream 21 k 2023-11-24T06:07:34.398 INFO:teuthology.orchestra.run.smithi092.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2023-11-24T06:07:34.398 INFO:teuthology.orchestra.run.smithi092.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2023-11-24T06:07:34.398 INFO:teuthology.orchestra.run.smithi092.stdout: libcephfs-devel x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 31 k 2023-11-24T06:07:34.398 INFO:teuthology.orchestra.run.smithi092.stdout: libcephfs2 x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 719 k 2023-11-24T06:07:34.399 INFO:teuthology.orchestra.run.smithi092.stdout: librados-devel x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 126 k 2023-11-24T06:07:34.399 INFO:teuthology.orchestra.run.smithi092.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2023-11-24T06:07:34.399 INFO:teuthology.orchestra.run.smithi092.stdout: libuuid-devel x86_64 2.37.4-15.el9 CentOS-AppStream 24 k 2023-11-24T06:07:34.399 INFO:teuthology.orchestra.run.smithi092.stdout: openssl-devel x86_64 1:3.0.7-25.el9 appstream 4.1 M 2023-11-24T06:07:34.399 INFO:teuthology.orchestra.run.smithi092.stdout: postgresql x86_64 13.11-1.el9 CentOS-AppStream 1.5 M 2023-11-24T06:07:34.399 INFO:teuthology.orchestra.run.smithi092.stdout: postgresql-contrib x86_64 13.11-1.el9 CentOS-AppStream 811 k 2023-11-24T06:07:34.399 INFO:teuthology.orchestra.run.smithi092.stdout: postgresql-server x86_64 13.11-1.el9 CentOS-AppStream 5.7 M 2023-11-24T06:07:34.399 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cephfs x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 162 k 2023-11-24T06:07:34.399 INFO:teuthology.orchestra.run.smithi092.stdout: python3-rados x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 319 k 2023-11-24T06:07:34.399 INFO:teuthology.orchestra.run.smithi092.stdout: python3-rbd x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 298 k 2023-11-24T06:07:34.399 INFO:teuthology.orchestra.run.smithi092.stdout: python3-rgw x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 99 k 2023-11-24T06:07:34.400 INFO:teuthology.orchestra.run.smithi092.stdout: rbd-fuse x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 85 k 2023-11-24T06:07:34.400 INFO:teuthology.orchestra.run.smithi092.stdout: rbd-mirror x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 3.0 M 2023-11-24T06:07:34.400 INFO:teuthology.orchestra.run.smithi092.stdout: rbd-nbd x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 170 k 2023-11-24T06:07:34.400 INFO:teuthology.orchestra.run.smithi092.stdout: xfsprogs-devel x86_64 5.19.0-4.el9 CentOS-AppStream 85 k 2023-11-24T06:07:34.400 INFO:teuthology.orchestra.run.smithi092.stdout:Upgrading: 2023-11-24T06:07:34.400 INFO:teuthology.orchestra.run.smithi092.stdout: NetworkManager x86_64 1:1.45.6-1.el9 CentOS-BaseOS 2.2 M 2023-11-24T06:07:34.400 INFO:teuthology.orchestra.run.smithi092.stdout: NetworkManager-libnm x86_64 1:1.45.6-1.el9 CentOS-BaseOS 1.8 M 2023-11-24T06:07:34.400 INFO:teuthology.orchestra.run.smithi092.stdout: NetworkManager-team x86_64 1:1.45.6-1.el9 CentOS-BaseOS 42 k 2023-11-24T06:07:34.400 INFO:teuthology.orchestra.run.smithi092.stdout: NetworkManager-tui x86_64 1:1.45.6-1.el9 CentOS-BaseOS 245 k 2023-11-24T06:07:34.400 INFO:teuthology.orchestra.run.smithi092.stdout: acl x86_64 2.3.1-4.el9 CentOS-BaseOS 69 k 2023-11-24T06:07:34.401 INFO:teuthology.orchestra.run.smithi092.stdout: elfutils-debuginfod-client x86_64 0.190-1.el9 CentOS-BaseOS 36 k 2023-11-24T06:07:34.401 INFO:teuthology.orchestra.run.smithi092.stdout: elfutils-libelf x86_64 0.190-1.el9 CentOS-BaseOS 192 k 2023-11-24T06:07:34.401 INFO:teuthology.orchestra.run.smithi092.stdout: elfutils-libs x86_64 0.190-1.el9 CentOS-BaseOS 255 k 2023-11-24T06:07:34.401 INFO:teuthology.orchestra.run.smithi092.stdout: libacl x86_64 2.3.1-4.el9 CentOS-BaseOS 22 k 2023-11-24T06:07:34.401 INFO:teuthology.orchestra.run.smithi092.stdout: libacl-devel x86_64 2.3.1-4.el9 CentOS-AppStream 68 k 2023-11-24T06:07:34.401 INFO:teuthology.orchestra.run.smithi092.stdout: librados2 x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 3.3 M 2023-11-24T06:07:34.401 INFO:teuthology.orchestra.run.smithi092.stdout: librbd1 x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 3.0 M 2023-11-24T06:07:34.401 INFO:teuthology.orchestra.run.smithi092.stdout: openssl x86_64 1:3.0.7-25.el9 baseos 1.2 M 2023-11-24T06:07:34.401 INFO:teuthology.orchestra.run.smithi092.stdout: openssl-libs x86_64 1:3.0.7-25.el9 baseos 2.2 M 2023-11-24T06:07:34.401 INFO:teuthology.orchestra.run.smithi092.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2023-11-24T06:07:34.401 INFO:teuthology.orchestra.run.smithi092.stdout:Installing dependencies: 2023-11-24T06:07:34.402 INFO:teuthology.orchestra.run.smithi092.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2023-11-24T06:07:34.402 INFO:teuthology.orchestra.run.smithi092.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2023-11-24T06:07:34.402 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-common x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 19 M 2023-11-24T06:07:34.402 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-grafana-dashboards noarch 2:18.0.0-7484.ga13ea5ff.el9 ceph-noarch 28 k 2023-11-24T06:07:34.402 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mds x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 2.1 M 2023-11-24T06:07:34.402 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-modules-core noarch 2:18.0.0-7484.ga13ea5ff.el9 ceph-noarch 243 k 2023-11-24T06:07:34.402 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mon x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 4.4 M 2023-11-24T06:07:34.402 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-osd x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 18 M 2023-11-24T06:07:34.402 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-prometheus-alerts noarch 2:18.0.0-7484.ga13ea5ff.el9 ceph-noarch 15 k 2023-11-24T06:07:34.403 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-selinux x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 24 k 2023-11-24T06:07:34.403 INFO:teuthology.orchestra.run.smithi092.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2023-11-24T06:07:34.403 INFO:teuthology.orchestra.run.smithi092.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2023-11-24T06:07:34.403 INFO:teuthology.orchestra.run.smithi092.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2023-11-24T06:07:34.403 INFO:teuthology.orchestra.run.smithi092.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2023-11-24T06:07:34.403 INFO:teuthology.orchestra.run.smithi092.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2023-11-24T06:07:34.403 INFO:teuthology.orchestra.run.smithi092.stdout: jq x86_64 1.6-16.el9 CentOS-AppStream 186 k 2023-11-24T06:07:34.403 INFO:teuthology.orchestra.run.smithi092.stdout: libarrow x86_64 9.0.0-9.el9 epel 4.4 M 2023-11-24T06:07:34.403 INFO:teuthology.orchestra.run.smithi092.stdout: libarrow-doc noarch 9.0.0-9.el9 epel 24 k 2023-11-24T06:07:34.403 INFO:teuthology.orchestra.run.smithi092.stdout: libcephsqlite x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 163 k 2023-11-24T06:07:34.403 INFO:teuthology.orchestra.run.smithi092.stdout: libgfortran x86_64 11.4.1-2.3.el9 CentOS-BaseOS 803 k 2023-11-24T06:07:34.404 INFO:teuthology.orchestra.run.smithi092.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2023-11-24T06:07:34.404 INFO:teuthology.orchestra.run.smithi092.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2023-11-24T06:07:34.404 INFO:teuthology.orchestra.run.smithi092.stdout: libquadmath x86_64 11.4.1-2.3.el9 CentOS-BaseOS 195 k 2023-11-24T06:07:34.404 INFO:teuthology.orchestra.run.smithi092.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2023-11-24T06:07:34.404 INFO:teuthology.orchestra.run.smithi092.stdout: libradosstriper1 x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 476 k 2023-11-24T06:07:34.404 INFO:teuthology.orchestra.run.smithi092.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2023-11-24T06:07:34.404 INFO:teuthology.orchestra.run.smithi092.stdout: librgw2 x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 4.8 M 2023-11-24T06:07:34.404 INFO:teuthology.orchestra.run.smithi092.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2023-11-24T06:07:34.404 INFO:teuthology.orchestra.run.smithi092.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2023-11-24T06:07:34.404 INFO:teuthology.orchestra.run.smithi092.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2023-11-24T06:07:34.404 INFO:teuthology.orchestra.run.smithi092.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2023-11-24T06:07:34.405 INFO:teuthology.orchestra.run.smithi092.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2023-11-24T06:07:34.405 INFO:teuthology.orchestra.run.smithi092.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2023-11-24T06:07:34.405 INFO:teuthology.orchestra.run.smithi092.stdout: parquet-libs x86_64 9.0.0-9.el9 epel 837 k 2023-11-24T06:07:34.405 INFO:teuthology.orchestra.run.smithi092.stdout: postgresql-private-libs x86_64 13.11-1.el9 CentOS-AppStream 134 k 2023-11-24T06:07:34.405 INFO:teuthology.orchestra.run.smithi092.stdout: python3-asyncssh noarch 2.13.2-3.el9 epel 548 k 2023-11-24T06:07:34.405 INFO:teuthology.orchestra.run.smithi092.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2023-11-24T06:07:34.405 INFO:teuthology.orchestra.run.smithi092.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2023-11-24T06:07:34.405 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2023-11-24T06:07:34.405 INFO:teuthology.orchestra.run.smithi092.stdout: python3-ceph-argparse x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 45 k 2023-11-24T06:07:34.405 INFO:teuthology.orchestra.run.smithi092.stdout: python3-ceph-common x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 129 k 2023-11-24T06:07:34.406 INFO:teuthology.orchestra.run.smithi092.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2023-11-24T06:07:34.406 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2023-11-24T06:07:34.406 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2023-11-24T06:07:34.406 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2023-11-24T06:07:34.406 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2023-11-24T06:07:34.406 INFO:teuthology.orchestra.run.smithi092.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2023-11-24T06:07:34.406 INFO:teuthology.orchestra.run.smithi092.stdout: python3-google-auth noarch 1:2.23.4-1.el9 epel 197 k 2023-11-24T06:07:34.406 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2023-11-24T06:07:34.406 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2023-11-24T06:07:34.406 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2023-11-24T06:07:34.406 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2023-11-24T06:07:34.407 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2023-11-24T06:07:34.407 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jinja2 noarch 2.11.3-4.el9 CentOS-AppStream 228 k 2023-11-24T06:07:34.407 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2023-11-24T06:07:34.407 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2023-11-24T06:07:34.407 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2023-11-24T06:07:34.407 INFO:teuthology.orchestra.run.smithi092.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2023-11-24T06:07:34.407 INFO:teuthology.orchestra.run.smithi092.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2023-11-24T06:07:34.407 INFO:teuthology.orchestra.run.smithi092.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2023-11-24T06:07:34.407 INFO:teuthology.orchestra.run.smithi092.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2023-11-24T06:07:34.407 INFO:teuthology.orchestra.run.smithi092.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2023-11-24T06:07:34.407 INFO:teuthology.orchestra.run.smithi092.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2023-11-24T06:07:34.408 INFO:teuthology.orchestra.run.smithi092.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2023-11-24T06:07:34.408 INFO:teuthology.orchestra.run.smithi092.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2023-11-24T06:07:34.408 INFO:teuthology.orchestra.run.smithi092.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2023-11-24T06:07:34.408 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2023-11-24T06:07:34.408 INFO:teuthology.orchestra.run.smithi092.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2023-11-24T06:07:34.408 INFO:teuthology.orchestra.run.smithi092.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2023-11-24T06:07:34.408 INFO:teuthology.orchestra.run.smithi092.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2023-11-24T06:07:34.408 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2023-11-24T06:07:34.408 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2023-11-24T06:07:34.408 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2023-11-24T06:07:34.408 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2023-11-24T06:07:34.409 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2023-11-24T06:07:34.409 INFO:teuthology.orchestra.run.smithi092.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2023-11-24T06:07:34.409 INFO:teuthology.orchestra.run.smithi092.stdout: python3-requests noarch 2.25.1-7.el9 CentOS-BaseOS 113 k 2023-11-24T06:07:34.409 INFO:teuthology.orchestra.run.smithi092.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2023-11-24T06:07:34.409 INFO:teuthology.orchestra.run.smithi092.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2023-11-24T06:07:34.409 INFO:teuthology.orchestra.run.smithi092.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2023-11-24T06:07:34.409 INFO:teuthology.orchestra.run.smithi092.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2023-11-24T06:07:34.409 INFO:teuthology.orchestra.run.smithi092.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2023-11-24T06:07:34.409 INFO:teuthology.orchestra.run.smithi092.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2023-11-24T06:07:34.409 INFO:teuthology.orchestra.run.smithi092.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2023-11-24T06:07:34.409 INFO:teuthology.orchestra.run.smithi092.stdout: python3-urllib3 noarch 1.26.5-4.el9 CentOS-BaseOS 187 k 2023-11-24T06:07:34.409 INFO:teuthology.orchestra.run.smithi092.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2023-11-24T06:07:34.410 INFO:teuthology.orchestra.run.smithi092.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2023-11-24T06:07:34.410 INFO:teuthology.orchestra.run.smithi092.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2023-11-24T06:07:34.410 INFO:teuthology.orchestra.run.smithi092.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2023-11-24T06:07:34.410 INFO:teuthology.orchestra.run.smithi092.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2023-11-24T06:07:34.410 INFO:teuthology.orchestra.run.smithi092.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2023-11-24T06:07:34.410 INFO:teuthology.orchestra.run.smithi092.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2023-11-24T06:07:34.410 INFO:teuthology.orchestra.run.smithi092.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2023-11-24T06:07:34.410 INFO:teuthology.orchestra.run.smithi092.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2023-11-24T06:07:34.410 INFO:teuthology.orchestra.run.smithi092.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2023-11-24T06:07:34.410 INFO:teuthology.orchestra.run.smithi092.stdout:Installing weak dependencies: 2023-11-24T06:07:34.411 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2023-11-24T06:07:34.411 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-24T06:07:34.411 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction Summary 2023-11-24T06:07:34.411 INFO:teuthology.orchestra.run.smithi092.stdout:============================================================================================ 2023-11-24T06:07:34.411 INFO:teuthology.orchestra.run.smithi092.stdout:Install 132 Packages 2023-11-24T06:07:34.411 INFO:teuthology.orchestra.run.smithi092.stdout:Upgrade 15 Packages 2023-11-24T06:07:34.411 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-24T06:07:34.412 INFO:teuthology.orchestra.run.smithi092.stdout:Total download size: 204 M 2023-11-24T06:07:34.412 INFO:teuthology.orchestra.run.smithi092.stdout:Downloading Packages: 2023-11-24T06:07:35.603 INFO:teuthology.orchestra.run.smithi092.stdout:(1/147): ceph-18.0.0-7484.ga13ea5ff.el9.x86_64. 15 kB/s | 6.5 kB 00:00 2023-11-24T06:07:36.434 INFO:teuthology.orchestra.run.smithi118.stdout:(10/147): ceph-test-18.0.0-7484.ga13ea5ff.el9.x 18 MB/s | 41 MB 00:02 2023-11-24T06:07:36.559 INFO:teuthology.orchestra.run.smithi118.stdout:(11/147): cephfs-mirror-18.0.0-7484.ga13ea5ff.e 1.7 MB/s | 217 kB 00:00 2023-11-24T06:07:36.651 INFO:teuthology.orchestra.run.smithi118.stdout:(12/147): libcephfs-devel-18.0.0-7484.ga13ea5ff 341 kB/s | 31 kB 00:00 2023-11-24T06:07:36.843 INFO:teuthology.orchestra.run.smithi118.stdout:(13/147): libcephfs2-18.0.0-7484.ga13ea5ff.el9. 3.7 MB/s | 719 kB 00:00 2023-11-24T06:07:36.977 INFO:teuthology.orchestra.run.smithi118.stdout:(14/147): libcephsqlite-18.0.0-7484.ga13ea5ff.e 1.2 MB/s | 163 kB 00:00 2023-11-24T06:07:37.094 INFO:teuthology.orchestra.run.smithi118.stdout:(15/147): librados-devel-18.0.0-7484.ga13ea5ff. 1.1 MB/s | 126 kB 00:00 2023-11-24T06:07:37.276 INFO:teuthology.orchestra.run.smithi118.stdout:(16/147): libradosstriper1-18.0.0-7484.ga13ea5f 2.6 MB/s | 476 kB 00:00 2023-11-24T06:07:37.712 INFO:teuthology.orchestra.run.smithi092.stdout:(2/147): ceph-fuse-18.0.0-7484.ga13ea5ff.el9.x8 408 kB/s | 859 kB 00:02 2023-11-24T06:07:38.062 INFO:teuthology.orchestra.run.smithi118.stdout:(17/147): librgw2-18.0.0-7484.ga13ea5ff.el9.x86 6.1 MB/s | 4.8 MB 00:00 2023-11-24T06:07:38.162 INFO:teuthology.orchestra.run.smithi092.stdout:(3/147): ceph-immutable-object-cache-18.0.0-748 314 kB/s | 141 kB 00:00 2023-11-24T06:07:38.178 INFO:teuthology.orchestra.run.smithi118.stdout:(18/147): python3-ceph-argparse-18.0.0-7484.ga1 389 kB/s | 45 kB 00:00 2023-11-24T06:07:38.278 INFO:teuthology.orchestra.run.smithi118.stdout:(19/147): python3-ceph-common-18.0.0-7484.ga13e 1.3 MB/s | 129 kB 00:00 2023-11-24T06:07:38.473 INFO:teuthology.orchestra.run.smithi118.stdout:(20/147): python3-cephfs-18.0.0-7484.ga13ea5ff. 835 kB/s | 162 kB 00:00 2023-11-24T06:07:38.870 INFO:teuthology.orchestra.run.smithi118.stdout:(21/147): python3-rados-18.0.0-7484.ga13ea5ff.e 805 kB/s | 319 kB 00:00 2023-11-24T06:07:39.031 INFO:teuthology.orchestra.run.smithi118.stdout:(22/147): python3-rbd-18.0.0-7484.ga13ea5ff.el9 1.8 MB/s | 298 kB 00:00 2023-11-24T06:07:39.157 INFO:teuthology.orchestra.run.smithi118.stdout:(23/147): python3-rgw-18.0.0-7484.ga13ea5ff.el9 801 kB/s | 99 kB 00:00 2023-11-24T06:07:39.265 INFO:teuthology.orchestra.run.smithi118.stdout:(24/147): ceph-mds-18.0.0-7484.ga13ea5ff.el9.x8 234 kB/s | 2.1 MB 00:09 2023-11-24T06:07:39.332 INFO:teuthology.orchestra.run.smithi118.stdout:(25/147): rbd-fuse-18.0.0-7484.ga13ea5ff.el9.x8 489 kB/s | 85 kB 00:00 2023-11-24T06:07:39.508 INFO:teuthology.orchestra.run.smithi118.stdout:(26/147): rbd-nbd-18.0.0-7484.ga13ea5ff.el9.x86 976 kB/s | 170 kB 00:00 2023-11-24T06:07:40.833 INFO:teuthology.orchestra.run.smithi118.stdout:(27/147): rbd-mirror-18.0.0-7484.ga13ea5ff.el9. 1.9 MB/s | 3.0 MB 00:01 2023-11-24T06:07:40.933 INFO:teuthology.orchestra.run.smithi118.stdout:(28/147): ceph-mgr-cephadm-18.0.0-7484.ga13ea5f 1.3 MB/s | 133 kB 00:00 2023-11-24T06:07:41.716 INFO:teuthology.orchestra.run.smithi118.stdout:(29/147): ceph-mgr-dashboard-18.0.0-7484.ga13ea 2.2 MB/s | 1.8 MB 00:00 2023-11-24T06:07:42.100 INFO:teuthology.orchestra.run.smithi118.stdout:(30/147): ceph-grafana-dashboards-18.0.0-7484.g 11 kB/s | 28 kB 00:02 2023-11-24T06:07:42.275 INFO:teuthology.orchestra.run.smithi118.stdout:(31/147): ceph-mgr-modules-core-18.0.0-7484.ga1 1.4 MB/s | 243 kB 00:00 2023-11-24T06:07:42.394 INFO:teuthology.orchestra.run.smithi092.stdout:(4/147): ceph-mds-18.0.0-7484.ga13ea5ff.el9.x86 505 kB/s | 2.1 MB 00:04 2023-11-24T06:07:42.796 INFO:teuthology.orchestra.run.smithi118.stdout:(32/147): ceph-mgr-rook-18.0.0-7484.ga13ea5ff.e 96 kB/s | 50 kB 00:00 2023-11-24T06:07:42.901 INFO:teuthology.orchestra.run.smithi118.stdout:(33/147): ceph-prometheus-alerts-18.0.0-7484.ga 146 kB/s | 15 kB 00:00 2023-11-24T06:07:43.020 INFO:teuthology.orchestra.run.smithi092.stdout:(5/147): ceph-mgr-18.0.0-7484.ga13ea5ff.el9.x86 2.3 MB/s | 1.5 MB 00:00 2023-11-24T06:07:43.145 INFO:teuthology.orchestra.run.smithi092.stdout:(6/147): ceph-base-18.0.0-7484.ga13ea5ff.el9.x8 670 kB/s | 5.2 MB 00:07 2023-11-24T06:07:43.166 INFO:teuthology.orchestra.run.smithi118.stdout:(34/147): ceph-volume-18.0.0-7484.ga13ea5ff.el9 995 kB/s | 262 kB 00:00 2023-11-24T06:07:43.327 INFO:teuthology.orchestra.run.smithi118.stdout:(35/147): cephadm-18.0.0-7484.ga13ea5ff.el9.noa 4.7 MB/s | 772 kB 00:00 2023-11-24T06:07:43.435 INFO:teuthology.orchestra.run.smithi118.stdout:(36/147): cephfs-top-18.0.0-7484.ga13ea5ff.el9. 169 kB/s | 18 kB 00:00 2023-11-24T06:07:43.769 INFO:teuthology.orchestra.run.smithi118.stdout:(37/147): automake-1.16.2-8.el9.noarch.rpm 1.9 MB/s | 662 kB 00:00 2023-11-24T06:07:43.852 INFO:teuthology.orchestra.run.smithi118.stdout:(38/147): bison-3.7.4-5.el9.x86_64.rpm 11 MB/s | 920 kB 00:00 2023-11-24T06:07:43.918 INFO:teuthology.orchestra.run.smithi118.stdout:(39/147): boost-program-options-1.75.0-8.el9.x8 1.6 MB/s | 106 kB 00:00 2023-11-24T06:07:43.970 INFO:teuthology.orchestra.run.smithi118.stdout:(40/147): elfutils-libelf-devel-0.190-1.el9.x86 429 kB/s | 21 kB 00:00 2023-11-24T06:07:44.028 INFO:teuthology.orchestra.run.smithi118.stdout:(41/147): flex-2.6.4-9.el9.x86_64.rpm 5.2 MB/s | 307 kB 00:00 2023-11-24T06:07:44.070 INFO:teuthology.orchestra.run.smithi118.stdout:(42/147): flexiblas-3.0.4-8.el9.x86_64.rpm 776 kB/s | 32 kB 00:00 2023-11-24T06:07:44.128 INFO:teuthology.orchestra.run.smithi092.stdout:(7/147): ceph-mon-18.0.0-7484.ga13ea5ff.el9.x86 4.0 MB/s | 4.4 MB 00:01 2023-11-24T06:07:44.237 INFO:teuthology.orchestra.run.smithi118.stdout:(43/147): flexiblas-netlib-3.0.4-8.el9.x86_64.r 18 MB/s | 3.0 MB 00:00 2023-11-24T06:07:44.315 INFO:teuthology.orchestra.run.smithi118.stdout:(44/147): flexiblas-openblas-openmp-3.0.4-8.el9 224 kB/s | 17 kB 00:00 2023-11-24T06:07:44.363 INFO:teuthology.orchestra.run.smithi118.stdout:(45/147): jq-1.6-16.el9.x86_64.rpm 3.9 MB/s | 186 kB 00:00 2023-11-24T06:07:44.405 INFO:teuthology.orchestra.run.smithi118.stdout:(46/147): libaio-devel-0.3.111-13.el9.x86_64.rp 245 kB/s | 10 kB 00:00 2023-11-24T06:07:44.446 INFO:teuthology.orchestra.run.smithi118.stdout:(47/147): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.8 MB/s | 159 kB 00:00 2023-11-24T06:07:44.489 INFO:teuthology.orchestra.run.smithi118.stdout:(48/147): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.0 MB/s | 44 kB 00:00 2023-11-24T06:07:44.547 INFO:teuthology.orchestra.run.smithi118.stdout:(49/147): librdkafka-1.6.1-102.el9.x86_64.rpm 11 MB/s | 661 kB 00:00 2023-11-24T06:07:44.606 INFO:teuthology.orchestra.run.smithi118.stdout:(50/147): libtool-2.4.6-45.el9.x86_64.rpm 9.7 MB/s | 578 kB 00:00 2023-11-24T06:07:44.657 INFO:teuthology.orchestra.run.smithi118.stdout:(51/147): libuuid-devel-2.37.4-15.el9.x86_64.rp 496 kB/s | 24 kB 00:00 2023-11-24T06:07:44.715 INFO:teuthology.orchestra.run.smithi118.stdout:(52/147): libzstd-devel-1.5.1-2.el9.x86_64.rpm 807 kB/s | 46 kB 00:00 2023-11-24T06:07:44.765 INFO:teuthology.orchestra.run.smithi118.stdout:(53/147): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.6 MB/s | 282 kB 00:00 2023-11-24T06:07:44.807 INFO:teuthology.orchestra.run.smithi118.stdout:(54/147): oniguruma-6.9.6-1.el9.5.x86_64.rpm 5.2 MB/s | 217 kB 00:00 2023-11-24T06:07:44.849 INFO:teuthology.orchestra.run.smithi118.stdout:(55/147): openblas-0.3.21-2.el9.x86_64.rpm 823 kB/s | 34 kB 00:00 2023-11-24T06:07:45.158 INFO:teuthology.orchestra.run.smithi118.stdout:(56/147): openblas-openmp-0.3.21-2.el9.x86_64.r 15 MB/s | 4.7 MB 00:00 2023-11-24T06:07:45.254 INFO:teuthology.orchestra.run.smithi092.stdout:(8/147): ceph-radosgw-18.0.0-7484.ga13ea5ff.el9 7.7 MB/s | 8.6 MB 00:01 2023-11-24T06:07:45.292 INFO:teuthology.orchestra.run.smithi118.stdout:(57/147): postgresql-13.11-1.el9.x86_64.rpm 11 MB/s | 1.5 MB 00:00 2023-11-24T06:07:45.359 INFO:teuthology.orchestra.run.smithi118.stdout:(58/147): postgresql-contrib-13.11-1.el9.x86_64 12 MB/s | 811 kB 00:00 2023-11-24T06:07:45.362 INFO:teuthology.orchestra.run.smithi092.stdout:(9/147): ceph-selinux-18.0.0-7484.ga13ea5ff.el9 224 kB/s | 24 kB 00:00 2023-11-24T06:07:45.409 INFO:teuthology.orchestra.run.smithi118.stdout:(59/147): postgresql-private-libs-13.11-1.el9.x 2.7 MB/s | 134 kB 00:00 2023-11-24T06:07:45.626 INFO:teuthology.orchestra.run.smithi118.stdout:(60/147): postgresql-server-13.11-1.el9.x86_64. 26 MB/s | 5.7 MB 00:00 2023-11-24T06:07:45.850 INFO:teuthology.orchestra.run.smithi118.stdout:(61/147): python3-babel-2.9.1-2.el9.noarch.rpm 26 MB/s | 5.8 MB 00:00 2023-11-24T06:07:45.910 INFO:teuthology.orchestra.run.smithi118.stdout:(62/147): python3-jinja2-2.11.3-4.el9.noarch.rp 3.8 MB/s | 228 kB 00:00 2023-11-24T06:07:45.952 INFO:teuthology.orchestra.run.smithi118.stdout:(63/147): python3-jsonpatch-1.21-16.el9.noarch. 599 kB/s | 24 kB 00:00 2023-11-24T06:07:45.994 INFO:teuthology.orchestra.run.smithi118.stdout:(64/147): python3-jsonpointer-2.0-4.el9.noarch. 441 kB/s | 18 kB 00:00 2023-11-24T06:07:46.044 INFO:teuthology.orchestra.run.smithi118.stdout:(65/147): python3-mako-1.1.4-6.el9.noarch.rpm 2.9 MB/s | 144 kB 00:00 2023-11-24T06:07:46.178 INFO:teuthology.orchestra.run.smithi118.stdout:(66/147): ceph-mgr-diskprediction-local-18.0.0- 1.7 MB/s | 7.4 MB 00:04 2023-11-24T06:07:46.195 INFO:teuthology.orchestra.run.smithi118.stdout:(67/147): python3-markupsafe-1.1.1-12.el9.x86_6 210 kB/s | 32 kB 00:00 2023-11-24T06:07:46.262 INFO:teuthology.orchestra.run.smithi118.stdout:(68/147): python3-numpy-f2py-1.20.1-5.el9.x86_6 3.2 MB/s | 217 kB 00:00 2023-11-24T06:07:46.404 INFO:teuthology.orchestra.run.smithi118.stdout:(69/147): python3-numpy-1.20.1-5.el9.x86_64.rpm 21 MB/s | 4.8 MB 00:00 2023-11-24T06:07:46.421 INFO:teuthology.orchestra.run.smithi118.stdout:(70/147): python3-oauthlib-3.1.1-5.el9.noarch.r 1.1 MB/s | 174 kB 00:00 2023-11-24T06:07:46.471 INFO:teuthology.orchestra.run.smithi118.stdout:(71/147): python3-prettytable-0.7.2-27.el9.noar 851 kB/s | 41 kB 00:00 2023-11-24T06:07:46.496 INFO:teuthology.orchestra.run.smithi118.stdout:(72/147): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.8 MB/s | 132 kB 00:00 2023-11-24T06:07:46.546 INFO:teuthology.orchestra.run.smithi118.stdout:(73/147): python3-pyasn1-modules-0.4.8-6.el9.no 4.2 MB/s | 211 kB 00:00 2023-11-24T06:07:46.572 INFO:teuthology.orchestra.run.smithi118.stdout:(74/147): python3-pytz-2021.1-5.el9.noarch.rpm 640 kB/s | 47 kB 00:00 2023-11-24T06:07:46.630 INFO:teuthology.orchestra.run.smithi118.stdout:(75/147): python3-requests-oauthlib-1.3.0-12.el 742 kB/s | 43 kB 00:00 2023-11-24T06:07:46.947 INFO:teuthology.orchestra.run.smithi118.stdout:(76/147): python3-scipy-1.6.2-8.el9.x86_64.rpm 42 MB/s | 16 MB 00:00 2023-11-24T06:07:47.003 INFO:teuthology.orchestra.run.smithi118.stdout:(77/147): python3-toml-0.10.2-6.el9.noarch.rpm 108 kB/s | 39 kB 00:00 2023-11-24T06:07:47.065 INFO:teuthology.orchestra.run.smithi118.stdout:(78/147): socat-1.7.4.1-5.el9.x86_64.rpm 4.9 MB/s | 300 kB 00:00 2023-11-24T06:07:47.089 INFO:teuthology.orchestra.run.smithi118.stdout:(79/147): uuid-1.6.2-55.el9.x86_64.rpm 667 kB/s | 56 kB 00:00 2023-11-24T06:07:47.140 INFO:teuthology.orchestra.run.smithi118.stdout:(80/147): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 63 kB 00:00 2023-11-24T06:07:47.157 INFO:teuthology.orchestra.run.smithi118.stdout:(81/147): xfsprogs-devel-5.19.0-4.el9.x86_64.rp 1.3 MB/s | 85 kB 00:00 2023-11-24T06:07:47.191 INFO:teuthology.orchestra.run.smithi118.stdout:(82/147): zlib-devel-1.2.11-40.el9.x86_64.rpm 906 kB/s | 44 kB 00:00 2023-11-24T06:07:47.216 INFO:teuthology.orchestra.run.smithi118.stdout:(83/147): fuse-2.9.9-15.el9.x86_64.rpm 1.3 MB/s | 79 kB 00:00 2023-11-24T06:07:47.258 INFO:teuthology.orchestra.run.smithi118.stdout:(84/147): libgfortran-11.4.1-2.3.el9.x86_64.rpm 12 MB/s | 803 kB 00:00 2023-11-24T06:07:47.292 INFO:teuthology.orchestra.run.smithi118.stdout:(85/147): libquadmath-11.4.1-2.3.el9.x86_64.rpm 2.8 MB/s | 195 kB 00:00 2023-11-24T06:07:47.325 INFO:teuthology.orchestra.run.smithi118.stdout:(86/147): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.7 MB/s | 241 kB 00:00 2023-11-24T06:07:47.401 INFO:teuthology.orchestra.run.smithi118.stdout:(87/147): python3-cryptography-36.0.1-4.el9.x86 11 MB/s | 1.1 MB 00:00 2023-11-24T06:07:47.668 INFO:teuthology.orchestra.run.smithi118.stdout:(88/147): ceph-common-18.0.0-7484.ga13ea5ff.el9 904 kB/s | 19 MB 00:21 2023-11-24T06:07:47.693 INFO:teuthology.orchestra.run.smithi118.stdout:(89/147): python3-ply-3.11-14.el9.noarch.rpm 288 kB/s | 103 kB 00:00 2023-11-24T06:07:47.718 INFO:teuthology.orchestra.run.smithi118.stdout:(90/147): python3-pycparser-2.20-6.el9.noarch.r 393 kB/s | 124 kB 00:00 2023-11-24T06:07:47.785 INFO:teuthology.orchestra.run.smithi118.stdout:(91/147): python3-requests-2.25.1-7.el9.noarch. 1.2 MB/s | 113 kB 00:00 2023-11-24T06:07:47.810 INFO:teuthology.orchestra.run.smithi118.stdout:(92/147): python3-urllib3-1.26.5-4.el9.noarch.r 1.6 MB/s | 187 kB 00:00 2023-11-24T06:07:48.004 INFO:teuthology.orchestra.run.smithi092.stdout:(10/147): ceph-test-18.0.0-7484.ga13ea5ff.el9.x 16 MB/s | 41 MB 00:02 2023-11-24T06:07:48.112 INFO:teuthology.orchestra.run.smithi092.stdout:(11/147): cephfs-mirror-18.0.0-7484.ga13ea5ff.e 2.0 MB/s | 217 kB 00:00 2023-11-24T06:07:48.140 INFO:teuthology.orchestra.run.smithi118.stdout:(93/147): python3-devel-3.9.17-2.el9.x86_64.rpm 747 kB/s | 245 kB 00:00 2023-11-24T06:07:48.204 INFO:teuthology.orchestra.run.smithi092.stdout:(12/147): libcephfs-devel-18.0.0-7484.ga13ea5ff 341 kB/s | 31 kB 00:00 2023-11-24T06:07:48.346 INFO:teuthology.orchestra.run.smithi092.stdout:(13/147): libcephfs2-18.0.0-7484.ga13ea5ff.el9. 5.0 MB/s | 719 kB 00:00 2023-11-24T06:07:48.446 INFO:teuthology.orchestra.run.smithi092.stdout:(14/147): libcephsqlite-18.0.0-7484.ga13ea5ff.e 1.6 MB/s | 163 kB 00:00 2023-11-24T06:07:48.469 INFO:teuthology.orchestra.run.smithi118.stdout:(94/147): openssl-devel-3.0.7-25.el9.x86_64.rpm 5.5 MB/s | 4.1 MB 00:00 2023-11-24T06:07:48.547 INFO:teuthology.orchestra.run.smithi092.stdout:(15/147): librados-devel-18.0.0-7484.ga13ea5ff. 1.2 MB/s | 126 kB 00:00 2023-11-24T06:07:48.644 INFO:teuthology.orchestra.run.smithi118.stdout:(95/147): gperftools-libs-2.9.1-2.el9.x86_64.rp 371 kB/s | 309 kB 00:00 2023-11-24T06:07:48.655 INFO:teuthology.orchestra.run.smithi092.stdout:(16/147): libradosstriper1-18.0.0-7484.ga13ea5f 4.3 MB/s | 476 kB 00:00 2023-11-24T06:07:48.761 INFO:teuthology.orchestra.run.smithi118.stdout:(96/147): libarrow-doc-9.0.0-9.el9.noarch.rpm 84 kB/s | 24 kB 00:00 2023-11-24T06:07:48.787 INFO:teuthology.orchestra.run.smithi118.stdout:(97/147): liboath-2.6.7-2.el9.x86_64.rpm 344 kB/s | 49 kB 00:00 2023-11-24T06:07:48.912 INFO:teuthology.orchestra.run.smithi118.stdout:(98/147): libunwind-1.6.2-1.el9.x86_64.rpm 451 kB/s | 67 kB 00:00 2023-11-24T06:07:49.039 INFO:teuthology.orchestra.run.smithi092.stdout:(17/147): librgw2-18.0.0-7484.ga13ea5ff.el9.x86 12 MB/s | 4.8 MB 00:00 2023-11-24T06:07:49.140 INFO:teuthology.orchestra.run.smithi092.stdout:(18/147): python3-ceph-argparse-18.0.0-7484.ga1 455 kB/s | 45 kB 00:00 2023-11-24T06:07:49.196 INFO:teuthology.orchestra.run.smithi118.stdout:(99/147): python3-asyncssh-2.13.2-3.el9.noarch. 1.9 MB/s | 548 kB 00:00 2023-11-24T06:07:49.240 INFO:teuthology.orchestra.run.smithi092.stdout:(19/147): python3-ceph-common-18.0.0-7484.ga13e 1.3 MB/s | 129 kB 00:00 2023-11-24T06:07:49.329 INFO:teuthology.orchestra.run.smithi118.stdout:(100/147): libarrow-9.0.0-9.el9.x86_64.rpm 3.7 MB/s | 4.4 MB 00:01 2023-11-24T06:07:49.340 INFO:teuthology.orchestra.run.smithi092.stdout:(20/147): python3-cephfs-18.0.0-7484.ga13ea5ff. 1.6 MB/s | 162 kB 00:00 2023-11-24T06:07:49.355 INFO:teuthology.orchestra.run.smithi118.stdout:(101/147): python3-bcrypt-3.2.2-1.el9.x86_64.rp 275 kB/s | 43 kB 00:00 2023-11-24T06:07:49.447 INFO:teuthology.orchestra.run.smithi118.stdout:(102/147): python3-cachetools-4.2.4-1.el9.noarc 276 kB/s | 32 kB 00:00 2023-11-24T06:07:49.468 INFO:teuthology.orchestra.run.smithi092.stdout:(21/147): python3-rados-18.0.0-7484.ga13ea5ff.e 2.4 MB/s | 319 kB 00:00 2023-11-24T06:07:49.489 INFO:teuthology.orchestra.run.smithi118.stdout:(103/147): parquet-libs-9.0.0-9.el9.x86_64.rpm 1.2 MB/s | 837 kB 00:00 2023-11-24T06:07:49.514 INFO:teuthology.orchestra.run.smithi118.stdout:(104/147): python3-certifi-2021.10.8-2.el9.noar 94 kB/s | 15 kB 00:00 2023-11-24T06:07:49.598 INFO:teuthology.orchestra.run.smithi118.stdout:(105/147): python3-cheroot-8.6.0-4.el9.noarch.r 1.1 MB/s | 172 kB 00:00 2023-11-24T06:07:49.607 INFO:teuthology.orchestra.run.smithi092.stdout:(22/147): python3-rbd-18.0.0-7484.ga13ea5ff.el9 2.1 MB/s | 298 kB 00:00 2023-11-24T06:07:49.682 INFO:teuthology.orchestra.run.smithi118.stdout:(106/147): python3-cherrypy-18.6.1-2.el9.noarch 2.1 MB/s | 358 kB 00:00 2023-11-24T06:07:49.699 INFO:teuthology.orchestra.run.smithi092.stdout:(23/147): python3-rgw-18.0.0-7484.ga13ea5ff.el9 1.1 MB/s | 99 kB 00:00 2023-11-24T06:07:49.707 INFO:teuthology.orchestra.run.smithi118.stdout:(107/147): python3-google-auth-2.23.4-1.el9.noa 1.0 MB/s | 197 kB 00:00 2023-11-24T06:07:49.724 INFO:teuthology.orchestra.run.smithi118.stdout:(108/147): python3-jaraco-8.2.1-3.el9.noarch.rp 85 kB/s | 11 kB 00:00 2023-11-24T06:07:49.800 INFO:teuthology.orchestra.run.smithi092.stdout:(24/147): rbd-fuse-18.0.0-7484.ga13ea5ff.el9.x8 858 kB/s | 85 kB 00:00 2023-11-24T06:07:49.816 INFO:teuthology.orchestra.run.smithi118.stdout:(109/147): python3-jaraco-classes-3.2.1-5.el9.n 133 kB/s | 18 kB 00:00 2023-11-24T06:07:49.858 INFO:teuthology.orchestra.run.smithi118.stdout:(110/147): python3-jaraco-collections-3.0.0-8.e 154 kB/s | 23 kB 00:00 2023-11-24T06:07:49.900 INFO:teuthology.orchestra.run.smithi118.stdout:(111/147): python3-jaraco-functools-3.5.0-2.el9 111 kB/s | 19 kB 00:00 2023-11-24T06:07:49.950 INFO:teuthology.orchestra.run.smithi118.stdout:(112/147): python3-jaraco-text-3.2.0-6.el9.noar 147 kB/s | 20 kB 00:00 2023-11-24T06:07:49.992 INFO:teuthology.orchestra.run.smithi118.stdout:(113/147): python3-jwt+crypto-2.4.0-1.el9.noarc 68 kB/s | 9.0 kB 00:00 2023-11-24T06:07:50.076 INFO:teuthology.orchestra.run.smithi118.stdout:(114/147): python3-jwt-2.4.0-1.el9.noarch.rpm 233 kB/s | 41 kB 00:00 2023-11-24T06:07:50.091 INFO:teuthology.orchestra.run.smithi092.stdout:(25/147): rbd-mirror-18.0.0-7484.ga13ea5ff.el9. 10 MB/s | 3.0 MB 00:00 2023-11-24T06:07:50.134 INFO:teuthology.orchestra.run.smithi118.stdout:(115/147): python3-kubernetes-26.1.0-2.el9.noar 5.6 MB/s | 1.0 MB 00:00 2023-11-24T06:07:50.176 INFO:teuthology.orchestra.run.smithi118.stdout:(116/147): python3-logutils-0.3.5-21.el9.noarch 252 kB/s | 46 kB 00:00 2023-11-24T06:07:50.192 INFO:teuthology.orchestra.run.smithi092.stdout:(26/147): rbd-nbd-18.0.0-7484.ga13ea5ff.el9.x86 1.7 MB/s | 170 kB 00:00 2023-11-24T06:07:50.210 INFO:teuthology.orchestra.run.smithi118.stdout:(117/147): python3-more-itertools-8.12.0-2.el9. 594 kB/s | 79 kB 00:00 2023-11-24T06:07:50.269 INFO:teuthology.orchestra.run.smithi118.stdout:(118/147): python3-natsort-7.1.1-5.el9.noarch.r 433 kB/s | 58 kB 00:00 2023-11-24T06:07:50.327 INFO:teuthology.orchestra.run.smithi118.stdout:(119/147): python3-pecan-1.4.2-3.el9.noarch.rpm 1.8 MB/s | 272 kB 00:00 2023-11-24T06:07:50.369 INFO:teuthology.orchestra.run.smithi118.stdout:(120/147): python3-portend-3.1.0-2.el9.noarch.r 103 kB/s | 16 kB 00:00 2023-11-24T06:07:50.428 INFO:teuthology.orchestra.run.smithi118.stdout:(121/147): python3-pyOpenSSL-21.0.0-1.el9.noarc 568 kB/s | 90 kB 00:00 2023-11-24T06:07:50.459 INFO:teuthology.orchestra.run.smithi092.stdout:(27/147): ceph-osd-18.0.0-7484.ga13ea5ff.el9.x8 2.4 MB/s | 18 MB 00:07 2023-11-24T06:07:50.470 INFO:teuthology.orchestra.run.smithi118.stdout:(122/147): python3-repoze-lru-0.7-16.el9.noarch 216 kB/s | 31 kB 00:00 2023-11-24T06:07:50.476 INFO:teuthology.orchestra.run.smithi092.stdout:(28/147): ceph-grafana-dashboards-18.0.0-7484.g 99 kB/s | 28 kB 00:00 2023-11-24T06:07:50.529 INFO:teuthology.orchestra.run.smithi118.stdout:(123/147): python3-routes-2.5.1-5.el9.noarch.rp 1.2 MB/s | 188 kB 00:00 2023-11-24T06:07:50.629 INFO:teuthology.orchestra.run.smithi118.stdout:(124/147): python3-tempora-5.0.0-2.el9.noarch.r 226 kB/s | 36 kB 00:00 2023-11-24T06:07:50.635 INFO:teuthology.orchestra.run.smithi092.stdout:(29/147): ceph-mgr-cephadm-18.0.0-7484.ga13ea5f 761 kB/s | 133 kB 00:00 2023-11-24T06:07:50.679 INFO:teuthology.orchestra.run.smithi118.stdout:(125/147): python3-typing-extensions-4.4.0-2.el 340 kB/s | 51 kB 00:00 2023-11-24T06:07:50.702 INFO:teuthology.orchestra.run.smithi092.stdout:(30/147): ceph-mgr-dashboard-18.0.0-7484.ga13ea 7.8 MB/s | 1.8 MB 00:00 2023-11-24T06:07:50.721 INFO:teuthology.orchestra.run.smithi118.stdout:(126/147): python3-rsa-4.9-2.el9.noarch.rpm 202 kB/s | 59 kB 00:00 2023-11-24T06:07:50.802 INFO:teuthology.orchestra.run.smithi092.stdout:(31/147): ceph-mgr-modules-core-18.0.0-7484.ga1 2.4 MB/s | 243 kB 00:00 2023-11-24T06:07:50.805 INFO:teuthology.orchestra.run.smithi118.stdout:(127/147): python3-webob-1.8.7-6.el9.noarch.rpm 1.3 MB/s | 230 kB 00:00 2023-11-24T06:07:50.839 INFO:teuthology.orchestra.run.smithi118.stdout:(128/147): python3-websocket-client-1.2.3-2.el9 566 kB/s | 90 kB 00:00 2023-11-24T06:07:50.881 INFO:teuthology.orchestra.run.smithi118.stdout:(129/147): python3-werkzeug-2.0.3-3.el9.1.noarc 2.6 MB/s | 427 kB 00:00 2023-11-24T06:07:50.894 INFO:teuthology.orchestra.run.smithi092.stdout:(32/147): ceph-mgr-rook-18.0.0-7484.ga13ea5ff.e 554 kB/s | 50 kB 00:00 2023-11-24T06:07:50.956 INFO:teuthology.orchestra.run.smithi118.stdout:(130/147): python3-zc-lockfile-2.0-10.el9.noarc 133 kB/s | 20 kB 00:00 2023-11-24T06:07:50.977 INFO:teuthology.orchestra.run.smithi092.stdout:(33/147): ceph-prometheus-alerts-18.0.0-7484.ga 182 kB/s | 15 kB 00:00 2023-11-24T06:07:50.990 INFO:teuthology.orchestra.run.smithi118.stdout:(131/147): re2-20211101-3.el9.x86_64.rpm 1.2 MB/s | 192 kB 00:00 2023-11-24T06:07:51.078 INFO:teuthology.orchestra.run.smithi092.stdout:(34/147): ceph-volume-18.0.0-7484.ga13ea5ff.el9 2.6 MB/s | 262 kB 00:00 2023-11-24T06:07:51.082 INFO:teuthology.orchestra.run.smithi118.stdout:(132/147): thrift-0.15.0-2.el9.x86_64.rpm 7.9 MB/s | 1.6 MB 00:00 2023-11-24T06:07:51.132 INFO:teuthology.orchestra.run.smithi118.stdout:(133/147): libacl-devel-2.3.1-4.el9.x86_64.rpm 1.3 MB/s | 68 kB 00:00 2023-11-24T06:07:51.233 INFO:teuthology.orchestra.run.smithi118.stdout:(134/147): NetworkManager-1.45.6-1.el9.x86_64.r 23 MB/s | 2.2 MB 00:00 2023-11-24T06:07:51.270 INFO:teuthology.orchestra.run.smithi092.stdout:(35/147): cephadm-18.0.0-7484.ga13ea5ff.el9.noa 3.9 MB/s | 772 kB 00:00 2023-11-24T06:07:51.325 INFO:teuthology.orchestra.run.smithi118.stdout:(135/147): NetworkManager-libnm-1.45.6-1.el9.x8 19 MB/s | 1.8 MB 00:00 2023-11-24T06:07:51.362 INFO:teuthology.orchestra.run.smithi092.stdout:(36/147): cephfs-top-18.0.0-7484.ga13ea5ff.el9. 200 kB/s | 18 kB 00:00 2023-11-24T06:07:51.375 INFO:teuthology.orchestra.run.smithi118.stdout:(136/147): NetworkManager-team-1.45.6-1.el9.x86 856 kB/s | 42 kB 00:00 2023-11-24T06:07:51.433 INFO:teuthology.orchestra.run.smithi118.stdout:(137/147): NetworkManager-tui-1.45.6-1.el9.x86_ 4.2 MB/s | 245 kB 00:00 2023-11-24T06:07:51.484 INFO:teuthology.orchestra.run.smithi118.stdout:(138/147): acl-2.3.1-4.el9.x86_64.rpm 1.3 MB/s | 69 kB 00:00 2023-11-24T06:07:51.565 INFO:teuthology.orchestra.run.smithi118.stdout:(139/147): elfutils-debuginfod-client-0.190-1.e 456 kB/s | 36 kB 00:00 2023-11-24T06:07:51.618 INFO:teuthology.orchestra.run.smithi118.stdout:(140/147): elfutils-libelf-0.190-1.el9.x86_64.r 3.6 MB/s | 192 kB 00:00 2023-11-24T06:07:51.629 INFO:teuthology.orchestra.run.smithi092.stdout:(37/147): automake-1.16.2-8.el9.noarch.rpm 2.4 MB/s | 662 kB 00:00 2023-11-24T06:07:51.668 INFO:teuthology.orchestra.run.smithi118.stdout:(141/147): elfutils-libs-0.190-1.el9.x86_64.rpm 5.0 MB/s | 255 kB 00:00 2023-11-24T06:07:51.705 INFO:teuthology.orchestra.run.smithi092.stdout:(38/147): bison-3.7.4-5.el9.x86_64.rpm 12 MB/s | 920 kB 00:00 2023-11-24T06:07:51.718 INFO:teuthology.orchestra.run.smithi118.stdout:(142/147): libacl-2.3.1-4.el9.x86_64.rpm 447 kB/s | 22 kB 00:00 2023-11-24T06:07:51.780 INFO:teuthology.orchestra.run.smithi092.stdout:(39/147): boost-program-options-1.75.0-8.el9.x8 1.4 MB/s | 106 kB 00:00 2023-11-24T06:07:51.793 INFO:teuthology.orchestra.run.smithi118.stdout:(143/147): librbd1-18.0.0-7484.ga13ea5ff.el9.x8 3.8 MB/s | 3.0 MB 00:00 2023-11-24T06:07:51.827 INFO:teuthology.orchestra.run.smithi118.stdout:(144/147): xfsdump-3.1.12-4.el9.x86_64.rpm 2.9 MB/s | 321 kB 00:00 2023-11-24T06:07:51.830 INFO:teuthology.orchestra.run.smithi092.stdout:(40/147): elfutils-libelf-devel-0.190-1.el9.x86 441 kB/s | 21 kB 00:00 2023-11-24T06:07:51.880 INFO:teuthology.orchestra.run.smithi092.stdout:(41/147): flex-2.6.4-9.el9.x86_64.rpm 6.1 MB/s | 307 kB 00:00 2023-11-24T06:07:51.922 INFO:teuthology.orchestra.run.smithi092.stdout:(42/147): flexiblas-3.0.4-8.el9.x86_64.rpm 766 kB/s | 32 kB 00:00 2023-11-24T06:07:52.056 INFO:teuthology.orchestra.run.smithi092.stdout:(43/147): flexiblas-netlib-3.0.4-8.el9.x86_64.r 22 MB/s | 3.0 MB 00:00 2023-11-24T06:07:52.098 INFO:teuthology.orchestra.run.smithi092.stdout:(44/147): flexiblas-openblas-openmp-3.0.4-8.el9 420 kB/s | 17 kB 00:00 2023-11-24T06:07:52.148 INFO:teuthology.orchestra.run.smithi092.stdout:(45/147): jq-1.6-16.el9.x86_64.rpm 3.7 MB/s | 186 kB 00:00 2023-11-24T06:07:52.190 INFO:teuthology.orchestra.run.smithi092.stdout:(46/147): libaio-devel-0.3.111-13.el9.x86_64.rp 246 kB/s | 10 kB 00:00 2023-11-24T06:07:52.232 INFO:teuthology.orchestra.run.smithi092.stdout:(47/147): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.8 MB/s | 159 kB 00:00 2023-11-24T06:07:52.274 INFO:teuthology.orchestra.run.smithi092.stdout:(48/147): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.0 MB/s | 44 kB 00:00 2023-11-24T06:07:52.366 INFO:teuthology.orchestra.run.smithi092.stdout:(49/147): librdkafka-1.6.1-102.el9.x86_64.rpm 7.1 MB/s | 661 kB 00:00 2023-11-24T06:07:52.433 INFO:teuthology.orchestra.run.smithi092.stdout:(50/147): libtool-2.4.6-45.el9.x86_64.rpm 8.6 MB/s | 578 kB 00:00 2023-11-24T06:07:52.475 INFO:teuthology.orchestra.run.smithi092.stdout:(51/147): libuuid-devel-2.37.4-15.el9.x86_64.rp 599 kB/s | 24 kB 00:00 2023-11-24T06:07:52.517 INFO:teuthology.orchestra.run.smithi092.stdout:(52/147): libzstd-devel-1.5.1-2.el9.x86_64.rpm 1.1 MB/s | 46 kB 00:00 2023-11-24T06:07:52.567 INFO:teuthology.orchestra.run.smithi092.stdout:(53/147): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.6 MB/s | 282 kB 00:00 2023-11-24T06:07:52.618 INFO:teuthology.orchestra.run.smithi092.stdout:(54/147): oniguruma-6.9.6-1.el9.5.x86_64.rpm 4.3 MB/s | 217 kB 00:00 2023-11-24T06:07:52.660 INFO:teuthology.orchestra.run.smithi092.stdout:(55/147): openblas-0.3.21-2.el9.x86_64.rpm 834 kB/s | 34 kB 00:00 2023-11-24T06:07:52.810 INFO:teuthology.orchestra.run.smithi092.stdout:(56/147): openblas-openmp-0.3.21-2.el9.x86_64.r 32 MB/s | 4.7 MB 00:00 2023-11-24T06:07:52.829 INFO:teuthology.orchestra.run.smithi118.stdout:(145/147): openssl-3.0.7-25.el9.x86_64.rpm 1.2 MB/s | 1.2 MB 00:01 2023-11-24T06:07:52.894 INFO:teuthology.orchestra.run.smithi092.stdout:(57/147): postgresql-13.11-1.el9.x86_64.rpm 18 MB/s | 1.5 MB 00:00 2023-11-24T06:07:52.903 INFO:teuthology.orchestra.run.smithi118.stdout:(146/147): openssl-libs-3.0.7-25.el9.x86_64.rpm 2.0 MB/s | 2.2 MB 00:01 2023-11-24T06:07:52.969 INFO:teuthology.orchestra.run.smithi092.stdout:(58/147): postgresql-contrib-13.11-1.el9.x86_64 11 MB/s | 811 kB 00:00 2023-11-24T06:07:53.011 INFO:teuthology.orchestra.run.smithi092.stdout:(59/147): postgresql-private-libs-13.11-1.el9.x 3.2 MB/s | 134 kB 00:00 2023-11-24T06:07:53.170 INFO:teuthology.orchestra.run.smithi092.stdout:(60/147): postgresql-server-13.11-1.el9.x86_64. 36 MB/s | 5.7 MB 00:00 2023-11-24T06:07:53.320 INFO:teuthology.orchestra.run.smithi092.stdout:(61/147): python3-babel-2.9.1-2.el9.noarch.rpm 39 MB/s | 5.8 MB 00:00 2023-11-24T06:07:53.371 INFO:teuthology.orchestra.run.smithi092.stdout:(62/147): python3-jinja2-2.11.3-4.el9.noarch.rp 4.6 MB/s | 228 kB 00:00 2023-11-24T06:07:53.412 INFO:teuthology.orchestra.run.smithi092.stdout:(63/147): python3-jsonpatch-1.21-16.el9.noarch. 593 kB/s | 24 kB 00:00 2023-11-24T06:07:53.454 INFO:teuthology.orchestra.run.smithi092.stdout:(64/147): python3-jsonpointer-2.0-4.el9.noarch. 442 kB/s | 18 kB 00:00 2023-11-24T06:07:53.522 INFO:teuthology.orchestra.run.smithi092.stdout:(65/147): python3-mako-1.1.4-6.el9.noarch.rpm 2.1 MB/s | 144 kB 00:00 2023-11-24T06:07:53.564 INFO:teuthology.orchestra.run.smithi092.stdout:(66/147): python3-markupsafe-1.1.1-12.el9.x86_6 768 kB/s | 32 kB 00:00 2023-11-24T06:07:53.706 INFO:teuthology.orchestra.run.smithi092.stdout:(67/147): python3-numpy-1.20.1-5.el9.x86_64.rpm 34 MB/s | 4.8 MB 00:00 2023-11-24T06:07:53.831 INFO:teuthology.orchestra.run.smithi092.stdout:(68/147): ceph-mgr-diskprediction-local-18.0.0- 2.3 MB/s | 7.4 MB 00:03 2023-11-24T06:07:53.864 INFO:teuthology.orchestra.run.smithi092.stdout:(69/147): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.3 MB/s | 217 kB 00:00 2023-11-24T06:07:53.898 INFO:teuthology.orchestra.run.smithi092.stdout:(70/147): python3-oauthlib-3.1.1-5.el9.noarch.r 2.7 MB/s | 174 kB 00:00 2023-11-24T06:07:53.923 INFO:teuthology.orchestra.run.smithi092.stdout:(71/147): python3-prettytable-0.7.2-27.el9.noar 731 kB/s | 41 kB 00:00 2023-11-24T06:07:53.949 INFO:teuthology.orchestra.run.smithi092.stdout:(72/147): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.6 MB/s | 132 kB 00:00 2023-11-24T06:07:53.974 INFO:teuthology.orchestra.run.smithi092.stdout:(73/147): python3-pyasn1-modules-0.4.8-6.el9.no 4.3 MB/s | 211 kB 00:00 2023-11-24T06:07:53.999 INFO:teuthology.orchestra.run.smithi092.stdout:(74/147): python3-pytz-2021.1-5.el9.noarch.rpm 992 kB/s | 47 kB 00:00 2023-11-24T06:07:54.024 INFO:teuthology.orchestra.run.smithi092.stdout:(75/147): python3-requests-oauthlib-1.3.0-12.el 876 kB/s | 43 kB 00:00 2023-11-24T06:07:54.342 INFO:teuthology.orchestra.run.smithi092.stdout:(76/147): python3-scipy-1.6.2-8.el9.x86_64.rpm 46 MB/s | 16 MB 00:00 2023-11-24T06:07:54.367 INFO:teuthology.orchestra.run.smithi092.stdout:(77/147): python3-toml-0.10.2-6.el9.noarch.rpm 118 kB/s | 39 kB 00:00 2023-11-24T06:07:54.417 INFO:teuthology.orchestra.run.smithi092.stdout:(78/147): socat-1.7.4.1-5.el9.x86_64.rpm 6.0 MB/s | 300 kB 00:00 2023-11-24T06:07:54.442 INFO:teuthology.orchestra.run.smithi092.stdout:(79/147): uuid-1.6.2-55.el9.x86_64.rpm 763 kB/s | 56 kB 00:00 2023-11-24T06:07:54.493 INFO:teuthology.orchestra.run.smithi092.stdout:(80/147): xfsprogs-devel-5.19.0-4.el9.x86_64.rp 1.7 MB/s | 85 kB 00:00 2023-11-24T06:07:54.518 INFO:teuthology.orchestra.run.smithi092.stdout:(81/147): xmlstarlet-1.6.1-20.el9.x86_64.rpm 849 kB/s | 63 kB 00:00 2023-11-24T06:07:54.568 INFO:teuthology.orchestra.run.smithi092.stdout:(82/147): zlib-devel-1.2.11-40.el9.x86_64.rpm 891 kB/s | 44 kB 00:00 2023-11-24T06:07:54.585 INFO:teuthology.orchestra.run.smithi092.stdout:(83/147): fuse-2.9.9-15.el9.x86_64.rpm 1.2 MB/s | 79 kB 00:00 2023-11-24T06:07:54.661 INFO:teuthology.orchestra.run.smithi092.stdout:(84/147): libquadmath-11.4.1-2.3.el9.x86_64.rpm 2.6 MB/s | 195 kB 00:00 2023-11-24T06:07:54.711 INFO:teuthology.orchestra.run.smithi092.stdout:(85/147): libgfortran-11.4.1-2.3.el9.x86_64.rpm 6.3 MB/s | 803 kB 00:00 2023-11-24T06:07:54.744 INFO:teuthology.orchestra.run.smithi092.stdout:(86/147): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.0 MB/s | 241 kB 00:00 2023-11-24T06:07:54.795 INFO:teuthology.orchestra.run.smithi092.stdout:(87/147): python3-cryptography-36.0.1-4.el9.x86 14 MB/s | 1.1 MB 00:00 2023-11-24T06:07:54.831 INFO:teuthology.orchestra.run.smithi092.stdout:(88/147): python3-ply-3.11-14.el9.noarch.rpm 1.3 MB/s | 103 kB 00:00 2023-11-24T06:07:54.856 INFO:teuthology.orchestra.run.smithi092.stdout:(89/147): python3-pycparser-2.20-6.el9.noarch.r 2.1 MB/s | 124 kB 00:00 2023-11-24T06:07:54.881 INFO:teuthology.orchestra.run.smithi092.stdout:(90/147): python3-requests-2.25.1-7.el9.noarch. 2.3 MB/s | 113 kB 00:00 2023-11-24T06:07:54.915 INFO:teuthology.orchestra.run.smithi092.stdout:(91/147): python3-urllib3-1.26.5-4.el9.noarch.r 3.2 MB/s | 187 kB 00:00 2023-11-24T06:07:55.115 INFO:teuthology.orchestra.run.smithi092.stdout:(92/147): python3-devel-3.9.17-2.el9.x86_64.rpm 1.2 MB/s | 245 kB 00:00 2023-11-24T06:07:55.824 INFO:teuthology.orchestra.run.smithi092.stdout:(93/147): gperftools-libs-2.9.1-2.el9.x86_64.rp 437 kB/s | 309 kB 00:00 2023-11-24T06:07:55.940 INFO:teuthology.orchestra.run.smithi092.stdout:(94/147): openssl-devel-3.0.7-25.el9.x86_64.rpm 3.9 MB/s | 4.1 MB 00:01 2023-11-24T06:07:56.216 INFO:teuthology.orchestra.run.smithi092.stdout:(95/147): libarrow-doc-9.0.0-9.el9.noarch.rpm 89 kB/s | 24 kB 00:00 2023-11-24T06:07:56.574 INFO:teuthology.orchestra.run.smithi092.stdout:(96/147): liboath-2.6.7-2.el9.x86_64.rpm 136 kB/s | 49 kB 00:00 2023-11-24T06:07:56.716 INFO:teuthology.orchestra.run.smithi092.stdout:(97/147): libunwind-1.6.2-1.el9.x86_64.rpm 477 kB/s | 67 kB 00:00 2023-11-24T06:07:57.050 INFO:teuthology.orchestra.run.smithi092.stdout:(98/147): parquet-libs-9.0.0-9.el9.x86_64.rpm 2.5 MB/s | 837 kB 00:00 2023-11-24T06:07:57.200 INFO:teuthology.orchestra.run.smithi092.stdout:(99/147): python3-asyncssh-2.13.2-3.el9.noarch. 3.6 MB/s | 548 kB 00:00 2023-11-24T06:07:57.317 INFO:teuthology.orchestra.run.smithi092.stdout:(100/147): python3-bcrypt-3.2.2-1.el9.x86_64.rp 375 kB/s | 43 kB 00:00 2023-11-24T06:07:57.434 INFO:teuthology.orchestra.run.smithi092.stdout:(101/147): python3-cachetools-4.2.4-1.el9.noarc 278 kB/s | 32 kB 00:00 2023-11-24T06:07:57.469 INFO:teuthology.orchestra.run.smithi118.stdout:(147/147): librados2-18.0.0-7484.ga13ea5ff.el9. 521 kB/s | 3.3 MB 00:06 2023-11-24T06:07:57.477 INFO:teuthology.orchestra.run.smithi118.stdout:-------------------------------------------------------------------------------- 2023-11-24T06:07:57.477 INFO:teuthology.orchestra.run.smithi118.stdout:Total 6.4 MB/s | 204 MB 00:31 2023-11-24T06:07:57.559 INFO:teuthology.orchestra.run.smithi092.stdout:(102/147): python3-certifi-2021.10.8-2.el9.noar 121 kB/s | 15 kB 00:00 2023-11-24T06:07:57.676 INFO:teuthology.orchestra.run.smithi092.stdout:(103/147): python3-cheroot-8.6.0-4.el9.noarch.r 1.4 MB/s | 172 kB 00:00 2023-11-24T06:07:57.704 INFO:teuthology.orchestra.run.smithi118.stdout:Running transaction check 2023-11-24T06:07:57.801 INFO:teuthology.orchestra.run.smithi092.stdout:(104/147): libarrow-9.0.0-9.el9.x86_64.rpm 2.2 MB/s | 4.4 MB 00:01 2023-11-24T06:07:57.843 INFO:teuthology.orchestra.run.smithi092.stdout:(105/147): python3-cherrypy-18.6.1-2.el9.noarch 2.1 MB/s | 358 kB 00:00 2023-11-24T06:07:57.981 INFO:teuthology.orchestra.run.smithi118.stdout:Transaction check succeeded. 2023-11-24T06:07:57.981 INFO:teuthology.orchestra.run.smithi118.stdout:Running transaction test 2023-11-24T06:07:58.019 INFO:teuthology.orchestra.run.smithi092.stdout:(106/147): python3-google-auth-2.23.4-1.el9.noa 913 kB/s | 197 kB 00:00 2023-11-24T06:07:58.044 INFO:teuthology.orchestra.run.smithi092.stdout:(107/147): python3-jaraco-8.2.1-3.el9.noarch.rp 53 kB/s | 11 kB 00:00 2023-11-24T06:07:58.136 INFO:teuthology.orchestra.run.smithi092.stdout:(108/147): python3-jaraco-classes-3.2.1-5.el9.n 152 kB/s | 18 kB 00:00 2023-11-24T06:07:58.161 INFO:teuthology.orchestra.run.smithi092.stdout:(109/147): python3-jaraco-collections-3.0.0-8.e 199 kB/s | 23 kB 00:00 2023-11-24T06:07:58.253 INFO:teuthology.orchestra.run.smithi092.stdout:(110/147): python3-jaraco-functools-3.5.0-2.el9 167 kB/s | 19 kB 00:00 2023-11-24T06:07:58.278 INFO:teuthology.orchestra.run.smithi092.stdout:(111/147): python3-jaraco-text-3.2.0-6.el9.noar 169 kB/s | 20 kB 00:00 2023-11-24T06:07:58.379 INFO:teuthology.orchestra.run.smithi092.stdout:(112/147): python3-jwt+crypto-2.4.0-1.el9.noarc 73 kB/s | 9.0 kB 00:00 2023-11-24T06:07:58.412 INFO:teuthology.orchestra.run.smithi092.stdout:(113/147): python3-jwt-2.4.0-1.el9.noarch.rpm 307 kB/s | 41 kB 00:00 2023-11-24T06:07:58.538 INFO:teuthology.orchestra.run.smithi092.stdout:(114/147): python3-logutils-0.3.5-21.el9.noarch 373 kB/s | 46 kB 00:00 2023-11-24T06:07:58.695 INFO:teuthology.orchestra.run.smithi092.stdout:(115/147): python3-more-itertools-8.12.0-2.el9. 502 kB/s | 79 kB 00:00 2023-11-24T06:07:58.822 INFO:teuthology.orchestra.run.smithi092.stdout:(116/147): python3-natsort-7.1.1-5.el9.noarch.r 460 kB/s | 58 kB 00:00 2023-11-24T06:07:58.939 INFO:teuthology.orchestra.run.smithi092.stdout:(117/147): python3-kubernetes-26.1.0-2.el9.noar 1.8 MB/s | 1.0 MB 00:00 2023-11-24T06:07:58.972 INFO:teuthology.orchestra.run.smithi092.stdout:(118/147): python3-pecan-1.4.2-3.el9.noarch.rpm 1.8 MB/s | 272 kB 00:00 2023-11-24T06:07:59.064 INFO:teuthology.orchestra.run.smithi092.stdout:(119/147): python3-portend-3.1.0-2.el9.noarch.r 132 kB/s | 16 kB 00:00 2023-11-24T06:07:59.098 INFO:teuthology.orchestra.run.smithi092.stdout:(120/147): python3-pyOpenSSL-21.0.0-1.el9.noarc 722 kB/s | 90 kB 00:00 2023-11-24T06:07:59.382 INFO:teuthology.orchestra.run.smithi092.stdout:(121/147): ceph-common-18.0.0-7484.ga13ea5ff.el 803 kB/s | 19 MB 00:24 2023-11-24T06:07:59.398 INFO:teuthology.orchestra.run.smithi092.stdout:(122/147): python3-repoze-lru-0.7-16.el9.noarch 92 kB/s | 31 kB 00:00 2023-11-24T06:07:59.432 INFO:teuthology.orchestra.run.smithi092.stdout:(123/147): python3-routes-2.5.1-5.el9.noarch.rp 566 kB/s | 188 kB 00:00 2023-11-24T06:07:59.524 INFO:teuthology.orchestra.run.smithi092.stdout:(124/147): python3-tempora-5.0.0-2.el9.noarch.r 287 kB/s | 36 kB 00:00 2023-11-24T06:07:59.558 INFO:teuthology.orchestra.run.smithi092.stdout:(125/147): python3-typing-extensions-4.4.0-2.el 408 kB/s | 51 kB 00:00 2023-11-24T06:07:59.699 INFO:teuthology.orchestra.run.smithi092.stdout:(126/147): python3-websocket-client-1.2.3-2.el9 636 kB/s | 90 kB 00:00 2023-11-24T06:07:59.750 INFO:teuthology.orchestra.run.smithi092.stdout:(127/147): python3-webob-1.8.7-6.el9.noarch.rpm 1.0 MB/s | 230 kB 00:00 2023-11-24T06:07:59.842 INFO:teuthology.orchestra.run.smithi092.stdout:(128/147): python3-rsa-4.9-2.el9.noarch.rpm 129 kB/s | 59 kB 00:00 2023-11-24T06:07:59.875 INFO:teuthology.orchestra.run.smithi092.stdout:(129/147): python3-zc-lockfile-2.0-10.el9.noarc 160 kB/s | 20 kB 00:00 2023-11-24T06:07:59.959 INFO:teuthology.orchestra.run.smithi092.stdout:(130/147): python3-werkzeug-2.0.3-3.el9.1.noarc 1.6 MB/s | 427 kB 00:00 2023-11-24T06:08:00.092 INFO:teuthology.orchestra.run.smithi092.stdout:(131/147): re2-20211101-3.el9.x86_64.rpm 770 kB/s | 192 kB 00:00 2023-11-24T06:08:00.118 INFO:teuthology.orchestra.run.smithi118.stdout:Transaction test succeeded. 2023-11-24T06:08:00.119 INFO:teuthology.orchestra.run.smithi118.stdout:Running transaction 2023-11-24T06:08:00.751 INFO:teuthology.orchestra.run.smithi092.stdout:(132/147): librados2-18.0.0-7484.ga13ea5ff.el9. 4.2 MB/s | 3.3 MB 00:00 2023-11-24T06:08:00.801 INFO:teuthology.orchestra.run.smithi092.stdout:(133/147): thrift-0.15.0-2.el9.x86_64.rpm 1.7 MB/s | 1.6 MB 00:00 2023-11-24T06:08:00.826 INFO:teuthology.orchestra.run.smithi092.stdout:(134/147): libacl-devel-2.3.1-4.el9.x86_64.rpm 909 kB/s | 68 kB 00:00 2023-11-24T06:08:00.910 INFO:teuthology.orchestra.run.smithi092.stdout:(135/147): NetworkManager-1.45.6-1.el9.x86_64.r 21 MB/s | 2.2 MB 00:00 2023-11-24T06:08:01.003 INFO:teuthology.orchestra.run.smithi092.stdout:(136/147): NetworkManager-libnm-1.45.6-1.el9.x8 11 MB/s | 1.8 MB 00:00 2023-11-24T06:08:01.028 INFO:teuthology.orchestra.run.smithi092.stdout:(137/147): NetworkManager-team-1.45.6-1.el9.x86 407 kB/s | 42 kB 00:00 2023-11-24T06:08:01.061 INFO:teuthology.orchestra.run.smithi092.stdout:(138/147): NetworkManager-tui-1.45.6-1.el9.x86_ 4.2 MB/s | 245 kB 00:00 2023-11-24T06:08:01.086 INFO:teuthology.orchestra.run.smithi092.stdout:(139/147): acl-2.3.1-4.el9.x86_64.rpm 1.2 MB/s | 69 kB 00:00 2023-11-24T06:08:01.112 INFO:teuthology.orchestra.run.smithi092.stdout:(140/147): elfutils-debuginfod-client-0.190-1.e 738 kB/s | 36 kB 00:00 2023-11-24T06:08:01.145 INFO:teuthology.orchestra.run.smithi092.stdout:(141/147): elfutils-libelf-0.190-1.el9.x86_64.r 3.3 MB/s | 192 kB 00:00 2023-11-24T06:08:01.179 INFO:teuthology.orchestra.run.smithi092.stdout:(142/147): elfutils-libs-0.190-1.el9.x86_64.rpm 3.9 MB/s | 255 kB 00:00 2023-11-24T06:08:01.204 INFO:teuthology.orchestra.run.smithi092.stdout:(143/147): libacl-2.3.1-4.el9.x86_64.rpm 396 kB/s | 22 kB 00:00 2023-11-24T06:08:01.238 INFO:teuthology.orchestra.run.smithi092.stdout:(144/147): xfsdump-3.1.12-4.el9.x86_64.rpm 5.4 MB/s | 321 kB 00:00 2023-11-24T06:08:01.871 INFO:teuthology.orchestra.run.smithi092.stdout:(145/147): openssl-3.0.7-25.el9.x86_64.rpm 1.8 MB/s | 1.2 MB 00:00 2023-11-24T06:08:02.438 INFO:teuthology.orchestra.run.smithi092.stdout:(146/147): librbd1-18.0.0-7484.ga13ea5ff.el9.x8 1.3 MB/s | 3.0 MB 00:02 2023-11-24T06:08:04.521 INFO:teuthology.orchestra.run.smithi092.stdout:(147/147): openssl-libs-3.0.7-25.el9.x86_64.rpm 672 kB/s | 2.2 MB 00:03 2023-11-24T06:08:04.530 INFO:teuthology.orchestra.run.smithi092.stdout:-------------------------------------------------------------------------------- 2023-11-24T06:08:04.530 INFO:teuthology.orchestra.run.smithi092.stdout:Total 6.8 MB/s | 204 MB 00:30 2023-11-24T06:08:04.759 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction check 2023-11-24T06:08:05.038 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction check succeeded. 2023-11-24T06:08:05.038 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction test 2023-11-24T06:08:06.557 INFO:teuthology.orchestra.run.smithi118.stdout: Preparing : 1/1 2023-11-24T06:08:06.725 INFO:teuthology.orchestra.run.smithi118.stdout: Upgrading : openssl-libs-1:3.0.7-25.el9.x86_64 1/162 2023-11-24T06:08:06.760 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : thrift-0.15.0-2.el9.x86_64 2/162 2023-11-24T06:08:06.844 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/162 2023-11-24T06:08:06.906 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 4/162 2023-11-24T06:08:06.928 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/162 2023-11-24T06:08:07.012 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/162 2023-11-24T06:08:07.270 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction test succeeded. 2023-11-24T06:08:07.270 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction 2023-11-24T06:08:07.407 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/162 2023-11-24T06:08:07.475 INFO:teuthology.orchestra.run.smithi118.stdout: Upgrading : librados2-2:18.0.0-7484.ga13ea5ff.el9.x86_64 8/162 2023-11-24T06:08:07.644 INFO:teuthology.orchestra.run.smithi118.stdout: Running scriptlet: librados2-2:18.0.0-7484.ga13ea5ff.el9.x86_64 8/162 2023-11-24T06:08:07.677 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : libcephfs2-2:18.0.0-7484.ga13ea5ff.el9.x86_64 9/162 2023-11-24T06:08:07.801 INFO:teuthology.orchestra.run.smithi118.stdout: Running scriptlet: libcephfs2-2:18.0.0-7484.ga13ea5ff.el9.x86_64 9/162 2023-11-24T06:08:07.842 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-rados-2:18.0.0-7484.ga13ea5ff.el9.x86_64 10/162 2023-11-24T06:08:07.911 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 11/162 2023-11-24T06:08:08.246 INFO:teuthology.orchestra.run.smithi118.stdout: Upgrading : elfutils-libelf-0.190-1.el9.x86_64 12/162 2023-11-24T06:08:08.298 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : postgresql-13.11-1.el9.x86_64 13/162 2023-11-24T06:08:08.323 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : libcephsqlite-2:18.0.0-7484.ga13ea5ff.el9.x86_64 14/162 2023-11-24T06:08:08.502 INFO:teuthology.orchestra.run.smithi118.stdout: Running scriptlet: libcephsqlite-2:18.0.0-7484.ga13ea5ff.el9.x86_64 14/162 2023-11-24T06:08:08.524 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : libradosstriper1-2:18.0.0-7484.ga13ea5ff.el9.x86 15/162 2023-11-24T06:08:08.616 INFO:teuthology.orchestra.run.smithi118.stdout: Running scriptlet: libradosstriper1-2:18.0.0-7484.ga13ea5ff.el9.x86 15/162 2023-11-24T06:08:08.970 INFO:teuthology.orchestra.run.smithi118.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 16/162 2023-11-24T06:08:09.427 INFO:teuthology.orchestra.run.smithi118.stdout: Upgrading : NetworkManager-libnm-1:1.45.6-1.el9.x86_64 17/162 2023-11-24T06:08:09.518 INFO:teuthology.orchestra.run.smithi118.stdout: Running scriptlet: NetworkManager-libnm-1:1.45.6-1.el9.x86_64 17/162 2023-11-24T06:08:09.718 INFO:teuthology.orchestra.run.smithi118.stdout: Running scriptlet: NetworkManager-1:1.45.6-1.el9.x86_64 18/162 2023-11-24T06:08:09.816 INFO:teuthology.orchestra.run.smithi118.stdout: Upgrading : NetworkManager-1:1.45.6-1.el9.x86_64 18/162 2023-11-24T06:08:10.330 INFO:teuthology.orchestra.run.smithi118.stdout: Running scriptlet: NetworkManager-1:1.45.6-1.el9.x86_64 18/162 2023-11-24T06:08:10.396 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/162 2023-11-24T06:08:10.505 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/162 2023-11-24T06:08:10.568 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-urllib3-1.26.5-4.el9.noarch 21/162 2023-11-24T06:08:10.608 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-requests-2.25.1-7.el9.noarch 22/162 2023-11-24T06:08:10.666 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : libquadmath-11.4.1-2.3.el9.x86_64 23/162 2023-11-24T06:08:10.729 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : libgfortran-11.4.1-2.3.el9.x86_64 24/162 2023-11-24T06:08:10.835 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 25/162 2023-11-24T06:08:10.878 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 26/162 2023-11-24T06:08:10.911 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 27/162 2023-11-24T06:08:11.045 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 28/162 2023-11-24T06:08:11.091 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-ceph-common-2:18.0.0-7484.ga13ea5ff.el9. 29/162 2023-11-24T06:08:11.142 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-ceph-argparse-2:18.0.0-7484.ga13ea5ff.el 30/162 2023-11-24T06:08:11.248 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-cephfs-2:18.0.0-7484.ga13ea5ff.el9.x86_6 31/162 2023-11-24T06:08:11.418 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 32/162 2023-11-24T06:08:11.485 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 33/162 2023-11-24T06:08:12.169 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-rsa-4.9-2.el9.noarch 34/162 2023-11-24T06:08:12.265 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 35/162 2023-11-24T06:08:12.315 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-jinja2-2.11.3-4.el9.noarch 36/162 2023-11-24T06:08:12.391 INFO:teuthology.orchestra.run.smithi118.stdout: Upgrading : elfutils-libs-0.190-1.el9.x86_64 37/162 2023-11-24T06:08:12.431 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : librados-devel-2:18.0.0-7484.ga13ea5ff.el9.x86_6 38/162 2023-11-24T06:08:12.526 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 39/162 2023-11-24T06:08:12.730 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 40/162 2023-11-24T06:08:12.793 INFO:teuthology.orchestra.run.smithi118.stdout: Upgrading : openssl-1:3.0.7-25.el9.x86_64 41/162 2023-11-24T06:08:12.935 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : re2-1:20211101-3.el9.x86_64 42/162 2023-11-24T06:08:13.005 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : libarrow-9.0.0-9.el9.x86_64 43/162 2023-11-24T06:08:13.072 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 44/162 2023-11-24T06:08:13.168 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 45/162 2023-11-24T06:08:13.259 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 46/162 2023-11-24T06:08:13.305 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 47/162 2023-11-24T06:08:13.389 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 48/162 2023-11-24T06:08:13.465 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 49/162 2023-11-24T06:08:13.495 INFO:teuthology.orchestra.run.smithi092.stdout: Preparing : 1/1 2023-11-24T06:08:13.533 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 50/162 2023-11-24T06:08:13.653 INFO:teuthology.orchestra.run.smithi092.stdout: Upgrading : openssl-libs-1:3.0.7-25.el9.x86_64 1/162 2023-11-24T06:08:13.694 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : thrift-0.15.0-2.el9.x86_64 2/162 2023-11-24T06:08:13.739 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 51/162 2023-11-24T06:08:13.755 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/162 2023-11-24T06:08:13.804 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 4/162 2023-11-24T06:08:13.820 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 52/162 2023-11-24T06:08:13.836 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/162 2023-11-24T06:08:13.868 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 53/162 2023-11-24T06:08:13.928 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/162 2023-11-24T06:08:13.996 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 54/162 2023-11-24T06:08:14.045 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-google-auth-1:2.23.4-1.el9.noarch 55/162 2023-11-24T06:08:14.094 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 56/162 2023-11-24T06:08:14.127 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 57/162 2023-11-24T06:08:14.282 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : libarrow-doc-9.0.0-9.el9.noarch 58/162 2023-11-24T06:08:14.340 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 59/162 2023-11-24T06:08:14.340 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/162 2023-11-24T06:08:14.385 INFO:teuthology.orchestra.run.smithi092.stdout: Upgrading : librados2-2:18.0.0-7484.ga13ea5ff.el9.x86_64 8/162 2023-11-24T06:08:14.444 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-ply-3.11-14.el9.noarch 60/162 2023-11-24T06:08:14.538 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 61/162 2023-11-24T06:08:14.548 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: librados2-2:18.0.0-7484.ga13ea5ff.el9.x86_64 8/162 2023-11-24T06:08:14.577 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libcephfs2-2:18.0.0-7484.ga13ea5ff.el9.x86_64 9/162 2023-11-24T06:08:14.730 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: libcephfs2-2:18.0.0-7484.ga13ea5ff.el9.x86_64 9/162 2023-11-24T06:08:14.767 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-rados-2:18.0.0-7484.ga13ea5ff.el9.x86_64 10/162 2023-11-24T06:08:14.790 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 62/162 2023-11-24T06:08:14.853 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 63/162 2023-11-24T06:08:14.872 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 11/162 2023-11-24T06:08:15.027 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 64/162 2023-11-24T06:08:15.080 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-asyncssh-2.13.2-3.el9.noarch 65/162 2023-11-24T06:08:15.113 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 66/162 2023-11-24T06:08:15.175 INFO:teuthology.orchestra.run.smithi092.stdout: Upgrading : elfutils-libelf-0.190-1.el9.x86_64 12/162 2023-11-24T06:08:15.180 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 67/162 2023-11-24T06:08:15.231 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 68/162 2023-11-24T06:08:15.232 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : postgresql-13.11-1.el9.x86_64 13/162 2023-11-24T06:08:15.249 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libcephsqlite-2:18.0.0-7484.ga13ea5ff.el9.x86_64 14/162 2023-11-24T06:08:15.276 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : fuse-2.9.9-15.el9.x86_64 69/162 2023-11-24T06:08:15.319 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 70/162 2023-11-24T06:08:15.379 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/162 2023-11-24T06:08:15.386 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: libcephsqlite-2:18.0.0-7484.ga13ea5ff.el9.x86_64 14/162 2023-11-24T06:08:15.408 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libradosstriper1-2:18.0.0-7484.ga13ea5ff.el9.x86 15/162 2023-11-24T06:08:15.443 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/162 2023-11-24T06:08:15.491 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: libradosstriper1-2:18.0.0-7484.ga13ea5ff.el9.x86 15/162 2023-11-24T06:08:15.492 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/162 2023-11-24T06:08:15.662 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/162 2023-11-24T06:08:15.712 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 75/162 2023-11-24T06:08:15.756 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/162 2023-11-24T06:08:15.791 INFO:teuthology.orchestra.run.smithi092.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 16/162 2023-11-24T06:08:15.830 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/162 2023-11-24T06:08:15.886 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/162 2023-11-24T06:08:15.988 INFO:teuthology.orchestra.run.smithi092.stdout: Upgrading : NetworkManager-libnm-1:1.45.6-1.el9.x86_64 17/162 2023-11-24T06:08:16.073 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/162 2023-11-24T06:08:16.076 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: NetworkManager-libnm-1:1.45.6-1.el9.x86_64 17/162 2023-11-24T06:08:16.133 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 80/162 2023-11-24T06:08:16.270 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: NetworkManager-1:1.45.6-1.el9.x86_64 18/162 2023-11-24T06:08:16.279 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/162 2023-11-24T06:08:16.325 INFO:teuthology.orchestra.run.smithi092.stdout: Upgrading : NetworkManager-1:1.45.6-1.el9.x86_64 18/162 2023-11-24T06:08:16.380 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/162 2023-11-24T06:08:16.786 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: NetworkManager-1:1.45.6-1.el9.x86_64 18/162 2023-11-24T06:08:16.821 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/162 2023-11-24T06:08:16.924 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/162 2023-11-24T06:08:17.010 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-urllib3-1.26.5-4.el9.noarch 21/162 2023-11-24T06:08:17.042 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-requests-2.25.1-7.el9.noarch 22/162 2023-11-24T06:08:17.098 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libquadmath-11.4.1-2.3.el9.x86_64 23/162 2023-11-24T06:08:17.161 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libgfortran-11.4.1-2.3.el9.x86_64 24/162 2023-11-24T06:08:17.266 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 25/162 2023-11-24T06:08:17.311 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 26/162 2023-11-24T06:08:17.344 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 27/162 2023-11-24T06:08:17.361 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/162 2023-11-24T06:08:17.413 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/162 2023-11-24T06:08:17.454 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 28/162 2023-11-24T06:08:17.476 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/162 2023-11-24T06:08:17.490 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-ceph-common-2:18.0.0-7484.ga13ea5ff.el9. 29/162 2023-11-24T06:08:17.513 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/162 2023-11-24T06:08:17.572 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-ceph-argparse-2:18.0.0-7484.ga13ea5ff.el 30/162 2023-11-24T06:08:17.684 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-cephfs-2:18.0.0-7484.ga13ea5ff.el9.x86_6 31/162 2023-11-24T06:08:17.772 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : openblas-0.3.21-2.el9.x86_64 87/162 2023-11-24T06:08:17.826 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 88/162 2023-11-24T06:08:17.862 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 32/162 2023-11-24T06:08:17.924 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 33/162 2023-11-24T06:08:17.947 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 89/162 2023-11-24T06:08:18.621 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-rsa-4.9-2.el9.noarch 34/162 2023-11-24T06:08:18.723 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 35/162 2023-11-24T06:08:18.752 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 90/162 2023-11-24T06:08:18.813 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-jinja2-2.11.3-4.el9.noarch 36/162 2023-11-24T06:08:18.913 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 91/162 2023-11-24T06:08:18.913 INFO:teuthology.orchestra.run.smithi092.stdout: Upgrading : elfutils-libs-0.190-1.el9.x86_64 37/162 2023-11-24T06:08:19.329 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : librados-devel-2:18.0.0-7484.ga13ea5ff.el9.x86_6 38/162 2023-11-24T06:08:19.421 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 39/162 2023-11-24T06:08:19.650 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 40/162 2023-11-24T06:08:19.699 INFO:teuthology.orchestra.run.smithi092.stdout: Upgrading : openssl-1:3.0.7-25.el9.x86_64 41/162 2023-11-24T06:08:19.846 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : re2-1:20211101-3.el9.x86_64 42/162 2023-11-24T06:08:19.885 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libarrow-9.0.0-9.el9.x86_64 43/162 2023-11-24T06:08:19.950 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 44/162 2023-11-24T06:08:20.064 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 45/162 2023-11-24T06:08:20.103 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 46/162 2023-11-24T06:08:20.141 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 47/162 2023-11-24T06:08:20.208 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 48/162 2023-11-24T06:08:20.264 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 49/162 2023-11-24T06:08:20.325 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 50/162 2023-11-24T06:08:20.506 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 92/162 2023-11-24T06:08:20.514 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 51/162 2023-11-24T06:08:20.591 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 52/162 2023-11-24T06:08:20.607 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 93/162 2023-11-24T06:08:20.645 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 94/162 2023-11-24T06:08:20.648 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 53/162 2023-11-24T06:08:20.676 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : jq-1.6-16.el9.x86_64 95/162 2023-11-24T06:08:20.720 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/162 2023-11-24T06:08:20.761 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : libuuid-devel-2.37.4-15.el9.x86_64 97/162 2023-11-24T06:08:20.778 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 54/162 2023-11-24T06:08:20.820 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-google-auth-1:2.23.4-1.el9.noarch 55/162 2023-11-24T06:08:20.875 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 56/162 2023-11-24T06:08:20.908 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 57/162 2023-11-24T06:08:21.062 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libarrow-doc-9.0.0-9.el9.noarch 58/162 2023-11-24T06:08:21.112 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 59/162 2023-11-24T06:08:21.126 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/162 2023-11-24T06:08:21.182 INFO:teuthology.orchestra.run.smithi118.stdout: Upgrading : librbd1-2:18.0.0-7484.ga13ea5ff.el9.x86_64 99/162 2023-11-24T06:08:21.207 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-ply-3.11-14.el9.noarch 60/162 2023-11-24T06:08:21.291 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 61/162 2023-11-24T06:08:21.328 INFO:teuthology.orchestra.run.smithi118.stdout: Running scriptlet: librbd1-2:18.0.0-7484.ga13ea5ff.el9.x86_64 99/162 2023-11-24T06:08:21.363 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-rbd-2:18.0.0-7484.ga13ea5ff.el9.x86_64 100/162 2023-11-24T06:08:21.426 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/162 2023-11-24T06:08:21.597 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 62/162 2023-11-24T06:08:21.649 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 63/162 2023-11-24T06:08:21.815 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 64/162 2023-11-24T06:08:21.866 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-asyncssh-2.13.2-3.el9.noarch 65/162 2023-11-24T06:08:21.893 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 66/162 2023-11-24T06:08:21.901 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : parquet-libs-9.0.0-9.el9.x86_64 102/162 2023-11-24T06:08:21.921 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 67/162 2023-11-24T06:08:21.935 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : librgw2-2:18.0.0-7484.ga13ea5ff.el9.x86_64 103/162 2023-11-24T06:08:21.956 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 68/162 2023-11-24T06:08:21.993 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : fuse-2.9.9-15.el9.x86_64 69/162 2023-11-24T06:08:22.035 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 70/162 2023-11-24T06:08:22.101 INFO:teuthology.orchestra.run.smithi118.stdout: Running scriptlet: librgw2-2:18.0.0-7484.ga13ea5ff.el9.x86_64 103/162 2023-11-24T06:08:22.107 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/162 2023-11-24T06:08:22.129 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-rgw-2:18.0.0-7484.ga13ea5ff.el9.x86_64 104/162 2023-11-24T06:08:22.172 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/162 2023-11-24T06:08:22.222 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/162 2023-11-24T06:08:22.352 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/162 2023-11-24T06:08:22.400 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 75/162 2023-11-24T06:08:22.444 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/162 2023-11-24T06:08:22.521 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/162 2023-11-24T06:08:22.563 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/162 2023-11-24T06:08:22.698 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/162 2023-11-24T06:08:22.743 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 80/162 2023-11-24T06:08:22.880 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/162 2023-11-24T06:08:22.949 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/162 2023-11-24T06:08:23.882 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/162 2023-11-24T06:08:23.932 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/162 2023-11-24T06:08:23.965 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/162 2023-11-24T06:08:23.998 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/162 2023-11-24T06:08:24.292 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : openblas-0.3.21-2.el9.x86_64 87/162 2023-11-24T06:08:24.320 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 88/162 2023-11-24T06:08:24.412 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 89/162 2023-11-24T06:08:24.662 INFO:teuthology.orchestra.run.smithi118.stdout: Running scriptlet: ceph-common-2:18.0.0-7484.ga13ea5ff.el9.x86_64 105/162 2023-11-24T06:08:24.735 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : ceph-common-2:18.0.0-7484.ga13ea5ff.el9.x86_64 105/162 2023-11-24T06:08:25.198 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 90/162 2023-11-24T06:08:25.339 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 91/162 2023-11-24T06:08:25.360 INFO:teuthology.orchestra.run.smithi118.stdout: Running scriptlet: ceph-common-2:18.0.0-7484.ga13ea5ff.el9.x86_64 105/162 2023-11-24T06:08:25.414 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : ceph-base-2:18.0.0-7484.ga13ea5ff.el9.x86_64 106/162 2023-11-24T06:08:25.593 INFO:teuthology.orchestra.run.smithi118.stdout: Running scriptlet: ceph-base-2:18.0.0-7484.ga13ea5ff.el9.x86_64 106/162 2023-11-24T06:08:25.593 INFO:teuthology.orchestra.run.smithi118.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2023-11-24T06:08:25.593 INFO:teuthology.orchestra.run.smithi118.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2023-11-24T06:08:25.593 INFO:teuthology.orchestra.run.smithi118.stdout: 2023-11-24T06:08:25.640 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : ceph-selinux-2:18.0.0-7484.ga13ea5ff.el9.x86_64 107/162 2023-11-24T06:08:26.883 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 92/162 2023-11-24T06:08:26.981 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 93/162 2023-11-24T06:08:27.019 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 94/162 2023-11-24T06:08:27.050 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : jq-1.6-16.el9.x86_64 95/162 2023-11-24T06:08:27.093 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/162 2023-11-24T06:08:27.135 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libuuid-devel-2.37.4-15.el9.x86_64 97/162 2023-11-24T06:08:27.507 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/162 2023-11-24T06:08:27.539 INFO:teuthology.orchestra.run.smithi092.stdout: Upgrading : librbd1-2:18.0.0-7484.ga13ea5ff.el9.x86_64 99/162 2023-11-24T06:08:27.658 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: librbd1-2:18.0.0-7484.ga13ea5ff.el9.x86_64 99/162 2023-11-24T06:08:27.686 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-rbd-2:18.0.0-7484.ga13ea5ff.el9.x86_64 100/162 2023-11-24T06:08:27.740 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/162 2023-11-24T06:08:28.215 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : parquet-libs-9.0.0-9.el9.x86_64 102/162 2023-11-24T06:08:28.242 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : librgw2-2:18.0.0-7484.ga13ea5ff.el9.x86_64 103/162 2023-11-24T06:08:28.349 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: librgw2-2:18.0.0-7484.ga13ea5ff.el9.x86_64 103/162 2023-11-24T06:08:28.377 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-rgw-2:18.0.0-7484.ga13ea5ff.el9.x86_64 104/162 2023-11-24T06:08:30.828 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-common-2:18.0.0-7484.ga13ea5ff.el9.x86_64 105/162 2023-11-24T06:08:31.590 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-common-2:18.0.0-7484.ga13ea5ff.el9.x86_64 105/162 2023-11-24T06:08:31.590 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-common-2:18.0.0-7484.ga13ea5ff.el9.x86_64 105/162 2023-11-24T06:08:31.590 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-base-2:18.0.0-7484.ga13ea5ff.el9.x86_64 106/162 2023-11-24T06:08:31.757 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-base-2:18.0.0-7484.ga13ea5ff.el9.x86_64 106/162 2023-11-24T06:08:31.757 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2023-11-24T06:08:31.757 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2023-11-24T06:08:31.757 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-24T06:08:31.812 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-selinux-2:18.0.0-7484.ga13ea5ff.el9.x86_64 107/162 2023-11-24T06:08:38.638 INFO:teuthology.orchestra.run.smithi118.stdout: Running scriptlet: ceph-selinux-2:18.0.0-7484.ga13ea5ff.el9.x86_64 107/162 2023-11-24T06:08:38.639 INFO:teuthology.orchestra.run.smithi118.stdout:skipping the directory /sys 2023-11-24T06:08:38.639 INFO:teuthology.orchestra.run.smithi118.stdout:skipping the directory /proc 2023-11-24T06:08:38.639 INFO:teuthology.orchestra.run.smithi118.stdout:skipping the directory /mnt 2023-11-24T06:08:38.639 INFO:teuthology.orchestra.run.smithi118.stdout:skipping the directory /var/tmp 2023-11-24T06:08:38.639 INFO:teuthology.orchestra.run.smithi118.stdout:skipping the directory /home 2023-11-24T06:08:38.639 INFO:teuthology.orchestra.run.smithi118.stdout:skipping the directory /root 2023-11-24T06:08:38.639 INFO:teuthology.orchestra.run.smithi118.stdout:skipping the directory /tmp 2023-11-24T06:08:38.639 INFO:teuthology.orchestra.run.smithi118.stdout: 2023-11-24T06:08:38.888 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : ceph-volume-2:18.0.0-7484.ga13ea5ff.el9.noarch 108/162 2023-11-24T06:08:38.939 INFO:teuthology.orchestra.run.smithi118.stdout: Running scriptlet: ceph-volume-2:18.0.0-7484.ga13ea5ff.el9.noarch 108/162 2023-11-24T06:08:38.939 INFO:teuthology.orchestra.run.smithi118.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-24T06:08:38.939 INFO:teuthology.orchestra.run.smithi118.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2023-11-24T06:08:38.939 INFO:teuthology.orchestra.run.smithi118.stdout: 2023-11-24T06:08:40.691 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : ceph-osd-2:18.0.0-7484.ga13ea5ff.el9.x86_64 109/162 2023-11-24T06:08:40.770 INFO:teuthology.orchestra.run.smithi118.stdout: Running scriptlet: ceph-osd-2:18.0.0-7484.ga13ea5ff.el9.x86_64 109/162 2023-11-24T06:08:40.771 INFO:teuthology.orchestra.run.smithi118.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-24T06:08:40.771 INFO:teuthology.orchestra.run.smithi118.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2023-11-24T06:08:40.771 INFO:teuthology.orchestra.run.smithi118.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2023-11-24T06:08:40.771 INFO:teuthology.orchestra.run.smithi118.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2023-11-24T06:08:40.771 INFO:teuthology.orchestra.run.smithi118.stdout: 2023-11-24T06:08:41.025 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : ceph-mds-2:18.0.0-7484.ga13ea5ff.el9.x86_64 110/162 2023-11-24T06:08:41.101 INFO:teuthology.orchestra.run.smithi118.stdout: Running scriptlet: ceph-mds-2:18.0.0-7484.ga13ea5ff.el9.x86_64 110/162 2023-11-24T06:08:41.101 INFO:teuthology.orchestra.run.smithi118.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-24T06:08:41.101 INFO:teuthology.orchestra.run.smithi118.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2023-11-24T06:08:41.101 INFO:teuthology.orchestra.run.smithi118.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2023-11-24T06:08:41.101 INFO:teuthology.orchestra.run.smithi118.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2023-11-24T06:08:41.101 INFO:teuthology.orchestra.run.smithi118.stdout: 2023-11-24T06:08:41.626 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : ceph-mon-2:18.0.0-7484.ga13ea5ff.el9.x86_64 111/162 2023-11-24T06:08:41.700 INFO:teuthology.orchestra.run.smithi118.stdout: Running scriptlet: ceph-mon-2:18.0.0-7484.ga13ea5ff.el9.x86_64 111/162 2023-11-24T06:08:41.700 INFO:teuthology.orchestra.run.smithi118.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-24T06:08:41.701 INFO:teuthology.orchestra.run.smithi118.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2023-11-24T06:08:41.701 INFO:teuthology.orchestra.run.smithi118.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2023-11-24T06:08:41.701 INFO:teuthology.orchestra.run.smithi118.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2023-11-24T06:08:41.701 INFO:teuthology.orchestra.run.smithi118.stdout: 2023-11-24T06:08:41.837 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : automake-1.16.2-8.el9.noarch 112/162 2023-11-24T06:08:42.656 INFO:teuthology.orchestra.run.smithi118.stdout: Running scriptlet: cephadm-2:18.0.0-7484.ga13ea5ff.el9.noarch 113/162 2023-11-24T06:08:42.706 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : cephadm-2:18.0.0-7484.ga13ea5ff.el9.noarch 113/162 2023-11-24T06:08:42.827 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-7484.ga13ea5ff.e 114/162 2023-11-24T06:08:42.928 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-7484.ga13ea5ff. 115/162 2023-11-24T06:08:42.960 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-7484.ga13ea5ff.el9.noa 116/162 2023-11-24T06:08:43.303 INFO:teuthology.orchestra.run.smithi118.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-7484.ga13ea5ff.el9.noa 116/162 2023-11-24T06:08:43.353 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-7484.ga13ea5ff.el9.n 117/162 2023-11-24T06:08:44.479 INFO:teuthology.orchestra.run.smithi118.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-7484.ga13ea5ff.el9.n 117/162 2023-11-24T06:08:44.512 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-7484.ga13 118/162 2023-11-24T06:08:44.609 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-selinux-2:18.0.0-7484.ga13ea5ff.el9.x86_64 107/162 2023-11-24T06:08:44.609 INFO:teuthology.orchestra.run.smithi092.stdout:skipping the directory /sys 2023-11-24T06:08:44.609 INFO:teuthology.orchestra.run.smithi092.stdout:skipping the directory /proc 2023-11-24T06:08:44.609 INFO:teuthology.orchestra.run.smithi092.stdout:skipping the directory /mnt 2023-11-24T06:08:44.609 INFO:teuthology.orchestra.run.smithi092.stdout:skipping the directory /var/tmp 2023-11-24T06:08:44.609 INFO:teuthology.orchestra.run.smithi092.stdout:skipping the directory /home 2023-11-24T06:08:44.609 INFO:teuthology.orchestra.run.smithi092.stdout:skipping the directory /root 2023-11-24T06:08:44.609 INFO:teuthology.orchestra.run.smithi092.stdout:skipping the directory /tmp 2023-11-24T06:08:44.610 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-24T06:08:44.654 INFO:teuthology.orchestra.run.smithi118.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-7484.ga13 118/162 2023-11-24T06:08:44.820 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-7484.ga13ea5ff.el 119/162 2023-11-24T06:08:44.822 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-volume-2:18.0.0-7484.ga13ea5ff.el9.noarch 108/162 2023-11-24T06:08:44.857 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : ceph-mgr-2:18.0.0-7484.ga13ea5ff.el9.x86_64 120/162 2023-11-24T06:08:44.868 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-volume-2:18.0.0-7484.ga13ea5ff.el9.noarch 108/162 2023-11-24T06:08:44.869 INFO:teuthology.orchestra.run.smithi092.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-24T06:08:44.869 INFO:teuthology.orchestra.run.smithi092.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2023-11-24T06:08:44.869 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-24T06:08:44.926 INFO:teuthology.orchestra.run.smithi118.stdout: Running scriptlet: ceph-mgr-2:18.0.0-7484.ga13ea5ff.el9.x86_64 120/162 2023-11-24T06:08:44.926 INFO:teuthology.orchestra.run.smithi118.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-24T06:08:44.926 INFO:teuthology.orchestra.run.smithi118.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2023-11-24T06:08:44.926 INFO:teuthology.orchestra.run.smithi118.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2023-11-24T06:08:44.926 INFO:teuthology.orchestra.run.smithi118.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2023-11-24T06:08:44.926 INFO:teuthology.orchestra.run.smithi118.stdout: 2023-11-24T06:08:44.990 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : ceph-mgr-rook-2:18.0.0-7484.ga13ea5ff.el9.noarch 121/162 2023-11-24T06:08:45.031 INFO:teuthology.orchestra.run.smithi118.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-7484.ga13ea5ff.el9.noarch 121/162 2023-11-24T06:08:45.120 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : ceph-2:18.0.0-7484.ga13ea5ff.el9.x86_64 122/162 2023-11-24T06:08:46.007 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : libtool-2.4.6-45.el9.x86_64 123/162 2023-11-24T06:08:46.068 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : ceph-radosgw-2:18.0.0-7484.ga13ea5ff.el9.x86_64 124/162 2023-11-24T06:08:46.139 INFO:teuthology.orchestra.run.smithi118.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-7484.ga13ea5ff.el9.x86_64 124/162 2023-11-24T06:08:46.140 INFO:teuthology.orchestra.run.smithi118.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-24T06:08:46.140 INFO:teuthology.orchestra.run.smithi118.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2023-11-24T06:08:46.140 INFO:teuthology.orchestra.run.smithi118.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2023-11-24T06:08:46.140 INFO:teuthology.orchestra.run.smithi118.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2023-11-24T06:08:46.140 INFO:teuthology.orchestra.run.smithi118.stdout: 2023-11-24T06:08:46.193 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-7484.ga13ea 125/162 2023-11-24T06:08:46.265 INFO:teuthology.orchestra.run.smithi118.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-7484.ga13ea 125/162 2023-11-24T06:08:46.266 INFO:teuthology.orchestra.run.smithi118.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-24T06:08:46.266 INFO:teuthology.orchestra.run.smithi118.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2023-11-24T06:08:46.266 INFO:teuthology.orchestra.run.smithi118.stdout: 2023-11-24T06:08:46.344 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : cephfs-mirror-2:18.0.0-7484.ga13ea5ff.el9.x86_64 126/162 2023-11-24T06:08:46.425 INFO:teuthology.orchestra.run.smithi118.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-7484.ga13ea5ff.el9.x86_64 126/162 2023-11-24T06:08:46.425 INFO:teuthology.orchestra.run.smithi118.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-24T06:08:46.425 INFO:teuthology.orchestra.run.smithi118.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2023-11-24T06:08:46.425 INFO:teuthology.orchestra.run.smithi118.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2023-11-24T06:08:46.425 INFO:teuthology.orchestra.run.smithi118.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2023-11-24T06:08:46.426 INFO:teuthology.orchestra.run.smithi118.stdout: 2023-11-24T06:08:46.557 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-osd-2:18.0.0-7484.ga13ea5ff.el9.x86_64 109/162 2023-11-24T06:08:46.639 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-osd-2:18.0.0-7484.ga13ea5ff.el9.x86_64 109/162 2023-11-24T06:08:46.639 INFO:teuthology.orchestra.run.smithi092.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-24T06:08:46.639 INFO:teuthology.orchestra.run.smithi092.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2023-11-24T06:08:46.639 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2023-11-24T06:08:46.639 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2023-11-24T06:08:46.639 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-24T06:08:46.842 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : rbd-mirror-2:18.0.0-7484.ga13ea5ff.el9.x86_64 127/162 2023-11-24T06:08:46.874 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-mds-2:18.0.0-7484.ga13ea5ff.el9.x86_64 110/162 2023-11-24T06:08:46.929 INFO:teuthology.orchestra.run.smithi118.stdout: Running scriptlet: rbd-mirror-2:18.0.0-7484.ga13ea5ff.el9.x86_64 127/162 2023-11-24T06:08:46.929 INFO:teuthology.orchestra.run.smithi118.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-24T06:08:46.929 INFO:teuthology.orchestra.run.smithi118.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2023-11-24T06:08:46.929 INFO:teuthology.orchestra.run.smithi118.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2023-11-24T06:08:46.929 INFO:teuthology.orchestra.run.smithi118.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2023-11-24T06:08:46.929 INFO:teuthology.orchestra.run.smithi118.stdout: 2023-11-24T06:08:46.953 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-mds-2:18.0.0-7484.ga13ea5ff.el9.x86_64 110/162 2023-11-24T06:08:46.953 INFO:teuthology.orchestra.run.smithi092.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-24T06:08:46.953 INFO:teuthology.orchestra.run.smithi092.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2023-11-24T06:08:46.954 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2023-11-24T06:08:46.954 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2023-11-24T06:08:46.954 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-24T06:08:47.442 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-mon-2:18.0.0-7484.ga13ea5ff.el9.x86_64 111/162 2023-11-24T06:08:47.504 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-mon-2:18.0.0-7484.ga13ea5ff.el9.x86_64 111/162 2023-11-24T06:08:47.504 INFO:teuthology.orchestra.run.smithi092.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-24T06:08:47.504 INFO:teuthology.orchestra.run.smithi092.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2023-11-24T06:08:47.504 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2023-11-24T06:08:47.504 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2023-11-24T06:08:47.504 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-24T06:08:47.619 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : automake-1.16.2-8.el9.noarch 112/162 2023-11-24T06:08:48.309 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: cephadm-2:18.0.0-7484.ga13ea5ff.el9.noarch 113/162 2023-11-24T06:08:48.344 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : cephadm-2:18.0.0-7484.ga13ea5ff.el9.noarch 113/162 2023-11-24T06:08:48.382 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-7484.ga13ea5ff.e 114/162 2023-11-24T06:08:48.458 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-7484.ga13ea5ff. 115/162 2023-11-24T06:08:48.490 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-7484.ga13ea5ff.el9.noa 116/162 2023-11-24T06:08:48.804 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-7484.ga13ea5ff.el9.noa 116/162 2023-11-24T06:08:48.841 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-7484.ga13ea5ff.el9.n 117/162 2023-11-24T06:08:49.961 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-7484.ga13ea5ff.el9.n 117/162 2023-11-24T06:08:49.991 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-7484.ga13 118/162 2023-11-24T06:08:50.131 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-7484.ga13 118/162 2023-11-24T06:08:50.298 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-7484.ga13ea5ff.el 119/162 2023-11-24T06:08:50.326 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-mgr-2:18.0.0-7484.ga13ea5ff.el9.x86_64 120/162 2023-11-24T06:08:50.422 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-mgr-2:18.0.0-7484.ga13ea5ff.el9.x86_64 120/162 2023-11-24T06:08:50.422 INFO:teuthology.orchestra.run.smithi092.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-24T06:08:50.423 INFO:teuthology.orchestra.run.smithi092.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2023-11-24T06:08:50.423 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2023-11-24T06:08:50.423 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2023-11-24T06:08:50.423 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-24T06:08:50.477 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-mgr-rook-2:18.0.0-7484.ga13ea5ff.el9.noarch 121/162 2023-11-24T06:08:50.514 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-7484.ga13ea5ff.el9.noarch 121/162 2023-11-24T06:08:50.574 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-2:18.0.0-7484.ga13ea5ff.el9.x86_64 122/162 2023-11-24T06:08:51.109 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : ceph-test-2:18.0.0-7484.ga13ea5ff.el9.x86_64 128/162 2023-11-24T06:08:51.172 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : rbd-fuse-2:18.0.0-7484.ga13ea5ff.el9.x86_64 129/162 2023-11-24T06:08:51.236 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : rbd-nbd-2:18.0.0-7484.ga13ea5ff.el9.x86_64 130/162 2023-11-24T06:08:51.275 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : xfsprogs-devel-5.19.0-4.el9.x86_64 131/162 2023-11-24T06:08:51.456 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libtool-2.4.6-45.el9.x86_64 123/162 2023-11-24T06:08:51.496 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-radosgw-2:18.0.0-7484.ga13ea5ff.el9.x86_64 124/162 2023-11-24T06:08:51.554 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : elfutils-libelf-devel-0.190-1.el9.x86_64 132/162 2023-11-24T06:08:51.573 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-7484.ga13ea5ff.el9.x86_64 124/162 2023-11-24T06:08:51.573 INFO:teuthology.orchestra.run.smithi092.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-24T06:08:51.573 INFO:teuthology.orchestra.run.smithi092.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2023-11-24T06:08:51.574 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2023-11-24T06:08:51.574 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2023-11-24T06:08:51.574 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-24T06:08:51.629 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-7484.ga13ea 125/162 2023-11-24T06:08:51.691 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 133/162 2023-11-24T06:08:51.725 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-7484.ga13ea 125/162 2023-11-24T06:08:51.725 INFO:teuthology.orchestra.run.smithi092.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-24T06:08:51.725 INFO:teuthology.orchestra.run.smithi092.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2023-11-24T06:08:51.725 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-24T06:08:51.759 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : ceph-fuse-2:18.0.0-7484.ga13ea5ff.el9.x86_64 134/162 2023-11-24T06:08:51.797 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : cephfs-mirror-2:18.0.0-7484.ga13ea5ff.el9.x86_64 126/162 2023-11-24T06:08:51.805 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : libcephfs-devel-2:18.0.0-7484.ga13ea5ff.el9.x86_ 135/162 2023-11-24T06:08:51.853 INFO:teuthology.orchestra.run.smithi118.stdout: Upgrading : elfutils-debuginfod-client-0.190-1.el9.x86_64 136/162 2023-11-24T06:08:51.899 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-7484.ga13ea5ff.el9.x86_64 126/162 2023-11-24T06:08:51.899 INFO:teuthology.orchestra.run.smithi092.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-24T06:08:51.899 INFO:teuthology.orchestra.run.smithi092.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2023-11-24T06:08:51.899 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2023-11-24T06:08:51.899 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2023-11-24T06:08:51.900 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-24T06:08:51.901 INFO:teuthology.orchestra.run.smithi118.stdout: Upgrading : NetworkManager-team-1:1.45.6-1.el9.x86_64 137/162 2023-11-24T06:08:52.023 INFO:teuthology.orchestra.run.smithi118.stdout: Upgrading : NetworkManager-tui-1:1.45.6-1.el9.x86_64 138/162 2023-11-24T06:08:52.095 INFO:teuthology.orchestra.run.smithi118.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 139/162 2023-11-24T06:08:52.165 INFO:teuthology.orchestra.run.smithi118.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 140/162 2023-11-24T06:08:52.272 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : rbd-mirror-2:18.0.0-7484.ga13ea5ff.el9.x86_64 127/162 2023-11-24T06:08:52.356 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: rbd-mirror-2:18.0.0-7484.ga13ea5ff.el9.x86_64 127/162 2023-11-24T06:08:52.356 INFO:teuthology.orchestra.run.smithi092.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-24T06:08:52.356 INFO:teuthology.orchestra.run.smithi092.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2023-11-24T06:08:52.357 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2023-11-24T06:08:52.357 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2023-11-24T06:08:52.357 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-24T06:08:53.143 INFO:teuthology.orchestra.run.smithi118.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 141/162 2023-11-24T06:08:53.189 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 141/162 2023-11-24T06:08:53.251 INFO:teuthology.orchestra.run.smithi118.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 141/162 2023-11-24T06:08:54.545 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : cephfs-top-2:18.0.0-7484.ga13ea5ff.el9.noarch 142/162 2023-11-24T06:08:54.661 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : openssl-devel-1:3.0.7-25.el9.x86_64 143/162 2023-11-24T06:08:54.711 INFO:teuthology.orchestra.run.smithi118.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 144/162 2023-11-24T06:08:54.810 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 145/162 2023-11-24T06:08:54.978 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : flex-2.6.4-9.el9.x86_64 146/162 2023-11-24T06:08:55.032 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : bison-3.7.4-5.el9.x86_64 147/162 2023-11-24T06:08:55.065 INFO:teuthology.orchestra.run.smithi118.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 148/162 2023-11-24T06:08:55.100 INFO:teuthology.orchestra.run.smithi118.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 149/162 2023-11-24T06:08:55.143 INFO:teuthology.orchestra.run.smithi118.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 150/162 2023-11-24T06:08:55.143 INFO:teuthology.orchestra.run.smithi118.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 151/162 2023-11-24T06:08:55.308 INFO:teuthology.orchestra.run.smithi118.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 151/162 2023-11-24T06:08:55.359 INFO:teuthology.orchestra.run.smithi118.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 152/162 2023-11-24T06:08:55.405 INFO:teuthology.orchestra.run.smithi118.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 153/162 2023-11-24T06:08:55.437 INFO:teuthology.orchestra.run.smithi118.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 154/162 2023-11-24T06:08:55.437 INFO:teuthology.orchestra.run.smithi118.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/162 2023-11-24T06:08:55.608 INFO:teuthology.orchestra.run.smithi118.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/162 2023-11-24T06:08:55.647 INFO:teuthology.orchestra.run.smithi118.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 156/162 2023-11-24T06:08:55.686 INFO:teuthology.orchestra.run.smithi118.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 157/162 2023-11-24T06:08:55.687 INFO:teuthology.orchestra.run.smithi118.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 157/162 2023-11-24T06:08:56.029 INFO:teuthology.orchestra.run.smithi118.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 157/162 2023-11-24T06:08:56.030 INFO:teuthology.orchestra.run.smithi118.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 158/162 2023-11-24T06:08:56.172 INFO:teuthology.orchestra.run.smithi118.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 158/162 2023-11-24T06:08:56.208 INFO:teuthology.orchestra.run.smithi118.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 159/162 2023-11-24T06:08:56.242 INFO:teuthology.orchestra.run.smithi118.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 160/162 2023-11-24T06:08:56.293 INFO:teuthology.orchestra.run.smithi118.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 161/162 2023-11-24T06:08:56.458 INFO:teuthology.orchestra.run.smithi118.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 162/162 2023-11-24T06:08:56.481 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-test-2:18.0.0-7484.ga13ea5ff.el9.x86_64 128/162 2023-11-24T06:08:56.552 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : rbd-fuse-2:18.0.0-7484.ga13ea5ff.el9.x86_64 129/162 2023-11-24T06:08:56.610 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : rbd-nbd-2:18.0.0-7484.ga13ea5ff.el9.x86_64 130/162 2023-11-24T06:08:56.645 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : xfsprogs-devel-5.19.0-4.el9.x86_64 131/162 2023-11-24T06:08:56.744 INFO:teuthology.orchestra.run.smithi118.stdout: Running scriptlet: xfsdump-3.1.12-3.el9.x86_64 162/162 2023-11-24T06:08:56.744 INFO:teuthology.orchestra.run.smithi118.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2023-11-24T06:08:56.744 INFO:teuthology.orchestra.run.smithi118.stdout: 2023-11-24T06:08:56.956 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : elfutils-libelf-devel-0.190-1.el9.x86_64 132/162 2023-11-24T06:08:57.078 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 133/162 2023-11-24T06:08:57.098 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-fuse-2:18.0.0-7484.ga13ea5ff.el9.x86_64 134/162 2023-11-24T06:08:57.123 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libcephfs-devel-2:18.0.0-7484.ga13ea5ff.el9.x86_ 135/162 2023-11-24T06:08:57.155 INFO:teuthology.orchestra.run.smithi092.stdout: Upgrading : elfutils-debuginfod-client-0.190-1.el9.x86_64 136/162 2023-11-24T06:08:57.188 INFO:teuthology.orchestra.run.smithi092.stdout: Upgrading : NetworkManager-team-1:1.45.6-1.el9.x86_64 137/162 2023-11-24T06:08:57.268 INFO:teuthology.orchestra.run.smithi092.stdout: Upgrading : NetworkManager-tui-1:1.45.6-1.el9.x86_64 138/162 2023-11-24T06:08:57.365 INFO:teuthology.orchestra.run.smithi092.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 139/162 2023-11-24T06:08:57.400 INFO:teuthology.orchestra.run.smithi092.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 140/162 2023-11-24T06:08:58.331 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 141/162 2023-11-24T06:08:58.381 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 141/162 2023-11-24T06:08:58.448 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 141/162 2023-11-24T06:08:59.802 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : cephfs-top-2:18.0.0-7484.ga13ea5ff.el9.noarch 142/162 2023-11-24T06:08:59.934 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : openssl-devel-1:3.0.7-25.el9.x86_64 143/162 2023-11-24T06:08:59.988 INFO:teuthology.orchestra.run.smithi092.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 144/162 2023-11-24T06:09:00.054 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 145/162 2023-11-24T06:09:00.199 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : flex-2.6.4-9.el9.x86_64 146/162 2023-11-24T06:09:00.233 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : bison-3.7.4-5.el9.x86_64 147/162 2023-11-24T06:09:00.266 INFO:teuthology.orchestra.run.smithi092.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 148/162 2023-11-24T06:09:00.300 INFO:teuthology.orchestra.run.smithi092.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 149/162 2023-11-24T06:09:00.342 INFO:teuthology.orchestra.run.smithi092.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 150/162 2023-11-24T06:09:00.342 INFO:teuthology.orchestra.run.smithi092.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 151/162 2023-11-24T06:09:00.474 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 151/162 2023-11-24T06:09:00.516 INFO:teuthology.orchestra.run.smithi092.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 152/162 2023-11-24T06:09:00.563 INFO:teuthology.orchestra.run.smithi092.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 153/162 2023-11-24T06:09:00.596 INFO:teuthology.orchestra.run.smithi092.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 154/162 2023-11-24T06:09:00.597 INFO:teuthology.orchestra.run.smithi092.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/162 2023-11-24T06:09:00.784 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/162 2023-11-24T06:09:00.814 INFO:teuthology.orchestra.run.smithi092.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 156/162 2023-11-24T06:09:00.854 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 157/162 2023-11-24T06:09:00.854 INFO:teuthology.orchestra.run.smithi092.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 157/162 2023-11-24T06:09:01.180 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 157/162 2023-11-24T06:09:01.180 INFO:teuthology.orchestra.run.smithi092.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 158/162 2023-11-24T06:09:01.347 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 158/162 2023-11-24T06:09:01.391 INFO:teuthology.orchestra.run.smithi092.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 159/162 2023-11-24T06:09:01.425 INFO:teuthology.orchestra.run.smithi092.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 160/162 2023-11-24T06:09:01.460 INFO:teuthology.orchestra.run.smithi092.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 161/162 2023-11-24T06:09:01.587 INFO:teuthology.orchestra.run.smithi092.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 162/162 2023-11-24T06:09:01.839 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: xfsdump-3.1.12-3.el9.x86_64 162/162 2023-11-24T06:09:01.839 INFO:teuthology.orchestra.run.smithi092.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2023-11-24T06:09:01.839 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-24T06:09:02.214 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : ceph-2:18.0.0-7484.ga13ea5ff.el9.x86_64 1/162 2023-11-24T06:09:02.214 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : ceph-base-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2/162 2023-11-24T06:09:02.214 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : ceph-common-2:18.0.0-7484.ga13ea5ff.el9.x86_64 3/162 2023-11-24T06:09:02.214 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : ceph-fuse-2:18.0.0-7484.ga13ea5ff.el9.x86_64 4/162 2023-11-24T06:09:02.214 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-7484.ga13ea 5/162 2023-11-24T06:09:02.215 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : ceph-mds-2:18.0.0-7484.ga13ea5ff.el9.x86_64 6/162 2023-11-24T06:09:02.215 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : ceph-mgr-2:18.0.0-7484.ga13ea5ff.el9.x86_64 7/162 2023-11-24T06:09:02.215 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : ceph-mon-2:18.0.0-7484.ga13ea5ff.el9.x86_64 8/162 2023-11-24T06:09:02.215 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : ceph-osd-2:18.0.0-7484.ga13ea5ff.el9.x86_64 9/162 2023-11-24T06:09:02.215 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : ceph-radosgw-2:18.0.0-7484.ga13ea5ff.el9.x86_64 10/162 2023-11-24T06:09:02.215 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : ceph-selinux-2:18.0.0-7484.ga13ea5ff.el9.x86_64 11/162 2023-11-24T06:09:02.215 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : ceph-test-2:18.0.0-7484.ga13ea5ff.el9.x86_64 12/162 2023-11-24T06:09:02.215 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : cephfs-mirror-2:18.0.0-7484.ga13ea5ff.el9.x86_64 13/162 2023-11-24T06:09:02.215 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : libcephfs-devel-2:18.0.0-7484.ga13ea5ff.el9.x86_ 14/162 2023-11-24T06:09:02.215 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : libcephfs2-2:18.0.0-7484.ga13ea5ff.el9.x86_64 15/162 2023-11-24T06:09:02.215 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : libcephsqlite-2:18.0.0-7484.ga13ea5ff.el9.x86_64 16/162 2023-11-24T06:09:02.215 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : librados-devel-2:18.0.0-7484.ga13ea5ff.el9.x86_6 17/162 2023-11-24T06:09:02.215 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : libradosstriper1-2:18.0.0-7484.ga13ea5ff.el9.x86 18/162 2023-11-24T06:09:02.216 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : librgw2-2:18.0.0-7484.ga13ea5ff.el9.x86_64 19/162 2023-11-24T06:09:02.216 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-ceph-argparse-2:18.0.0-7484.ga13ea5ff.el 20/162 2023-11-24T06:09:02.216 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-ceph-common-2:18.0.0-7484.ga13ea5ff.el9. 21/162 2023-11-24T06:09:02.216 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-cephfs-2:18.0.0-7484.ga13ea5ff.el9.x86_6 22/162 2023-11-24T06:09:02.216 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-rados-2:18.0.0-7484.ga13ea5ff.el9.x86_64 23/162 2023-11-24T06:09:02.216 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-rbd-2:18.0.0-7484.ga13ea5ff.el9.x86_64 24/162 2023-11-24T06:09:02.216 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-rgw-2:18.0.0-7484.ga13ea5ff.el9.x86_64 25/162 2023-11-24T06:09:02.216 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : rbd-fuse-2:18.0.0-7484.ga13ea5ff.el9.x86_64 26/162 2023-11-24T06:09:02.216 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : rbd-mirror-2:18.0.0-7484.ga13ea5ff.el9.x86_64 27/162 2023-11-24T06:09:02.216 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : rbd-nbd-2:18.0.0-7484.ga13ea5ff.el9.x86_64 28/162 2023-11-24T06:09:02.216 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-7484.ga13ea5ff. 29/162 2023-11-24T06:09:02.216 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-7484.ga13ea5ff.el9.noa 30/162 2023-11-24T06:09:02.216 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-7484.ga13ea5ff.el9.n 31/162 2023-11-24T06:09:02.217 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-7484.ga13 32/162 2023-11-24T06:09:02.217 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-7484.ga13ea5ff.el 33/162 2023-11-24T06:09:02.217 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : ceph-mgr-rook-2:18.0.0-7484.ga13ea5ff.el9.noarch 34/162 2023-11-24T06:09:02.217 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-7484.ga13ea5ff.e 35/162 2023-11-24T06:09:02.217 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : ceph-volume-2:18.0.0-7484.ga13ea5ff.el9.noarch 36/162 2023-11-24T06:09:02.217 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : cephadm-2:18.0.0-7484.ga13ea5ff.el9.noarch 37/162 2023-11-24T06:09:02.217 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : cephfs-top-2:18.0.0-7484.ga13ea5ff.el9.noarch 38/162 2023-11-24T06:09:02.217 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : automake-1.16.2-8.el9.noarch 39/162 2023-11-24T06:09:02.217 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : bison-3.7.4-5.el9.x86_64 40/162 2023-11-24T06:09:02.217 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 41/162 2023-11-24T06:09:02.217 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : elfutils-libelf-devel-0.190-1.el9.x86_64 42/162 2023-11-24T06:09:02.217 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/162 2023-11-24T06:09:02.217 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/162 2023-11-24T06:09:02.218 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/162 2023-11-24T06:09:02.218 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/162 2023-11-24T06:09:02.218 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : jq-1.6-16.el9.x86_64 47/162 2023-11-24T06:09:02.218 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 48/162 2023-11-24T06:09:02.218 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 49/162 2023-11-24T06:09:02.218 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 50/162 2023-11-24T06:09:02.218 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 51/162 2023-11-24T06:09:02.218 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 52/162 2023-11-24T06:09:02.218 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : libuuid-devel-2.37.4-15.el9.x86_64 53/162 2023-11-24T06:09:02.218 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 54/162 2023-11-24T06:09:02.218 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 55/162 2023-11-24T06:09:02.218 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 56/162 2023-11-24T06:09:02.218 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 57/162 2023-11-24T06:09:02.219 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 58/162 2023-11-24T06:09:02.219 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : postgresql-13.11-1.el9.x86_64 59/162 2023-11-24T06:09:02.219 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 60/162 2023-11-24T06:09:02.219 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 61/162 2023-11-24T06:09:02.219 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 62/162 2023-11-24T06:09:02.219 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 63/162 2023-11-24T06:09:02.219 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-jinja2-2.11.3-4.el9.noarch 64/162 2023-11-24T06:09:02.219 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 65/162 2023-11-24T06:09:02.219 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 66/162 2023-11-24T06:09:02.219 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 67/162 2023-11-24T06:09:02.219 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 68/162 2023-11-24T06:09:02.219 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 69/162 2023-11-24T06:09:02.220 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 70/162 2023-11-24T06:09:02.220 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 71/162 2023-11-24T06:09:02.220 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 72/162 2023-11-24T06:09:02.220 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 73/162 2023-11-24T06:09:02.220 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 74/162 2023-11-24T06:09:02.220 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 75/162 2023-11-24T06:09:02.220 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 76/162 2023-11-24T06:09:02.220 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 77/162 2023-11-24T06:09:02.220 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 78/162 2023-11-24T06:09:02.220 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 79/162 2023-11-24T06:09:02.220 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 80/162 2023-11-24T06:09:02.220 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : xfsprogs-devel-5.19.0-4.el9.x86_64 81/162 2023-11-24T06:09:02.220 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 82/162 2023-11-24T06:09:02.221 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 83/162 2023-11-24T06:09:02.221 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 84/162 2023-11-24T06:09:02.221 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : libgfortran-11.4.1-2.3.el9.x86_64 85/162 2023-11-24T06:09:02.221 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : libquadmath-11.4.1-2.3.el9.x86_64 86/162 2023-11-24T06:09:02.221 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 87/162 2023-11-24T06:09:02.221 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 88/162 2023-11-24T06:09:02.221 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-ply-3.11-14.el9.noarch 89/162 2023-11-24T06:09:02.221 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 90/162 2023-11-24T06:09:02.221 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-requests-2.25.1-7.el9.noarch 91/162 2023-11-24T06:09:02.221 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-urllib3-1.26.5-4.el9.noarch 92/162 2023-11-24T06:09:02.221 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : openssl-devel-1:3.0.7-25.el9.x86_64 93/162 2023-11-24T06:09:02.221 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 94/162 2023-11-24T06:09:02.221 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 95/162 2023-11-24T06:09:02.222 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : libarrow-9.0.0-9.el9.x86_64 96/162 2023-11-24T06:09:02.222 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : libarrow-doc-9.0.0-9.el9.noarch 97/162 2023-11-24T06:09:02.222 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 98/162 2023-11-24T06:09:02.222 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 99/162 2023-11-24T06:09:02.222 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : parquet-libs-9.0.0-9.el9.x86_64 100/162 2023-11-24T06:09:02.222 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-asyncssh-2.13.2-3.el9.noarch 101/162 2023-11-24T06:09:02.222 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 102/162 2023-11-24T06:09:02.222 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 103/162 2023-11-24T06:09:02.222 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 104/162 2023-11-24T06:09:02.222 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 105/162 2023-11-24T06:09:02.222 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 106/162 2023-11-24T06:09:02.222 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-google-auth-1:2.23.4-1.el9.noarch 107/162 2023-11-24T06:09:02.223 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 108/162 2023-11-24T06:09:02.223 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 109/162 2023-11-24T06:09:02.223 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 110/162 2023-11-24T06:09:02.223 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 111/162 2023-11-24T06:09:02.223 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 112/162 2023-11-24T06:09:02.223 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 113/162 2023-11-24T06:09:02.223 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 114/162 2023-11-24T06:09:02.223 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 115/162 2023-11-24T06:09:02.224 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 116/162 2023-11-24T06:09:02.224 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 117/162 2023-11-24T06:09:02.224 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 118/162 2023-11-24T06:09:02.224 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 119/162 2023-11-24T06:09:02.224 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 120/162 2023-11-24T06:09:02.224 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 121/162 2023-11-24T06:09:02.224 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 122/162 2023-11-24T06:09:02.224 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 123/162 2023-11-24T06:09:02.224 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 124/162 2023-11-24T06:09:02.224 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 125/162 2023-11-24T06:09:02.224 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 126/162 2023-11-24T06:09:02.224 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 127/162 2023-11-24T06:09:02.225 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 128/162 2023-11-24T06:09:02.225 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 129/162 2023-11-24T06:09:02.225 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 130/162 2023-11-24T06:09:02.225 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : re2-1:20211101-3.el9.x86_64 131/162 2023-11-24T06:09:02.225 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 132/162 2023-11-24T06:09:02.225 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : librados2-2:18.0.0-7484.ga13ea5ff.el9.x86_64 133/162 2023-11-24T06:09:02.225 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 134/162 2023-11-24T06:09:02.225 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : librbd1-2:18.0.0-7484.ga13ea5ff.el9.x86_64 135/162 2023-11-24T06:09:02.225 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 136/162 2023-11-24T06:09:02.225 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 137/162 2023-11-24T06:09:02.225 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 138/162 2023-11-24T06:09:02.226 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : NetworkManager-1:1.45.6-1.el9.x86_64 139/162 2023-11-24T06:09:02.226 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 140/162 2023-11-24T06:09:02.226 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : NetworkManager-libnm-1:1.45.6-1.el9.x86_64 141/162 2023-11-24T06:09:02.226 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 142/162 2023-11-24T06:09:02.226 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : NetworkManager-team-1:1.45.6-1.el9.x86_64 143/162 2023-11-24T06:09:02.226 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 144/162 2023-11-24T06:09:02.226 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : NetworkManager-tui-1:1.45.6-1.el9.x86_64 145/162 2023-11-24T06:09:02.226 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 146/162 2023-11-24T06:09:02.226 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : acl-2.3.1-4.el9.x86_64 147/162 2023-11-24T06:09:02.226 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : acl-2.3.1-3.el9.x86_64 148/162 2023-11-24T06:09:02.226 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : elfutils-debuginfod-client-0.190-1.el9.x86_64 149/162 2023-11-24T06:09:02.227 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 150/162 2023-11-24T06:09:02.227 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : elfutils-libelf-0.190-1.el9.x86_64 151/162 2023-11-24T06:09:02.227 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 152/162 2023-11-24T06:09:02.227 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : elfutils-libs-0.190-1.el9.x86_64 153/162 2023-11-24T06:09:02.227 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 154/162 2023-11-24T06:09:02.227 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 155/162 2023-11-24T06:09:02.227 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 156/162 2023-11-24T06:09:02.227 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 157/162 2023-11-24T06:09:02.227 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 158/162 2023-11-24T06:09:02.227 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : openssl-1:3.0.7-25.el9.x86_64 159/162 2023-11-24T06:09:02.228 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 160/162 2023-11-24T06:09:02.228 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : openssl-libs-1:3.0.7-25.el9.x86_64 161/162 2023-11-24T06:09:04.465 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 162/162 2023-11-24T06:09:04.465 INFO:teuthology.orchestra.run.smithi118.stdout: 2023-11-24T06:09:04.465 INFO:teuthology.orchestra.run.smithi118.stdout:Upgraded: 2023-11-24T06:09:04.465 INFO:teuthology.orchestra.run.smithi118.stdout: NetworkManager-1:1.45.6-1.el9.x86_64 2023-11-24T06:09:04.465 INFO:teuthology.orchestra.run.smithi118.stdout: NetworkManager-libnm-1:1.45.6-1.el9.x86_64 2023-11-24T06:09:04.466 INFO:teuthology.orchestra.run.smithi118.stdout: NetworkManager-team-1:1.45.6-1.el9.x86_64 2023-11-24T06:09:04.466 INFO:teuthology.orchestra.run.smithi118.stdout: NetworkManager-tui-1:1.45.6-1.el9.x86_64 2023-11-24T06:09:04.466 INFO:teuthology.orchestra.run.smithi118.stdout: acl-2.3.1-4.el9.x86_64 2023-11-24T06:09:04.466 INFO:teuthology.orchestra.run.smithi118.stdout: elfutils-debuginfod-client-0.190-1.el9.x86_64 2023-11-24T06:09:04.466 INFO:teuthology.orchestra.run.smithi118.stdout: elfutils-libelf-0.190-1.el9.x86_64 2023-11-24T06:09:04.466 INFO:teuthology.orchestra.run.smithi118.stdout: elfutils-libs-0.190-1.el9.x86_64 2023-11-24T06:09:04.466 INFO:teuthology.orchestra.run.smithi118.stdout: libacl-2.3.1-4.el9.x86_64 2023-11-24T06:09:04.466 INFO:teuthology.orchestra.run.smithi118.stdout: libacl-devel-2.3.1-4.el9.x86_64 2023-11-24T06:09:04.466 INFO:teuthology.orchestra.run.smithi118.stdout: librados2-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:09:04.466 INFO:teuthology.orchestra.run.smithi118.stdout: librbd1-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:09:04.466 INFO:teuthology.orchestra.run.smithi118.stdout: openssl-1:3.0.7-25.el9.x86_64 2023-11-24T06:09:04.467 INFO:teuthology.orchestra.run.smithi118.stdout: openssl-libs-1:3.0.7-25.el9.x86_64 2023-11-24T06:09:04.467 INFO:teuthology.orchestra.run.smithi118.stdout: xfsdump-3.1.12-4.el9.x86_64 2023-11-24T06:09:04.467 INFO:teuthology.orchestra.run.smithi118.stdout:Installed: 2023-11-24T06:09:04.467 INFO:teuthology.orchestra.run.smithi118.stdout: automake-1.16.2-8.el9.noarch 2023-11-24T06:09:04.467 INFO:teuthology.orchestra.run.smithi118.stdout: bison-3.7.4-5.el9.x86_64 2023-11-24T06:09:04.467 INFO:teuthology.orchestra.run.smithi118.stdout: boost-program-options-1.75.0-8.el9.x86_64 2023-11-24T06:09:04.467 INFO:teuthology.orchestra.run.smithi118.stdout: ceph-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:09:04.467 INFO:teuthology.orchestra.run.smithi118.stdout: ceph-base-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:09:04.467 INFO:teuthology.orchestra.run.smithi118.stdout: ceph-common-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:09:04.467 INFO:teuthology.orchestra.run.smithi118.stdout: ceph-fuse-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:09:04.467 INFO:teuthology.orchestra.run.smithi118.stdout: ceph-grafana-dashboards-2:18.0.0-7484.ga13ea5ff.el9.noarch 2023-11-24T06:09:04.467 INFO:teuthology.orchestra.run.smithi118.stdout: ceph-immutable-object-cache-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:09:04.468 INFO:teuthology.orchestra.run.smithi118.stdout: ceph-mds-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:09:04.468 INFO:teuthology.orchestra.run.smithi118.stdout: ceph-mgr-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:09:04.468 INFO:teuthology.orchestra.run.smithi118.stdout: ceph-mgr-cephadm-2:18.0.0-7484.ga13ea5ff.el9.noarch 2023-11-24T06:09:04.468 INFO:teuthology.orchestra.run.smithi118.stdout: ceph-mgr-dashboard-2:18.0.0-7484.ga13ea5ff.el9.noarch 2023-11-24T06:09:04.468 INFO:teuthology.orchestra.run.smithi118.stdout: ceph-mgr-diskprediction-local-2:18.0.0-7484.ga13ea5ff.el9.noarch 2023-11-24T06:09:04.468 INFO:teuthology.orchestra.run.smithi118.stdout: ceph-mgr-modules-core-2:18.0.0-7484.ga13ea5ff.el9.noarch 2023-11-24T06:09:04.468 INFO:teuthology.orchestra.run.smithi118.stdout: ceph-mgr-rook-2:18.0.0-7484.ga13ea5ff.el9.noarch 2023-11-24T06:09:04.468 INFO:teuthology.orchestra.run.smithi118.stdout: ceph-mon-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:09:04.468 INFO:teuthology.orchestra.run.smithi118.stdout: ceph-osd-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:09:04.468 INFO:teuthology.orchestra.run.smithi118.stdout: ceph-prometheus-alerts-2:18.0.0-7484.ga13ea5ff.el9.noarch 2023-11-24T06:09:04.468 INFO:teuthology.orchestra.run.smithi118.stdout: ceph-radosgw-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:09:04.468 INFO:teuthology.orchestra.run.smithi118.stdout: ceph-selinux-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:09:04.469 INFO:teuthology.orchestra.run.smithi118.stdout: ceph-test-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:09:04.469 INFO:teuthology.orchestra.run.smithi118.stdout: ceph-volume-2:18.0.0-7484.ga13ea5ff.el9.noarch 2023-11-24T06:09:04.469 INFO:teuthology.orchestra.run.smithi118.stdout: cephadm-2:18.0.0-7484.ga13ea5ff.el9.noarch 2023-11-24T06:09:04.469 INFO:teuthology.orchestra.run.smithi118.stdout: cephfs-mirror-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:09:04.469 INFO:teuthology.orchestra.run.smithi118.stdout: cephfs-top-2:18.0.0-7484.ga13ea5ff.el9.noarch 2023-11-24T06:09:04.469 INFO:teuthology.orchestra.run.smithi118.stdout: elfutils-libelf-devel-0.190-1.el9.x86_64 2023-11-24T06:09:04.469 INFO:teuthology.orchestra.run.smithi118.stdout: flex-2.6.4-9.el9.x86_64 2023-11-24T06:09:04.469 INFO:teuthology.orchestra.run.smithi118.stdout: flexiblas-3.0.4-8.el9.x86_64 2023-11-24T06:09:04.469 INFO:teuthology.orchestra.run.smithi118.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2023-11-24T06:09:04.469 INFO:teuthology.orchestra.run.smithi118.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2023-11-24T06:09:04.469 INFO:teuthology.orchestra.run.smithi118.stdout: fuse-2.9.9-15.el9.x86_64 2023-11-24T06:09:04.469 INFO:teuthology.orchestra.run.smithi118.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2023-11-24T06:09:04.470 INFO:teuthology.orchestra.run.smithi118.stdout: jq-1.6-16.el9.x86_64 2023-11-24T06:09:04.470 INFO:teuthology.orchestra.run.smithi118.stdout: libaio-devel-0.3.111-13.el9.x86_64 2023-11-24T06:09:04.470 INFO:teuthology.orchestra.run.smithi118.stdout: libarrow-9.0.0-9.el9.x86_64 2023-11-24T06:09:04.470 INFO:teuthology.orchestra.run.smithi118.stdout: libarrow-doc-9.0.0-9.el9.noarch 2023-11-24T06:09:04.470 INFO:teuthology.orchestra.run.smithi118.stdout: libcephfs-devel-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:09:04.470 INFO:teuthology.orchestra.run.smithi118.stdout: libcephfs2-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:09:04.470 INFO:teuthology.orchestra.run.smithi118.stdout: libcephsqlite-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:09:04.470 INFO:teuthology.orchestra.run.smithi118.stdout: libgfortran-11.4.1-2.3.el9.x86_64 2023-11-24T06:09:04.470 INFO:teuthology.orchestra.run.smithi118.stdout: liboath-2.6.7-2.el9.x86_64 2023-11-24T06:09:04.470 INFO:teuthology.orchestra.run.smithi118.stdout: libpmemobj-1.12.1-1.el9.x86_64 2023-11-24T06:09:04.470 INFO:teuthology.orchestra.run.smithi118.stdout: libquadmath-11.4.1-2.3.el9.x86_64 2023-11-24T06:09:04.470 INFO:teuthology.orchestra.run.smithi118.stdout: librabbitmq-0.11.0-7.el9.x86_64 2023-11-24T06:09:04.471 INFO:teuthology.orchestra.run.smithi118.stdout: librados-devel-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:09:04.471 INFO:teuthology.orchestra.run.smithi118.stdout: libradosstriper1-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:09:04.471 INFO:teuthology.orchestra.run.smithi118.stdout: librdkafka-1.6.1-102.el9.x86_64 2023-11-24T06:09:04.471 INFO:teuthology.orchestra.run.smithi118.stdout: librgw2-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:09:04.471 INFO:teuthology.orchestra.run.smithi118.stdout: libtool-2.4.6-45.el9.x86_64 2023-11-24T06:09:04.471 INFO:teuthology.orchestra.run.smithi118.stdout: libunwind-1.6.2-1.el9.x86_64 2023-11-24T06:09:04.471 INFO:teuthology.orchestra.run.smithi118.stdout: libuuid-devel-2.37.4-15.el9.x86_64 2023-11-24T06:09:04.471 INFO:teuthology.orchestra.run.smithi118.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2023-11-24T06:09:04.471 INFO:teuthology.orchestra.run.smithi118.stdout: lttng-ust-2.12.0-6.el9.x86_64 2023-11-24T06:09:04.471 INFO:teuthology.orchestra.run.smithi118.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2023-11-24T06:09:04.471 INFO:teuthology.orchestra.run.smithi118.stdout: openblas-0.3.21-2.el9.x86_64 2023-11-24T06:09:04.471 INFO:teuthology.orchestra.run.smithi118.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2023-11-24T06:09:04.472 INFO:teuthology.orchestra.run.smithi118.stdout: openssl-devel-1:3.0.7-25.el9.x86_64 2023-11-24T06:09:04.472 INFO:teuthology.orchestra.run.smithi118.stdout: parquet-libs-9.0.0-9.el9.x86_64 2023-11-24T06:09:04.472 INFO:teuthology.orchestra.run.smithi118.stdout: postgresql-13.11-1.el9.x86_64 2023-11-24T06:09:04.472 INFO:teuthology.orchestra.run.smithi118.stdout: postgresql-contrib-13.11-1.el9.x86_64 2023-11-24T06:09:04.472 INFO:teuthology.orchestra.run.smithi118.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2023-11-24T06:09:04.472 INFO:teuthology.orchestra.run.smithi118.stdout: postgresql-server-13.11-1.el9.x86_64 2023-11-24T06:09:04.472 INFO:teuthology.orchestra.run.smithi118.stdout: python3-asyncssh-2.13.2-3.el9.noarch 2023-11-24T06:09:04.472 INFO:teuthology.orchestra.run.smithi118.stdout: python3-babel-2.9.1-2.el9.noarch 2023-11-24T06:09:04.472 INFO:teuthology.orchestra.run.smithi118.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2023-11-24T06:09:04.472 INFO:teuthology.orchestra.run.smithi118.stdout: python3-cachetools-4.2.4-1.el9.noarch 2023-11-24T06:09:04.472 INFO:teuthology.orchestra.run.smithi118.stdout: python3-ceph-argparse-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:09:04.473 INFO:teuthology.orchestra.run.smithi118.stdout: python3-ceph-common-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:09:04.473 INFO:teuthology.orchestra.run.smithi118.stdout: python3-cephfs-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:09:04.473 INFO:teuthology.orchestra.run.smithi118.stdout: python3-certifi-2021.10.8-2.el9.noarch 2023-11-24T06:09:04.473 INFO:teuthology.orchestra.run.smithi118.stdout: python3-cffi-1.14.5-5.el9.x86_64 2023-11-24T06:09:04.473 INFO:teuthology.orchestra.run.smithi118.stdout: python3-cheroot-8.6.0-4.el9.noarch 2023-11-24T06:09:04.473 INFO:teuthology.orchestra.run.smithi118.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2023-11-24T06:09:04.473 INFO:teuthology.orchestra.run.smithi118.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2023-11-24T06:09:04.473 INFO:teuthology.orchestra.run.smithi118.stdout: python3-devel-3.9.17-2.el9.x86_64 2023-11-24T06:09:04.473 INFO:teuthology.orchestra.run.smithi118.stdout: python3-google-auth-1:2.23.4-1.el9.noarch 2023-11-24T06:09:04.473 INFO:teuthology.orchestra.run.smithi118.stdout: python3-jaraco-8.2.1-3.el9.noarch 2023-11-24T06:09:04.473 INFO:teuthology.orchestra.run.smithi118.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2023-11-24T06:09:04.473 INFO:teuthology.orchestra.run.smithi118.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2023-11-24T06:09:04.473 INFO:teuthology.orchestra.run.smithi118.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2023-11-24T06:09:04.474 INFO:teuthology.orchestra.run.smithi118.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2023-11-24T06:09:04.474 INFO:teuthology.orchestra.run.smithi118.stdout: python3-jinja2-2.11.3-4.el9.noarch 2023-11-24T06:09:04.474 INFO:teuthology.orchestra.run.smithi118.stdout: python3-jsonpatch-1.21-16.el9.noarch 2023-11-24T06:09:04.474 INFO:teuthology.orchestra.run.smithi118.stdout: python3-jsonpointer-2.0-4.el9.noarch 2023-11-24T06:09:04.474 INFO:teuthology.orchestra.run.smithi118.stdout: python3-jwt-2.4.0-1.el9.noarch 2023-11-24T06:09:04.474 INFO:teuthology.orchestra.run.smithi118.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2023-11-24T06:09:04.474 INFO:teuthology.orchestra.run.smithi118.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2023-11-24T06:09:04.474 INFO:teuthology.orchestra.run.smithi118.stdout: python3-logutils-0.3.5-21.el9.noarch 2023-11-24T06:09:04.474 INFO:teuthology.orchestra.run.smithi118.stdout: python3-mako-1.1.4-6.el9.noarch 2023-11-24T06:09:04.474 INFO:teuthology.orchestra.run.smithi118.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2023-11-24T06:09:04.474 INFO:teuthology.orchestra.run.smithi118.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2023-11-24T06:09:04.475 INFO:teuthology.orchestra.run.smithi118.stdout: python3-natsort-7.1.1-5.el9.noarch 2023-11-24T06:09:04.475 INFO:teuthology.orchestra.run.smithi118.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2023-11-24T06:09:04.475 INFO:teuthology.orchestra.run.smithi118.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2023-11-24T06:09:04.475 INFO:teuthology.orchestra.run.smithi118.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2023-11-24T06:09:04.475 INFO:teuthology.orchestra.run.smithi118.stdout: python3-pecan-1.4.2-3.el9.noarch 2023-11-24T06:09:04.475 INFO:teuthology.orchestra.run.smithi118.stdout: python3-ply-3.11-14.el9.noarch 2023-11-24T06:09:04.475 INFO:teuthology.orchestra.run.smithi118.stdout: python3-portend-3.1.0-2.el9.noarch 2023-11-24T06:09:04.475 INFO:teuthology.orchestra.run.smithi118.stdout: python3-prettytable-0.7.2-27.el9.noarch 2023-11-24T06:09:04.475 INFO:teuthology.orchestra.run.smithi118.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2023-11-24T06:09:04.475 INFO:teuthology.orchestra.run.smithi118.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2023-11-24T06:09:04.475 INFO:teuthology.orchestra.run.smithi118.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2023-11-24T06:09:04.475 INFO:teuthology.orchestra.run.smithi118.stdout: python3-pycparser-2.20-6.el9.noarch 2023-11-24T06:09:04.476 INFO:teuthology.orchestra.run.smithi118.stdout: python3-pytz-2021.1-5.el9.noarch 2023-11-24T06:09:04.476 INFO:teuthology.orchestra.run.smithi118.stdout: python3-rados-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:09:04.476 INFO:teuthology.orchestra.run.smithi118.stdout: python3-rbd-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:09:04.476 INFO:teuthology.orchestra.run.smithi118.stdout: python3-repoze-lru-0.7-16.el9.noarch 2023-11-24T06:09:04.476 INFO:teuthology.orchestra.run.smithi118.stdout: python3-requests-2.25.1-7.el9.noarch 2023-11-24T06:09:04.476 INFO:teuthology.orchestra.run.smithi118.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2023-11-24T06:09:04.476 INFO:teuthology.orchestra.run.smithi118.stdout: python3-rgw-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:09:04.476 INFO:teuthology.orchestra.run.smithi118.stdout: python3-routes-2.5.1-5.el9.noarch 2023-11-24T06:09:04.476 INFO:teuthology.orchestra.run.smithi118.stdout: python3-rsa-4.9-2.el9.noarch 2023-11-24T06:09:04.476 INFO:teuthology.orchestra.run.smithi118.stdout: python3-scipy-1.6.2-8.el9.x86_64 2023-11-24T06:09:04.476 INFO:teuthology.orchestra.run.smithi118.stdout: python3-tempora-5.0.0-2.el9.noarch 2023-11-24T06:09:04.476 INFO:teuthology.orchestra.run.smithi118.stdout: python3-toml-0.10.2-6.el9.noarch 2023-11-24T06:09:04.476 INFO:teuthology.orchestra.run.smithi118.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2023-11-24T06:09:04.477 INFO:teuthology.orchestra.run.smithi118.stdout: python3-urllib3-1.26.5-4.el9.noarch 2023-11-24T06:09:04.477 INFO:teuthology.orchestra.run.smithi118.stdout: python3-webob-1.8.7-6.el9.noarch 2023-11-24T06:09:04.477 INFO:teuthology.orchestra.run.smithi118.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2023-11-24T06:09:04.477 INFO:teuthology.orchestra.run.smithi118.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2023-11-24T06:09:04.477 INFO:teuthology.orchestra.run.smithi118.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2023-11-24T06:09:04.477 INFO:teuthology.orchestra.run.smithi118.stdout: rbd-fuse-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:09:04.477 INFO:teuthology.orchestra.run.smithi118.stdout: rbd-mirror-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:09:04.477 INFO:teuthology.orchestra.run.smithi118.stdout: rbd-nbd-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:09:04.477 INFO:teuthology.orchestra.run.smithi118.stdout: re2-1:20211101-3.el9.x86_64 2023-11-24T06:09:04.477 INFO:teuthology.orchestra.run.smithi118.stdout: socat-1.7.4.1-5.el9.x86_64 2023-11-24T06:09:04.477 INFO:teuthology.orchestra.run.smithi118.stdout: thrift-0.15.0-2.el9.x86_64 2023-11-24T06:09:04.477 INFO:teuthology.orchestra.run.smithi118.stdout: uuid-1.6.2-55.el9.x86_64 2023-11-24T06:09:04.477 INFO:teuthology.orchestra.run.smithi118.stdout: xfsprogs-devel-5.19.0-4.el9.x86_64 2023-11-24T06:09:04.478 INFO:teuthology.orchestra.run.smithi118.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2023-11-24T06:09:04.478 INFO:teuthology.orchestra.run.smithi118.stdout: zlib-devel-1.2.11-40.el9.x86_64 2023-11-24T06:09:04.478 INFO:teuthology.orchestra.run.smithi118.stdout: 2023-11-24T06:09:04.478 INFO:teuthology.orchestra.run.smithi118.stdout:Complete! 2023-11-24T06:09:05.101 DEBUG:teuthology.parallel:result is None 2023-11-24T06:09:06.517 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-2:18.0.0-7484.ga13ea5ff.el9.x86_64 1/162 2023-11-24T06:09:06.517 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-base-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2/162 2023-11-24T06:09:06.517 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-common-2:18.0.0-7484.ga13ea5ff.el9.x86_64 3/162 2023-11-24T06:09:06.517 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-fuse-2:18.0.0-7484.ga13ea5ff.el9.x86_64 4/162 2023-11-24T06:09:06.517 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-7484.ga13ea 5/162 2023-11-24T06:09:06.517 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-mds-2:18.0.0-7484.ga13ea5ff.el9.x86_64 6/162 2023-11-24T06:09:06.517 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-mgr-2:18.0.0-7484.ga13ea5ff.el9.x86_64 7/162 2023-11-24T06:09:06.517 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-mon-2:18.0.0-7484.ga13ea5ff.el9.x86_64 8/162 2023-11-24T06:09:06.517 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-osd-2:18.0.0-7484.ga13ea5ff.el9.x86_64 9/162 2023-11-24T06:09:06.517 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-radosgw-2:18.0.0-7484.ga13ea5ff.el9.x86_64 10/162 2023-11-24T06:09:06.518 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-selinux-2:18.0.0-7484.ga13ea5ff.el9.x86_64 11/162 2023-11-24T06:09:06.518 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-test-2:18.0.0-7484.ga13ea5ff.el9.x86_64 12/162 2023-11-24T06:09:06.518 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : cephfs-mirror-2:18.0.0-7484.ga13ea5ff.el9.x86_64 13/162 2023-11-24T06:09:06.518 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libcephfs-devel-2:18.0.0-7484.ga13ea5ff.el9.x86_ 14/162 2023-11-24T06:09:06.518 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libcephfs2-2:18.0.0-7484.ga13ea5ff.el9.x86_64 15/162 2023-11-24T06:09:06.518 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libcephsqlite-2:18.0.0-7484.ga13ea5ff.el9.x86_64 16/162 2023-11-24T06:09:06.518 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : librados-devel-2:18.0.0-7484.ga13ea5ff.el9.x86_6 17/162 2023-11-24T06:09:06.518 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libradosstriper1-2:18.0.0-7484.ga13ea5ff.el9.x86 18/162 2023-11-24T06:09:06.518 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : librgw2-2:18.0.0-7484.ga13ea5ff.el9.x86_64 19/162 2023-11-24T06:09:06.518 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-ceph-argparse-2:18.0.0-7484.ga13ea5ff.el 20/162 2023-11-24T06:09:06.518 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-ceph-common-2:18.0.0-7484.ga13ea5ff.el9. 21/162 2023-11-24T06:09:06.519 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-cephfs-2:18.0.0-7484.ga13ea5ff.el9.x86_6 22/162 2023-11-24T06:09:06.519 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-rados-2:18.0.0-7484.ga13ea5ff.el9.x86_64 23/162 2023-11-24T06:09:06.519 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-rbd-2:18.0.0-7484.ga13ea5ff.el9.x86_64 24/162 2023-11-24T06:09:06.519 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-rgw-2:18.0.0-7484.ga13ea5ff.el9.x86_64 25/162 2023-11-24T06:09:06.519 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : rbd-fuse-2:18.0.0-7484.ga13ea5ff.el9.x86_64 26/162 2023-11-24T06:09:06.519 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : rbd-mirror-2:18.0.0-7484.ga13ea5ff.el9.x86_64 27/162 2023-11-24T06:09:06.531 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : rbd-nbd-2:18.0.0-7484.ga13ea5ff.el9.x86_64 28/162 2023-11-24T06:09:06.531 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-7484.ga13ea5ff. 29/162 2023-11-24T06:09:06.531 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-7484.ga13ea5ff.el9.noa 30/162 2023-11-24T06:09:06.531 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-7484.ga13ea5ff.el9.n 31/162 2023-11-24T06:09:06.531 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-7484.ga13 32/162 2023-11-24T06:09:06.531 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-7484.ga13ea5ff.el 33/162 2023-11-24T06:09:06.531 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-mgr-rook-2:18.0.0-7484.ga13ea5ff.el9.noarch 34/162 2023-11-24T06:09:06.531 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-7484.ga13ea5ff.e 35/162 2023-11-24T06:09:06.532 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-volume-2:18.0.0-7484.ga13ea5ff.el9.noarch 36/162 2023-11-24T06:09:06.532 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : cephadm-2:18.0.0-7484.ga13ea5ff.el9.noarch 37/162 2023-11-24T06:09:06.532 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : cephfs-top-2:18.0.0-7484.ga13ea5ff.el9.noarch 38/162 2023-11-24T06:09:06.532 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : automake-1.16.2-8.el9.noarch 39/162 2023-11-24T06:09:06.532 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : bison-3.7.4-5.el9.x86_64 40/162 2023-11-24T06:09:06.532 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 41/162 2023-11-24T06:09:06.532 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : elfutils-libelf-devel-0.190-1.el9.x86_64 42/162 2023-11-24T06:09:06.532 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/162 2023-11-24T06:09:06.532 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/162 2023-11-24T06:09:06.532 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/162 2023-11-24T06:09:06.532 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/162 2023-11-24T06:09:06.533 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : jq-1.6-16.el9.x86_64 47/162 2023-11-24T06:09:06.533 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 48/162 2023-11-24T06:09:06.533 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 49/162 2023-11-24T06:09:06.533 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 50/162 2023-11-24T06:09:06.533 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 51/162 2023-11-24T06:09:06.533 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 52/162 2023-11-24T06:09:06.533 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libuuid-devel-2.37.4-15.el9.x86_64 53/162 2023-11-24T06:09:06.533 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 54/162 2023-11-24T06:09:06.533 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 55/162 2023-11-24T06:09:06.533 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 56/162 2023-11-24T06:09:06.533 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 57/162 2023-11-24T06:09:06.534 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 58/162 2023-11-24T06:09:06.534 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : postgresql-13.11-1.el9.x86_64 59/162 2023-11-24T06:09:06.534 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 60/162 2023-11-24T06:09:06.534 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 61/162 2023-11-24T06:09:06.534 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 62/162 2023-11-24T06:09:06.534 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 63/162 2023-11-24T06:09:06.534 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-jinja2-2.11.3-4.el9.noarch 64/162 2023-11-24T06:09:06.534 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 65/162 2023-11-24T06:09:06.534 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 66/162 2023-11-24T06:09:06.534 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 67/162 2023-11-24T06:09:06.534 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 68/162 2023-11-24T06:09:06.535 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 69/162 2023-11-24T06:09:06.535 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 70/162 2023-11-24T06:09:06.535 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 71/162 2023-11-24T06:09:06.535 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 72/162 2023-11-24T06:09:06.535 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 73/162 2023-11-24T06:09:06.535 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 74/162 2023-11-24T06:09:06.535 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 75/162 2023-11-24T06:09:06.535 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 76/162 2023-11-24T06:09:06.535 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 77/162 2023-11-24T06:09:06.535 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 78/162 2023-11-24T06:09:06.536 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 79/162 2023-11-24T06:09:06.536 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 80/162 2023-11-24T06:09:06.536 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : xfsprogs-devel-5.19.0-4.el9.x86_64 81/162 2023-11-24T06:09:06.536 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 82/162 2023-11-24T06:09:06.536 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 83/162 2023-11-24T06:09:06.536 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 84/162 2023-11-24T06:09:06.536 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libgfortran-11.4.1-2.3.el9.x86_64 85/162 2023-11-24T06:09:06.536 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libquadmath-11.4.1-2.3.el9.x86_64 86/162 2023-11-24T06:09:06.536 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 87/162 2023-11-24T06:09:06.536 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 88/162 2023-11-24T06:09:06.536 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-ply-3.11-14.el9.noarch 89/162 2023-11-24T06:09:06.537 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 90/162 2023-11-24T06:09:06.537 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-requests-2.25.1-7.el9.noarch 91/162 2023-11-24T06:09:06.537 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-urllib3-1.26.5-4.el9.noarch 92/162 2023-11-24T06:09:06.537 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : openssl-devel-1:3.0.7-25.el9.x86_64 93/162 2023-11-24T06:09:06.537 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 94/162 2023-11-24T06:09:06.537 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 95/162 2023-11-24T06:09:06.537 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libarrow-9.0.0-9.el9.x86_64 96/162 2023-11-24T06:09:06.537 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libarrow-doc-9.0.0-9.el9.noarch 97/162 2023-11-24T06:09:06.537 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 98/162 2023-11-24T06:09:06.537 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 99/162 2023-11-24T06:09:06.537 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : parquet-libs-9.0.0-9.el9.x86_64 100/162 2023-11-24T06:09:06.538 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-asyncssh-2.13.2-3.el9.noarch 101/162 2023-11-24T06:09:06.538 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 102/162 2023-11-24T06:09:06.538 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 103/162 2023-11-24T06:09:06.538 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 104/162 2023-11-24T06:09:06.538 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 105/162 2023-11-24T06:09:06.538 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 106/162 2023-11-24T06:09:06.538 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-google-auth-1:2.23.4-1.el9.noarch 107/162 2023-11-24T06:09:06.538 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 108/162 2023-11-24T06:09:06.538 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 109/162 2023-11-24T06:09:06.538 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 110/162 2023-11-24T06:09:06.539 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 111/162 2023-11-24T06:09:06.539 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 112/162 2023-11-24T06:09:06.539 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 113/162 2023-11-24T06:09:06.539 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 114/162 2023-11-24T06:09:06.539 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 115/162 2023-11-24T06:09:06.539 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 116/162 2023-11-24T06:09:06.539 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 117/162 2023-11-24T06:09:06.539 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 118/162 2023-11-24T06:09:06.539 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 119/162 2023-11-24T06:09:06.539 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 120/162 2023-11-24T06:09:06.539 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 121/162 2023-11-24T06:09:06.539 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 122/162 2023-11-24T06:09:06.539 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 123/162 2023-11-24T06:09:06.540 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 124/162 2023-11-24T06:09:06.540 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 125/162 2023-11-24T06:09:06.540 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 126/162 2023-11-24T06:09:06.540 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 127/162 2023-11-24T06:09:06.540 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 128/162 2023-11-24T06:09:06.540 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 129/162 2023-11-24T06:09:06.540 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 130/162 2023-11-24T06:09:06.540 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : re2-1:20211101-3.el9.x86_64 131/162 2023-11-24T06:09:06.540 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 132/162 2023-11-24T06:09:06.540 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : librados2-2:18.0.0-7484.ga13ea5ff.el9.x86_64 133/162 2023-11-24T06:09:06.541 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 134/162 2023-11-24T06:09:06.541 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : librbd1-2:18.0.0-7484.ga13ea5ff.el9.x86_64 135/162 2023-11-24T06:09:06.541 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 136/162 2023-11-24T06:09:06.541 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 137/162 2023-11-24T06:09:06.541 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 138/162 2023-11-24T06:09:06.541 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : NetworkManager-1:1.45.6-1.el9.x86_64 139/162 2023-11-24T06:09:06.541 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 140/162 2023-11-24T06:09:06.541 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : NetworkManager-libnm-1:1.45.6-1.el9.x86_64 141/162 2023-11-24T06:09:06.541 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 142/162 2023-11-24T06:09:06.541 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : NetworkManager-team-1:1.45.6-1.el9.x86_64 143/162 2023-11-24T06:09:06.541 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 144/162 2023-11-24T06:09:06.541 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : NetworkManager-tui-1:1.45.6-1.el9.x86_64 145/162 2023-11-24T06:09:06.542 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 146/162 2023-11-24T06:09:06.542 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : acl-2.3.1-4.el9.x86_64 147/162 2023-11-24T06:09:06.542 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : acl-2.3.1-3.el9.x86_64 148/162 2023-11-24T06:09:06.542 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : elfutils-debuginfod-client-0.190-1.el9.x86_64 149/162 2023-11-24T06:09:06.542 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 150/162 2023-11-24T06:09:06.542 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : elfutils-libelf-0.190-1.el9.x86_64 151/162 2023-11-24T06:09:06.542 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 152/162 2023-11-24T06:09:06.542 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : elfutils-libs-0.190-1.el9.x86_64 153/162 2023-11-24T06:09:06.542 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 154/162 2023-11-24T06:09:06.543 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 155/162 2023-11-24T06:09:06.543 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 156/162 2023-11-24T06:09:06.543 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 157/162 2023-11-24T06:09:06.543 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 158/162 2023-11-24T06:09:06.543 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : openssl-1:3.0.7-25.el9.x86_64 159/162 2023-11-24T06:09:06.543 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 160/162 2023-11-24T06:09:06.543 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : openssl-libs-1:3.0.7-25.el9.x86_64 161/162 2023-11-24T06:09:08.719 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 162/162 2023-11-24T06:09:08.719 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-24T06:09:08.719 INFO:teuthology.orchestra.run.smithi092.stdout:Upgraded: 2023-11-24T06:09:08.719 INFO:teuthology.orchestra.run.smithi092.stdout: NetworkManager-1:1.45.6-1.el9.x86_64 2023-11-24T06:09:08.719 INFO:teuthology.orchestra.run.smithi092.stdout: NetworkManager-libnm-1:1.45.6-1.el9.x86_64 2023-11-24T06:09:08.720 INFO:teuthology.orchestra.run.smithi092.stdout: NetworkManager-team-1:1.45.6-1.el9.x86_64 2023-11-24T06:09:08.720 INFO:teuthology.orchestra.run.smithi092.stdout: NetworkManager-tui-1:1.45.6-1.el9.x86_64 2023-11-24T06:09:08.720 INFO:teuthology.orchestra.run.smithi092.stdout: acl-2.3.1-4.el9.x86_64 2023-11-24T06:09:08.720 INFO:teuthology.orchestra.run.smithi092.stdout: elfutils-debuginfod-client-0.190-1.el9.x86_64 2023-11-24T06:09:08.720 INFO:teuthology.orchestra.run.smithi092.stdout: elfutils-libelf-0.190-1.el9.x86_64 2023-11-24T06:09:08.720 INFO:teuthology.orchestra.run.smithi092.stdout: elfutils-libs-0.190-1.el9.x86_64 2023-11-24T06:09:08.720 INFO:teuthology.orchestra.run.smithi092.stdout: libacl-2.3.1-4.el9.x86_64 2023-11-24T06:09:08.720 INFO:teuthology.orchestra.run.smithi092.stdout: libacl-devel-2.3.1-4.el9.x86_64 2023-11-24T06:09:08.720 INFO:teuthology.orchestra.run.smithi092.stdout: librados2-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:09:08.720 INFO:teuthology.orchestra.run.smithi092.stdout: librbd1-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:09:08.720 INFO:teuthology.orchestra.run.smithi092.stdout: openssl-1:3.0.7-25.el9.x86_64 2023-11-24T06:09:08.720 INFO:teuthology.orchestra.run.smithi092.stdout: openssl-libs-1:3.0.7-25.el9.x86_64 2023-11-24T06:09:08.721 INFO:teuthology.orchestra.run.smithi092.stdout: xfsdump-3.1.12-4.el9.x86_64 2023-11-24T06:09:08.721 INFO:teuthology.orchestra.run.smithi092.stdout:Installed: 2023-11-24T06:09:08.721 INFO:teuthology.orchestra.run.smithi092.stdout: automake-1.16.2-8.el9.noarch 2023-11-24T06:09:08.721 INFO:teuthology.orchestra.run.smithi092.stdout: bison-3.7.4-5.el9.x86_64 2023-11-24T06:09:08.721 INFO:teuthology.orchestra.run.smithi092.stdout: boost-program-options-1.75.0-8.el9.x86_64 2023-11-24T06:09:08.721 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:09:08.721 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-base-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:09:08.721 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-common-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:09:08.721 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-fuse-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:09:08.721 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-grafana-dashboards-2:18.0.0-7484.ga13ea5ff.el9.noarch 2023-11-24T06:09:08.721 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-immutable-object-cache-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:09:08.721 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mds-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:09:08.721 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:09:08.722 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-cephadm-2:18.0.0-7484.ga13ea5ff.el9.noarch 2023-11-24T06:09:08.722 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-dashboard-2:18.0.0-7484.ga13ea5ff.el9.noarch 2023-11-24T06:09:08.722 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-diskprediction-local-2:18.0.0-7484.ga13ea5ff.el9.noarch 2023-11-24T06:09:08.722 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-modules-core-2:18.0.0-7484.ga13ea5ff.el9.noarch 2023-11-24T06:09:08.722 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-rook-2:18.0.0-7484.ga13ea5ff.el9.noarch 2023-11-24T06:09:08.722 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mon-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:09:08.722 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-osd-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:09:08.722 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-prometheus-alerts-2:18.0.0-7484.ga13ea5ff.el9.noarch 2023-11-24T06:09:08.722 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-radosgw-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:09:08.722 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-selinux-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:09:08.722 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-test-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:09:08.722 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-volume-2:18.0.0-7484.ga13ea5ff.el9.noarch 2023-11-24T06:09:08.722 INFO:teuthology.orchestra.run.smithi092.stdout: cephadm-2:18.0.0-7484.ga13ea5ff.el9.noarch 2023-11-24T06:09:08.723 INFO:teuthology.orchestra.run.smithi092.stdout: cephfs-mirror-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:09:08.723 INFO:teuthology.orchestra.run.smithi092.stdout: cephfs-top-2:18.0.0-7484.ga13ea5ff.el9.noarch 2023-11-24T06:09:08.723 INFO:teuthology.orchestra.run.smithi092.stdout: elfutils-libelf-devel-0.190-1.el9.x86_64 2023-11-24T06:09:08.723 INFO:teuthology.orchestra.run.smithi092.stdout: flex-2.6.4-9.el9.x86_64 2023-11-24T06:09:08.723 INFO:teuthology.orchestra.run.smithi092.stdout: flexiblas-3.0.4-8.el9.x86_64 2023-11-24T06:09:08.723 INFO:teuthology.orchestra.run.smithi092.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2023-11-24T06:09:08.723 INFO:teuthology.orchestra.run.smithi092.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2023-11-24T06:09:08.723 INFO:teuthology.orchestra.run.smithi092.stdout: fuse-2.9.9-15.el9.x86_64 2023-11-24T06:09:08.723 INFO:teuthology.orchestra.run.smithi092.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2023-11-24T06:09:08.723 INFO:teuthology.orchestra.run.smithi092.stdout: jq-1.6-16.el9.x86_64 2023-11-24T06:09:08.723 INFO:teuthology.orchestra.run.smithi092.stdout: libaio-devel-0.3.111-13.el9.x86_64 2023-11-24T06:09:08.723 INFO:teuthology.orchestra.run.smithi092.stdout: libarrow-9.0.0-9.el9.x86_64 2023-11-24T06:09:08.724 INFO:teuthology.orchestra.run.smithi092.stdout: libarrow-doc-9.0.0-9.el9.noarch 2023-11-24T06:09:08.724 INFO:teuthology.orchestra.run.smithi092.stdout: libcephfs-devel-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:09:08.724 INFO:teuthology.orchestra.run.smithi092.stdout: libcephfs2-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:09:08.724 INFO:teuthology.orchestra.run.smithi092.stdout: libcephsqlite-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:09:08.724 INFO:teuthology.orchestra.run.smithi092.stdout: libgfortran-11.4.1-2.3.el9.x86_64 2023-11-24T06:09:08.724 INFO:teuthology.orchestra.run.smithi092.stdout: liboath-2.6.7-2.el9.x86_64 2023-11-24T06:09:08.724 INFO:teuthology.orchestra.run.smithi092.stdout: libpmemobj-1.12.1-1.el9.x86_64 2023-11-24T06:09:08.724 INFO:teuthology.orchestra.run.smithi092.stdout: libquadmath-11.4.1-2.3.el9.x86_64 2023-11-24T06:09:08.724 INFO:teuthology.orchestra.run.smithi092.stdout: librabbitmq-0.11.0-7.el9.x86_64 2023-11-24T06:09:08.724 INFO:teuthology.orchestra.run.smithi092.stdout: librados-devel-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:09:08.724 INFO:teuthology.orchestra.run.smithi092.stdout: libradosstriper1-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:09:08.724 INFO:teuthology.orchestra.run.smithi092.stdout: librdkafka-1.6.1-102.el9.x86_64 2023-11-24T06:09:08.724 INFO:teuthology.orchestra.run.smithi092.stdout: librgw2-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:09:08.725 INFO:teuthology.orchestra.run.smithi092.stdout: libtool-2.4.6-45.el9.x86_64 2023-11-24T06:09:08.725 INFO:teuthology.orchestra.run.smithi092.stdout: libunwind-1.6.2-1.el9.x86_64 2023-11-24T06:09:08.725 INFO:teuthology.orchestra.run.smithi092.stdout: libuuid-devel-2.37.4-15.el9.x86_64 2023-11-24T06:09:08.725 INFO:teuthology.orchestra.run.smithi092.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2023-11-24T06:09:08.725 INFO:teuthology.orchestra.run.smithi092.stdout: lttng-ust-2.12.0-6.el9.x86_64 2023-11-24T06:09:08.725 INFO:teuthology.orchestra.run.smithi092.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2023-11-24T06:09:08.725 INFO:teuthology.orchestra.run.smithi092.stdout: openblas-0.3.21-2.el9.x86_64 2023-11-24T06:09:08.725 INFO:teuthology.orchestra.run.smithi092.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2023-11-24T06:09:08.725 INFO:teuthology.orchestra.run.smithi092.stdout: openssl-devel-1:3.0.7-25.el9.x86_64 2023-11-24T06:09:08.725 INFO:teuthology.orchestra.run.smithi092.stdout: parquet-libs-9.0.0-9.el9.x86_64 2023-11-24T06:09:08.725 INFO:teuthology.orchestra.run.smithi092.stdout: postgresql-13.11-1.el9.x86_64 2023-11-24T06:09:08.725 INFO:teuthology.orchestra.run.smithi092.stdout: postgresql-contrib-13.11-1.el9.x86_64 2023-11-24T06:09:08.725 INFO:teuthology.orchestra.run.smithi092.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2023-11-24T06:09:08.726 INFO:teuthology.orchestra.run.smithi092.stdout: postgresql-server-13.11-1.el9.x86_64 2023-11-24T06:09:08.726 INFO:teuthology.orchestra.run.smithi092.stdout: python3-asyncssh-2.13.2-3.el9.noarch 2023-11-24T06:09:08.726 INFO:teuthology.orchestra.run.smithi092.stdout: python3-babel-2.9.1-2.el9.noarch 2023-11-24T06:09:08.726 INFO:teuthology.orchestra.run.smithi092.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2023-11-24T06:09:08.726 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cachetools-4.2.4-1.el9.noarch 2023-11-24T06:09:08.726 INFO:teuthology.orchestra.run.smithi092.stdout: python3-ceph-argparse-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:09:08.726 INFO:teuthology.orchestra.run.smithi092.stdout: python3-ceph-common-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:09:08.726 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cephfs-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:09:08.726 INFO:teuthology.orchestra.run.smithi092.stdout: python3-certifi-2021.10.8-2.el9.noarch 2023-11-24T06:09:08.726 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cffi-1.14.5-5.el9.x86_64 2023-11-24T06:09:08.726 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cheroot-8.6.0-4.el9.noarch 2023-11-24T06:09:08.727 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2023-11-24T06:09:08.727 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2023-11-24T06:09:08.727 INFO:teuthology.orchestra.run.smithi092.stdout: python3-devel-3.9.17-2.el9.x86_64 2023-11-24T06:09:08.727 INFO:teuthology.orchestra.run.smithi092.stdout: python3-google-auth-1:2.23.4-1.el9.noarch 2023-11-24T06:09:08.727 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco-8.2.1-3.el9.noarch 2023-11-24T06:09:08.727 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2023-11-24T06:09:08.727 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2023-11-24T06:09:08.727 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2023-11-24T06:09:08.727 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2023-11-24T06:09:08.727 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jinja2-2.11.3-4.el9.noarch 2023-11-24T06:09:08.727 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jsonpatch-1.21-16.el9.noarch 2023-11-24T06:09:08.727 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jsonpointer-2.0-4.el9.noarch 2023-11-24T06:09:08.727 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jwt-2.4.0-1.el9.noarch 2023-11-24T06:09:08.728 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2023-11-24T06:09:08.728 INFO:teuthology.orchestra.run.smithi092.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2023-11-24T06:09:08.728 INFO:teuthology.orchestra.run.smithi092.stdout: python3-logutils-0.3.5-21.el9.noarch 2023-11-24T06:09:08.728 INFO:teuthology.orchestra.run.smithi092.stdout: python3-mako-1.1.4-6.el9.noarch 2023-11-24T06:09:08.728 INFO:teuthology.orchestra.run.smithi092.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2023-11-24T06:09:08.728 INFO:teuthology.orchestra.run.smithi092.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2023-11-24T06:09:08.728 INFO:teuthology.orchestra.run.smithi092.stdout: python3-natsort-7.1.1-5.el9.noarch 2023-11-24T06:09:08.728 INFO:teuthology.orchestra.run.smithi092.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2023-11-24T06:09:08.728 INFO:teuthology.orchestra.run.smithi092.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2023-11-24T06:09:08.728 INFO:teuthology.orchestra.run.smithi092.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2023-11-24T06:09:08.728 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pecan-1.4.2-3.el9.noarch 2023-11-24T06:09:08.728 INFO:teuthology.orchestra.run.smithi092.stdout: python3-ply-3.11-14.el9.noarch 2023-11-24T06:09:08.729 INFO:teuthology.orchestra.run.smithi092.stdout: python3-portend-3.1.0-2.el9.noarch 2023-11-24T06:09:08.729 INFO:teuthology.orchestra.run.smithi092.stdout: python3-prettytable-0.7.2-27.el9.noarch 2023-11-24T06:09:08.729 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2023-11-24T06:09:08.729 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2023-11-24T06:09:08.729 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2023-11-24T06:09:08.729 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pycparser-2.20-6.el9.noarch 2023-11-24T06:09:08.729 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pytz-2021.1-5.el9.noarch 2023-11-24T06:09:08.729 INFO:teuthology.orchestra.run.smithi092.stdout: python3-rados-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:09:08.729 INFO:teuthology.orchestra.run.smithi092.stdout: python3-rbd-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:09:08.729 INFO:teuthology.orchestra.run.smithi092.stdout: python3-repoze-lru-0.7-16.el9.noarch 2023-11-24T06:09:08.729 INFO:teuthology.orchestra.run.smithi092.stdout: python3-requests-2.25.1-7.el9.noarch 2023-11-24T06:09:08.729 INFO:teuthology.orchestra.run.smithi092.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2023-11-24T06:09:08.729 INFO:teuthology.orchestra.run.smithi092.stdout: python3-rgw-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:09:08.730 INFO:teuthology.orchestra.run.smithi092.stdout: python3-routes-2.5.1-5.el9.noarch 2023-11-24T06:09:08.730 INFO:teuthology.orchestra.run.smithi092.stdout: python3-rsa-4.9-2.el9.noarch 2023-11-24T06:09:08.730 INFO:teuthology.orchestra.run.smithi092.stdout: python3-scipy-1.6.2-8.el9.x86_64 2023-11-24T06:09:08.730 INFO:teuthology.orchestra.run.smithi092.stdout: python3-tempora-5.0.0-2.el9.noarch 2023-11-24T06:09:08.730 INFO:teuthology.orchestra.run.smithi092.stdout: python3-toml-0.10.2-6.el9.noarch 2023-11-24T06:09:08.730 INFO:teuthology.orchestra.run.smithi092.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2023-11-24T06:09:08.730 INFO:teuthology.orchestra.run.smithi092.stdout: python3-urllib3-1.26.5-4.el9.noarch 2023-11-24T06:09:08.730 INFO:teuthology.orchestra.run.smithi092.stdout: python3-webob-1.8.7-6.el9.noarch 2023-11-24T06:09:08.730 INFO:teuthology.orchestra.run.smithi092.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2023-11-24T06:09:08.730 INFO:teuthology.orchestra.run.smithi092.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2023-11-24T06:09:08.730 INFO:teuthology.orchestra.run.smithi092.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2023-11-24T06:09:08.730 INFO:teuthology.orchestra.run.smithi092.stdout: rbd-fuse-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:09:08.731 INFO:teuthology.orchestra.run.smithi092.stdout: rbd-mirror-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:09:08.731 INFO:teuthology.orchestra.run.smithi092.stdout: rbd-nbd-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:09:08.731 INFO:teuthology.orchestra.run.smithi092.stdout: re2-1:20211101-3.el9.x86_64 2023-11-24T06:09:08.731 INFO:teuthology.orchestra.run.smithi092.stdout: socat-1.7.4.1-5.el9.x86_64 2023-11-24T06:09:08.731 INFO:teuthology.orchestra.run.smithi092.stdout: thrift-0.15.0-2.el9.x86_64 2023-11-24T06:09:08.731 INFO:teuthology.orchestra.run.smithi092.stdout: uuid-1.6.2-55.el9.x86_64 2023-11-24T06:09:08.731 INFO:teuthology.orchestra.run.smithi092.stdout: xfsprogs-devel-5.19.0-4.el9.x86_64 2023-11-24T06:09:08.731 INFO:teuthology.orchestra.run.smithi092.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2023-11-24T06:09:08.731 INFO:teuthology.orchestra.run.smithi092.stdout: zlib-devel-1.2.11-40.el9.x86_64 2023-11-24T06:09:08.732 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-24T06:09:08.732 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2023-11-24T06:09:09.301 DEBUG:teuthology.parallel:result is None 2023-11-24T06:09:09.301 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a13ea5ff00cb4df7a9cc941166ddeb358881144c 2023-11-24T06:09:09.604 DEBUG:teuthology.orchestra.run.smithi092:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-11-24T06:09:13.103 INFO:teuthology.orchestra.run.smithi092.stdout:18.0.0-7484.ga13ea5ff.el9 2023-11-24T06:09:13.104 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-7484.ga13ea5ff.el9 2023-11-24T06:09:13.104 INFO:teuthology.task.install:The correct ceph version 18.0.0-7484.ga13ea5ff is installed. 2023-11-24T06:09:13.104 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a13ea5ff00cb4df7a9cc941166ddeb358881144c 2023-11-24T06:09:13.239 DEBUG:teuthology.orchestra.run.smithi118:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-11-24T06:09:13.313 INFO:teuthology.orchestra.run.smithi118.stdout:18.0.0-7484.ga13ea5ff.el9 2023-11-24T06:09:13.313 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-7484.ga13ea5ff.el9 2023-11-24T06:09:13.313 INFO:teuthology.task.install:The correct ceph version 18.0.0-7484.ga13ea5ff is installed. 2023-11-24T06:09:13.314 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-11-24T06:09:13.314 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-11-24T06:09:13.314 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-11-24T06:09:13.394 DEBUG:teuthology.orchestra.run.smithi118:> set -ex 2023-11-24T06:09:13.394 DEBUG:teuthology.orchestra.run.smithi118:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-11-24T06:09:13.501 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-11-24T06:09:13.501 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-11-24T06:09:13.501 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/usr/bin/daemon-helper 2023-11-24T06:09:13.578 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-11-24T06:09:13.694 DEBUG:teuthology.orchestra.run.smithi118:> set -ex 2023-11-24T06:09:13.694 DEBUG:teuthology.orchestra.run.smithi118:> sudo dd of=/usr/bin/daemon-helper 2023-11-24T06:09:13.800 DEBUG:teuthology.orchestra.run.smithi118:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-11-24T06:09:13.895 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-11-24T06:09:13.895 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-11-24T06:09:13.896 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/usr/bin/adjust-ulimits 2023-11-24T06:09:13.972 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-11-24T06:09:14.092 DEBUG:teuthology.orchestra.run.smithi118:> set -ex 2023-11-24T06:09:14.092 DEBUG:teuthology.orchestra.run.smithi118:> sudo dd of=/usr/bin/adjust-ulimits 2023-11-24T06:09:14.196 DEBUG:teuthology.orchestra.run.smithi118:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-11-24T06:09:14.340 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2023-11-24T06:09:14.340 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-11-24T06:09:14.340 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/usr/bin/stdin-killer 2023-11-24T06:09:14.421 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-11-24T06:09:14.540 DEBUG:teuthology.orchestra.run.smithi118:> set -ex 2023-11-24T06:09:14.540 DEBUG:teuthology.orchestra.run.smithi118:> sudo dd of=/usr/bin/stdin-killer 2023-11-24T06:09:14.642 DEBUG:teuthology.orchestra.run.smithi118:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-11-24T06:09:14.738 INFO:teuthology.run_tasks:Running task ceph... 2023-11-24T06:09:14.800 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2023-11-24T06:09:14.800 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod 777 /var/log/ceph 2023-11-24T06:09:14.806 DEBUG:teuthology.orchestra.run.smithi118:> sudo chmod 777 /var/log/ceph 2023-11-24T06:09:14.908 INFO:tasks.ceph:Disabling ceph logrotate... 2023-11-24T06:09:14.908 DEBUG:teuthology.orchestra.run.smithi092:> sudo rm -f -- /etc/logrotate.d/ceph 2023-11-24T06:09:14.917 DEBUG:teuthology.orchestra.run.smithi118:> sudo rm -f -- /etc/logrotate.d/ceph 2023-11-24T06:09:15.011 INFO:tasks.ceph:Creating extra log directories... 2023-11-24T06:09:15.011 DEBUG:teuthology.orchestra.run.smithi092:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-11-24T06:09:15.020 DEBUG:teuthology.orchestra.run.smithi118:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-11-24T06:09:15.123 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2023-11-24T06:09:15.123 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2023-11-24T06:09:15.123 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2023-11-24T06:09:15.123 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-11-24T06:09:15.123 DEBUG:teuthology.orchestra.run.smithi092:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-11-24T06:09:15.179 DEBUG:teuthology.orchestra.run.smithi092:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-11-24T06:09:15.422 DEBUG:teuthology.orchestra.run.smithi092:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2023-11-24T06:09:15.529 DEBUG:teuthology.orchestra.run.smithi118:> set -ex 2023-11-24T06:09:15.530 DEBUG:teuthology.orchestra.run.smithi118:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-11-24T06:09:15.587 DEBUG:teuthology.orchestra.run.smithi118:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-11-24T06:09:15.858 DEBUG:teuthology.orchestra.run.smithi118:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2023-11-24T06:09:15.968 INFO:tasks.ceph:Creating ceph cluster ceph... 2023-11-24T06:09:15.968 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2023-11-24T06:09:15.968 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/rishabh-2023-11-24_05:41:33-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7465546', 'branch': 'rishabh-mgr-vol-misc', 'description': 'fs:volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/basic}}', 'email': None, 'first_in_suite': False, 'job_id': '7465546', 'kernel': {'flavor': 'default', 'kdb': True, 'sha1': '8247bbb5b470f7e2a1f3eefcc16e24cf1338f606'}, 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'rishabh-2023-11-24_05:41:33-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'rishabh-mgr-vol-misc'}, 'ceph': {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'sha1': 'a13ea5ff00cb4df7a9cc941166ddeb358881144c'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'a13ea5ff00cb4df7a9cc941166ddeb358881144c'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'rishabh-mgr-vol-misc', 'sha1': 'a13ea5ff00cb4df7a9cc941166ddeb358881144c'}}, 'owner': 'scheduled_rishabh@teuthology', 'priority': 50, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 3305, 'sha1': 'a13ea5ff00cb4df7a9cc941166ddeb358881144c', 'sleep_before_teardown': 0, 'suite': 'fs:volumes', 'suite_branch': 'rishabh-mgr-vol-misc', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_a13ea5ff00cb4df7a9cc941166ddeb358881144c/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': 'a13ea5ff00cb4df7a9cc941166ddeb358881144c', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi092.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBM/6AgknJyX69rVjCUaIgHxhqMmdlXcpfMRywOSR//a3BdC53SvAjaxmI9LSZboTEgD8EzutTJWZfDUDZG21qko=', 'smithi118.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDtNpRTYzJZfvzdUULaqo1CEoH8cnZH6jMFDwICFAvv6nLJJyaEH9q71cRsl+4OgPQO1/cgIH3Ch5W2Ttk4w7N8='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'flavor': 'default', 'kdb': True, 'sha1': '8247bbb5b470f7e2a1f3eefcc16e24cf1338f606'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'a13ea5ff00cb4df7a9cc941166ddeb358881144c'}}, {'ceph': {'create_rbd_pool': False, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'sha1': 'a13ea5ff00cb4df7a9cc941166ddeb358881144c', 'cluster': 'ceph'}}, {'ceph-fuse': None}, {'cephfs_test_runner': {'fail_on_skip': False, 'modules': ['tasks.cephfs.test_volumes.TestVolumes', 'tasks.cephfs.test_volumes.TestSubvolumeGroups', 'tasks.cephfs.test_volumes.TestSubvolumes', 'tasks.cephfs.test_subvolume.TestSubvolume']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ['local function is_kupstream() return false end', 'local function is_kdistro() return false end', 'local function is_fuse() return true end', "local function syntax_version() return '' end"]}, 'teuthology_branch': 'main', 'teuthology_sha1': '2442239d2653456406c25ae0c71b689c8f2657b6', 'timestamp': '2023-11-24_05:41:33', 'tube': 'smithi', 'user': 'rishabh', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.232157'} 2023-11-24T06:09:15.969 DEBUG:teuthology.orchestra.run.smithi092:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-11-24T06:09:15.976 DEBUG:teuthology.orchestra.run.smithi118:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-11-24T06:09:16.035 DEBUG:teuthology.orchestra.run.smithi092:> sudo install -d -m0777 -- /var/run/ceph 2023-11-24T06:09:16.041 DEBUG:teuthology.orchestra.run.smithi118:> sudo install -d -m0777 -- /var/run/ceph 2023-11-24T06:09:16.137 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-11-24T06:09:16.137 DEBUG:teuthology.orchestra.run.smithi092:> dd if=/scratch_devs of=/dev/stdout 2023-11-24T06:09:16.193 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-11-24T06:09:16.194 DEBUG:teuthology.orchestra.run.smithi092:> stat /dev/vg_nvme/lv_1 2023-11-24T06:09:16.247 INFO:teuthology.orchestra.run.smithi092.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-11-24T06:09:16.247 INFO:teuthology.orchestra.run.smithi092.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-24T06:09:16.247 INFO:teuthology.orchestra.run.smithi092.stdout:Device: 5h/5d Inode: 633 Links: 1 2023-11-24T06:09:16.247 INFO:teuthology.orchestra.run.smithi092.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-24T06:09:16.247 INFO:teuthology.orchestra.run.smithi092.stdout:Context: system_u:object_r:device_t:s0 2023-11-24T06:09:16.247 INFO:teuthology.orchestra.run.smithi092.stdout:Access: 2023-11-24 06:09:02.278241865 +0000 2023-11-24T06:09:16.247 INFO:teuthology.orchestra.run.smithi092.stdout:Modify: 2023-11-24 06:05:59.760992101 +0000 2023-11-24T06:09:16.247 INFO:teuthology.orchestra.run.smithi092.stdout:Change: 2023-11-24 06:05:59.760992101 +0000 2023-11-24T06:09:16.247 INFO:teuthology.orchestra.run.smithi092.stdout: Birth: 2023-11-24 06:05:59.760992101 +0000 2023-11-24T06:09:16.248 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-11-24T06:09:16.335 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records in 2023-11-24T06:09:16.335 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records out 2023-11-24T06:09:16.335 INFO:teuthology.orchestra.run.smithi092.stderr:512 bytes copied, 0.000353028 s, 1.5 MB/s 2023-11-24T06:09:16.338 DEBUG:teuthology.orchestra.run.smithi092:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-11-24T06:09:16.395 DEBUG:teuthology.orchestra.run.smithi092:> stat /dev/vg_nvme/lv_2 2023-11-24T06:09:16.445 INFO:teuthology.orchestra.run.smithi092.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-11-24T06:09:16.445 INFO:teuthology.orchestra.run.smithi092.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-24T06:09:16.445 INFO:teuthology.orchestra.run.smithi092.stdout:Device: 5h/5d Inode: 653 Links: 1 2023-11-24T06:09:16.445 INFO:teuthology.orchestra.run.smithi092.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-24T06:09:16.446 INFO:teuthology.orchestra.run.smithi092.stdout:Context: system_u:object_r:device_t:s0 2023-11-24T06:09:16.446 INFO:teuthology.orchestra.run.smithi092.stdout:Access: 2023-11-24 06:09:02.280241808 +0000 2023-11-24T06:09:16.446 INFO:teuthology.orchestra.run.smithi092.stdout:Modify: 2023-11-24 06:06:00.785962805 +0000 2023-11-24T06:09:16.446 INFO:teuthology.orchestra.run.smithi092.stdout:Change: 2023-11-24 06:06:00.785962805 +0000 2023-11-24T06:09:16.446 INFO:teuthology.orchestra.run.smithi092.stdout: Birth: 2023-11-24 06:06:00.785962805 +0000 2023-11-24T06:09:16.446 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-11-24T06:09:16.536 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records in 2023-11-24T06:09:16.536 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records out 2023-11-24T06:09:16.537 INFO:teuthology.orchestra.run.smithi092.stderr:512 bytes copied, 0.000316677 s, 1.6 MB/s 2023-11-24T06:09:16.538 DEBUG:teuthology.orchestra.run.smithi092:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-11-24T06:09:16.592 DEBUG:teuthology.orchestra.run.smithi092:> stat /dev/vg_nvme/lv_3 2023-11-24T06:09:16.640 INFO:teuthology.orchestra.run.smithi092.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-11-24T06:09:16.641 INFO:teuthology.orchestra.run.smithi092.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-24T06:09:16.641 INFO:teuthology.orchestra.run.smithi092.stdout:Device: 5h/5d Inode: 671 Links: 1 2023-11-24T06:09:16.641 INFO:teuthology.orchestra.run.smithi092.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-24T06:09:16.641 INFO:teuthology.orchestra.run.smithi092.stdout:Context: system_u:object_r:device_t:s0 2023-11-24T06:09:16.641 INFO:teuthology.orchestra.run.smithi092.stdout:Access: 2023-11-24 06:09:02.280241808 +0000 2023-11-24T06:09:16.641 INFO:teuthology.orchestra.run.smithi092.stdout:Modify: 2023-11-24 06:06:01.802933736 +0000 2023-11-24T06:09:16.641 INFO:teuthology.orchestra.run.smithi092.stdout:Change: 2023-11-24 06:06:01.802933736 +0000 2023-11-24T06:09:16.641 INFO:teuthology.orchestra.run.smithi092.stdout: Birth: 2023-11-24 06:06:01.802933736 +0000 2023-11-24T06:09:16.641 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-11-24T06:09:16.729 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records in 2023-11-24T06:09:16.729 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records out 2023-11-24T06:09:16.729 INFO:teuthology.orchestra.run.smithi092.stderr:512 bytes copied, 0.00030502 s, 1.7 MB/s 2023-11-24T06:09:16.733 DEBUG:teuthology.orchestra.run.smithi092:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-11-24T06:09:16.789 DEBUG:teuthology.orchestra.run.smithi092:> stat /dev/vg_nvme/lv_4 2023-11-24T06:09:16.838 INFO:teuthology.orchestra.run.smithi092.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-11-24T06:09:16.839 INFO:teuthology.orchestra.run.smithi092.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-24T06:09:16.839 INFO:teuthology.orchestra.run.smithi092.stdout:Device: 5h/5d Inode: 690 Links: 1 2023-11-24T06:09:16.839 INFO:teuthology.orchestra.run.smithi092.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-24T06:09:16.839 INFO:teuthology.orchestra.run.smithi092.stdout:Context: system_u:object_r:device_t:s0 2023-11-24T06:09:16.839 INFO:teuthology.orchestra.run.smithi092.stdout:Access: 2023-11-24 06:09:02.281241779 +0000 2023-11-24T06:09:16.839 INFO:teuthology.orchestra.run.smithi092.stdout:Modify: 2023-11-24 06:06:02.805905068 +0000 2023-11-24T06:09:16.839 INFO:teuthology.orchestra.run.smithi092.stdout:Change: 2023-11-24 06:06:02.805905068 +0000 2023-11-24T06:09:16.839 INFO:teuthology.orchestra.run.smithi092.stdout: Birth: 2023-11-24 06:06:02.805905068 +0000 2023-11-24T06:09:16.839 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-11-24T06:09:16.928 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records in 2023-11-24T06:09:16.928 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records out 2023-11-24T06:09:16.928 INFO:teuthology.orchestra.run.smithi092.stderr:512 bytes copied, 0.000301926 s, 1.7 MB/s 2023-11-24T06:09:16.931 DEBUG:teuthology.orchestra.run.smithi092:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-11-24T06:09:16.990 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-11-24T06:09:16.991 DEBUG:teuthology.orchestra.run.smithi118:> set -ex 2023-11-24T06:09:16.991 DEBUG:teuthology.orchestra.run.smithi118:> dd if=/scratch_devs of=/dev/stdout 2023-11-24T06:09:17.050 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-11-24T06:09:17.050 DEBUG:teuthology.orchestra.run.smithi118:> stat /dev/vg_nvme/lv_1 2023-11-24T06:09:17.103 INFO:teuthology.orchestra.run.smithi118.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-11-24T06:09:17.103 INFO:teuthology.orchestra.run.smithi118.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-24T06:09:17.103 INFO:teuthology.orchestra.run.smithi118.stdout:Device: 5h/5d Inode: 638 Links: 1 2023-11-24T06:09:17.103 INFO:teuthology.orchestra.run.smithi118.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-24T06:09:17.103 INFO:teuthology.orchestra.run.smithi118.stdout:Context: system_u:object_r:device_t:s0 2023-11-24T06:09:17.104 INFO:teuthology.orchestra.run.smithi118.stdout:Access: 2023-11-24 06:08:57.151779340 +0000 2023-11-24T06:09:17.104 INFO:teuthology.orchestra.run.smithi118.stdout:Modify: 2023-11-24 06:06:01.306891502 +0000 2023-11-24T06:09:17.104 INFO:teuthology.orchestra.run.smithi118.stdout:Change: 2023-11-24 06:06:01.306891502 +0000 2023-11-24T06:09:17.104 INFO:teuthology.orchestra.run.smithi118.stdout: Birth: 2023-11-24 06:06:01.306891502 +0000 2023-11-24T06:09:17.104 DEBUG:teuthology.orchestra.run.smithi118:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-11-24T06:09:17.191 INFO:teuthology.orchestra.run.smithi118.stderr:1+0 records in 2023-11-24T06:09:17.192 INFO:teuthology.orchestra.run.smithi118.stderr:1+0 records out 2023-11-24T06:09:17.192 INFO:teuthology.orchestra.run.smithi118.stderr:512 bytes copied, 0.000199236 s, 2.6 MB/s 2023-11-24T06:09:17.195 DEBUG:teuthology.orchestra.run.smithi118:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-11-24T06:09:17.250 DEBUG:teuthology.orchestra.run.smithi118:> stat /dev/vg_nvme/lv_2 2023-11-24T06:09:17.299 INFO:teuthology.orchestra.run.smithi118.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-11-24T06:09:17.299 INFO:teuthology.orchestra.run.smithi118.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-24T06:09:17.299 INFO:teuthology.orchestra.run.smithi118.stdout:Device: 5h/5d Inode: 657 Links: 1 2023-11-24T06:09:17.299 INFO:teuthology.orchestra.run.smithi118.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-24T06:09:17.300 INFO:teuthology.orchestra.run.smithi118.stdout:Context: system_u:object_r:device_t:s0 2023-11-24T06:09:17.300 INFO:teuthology.orchestra.run.smithi118.stdout:Access: 2023-11-24 06:08:57.152779318 +0000 2023-11-24T06:09:17.300 INFO:teuthology.orchestra.run.smithi118.stdout:Modify: 2023-11-24 06:06:02.295869503 +0000 2023-11-24T06:09:17.300 INFO:teuthology.orchestra.run.smithi118.stdout:Change: 2023-11-24 06:06:02.295869503 +0000 2023-11-24T06:09:17.300 INFO:teuthology.orchestra.run.smithi118.stdout: Birth: 2023-11-24 06:06:02.295869503 +0000 2023-11-24T06:09:17.300 DEBUG:teuthology.orchestra.run.smithi118:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-11-24T06:09:17.390 INFO:teuthology.orchestra.run.smithi118.stderr:1+0 records in 2023-11-24T06:09:17.390 INFO:teuthology.orchestra.run.smithi118.stderr:1+0 records out 2023-11-24T06:09:17.390 INFO:teuthology.orchestra.run.smithi118.stderr:512 bytes copied, 0.000254324 s, 2.0 MB/s 2023-11-24T06:09:17.393 DEBUG:teuthology.orchestra.run.smithi118:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-11-24T06:09:17.449 DEBUG:teuthology.orchestra.run.smithi118:> stat /dev/vg_nvme/lv_3 2023-11-24T06:09:17.500 INFO:teuthology.orchestra.run.smithi118.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-11-24T06:09:17.500 INFO:teuthology.orchestra.run.smithi118.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-24T06:09:17.500 INFO:teuthology.orchestra.run.smithi118.stdout:Device: 5h/5d Inode: 675 Links: 1 2023-11-24T06:09:17.500 INFO:teuthology.orchestra.run.smithi118.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-24T06:09:17.501 INFO:teuthology.orchestra.run.smithi118.stdout:Context: system_u:object_r:device_t:s0 2023-11-24T06:09:17.501 INFO:teuthology.orchestra.run.smithi118.stdout:Access: 2023-11-24 06:08:57.153779296 +0000 2023-11-24T06:09:17.501 INFO:teuthology.orchestra.run.smithi118.stdout:Modify: 2023-11-24 06:06:03.345846129 +0000 2023-11-24T06:09:17.501 INFO:teuthology.orchestra.run.smithi118.stdout:Change: 2023-11-24 06:06:03.345846129 +0000 2023-11-24T06:09:17.501 INFO:teuthology.orchestra.run.smithi118.stdout: Birth: 2023-11-24 06:06:03.345846129 +0000 2023-11-24T06:09:17.501 DEBUG:teuthology.orchestra.run.smithi118:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-11-24T06:09:17.594 INFO:teuthology.orchestra.run.smithi118.stderr:1+0 records in 2023-11-24T06:09:17.594 INFO:teuthology.orchestra.run.smithi118.stderr:1+0 records out 2023-11-24T06:09:17.594 INFO:teuthology.orchestra.run.smithi118.stderr:512 bytes copied, 0.000238357 s, 2.1 MB/s 2023-11-24T06:09:17.597 DEBUG:teuthology.orchestra.run.smithi118:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-11-24T06:09:17.654 DEBUG:teuthology.orchestra.run.smithi118:> stat /dev/vg_nvme/lv_4 2023-11-24T06:09:17.703 INFO:teuthology.orchestra.run.smithi118.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-11-24T06:09:17.703 INFO:teuthology.orchestra.run.smithi118.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-24T06:09:17.703 INFO:teuthology.orchestra.run.smithi118.stdout:Device: 5h/5d Inode: 697 Links: 1 2023-11-24T06:09:17.704 INFO:teuthology.orchestra.run.smithi118.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-24T06:09:17.704 INFO:teuthology.orchestra.run.smithi118.stdout:Context: system_u:object_r:device_t:s0 2023-11-24T06:09:17.704 INFO:teuthology.orchestra.run.smithi118.stdout:Access: 2023-11-24 06:08:57.154779273 +0000 2023-11-24T06:09:17.704 INFO:teuthology.orchestra.run.smithi118.stdout:Modify: 2023-11-24 06:06:04.351823565 +0000 2023-11-24T06:09:17.704 INFO:teuthology.orchestra.run.smithi118.stdout:Change: 2023-11-24 06:06:04.351823565 +0000 2023-11-24T06:09:17.704 INFO:teuthology.orchestra.run.smithi118.stdout: Birth: 2023-11-24 06:06:04.351823565 +0000 2023-11-24T06:09:17.704 DEBUG:teuthology.orchestra.run.smithi118:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-11-24T06:09:17.791 INFO:teuthology.orchestra.run.smithi118.stderr:1+0 records in 2023-11-24T06:09:17.792 INFO:teuthology.orchestra.run.smithi118.stderr:1+0 records out 2023-11-24T06:09:17.792 INFO:teuthology.orchestra.run.smithi118.stderr:512 bytes copied, 0.000224554 s, 2.3 MB/s 2023-11-24T06:09:17.795 DEBUG:teuthology.orchestra.run.smithi118:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-11-24T06:09:17.854 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-11-24T06:09:17.854 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi092.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi118.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-11-24T06:09:17.854 INFO:tasks.ceph:Generating config... 2023-11-24T06:09:17.855 INFO:tasks.ceph:[client] client mount timeout = 600 2023-11-24T06:09:17.855 INFO:tasks.ceph:[client] debug client = 20 2023-11-24T06:09:17.856 INFO:tasks.ceph:[client] debug ms = 1 2023-11-24T06:09:17.856 INFO:tasks.ceph:[client] pid file = 2023-11-24T06:09:17.856 INFO:tasks.ceph:[client] rados mon op timeout = 900 2023-11-24T06:09:17.856 INFO:tasks.ceph:[client] rados osd op timeout = 900 2023-11-24T06:09:17.856 INFO:tasks.ceph:[mds] debug mds = 20 2023-11-24T06:09:17.856 INFO:tasks.ceph:[mds] debug mds balancer = 20 2023-11-24T06:09:17.856 INFO:tasks.ceph:[mds] debug ms = 1 2023-11-24T06:09:17.856 INFO:tasks.ceph:[mds] mds debug frag = True 2023-11-24T06:09:17.856 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2023-11-24T06:09:17.856 INFO:tasks.ceph:[mds] mds op complaint time = 180 2023-11-24T06:09:17.856 INFO:tasks.ceph:[mds] mds verify scatter = True 2023-11-24T06:09:17.857 INFO:tasks.ceph:[mds] osd op complaint time = 180 2023-11-24T06:09:17.857 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2023-11-24T06:09:17.857 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2023-11-24T06:09:17.857 INFO:tasks.ceph:[mgr] debug client = 20 2023-11-24T06:09:17.857 INFO:tasks.ceph:[mgr] debug finisher = 20 2023-11-24T06:09:17.857 INFO:tasks.ceph:[mgr] debug mgr = 20 2023-11-24T06:09:17.857 INFO:tasks.ceph:[mgr] debug ms = 1 2023-11-24T06:09:17.857 INFO:tasks.ceph:[mon] debug mon = 20 2023-11-24T06:09:17.857 INFO:tasks.ceph:[mon] debug ms = 1 2023-11-24T06:09:17.857 INFO:tasks.ceph:[mon] debug paxos = 20 2023-11-24T06:09:17.857 INFO:tasks.ceph:[mon] mon op complaint time = 120 2023-11-24T06:09:17.857 INFO:tasks.ceph:[osd] bdev async discard = True 2023-11-24T06:09:17.857 INFO:tasks.ceph:[osd] bdev enable discard = True 2023-11-24T06:09:17.858 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2023-11-24T06:09:17.858 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2023-11-24T06:09:17.858 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2023-11-24T06:09:17.858 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2023-11-24T06:09:17.858 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2023-11-24T06:09:17.858 INFO:tasks.ceph:[osd] debug ms = 1 2023-11-24T06:09:17.858 INFO:tasks.ceph:[osd] debug osd = 20 2023-11-24T06:09:17.858 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2023-11-24T06:09:17.858 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2023-11-24T06:09:17.858 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2023-11-24T06:09:17.858 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2023-11-24T06:09:17.859 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2023-11-24T06:09:17.859 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2023-11-24T06:09:17.859 INFO:tasks.ceph:[osd] osd op complaint time = 180 2023-11-24T06:09:17.859 INFO:tasks.ceph:Setting up mon.a... 2023-11-24T06:09:17.859 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2023-11-24T06:09:18.002 INFO:teuthology.orchestra.run.smithi092.stdout:creating /etc/ceph/ceph.keyring 2023-11-24T06:09:18.011 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2023-11-24T06:09:18.144 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-11-24T06:09:18.238 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.92'), ('mon.b', '172.21.15.118'), ('mon.c', '[v2:172.21.15.118:3301,v1:172.21.15.118:6790]')] 2023-11-24T06:09:18.239 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.92,172.21.15.118,[v2:172.21.15.118:3301,v1:172.21.15.118:6790]'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'debug client': 20, 'debug finisher': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2023-11-24T06:09:18.240 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-11-24T06:09:18.240 DEBUG:teuthology.orchestra.run.smithi092:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2023-11-24T06:09:18.294 DEBUG:teuthology.orchestra.run.smithi092:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.92 --add b 172.21.15.118 --addv c '[v2:172.21.15.118:3301,v1:172.21.15.118:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2023-11-24T06:09:18.449 INFO:teuthology.orchestra.run.smithi092.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2023-11-24T06:09:18.449 INFO:teuthology.orchestra.run.smithi092.stdout:monmaptool: generated fsid 2d6dee5f-8aa3-470f-931a-50ade963b1cb 2023-11-24T06:09:18.449 INFO:teuthology.orchestra.run.smithi092.stdout:setting min_mon_release = pacific 2023-11-24T06:09:18.449 INFO:teuthology.orchestra.run.smithi092.stdout:epoch 0 2023-11-24T06:09:18.450 INFO:teuthology.orchestra.run.smithi092.stdout:fsid 2d6dee5f-8aa3-470f-931a-50ade963b1cb 2023-11-24T06:09:18.450 INFO:teuthology.orchestra.run.smithi092.stdout:last_changed 2023-11-24T06:09:18.447498+0000 2023-11-24T06:09:18.450 INFO:teuthology.orchestra.run.smithi092.stdout:created 2023-11-24T06:09:18.447498+0000 2023-11-24T06:09:18.450 INFO:teuthology.orchestra.run.smithi092.stdout:min_mon_release 16 (pacific) 2023-11-24T06:09:18.450 INFO:teuthology.orchestra.run.smithi092.stdout:election_strategy: 1 2023-11-24T06:09:18.450 INFO:teuthology.orchestra.run.smithi092.stdout:0: [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] mon.a 2023-11-24T06:09:18.450 INFO:teuthology.orchestra.run.smithi092.stdout:1: [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] mon.b 2023-11-24T06:09:18.450 INFO:teuthology.orchestra.run.smithi092.stdout:2: [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] mon.c 2023-11-24T06:09:18.450 INFO:teuthology.orchestra.run.smithi092.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2023-11-24T06:09:18.453 DEBUG:teuthology.orchestra.run.smithi092:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2023-11-24T06:09:18.505 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 2d6dee5f-8aa3-470f-931a-50ade963b1cb... 2023-11-24T06:09:18.506 DEBUG:teuthology.orchestra.run.smithi092:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-11-24T06:09:18.511 DEBUG:teuthology.orchestra.run.smithi118:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-11-24T06:09:18.677 INFO:teuthology.orchestra.run.smithi092.stdout:[global] 2023-11-24T06:09:18.677 INFO:teuthology.orchestra.run.smithi092.stdout: chdir = "" 2023-11-24T06:09:18.677 INFO:teuthology.orchestra.run.smithi092.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-11-24T06:09:18.677 INFO:teuthology.orchestra.run.smithi092.stdout: auth supported = cephx 2023-11-24T06:09:18.677 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-24T06:09:18.677 INFO:teuthology.orchestra.run.smithi092.stdout: filestore xattr use omap = true 2023-11-24T06:09:18.677 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-24T06:09:18.677 INFO:teuthology.orchestra.run.smithi092.stdout: mon clock drift allowed = 1.000 2023-11-24T06:09:18.677 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-24T06:09:18.678 INFO:teuthology.orchestra.run.smithi092.stdout: osd crush chooseleaf type = 0 2023-11-24T06:09:18.678 INFO:teuthology.orchestra.run.smithi092.stdout: auth debug = true 2023-11-24T06:09:18.678 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-24T06:09:18.678 INFO:teuthology.orchestra.run.smithi092.stdout: ms die on old message = true 2023-11-24T06:09:18.678 INFO:teuthology.orchestra.run.smithi092.stdout: ms die on bug = true 2023-11-24T06:09:18.678 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-24T06:09:18.678 INFO:teuthology.orchestra.run.smithi092.stdout: mon max pg per osd = 10000 # >= luminous 2023-11-24T06:09:18.678 INFO:teuthology.orchestra.run.smithi092.stdout: mon pg warn max object skew = 0 2023-11-24T06:09:18.678 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-24T06:09:18.678 INFO:teuthology.orchestra.run.smithi092.stdout: # disable pg_autoscaler by default for new pools 2023-11-24T06:09:18.678 INFO:teuthology.orchestra.run.smithi092.stdout: osd_pool_default_pg_autoscale_mode = off 2023-11-24T06:09:18.679 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-24T06:09:18.679 INFO:teuthology.orchestra.run.smithi092.stdout: osd pool default size = 2 2023-11-24T06:09:18.679 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-24T06:09:18.679 INFO:teuthology.orchestra.run.smithi092.stdout: mon osd allow primary affinity = true 2023-11-24T06:09:18.679 INFO:teuthology.orchestra.run.smithi092.stdout: mon osd allow pg remap = true 2023-11-24T06:09:18.679 INFO:teuthology.orchestra.run.smithi092.stdout: mon warn on legacy crush tunables = false 2023-11-24T06:09:18.679 INFO:teuthology.orchestra.run.smithi092.stdout: mon warn on crush straw calc version zero = false 2023-11-24T06:09:18.679 INFO:teuthology.orchestra.run.smithi092.stdout: mon warn on no sortbitwise = false 2023-11-24T06:09:18.679 INFO:teuthology.orchestra.run.smithi092.stdout: mon warn on osd down out interval zero = false 2023-11-24T06:09:18.679 INFO:teuthology.orchestra.run.smithi092.stdout: mon warn on too few osds = false 2023-11-24T06:09:18.679 INFO:teuthology.orchestra.run.smithi092.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-11-24T06:09:18.679 INFO:teuthology.orchestra.run.smithi092.stdout: mon_warn_on_pool_no_redundancy = false 2023-11-24T06:09:18.680 INFO:teuthology.orchestra.run.smithi092.stdout: mon_allow_pool_size_one = true 2023-11-24T06:09:18.680 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-24T06:09:18.680 INFO:teuthology.orchestra.run.smithi092.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-11-24T06:09:18.680 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-24T06:09:18.680 INFO:teuthology.orchestra.run.smithi092.stdout: osd default data pool replay window = 5 2023-11-24T06:09:18.680 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-24T06:09:18.680 INFO:teuthology.orchestra.run.smithi092.stdout: mon allow pool delete = true 2023-11-24T06:09:18.680 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-24T06:09:18.680 INFO:teuthology.orchestra.run.smithi092.stdout: mon cluster log file level = debug 2023-11-24T06:09:18.680 INFO:teuthology.orchestra.run.smithi092.stdout: debug asserts on shutdown = true 2023-11-24T06:09:18.680 INFO:teuthology.orchestra.run.smithi092.stdout: mon health detail to clog = false 2023-11-24T06:09:18.680 INFO:teuthology.orchestra.run.smithi092.stdout: mon host = "172.21.15.92,172.21.15.118,[v2:172.21.15.118:3301,v1:172.21.15.118:6790]" 2023-11-24T06:09:18.681 INFO:teuthology.orchestra.run.smithi092.stdout: fsid = 2d6dee5f-8aa3-470f-931a-50ade963b1cb 2023-11-24T06:09:18.681 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-24T06:09:18.681 INFO:teuthology.orchestra.run.smithi092.stdout:[osd] 2023-11-24T06:09:18.681 INFO:teuthology.orchestra.run.smithi092.stdout: osd journal size = 100 2023-11-24T06:09:18.681 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-24T06:09:18.681 INFO:teuthology.orchestra.run.smithi092.stdout: osd scrub load threshold = 5.0 2023-11-24T06:09:18.681 INFO:teuthology.orchestra.run.smithi092.stdout: osd scrub max interval = 600 2023-11-24T06:09:18.681 INFO:teuthology.orchestra.run.smithi092.stdout: osd mclock profile = high_recovery_ops 2023-11-24T06:09:18.681 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-24T06:09:18.681 INFO:teuthology.orchestra.run.smithi092.stdout: osd recover clone overlap = true 2023-11-24T06:09:18.681 INFO:teuthology.orchestra.run.smithi092.stdout: osd recovery max chunk = 1048576 2023-11-24T06:09:18.681 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-24T06:09:18.682 INFO:teuthology.orchestra.run.smithi092.stdout: osd debug shutdown = true 2023-11-24T06:09:18.682 INFO:teuthology.orchestra.run.smithi092.stdout: osd debug op order = true 2023-11-24T06:09:18.682 INFO:teuthology.orchestra.run.smithi092.stdout: osd debug verify stray on activate = true 2023-11-24T06:09:18.682 INFO:teuthology.orchestra.run.smithi092.stdout: osd debug trim objects = true 2023-11-24T06:09:18.682 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-24T06:09:18.682 INFO:teuthology.orchestra.run.smithi092.stdout: osd open classes on start = true 2023-11-24T06:09:18.682 INFO:teuthology.orchestra.run.smithi092.stdout: osd debug pg log writeout = true 2023-11-24T06:09:18.682 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-24T06:09:18.682 INFO:teuthology.orchestra.run.smithi092.stdout: osd deep scrub update digest min age = 30 2023-11-24T06:09:18.682 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-24T06:09:18.682 INFO:teuthology.orchestra.run.smithi092.stdout: osd map max advance = 10 2023-11-24T06:09:18.682 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-24T06:09:18.683 INFO:teuthology.orchestra.run.smithi092.stdout: journal zero on create = true 2023-11-24T06:09:18.683 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-24T06:09:18.683 INFO:teuthology.orchestra.run.smithi092.stdout: filestore ondisk finisher threads = 3 2023-11-24T06:09:18.683 INFO:teuthology.orchestra.run.smithi092.stdout: filestore apply finisher threads = 3 2023-11-24T06:09:18.683 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-24T06:09:18.683 INFO:teuthology.orchestra.run.smithi092.stdout: bdev debug aio = true 2023-11-24T06:09:18.683 INFO:teuthology.orchestra.run.smithi092.stdout: osd debug misdirected ops = true 2023-11-24T06:09:18.683 INFO:teuthology.orchestra.run.smithi092.stdout: bdev async discard = True 2023-11-24T06:09:18.683 INFO:teuthology.orchestra.run.smithi092.stdout: bdev enable discard = True 2023-11-24T06:09:18.683 INFO:teuthology.orchestra.run.smithi092.stdout: bluestore allocator = bitmap 2023-11-24T06:09:18.683 INFO:teuthology.orchestra.run.smithi092.stdout: bluestore block size = 96636764160 2023-11-24T06:09:18.683 INFO:teuthology.orchestra.run.smithi092.stdout: bluestore fsck on mount = True 2023-11-24T06:09:18.683 INFO:teuthology.orchestra.run.smithi092.stdout: debug bluefs = 1/20 2023-11-24T06:09:18.684 INFO:teuthology.orchestra.run.smithi092.stdout: debug bluestore = 1/20 2023-11-24T06:09:18.684 INFO:teuthology.orchestra.run.smithi092.stdout: debug ms = 1 2023-11-24T06:09:18.684 INFO:teuthology.orchestra.run.smithi092.stdout: debug osd = 20 2023-11-24T06:09:18.684 INFO:teuthology.orchestra.run.smithi092.stdout: debug rocksdb = 4/10 2023-11-24T06:09:18.684 INFO:teuthology.orchestra.run.smithi092.stdout: mon osd backfillfull_ratio = 0.85 2023-11-24T06:09:18.684 INFO:teuthology.orchestra.run.smithi092.stdout: mon osd full ratio = 0.9 2023-11-24T06:09:18.684 INFO:teuthology.orchestra.run.smithi092.stdout: mon osd nearfull ratio = 0.8 2023-11-24T06:09:18.684 INFO:teuthology.orchestra.run.smithi092.stdout: osd failsafe full ratio = 0.95 2023-11-24T06:09:18.684 INFO:teuthology.orchestra.run.smithi092.stdout: osd objectstore = bluestore 2023-11-24T06:09:18.684 INFO:teuthology.orchestra.run.smithi092.stdout: osd op complaint time = 180 2023-11-24T06:09:18.684 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-24T06:09:18.684 INFO:teuthology.orchestra.run.smithi092.stdout:[mgr] 2023-11-24T06:09:18.685 INFO:teuthology.orchestra.run.smithi092.stdout: debug ms = 1 2023-11-24T06:09:18.685 INFO:teuthology.orchestra.run.smithi092.stdout: debug mgr = 20 2023-11-24T06:09:18.685 INFO:teuthology.orchestra.run.smithi092.stdout: debug mon = 20 2023-11-24T06:09:18.685 INFO:teuthology.orchestra.run.smithi092.stdout: debug auth = 20 2023-11-24T06:09:18.685 INFO:teuthology.orchestra.run.smithi092.stdout: mon reweight min pgs per osd = 4 2023-11-24T06:09:18.685 INFO:teuthology.orchestra.run.smithi092.stdout: mon reweight min bytes per osd = 10 2023-11-24T06:09:18.685 INFO:teuthology.orchestra.run.smithi092.stdout: mgr/telemetry/nag = false 2023-11-24T06:09:18.685 INFO:teuthology.orchestra.run.smithi092.stdout: debug client = 20 2023-11-24T06:09:18.685 INFO:teuthology.orchestra.run.smithi092.stdout: debug finisher = 20 2023-11-24T06:09:18.685 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-24T06:09:18.685 INFO:teuthology.orchestra.run.smithi092.stdout:[mon] 2023-11-24T06:09:18.685 INFO:teuthology.orchestra.run.smithi092.stdout: debug ms = 1 2023-11-24T06:09:18.685 INFO:teuthology.orchestra.run.smithi092.stdout: debug mon = 20 2023-11-24T06:09:18.686 INFO:teuthology.orchestra.run.smithi092.stdout: debug paxos = 20 2023-11-24T06:09:18.686 INFO:teuthology.orchestra.run.smithi092.stdout: debug auth = 20 2023-11-24T06:09:18.686 INFO:teuthology.orchestra.run.smithi092.stdout: mon data avail warn = 5 2023-11-24T06:09:18.686 INFO:teuthology.orchestra.run.smithi092.stdout: mon mgr mkfs grace = 240 2023-11-24T06:09:18.686 INFO:teuthology.orchestra.run.smithi092.stdout: mon reweight min pgs per osd = 4 2023-11-24T06:09:18.686 INFO:teuthology.orchestra.run.smithi092.stdout: mon osd reporter subtree level = osd 2023-11-24T06:09:18.686 INFO:teuthology.orchestra.run.smithi092.stdout: mon osd prime pg temp = true 2023-11-24T06:09:18.686 INFO:teuthology.orchestra.run.smithi092.stdout: mon reweight min bytes per osd = 10 2023-11-24T06:09:18.686 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-24T06:09:18.686 INFO:teuthology.orchestra.run.smithi092.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-11-24T06:09:18.686 INFO:teuthology.orchestra.run.smithi092.stdout: auth mon ticket ttl = 660 # 11m 2023-11-24T06:09:18.686 INFO:teuthology.orchestra.run.smithi092.stdout: auth service ticket ttl = 240 # 4m 2023-11-24T06:09:18.686 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-24T06:09:18.687 INFO:teuthology.orchestra.run.smithi092.stdout: # don't complain about insecure global_id in the test suite 2023-11-24T06:09:18.687 INFO:teuthology.orchestra.run.smithi092.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-11-24T06:09:18.687 INFO:teuthology.orchestra.run.smithi092.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-11-24T06:09:18.687 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-24T06:09:18.687 INFO:teuthology.orchestra.run.smithi092.stdout: # 1m isn't quite enough 2023-11-24T06:09:18.687 INFO:teuthology.orchestra.run.smithi092.stdout: mon_down_mkfs_grace = 2m 2023-11-24T06:09:18.687 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-24T06:09:18.687 INFO:teuthology.orchestra.run.smithi092.stdout: mon_warn_on_filestore_osds = false 2023-11-24T06:09:18.687 INFO:teuthology.orchestra.run.smithi092.stdout: mon op complaint time = 120 2023-11-24T06:09:18.687 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-24T06:09:18.687 INFO:teuthology.orchestra.run.smithi092.stdout:[client] 2023-11-24T06:09:18.687 INFO:teuthology.orchestra.run.smithi092.stdout: rgw cache enabled = true 2023-11-24T06:09:18.688 INFO:teuthology.orchestra.run.smithi092.stdout: rgw enable ops log = true 2023-11-24T06:09:18.688 INFO:teuthology.orchestra.run.smithi092.stdout: rgw enable usage log = true 2023-11-24T06:09:18.688 INFO:teuthology.orchestra.run.smithi092.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-11-24T06:09:18.688 INFO:teuthology.orchestra.run.smithi092.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-11-24T06:09:18.688 INFO:teuthology.orchestra.run.smithi092.stdout: client mount timeout = 600 2023-11-24T06:09:18.688 INFO:teuthology.orchestra.run.smithi092.stdout: debug client = 20 2023-11-24T06:09:18.688 INFO:teuthology.orchestra.run.smithi092.stdout: debug ms = 1 2023-11-24T06:09:18.688 INFO:teuthology.orchestra.run.smithi092.stdout: pid file = "" 2023-11-24T06:09:18.688 INFO:teuthology.orchestra.run.smithi092.stdout: rados mon op timeout = 900 2023-11-24T06:09:18.688 INFO:teuthology.orchestra.run.smithi092.stdout: rados osd op timeout = 900 2023-11-24T06:09:18.688 INFO:teuthology.orchestra.run.smithi092.stdout:[mon.a] 2023-11-24T06:09:18.688 INFO:teuthology.orchestra.run.smithi092.stdout:[mon.b] 2023-11-24T06:09:18.688 INFO:teuthology.orchestra.run.smithi092.stdout:[mon.c] 2023-11-24T06:09:18.689 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.a] 2023-11-24T06:09:18.689 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.c] 2023-11-24T06:09:18.689 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.b] 2023-11-24T06:09:18.689 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.d] 2023-11-24T06:09:18.689 INFO:teuthology.orchestra.run.smithi092.stdout:[mds] 2023-11-24T06:09:18.689 INFO:teuthology.orchestra.run.smithi092.stdout: debug mds = 20 2023-11-24T06:09:18.689 INFO:teuthology.orchestra.run.smithi092.stdout: debug mds balancer = 20 2023-11-24T06:09:18.689 INFO:teuthology.orchestra.run.smithi092.stdout: debug ms = 1 2023-11-24T06:09:18.689 INFO:teuthology.orchestra.run.smithi092.stdout: mds debug frag = True 2023-11-24T06:09:18.689 INFO:teuthology.orchestra.run.smithi092.stdout: mds debug scatterstat = True 2023-11-24T06:09:18.689 INFO:teuthology.orchestra.run.smithi092.stdout: mds op complaint time = 180 2023-11-24T06:09:18.689 INFO:teuthology.orchestra.run.smithi092.stdout: mds verify scatter = True 2023-11-24T06:09:18.689 INFO:teuthology.orchestra.run.smithi092.stdout: osd op complaint time = 180 2023-11-24T06:09:18.690 INFO:teuthology.orchestra.run.smithi092.stdout: rados mon op timeout = 900 2023-11-24T06:09:18.690 INFO:teuthology.orchestra.run.smithi092.stdout: rados osd op timeout = 900 2023-11-24T06:09:18.690 INFO:teuthology.orchestra.run.smithi118.stdout:[global] 2023-11-24T06:09:18.690 INFO:teuthology.orchestra.run.smithi118.stdout: chdir = "" 2023-11-24T06:09:18.690 INFO:teuthology.orchestra.run.smithi118.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-11-24T06:09:18.690 INFO:teuthology.orchestra.run.smithi118.stdout: auth supported = cephx 2023-11-24T06:09:18.690 INFO:teuthology.orchestra.run.smithi118.stdout: 2023-11-24T06:09:18.691 INFO:teuthology.orchestra.run.smithi118.stdout: filestore xattr use omap = true 2023-11-24T06:09:18.691 INFO:teuthology.orchestra.run.smithi118.stdout: 2023-11-24T06:09:18.691 INFO:teuthology.orchestra.run.smithi118.stdout: mon clock drift allowed = 1.000 2023-11-24T06:09:18.691 INFO:teuthology.orchestra.run.smithi118.stdout: 2023-11-24T06:09:18.691 INFO:teuthology.orchestra.run.smithi118.stdout: osd crush chooseleaf type = 0 2023-11-24T06:09:18.691 INFO:teuthology.orchestra.run.smithi118.stdout: auth debug = true 2023-11-24T06:09:18.691 INFO:teuthology.orchestra.run.smithi118.stdout: 2023-11-24T06:09:18.691 INFO:teuthology.orchestra.run.smithi118.stdout: ms die on old message = true 2023-11-24T06:09:18.691 INFO:teuthology.orchestra.run.smithi118.stdout: ms die on bug = true 2023-11-24T06:09:18.691 INFO:teuthology.orchestra.run.smithi118.stdout: 2023-11-24T06:09:18.691 INFO:teuthology.orchestra.run.smithi118.stdout: mon max pg per osd = 10000 # >= luminous 2023-11-24T06:09:18.692 INFO:teuthology.orchestra.run.smithi118.stdout: mon pg warn max object skew = 0 2023-11-24T06:09:18.692 INFO:teuthology.orchestra.run.smithi118.stdout: 2023-11-24T06:09:18.692 INFO:teuthology.orchestra.run.smithi118.stdout: # disable pg_autoscaler by default for new pools 2023-11-24T06:09:18.692 INFO:teuthology.orchestra.run.smithi118.stdout: osd_pool_default_pg_autoscale_mode = off 2023-11-24T06:09:18.692 INFO:teuthology.orchestra.run.smithi118.stdout: 2023-11-24T06:09:18.692 INFO:teuthology.orchestra.run.smithi118.stdout: osd pool default size = 2 2023-11-24T06:09:18.692 INFO:teuthology.orchestra.run.smithi118.stdout: 2023-11-24T06:09:18.692 INFO:teuthology.orchestra.run.smithi118.stdout: mon osd allow primary affinity = true 2023-11-24T06:09:18.692 INFO:teuthology.orchestra.run.smithi118.stdout: mon osd allow pg remap = true 2023-11-24T06:09:18.692 INFO:teuthology.orchestra.run.smithi118.stdout: mon warn on legacy crush tunables = false 2023-11-24T06:09:18.692 INFO:teuthology.orchestra.run.smithi118.stdout: mon warn on crush straw calc version zero = false 2023-11-24T06:09:18.692 INFO:teuthology.orchestra.run.smithi118.stdout: mon warn on no sortbitwise = false 2023-11-24T06:09:18.692 INFO:teuthology.orchestra.run.smithi118.stdout: mon warn on osd down out interval zero = false 2023-11-24T06:09:18.693 INFO:teuthology.orchestra.run.smithi118.stdout: mon warn on too few osds = false 2023-11-24T06:09:18.693 INFO:teuthology.orchestra.run.smithi118.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-11-24T06:09:18.693 INFO:teuthology.orchestra.run.smithi118.stdout: mon_warn_on_pool_no_redundancy = false 2023-11-24T06:09:18.693 INFO:teuthology.orchestra.run.smithi118.stdout: mon_allow_pool_size_one = true 2023-11-24T06:09:18.693 INFO:teuthology.orchestra.run.smithi118.stdout: 2023-11-24T06:09:18.693 INFO:teuthology.orchestra.run.smithi118.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-11-24T06:09:18.693 INFO:teuthology.orchestra.run.smithi118.stdout: 2023-11-24T06:09:18.693 INFO:teuthology.orchestra.run.smithi118.stdout: osd default data pool replay window = 5 2023-11-24T06:09:18.693 INFO:teuthology.orchestra.run.smithi118.stdout: 2023-11-24T06:09:18.693 INFO:teuthology.orchestra.run.smithi118.stdout: mon allow pool delete = true 2023-11-24T06:09:18.693 INFO:teuthology.orchestra.run.smithi118.stdout: 2023-11-24T06:09:18.693 INFO:teuthology.orchestra.run.smithi118.stdout: mon cluster log file level = debug 2023-11-24T06:09:18.694 INFO:teuthology.orchestra.run.smithi118.stdout: debug asserts on shutdown = true 2023-11-24T06:09:18.694 INFO:teuthology.orchestra.run.smithi118.stdout: mon health detail to clog = false 2023-11-24T06:09:18.694 INFO:teuthology.orchestra.run.smithi118.stdout: mon host = "172.21.15.92,172.21.15.118,[v2:172.21.15.118:3301,v1:172.21.15.118:6790]" 2023-11-24T06:09:18.694 INFO:teuthology.orchestra.run.smithi118.stdout: fsid = 2d6dee5f-8aa3-470f-931a-50ade963b1cb 2023-11-24T06:09:18.694 INFO:teuthology.orchestra.run.smithi118.stdout: 2023-11-24T06:09:18.694 INFO:teuthology.orchestra.run.smithi118.stdout:[osd] 2023-11-24T06:09:18.694 INFO:teuthology.orchestra.run.smithi118.stdout: osd journal size = 100 2023-11-24T06:09:18.694 INFO:teuthology.orchestra.run.smithi118.stdout: 2023-11-24T06:09:18.694 INFO:teuthology.orchestra.run.smithi118.stdout: osd scrub load threshold = 5.0 2023-11-24T06:09:18.694 INFO:teuthology.orchestra.run.smithi118.stdout: osd scrub max interval = 600 2023-11-24T06:09:18.694 INFO:teuthology.orchestra.run.smithi118.stdout: osd mclock profile = high_recovery_ops 2023-11-24T06:09:18.695 INFO:teuthology.orchestra.run.smithi118.stdout: 2023-11-24T06:09:18.695 INFO:teuthology.orchestra.run.smithi118.stdout: osd recover clone overlap = true 2023-11-24T06:09:18.695 INFO:teuthology.orchestra.run.smithi118.stdout: osd recovery max chunk = 1048576 2023-11-24T06:09:18.695 INFO:teuthology.orchestra.run.smithi118.stdout: 2023-11-24T06:09:18.695 INFO:teuthology.orchestra.run.smithi118.stdout: osd debug shutdown = true 2023-11-24T06:09:18.695 INFO:teuthology.orchestra.run.smithi118.stdout: osd debug op order = true 2023-11-24T06:09:18.695 INFO:teuthology.orchestra.run.smithi118.stdout: osd debug verify stray on activate = true 2023-11-24T06:09:18.695 INFO:teuthology.orchestra.run.smithi118.stdout: osd debug trim objects = true 2023-11-24T06:09:18.695 INFO:teuthology.orchestra.run.smithi118.stdout: 2023-11-24T06:09:18.695 INFO:teuthology.orchestra.run.smithi118.stdout: osd open classes on start = true 2023-11-24T06:09:18.695 INFO:teuthology.orchestra.run.smithi118.stdout: osd debug pg log writeout = true 2023-11-24T06:09:18.695 INFO:teuthology.orchestra.run.smithi118.stdout: 2023-11-24T06:09:18.695 INFO:teuthology.orchestra.run.smithi118.stdout: osd deep scrub update digest min age = 30 2023-11-24T06:09:18.696 INFO:teuthology.orchestra.run.smithi118.stdout: 2023-11-24T06:09:18.696 INFO:teuthology.orchestra.run.smithi118.stdout: osd map max advance = 10 2023-11-24T06:09:18.696 INFO:teuthology.orchestra.run.smithi118.stdout: 2023-11-24T06:09:18.696 INFO:teuthology.orchestra.run.smithi118.stdout: journal zero on create = true 2023-11-24T06:09:18.696 INFO:teuthology.orchestra.run.smithi118.stdout: 2023-11-24T06:09:18.696 INFO:teuthology.orchestra.run.smithi118.stdout: filestore ondisk finisher threads = 3 2023-11-24T06:09:18.696 INFO:teuthology.orchestra.run.smithi118.stdout: filestore apply finisher threads = 3 2023-11-24T06:09:18.696 INFO:teuthology.orchestra.run.smithi118.stdout: 2023-11-24T06:09:18.696 INFO:teuthology.orchestra.run.smithi118.stdout: bdev debug aio = true 2023-11-24T06:09:18.696 INFO:teuthology.orchestra.run.smithi118.stdout: osd debug misdirected ops = true 2023-11-24T06:09:18.696 INFO:teuthology.orchestra.run.smithi118.stdout: bdev async discard = True 2023-11-24T06:09:18.696 INFO:teuthology.orchestra.run.smithi118.stdout: bdev enable discard = True 2023-11-24T06:09:18.696 INFO:teuthology.orchestra.run.smithi118.stdout: bluestore allocator = bitmap 2023-11-24T06:09:18.697 INFO:teuthology.orchestra.run.smithi118.stdout: bluestore block size = 96636764160 2023-11-24T06:09:18.697 INFO:teuthology.orchestra.run.smithi118.stdout: bluestore fsck on mount = True 2023-11-24T06:09:18.697 INFO:teuthology.orchestra.run.smithi118.stdout: debug bluefs = 1/20 2023-11-24T06:09:18.697 INFO:teuthology.orchestra.run.smithi118.stdout: debug bluestore = 1/20 2023-11-24T06:09:18.697 INFO:teuthology.orchestra.run.smithi118.stdout: debug ms = 1 2023-11-24T06:09:18.697 INFO:teuthology.orchestra.run.smithi118.stdout: debug osd = 20 2023-11-24T06:09:18.697 INFO:teuthology.orchestra.run.smithi118.stdout: debug rocksdb = 4/10 2023-11-24T06:09:18.697 INFO:teuthology.orchestra.run.smithi118.stdout: mon osd backfillfull_ratio = 0.85 2023-11-24T06:09:18.697 INFO:teuthology.orchestra.run.smithi118.stdout: mon osd full ratio = 0.9 2023-11-24T06:09:18.697 INFO:teuthology.orchestra.run.smithi118.stdout: mon osd nearfull ratio = 0.8 2023-11-24T06:09:18.697 INFO:teuthology.orchestra.run.smithi118.stdout: osd failsafe full ratio = 0.95 2023-11-24T06:09:18.697 INFO:teuthology.orchestra.run.smithi118.stdout: osd objectstore = bluestore 2023-11-24T06:09:18.698 INFO:teuthology.orchestra.run.smithi118.stdout: osd op complaint time = 180 2023-11-24T06:09:18.698 INFO:teuthology.orchestra.run.smithi118.stdout: 2023-11-24T06:09:18.698 INFO:teuthology.orchestra.run.smithi118.stdout:[mgr] 2023-11-24T06:09:18.698 INFO:teuthology.orchestra.run.smithi118.stdout: debug ms = 1 2023-11-24T06:09:18.698 INFO:teuthology.orchestra.run.smithi118.stdout: debug mgr = 20 2023-11-24T06:09:18.698 INFO:teuthology.orchestra.run.smithi118.stdout: debug mon = 20 2023-11-24T06:09:18.698 INFO:teuthology.orchestra.run.smithi118.stdout: debug auth = 20 2023-11-24T06:09:18.698 INFO:teuthology.orchestra.run.smithi118.stdout: mon reweight min pgs per osd = 4 2023-11-24T06:09:18.698 INFO:teuthology.orchestra.run.smithi118.stdout: mon reweight min bytes per osd = 10 2023-11-24T06:09:18.698 INFO:teuthology.orchestra.run.smithi118.stdout: mgr/telemetry/nag = false 2023-11-24T06:09:18.698 INFO:teuthology.orchestra.run.smithi118.stdout: debug client = 20 2023-11-24T06:09:18.698 INFO:teuthology.orchestra.run.smithi118.stdout: debug finisher = 20 2023-11-24T06:09:18.699 INFO:teuthology.orchestra.run.smithi118.stdout: 2023-11-24T06:09:18.699 INFO:teuthology.orchestra.run.smithi118.stdout:[mon] 2023-11-24T06:09:18.699 INFO:teuthology.orchestra.run.smithi118.stdout: debug ms = 1 2023-11-24T06:09:18.699 INFO:teuthology.orchestra.run.smithi118.stdout: debug mon = 20 2023-11-24T06:09:18.699 INFO:teuthology.orchestra.run.smithi118.stdout: debug paxos = 20 2023-11-24T06:09:18.699 INFO:teuthology.orchestra.run.smithi118.stdout: debug auth = 20 2023-11-24T06:09:18.699 INFO:teuthology.orchestra.run.smithi118.stdout: mon data avail warn = 5 2023-11-24T06:09:18.699 INFO:teuthology.orchestra.run.smithi118.stdout: mon mgr mkfs grace = 240 2023-11-24T06:09:18.699 INFO:teuthology.orchestra.run.smithi118.stdout: mon reweight min pgs per osd = 4 2023-11-24T06:09:18.699 INFO:teuthology.orchestra.run.smithi118.stdout: mon osd reporter subtree level = osd 2023-11-24T06:09:18.699 INFO:teuthology.orchestra.run.smithi118.stdout: mon osd prime pg temp = true 2023-11-24T06:09:18.699 INFO:teuthology.orchestra.run.smithi118.stdout: mon reweight min bytes per osd = 10 2023-11-24T06:09:18.700 INFO:teuthology.orchestra.run.smithi118.stdout: 2023-11-24T06:09:18.700 INFO:teuthology.orchestra.run.smithi118.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-11-24T06:09:18.700 INFO:teuthology.orchestra.run.smithi118.stdout: auth mon ticket ttl = 660 # 11m 2023-11-24T06:09:18.700 INFO:teuthology.orchestra.run.smithi118.stdout: auth service ticket ttl = 240 # 4m 2023-11-24T06:09:18.700 INFO:teuthology.orchestra.run.smithi118.stdout: 2023-11-24T06:09:18.700 INFO:teuthology.orchestra.run.smithi118.stdout: # don't complain about insecure global_id in the test suite 2023-11-24T06:09:18.700 INFO:teuthology.orchestra.run.smithi118.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-11-24T06:09:18.700 INFO:teuthology.orchestra.run.smithi118.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-11-24T06:09:18.700 INFO:teuthology.orchestra.run.smithi118.stdout: 2023-11-24T06:09:18.700 INFO:teuthology.orchestra.run.smithi118.stdout: # 1m isn't quite enough 2023-11-24T06:09:18.700 INFO:teuthology.orchestra.run.smithi118.stdout: mon_down_mkfs_grace = 2m 2023-11-24T06:09:18.701 INFO:teuthology.orchestra.run.smithi118.stdout: 2023-11-24T06:09:18.701 INFO:teuthology.orchestra.run.smithi118.stdout: mon_warn_on_filestore_osds = false 2023-11-24T06:09:18.701 INFO:teuthology.orchestra.run.smithi118.stdout: mon op complaint time = 120 2023-11-24T06:09:18.701 INFO:teuthology.orchestra.run.smithi118.stdout: 2023-11-24T06:09:18.701 INFO:teuthology.orchestra.run.smithi118.stdout:[client] 2023-11-24T06:09:18.701 INFO:teuthology.orchestra.run.smithi118.stdout: rgw cache enabled = true 2023-11-24T06:09:18.701 INFO:teuthology.orchestra.run.smithi118.stdout: rgw enable ops log = true 2023-11-24T06:09:18.701 INFO:teuthology.orchestra.run.smithi118.stdout: rgw enable usage log = true 2023-11-24T06:09:18.701 INFO:teuthology.orchestra.run.smithi118.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-11-24T06:09:18.701 INFO:teuthology.orchestra.run.smithi118.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-11-24T06:09:18.701 INFO:teuthology.orchestra.run.smithi118.stdout: client mount timeout = 600 2023-11-24T06:09:18.701 INFO:teuthology.orchestra.run.smithi118.stdout: debug client = 20 2023-11-24T06:09:18.702 INFO:teuthology.orchestra.run.smithi118.stdout: debug ms = 1 2023-11-24T06:09:18.702 INFO:teuthology.orchestra.run.smithi118.stdout: pid file = "" 2023-11-24T06:09:18.702 INFO:teuthology.orchestra.run.smithi118.stdout: rados mon op timeout = 900 2023-11-24T06:09:18.702 INFO:teuthology.orchestra.run.smithi118.stdout: rados osd op timeout = 900 2023-11-24T06:09:18.702 INFO:teuthology.orchestra.run.smithi118.stdout:[mon.a] 2023-11-24T06:09:18.702 INFO:teuthology.orchestra.run.smithi118.stdout:[mon.b] 2023-11-24T06:09:18.702 INFO:teuthology.orchestra.run.smithi118.stdout:[mon.c] 2023-11-24T06:09:18.702 INFO:teuthology.orchestra.run.smithi118.stdout:[mds.a] 2023-11-24T06:09:18.702 INFO:teuthology.orchestra.run.smithi118.stdout:[mds.c] 2023-11-24T06:09:18.702 INFO:teuthology.orchestra.run.smithi118.stdout:[mds.b] 2023-11-24T06:09:18.703 INFO:teuthology.orchestra.run.smithi118.stdout:[mds.d] 2023-11-24T06:09:18.703 INFO:teuthology.orchestra.run.smithi118.stdout:[mds] 2023-11-24T06:09:18.703 INFO:teuthology.orchestra.run.smithi118.stdout: debug mds = 20 2023-11-24T06:09:18.703 INFO:teuthology.orchestra.run.smithi118.stdout: debug mds balancer = 20 2023-11-24T06:09:18.703 INFO:teuthology.orchestra.run.smithi118.stdout: debug ms = 1 2023-11-24T06:09:18.703 INFO:teuthology.orchestra.run.smithi118.stdout: mds debug frag = True 2023-11-24T06:09:18.703 INFO:teuthology.orchestra.run.smithi118.stdout: mds debug scatterstat = True 2023-11-24T06:09:18.703 INFO:teuthology.orchestra.run.smithi118.stdout: mds op complaint time = 180 2023-11-24T06:09:18.703 INFO:teuthology.orchestra.run.smithi118.stdout: mds verify scatter = True 2023-11-24T06:09:18.704 INFO:teuthology.orchestra.run.smithi118.stdout: osd op complaint time = 180 2023-11-24T06:09:18.704 INFO:teuthology.orchestra.run.smithi118.stdout: rados mon op timeout = 900 2023-11-24T06:09:18.704 INFO:teuthology.orchestra.run.smithi118.stdout: rados osd op timeout = 900 2023-11-24T06:09:18.728 INFO:tasks.ceph:Creating admin key on mon.a... 2023-11-24T06:09:18.728 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2023-11-24T06:09:18.873 INFO:tasks.ceph:Copying monmap to all nodes... 2023-11-24T06:09:18.874 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-11-24T06:09:18.874 DEBUG:teuthology.orchestra.run.smithi092:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2023-11-24T06:09:18.925 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-11-24T06:09:18.925 DEBUG:teuthology.orchestra.run.smithi092:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2023-11-24T06:09:19.022 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi092.front.sepia.ceph.com 2023-11-24T06:09:19.023 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-11-24T06:09:19.023 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/etc/ceph/ceph.keyring 2023-11-24T06:09:19.023 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-11-24T06:09:19.168 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-11-24T06:09:19.168 DEBUG:teuthology.orchestra.run.smithi092:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-11-24T06:09:19.267 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi118.front.sepia.ceph.com 2023-11-24T06:09:19.267 DEBUG:teuthology.orchestra.run.smithi118:> set -ex 2023-11-24T06:09:19.267 DEBUG:teuthology.orchestra.run.smithi118:> sudo dd of=/etc/ceph/ceph.keyring 2023-11-24T06:09:19.267 DEBUG:teuthology.orchestra.run.smithi118:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-11-24T06:09:19.424 DEBUG:teuthology.orchestra.run.smithi118:> set -ex 2023-11-24T06:09:19.424 DEBUG:teuthology.orchestra.run.smithi118:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-11-24T06:09:19.522 INFO:tasks.ceph:Setting up mon nodes... 2023-11-24T06:09:19.523 INFO:tasks.ceph:Setting up mgr nodes... 2023-11-24T06:09:19.523 DEBUG:teuthology.orchestra.run.smithi092:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2023-11-24T06:09:19.712 INFO:teuthology.orchestra.run.smithi092.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2023-11-24T06:09:19.722 DEBUG:teuthology.orchestra.run.smithi118:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2023-11-24T06:09:19.927 INFO:teuthology.orchestra.run.smithi118.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2023-11-24T06:09:19.938 INFO:tasks.ceph:Setting up mds nodes... 2023-11-24T06:09:19.938 DEBUG:teuthology.orchestra.run.smithi092:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2023-11-24T06:09:20.127 INFO:teuthology.orchestra.run.smithi092.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2023-11-24T06:09:20.136 DEBUG:teuthology.orchestra.run.smithi092:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2023-11-24T06:09:20.235 DEBUG:teuthology.orchestra.run.smithi092:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2023-11-24T06:09:20.419 INFO:teuthology.orchestra.run.smithi092.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2023-11-24T06:09:20.428 DEBUG:teuthology.orchestra.run.smithi092:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2023-11-24T06:09:20.525 DEBUG:teuthology.orchestra.run.smithi118:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2023-11-24T06:09:20.719 INFO:teuthology.orchestra.run.smithi118.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2023-11-24T06:09:20.728 DEBUG:teuthology.orchestra.run.smithi118:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2023-11-24T06:09:20.826 DEBUG:teuthology.orchestra.run.smithi118:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2023-11-24T06:09:21.004 INFO:teuthology.orchestra.run.smithi118.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2023-11-24T06:09:21.013 DEBUG:teuthology.orchestra.run.smithi118:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2023-11-24T06:09:21.106 INFO:tasks.ceph_client:Setting up client nodes... 2023-11-24T06:09:21.107 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-11-24T06:09:21.239 INFO:teuthology.orchestra.run.smithi092.stdout:creating /etc/ceph/ceph.client.0.keyring 2023-11-24T06:09:21.287 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2023-11-24T06:09:21.412 INFO:teuthology.orchestra.run.smithi092.stdout:creating /etc/ceph/ceph.client.2.keyring 2023-11-24T06:09:21.468 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2023-11-24T06:09:21.603 INFO:teuthology.orchestra.run.smithi118.stdout:creating /etc/ceph/ceph.client.1.keyring 2023-11-24T06:09:21.656 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2023-11-24T06:09:21.782 INFO:teuthology.orchestra.run.smithi118.stdout:creating /etc/ceph/ceph.client.3.keyring 2023-11-24T06:09:21.838 INFO:tasks.ceph:Running mkfs on osd nodes... 2023-11-24T06:09:21.838 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi092.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi118.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-11-24T06:09:21.838 DEBUG:teuthology.orchestra.run.smithi092:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2023-11-24T06:09:21.938 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-11-24T06:09:21.938 INFO:tasks.ceph:role: osd.0 2023-11-24T06:09:21.938 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi092.front.sepia.ceph.com 2023-11-24T06:09:21.939 DEBUG:teuthology.orchestra.run.smithi092:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-11-24T06:09:22.037 INFO:teuthology.orchestra.run.smithi092.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-11-24T06:09:22.038 INFO:teuthology.orchestra.run.smithi092.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-24T06:09:22.038 INFO:teuthology.orchestra.run.smithi092.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-24T06:09:22.038 INFO:teuthology.orchestra.run.smithi092.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-11-24T06:09:22.038 INFO:teuthology.orchestra.run.smithi092.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-24T06:09:22.038 INFO:teuthology.orchestra.run.smithi092.stdout: = sunit=0 swidth=0 blks 2023-11-24T06:09:22.038 INFO:teuthology.orchestra.run.smithi092.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-24T06:09:22.038 INFO:teuthology.orchestra.run.smithi092.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-11-24T06:09:22.038 INFO:teuthology.orchestra.run.smithi092.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-24T06:09:22.038 INFO:teuthology.orchestra.run.smithi092.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-24T06:09:22.559 INFO:teuthology.orchestra.run.smithi092.stdout:Discarding blocks...Done. 2023-11-24T06:09:22.566 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi092.front.sepia.ceph.com -o noatime 2023-11-24T06:09:22.567 DEBUG:teuthology.orchestra.run.smithi092:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2023-11-24T06:09:22.759 DEBUG:teuthology.orchestra.run.smithi092:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2023-11-24T06:09:22.871 DEBUG:teuthology.orchestra.run.smithi092:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2023-11-24T06:09:22.955 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-11-24T06:09:22.955 INFO:tasks.ceph:role: osd.1 2023-11-24T06:09:22.955 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi092.front.sepia.ceph.com 2023-11-24T06:09:22.955 DEBUG:teuthology.orchestra.run.smithi092:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-11-24T06:09:23.046 INFO:teuthology.orchestra.run.smithi092.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-11-24T06:09:23.046 INFO:teuthology.orchestra.run.smithi092.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-24T06:09:23.046 INFO:teuthology.orchestra.run.smithi092.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-24T06:09:23.046 INFO:teuthology.orchestra.run.smithi092.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-11-24T06:09:23.046 INFO:teuthology.orchestra.run.smithi092.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-24T06:09:23.046 INFO:teuthology.orchestra.run.smithi092.stdout: = sunit=0 swidth=0 blks 2023-11-24T06:09:23.046 INFO:teuthology.orchestra.run.smithi092.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-24T06:09:23.047 INFO:teuthology.orchestra.run.smithi092.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-11-24T06:09:23.047 INFO:teuthology.orchestra.run.smithi092.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-24T06:09:23.047 INFO:teuthology.orchestra.run.smithi092.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-24T06:09:23.522 INFO:teuthology.orchestra.run.smithi092.stdout:Discarding blocks...Done. 2023-11-24T06:09:23.529 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi092.front.sepia.ceph.com -o noatime 2023-11-24T06:09:23.529 DEBUG:teuthology.orchestra.run.smithi092:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2023-11-24T06:09:23.656 DEBUG:teuthology.orchestra.run.smithi092:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2023-11-24T06:09:23.738 DEBUG:teuthology.orchestra.run.smithi092:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2023-11-24T06:09:23.817 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-11-24T06:09:23.817 INFO:tasks.ceph:role: osd.2 2023-11-24T06:09:23.817 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi092.front.sepia.ceph.com 2023-11-24T06:09:23.817 DEBUG:teuthology.orchestra.run.smithi092:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-11-24T06:09:23.896 INFO:teuthology.orchestra.run.smithi092.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-11-24T06:09:23.896 INFO:teuthology.orchestra.run.smithi092.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-24T06:09:23.896 INFO:teuthology.orchestra.run.smithi092.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-24T06:09:23.897 INFO:teuthology.orchestra.run.smithi092.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-11-24T06:09:23.897 INFO:teuthology.orchestra.run.smithi092.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-24T06:09:23.897 INFO:teuthology.orchestra.run.smithi092.stdout: = sunit=0 swidth=0 blks 2023-11-24T06:09:23.897 INFO:teuthology.orchestra.run.smithi092.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-24T06:09:23.897 INFO:teuthology.orchestra.run.smithi092.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-11-24T06:09:23.897 INFO:teuthology.orchestra.run.smithi092.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-24T06:09:23.897 INFO:teuthology.orchestra.run.smithi092.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-24T06:09:24.414 INFO:teuthology.orchestra.run.smithi092.stdout:Discarding blocks...Done. 2023-11-24T06:09:24.418 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi092.front.sepia.ceph.com -o noatime 2023-11-24T06:09:24.419 DEBUG:teuthology.orchestra.run.smithi092:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2023-11-24T06:09:24.606 DEBUG:teuthology.orchestra.run.smithi092:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2023-11-24T06:09:24.707 DEBUG:teuthology.orchestra.run.smithi092:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2023-11-24T06:09:24.800 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-11-24T06:09:24.800 INFO:tasks.ceph:role: osd.3 2023-11-24T06:09:24.801 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi092.front.sepia.ceph.com 2023-11-24T06:09:24.801 DEBUG:teuthology.orchestra.run.smithi092:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-11-24T06:09:24.902 INFO:teuthology.orchestra.run.smithi092.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-11-24T06:09:24.902 INFO:teuthology.orchestra.run.smithi092.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-24T06:09:24.903 INFO:teuthology.orchestra.run.smithi092.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-24T06:09:24.903 INFO:teuthology.orchestra.run.smithi092.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-11-24T06:09:24.903 INFO:teuthology.orchestra.run.smithi092.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-24T06:09:24.903 INFO:teuthology.orchestra.run.smithi092.stdout: = sunit=0 swidth=0 blks 2023-11-24T06:09:24.903 INFO:teuthology.orchestra.run.smithi092.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-24T06:09:24.903 INFO:teuthology.orchestra.run.smithi092.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-11-24T06:09:24.903 INFO:teuthology.orchestra.run.smithi092.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-24T06:09:24.903 INFO:teuthology.orchestra.run.smithi092.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-24T06:09:25.363 INFO:teuthology.orchestra.run.smithi092.stdout:Discarding blocks...Done. 2023-11-24T06:09:25.371 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi092.front.sepia.ceph.com -o noatime 2023-11-24T06:09:25.371 DEBUG:teuthology.orchestra.run.smithi092:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2023-11-24T06:09:25.519 DEBUG:teuthology.orchestra.run.smithi092:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2023-11-24T06:09:25.614 DEBUG:teuthology.orchestra.run.smithi092:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-24T06:09:25.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:09:25.756+0000 7f9b70a11500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2023-11-24T06:09:25.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:09:25.758+0000 7f9b70a11500 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2023-11-24T06:09:25.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:09:25.758+0000 7f9b70a11500 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-11-24T06:09:25.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:09:25.758+0000 7f9b70a11500 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-11-24T06:09:25.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:09:25.758+0000 7f9b70a11500 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-11-24T06:09:25.766 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:09:25.763+0000 7f9b70a11500 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2023-11-24T06:09:28.929 DEBUG:teuthology.orchestra.run.smithi092:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2023-11-24T06:09:29.031 DEBUG:teuthology.orchestra.run.smithi092:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-24T06:09:29.170 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:09:29.166+0000 7f09ef00c500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2023-11-24T06:09:29.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:09:29.167+0000 7f09ef00c500 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2023-11-24T06:09:29.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:09:29.167+0000 7f09ef00c500 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-11-24T06:09:29.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:09:29.167+0000 7f09ef00c500 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-11-24T06:09:29.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:09:29.167+0000 7f09ef00c500 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-11-24T06:09:29.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:09:29.170+0000 7f09ef00c500 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2023-11-24T06:09:32.346 DEBUG:teuthology.orchestra.run.smithi092:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2023-11-24T06:09:32.444 DEBUG:teuthology.orchestra.run.smithi092:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-24T06:09:32.578 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:09:32.575+0000 7f15cd81a500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2023-11-24T06:09:32.579 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:09:32.575+0000 7f15cd81a500 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2023-11-24T06:09:32.579 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:09:32.575+0000 7f15cd81a500 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-11-24T06:09:32.579 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:09:32.575+0000 7f15cd81a500 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-11-24T06:09:32.579 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:09:32.575+0000 7f15cd81a500 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-11-24T06:09:32.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:09:32.577+0000 7f15cd81a500 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2023-11-24T06:09:35.497 DEBUG:teuthology.orchestra.run.smithi092:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2023-11-24T06:09:35.597 DEBUG:teuthology.orchestra.run.smithi092:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-24T06:09:35.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:09:35.733+0000 7f0e28407500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2023-11-24T06:09:35.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:09:35.733+0000 7f0e28407500 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2023-11-24T06:09:35.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:09:35.733+0000 7f0e28407500 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-11-24T06:09:35.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:09:35.733+0000 7f0e28407500 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-11-24T06:09:35.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:09:35.733+0000 7f0e28407500 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-11-24T06:09:35.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:09:35.735+0000 7f0e28407500 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2023-11-24T06:09:38.907 DEBUG:teuthology.orchestra.run.smithi092:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2023-11-24T06:09:39.008 DEBUG:teuthology.orchestra.run.smithi118:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2023-11-24T06:09:39.117 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-11-24T06:09:39.117 INFO:tasks.ceph:role: osd.4 2023-11-24T06:09:39.117 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi118.front.sepia.ceph.com 2023-11-24T06:09:39.118 DEBUG:teuthology.orchestra.run.smithi118:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-11-24T06:09:39.216 INFO:teuthology.orchestra.run.smithi118.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-11-24T06:09:39.216 INFO:teuthology.orchestra.run.smithi118.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-24T06:09:39.216 INFO:teuthology.orchestra.run.smithi118.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-24T06:09:39.217 INFO:teuthology.orchestra.run.smithi118.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-11-24T06:09:39.217 INFO:teuthology.orchestra.run.smithi118.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-24T06:09:39.217 INFO:teuthology.orchestra.run.smithi118.stdout: = sunit=0 swidth=0 blks 2023-11-24T06:09:39.217 INFO:teuthology.orchestra.run.smithi118.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-24T06:09:39.217 INFO:teuthology.orchestra.run.smithi118.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-11-24T06:09:39.217 INFO:teuthology.orchestra.run.smithi118.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-24T06:09:39.217 INFO:teuthology.orchestra.run.smithi118.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-24T06:09:39.251 INFO:teuthology.orchestra.run.smithi118.stdout:Discarding blocks...Done. 2023-11-24T06:09:39.258 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi118.front.sepia.ceph.com -o noatime 2023-11-24T06:09:39.258 DEBUG:teuthology.orchestra.run.smithi118:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2023-11-24T06:09:39.376 DEBUG:teuthology.orchestra.run.smithi118:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2023-11-24T06:09:39.463 DEBUG:teuthology.orchestra.run.smithi118:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2023-11-24T06:09:39.542 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-11-24T06:09:39.542 INFO:tasks.ceph:role: osd.5 2023-11-24T06:09:39.542 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi118.front.sepia.ceph.com 2023-11-24T06:09:39.542 DEBUG:teuthology.orchestra.run.smithi118:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-11-24T06:09:39.618 INFO:teuthology.orchestra.run.smithi118.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-11-24T06:09:39.618 INFO:teuthology.orchestra.run.smithi118.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-24T06:09:39.618 INFO:teuthology.orchestra.run.smithi118.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-24T06:09:39.618 INFO:teuthology.orchestra.run.smithi118.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-11-24T06:09:39.619 INFO:teuthology.orchestra.run.smithi118.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-24T06:09:39.619 INFO:teuthology.orchestra.run.smithi118.stdout: = sunit=0 swidth=0 blks 2023-11-24T06:09:39.619 INFO:teuthology.orchestra.run.smithi118.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-24T06:09:39.619 INFO:teuthology.orchestra.run.smithi118.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-11-24T06:09:39.619 INFO:teuthology.orchestra.run.smithi118.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-24T06:09:39.619 INFO:teuthology.orchestra.run.smithi118.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-24T06:09:39.652 INFO:teuthology.orchestra.run.smithi118.stdout:Discarding blocks...Done. 2023-11-24T06:09:39.657 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi118.front.sepia.ceph.com -o noatime 2023-11-24T06:09:39.657 DEBUG:teuthology.orchestra.run.smithi118:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2023-11-24T06:09:39.766 DEBUG:teuthology.orchestra.run.smithi118:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2023-11-24T06:09:39.844 DEBUG:teuthology.orchestra.run.smithi118:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2023-11-24T06:09:39.913 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-11-24T06:09:39.913 INFO:tasks.ceph:role: osd.6 2023-11-24T06:09:39.913 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi118.front.sepia.ceph.com 2023-11-24T06:09:39.914 DEBUG:teuthology.orchestra.run.smithi118:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-11-24T06:09:39.988 INFO:teuthology.orchestra.run.smithi118.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-11-24T06:09:39.988 INFO:teuthology.orchestra.run.smithi118.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-24T06:09:39.988 INFO:teuthology.orchestra.run.smithi118.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-24T06:09:39.988 INFO:teuthology.orchestra.run.smithi118.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-11-24T06:09:39.988 INFO:teuthology.orchestra.run.smithi118.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-24T06:09:39.988 INFO:teuthology.orchestra.run.smithi118.stdout: = sunit=0 swidth=0 blks 2023-11-24T06:09:39.988 INFO:teuthology.orchestra.run.smithi118.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-24T06:09:39.988 INFO:teuthology.orchestra.run.smithi118.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-11-24T06:09:39.988 INFO:teuthology.orchestra.run.smithi118.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-24T06:09:39.989 INFO:teuthology.orchestra.run.smithi118.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-24T06:09:40.021 INFO:teuthology.orchestra.run.smithi118.stdout:Discarding blocks...Done. 2023-11-24T06:09:40.027 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi118.front.sepia.ceph.com -o noatime 2023-11-24T06:09:40.027 DEBUG:teuthology.orchestra.run.smithi118:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2023-11-24T06:09:40.129 DEBUG:teuthology.orchestra.run.smithi118:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2023-11-24T06:09:40.204 DEBUG:teuthology.orchestra.run.smithi118:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2023-11-24T06:09:40.278 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-11-24T06:09:40.278 INFO:tasks.ceph:role: osd.7 2023-11-24T06:09:40.278 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi118.front.sepia.ceph.com 2023-11-24T06:09:40.279 DEBUG:teuthology.orchestra.run.smithi118:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-11-24T06:09:40.355 INFO:teuthology.orchestra.run.smithi118.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-11-24T06:09:40.355 INFO:teuthology.orchestra.run.smithi118.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-24T06:09:40.355 INFO:teuthology.orchestra.run.smithi118.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-24T06:09:40.355 INFO:teuthology.orchestra.run.smithi118.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-11-24T06:09:40.355 INFO:teuthology.orchestra.run.smithi118.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-24T06:09:40.355 INFO:teuthology.orchestra.run.smithi118.stdout: = sunit=0 swidth=0 blks 2023-11-24T06:09:40.355 INFO:teuthology.orchestra.run.smithi118.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-24T06:09:40.356 INFO:teuthology.orchestra.run.smithi118.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-11-24T06:09:40.356 INFO:teuthology.orchestra.run.smithi118.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-24T06:09:40.356 INFO:teuthology.orchestra.run.smithi118.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-24T06:09:40.392 INFO:teuthology.orchestra.run.smithi118.stdout:Discarding blocks...Done. 2023-11-24T06:09:40.396 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi118.front.sepia.ceph.com -o noatime 2023-11-24T06:09:40.397 DEBUG:teuthology.orchestra.run.smithi118:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2023-11-24T06:09:40.494 DEBUG:teuthology.orchestra.run.smithi118:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2023-11-24T06:09:40.575 DEBUG:teuthology.orchestra.run.smithi118:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-24T06:09:40.731 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:09:40.727+0000 7f886ee12500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2023-11-24T06:09:40.732 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:09:40.728+0000 7f886ee12500 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2023-11-24T06:09:40.732 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:09:40.728+0000 7f886ee12500 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-11-24T06:09:40.732 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:09:40.728+0000 7f886ee12500 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-11-24T06:09:40.732 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:09:40.728+0000 7f886ee12500 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-11-24T06:09:40.734 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:09:40.731+0000 7f886ee12500 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2023-11-24T06:09:43.918 DEBUG:teuthology.orchestra.run.smithi118:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2023-11-24T06:09:44.015 DEBUG:teuthology.orchestra.run.smithi118:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-24T06:09:44.150 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:09:44.146+0000 7fce2f1f7500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2023-11-24T06:09:44.151 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:09:44.147+0000 7fce2f1f7500 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2023-11-24T06:09:44.151 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:09:44.147+0000 7fce2f1f7500 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-11-24T06:09:44.151 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:09:44.147+0000 7fce2f1f7500 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-11-24T06:09:44.152 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:09:44.147+0000 7fce2f1f7500 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-11-24T06:09:44.153 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:09:44.149+0000 7fce2f1f7500 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2023-11-24T06:09:45.971 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T06:09:45.980 DEBUG:teuthology.orchestra.run.smithi118:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T06:09:47.322 DEBUG:teuthology.orchestra.run.smithi118:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2023-11-24T06:09:47.420 DEBUG:teuthology.orchestra.run.smithi118:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-24T06:09:47.558 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:09:47.554+0000 7f7708414500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2023-11-24T06:09:47.559 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:09:47.555+0000 7f7708414500 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2023-11-24T06:09:47.559 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:09:47.555+0000 7f7708414500 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-11-24T06:09:47.559 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:09:47.555+0000 7f7708414500 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-11-24T06:09:47.559 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:09:47.555+0000 7f7708414500 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-11-24T06:09:47.560 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:09:47.557+0000 7f7708414500 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2023-11-24T06:09:50.472 DEBUG:teuthology.orchestra.run.smithi118:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2023-11-24T06:09:50.569 DEBUG:teuthology.orchestra.run.smithi118:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-24T06:09:50.698 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:09:50.695+0000 7fdb63e1c500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2023-11-24T06:09:50.699 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:09:50.696+0000 7fdb63e1c500 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2023-11-24T06:09:50.700 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:09:50.696+0000 7fdb63e1c500 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-11-24T06:09:50.700 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:09:50.696+0000 7fdb63e1c500 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-11-24T06:09:50.700 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:09:50.696+0000 7fdb63e1c500 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-11-24T06:09:50.701 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:09:50.698+0000 7fdb63e1c500 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2023-11-24T06:09:53.635 DEBUG:teuthology.orchestra.run.smithi118:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2023-11-24T06:09:53.738 INFO:tasks.ceph:Reading keys from all nodes... 2023-11-24T06:09:53.738 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-11-24T06:09:53.738 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2023-11-24T06:09:53.840 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-11-24T06:09:53.840 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2023-11-24T06:09:53.935 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-11-24T06:09:53.935 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2023-11-24T06:09:54.029 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-11-24T06:09:54.029 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2023-11-24T06:09:54.124 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-11-24T06:09:54.124 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2023-11-24T06:09:54.214 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-11-24T06:09:54.215 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2023-11-24T06:09:54.311 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-11-24T06:09:54.311 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2023-11-24T06:09:54.411 DEBUG:teuthology.orchestra.run.smithi118:> set -ex 2023-11-24T06:09:54.411 DEBUG:teuthology.orchestra.run.smithi118:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2023-11-24T06:09:54.512 DEBUG:teuthology.orchestra.run.smithi118:> set -ex 2023-11-24T06:09:54.512 DEBUG:teuthology.orchestra.run.smithi118:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2023-11-24T06:09:54.604 DEBUG:teuthology.orchestra.run.smithi118:> set -ex 2023-11-24T06:09:54.604 DEBUG:teuthology.orchestra.run.smithi118:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2023-11-24T06:09:54.698 DEBUG:teuthology.orchestra.run.smithi118:> set -ex 2023-11-24T06:09:54.698 DEBUG:teuthology.orchestra.run.smithi118:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2023-11-24T06:09:54.798 DEBUG:teuthology.orchestra.run.smithi118:> set -ex 2023-11-24T06:09:54.798 DEBUG:teuthology.orchestra.run.smithi118:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2023-11-24T06:09:54.891 DEBUG:teuthology.orchestra.run.smithi118:> set -ex 2023-11-24T06:09:54.891 DEBUG:teuthology.orchestra.run.smithi118:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2023-11-24T06:09:54.983 DEBUG:teuthology.orchestra.run.smithi118:> set -ex 2023-11-24T06:09:54.983 DEBUG:teuthology.orchestra.run.smithi118:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2023-11-24T06:09:55.082 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-11-24T06:09:55.082 DEBUG:teuthology.orchestra.run.smithi092:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2023-11-24T06:09:55.142 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-11-24T06:09:55.143 DEBUG:teuthology.orchestra.run.smithi092:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2023-11-24T06:09:55.197 DEBUG:teuthology.orchestra.run.smithi118:> set -ex 2023-11-24T06:09:55.197 DEBUG:teuthology.orchestra.run.smithi118:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2023-11-24T06:09:55.253 DEBUG:teuthology.orchestra.run.smithi118:> set -ex 2023-11-24T06:09:55.254 DEBUG:teuthology.orchestra.run.smithi118:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2023-11-24T06:09:55.300 INFO:tasks.ceph:Adding keys to all mons... 2023-11-24T06:09:55.300 DEBUG:teuthology.orchestra.run.smithi092:> sudo tee -a /etc/ceph/ceph.keyring 2023-11-24T06:09:55.309 DEBUG:teuthology.orchestra.run.smithi118:> sudo tee -a /etc/ceph/ceph.keyring 2023-11-24T06:09:55.400 INFO:teuthology.orchestra.run.smithi092.stdout:[mgr.y] 2023-11-24T06:09:55.400 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQAPPmBlyGxUKhAAWOTBKusZ4JOcJ1SaHJ6ZmQ== 2023-11-24T06:09:55.401 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.a] 2023-11-24T06:09:55.401 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQAQPmBl501xBxAAEtXTFSRjPeIoKJpcPILQwA== 2023-11-24T06:09:55.401 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.c] 2023-11-24T06:09:55.401 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQAQPmBl4VbbGBAAkJwrHXAC4d5S2hdPIcE/Cg== 2023-11-24T06:09:55.401 INFO:teuthology.orchestra.run.smithi092.stdout:[osd.0] 2023-11-24T06:09:55.401 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQAVPmBlr+4mLRAAgsX9kvv5frsFc+E1VoO8Fw== 2023-11-24T06:09:55.401 INFO:teuthology.orchestra.run.smithi092.stdout:[osd.1] 2023-11-24T06:09:55.401 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQAZPmBl7IUIChAA/DL645CltqHdzRS8r4Lksg== 2023-11-24T06:09:55.401 INFO:teuthology.orchestra.run.smithi092.stdout:[osd.2] 2023-11-24T06:09:55.401 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQAcPmBlJ5VbIhAAEnc+QX3whpwM0Ok9WD8dcA== 2023-11-24T06:09:55.401 INFO:teuthology.orchestra.run.smithi092.stdout:[osd.3] 2023-11-24T06:09:55.401 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQAfPmBlKdS9KxAAbdaBTd1CEf/A6RyKxc9t6A== 2023-11-24T06:09:55.402 INFO:teuthology.orchestra.run.smithi092.stdout:[mgr.x] 2023-11-24T06:09:55.402 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQAPPmBl1LwlNxAAsk8YjJh9jW0OCcGdfnBA1w== 2023-11-24T06:09:55.402 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.b] 2023-11-24T06:09:55.402 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQAQPmBlaY/AKhAArTolS+ALdzAB8M0lQ3/Mjg== 2023-11-24T06:09:55.402 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.d] 2023-11-24T06:09:55.402 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQARPmBl00MmABAAvqEvZ0bL8D1oHg4gcKXEyA== 2023-11-24T06:09:55.402 INFO:teuthology.orchestra.run.smithi092.stdout:[osd.4] 2023-11-24T06:09:55.402 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQAkPmBlnOJ6KxAAuLTyFpOBStxC5FevOc4pKw== 2023-11-24T06:09:55.402 INFO:teuthology.orchestra.run.smithi092.stdout:[osd.5] 2023-11-24T06:09:55.402 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQAoPmBlrkDXCBAAwv3u79yJeeTjreYWfr/yvQ== 2023-11-24T06:09:55.402 INFO:teuthology.orchestra.run.smithi092.stdout:[osd.6] 2023-11-24T06:09:55.403 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQArPmBlP1AoIRAA8D8nUCwXZRHZmB9d73fYbQ== 2023-11-24T06:09:55.403 INFO:teuthology.orchestra.run.smithi092.stdout:[osd.7] 2023-11-24T06:09:55.403 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQAuPmBlUBKMKRAA3obYjr9s13SuYnbFt+Y0bQ== 2023-11-24T06:09:55.403 INFO:teuthology.orchestra.run.smithi092.stdout:[client.0] 2023-11-24T06:09:55.403 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQARPmBlGMUsDhAAlgrJvA/TT5hpUYd5o9OzIQ== 2023-11-24T06:09:55.403 INFO:teuthology.orchestra.run.smithi092.stdout:[client.2] 2023-11-24T06:09:55.403 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQARPmBlIqJ9GBAA46UmMl1hl/522waZSndXhw== 2023-11-24T06:09:55.403 INFO:teuthology.orchestra.run.smithi092.stdout:[client.1] 2023-11-24T06:09:55.403 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQARPmBlOeviIxAA/egv9DYEDJ+1yKDFbsY1pw== 2023-11-24T06:09:55.403 INFO:teuthology.orchestra.run.smithi092.stdout:[client.3] 2023-11-24T06:09:55.403 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQARPmBlQRCDLhAA80Oc0yKHPMD5EjVxqurfPw== 2023-11-24T06:09:55.437 INFO:teuthology.orchestra.run.smithi118.stdout:[mgr.y] 2023-11-24T06:09:55.437 INFO:teuthology.orchestra.run.smithi118.stdout: key = AQAPPmBlyGxUKhAAWOTBKusZ4JOcJ1SaHJ6ZmQ== 2023-11-24T06:09:55.437 INFO:teuthology.orchestra.run.smithi118.stdout:[mds.a] 2023-11-24T06:09:55.438 INFO:teuthology.orchestra.run.smithi118.stdout: key = AQAQPmBl501xBxAAEtXTFSRjPeIoKJpcPILQwA== 2023-11-24T06:09:55.438 INFO:teuthology.orchestra.run.smithi118.stdout:[mds.c] 2023-11-24T06:09:55.438 INFO:teuthology.orchestra.run.smithi118.stdout: key = AQAQPmBl4VbbGBAAkJwrHXAC4d5S2hdPIcE/Cg== 2023-11-24T06:09:55.438 INFO:teuthology.orchestra.run.smithi118.stdout:[osd.0] 2023-11-24T06:09:55.438 INFO:teuthology.orchestra.run.smithi118.stdout: key = AQAVPmBlr+4mLRAAgsX9kvv5frsFc+E1VoO8Fw== 2023-11-24T06:09:55.438 INFO:teuthology.orchestra.run.smithi118.stdout:[osd.1] 2023-11-24T06:09:55.438 INFO:teuthology.orchestra.run.smithi118.stdout: key = AQAZPmBl7IUIChAA/DL645CltqHdzRS8r4Lksg== 2023-11-24T06:09:55.439 INFO:teuthology.orchestra.run.smithi118.stdout:[osd.2] 2023-11-24T06:09:55.439 INFO:teuthology.orchestra.run.smithi118.stdout: key = AQAcPmBlJ5VbIhAAEnc+QX3whpwM0Ok9WD8dcA== 2023-11-24T06:09:55.439 INFO:teuthology.orchestra.run.smithi118.stdout:[osd.3] 2023-11-24T06:09:55.439 INFO:teuthology.orchestra.run.smithi118.stdout: key = AQAfPmBlKdS9KxAAbdaBTd1CEf/A6RyKxc9t6A== 2023-11-24T06:09:55.439 INFO:teuthology.orchestra.run.smithi118.stdout:[mgr.x] 2023-11-24T06:09:55.439 INFO:teuthology.orchestra.run.smithi118.stdout: key = AQAPPmBl1LwlNxAAsk8YjJh9jW0OCcGdfnBA1w== 2023-11-24T06:09:55.439 INFO:teuthology.orchestra.run.smithi118.stdout:[mds.b] 2023-11-24T06:09:55.439 INFO:teuthology.orchestra.run.smithi118.stdout: key = AQAQPmBlaY/AKhAArTolS+ALdzAB8M0lQ3/Mjg== 2023-11-24T06:09:55.440 INFO:teuthology.orchestra.run.smithi118.stdout:[mds.d] 2023-11-24T06:09:55.440 INFO:teuthology.orchestra.run.smithi118.stdout: key = AQARPmBl00MmABAAvqEvZ0bL8D1oHg4gcKXEyA== 2023-11-24T06:09:55.440 INFO:teuthology.orchestra.run.smithi118.stdout:[osd.4] 2023-11-24T06:09:55.440 INFO:teuthology.orchestra.run.smithi118.stdout: key = AQAkPmBlnOJ6KxAAuLTyFpOBStxC5FevOc4pKw== 2023-11-24T06:09:55.440 INFO:teuthology.orchestra.run.smithi118.stdout:[osd.5] 2023-11-24T06:09:55.440 INFO:teuthology.orchestra.run.smithi118.stdout: key = AQAoPmBlrkDXCBAAwv3u79yJeeTjreYWfr/yvQ== 2023-11-24T06:09:55.440 INFO:teuthology.orchestra.run.smithi118.stdout:[osd.6] 2023-11-24T06:09:55.441 INFO:teuthology.orchestra.run.smithi118.stdout: key = AQArPmBlP1AoIRAA8D8nUCwXZRHZmB9d73fYbQ== 2023-11-24T06:09:55.441 INFO:teuthology.orchestra.run.smithi118.stdout:[osd.7] 2023-11-24T06:09:55.441 INFO:teuthology.orchestra.run.smithi118.stdout: key = AQAuPmBlUBKMKRAA3obYjr9s13SuYnbFt+Y0bQ== 2023-11-24T06:09:55.441 INFO:teuthology.orchestra.run.smithi118.stdout:[client.0] 2023-11-24T06:09:55.441 INFO:teuthology.orchestra.run.smithi118.stdout: key = AQARPmBlGMUsDhAAlgrJvA/TT5hpUYd5o9OzIQ== 2023-11-24T06:09:55.441 INFO:teuthology.orchestra.run.smithi118.stdout:[client.2] 2023-11-24T06:09:55.442 INFO:teuthology.orchestra.run.smithi118.stdout: key = AQARPmBlIqJ9GBAA46UmMl1hl/522waZSndXhw== 2023-11-24T06:09:55.442 INFO:teuthology.orchestra.run.smithi118.stdout:[client.1] 2023-11-24T06:09:55.442 INFO:teuthology.orchestra.run.smithi118.stdout: key = AQARPmBlOeviIxAA/egv9DYEDJ+1yKDFbsY1pw== 2023-11-24T06:09:55.442 INFO:teuthology.orchestra.run.smithi118.stdout:[client.3] 2023-11-24T06:09:55.442 INFO:teuthology.orchestra.run.smithi118.stdout: key = AQARPmBlQRCDLhAA80Oc0yKHPMD5EjVxqurfPw== 2023-11-24T06:09:55.444 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-11-24T06:09:55.452 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-11-24T06:09:55.596 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-24T06:09:55.601 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-24T06:09:55.738 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-24T06:09:55.744 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-24T06:09:55.891 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-24T06:09:55.900 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-24T06:09:56.043 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-24T06:09:56.050 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-24T06:09:56.198 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-24T06:09:56.204 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-24T06:09:56.343 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-24T06:09:56.349 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-24T06:09:56.491 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-11-24T06:09:56.500 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-11-24T06:09:56.635 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-24T06:09:56.642 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-24T06:09:56.791 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-24T06:09:56.799 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-24T06:09:56.946 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-24T06:09:56.951 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-24T06:09:57.095 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-24T06:09:57.100 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-24T06:09:57.246 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-24T06:09:57.251 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-24T06:09:57.399 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-24T06:09:57.405 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-24T06:09:57.547 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-24T06:09:57.551 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-24T06:09:57.701 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-24T06:09:57.709 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-24T06:09:57.862 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-24T06:09:57.867 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-24T06:09:58.011 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-24T06:09:58.016 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-24T06:09:58.164 INFO:tasks.ceph:Running mkfs on mon nodes... 2023-11-24T06:09:58.164 DEBUG:teuthology.orchestra.run.smithi092:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2023-11-24T06:09:58.261 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-11-24T06:09:58.420 DEBUG:teuthology.orchestra.run.smithi092:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2023-11-24T06:09:58.513 DEBUG:teuthology.orchestra.run.smithi118:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2023-11-24T06:09:58.611 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-11-24T06:09:58.770 DEBUG:teuthology.orchestra.run.smithi118:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2023-11-24T06:09:58.866 DEBUG:teuthology.orchestra.run.smithi118:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2023-11-24T06:09:58.962 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-11-24T06:09:59.111 DEBUG:teuthology.orchestra.run.smithi118:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2023-11-24T06:09:59.203 DEBUG:teuthology.orchestra.run.smithi092:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-11-24T06:09:59.212 DEBUG:teuthology.orchestra.run.smithi118:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-11-24T06:09:59.265 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2023-11-24T06:09:59.265 INFO:tasks.ceph.mon.a:Restarting daemon 2023-11-24T06:09:59.265 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2023-11-24T06:09:59.271 INFO:tasks.ceph.mon.a:Started 2023-11-24T06:09:59.272 INFO:tasks.ceph.mon.b:Restarting daemon 2023-11-24T06:09:59.272 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2023-11-24T06:09:59.280 INFO:tasks.ceph.mon.b:Started 2023-11-24T06:09:59.280 INFO:tasks.ceph.mon.c:Restarting daemon 2023-11-24T06:09:59.280 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2023-11-24T06:09:59.288 INFO:tasks.ceph.mon.c:Started 2023-11-24T06:09:59.288 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2023-11-24T06:09:59.288 INFO:tasks.ceph.mgr.y:Restarting daemon 2023-11-24T06:09:59.288 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2023-11-24T06:09:59.296 INFO:tasks.ceph.mgr.y:Started 2023-11-24T06:09:59.296 INFO:tasks.ceph.mgr.x:Restarting daemon 2023-11-24T06:09:59.296 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2023-11-24T06:09:59.301 INFO:tasks.ceph.mgr.x:Started 2023-11-24T06:09:59.301 INFO:tasks.ceph:Setting crush tunables to default 2023-11-24T06:09:59.301 DEBUG:teuthology.orchestra.run.smithi092:> sudo ceph --cluster ceph osd crush tunables default 2023-11-24T06:09:59.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:09:59.475+0000 7f0f738cf640 1 Processor -- start 2023-11-24T06:09:59.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:09:59.475+0000 7f0f738cf640 1 -- start start 2023-11-24T06:09:59.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:09:59.477+0000 7f0f738cf640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f0f6c143ae0 0x7f0f6c0cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:59.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:09:59.477+0000 7f0f738cf640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f6c0cfc30 con 0x7f0f6c0cf7d0 2023-11-24T06:09:59.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:09:59.477+0000 7f0f738cf640 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f6c0b1ee0 con 0x7f0f6c15e3b0 2023-11-24T06:09:59.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:09:59.477+0000 7f0f738cf640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f6c0b2780 con 0x7f0f6c143ae0 2023-11-24T06:09:59.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:09:59.477+0000 7f0f7166b640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f0f6c143ae0 0x7f0f6c0cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:59.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:09:59.477+0000 7f0f7166b640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f0f6c143ae0 0x7f0f6c0cf290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44328/0 (socket says 172.21.15.92:44328) 2023-11-24T06:09:59.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:09:59.477+0000 7f0f7166b640 1 -- 172.21.15.92:0/3333076453 learned_addr learned my addr 172.21.15.92:0/3333076453 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:09:59.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:09:59.477+0000 7f0f7166b640 1 -- 172.21.15.92:0/3333076453 >> v2:172.21.15.92:3300/0 conn(0x7f0f6c143ae0 msgr2=0x7f0f6c0cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-11-24T06:09:59.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:09:59.477+0000 7f0f7166b640 1 -- 172.21.15.92:0/3333076453 >> v2:172.21.15.92:3300/0 conn(0x7f0f6c143ae0 msgr2=0x7f0f6c0cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-24T06:09:59.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:09:59.477+0000 7f0f7166b640 1 --2- 172.21.15.92:0/3333076453 >> v2:172.21.15.92:3300/0 conn(0x7f0f6c143ae0 0x7f0f6c0cf290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-24T06:09:59.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:09:59.477+0000 7f0f7166b640 1 --2- 172.21.15.92:0/3333076453 >> v2:172.21.15.92:3300/0 conn(0x7f0f6c143ae0 0x7f0f6c0cf290 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-24T06:09:59.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:09:59.678+0000 7f0f7166b640 1 --2- 172.21.15.92:0/3333076453 >> v2:172.21.15.92:3300/0 conn(0x7f0f6c143ae0 0x7f0f6c0cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:59.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:09:59.678+0000 7f0f7166b640 1 -- 172.21.15.92:0/3333076453 >> v2:172.21.15.92:3300/0 conn(0x7f0f6c143ae0 msgr2=0x7f0f6c0cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-11-24T06:09:59.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:09:59.678+0000 7f0f7166b640 1 -- 172.21.15.92:0/3333076453 >> v2:172.21.15.92:3300/0 conn(0x7f0f6c143ae0 msgr2=0x7f0f6c0cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-24T06:09:59.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:09:59.678+0000 7f0f7166b640 1 --2- 172.21.15.92:0/3333076453 >> v2:172.21.15.92:3300/0 conn(0x7f0f6c143ae0 0x7f0f6c0cf290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-24T06:09:59.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:09:59.678+0000 7f0f7166b640 1 --2- 172.21.15.92:0/3333076453 >> v2:172.21.15.92:3300/0 conn(0x7f0f6c143ae0 0x7f0f6c0cf290 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-24T06:10:00.083 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:00.080+0000 7f0f7166b640 1 --2- 172.21.15.92:0/3333076453 >> v2:172.21.15.92:3300/0 conn(0x7f0f6c143ae0 0x7f0f6c0cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:00.083 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:00.080+0000 7f0f7166b640 1 -- 172.21.15.92:0/3333076453 >> v2:172.21.15.92:3300/0 conn(0x7f0f6c143ae0 msgr2=0x7f0f6c0cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-11-24T06:10:00.084 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:00.080+0000 7f0f7166b640 1 -- 172.21.15.92:0/3333076453 >> v2:172.21.15.92:3300/0 conn(0x7f0f6c143ae0 msgr2=0x7f0f6c0cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-24T06:10:00.084 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:00.080+0000 7f0f7166b640 1 --2- 172.21.15.92:0/3333076453 >> v2:172.21.15.92:3300/0 conn(0x7f0f6c143ae0 0x7f0f6c0cf290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-24T06:10:00.084 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:00.080+0000 7f0f7166b640 1 --2- 172.21.15.92:0/3333076453 >> v2:172.21.15.92:3300/0 conn(0x7f0f6c143ae0 0x7f0f6c0cf290 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-24T06:10:00.885 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:00.882+0000 7f0f7166b640 1 --2- 172.21.15.92:0/3333076453 >> v2:172.21.15.92:3300/0 conn(0x7f0f6c143ae0 0x7f0f6c0cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:00.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:00.882+0000 7f0f7166b640 1 -- 172.21.15.92:0/3333076453 >> v2:172.21.15.92:3300/0 conn(0x7f0f6c143ae0 msgr2=0x7f0f6c0cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-11-24T06:10:00.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:00.882+0000 7f0f7166b640 1 -- 172.21.15.92:0/3333076453 >> v2:172.21.15.92:3300/0 conn(0x7f0f6c143ae0 msgr2=0x7f0f6c0cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-24T06:10:00.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:00.882+0000 7f0f7166b640 1 --2- 172.21.15.92:0/3333076453 >> v2:172.21.15.92:3300/0 conn(0x7f0f6c143ae0 0x7f0f6c0cf290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-24T06:10:00.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:00.882+0000 7f0f7166b640 1 --2- 172.21.15.92:0/3333076453 >> v2:172.21.15.92:3300/0 conn(0x7f0f6c143ae0 0x7f0f6c0cf290 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-24T06:10:02.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:02.477+0000 7f0f62ffd640 1 -- 172.21.15.92:0/3333076453 >> v2:172.21.15.92:3300/0 conn(0x7f0f6c143ae0 msgr2=0x7f0f6c0cf290 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-24T06:10:02.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:02.477+0000 7f0f62ffd640 1 --2- 172.21.15.92:0/3333076453 >> v2:172.21.15.92:3300/0 conn(0x7f0f6c143ae0 0x7f0f6c0cf290 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:02.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:02.477+0000 7f0f62ffd640 1 -- 172.21.15.92:0/3333076453 >> v1:172.21.15.118:6789/0 conn(0x7f0f6c15e3b0 legacy=0x7f0f6c0702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:02.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:02.477+0000 7f0f62ffd640 1 -- 172.21.15.92:0/3333076453 >> v1:172.21.15.92:6789/0 conn(0x7f0f6c0cf7d0 legacy=0x7f0f6c15dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:02.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:02.477+0000 7f0f62ffd640 1 --2- 172.21.15.92:0/3333076453 >> v2:172.21.15.118:3300/0 conn(0x7f0f500036c0 0x7f0f50005b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:02.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:02.477+0000 7f0f62ffd640 1 --2- 172.21.15.92:0/3333076453 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0f50006180 0x7f0f50008620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:02.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:02.477+0000 7f0f62ffd640 1 -- 172.21.15.92:0/3333076453 --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f6c0b2780 con 0x7f0f500036c0 2023-11-24T06:10:02.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:02.477+0000 7f0f62ffd640 1 -- 172.21.15.92:0/3333076453 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f50008d20 con 0x7f0f50006180 2023-11-24T06:10:02.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:02.478+0000 7f0f7166b640 1 --2- 172.21.15.92:0/3333076453 >> v2:172.21.15.118:3300/0 conn(0x7f0f500036c0 0x7f0f50005b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:02.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:02.478+0000 7f0f70e6a640 1 --2- 172.21.15.92:0/3333076453 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0f50006180 0x7f0f50008620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:02.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:02.478+0000 7f0f7166b640 1 -- 172.21.15.92:0/3333076453 >> v2:172.21.15.118:3300/0 conn(0x7f0f500036c0 msgr2=0x7f0f50005b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-11-24T06:10:02.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:02.478+0000 7f0f7166b640 1 -- 172.21.15.92:0/3333076453 >> v2:172.21.15.118:3300/0 conn(0x7f0f500036c0 msgr2=0x7f0f50005b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-24T06:10:02.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:02.478+0000 7f0f7166b640 1 --2- 172.21.15.92:0/3333076453 >> v2:172.21.15.118:3300/0 conn(0x7f0f500036c0 0x7f0f50005b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-24T06:10:02.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:02.478+0000 7f0f7166b640 1 --2- 172.21.15.92:0/3333076453 >> v2:172.21.15.118:3300/0 conn(0x7f0f500036c0 0x7f0f50005b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-24T06:10:02.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:02.478+0000 7f0f70e6a640 1 -- 172.21.15.92:0/3333076453 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0f50006180 msgr2=0x7f0f50008620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-11-24T06:10:02.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:02.478+0000 7f0f70e6a640 1 -- 172.21.15.92:0/3333076453 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0f50006180 msgr2=0x7f0f50008620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-24T06:10:02.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:02.478+0000 7f0f70e6a640 1 --2- 172.21.15.92:0/3333076453 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0f50006180 0x7f0f50008620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-24T06:10:02.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:02.478+0000 7f0f70e6a640 1 --2- 172.21.15.92:0/3333076453 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0f50006180 0x7f0f50008620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-24T06:10:02.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:02.679+0000 7f0f7166b640 1 --2- 172.21.15.92:0/3333076453 >> v2:172.21.15.118:3300/0 conn(0x7f0f500036c0 0x7f0f50005b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:02.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:02.679+0000 7f0f70e6a640 1 --2- 172.21.15.92:0/3333076453 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0f50006180 0x7f0f50008620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:02.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:02.680+0000 7f0f7166b640 1 -- 172.21.15.92:0/3333076453 >> v2:172.21.15.118:3300/0 conn(0x7f0f500036c0 msgr2=0x7f0f50005b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-11-24T06:10:02.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:02.680+0000 7f0f7166b640 1 -- 172.21.15.92:0/3333076453 >> v2:172.21.15.118:3300/0 conn(0x7f0f500036c0 msgr2=0x7f0f50005b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-24T06:10:02.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:02.680+0000 7f0f7166b640 1 --2- 172.21.15.92:0/3333076453 >> v2:172.21.15.118:3300/0 conn(0x7f0f500036c0 0x7f0f50005b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-24T06:10:02.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:02.680+0000 7f0f70e6a640 1 -- 172.21.15.92:0/3333076453 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0f50006180 msgr2=0x7f0f50008620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-11-24T06:10:02.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:02.680+0000 7f0f70e6a640 1 -- 172.21.15.92:0/3333076453 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0f50006180 msgr2=0x7f0f50008620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-24T06:10:02.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:02.680+0000 7f0f70e6a640 1 --2- 172.21.15.92:0/3333076453 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0f50006180 0x7f0f50008620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-24T06:10:02.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:02.680+0000 7f0f7166b640 1 --2- 172.21.15.92:0/3333076453 >> v2:172.21.15.118:3300/0 conn(0x7f0f500036c0 0x7f0f50005b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-24T06:10:02.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:02.680+0000 7f0f70e6a640 1 --2- 172.21.15.92:0/3333076453 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0f50006180 0x7f0f50008620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-24T06:10:03.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:03.081+0000 7f0f7166b640 1 --2- 172.21.15.92:0/3333076453 >> v2:172.21.15.118:3300/0 conn(0x7f0f500036c0 0x7f0f50005b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:03.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:03.081+0000 7f0f70e6a640 1 --2- 172.21.15.92:0/3333076453 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0f50006180 0x7f0f50008620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:03.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:03.081+0000 7f0f7166b640 1 -- 172.21.15.92:0/3333076453 >> v2:172.21.15.118:3300/0 conn(0x7f0f500036c0 msgr2=0x7f0f50005b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-11-24T06:10:03.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:03.081+0000 7f0f7166b640 1 -- 172.21.15.92:0/3333076453 >> v2:172.21.15.118:3300/0 conn(0x7f0f500036c0 msgr2=0x7f0f50005b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-24T06:10:03.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:03.081+0000 7f0f7166b640 1 --2- 172.21.15.92:0/3333076453 >> v2:172.21.15.118:3300/0 conn(0x7f0f500036c0 0x7f0f50005b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-24T06:10:03.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:03.082+0000 7f0f70e6a640 1 -- 172.21.15.92:0/3333076453 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0f50006180 msgr2=0x7f0f50008620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-11-24T06:10:03.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:03.082+0000 7f0f70e6a640 1 -- 172.21.15.92:0/3333076453 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0f50006180 msgr2=0x7f0f50008620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-24T06:10:03.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:03.082+0000 7f0f70e6a640 1 --2- 172.21.15.92:0/3333076453 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0f50006180 0x7f0f50008620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-24T06:10:03.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:03.082+0000 7f0f7166b640 1 --2- 172.21.15.92:0/3333076453 >> v2:172.21.15.118:3300/0 conn(0x7f0f500036c0 0x7f0f50005b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-24T06:10:03.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:03.082+0000 7f0f70e6a640 1 --2- 172.21.15.92:0/3333076453 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0f50006180 0x7f0f50008620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-24T06:10:03.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:03.883+0000 7f0f70e6a640 1 --2- 172.21.15.92:0/3333076453 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0f50006180 0x7f0f50008620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:03.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:03.883+0000 7f0f7166b640 1 --2- 172.21.15.92:0/3333076453 >> v2:172.21.15.118:3300/0 conn(0x7f0f500036c0 0x7f0f50005b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:03.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:03.884+0000 7f0f70e6a640 1 -- 172.21.15.92:0/3333076453 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0f50006180 msgr2=0x7f0f50008620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-11-24T06:10:03.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:03.884+0000 7f0f70e6a640 1 -- 172.21.15.92:0/3333076453 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0f50006180 msgr2=0x7f0f50008620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-24T06:10:03.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:03.884+0000 7f0f70e6a640 1 --2- 172.21.15.92:0/3333076453 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0f50006180 0x7f0f50008620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-24T06:10:03.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:03.884+0000 7f0f7166b640 1 -- 172.21.15.92:0/3333076453 >> v2:172.21.15.118:3300/0 conn(0x7f0f500036c0 msgr2=0x7f0f50005b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-11-24T06:10:03.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:03.884+0000 7f0f7166b640 1 -- 172.21.15.92:0/3333076453 >> v2:172.21.15.118:3300/0 conn(0x7f0f500036c0 msgr2=0x7f0f50005b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-24T06:10:03.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:03.884+0000 7f0f7166b640 1 --2- 172.21.15.92:0/3333076453 >> v2:172.21.15.118:3300/0 conn(0x7f0f500036c0 0x7f0f50005b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-24T06:10:03.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:03.884+0000 7f0f70e6a640 1 --2- 172.21.15.92:0/3333076453 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0f50006180 0x7f0f50008620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-24T06:10:03.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:03.884+0000 7f0f7166b640 1 --2- 172.21.15.92:0/3333076453 >> v2:172.21.15.118:3300/0 conn(0x7f0f500036c0 0x7f0f50005b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-24T06:10:05.346 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-11-24T06:10:05.343+0000 7ff6ac62e040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-11-24T06:10:05.411 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-11-24T06:10:05.407+0000 7ff6ac62e040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-24T06:10:05.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:05.477+0000 7f0f62ffd640 1 -- 172.21.15.92:0/3333076453 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0f50006180 msgr2=0x7f0f50008620 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-24T06:10:05.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:05.477+0000 7f0f62ffd640 1 --2- 172.21.15.92:0/3333076453 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0f50006180 0x7f0f50008620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:05.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:05.477+0000 7f0f62ffd640 1 -- 172.21.15.92:0/3333076453 >> v2:172.21.15.118:3300/0 conn(0x7f0f500036c0 msgr2=0x7f0f50005b40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-24T06:10:05.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:05.477+0000 7f0f62ffd640 1 --2- 172.21.15.92:0/3333076453 >> v2:172.21.15.118:3300/0 conn(0x7f0f500036c0 0x7f0f50005b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:05.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:05.478+0000 7f0f71e6c640 1 -- 172.21.15.92:0/3333076453 reap_dead start 2023-11-24T06:10:05.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:05.478+0000 7f0f62ffd640 1 --2- 172.21.15.92:0/3333076453 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0f50006180 0x7f0f50008620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:05.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:05.478+0000 7f0f62ffd640 1 --2- 172.21.15.92:0/3333076453 >> v2:172.21.15.92:3300/0 conn(0x7f0f50003ad0 0x7f0f50009180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:05.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:05.478+0000 7f0f62ffd640 1 -- 172.21.15.92:0/3333076453 --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f6c15e990 con 0x7f0f500096c0 2023-11-24T06:10:05.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:05.478+0000 7f0f62ffd640 1 -- 172.21.15.92:0/3333076453 --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f6c0b2780 con 0x7f0f50003ad0 2023-11-24T06:10:05.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:05.478+0000 7f0f62ffd640 1 -- 172.21.15.92:0/3333076453 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f50008d20 con 0x7f0f50006180 2023-11-24T06:10:05.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:05.478+0000 7f0f7166b640 1 --2- 172.21.15.92:0/3333076453 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0f50006180 0x7f0f50008620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:05.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:05.478+0000 7f0f70e6a640 1 --2- 172.21.15.92:0/3333076453 >> v2:172.21.15.92:3300/0 conn(0x7f0f50003ad0 0x7f0f50009180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:05.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:05.479+0000 7f0f7166b640 1 -- 172.21.15.92:0/3333076453 >> v2:172.21.15.92:3300/0 conn(0x7f0f50003ad0 msgr2=0x7f0f50009180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:05.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:05.479+0000 7f0f7166b640 1 --2- 172.21.15.92:0/3333076453 >> v2:172.21.15.92:3300/0 conn(0x7f0f50003ad0 0x7f0f50009180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:05.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:05.479+0000 7f0f7166b640 1 -- 172.21.15.92:0/3333076453 >> v1:172.21.15.118:6789/0 conn(0x7f0f500096c0 legacy=0x7f0f5000bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:05.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:05.479+0000 7f0f63fff640 1 -- 172.21.15.92:0/3333076453 <== mon.1 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3582513749 0 0) 0x7f0f6c15e990 con 0x7f0f500096c0 2023-11-24T06:10:05.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:05.479+0000 7f0f7166b640 1 -- 172.21.15.92:0/3333076453 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f5000e4a0 con 0x7f0f50006180 2023-11-24T06:10:05.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:05.482+0000 7f0f7166b640 1 --2- 172.21.15.92:0/3333076453 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0f50006180 0x7f0f50008620 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f0f68009100 tx=0x7f0f6802d6d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5176b6244b86ae36 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:05.486 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:05.482+0000 7f0f63fff640 1 -- 172.21.15.92:0/3333076453 <== mon.2 v2:172.21.15.118:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f68034600 con 0x7f0f50006180 2023-11-24T06:10:05.486 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:05.482+0000 7f0f63fff640 1 -- 172.21.15.92:0/3333076453 <== mon.2 v2:172.21.15.118:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f68030050 con 0x7f0f50006180 2023-11-24T06:10:05.486 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:05.482+0000 7f0f63fff640 1 -- 172.21.15.92:0/3333076453 <== mon.2 v2:172.21.15.118:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f680349a0 con 0x7f0f50006180 2023-11-24T06:10:05.487 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:05.484+0000 7f0f738cf640 1 -- 172.21.15.92:0/3333076453 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0f50006180 msgr2=0x7f0f50008620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:05.487 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:05.484+0000 7f0f738cf640 1 --2- 172.21.15.92:0/3333076453 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0f50006180 0x7f0f50008620 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f0f68009100 tx=0x7f0f6802d6d0 comp rx=0 tx=0).stop 2023-11-24T06:10:05.487 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:05.484+0000 7f0f738cf640 1 -- 172.21.15.92:0/3333076453 shutdown_connections 2023-11-24T06:10:05.487 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:05.484+0000 7f0f738cf640 1 --2- 172.21.15.92:0/3333076453 >> v2:172.21.15.92:3300/0 conn(0x7f0f50003ad0 0x7f0f50009180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:05.488 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:05.484+0000 7f0f738cf640 1 --2- 172.21.15.92:0/3333076453 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0f50006180 0x7f0f50008620 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:05.488 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:05.484+0000 7f0f738cf640 1 -- 172.21.15.92:0/3333076453 >> 172.21.15.92:0/3333076453 conn(0x7f0f6c06a800 msgr2=0x7f0f6c06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:05.488 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:05.484+0000 7f0f738cf640 1 -- 172.21.15.92:0/3333076453 shutdown_connections 2023-11-24T06:10:05.488 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:05.484+0000 7f0f738cf640 1 -- 172.21.15.92:0/3333076453 wait complete. 2023-11-24T06:10:05.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:05.486+0000 7f0f738cf640 1 Processor -- start 2023-11-24T06:10:05.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:05.487+0000 7f0f738cf640 1 -- start start 2023-11-24T06:10:05.492 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:05.488+0000 7f0f738cf640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0f50003ad0 0x7f0f6c1f6ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:05.492 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:05.488+0000 7f0f738cf640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0f50006180 0x7f0f6c070530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:05.492 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:05.488+0000 7f0f738cf640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0f500096c0 0x7f0f6c0cace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:05.492 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:05.488+0000 7f0f738cf640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f6c1f70e0 con 0x7f0f50003ad0 2023-11-24T06:10:05.492 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:05.488+0000 7f0f738cf640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f6c137670 con 0x7f0f500096c0 2023-11-24T06:10:05.492 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:05.488+0000 7f0f738cf640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f6c136af0 con 0x7f0f50006180 2023-11-24T06:10:05.492 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:05.488+0000 7f0f7166b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0f50003ad0 0x7f0f6c1f6ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:05.493 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:05.488+0000 7f0f7166b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0f50003ad0 0x7f0f6c1f6ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44442/0 (socket says 172.21.15.92:44442) 2023-11-24T06:10:05.493 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:05.488+0000 7f0f7166b640 1 -- 172.21.15.92:0/3513522888 learned_addr learned my addr 172.21.15.92:0/3513522888 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:10:05.493 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:05.489+0000 7f0f71e6c640 1 --2- 172.21.15.92:0/3513522888 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0f500096c0 0x7f0f6c0cace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:05.493 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:05.489+0000 7f0f70e6a640 1 --2- 172.21.15.92:0/3513522888 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0f50006180 0x7f0f6c070530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:05.493 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:05.489+0000 7f0f7166b640 1 -- 172.21.15.92:0/3513522888 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0f50006180 msgr2=0x7f0f6c070530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:05.493 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:05.489+0000 7f0f7166b640 1 --2- 172.21.15.92:0/3513522888 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0f50006180 0x7f0f6c070530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:05.493 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:05.489+0000 7f0f7166b640 1 -- 172.21.15.92:0/3513522888 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0f500096c0 msgr2=0x7f0f6c0cace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:05.493 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:05.489+0000 7f0f7166b640 1 --2- 172.21.15.92:0/3513522888 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0f500096c0 0x7f0f6c0cace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:05.493 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:05.489+0000 7f0f7166b640 1 -- 172.21.15.92:0/3513522888 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f6800e040 con 0x7f0f50003ad0 2023-11-24T06:10:05.493 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:05.489+0000 7f0f70e6a640 1 --2- 172.21.15.92:0/3513522888 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0f50006180 0x7f0f6c070530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:10:05.494 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:05.489+0000 7f0f7166b640 1 --2- 172.21.15.92:0/3513522888 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0f50003ad0 0x7f0f6c1f6ba0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f0f68000c00 tx=0x7f0f680046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:05.494 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:05.489+0000 7f0f61ffb640 1 -- 172.21.15.92:0/3513522888 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f6803f830 con 0x7f0f50003ad0 2023-11-24T06:10:05.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:05.489+0000 7f0f61ffb640 1 -- 172.21.15.92:0/3513522888 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f6800c040 con 0x7f0f50003ad0 2023-11-24T06:10:05.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:05.489+0000 7f0f61ffb640 1 -- 172.21.15.92:0/3513522888 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f6803fbf0 con 0x7f0f50003ad0 2023-11-24T06:10:05.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:05.489+0000 7f0f738cf640 1 -- 172.21.15.92:0/3513522888 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f6c136dd0 con 0x7f0f50003ad0 2023-11-24T06:10:05.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:05.490+0000 7f0f738cf640 1 -- 172.21.15.92:0/3513522888 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f6c137390 con 0x7f0f50003ad0 2023-11-24T06:10:05.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:05.490+0000 7f0f61ffb640 1 -- 172.21.15.92:0/3513522888 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f0f6803fd50 con 0x7f0f50003ad0 2023-11-24T06:10:05.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:05.490+0000 7f0f61ffb640 1 -- 172.21.15.92:0/3513522888 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f0f6803e3b0 con 0x7f0f50003ad0 2023-11-24T06:10:05.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:05.491+0000 7f0f738cf640 1 -- 172.21.15.92:0/3513522888 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f6c0675d0 con 0x7f0f50003ad0 2023-11-24T06:10:05.498 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:05.495+0000 7f0f61ffb640 1 -- 172.21.15.92:0/3513522888 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7f0f68067ce0 con 0x7f0f50003ad0 2023-11-24T06:10:05.584 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:05.581+0000 7f0f738cf640 1 -- 172.21.15.92:0/3513522888 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f0f6c0cea20 con 0x7f0f50003ad0 2023-11-24T06:10:05.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:05.584+0000 7f0f61ffb640 1 -- 172.21.15.92:0/3513522888 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7f0f68056020 con 0x7f0f50003ad0 2023-11-24T06:10:05.588 INFO:teuthology.orchestra.run.smithi092.stderr:adjusted tunables profile to default 2023-11-24T06:10:05.589 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:05.586+0000 7f0f738cf640 1 -- 172.21.15.92:0/3513522888 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0f50003ad0 msgr2=0x7f0f6c1f6ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:05.589 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:05.586+0000 7f0f738cf640 1 --2- 172.21.15.92:0/3513522888 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0f50003ad0 0x7f0f6c1f6ba0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f0f68000c00 tx=0x7f0f680046c0 comp rx=0 tx=0).stop 2023-11-24T06:10:05.589 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:05.586+0000 7f0f738cf640 1 -- 172.21.15.92:0/3513522888 shutdown_connections 2023-11-24T06:10:05.589 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:05.586+0000 7f0f738cf640 1 --2- 172.21.15.92:0/3513522888 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0f500096c0 0x7f0f6c0cace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:05.590 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:05.586+0000 7f0f738cf640 1 --2- 172.21.15.92:0/3513522888 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0f50006180 0x7f0f6c070530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:05.590 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:05.586+0000 7f0f738cf640 1 --2- 172.21.15.92:0/3513522888 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0f50003ad0 0x7f0f6c1f6ba0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:05.590 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:05.586+0000 7f0f738cf640 1 -- 172.21.15.92:0/3513522888 >> 172.21.15.92:0/3513522888 conn(0x7f0f6c06a800 msgr2=0x7f0f6c0afd70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:05.590 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:05.586+0000 7f0f738cf640 1 -- 172.21.15.92:0/3513522888 shutdown_connections 2023-11-24T06:10:05.590 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:05.586+0000 7f0f738cf640 1 -- 172.21.15.92:0/3513522888 wait complete. 2023-11-24T06:10:05.608 INFO:tasks.ceph:check_enable_crimson: False 2023-11-24T06:10:05.608 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2023-11-24T06:10:05.608 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-11-24T06:10:05.608 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2023-11-24T06:10:05.691 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-11-24T06:10:05.691 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2023-11-24T06:10:05.816 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-11-24T06:10:05.817 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2023-11-24T06:10:05.949 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-11-24T06:10:05.949 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2023-11-24T06:10:06.024 INFO:tasks.ceph.mgr.x.smithi118.stderr:2023-11-24T06:10:06.022+0000 7f538f62c040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-11-24T06:10:06.033 DEBUG:teuthology.orchestra.run.smithi118:> set -ex 2023-11-24T06:10:06.033 DEBUG:teuthology.orchestra.run.smithi118:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2023-11-24T06:10:06.064 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-11-24T06:10:06.061+0000 7ff6ac62e040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-24T06:10:06.086 INFO:tasks.ceph.mgr.x.smithi118.stderr:2023-11-24T06:10:06.084+0000 7f538f62c040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-24T06:10:06.121 DEBUG:teuthology.orchestra.run.smithi118:> set -ex 2023-11-24T06:10:06.121 DEBUG:teuthology.orchestra.run.smithi118:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2023-11-24T06:10:06.125 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-11-24T06:10:06.121+0000 7ff6ac62e040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-24T06:10:06.175 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-11-24T06:10:06.171+0000 7ff6ac62e040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-24T06:10:06.236 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-11-24T06:10:06.233+0000 7ff6ac62e040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-24T06:10:06.246 DEBUG:teuthology.orchestra.run.smithi118:> set -ex 2023-11-24T06:10:06.246 DEBUG:teuthology.orchestra.run.smithi118:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2023-11-24T06:10:06.285 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-11-24T06:10:06.281+0000 7ff6ac62e040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-24T06:10:06.375 DEBUG:teuthology.orchestra.run.smithi118:> set -ex 2023-11-24T06:10:06.375 DEBUG:teuthology.orchestra.run.smithi118:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2023-11-24T06:10:06.396 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-11-24T06:10:06.393+0000 7ff6ac62e040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-11-24T06:10:06.456 DEBUG:teuthology.orchestra.run.smithi118:> sudo ceph --cluster ceph osd new ff37a9fb-135f-471f-b4b0-f7b5772006c7 0 2023-11-24T06:10:06.475 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-11-24T06:10:06.472+0000 7ff6ac62e040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-24T06:10:06.527 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-11-24T06:10:06.524+0000 7ff6ac62e040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-24T06:10:06.586 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-11-24T06:10:06.583+0000 7ff6ac62e040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-24T06:10:06.640 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-11-24T06:10:06.637+0000 7ff6ac62e040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-11-24T06:10:06.691 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:06.688+0000 7fc79d3e9640 1 Processor -- start 2023-11-24T06:10:06.691 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:06.688+0000 7fc79d3e9640 1 -- start start 2023-11-24T06:10:06.693 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:06.689+0000 7fc79d3e9640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc79813cf10 0x7fc79813f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:06.693 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:06.689+0000 7fc79d3e9640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fc79813fae0 0x7fc7980cd6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:06.693 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:06.690+0000 7fc79d3e9640 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7980ce440 con 0x7fc7980cdc20 2023-11-24T06:10:06.693 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:06.690+0000 7fc79d3e9640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc79813ff60 con 0x7fc79813fae0 2023-11-24T06:10:06.693 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:06.690+0000 7fc79d3e9640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7980ce720 con 0x7fc79813cf10 2023-11-24T06:10:06.693 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:06.690+0000 7fc797fff640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc79813cf10 0x7fc79813f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:06.693 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:06.690+0000 7fc79cbe8640 1 --1- >> v1:172.21.15.118:6789/0 conn(0x7fc7980cdc20 0x7fc79815d9b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.118:6789/0 says I am v1:172.21.15.118:51146/0 (socket says 172.21.15.118:51146) 2023-11-24T06:10:06.693 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:06.690+0000 7fc79cbe8640 1 -- 172.21.15.118:0/4127356957 learned_addr learned my addr 172.21.15.118:0/4127356957 (peer_addr_for_me v1:172.21.15.118:0/0) 2023-11-24T06:10:06.693 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:06.690+0000 7fc7977fe640 1 --2- 172.21.15.118:0/4127356957 >> v2:172.21.15.92:3300/0 conn(0x7fc79813fae0 0x7fc7980cd6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:06.693 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:06.690+0000 7fc797fff640 1 -- 172.21.15.118:0/4127356957 >> v2:172.21.15.92:3300/0 conn(0x7fc79813fae0 msgr2=0x7fc7980cd6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:06.694 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:06.690+0000 7fc797fff640 1 --2- 172.21.15.118:0/4127356957 >> v2:172.21.15.92:3300/0 conn(0x7fc79813fae0 0x7fc7980cd6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:06.694 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:06.690+0000 7fc797fff640 1 -- 172.21.15.118:0/4127356957 >> v1:172.21.15.118:6789/0 conn(0x7fc7980cdc20 legacy=0x7fc79815d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:06.694 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:06.690+0000 7fc797fff640 1 -- 172.21.15.118:0/4127356957 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7980cea00 con 0x7fc79813cf10 2023-11-24T06:10:06.694 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:06.690+0000 7fc796ffd640 1 -- 172.21.15.118:0/4127356957 <== mon.1 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2962162013 0 0) 0x7fc7980ce440 con 0x7fc7980cdc20 2023-11-24T06:10:06.695 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:06.692+0000 7fc797fff640 1 --2- 172.21.15.118:0/4127356957 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc79813cf10 0x7fc79813f5a0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fc7880097a0 tx=0x7fc78802b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=39fdaf0813e66f18 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:06.695 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:06.692+0000 7fc796ffd640 1 -- 172.21.15.118:0/4127356957 <== mon.2 v2:172.21.15.118:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc788031830 con 0x7fc79813cf10 2023-11-24T06:10:06.695 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:06.692+0000 7fc796ffd640 1 -- 172.21.15.118:0/4127356957 <== mon.2 v2:172.21.15.118:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc788031990 con 0x7fc79813cf10 2023-11-24T06:10:06.695 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:06.692+0000 7fc796ffd640 1 -- 172.21.15.118:0/4127356957 <== mon.2 v2:172.21.15.118:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc788031d20 con 0x7fc79813cf10 2023-11-24T06:10:06.695 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:06.692+0000 7fc79d3e9640 1 -- 172.21.15.118:0/4127356957 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc79813cf10 msgr2=0x7fc79813f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:06.695 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:06.692+0000 7fc79d3e9640 1 --2- 172.21.15.118:0/4127356957 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc79813cf10 0x7fc79813f5a0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fc7880097a0 tx=0x7fc78802b8d0 comp rx=0 tx=0).stop 2023-11-24T06:10:06.696 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:06.692+0000 7fc79d3e9640 1 -- 172.21.15.118:0/4127356957 shutdown_connections 2023-11-24T06:10:06.696 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:06.692+0000 7fc79d3e9640 1 --2- 172.21.15.118:0/4127356957 >> v2:172.21.15.92:3300/0 conn(0x7fc79813fae0 0x7fc7980cd6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:06.696 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:06.692+0000 7fc79d3e9640 1 --2- 172.21.15.118:0/4127356957 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc79813cf10 0x7fc79813f5a0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:06.696 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:06.692+0000 7fc79d3e9640 1 -- 172.21.15.118:0/4127356957 >> 172.21.15.118:0/4127356957 conn(0x7fc79806a7d0 msgr2=0x7fc79806abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:06.696 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:06.693+0000 7fc79d3e9640 1 -- 172.21.15.118:0/4127356957 shutdown_connections 2023-11-24T06:10:06.696 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:06.693+0000 7fc79d3e9640 1 -- 172.21.15.118:0/4127356957 wait complete. 2023-11-24T06:10:06.696 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:06.694+0000 7fc79d3e9640 1 Processor -- start 2023-11-24T06:10:06.697 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:06.694+0000 7fc79d3e9640 1 -- start start 2023-11-24T06:10:06.698 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:06.695+0000 7fc79d3e9640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc7980cdc20 0x7fc79813b140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:06.699 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:06.695+0000 7fc79d3e9640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc79813cf10 0x7fc798156b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:06.699 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:06.695+0000 7fc79d3e9640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc79813fae0 0x7fc79813e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:06.699 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:06.695+0000 7fc79d3e9640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc798156430 con 0x7fc79813fae0 2023-11-24T06:10:06.699 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:06.695+0000 7fc79d3e9640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc798157390 con 0x7fc7980cdc20 2023-11-24T06:10:06.699 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:06.695+0000 7fc79d3e9640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7981f4520 con 0x7fc79813cf10 2023-11-24T06:10:06.699 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:06.696+0000 7fc797fff640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc7980cdc20 0x7fc79813b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:06.699 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:06.696+0000 7fc7977fe640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc79813cf10 0x7fc798156b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:06.699 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:06.696+0000 7fc797fff640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc7980cdc20 0x7fc79813b140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:52112/0 (socket says 172.21.15.118:52112) 2023-11-24T06:10:06.699 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:06.696+0000 7fc797fff640 1 -- 172.21.15.118:0/2712388515 learned_addr learned my addr 172.21.15.118:0/2712388515 (peer_addr_for_me v2:172.21.15.118:0/0) 2023-11-24T06:10:06.699 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:06.696+0000 7fc79cbe8640 1 --2- 172.21.15.118:0/2712388515 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc79813fae0 0x7fc79813e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:06.699 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:06.696+0000 7fc7977fe640 1 -- 172.21.15.118:0/2712388515 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc7980cdc20 msgr2=0x7fc79813b140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:06.699 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:06.696+0000 7fc7977fe640 1 --2- 172.21.15.118:0/2712388515 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc7980cdc20 0x7fc79813b140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:06.700 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:06.696+0000 7fc7977fe640 1 -- 172.21.15.118:0/2712388515 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc79813fae0 msgr2=0x7fc79813e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:06.700 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:06.696+0000 7fc7977fe640 1 --2- 172.21.15.118:0/2712388515 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc79813fae0 0x7fc79813e8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:06.700 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:06.696+0000 7fc7977fe640 1 -- 172.21.15.118:0/2712388515 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc788009400 con 0x7fc79813cf10 2023-11-24T06:10:06.700 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:06.696+0000 7fc797fff640 1 --2- 172.21.15.118:0/2712388515 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc7980cdc20 0x7fc79813b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:10:06.700 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:06.696+0000 7fc7977fe640 1 --2- 172.21.15.118:0/2712388515 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc79813cf10 0x7fc798156b40 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fc78c00d550 tx=0x7fc78c00da10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:06.701 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:06.697+0000 7fc794ff9640 1 -- 172.21.15.118:0/2712388515 <== mon.2 v2:172.21.15.118:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc78c0125f0 con 0x7fc79813cf10 2023-11-24T06:10:06.701 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:06.697+0000 7fc794ff9640 1 -- 172.21.15.118:0/2712388515 <== mon.2 v2:172.21.15.118:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc78c012750 con 0x7fc79813cf10 2023-11-24T06:10:06.701 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:06.697+0000 7fc79d3e9640 1 -- 172.21.15.118:0/2712388515 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7981f4800 con 0x7fc79813cf10 2023-11-24T06:10:06.701 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:06.697+0000 7fc794ff9640 1 -- 172.21.15.118:0/2712388515 <== mon.2 v2:172.21.15.118:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc78c012ae0 con 0x7fc79813cf10 2023-11-24T06:10:06.701 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:06.697+0000 7fc79d3e9640 1 -- 172.21.15.118:0/2712388515 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc79813b680 con 0x7fc79813cf10 2023-11-24T06:10:06.701 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:06.697+0000 7fc794ff9640 1 -- 172.21.15.118:0/2712388515 <== mon.2 v2:172.21.15.118:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fc78c01e460 con 0x7fc79813cf10 2023-11-24T06:10:06.701 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:06.697+0000 7fc794ff9640 1 -- 172.21.15.118:0/2712388515 <== mon.2 v2:172.21.15.118:3301/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 725+0+0 (secure 0 0 0) 0x7fc78c014550 con 0x7fc79813cf10 2023-11-24T06:10:06.701 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:06.698+0000 7fc79d3e9640 1 -- 172.21.15.118:0/2712388515 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7980b3560 con 0x7fc79813cf10 2023-11-24T06:10:06.703 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:06.701+0000 7fc794ff9640 1 -- 172.21.15.118:0/2712388515 <== mon.2 v2:172.21.15.118:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7fc78c01e8f0 con 0x7fc79813cf10 2023-11-24T06:10:06.721 INFO:tasks.ceph.mgr.x.smithi118.stderr:2023-11-24T06:10:06.719+0000 7f538f62c040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-24T06:10:06.782 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:06.779+0000 7fc79d3e9640 1 -- 172.21.15.118:0/2712388515 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_command({"prefix": "osd new", "uuid": "ff37a9fb-135f-471f-b4b0-f7b5772006c7", "id": 0} v 0) v1 -- 0x7fc798067450 con 0x7fc79813cf10 2023-11-24T06:10:06.785 INFO:tasks.ceph.mgr.x.smithi118.stderr:2023-11-24T06:10:06.783+0000 7f538f62c040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-24T06:10:06.787 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:06.785+0000 7fc794ff9640 1 -- 172.21.15.118:0/2712388515 <== mon.2 v2:172.21.15.118:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "ff37a9fb-135f-471f-b4b0-f7b5772006c7", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7fc78c01ea90 con 0x7fc79813cf10 2023-11-24T06:10:06.788 INFO:teuthology.orchestra.run.smithi118.stdout:0 2023-11-24T06:10:06.789 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:06.787+0000 7fc79d3e9640 1 -- 172.21.15.118:0/2712388515 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc79813cf10 msgr2=0x7fc798156b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:06.790 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:06.787+0000 7fc79d3e9640 1 --2- 172.21.15.118:0/2712388515 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc79813cf10 0x7fc798156b40 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fc78c00d550 tx=0x7fc78c00da10 comp rx=0 tx=0).stop 2023-11-24T06:10:06.790 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:06.787+0000 7fc79d3e9640 1 -- 172.21.15.118:0/2712388515 shutdown_connections 2023-11-24T06:10:06.790 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:06.787+0000 7fc79d3e9640 1 --2- 172.21.15.118:0/2712388515 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc79813fae0 0x7fc79813e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:06.790 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:06.787+0000 7fc79d3e9640 1 --2- 172.21.15.118:0/2712388515 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc79813cf10 0x7fc798156b40 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:06.790 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:06.787+0000 7fc79d3e9640 1 --2- 172.21.15.118:0/2712388515 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc7980cdc20 0x7fc79813b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:06.790 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:06.787+0000 7fc79d3e9640 1 -- 172.21.15.118:0/2712388515 >> 172.21.15.118:0/2712388515 conn(0x7fc79806a7d0 msgr2=0x7fc798158b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:06.790 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:06.787+0000 7fc79d3e9640 1 -- 172.21.15.118:0/2712388515 shutdown_connections 2023-11-24T06:10:06.790 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:06.787+0000 7fc79d3e9640 1 -- 172.21.15.118:0/2712388515 wait complete. 2023-11-24T06:10:06.806 DEBUG:teuthology.orchestra.run.smithi118:> sudo ceph --cluster ceph osd new 03cdd353-dcf1-4f38-94fe-a795137c06b1 1 2023-11-24T06:10:06.841 INFO:tasks.ceph.mgr.x.smithi118.stderr:2023-11-24T06:10:06.838+0000 7f538f62c040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-24T06:10:06.915 INFO:tasks.ceph.mgr.x.smithi118.stderr:2023-11-24T06:10:06.912+0000 7f538f62c040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-24T06:10:06.967 INFO:tasks.ceph.mgr.x.smithi118.stderr:2023-11-24T06:10:06.964+0000 7f538f62c040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-24T06:10:06.988 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:06.986+0000 7fd3e370f640 1 Processor -- start 2023-11-24T06:10:06.989 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:06.986+0000 7fd3e370f640 1 -- start start 2023-11-24T06:10:06.990 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:06.987+0000 7fd3e370f640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fd3dc0cd240 0x7fd3dc0b3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:06.990 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:06.987+0000 7fd3e370f640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fd3dc06c6b0 0x7fd3dc099a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:06.990 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:06.987+0000 7fd3e370f640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd3dc099fa0 0x7fd3dc06b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:06.990 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:06.987+0000 7fd3e370f640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3dc09a430 con 0x7fd3dc06c6b0 2023-11-24T06:10:06.990 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:06.987+0000 7fd3e370f640 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3dc06cea0 con 0x7fd3dc0cd240 2023-11-24T06:10:06.991 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:06.987+0000 7fd3e370f640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3dc06cbc0 con 0x7fd3dc099fa0 2023-11-24T06:10:06.991 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:06.988+0000 7fd3e14ab640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fd3dc0cd240 0x7fd3dc0b3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:06.991 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:06.988+0000 7fd3e1cac640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd3dc099fa0 0x7fd3dc06b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:06.991 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:06.988+0000 7fd3e14ab640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fd3dc0cd240 0x7fd3dc0b3820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:52126/0 (socket says 172.21.15.118:52126) 2023-11-24T06:10:06.991 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:06.988+0000 7fd3e14ab640 1 -- 172.21.15.118:0/3516611639 learned_addr learned my addr 172.21.15.118:0/3516611639 (peer_addr_for_me v2:172.21.15.118:0/0) 2023-11-24T06:10:06.991 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:06.988+0000 7fd3e14ab640 1 -- 172.21.15.118:0/3516611639 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd3dc099fa0 msgr2=0x7fd3dc06b4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:06.991 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:06.988+0000 7fd3e14ab640 1 --2- 172.21.15.118:0/3516611639 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd3dc099fa0 0x7fd3dc06b4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:06.992 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:06.988+0000 7fd3e14ab640 1 -- 172.21.15.118:0/3516611639 >> v2:172.21.15.92:3300/0 conn(0x7fd3dc06c6b0 msgr2=0x7fd3dc099a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:06.992 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:06.988+0000 7fd3e14ab640 1 --2- 172.21.15.118:0/3516611639 >> v2:172.21.15.92:3300/0 conn(0x7fd3dc06c6b0 0x7fd3dc099a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:06.992 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:06.988+0000 7fd3e14ab640 1 -- 172.21.15.118:0/3516611639 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3dc06f350 con 0x7fd3dc0cd240 2023-11-24T06:10:06.992 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:06.989+0000 7fd3e14ab640 1 --2- 172.21.15.118:0/3516611639 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd3dc0cd240 0x7fd3dc0b3820 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fd3dc070a10 tx=0x7fd3d802b910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=60b83d3a3c99d7d server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:06.992 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:06.990+0000 7fd3d3fff640 1 -- 172.21.15.118:0/3516611639 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3d8031870 con 0x7fd3dc0cd240 2023-11-24T06:10:06.993 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:06.990+0000 7fd3d3fff640 1 -- 172.21.15.118:0/3516611639 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3d80319d0 con 0x7fd3dc0cd240 2023-11-24T06:10:06.993 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:06.990+0000 7fd3d3fff640 1 -- 172.21.15.118:0/3516611639 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3d8031d60 con 0x7fd3dc0cd240 2023-11-24T06:10:06.993 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:06.990+0000 7fd3e370f640 1 -- 172.21.15.118:0/3516611639 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd3dc0cd240 msgr2=0x7fd3dc0b3820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:06.993 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:06.990+0000 7fd3e370f640 1 --2- 172.21.15.118:0/3516611639 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd3dc0cd240 0x7fd3dc0b3820 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fd3dc070a10 tx=0x7fd3d802b910 comp rx=0 tx=0).stop 2023-11-24T06:10:06.994 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:06.990+0000 7fd3e370f640 1 -- 172.21.15.118:0/3516611639 shutdown_connections 2023-11-24T06:10:06.994 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:06.990+0000 7fd3e370f640 1 --2- 172.21.15.118:0/3516611639 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd3dc099fa0 0x7fd3dc06b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:06.994 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:06.990+0000 7fd3e370f640 1 --2- 172.21.15.118:0/3516611639 >> v2:172.21.15.92:3300/0 conn(0x7fd3dc06c6b0 0x7fd3dc099a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:06.994 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:06.990+0000 7fd3e370f640 1 --2- 172.21.15.118:0/3516611639 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd3dc0cd240 0x7fd3dc0b3820 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:06.994 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:06.990+0000 7fd3e370f640 1 -- 172.21.15.118:0/3516611639 >> 172.21.15.118:0/3516611639 conn(0x7fd3dc0b2d00 msgr2=0x7fd3dc0b3110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:06.994 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:06.991+0000 7fd3e370f640 1 -- 172.21.15.118:0/3516611639 shutdown_connections 2023-11-24T06:10:06.994 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:06.991+0000 7fd3e370f640 1 -- 172.21.15.118:0/3516611639 wait complete. 2023-11-24T06:10:06.995 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:06.992+0000 7fd3e370f640 1 Processor -- start 2023-11-24T06:10:06.995 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:06.993+0000 7fd3e370f640 1 -- start start 2023-11-24T06:10:06.997 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:06.993+0000 7fd3e370f640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd3dc06c6b0 0x7fd3dc1569c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:06.997 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:06.993+0000 7fd3e370f640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd3dc099fa0 0x7fd3dc0cadc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:06.997 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:06.994+0000 7fd3e370f640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd3dc0cd240 0x7fd3dc15f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:06.998 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:06.994+0000 7fd3e370f640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3dc156820 con 0x7fd3dc0cd240 2023-11-24T06:10:06.998 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:06.994+0000 7fd3e370f640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3dc0c61e0 con 0x7fd3dc06c6b0 2023-11-24T06:10:06.998 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:06.994+0000 7fd3e370f640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3dc0c6480 con 0x7fd3dc099fa0 2023-11-24T06:10:06.998 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:06.994+0000 7fd3e14ab640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd3dc06c6b0 0x7fd3dc1569c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:06.998 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:06.994+0000 7fd3e0caa640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd3dc099fa0 0x7fd3dc0cadc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:06.998 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:06.994+0000 7fd3e14ab640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd3dc06c6b0 0x7fd3dc1569c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:52130/0 (socket says 172.21.15.118:52130) 2023-11-24T06:10:06.998 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:06.994+0000 7fd3e14ab640 1 -- 172.21.15.118:0/3170880516 learned_addr learned my addr 172.21.15.118:0/3170880516 (peer_addr_for_me v2:172.21.15.118:0/0) 2023-11-24T06:10:06.998 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:06.994+0000 7fd3e1cac640 1 --2- 172.21.15.118:0/3170880516 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd3dc0cd240 0x7fd3dc15f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:06.998 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:06.994+0000 7fd3e0caa640 1 -- 172.21.15.118:0/3170880516 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd3dc06c6b0 msgr2=0x7fd3dc1569c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:06.998 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:06.994+0000 7fd3e0caa640 1 --2- 172.21.15.118:0/3170880516 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd3dc06c6b0 0x7fd3dc1569c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:06.999 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:06.994+0000 7fd3e0caa640 1 -- 172.21.15.118:0/3170880516 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd3dc0cd240 msgr2=0x7fd3dc15f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:06.999 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:06.994+0000 7fd3e0caa640 1 --2- 172.21.15.118:0/3170880516 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd3dc0cd240 0x7fd3dc15f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:06.999 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:06.994+0000 7fd3e0caa640 1 -- 172.21.15.118:0/3170880516 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3d80094d0 con 0x7fd3dc099fa0 2023-11-24T06:10:06.999 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:06.995+0000 7fd3e0caa640 1 --2- 172.21.15.118:0/3170880516 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd3dc099fa0 0x7fd3dc0cadc0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fd3cc00b3e0 tx=0x7fd3cc00b8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:06.999 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:06.995+0000 7fd3d1ffb640 1 -- 172.21.15.118:0/3170880516 <== mon.2 v2:172.21.15.118:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3cc011830 con 0x7fd3dc099fa0 2023-11-24T06:10:06.999 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:06.995+0000 7fd3d1ffb640 1 -- 172.21.15.118:0/3170880516 <== mon.2 v2:172.21.15.118:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3cc011990 con 0x7fd3dc099fa0 2023-11-24T06:10:06.999 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:06.995+0000 7fd3e370f640 1 -- 172.21.15.118:0/3170880516 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3dc0c6760 con 0x7fd3dc099fa0 2023-11-24T06:10:06.999 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:06.995+0000 7fd3d1ffb640 1 -- 172.21.15.118:0/3170880516 <== mon.2 v2:172.21.15.118:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3cc011d40 con 0x7fd3dc099fa0 2023-11-24T06:10:06.999 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:06.995+0000 7fd3e370f640 1 -- 172.21.15.118:0/3170880516 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3dc0c6d20 con 0x7fd3dc099fa0 2023-11-24T06:10:06.999 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:06.995+0000 7fd3d1ffb640 1 -- 172.21.15.118:0/3170880516 <== mon.2 v2:172.21.15.118:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fd3cc01a700 con 0x7fd3dc099fa0 2023-11-24T06:10:06.999 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:06.995+0000 7fd3d1ffb640 1 -- 172.21.15.118:0/3170880516 <== mon.2 v2:172.21.15.118:3301/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 840+0+0 (secure 0 0 0) 0x7fd3cc019710 con 0x7fd3dc099fa0 2023-11-24T06:10:07.000 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:06.996+0000 7fd3e370f640 1 -- 172.21.15.118:0/3170880516 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3dc0675d0 con 0x7fd3dc099fa0 2023-11-24T06:10:07.001 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:06.998+0000 7fd3d1ffb640 1 -- 172.21.15.118:0/3170880516 <== mon.2 v2:172.21.15.118:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7fd3cc039ce0 con 0x7fd3dc099fa0 2023-11-24T06:10:07.079 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:07.076+0000 7fd3e370f640 1 -- 172.21.15.118:0/3170880516 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_command({"prefix": "osd new", "uuid": "03cdd353-dcf1-4f38-94fe-a795137c06b1", "id": 1} v 0) v1 -- 0x7fd3dc067e00 con 0x7fd3dc099fa0 2023-11-24T06:10:07.086 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:07.083+0000 7fd3d1ffb640 1 -- 172.21.15.118:0/3170880516 <== mon.2 v2:172.21.15.118:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "03cdd353-dcf1-4f38-94fe-a795137c06b1", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7fd3cc028030 con 0x7fd3dc099fa0 2023-11-24T06:10:07.086 INFO:teuthology.orchestra.run.smithi118.stdout:1 2023-11-24T06:10:07.087 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:07.084+0000 7fd3e370f640 1 -- 172.21.15.118:0/3170880516 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd3dc099fa0 msgr2=0x7fd3dc0cadc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:07.087 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:07.084+0000 7fd3e370f640 1 --2- 172.21.15.118:0/3170880516 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd3dc099fa0 0x7fd3dc0cadc0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fd3cc00b3e0 tx=0x7fd3cc00b8a0 comp rx=0 tx=0).stop 2023-11-24T06:10:07.087 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:07.085+0000 7fd3e370f640 1 -- 172.21.15.118:0/3170880516 shutdown_connections 2023-11-24T06:10:07.087 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:07.085+0000 7fd3e370f640 1 --2- 172.21.15.118:0/3170880516 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd3dc0cd240 0x7fd3dc15f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:07.088 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:07.085+0000 7fd3e370f640 1 --2- 172.21.15.118:0/3170880516 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd3dc099fa0 0x7fd3dc0cadc0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:07.088 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:07.085+0000 7fd3e370f640 1 --2- 172.21.15.118:0/3170880516 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd3dc06c6b0 0x7fd3dc1569c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:07.088 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:07.085+0000 7fd3e370f640 1 -- 172.21.15.118:0/3170880516 >> 172.21.15.118:0/3170880516 conn(0x7fd3dc0b2d00 msgr2=0x7fd3dc0b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:07.088 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:07.085+0000 7fd3e370f640 1 -- 172.21.15.118:0/3170880516 shutdown_connections 2023-11-24T06:10:07.088 INFO:tasks.ceph.mgr.x.smithi118.stderr:2023-11-24T06:10:07.085+0000 7f538f62c040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-11-24T06:10:07.088 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:07.085+0000 7fd3e370f640 1 -- 172.21.15.118:0/3170880516 wait complete. 2023-11-24T06:10:07.090 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-11-24T06:10:07.087+0000 7ff6ac62e040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-11-24T06:10:07.103 DEBUG:teuthology.orchestra.run.smithi118:> sudo ceph --cluster ceph osd new a4a20d80-5d57-484b-becd-9aa8070dd3aa 2 2023-11-24T06:10:07.172 INFO:tasks.ceph.mgr.x.smithi118.stderr:2023-11-24T06:10:07.169+0000 7f538f62c040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-24T06:10:07.228 INFO:tasks.ceph.mgr.x.smithi118.stderr:2023-11-24T06:10:07.225+0000 7f538f62c040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-24T06:10:07.247 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-11-24T06:10:07.244+0000 7ff6ac62e040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-24T06:10:07.291 INFO:tasks.ceph.mgr.x.smithi118.stderr:2023-11-24T06:10:07.289+0000 7f538f62c040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-24T06:10:07.340 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:07.337+0000 7f8445f83640 1 Processor -- start 2023-11-24T06:10:07.340 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:07.337+0000 7f8445f83640 1 -- start start 2023-11-24T06:10:07.342 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:07.338+0000 7f8445f83640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f84400cf9a0 0x7f84400cfde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:07.342 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:07.339+0000 7f8445f83640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f84400b31d0 0x7f84400b3660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:07.342 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:07.339+0000 7f8445f83640 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84400d0320 con 0x7f844015d4b0 2023-11-24T06:10:07.342 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:07.339+0000 7f8445f83640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84400d1c60 con 0x7f84400cf9a0 2023-11-24T06:10:07.342 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:07.339+0000 7f8445f83640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84400b2730 con 0x7f84400b31d0 2023-11-24T06:10:07.342 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:07.339+0000 7f8444f81640 1 --1- >> v1:172.21.15.118:6789/0 conn(0x7f844015d4b0 0x7f84400d1550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.118:6789/0 says I am v1:172.21.15.118:51154/0 (socket says 172.21.15.118:51154) 2023-11-24T06:10:07.342 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:07.339+0000 7f8444f81640 1 -- 172.21.15.118:0/2246454412 learned_addr learned my addr 172.21.15.118:0/2246454412 (peer_addr_for_me v1:172.21.15.118:0/0) 2023-11-24T06:10:07.342 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:07.339+0000 7f8445782640 1 --2- 172.21.15.118:0/2246454412 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f84400b31d0 0x7f84400b3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:07.342 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:07.339+0000 7f843ffff640 1 --2- 172.21.15.118:0/2246454412 >> v2:172.21.15.92:3300/0 conn(0x7f84400cf9a0 0x7f84400cfde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:07.343 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:07.339+0000 7f8445782640 1 -- 172.21.15.118:0/2246454412 >> v2:172.21.15.92:3300/0 conn(0x7f84400cf9a0 msgr2=0x7f84400cfde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:07.343 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:07.339+0000 7f8445782640 1 --2- 172.21.15.118:0/2246454412 >> v2:172.21.15.92:3300/0 conn(0x7f84400cf9a0 0x7f84400cfde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:07.343 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:07.339+0000 7f8445782640 1 -- 172.21.15.118:0/2246454412 >> v1:172.21.15.118:6789/0 conn(0x7f844015d4b0 legacy=0x7f84400d1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:07.343 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:07.339+0000 7f843f7fe640 1 -- 172.21.15.118:0/2246454412 <== mon.1 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2041937438 0 0) 0x7f84400d0320 con 0x7f844015d4b0 2023-11-24T06:10:07.343 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:07.339+0000 7f8445782640 1 -- 172.21.15.118:0/2246454412 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84400d1ec0 con 0x7f84400b31d0 2023-11-24T06:10:07.344 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:07.340+0000 7f8445782640 1 --2- 172.21.15.118:0/2246454412 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f84400b31d0 0x7f84400b3660 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f8438009d90 tx=0x7f843802bea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f667ff9c71e0c7a0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:07.344 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:07.341+0000 7f843f7fe640 1 -- 172.21.15.118:0/2246454412 <== mon.2 v2:172.21.15.118:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f843802cc50 con 0x7f84400b31d0 2023-11-24T06:10:07.344 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:07.341+0000 7f843f7fe640 1 -- 172.21.15.118:0/2246454412 <== mon.2 v2:172.21.15.118:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f843802cdb0 con 0x7f84400b31d0 2023-11-24T06:10:07.344 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:07.341+0000 7f843f7fe640 1 -- 172.21.15.118:0/2246454412 <== mon.2 v2:172.21.15.118:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8438034660 con 0x7f84400b31d0 2023-11-24T06:10:07.344 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:07.341+0000 7f8445f83640 1 -- 172.21.15.118:0/2246454412 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f84400b31d0 msgr2=0x7f84400b3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:07.344 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:07.341+0000 7f8445f83640 1 --2- 172.21.15.118:0/2246454412 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f84400b31d0 0x7f84400b3660 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f8438009d90 tx=0x7f843802bea0 comp rx=0 tx=0).stop 2023-11-24T06:10:07.345 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:07.341+0000 7f8445f83640 1 -- 172.21.15.118:0/2246454412 shutdown_connections 2023-11-24T06:10:07.345 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:07.341+0000 7f8445f83640 1 --2- 172.21.15.118:0/2246454412 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f84400b31d0 0x7f84400b3660 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:07.345 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:07.341+0000 7f8445f83640 1 --2- 172.21.15.118:0/2246454412 >> v2:172.21.15.92:3300/0 conn(0x7f84400cf9a0 0x7f84400cfde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:07.345 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:07.341+0000 7f8445f83640 1 -- 172.21.15.118:0/2246454412 >> 172.21.15.118:0/2246454412 conn(0x7f84400d3000 msgr2=0x7f84400d3410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:07.345 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:07.341+0000 7f8445f83640 1 -- 172.21.15.118:0/2246454412 shutdown_connections 2023-11-24T06:10:07.345 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:07.342+0000 7f8445f83640 1 -- 172.21.15.118:0/2246454412 wait complete. 2023-11-24T06:10:07.345 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:07.343+0000 7f8445f83640 1 Processor -- start 2023-11-24T06:10:07.345 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:07.343+0000 7f8445f83640 1 -- start start 2023-11-24T06:10:07.348 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:07.344+0000 7f8445f83640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f84400b31d0 0x7f84401545b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:07.348 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:07.344+0000 7f8445f83640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f84400cf9a0 0x7f844009fe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:07.348 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:07.344+0000 7f8445f83640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f844015d4b0 0x7f84400cddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:07.348 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:07.344+0000 7f8445f83640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84400a0da0 con 0x7f844015d4b0 2023-11-24T06:10:07.349 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:07.344+0000 7f8445f83640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84400cb6a0 con 0x7f84400cf9a0 2023-11-24T06:10:07.349 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:07.344+0000 7f8445f83640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8440154af0 con 0x7f84400b31d0 2023-11-24T06:10:07.349 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:07.344+0000 7f843ffff640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f84400cf9a0 0x7f844009fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:07.349 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:07.344+0000 7f8444f81640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f84400b31d0 0x7f84401545b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:07.349 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:07.345+0000 7f8444f81640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f84400b31d0 0x7f84401545b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3301/0 says I am v2:172.21.15.118:60964/0 (socket says 172.21.15.118:60964) 2023-11-24T06:10:07.349 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:07.345+0000 7f8444f81640 1 -- 172.21.15.118:0/948401985 learned_addr learned my addr 172.21.15.118:0/948401985 (peer_addr_for_me v2:172.21.15.118:0/0) 2023-11-24T06:10:07.349 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:07.345+0000 7f843ffff640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f84400cf9a0 0x7f844009fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:52136/0 (socket says 172.21.15.118:52136) 2023-11-24T06:10:07.349 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:07.345+0000 7f8445782640 1 --2- 172.21.15.118:0/948401985 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f844015d4b0 0x7f84400cddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:07.349 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:07.345+0000 7f8444f81640 1 -- 172.21.15.118:0/948401985 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f84400cf9a0 msgr2=0x7f844009fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:07.349 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:07.345+0000 7f8444f81640 1 --2- 172.21.15.118:0/948401985 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f84400cf9a0 0x7f844009fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:07.349 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:07.345+0000 7f8444f81640 1 -- 172.21.15.118:0/948401985 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f844015d4b0 msgr2=0x7f84400cddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:07.349 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:07.345+0000 7f8444f81640 1 --2- 172.21.15.118:0/948401985 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f844015d4b0 0x7f84400cddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:07.350 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:07.345+0000 7f8444f81640 1 -- 172.21.15.118:0/948401985 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8438009a70 con 0x7f84400b31d0 2023-11-24T06:10:07.350 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:07.345+0000 7f8445782640 1 --2- 172.21.15.118:0/948401985 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f844015d4b0 0x7f84400cddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:10:07.350 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:07.345+0000 7f8444f81640 1 --2- 172.21.15.118:0/948401985 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f84400b31d0 0x7f84401545b0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f843400c500 tx=0x7f843400c9c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:07.350 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:07.345+0000 7f843d7fa640 1 -- 172.21.15.118:0/948401985 <== mon.2 v2:172.21.15.118:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8434007560 con 0x7f84400b31d0 2023-11-24T06:10:07.350 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:07.346+0000 7f843d7fa640 1 -- 172.21.15.118:0/948401985 <== mon.2 v2:172.21.15.118:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84340047e0 con 0x7f84400b31d0 2023-11-24T06:10:07.350 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:07.346+0000 7f843d7fa640 1 -- 172.21.15.118:0/948401985 <== mon.2 v2:172.21.15.118:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8434004b90 con 0x7f84400b31d0 2023-11-24T06:10:07.350 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:07.346+0000 7f8445f83640 1 -- 172.21.15.118:0/948401985 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84400a07e0 con 0x7f84400b31d0 2023-11-24T06:10:07.350 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:07.346+0000 7f8445f83640 1 -- 172.21.15.118:0/948401985 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84400cb330 con 0x7f84400b31d0 2023-11-24T06:10:07.350 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:07.346+0000 7f843d7fa640 1 -- 172.21.15.118:0/948401985 <== mon.2 v2:172.21.15.118:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f8434012400 con 0x7f84400b31d0 2023-11-24T06:10:07.350 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:07.346+0000 7f843d7fa640 1 -- 172.21.15.118:0/948401985 <== mon.2 v2:172.21.15.118:3301/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 955+0+0 (secure 0 0 0) 0x7f8434012ad0 con 0x7f84400b31d0 2023-11-24T06:10:07.351 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:07.347+0000 7f8445f83640 1 -- 172.21.15.118:0/948401985 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8440067450 con 0x7f84400b31d0 2023-11-24T06:10:07.351 INFO:tasks.ceph.mgr.x.smithi118.stderr:2023-11-24T06:10:07.347+0000 7f538f62c040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-11-24T06:10:07.351 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:07.349+0000 7f843d7fa640 1 -- 172.21.15.118:0/948401985 <== mon.2 v2:172.21.15.118:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7f8434018070 con 0x7f84400b31d0 2023-11-24T06:10:07.372 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-11-24T06:10:07.368+0000 7ff6ac62e040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-24T06:10:07.425 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-11-24T06:10:07.421+0000 7ff6ac62e040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-24T06:10:07.429 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:07.427+0000 7f8445f83640 1 -- 172.21.15.118:0/948401985 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_command({"prefix": "osd new", "uuid": "a4a20d80-5d57-484b-becd-9aa8070dd3aa", "id": 2} v 0) v1 -- 0x7f84400b1560 con 0x7f84400b31d0 2023-11-24T06:10:07.437 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:07.434+0000 7f843d7fa640 1 -- 172.21.15.118:0/948401985 <== mon.2 v2:172.21.15.118:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "a4a20d80-5d57-484b-becd-9aa8070dd3aa", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7f84340126b0 con 0x7f84400b31d0 2023-11-24T06:10:07.437 INFO:teuthology.orchestra.run.smithi118.stdout:2 2023-11-24T06:10:07.438 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:07.435+0000 7f8445f83640 1 -- 172.21.15.118:0/948401985 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f84400b31d0 msgr2=0x7f84401545b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:07.438 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:07.435+0000 7f8445f83640 1 --2- 172.21.15.118:0/948401985 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f84400b31d0 0x7f84401545b0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f843400c500 tx=0x7f843400c9c0 comp rx=0 tx=0).stop 2023-11-24T06:10:07.438 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:07.436+0000 7f8445f83640 1 -- 172.21.15.118:0/948401985 shutdown_connections 2023-11-24T06:10:07.439 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:07.436+0000 7f8445f83640 1 --2- 172.21.15.118:0/948401985 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f844015d4b0 0x7f84400cddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:07.439 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:07.436+0000 7f8445f83640 1 --2- 172.21.15.118:0/948401985 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f84400cf9a0 0x7f844009fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:07.439 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:07.436+0000 7f8445f83640 1 --2- 172.21.15.118:0/948401985 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f84400b31d0 0x7f84401545b0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:07.439 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:07.436+0000 7f8445f83640 1 -- 172.21.15.118:0/948401985 >> 172.21.15.118:0/948401985 conn(0x7f84400d3000 msgr2=0x7f844006ae20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:07.439 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:07.436+0000 7f8445f83640 1 -- 172.21.15.118:0/948401985 shutdown_connections 2023-11-24T06:10:07.439 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:07.436+0000 7f8445f83640 1 -- 172.21.15.118:0/948401985 wait complete. 2023-11-24T06:10:07.455 DEBUG:teuthology.orchestra.run.smithi118:> sudo ceph --cluster ceph osd new 4648d7ce-9b29-4583-9425-4ba8fd2b1903 3 2023-11-24T06:10:07.537 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-11-24T06:10:07.533+0000 7ff6ac62e040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-24T06:10:07.647 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-11-24T06:10:07.643+0000 7ff6ac62e040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-24T06:10:07.695 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:07.693+0000 7fe9a676f640 1 Processor -- start 2023-11-24T06:10:07.696 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:07.693+0000 7fe9a676f640 1 -- start start 2023-11-24T06:10:07.697 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:07.695+0000 7fe9a676f640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe9a0072230 0x7fe9a00b32e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:07.698 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:07.695+0000 7fe9a676f640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9a0071190 con 0x7fe9a015da10 2023-11-24T06:10:07.698 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:07.695+0000 7fe9a676f640 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9a0071470 con 0x7fe9a0150760 2023-11-24T06:10:07.698 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:07.695+0000 7fe9a676f640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9a0071750 con 0x7fe9a0072230 2023-11-24T06:10:07.698 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:07.695+0000 7fe99ffff640 1 --1- >> v1:172.21.15.118:6789/0 conn(0x7fe9a0150760 0x7fe9a015d300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.118:6789/0 says I am v1:172.21.15.118:51162/0 (socket says 172.21.15.118:51162) 2023-11-24T06:10:07.698 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:07.695+0000 7fe99ffff640 1 -- 172.21.15.118:0/1710982159 learned_addr learned my addr 172.21.15.118:0/1710982159 (peer_addr_for_me v1:172.21.15.118:0/0) 2023-11-24T06:10:07.698 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:07.695+0000 7fe9a4d0c640 1 --2- 172.21.15.118:0/1710982159 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe9a0072230 0x7fe9a00b32e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:07.699 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:07.696+0000 7fe99effd640 1 -- 172.21.15.118:0/1710982159 <== mon.1 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 218327243 0 0) 0x7fe9a0071470 con 0x7fe9a0150760 2023-11-24T06:10:07.699 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:07.696+0000 7fe99effd640 1 -- 172.21.15.118:0/1710982159 --> v1:172.21.15.118:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe988003520 con 0x7fe9a0150760 2023-11-24T06:10:07.699 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:07.696+0000 7fe9a4d0c640 1 -- 172.21.15.118:0/1710982159 >> v1:172.21.15.118:6789/0 conn(0x7fe9a0150760 legacy=0x7fe9a015d300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:07.699 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:07.696+0000 7fe9a4d0c640 1 -- 172.21.15.118:0/1710982159 >> v1:172.21.15.92:6789/0 conn(0x7fe9a015da10 legacy=0x7fe9a00b2bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:07.699 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:07.696+0000 7fe9a4d0c640 1 -- 172.21.15.118:0/1710982159 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9a0071a30 con 0x7fe9a0072230 2023-11-24T06:10:07.700 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:07.697+0000 7fe9a4d0c640 1 --2- 172.21.15.118:0/1710982159 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe9a0072230 0x7fe9a00b32e0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fe99800aed0 tx=0x7fe99802d020 comp rx=0 tx=0).ready entity=mon.2 client_cookie=957e9f9002433483 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:07.700 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:07.697+0000 7fe99effd640 1 -- 172.21.15.118:0/1710982159 <== mon.2 v2:172.21.15.118:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe99802dbc0 con 0x7fe9a0072230 2023-11-24T06:10:07.700 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:07.697+0000 7fe99effd640 1 -- 172.21.15.118:0/1710982159 <== mon.2 v2:172.21.15.118:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe99802dd20 con 0x7fe9a0072230 2023-11-24T06:10:07.700 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:07.697+0000 7fe99effd640 1 -- 172.21.15.118:0/1710982159 <== mon.2 v2:172.21.15.118:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe998008660 con 0x7fe9a0072230 2023-11-24T06:10:07.701 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:07.698+0000 7fe9a676f640 1 -- 172.21.15.118:0/1710982159 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe9a0072230 msgr2=0x7fe9a00b32e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:07.701 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:07.698+0000 7fe9a676f640 1 --2- 172.21.15.118:0/1710982159 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe9a0072230 0x7fe9a00b32e0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fe99800aed0 tx=0x7fe99802d020 comp rx=0 tx=0).stop 2023-11-24T06:10:07.701 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:07.698+0000 7fe9a676f640 1 -- 172.21.15.118:0/1710982159 shutdown_connections 2023-11-24T06:10:07.701 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:07.698+0000 7fe9a676f640 1 --2- 172.21.15.118:0/1710982159 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe9a0072230 0x7fe9a00b32e0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:07.701 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:07.698+0000 7fe9a676f640 1 -- 172.21.15.118:0/1710982159 >> 172.21.15.118:0/1710982159 conn(0x7fe9a006a800 msgr2=0x7fe9a006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:07.701 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:07.698+0000 7fe9a676f640 1 -- 172.21.15.118:0/1710982159 shutdown_connections 2023-11-24T06:10:07.701 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:07.698+0000 7fe9a676f640 1 -- 172.21.15.118:0/1710982159 wait complete. 2023-11-24T06:10:07.702 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:07.700+0000 7fe9a676f640 1 Processor -- start 2023-11-24T06:10:07.702 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:07.700+0000 7fe9a676f640 1 -- start start 2023-11-24T06:10:07.705 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:07.701+0000 7fe9a676f640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe9a0072230 0x7fe9a015c770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:07.705 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:07.701+0000 7fe9a676f640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe9a0150760 0x7fe9a00aeb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:07.705 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:07.701+0000 7fe9a676f640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe9a015da10 0x7fe9a0143b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:07.705 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:07.701+0000 7fe9a676f640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9a01440b0 con 0x7fe9a0072230 2023-11-24T06:10:07.705 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:07.701+0000 7fe9a676f640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9a0141200 con 0x7fe9a0150760 2023-11-24T06:10:07.705 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:07.701+0000 7fe9a676f640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9a01414e0 con 0x7fe9a015da10 2023-11-24T06:10:07.705 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:07.701+0000 7fe99f7fe640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe9a0150760 0x7fe9a00aeb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:07.705 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:07.701+0000 7fe9a4d0c640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe9a015da10 0x7fe9a0143b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:07.705 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:07.701+0000 7fe99f7fe640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe9a0150760 0x7fe9a00aeb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:52140/0 (socket says 172.21.15.118:52140) 2023-11-24T06:10:07.705 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:07.701+0000 7fe99f7fe640 1 -- 172.21.15.118:0/540090546 learned_addr learned my addr 172.21.15.118:0/540090546 (peer_addr_for_me v2:172.21.15.118:0/0) 2023-11-24T06:10:07.705 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:07.701+0000 7fe99ffff640 1 --2- 172.21.15.118:0/540090546 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe9a0072230 0x7fe9a015c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:07.706 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:07.702+0000 7fe99f7fe640 1 -- 172.21.15.118:0/540090546 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe9a015da10 msgr2=0x7fe9a0143b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:07.706 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:07.702+0000 7fe99f7fe640 1 --2- 172.21.15.118:0/540090546 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe9a015da10 0x7fe9a0143b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:07.706 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:07.702+0000 7fe99f7fe640 1 -- 172.21.15.118:0/540090546 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe9a0072230 msgr2=0x7fe9a015c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:07.706 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:07.702+0000 7fe99f7fe640 1 --2- 172.21.15.118:0/540090546 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe9a0072230 0x7fe9a015c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:07.706 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:07.702+0000 7fe99f7fe640 1 -- 172.21.15.118:0/540090546 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe99800ab80 con 0x7fe9a0150760 2023-11-24T06:10:07.706 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:07.702+0000 7fe9a4d0c640 1 --2- 172.21.15.118:0/540090546 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe9a015da10 0x7fe9a0143b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:10:07.706 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:07.702+0000 7fe99f7fe640 1 --2- 172.21.15.118:0/540090546 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe9a0150760 0x7fe9a00aeb50 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fe99400c400 tx=0x7fe99400c8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:07.706 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:07.702+0000 7fe99cff9640 1 -- 172.21.15.118:0/540090546 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9940052d0 con 0x7fe9a0150760 2023-11-24T06:10:07.706 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:07.702+0000 7fe99cff9640 1 -- 172.21.15.118:0/540090546 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe994005430 con 0x7fe9a0150760 2023-11-24T06:10:07.706 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:07.702+0000 7fe99cff9640 1 -- 172.21.15.118:0/540090546 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe994003830 con 0x7fe9a0150760 2023-11-24T06:10:07.706 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:07.702+0000 7fe9a676f640 1 -- 172.21.15.118:0/540090546 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9a01417c0 con 0x7fe9a0150760 2023-11-24T06:10:07.706 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:07.702+0000 7fe9a676f640 1 -- 172.21.15.118:0/540090546 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9a0141d80 con 0x7fe9a0150760 2023-11-24T06:10:07.707 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:07.702+0000 7fe99cff9640 1 -- 172.21.15.118:0/540090546 <== mon.1 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fe994003c90 con 0x7fe9a0150760 2023-11-24T06:10:07.707 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:07.703+0000 7fe99cff9640 1 -- 172.21.15.118:0/540090546 <== mon.1 v2:172.21.15.118:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1070+0+0 (secure 0 0 0) 0x7fe994011a20 con 0x7fe9a0150760 2023-11-24T06:10:07.707 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:07.703+0000 7fe9a676f640 1 -- 172.21.15.118:0/540090546 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9a0141fa0 con 0x7fe9a0150760 2023-11-24T06:10:07.708 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:07.706+0000 7fe99cff9640 1 -- 172.21.15.118:0/540090546 <== mon.1 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7fe994017070 con 0x7fe9a0150760 2023-11-24T06:10:07.786 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:07.784+0000 7fe9a676f640 1 -- 172.21.15.118:0/540090546 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "osd new", "uuid": "4648d7ce-9b29-4583-9425-4ba8fd2b1903", "id": 3} v 0) v1 -- 0x7fe9a00675d0 con 0x7fe9a0150760 2023-11-24T06:10:07.791 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:07.789+0000 7fe99cff9640 1 -- 172.21.15.118:0/540090546 <== mon.1 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "4648d7ce-9b29-4583-9425-4ba8fd2b1903", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7fe994011590 con 0x7fe9a0150760 2023-11-24T06:10:07.791 INFO:teuthology.orchestra.run.smithi118.stdout:3 2023-11-24T06:10:07.793 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:07.790+0000 7fe9a676f640 1 -- 172.21.15.118:0/540090546 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe9a0150760 msgr2=0x7fe9a00aeb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:07.793 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:07.790+0000 7fe9a676f640 1 --2- 172.21.15.118:0/540090546 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe9a0150760 0x7fe9a00aeb50 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fe99400c400 tx=0x7fe99400c8c0 comp rx=0 tx=0).stop 2023-11-24T06:10:07.793 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:07.790+0000 7fe9a676f640 1 -- 172.21.15.118:0/540090546 shutdown_connections 2023-11-24T06:10:07.793 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:07.790+0000 7fe9a676f640 1 --2- 172.21.15.118:0/540090546 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe9a015da10 0x7fe9a0143b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:07.793 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:07.790+0000 7fe9a676f640 1 --2- 172.21.15.118:0/540090546 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe9a0150760 0x7fe9a00aeb50 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:07.793 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:07.790+0000 7fe9a676f640 1 --2- 172.21.15.118:0/540090546 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe9a0072230 0x7fe9a015c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:07.793 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:07.790+0000 7fe9a676f640 1 -- 172.21.15.118:0/540090546 >> 172.21.15.118:0/540090546 conn(0x7fe9a006a800 msgr2=0x7fe9a006cd00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:07.793 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:07.790+0000 7fe9a676f640 1 -- 172.21.15.118:0/540090546 shutdown_connections 2023-11-24T06:10:07.793 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:07.790+0000 7fe9a676f640 1 -- 172.21.15.118:0/540090546 wait complete. 2023-11-24T06:10:07.808 DEBUG:teuthology.orchestra.run.smithi118:> sudo ceph --cluster ceph osd new 3bc9d7eb-c6fa-4fcf-a3ec-ade604e05aac 4 2023-11-24T06:10:07.808 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-11-24T06:10:07.805+0000 7ff6ac62e040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-24T06:10:07.829 INFO:tasks.ceph.mgr.x.smithi118.stderr:2023-11-24T06:10:07.826+0000 7f538f62c040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-11-24T06:10:07.858 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-11-24T06:10:07.855+0000 7ff6ac62e040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-24T06:10:07.987 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:07.984+0000 7f6674cf8640 1 Processor -- start 2023-11-24T06:10:07.987 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:07.985+0000 7f6674cf8640 1 -- start start 2023-11-24T06:10:07.989 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:07.986+0000 7f6674cf8640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6670160e00 0x7f66700cdd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:07.989 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:07.986+0000 7f6674cf8640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f66700cff60 0x7f66700ce270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:07.989 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:07.986+0000 7f6674cf8640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66700d1e00 con 0x7f6670160a20 2023-11-24T06:10:07.989 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:07.986+0000 7f6674cf8640 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66700d3c40 con 0x7f66700cff60 2023-11-24T06:10:07.989 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:07.986+0000 7f6674cf8640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66700d3e50 con 0x7f6670160e00 2023-11-24T06:10:07.989 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:07.986+0000 7f6667fff640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6670160e00 0x7f66700cdd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:07.989 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:07.986+0000 7f666ffff640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f66700cff60 0x7f66700ce270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:07.989 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:07.986+0000 7f6667fff640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6670160e00 0x7f66700cdd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3301/0 says I am v2:172.21.15.118:60996/0 (socket says 172.21.15.118:60996) 2023-11-24T06:10:07.989 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:07.986+0000 7f6667fff640 1 -- 172.21.15.118:0/1063009004 learned_addr learned my addr 172.21.15.118:0/1063009004 (peer_addr_for_me v2:172.21.15.118:0/0) 2023-11-24T06:10:07.990 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:07.987+0000 7f6667fff640 1 -- 172.21.15.118:0/1063009004 >> v2:172.21.15.118:3300/0 conn(0x7f66700cff60 msgr2=0x7f66700ce270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:07.990 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:07.987+0000 7f6667fff640 1 --2- 172.21.15.118:0/1063009004 >> v2:172.21.15.118:3300/0 conn(0x7f66700cff60 0x7f66700ce270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:07.990 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:07.987+0000 7f6667fff640 1 -- 172.21.15.118:0/1063009004 >> v1:172.21.15.92:6789/0 conn(0x7f6670160a20 legacy=0x7f66700cf850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:07.990 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:07.987+0000 7f6667fff640 1 -- 172.21.15.118:0/1063009004 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66700d1a90 con 0x7f6670160e00 2023-11-24T06:10:07.990 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:07.988+0000 7f6667fff640 1 --2- 172.21.15.118:0/1063009004 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6670160e00 0x7f66700cdd30 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f6658009870 tx=0x7f665802b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=48bde056b0b06db7 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:07.991 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:07.988+0000 7f666effd640 1 -- 172.21.15.118:0/1063009004 <== mon.2 v2:172.21.15.118:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6658031640 con 0x7f6670160e00 2023-11-24T06:10:07.992 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:07.988+0000 7f666effd640 1 -- 172.21.15.118:0/1063009004 <== mon.2 v2:172.21.15.118:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66580317a0 con 0x7f6670160e00 2023-11-24T06:10:07.992 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:07.988+0000 7f666effd640 1 -- 172.21.15.118:0/1063009004 <== mon.2 v2:172.21.15.118:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6658031b30 con 0x7f6670160e00 2023-11-24T06:10:07.992 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:07.989+0000 7f6674cf8640 1 -- 172.21.15.118:0/1063009004 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6670160e00 msgr2=0x7f66700cdd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:07.992 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:07.989+0000 7f6674cf8640 1 --2- 172.21.15.118:0/1063009004 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6670160e00 0x7f66700cdd30 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f6658009870 tx=0x7f665802b8f0 comp rx=0 tx=0).stop 2023-11-24T06:10:07.992 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:07.989+0000 7f6674cf8640 1 -- 172.21.15.118:0/1063009004 shutdown_connections 2023-11-24T06:10:07.992 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:07.989+0000 7f6674cf8640 1 --2- 172.21.15.118:0/1063009004 >> v2:172.21.15.118:3300/0 conn(0x7f66700cff60 0x7f66700ce270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:07.992 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:07.989+0000 7f6674cf8640 1 --2- 172.21.15.118:0/1063009004 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6670160e00 0x7f66700cdd30 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:07.992 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:07.989+0000 7f6674cf8640 1 -- 172.21.15.118:0/1063009004 >> 172.21.15.118:0/1063009004 conn(0x7f66700d3120 msgr2=0x7f66700d3530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:07.992 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:07.989+0000 7f6674cf8640 1 -- 172.21.15.118:0/1063009004 shutdown_connections 2023-11-24T06:10:07.992 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:07.989+0000 7f6674cf8640 1 -- 172.21.15.118:0/1063009004 wait complete. 2023-11-24T06:10:07.993 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:07.991+0000 7f6674cf8640 1 Processor -- start 2023-11-24T06:10:07.993 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:07.991+0000 7f6674cf8640 1 -- start start 2023-11-24T06:10:07.995 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:07.992+0000 7f6674cf8640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f66700cff60 0x7f667014d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:07.995 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:07.992+0000 7f6674cf8640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6670160a20 0x7f66701f2900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:07.995 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:07.992+0000 7f6674cf8640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6670160e00 0x7f66701f6490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:07.995 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:07.992+0000 7f6674cf8640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66700b7ed0 con 0x7f6670160e00 2023-11-24T06:10:07.995 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:07.992+0000 7f6674cf8640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66701f2e40 con 0x7f66700cff60 2023-11-24T06:10:07.995 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:07.992+0000 7f6674cf8640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f667014d030 con 0x7f6670160a20 2023-11-24T06:10:07.996 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:07.992+0000 7f666f7fe640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f66700cff60 0x7f667014d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:07.996 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:07.992+0000 7f6667fff640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6670160a20 0x7f66701f2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:07.996 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:07.992+0000 7f666f7fe640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f66700cff60 0x7f667014d2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:52164/0 (socket says 172.21.15.118:52164) 2023-11-24T06:10:07.996 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:07.992+0000 7f666f7fe640 1 -- 172.21.15.118:0/4021322710 learned_addr learned my addr 172.21.15.118:0/4021322710 (peer_addr_for_me v2:172.21.15.118:0/0) 2023-11-24T06:10:07.996 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:07.992+0000 7f666ffff640 1 --2- 172.21.15.118:0/4021322710 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6670160e00 0x7f66701f6490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:07.996 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:07.992+0000 7f6667fff640 1 -- 172.21.15.118:0/4021322710 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f66700cff60 msgr2=0x7f667014d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:07.996 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:07.992+0000 7f6667fff640 1 --2- 172.21.15.118:0/4021322710 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f66700cff60 0x7f667014d2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:07.996 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:07.992+0000 7f6667fff640 1 -- 172.21.15.118:0/4021322710 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6670160e00 msgr2=0x7f66701f6490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:07.996 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:07.992+0000 7f6667fff640 1 --2- 172.21.15.118:0/4021322710 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6670160e00 0x7f66701f6490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:07.996 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:07.993+0000 7f6667fff640 1 -- 172.21.15.118:0/4021322710 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66580094d0 con 0x7f6670160a20 2023-11-24T06:10:07.996 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:07.993+0000 7f6667fff640 1 --2- 172.21.15.118:0/4021322710 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6670160a20 0x7f66701f2900 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f6658033000 tx=0x7f66580046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:07.997 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:07.993+0000 7f666cff9640 1 -- 172.21.15.118:0/4021322710 <== mon.2 v2:172.21.15.118:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f665803ac60 con 0x7f6670160a20 2023-11-24T06:10:07.998 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:07.993+0000 7f666cff9640 1 -- 172.21.15.118:0/4021322710 <== mon.2 v2:172.21.15.118:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f665803adc0 con 0x7f6670160a20 2023-11-24T06:10:07.998 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:07.993+0000 7f6674cf8640 1 -- 172.21.15.118:0/4021322710 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66700ccb10 con 0x7f6670160a20 2023-11-24T06:10:07.998 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:07.993+0000 7f666cff9640 1 -- 172.21.15.118:0/4021322710 <== mon.2 v2:172.21.15.118:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6658044610 con 0x7f6670160a20 2023-11-24T06:10:07.998 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:07.993+0000 7f6674cf8640 1 -- 172.21.15.118:0/4021322710 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66700b7810 con 0x7f6670160a20 2023-11-24T06:10:07.998 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:07.994+0000 7f666cff9640 1 -- 172.21.15.118:0/4021322710 <== mon.2 v2:172.21.15.118:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f665802f050 con 0x7f6670160a20 2023-11-24T06:10:07.998 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:07.994+0000 7f666cff9640 1 -- 172.21.15.118:0/4021322710 <== mon.2 v2:172.21.15.118:3301/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1185+0+0 (secure 0 0 0) 0x7f66580398c0 con 0x7f6670160a20 2023-11-24T06:10:07.998 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:07.994+0000 7f6674cf8640 1 -- 172.21.15.118:0/4021322710 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6670067b60 con 0x7f6670160a20 2023-11-24T06:10:07.998 INFO:tasks.ceph.mgr.x.smithi118.stderr:2023-11-24T06:10:07.995+0000 7f538f62c040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-24T06:10:07.999 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:07.997+0000 7f666cff9640 1 -- 172.21.15.118:0/4021322710 <== mon.2 v2:172.21.15.118:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7f6658035020 con 0x7f6670160a20 2023-11-24T06:10:08.015 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-11-24T06:10:08.012+0000 7ff6ac62e040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-24T06:10:08.077 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:08.074+0000 7f6674cf8640 1 -- 172.21.15.118:0/4021322710 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_command({"prefix": "osd new", "uuid": "3bc9d7eb-c6fa-4fcf-a3ec-ade604e05aac", "id": 4} v 0) v1 -- 0x7f66700cde70 con 0x7f6670160a20 2023-11-24T06:10:08.084 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:08.082+0000 7f666cff9640 1 -- 172.21.15.118:0/4021322710 <== mon.2 v2:172.21.15.118:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "3bc9d7eb-c6fa-4fcf-a3ec-ade604e05aac", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7f665802d040 con 0x7f6670160a20 2023-11-24T06:10:08.084 INFO:teuthology.orchestra.run.smithi118.stdout:4 2023-11-24T06:10:08.086 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:08.083+0000 7f6674cf8640 1 -- 172.21.15.118:0/4021322710 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6670160a20 msgr2=0x7f66701f2900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:08.086 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:08.083+0000 7f6674cf8640 1 --2- 172.21.15.118:0/4021322710 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6670160a20 0x7f66701f2900 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f6658033000 tx=0x7f66580046c0 comp rx=0 tx=0).stop 2023-11-24T06:10:08.086 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:08.083+0000 7f6674cf8640 1 -- 172.21.15.118:0/4021322710 shutdown_connections 2023-11-24T06:10:08.086 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:08.083+0000 7f6674cf8640 1 --2- 172.21.15.118:0/4021322710 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6670160e00 0x7f66701f6490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:08.086 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:08.083+0000 7f6674cf8640 1 --2- 172.21.15.118:0/4021322710 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6670160a20 0x7f66701f2900 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:08.087 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:08.083+0000 7f6674cf8640 1 --2- 172.21.15.118:0/4021322710 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f66700cff60 0x7f667014d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:08.087 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:08.083+0000 7f6674cf8640 1 -- 172.21.15.118:0/4021322710 >> 172.21.15.118:0/4021322710 conn(0x7f66700d3120 msgr2=0x7f66700b2180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:08.087 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:08.083+0000 7f6674cf8640 1 -- 172.21.15.118:0/4021322710 shutdown_connections 2023-11-24T06:10:08.087 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:08.084+0000 7f6674cf8640 1 -- 172.21.15.118:0/4021322710 wait complete. 2023-11-24T06:10:08.103 DEBUG:teuthology.orchestra.run.smithi118:> sudo ceph --cluster ceph osd new dce9ba6d-59ed-46d1-84fc-a84b13de10c5 5 2023-11-24T06:10:08.167 INFO:tasks.ceph.mgr.x.smithi118.stderr:2023-11-24T06:10:08.165+0000 7f538f62c040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-24T06:10:08.226 INFO:tasks.ceph.mgr.x.smithi118.stderr:2023-11-24T06:10:08.224+0000 7f538f62c040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-24T06:10:08.336 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:08.334+0000 7f6141859640 1 Processor -- start 2023-11-24T06:10:08.336 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:08.334+0000 7f6141859640 1 -- start start 2023-11-24T06:10:08.338 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:08.335+0000 7f6141859640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f613c15d4b0 0x7f613c0d1550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:08.338 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:08.335+0000 7f6141859640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f613c0b41e0 0x7f613c0cfde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:08.339 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:08.335+0000 7f6141859640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f613c0d0320 con 0x7f613c0cf9a0 2023-11-24T06:10:08.339 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:08.335+0000 7f6141859640 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f613c0b2730 con 0x7f613c15d4b0 2023-11-24T06:10:08.339 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:08.335+0000 7f6141859640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f613c0d1ec0 con 0x7f613c0b41e0 2023-11-24T06:10:08.339 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:08.336+0000 7f6141058640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f613c0b41e0 0x7f613c0cfde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:08.339 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:08.336+0000 7f6140857640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f613c15d4b0 0x7f613c0d1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:08.339 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:08.336+0000 7f6141058640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f613c0b41e0 0x7f613c0cfde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3301/0 says I am v2:172.21.15.118:32794/0 (socket says 172.21.15.118:32794) 2023-11-24T06:10:08.339 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:08.336+0000 7f6141058640 1 -- 172.21.15.118:0/2912101805 learned_addr learned my addr 172.21.15.118:0/2912101805 (peer_addr_for_me v2:172.21.15.118:0/0) 2023-11-24T06:10:08.339 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:08.336+0000 7f6141058640 1 -- 172.21.15.118:0/2912101805 >> v2:172.21.15.118:3300/0 conn(0x7f613c15d4b0 msgr2=0x7f613c0d1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:08.339 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:08.336+0000 7f6141058640 1 --2- 172.21.15.118:0/2912101805 >> v2:172.21.15.118:3300/0 conn(0x7f613c15d4b0 0x7f613c0d1550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:08.339 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:08.336+0000 7f6141058640 1 -- 172.21.15.118:0/2912101805 >> v1:172.21.15.92:6789/0 conn(0x7f613c0cf9a0 legacy=0x7f613c0cddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:08.339 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:08.336+0000 7f6141058640 1 -- 172.21.15.118:0/2912101805 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f613c0b29d0 con 0x7f613c0b41e0 2023-11-24T06:10:08.341 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:08.337+0000 7f6141058640 1 --2- 172.21.15.118:0/2912101805 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f613c0b41e0 0x7f613c0cfde0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f6134009870 tx=0x7f613402b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5a72c898e1d5aa9a server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:08.341 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:08.337+0000 7f613b7fe640 1 -- 172.21.15.118:0/2912101805 <== mon.2 v2:172.21.15.118:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6134031640 con 0x7f613c0b41e0 2023-11-24T06:10:08.341 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:08.338+0000 7f613b7fe640 1 -- 172.21.15.118:0/2912101805 <== mon.2 v2:172.21.15.118:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61340317a0 con 0x7f613c0b41e0 2023-11-24T06:10:08.341 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:08.338+0000 7f613b7fe640 1 -- 172.21.15.118:0/2912101805 <== mon.2 v2:172.21.15.118:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6134031b30 con 0x7f613c0b41e0 2023-11-24T06:10:08.341 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:08.338+0000 7f6141859640 1 -- 172.21.15.118:0/2912101805 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f613c0b41e0 msgr2=0x7f613c0cfde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:08.341 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:08.338+0000 7f6141859640 1 --2- 172.21.15.118:0/2912101805 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f613c0b41e0 0x7f613c0cfde0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f6134009870 tx=0x7f613402b8f0 comp rx=0 tx=0).stop 2023-11-24T06:10:08.341 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:08.338+0000 7f6141859640 1 -- 172.21.15.118:0/2912101805 shutdown_connections 2023-11-24T06:10:08.341 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:08.338+0000 7f6141859640 1 --2- 172.21.15.118:0/2912101805 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f613c0b41e0 0x7f613c0cfde0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:08.341 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:08.338+0000 7f6141859640 1 --2- 172.21.15.118:0/2912101805 >> v2:172.21.15.118:3300/0 conn(0x7f613c15d4b0 0x7f613c0d1550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:08.341 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:08.338+0000 7f6141859640 1 -- 172.21.15.118:0/2912101805 >> 172.21.15.118:0/2912101805 conn(0x7f613c0d3000 msgr2=0x7f613c0d3410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:08.341 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:08.338+0000 7f6141859640 1 -- 172.21.15.118:0/2912101805 shutdown_connections 2023-11-24T06:10:08.341 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:08.338+0000 7f6141859640 1 -- 172.21.15.118:0/2912101805 wait complete. 2023-11-24T06:10:08.342 INFO:tasks.ceph.mgr.x.smithi118.stderr:2023-11-24T06:10:08.339+0000 7f538f62c040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-24T06:10:08.342 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:08.340+0000 7f6141859640 1 Processor -- start 2023-11-24T06:10:08.342 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:08.340+0000 7f6141859640 1 -- start start 2023-11-24T06:10:08.345 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:08.341+0000 7f6141859640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f613c0b41e0 0x7f613c14c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:08.345 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:08.341+0000 7f6141859640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f613c0cf9a0 0x7f613c09f980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:08.345 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:08.341+0000 7f6141859640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f613c15d4b0 0x7f613c09fec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:08.345 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:08.341+0000 7f6141859640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f613c09d090 con 0x7f613c0cf9a0 2023-11-24T06:10:08.345 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:08.341+0000 7f6141859640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f613c0b7d20 con 0x7f613c15d4b0 2023-11-24T06:10:08.345 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:08.341+0000 7f6141859640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f613c09d270 con 0x7f613c0b41e0 2023-11-24T06:10:08.345 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:08.341+0000 7f6140857640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f613c0b41e0 0x7f613c14c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:08.345 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:08.341+0000 7f6141058640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f613c15d4b0 0x7f613c09fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:08.345 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:08.341+0000 7f6140857640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f613c0b41e0 0x7f613c14c3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3301/0 says I am v2:172.21.15.118:32798/0 (socket says 172.21.15.118:32798) 2023-11-24T06:10:08.345 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:08.341+0000 7f6140857640 1 -- 172.21.15.118:0/3115362304 learned_addr learned my addr 172.21.15.118:0/3115362304 (peer_addr_for_me v2:172.21.15.118:0/0) 2023-11-24T06:10:08.346 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:08.342+0000 7f613bfff640 1 --2- 172.21.15.118:0/3115362304 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f613c0cf9a0 0x7f613c09f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:08.346 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:08.342+0000 7f6141058640 1 -- 172.21.15.118:0/3115362304 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f613c0b41e0 msgr2=0x7f613c14c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:08.346 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:08.342+0000 7f6141058640 1 --2- 172.21.15.118:0/3115362304 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f613c0b41e0 0x7f613c14c3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:08.346 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:08.342+0000 7f6141058640 1 -- 172.21.15.118:0/3115362304 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f613c0cf9a0 msgr2=0x7f613c09f980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:08.346 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:08.342+0000 7f6141058640 1 --2- 172.21.15.118:0/3115362304 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f613c0cf9a0 0x7f613c09f980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:08.346 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:08.342+0000 7f6141058640 1 -- 172.21.15.118:0/3115362304 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61340094d0 con 0x7f613c15d4b0 2023-11-24T06:10:08.346 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:08.342+0000 7f6140857640 1 --2- 172.21.15.118:0/3115362304 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f613c0b41e0 0x7f613c14c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:10:08.346 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:08.342+0000 7f6141058640 1 --2- 172.21.15.118:0/3115362304 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f613c15d4b0 0x7f613c09fec0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f6134033000 tx=0x7f61340046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:08.346 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:08.342+0000 7f61397fa640 1 -- 172.21.15.118:0/3115362304 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f613403ac60 con 0x7f613c15d4b0 2023-11-24T06:10:08.346 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:08.342+0000 7f61397fa640 1 -- 172.21.15.118:0/3115362304 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f613403adc0 con 0x7f613c15d4b0 2023-11-24T06:10:08.346 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:08.342+0000 7f61397fa640 1 -- 172.21.15.118:0/3115362304 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6134044630 con 0x7f613c15d4b0 2023-11-24T06:10:08.347 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:08.342+0000 7f6141859640 1 -- 172.21.15.118:0/3115362304 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f613c15da10 con 0x7f613c15d4b0 2023-11-24T06:10:08.347 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:08.342+0000 7f6141859640 1 -- 172.21.15.118:0/3115362304 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f613c09d830 con 0x7f613c15d4b0 2023-11-24T06:10:08.347 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:08.343+0000 7f61397fa640 1 -- 172.21.15.118:0/3115362304 <== mon.1 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f613402f050 con 0x7f613c15d4b0 2023-11-24T06:10:08.347 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:08.343+0000 7f61397fa640 1 -- 172.21.15.118:0/3115362304 <== mon.1 v2:172.21.15.118:3300/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1300+0+0 (secure 0 0 0) 0x7f6134044a90 con 0x7f613c15d4b0 2023-11-24T06:10:08.347 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:08.344+0000 7f6141859640 1 -- 172.21.15.118:0/3115362304 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f613c09da20 con 0x7f613c15d4b0 2023-11-24T06:10:08.348 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:08.346+0000 7f61397fa640 1 -- 172.21.15.118:0/3115362304 <== mon.1 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7f6134035020 con 0x7f613c15d4b0 2023-11-24T06:10:08.427 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:08.424+0000 7f6141859640 1 -- 172.21.15.118:0/3115362304 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "osd new", "uuid": "dce9ba6d-59ed-46d1-84fc-a84b13de10c5", "id": 5} v 0) v1 -- 0x7f613c067450 con 0x7f613c15d4b0 2023-11-24T06:10:08.433 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:08.431+0000 7f61397fa640 1 -- 172.21.15.118:0/3115362304 <== mon.1 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "dce9ba6d-59ed-46d1-84fc-a84b13de10c5", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7f613402d040 con 0x7f613c15d4b0 2023-11-24T06:10:08.433 INFO:teuthology.orchestra.run.smithi118.stdout:5 2023-11-24T06:10:08.435 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:08.432+0000 7f6141859640 1 -- 172.21.15.118:0/3115362304 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f613c15d4b0 msgr2=0x7f613c09fec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:08.435 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:08.432+0000 7f6141859640 1 --2- 172.21.15.118:0/3115362304 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f613c15d4b0 0x7f613c09fec0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f6134033000 tx=0x7f61340046c0 comp rx=0 tx=0).stop 2023-11-24T06:10:08.435 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:08.432+0000 7f6141859640 1 -- 172.21.15.118:0/3115362304 shutdown_connections 2023-11-24T06:10:08.435 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:08.432+0000 7f6141859640 1 --2- 172.21.15.118:0/3115362304 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f613c15d4b0 0x7f613c09fec0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:08.435 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:08.432+0000 7f6141859640 1 --2- 172.21.15.118:0/3115362304 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f613c0cf9a0 0x7f613c09f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:08.435 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:08.432+0000 7f6141859640 1 --2- 172.21.15.118:0/3115362304 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f613c0b41e0 0x7f613c14c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:08.435 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:08.432+0000 7f6141859640 1 -- 172.21.15.118:0/3115362304 >> 172.21.15.118:0/3115362304 conn(0x7f613c0d3000 msgr2=0x7f613c06c100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:08.435 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:08.432+0000 7f6141859640 1 -- 172.21.15.118:0/3115362304 shutdown_connections 2023-11-24T06:10:08.435 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:08.432+0000 7f6141859640 1 -- 172.21.15.118:0/3115362304 wait complete. 2023-11-24T06:10:08.451 DEBUG:teuthology.orchestra.run.smithi118:> sudo ceph --cluster ceph osd new 82aec504-ae58-4709-8d30-46f2b12081b8 6 2023-11-24T06:10:08.460 INFO:tasks.ceph.mgr.x.smithi118.stderr:2023-11-24T06:10:08.457+0000 7f538f62c040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-24T06:10:08.626 INFO:tasks.ceph.mgr.x.smithi118.stderr:2023-11-24T06:10:08.623+0000 7f538f62c040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-24T06:10:08.648 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:08.646+0000 7f43730ad640 1 Processor -- start 2023-11-24T06:10:08.648 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:08.646+0000 7f43730ad640 1 -- start start 2023-11-24T06:10:08.650 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:08.647+0000 7f43730ad640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f436c0b1f70 0x7f436c0b08d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:08.651 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:08.647+0000 7f43730ad640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f436c0d0090 con 0x7f436c15d1d0 2023-11-24T06:10:08.651 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:08.647+0000 7f43730ad640 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f436c0d02e0 con 0x7f436c0d1d40 2023-11-24T06:10:08.651 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:08.647+0000 7f43730ad640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f436c0d05c0 con 0x7f436c0b1f70 2023-11-24T06:10:08.651 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:08.648+0000 7f43718aa640 1 --1- >> v1:172.21.15.118:6789/0 conn(0x7f436c0d1d40 0x7f436c0b01c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.118:6789/0 says I am v1:172.21.15.118:51170/0 (socket says 172.21.15.118:51170) 2023-11-24T06:10:08.651 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:08.648+0000 7f43718aa640 1 -- 172.21.15.118:0/3028973741 learned_addr learned my addr 172.21.15.118:0/3028973741 (peer_addr_for_me v1:172.21.15.118:0/0) 2023-11-24T06:10:08.651 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:08.648+0000 7f43728ac640 1 --2- 172.21.15.118:0/3028973741 >> v2:172.21.15.92:3300/0 conn(0x7f436c0b1f70 0x7f436c0b08d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:08.651 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:08.648+0000 7f43710a9640 1 -- 172.21.15.118:0/3028973741 <== mon.1 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2715821065 0 0) 0x7f436c0d02e0 con 0x7f436c0d1d40 2023-11-24T06:10:08.651 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:08.648+0000 7f43710a9640 1 -- 172.21.15.118:0/3028973741 --> v1:172.21.15.118:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4358003520 con 0x7f436c0d1d40 2023-11-24T06:10:08.651 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:08.648+0000 7f43728ac640 1 -- 172.21.15.118:0/3028973741 >> v1:172.21.15.118:6789/0 conn(0x7f436c0d1d40 legacy=0x7f436c0b01c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:08.651 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:08.648+0000 7f43728ac640 1 -- 172.21.15.118:0/3028973741 >> v1:172.21.15.92:6789/0 conn(0x7f436c15d1d0 legacy=0x7f436c0d1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:08.652 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:08.648+0000 7f43728ac640 1 -- 172.21.15.118:0/3028973741 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f436c0d08a0 con 0x7f436c0b1f70 2023-11-24T06:10:08.652 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:08.649+0000 7f43728ac640 1 --2- 172.21.15.118:0/3028973741 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f436c0b1f70 0x7f436c0b08d0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f4364009dc0 tx=0x7f436402bf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=934e85a14316a7d4 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:08.652 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:08.650+0000 7f43710a9640 1 -- 172.21.15.118:0/3028973741 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f436402cce0 con 0x7f436c0b1f70 2023-11-24T06:10:08.652 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:08.650+0000 7f43710a9640 1 -- 172.21.15.118:0/3028973741 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f436402ce40 con 0x7f436c0b1f70 2023-11-24T06:10:08.653 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:08.650+0000 7f43710a9640 1 -- 172.21.15.118:0/3028973741 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4364034660 con 0x7f436c0b1f70 2023-11-24T06:10:08.653 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:08.650+0000 7f43730ad640 1 -- 172.21.15.118:0/3028973741 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f436c0b1f70 msgr2=0x7f436c0b08d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:08.653 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:08.650+0000 7f43730ad640 1 --2- 172.21.15.118:0/3028973741 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f436c0b1f70 0x7f436c0b08d0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f4364009dc0 tx=0x7f436402bf30 comp rx=0 tx=0).stop 2023-11-24T06:10:08.654 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:08.650+0000 7f43730ad640 1 -- 172.21.15.118:0/3028973741 shutdown_connections 2023-11-24T06:10:08.654 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:08.650+0000 7f43730ad640 1 --2- 172.21.15.118:0/3028973741 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f436c0b1f70 0x7f436c0b08d0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:08.654 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:08.650+0000 7f43730ad640 1 -- 172.21.15.118:0/3028973741 >> 172.21.15.118:0/3028973741 conn(0x7f436c0d3050 msgr2=0x7f436c0d3460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:08.654 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:08.651+0000 7f43730ad640 1 -- 172.21.15.118:0/3028973741 shutdown_connections 2023-11-24T06:10:08.654 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:08.651+0000 7f43730ad640 1 -- 172.21.15.118:0/3028973741 wait complete. 2023-11-24T06:10:08.654 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:08.652+0000 7f43730ad640 1 Processor -- start 2023-11-24T06:10:08.655 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:08.652+0000 7f43730ad640 1 -- start start 2023-11-24T06:10:08.656 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:08.653+0000 7f43730ad640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f436c0b1f70 0x7f436c145ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:08.657 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:08.653+0000 7f43730ad640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f436c0d1d40 0x7f436c148930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:08.657 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:08.653+0000 7f43730ad640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f436c15d1d0 0x7f436c0c93b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:08.657 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:08.653+0000 7f43730ad640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f436c0cd0b0 con 0x7f436c0b1f70 2023-11-24T06:10:08.657 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:08.653+0000 7f43730ad640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f436c0b73e0 con 0x7f436c0d1d40 2023-11-24T06:10:08.657 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:08.653+0000 7f43730ad640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f436c0ccaf0 con 0x7f436c15d1d0 2023-11-24T06:10:08.657 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:08.654+0000 7f43728ac640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f436c15d1d0 0x7f436c0c93b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:08.657 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:08.654+0000 7f43728ac640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f436c15d1d0 0x7f436c0c93b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3301/0 says I am v2:172.21.15.118:32814/0 (socket says 172.21.15.118:32814) 2023-11-24T06:10:08.657 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:08.654+0000 7f43728ac640 1 -- 172.21.15.118:0/1954479885 learned_addr learned my addr 172.21.15.118:0/1954479885 (peer_addr_for_me v2:172.21.15.118:0/0) 2023-11-24T06:10:08.657 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:08.654+0000 7f43718aa640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f436c0d1d40 0x7f436c148930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:08.658 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:08.654+0000 7f43720ab640 1 --2- 172.21.15.118:0/1954479885 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f436c0b1f70 0x7f436c145ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:08.658 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:08.654+0000 7f43728ac640 1 -- 172.21.15.118:0/1954479885 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f436c0d1d40 msgr2=0x7f436c148930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:08.658 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:08.654+0000 7f43728ac640 1 --2- 172.21.15.118:0/1954479885 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f436c0d1d40 0x7f436c148930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:08.658 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:08.654+0000 7f43728ac640 1 -- 172.21.15.118:0/1954479885 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f436c0b1f70 msgr2=0x7f436c145ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:08.658 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:08.654+0000 7f43728ac640 1 --2- 172.21.15.118:0/1954479885 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f436c0b1f70 0x7f436c145ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:08.658 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:08.654+0000 7f43728ac640 1 -- 172.21.15.118:0/1954479885 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4364009a70 con 0x7f436c15d1d0 2023-11-24T06:10:08.658 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:08.654+0000 7f43728ac640 1 --2- 172.21.15.118:0/1954479885 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f436c15d1d0 0x7f436c0c93b0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f4364037040 tx=0x7f4364004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:08.659 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:08.655+0000 7f4362ffd640 1 -- 172.21.15.118:0/1954479885 <== mon.2 v2:172.21.15.118:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f436403d600 con 0x7f436c15d1d0 2023-11-24T06:10:08.659 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:08.655+0000 7f4362ffd640 1 -- 172.21.15.118:0/1954479885 <== mon.2 v2:172.21.15.118:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f436403d7a0 con 0x7f436c15d1d0 2023-11-24T06:10:08.659 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:08.655+0000 7f4362ffd640 1 -- 172.21.15.118:0/1954479885 <== mon.2 v2:172.21.15.118:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f436403db50 con 0x7f436c15d1d0 2023-11-24T06:10:08.659 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:08.655+0000 7f43730ad640 1 -- 172.21.15.118:0/1954479885 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f436c0ccdd0 con 0x7f436c15d1d0 2023-11-24T06:10:08.659 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:08.655+0000 7f43730ad640 1 -- 172.21.15.118:0/1954479885 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f436c0b7960 con 0x7f436c15d1d0 2023-11-24T06:10:08.659 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:08.655+0000 7f4362ffd640 1 -- 172.21.15.118:0/1954479885 <== mon.2 v2:172.21.15.118:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f436403b040 con 0x7f436c15d1d0 2023-11-24T06:10:08.660 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:08.655+0000 7f4362ffd640 1 -- 172.21.15.118:0/1954479885 <== mon.2 v2:172.21.15.118:3301/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1415+0+0 (secure 0 0 0) 0x7f4364048a40 con 0x7f436c15d1d0 2023-11-24T06:10:08.660 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:08.656+0000 7f43730ad640 1 -- 172.21.15.118:0/1954479885 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f436c067450 con 0x7f436c15d1d0 2023-11-24T06:10:08.660 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:08.658+0000 7f4362ffd640 1 -- 172.21.15.118:0/1954479885 <== mon.2 v2:172.21.15.118:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7f436404d060 con 0x7f436c15d1d0 2023-11-24T06:10:08.679 INFO:tasks.ceph.mgr.x.smithi118.stderr:2023-11-24T06:10:08.676+0000 7f538f62c040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-24T06:10:08.739 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:08.737+0000 7f43730ad640 1 -- 172.21.15.118:0/1954479885 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_command({"prefix": "osd new", "uuid": "82aec504-ae58-4709-8d30-46f2b12081b8", "id": 6} v 0) v1 -- 0x7f436c0d2e30 con 0x7f436c15d1d0 2023-11-24T06:10:08.745 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:08.743+0000 7f4362ffd640 1 -- 172.21.15.118:0/1954479885 <== mon.2 v2:172.21.15.118:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "82aec504-ae58-4709-8d30-46f2b12081b8", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7f4364044070 con 0x7f436c15d1d0 2023-11-24T06:10:08.746 INFO:teuthology.orchestra.run.smithi118.stdout:6 2023-11-24T06:10:08.747 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:08.745+0000 7f43730ad640 1 -- 172.21.15.118:0/1954479885 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f436c15d1d0 msgr2=0x7f436c0c93b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:08.748 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:08.745+0000 7f43730ad640 1 --2- 172.21.15.118:0/1954479885 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f436c15d1d0 0x7f436c0c93b0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f4364037040 tx=0x7f4364004a40 comp rx=0 tx=0).stop 2023-11-24T06:10:08.748 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:08.745+0000 7f43730ad640 1 -- 172.21.15.118:0/1954479885 shutdown_connections 2023-11-24T06:10:08.748 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:08.745+0000 7f43730ad640 1 --2- 172.21.15.118:0/1954479885 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f436c15d1d0 0x7f436c0c93b0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:08.748 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:08.745+0000 7f43730ad640 1 --2- 172.21.15.118:0/1954479885 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f436c0d1d40 0x7f436c148930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:08.748 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:08.745+0000 7f43730ad640 1 --2- 172.21.15.118:0/1954479885 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f436c0b1f70 0x7f436c145ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:08.748 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:08.745+0000 7f43730ad640 1 -- 172.21.15.118:0/1954479885 >> 172.21.15.118:0/1954479885 conn(0x7f436c0d3050 msgr2=0x7f436c06b8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:08.748 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:08.745+0000 7f43730ad640 1 -- 172.21.15.118:0/1954479885 shutdown_connections 2023-11-24T06:10:08.748 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:08.745+0000 7f43730ad640 1 -- 172.21.15.118:0/1954479885 wait complete. 2023-11-24T06:10:08.764 DEBUG:teuthology.orchestra.run.smithi118:> sudo ceph --cluster ceph osd new 4ed4c624-c88f-4d35-bdb1-bf209b5dffb1 7 2023-11-24T06:10:08.844 INFO:tasks.ceph.mgr.x.smithi118.stderr:2023-11-24T06:10:08.841+0000 7f538f62c040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-24T06:10:08.950 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:08.948+0000 7ffba359e640 1 Processor -- start 2023-11-24T06:10:08.951 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:08.948+0000 7ffba359e640 1 -- start start 2023-11-24T06:10:08.952 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:08.949+0000 7ffba359e640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7ffba4160770 0x7ffba40ac8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:08.953 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:08.950+0000 7ffba359e640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ffba40714a0 0x7ffba4160e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:08.953 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:08.950+0000 7ffba359e640 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffba4066430 con 0x7ffba406e5a0 2023-11-24T06:10:08.953 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:08.950+0000 7ffba359e640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffba406a560 con 0x7ffba40714a0 2023-11-24T06:10:08.953 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:08.950+0000 7ffba359e640 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffba406a820 con 0x7ffba4160770 2023-11-24T06:10:08.953 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:08.950+0000 7ffba259c640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7ffba4160770 0x7ffba40ac8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:08.953 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:08.950+0000 7ffba259c640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7ffba4160770 0x7ffba40ac8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:52194/0 (socket says 172.21.15.118:52194) 2023-11-24T06:10:08.953 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:08.950+0000 7ffba259c640 1 -- 172.21.15.118:0/1971667751 learned_addr learned my addr 172.21.15.118:0/1971667751 (peer_addr_for_me v2:172.21.15.118:0/0) 2023-11-24T06:10:08.953 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:08.950+0000 7ffba259c640 1 -- 172.21.15.118:0/1971667751 >> v2:172.21.15.92:3300/0 conn(0x7ffba40714a0 msgr2=0x7ffba4160e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:08.953 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:08.950+0000 7ffba259c640 1 --2- 172.21.15.118:0/1971667751 >> v2:172.21.15.92:3300/0 conn(0x7ffba40714a0 0x7ffba4160e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:08.953 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:08.950+0000 7ffba259c640 1 -- 172.21.15.118:0/1971667751 >> v1:172.21.15.118:6789/0 conn(0x7ffba406e5a0 legacy=0x7ffba4070d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:08.953 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:08.950+0000 7ffba259c640 1 -- 172.21.15.118:0/1971667751 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffba406ab00 con 0x7ffba4160770 2023-11-24T06:10:08.954 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:08.951+0000 7ffba259c640 1 --2- 172.21.15.118:0/1971667751 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ffba4160770 0x7ffba40ac8d0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7ffba4071c30 tx=0x7ffb9402b9c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=766bf0fdf7876523 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:08.954 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:08.952+0000 7ffba159a640 1 -- 172.21.15.118:0/1971667751 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb94030880 con 0x7ffba4160770 2023-11-24T06:10:08.954 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:08.952+0000 7ffba159a640 1 -- 172.21.15.118:0/1971667751 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb940309e0 con 0x7ffba4160770 2023-11-24T06:10:08.954 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:08.952+0000 7ffba159a640 1 -- 172.21.15.118:0/1971667751 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb94030d70 con 0x7ffba4160770 2023-11-24T06:10:08.955 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:08.952+0000 7ffba359e640 1 -- 172.21.15.118:0/1971667751 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ffba4160770 msgr2=0x7ffba40ac8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:08.955 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:08.952+0000 7ffba359e640 1 --2- 172.21.15.118:0/1971667751 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ffba4160770 0x7ffba40ac8d0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7ffba4071c30 tx=0x7ffb9402b9c0 comp rx=0 tx=0).stop 2023-11-24T06:10:08.955 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:08.952+0000 7ffba359e640 1 -- 172.21.15.118:0/1971667751 shutdown_connections 2023-11-24T06:10:08.955 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:08.952+0000 7ffba359e640 1 --2- 172.21.15.118:0/1971667751 >> v2:172.21.15.92:3300/0 conn(0x7ffba40714a0 0x7ffba4160e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:08.955 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:08.952+0000 7ffba359e640 1 --2- 172.21.15.118:0/1971667751 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ffba4160770 0x7ffba40ac8d0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:08.956 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:08.952+0000 7ffba359e640 1 -- 172.21.15.118:0/1971667751 >> 172.21.15.118:0/1971667751 conn(0x7ffba40b15b0 msgr2=0x7ffba40b19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:08.956 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:08.952+0000 7ffba359e640 1 -- 172.21.15.118:0/1971667751 shutdown_connections 2023-11-24T06:10:08.956 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:08.953+0000 7ffba359e640 1 -- 172.21.15.118:0/1971667751 wait complete. 2023-11-24T06:10:08.956 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:08.954+0000 7ffba359e640 1 Processor -- start 2023-11-24T06:10:08.956 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:08.954+0000 7ffba359e640 1 -- start start 2023-11-24T06:10:08.959 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:08.955+0000 7ffba359e640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ffba406e5a0 0x7ffba4145ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:08.959 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:08.955+0000 7ffba359e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffba40714a0 0x7ffba415c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:08.959 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:08.955+0000 7ffba359e640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ffba4160770 0x7ffba415a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:08.959 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:08.955+0000 7ffba359e640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffba4146220 con 0x7ffba40714a0 2023-11-24T06:10:08.959 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:08.955+0000 7ffba359e640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffba40afcb0 con 0x7ffba4160770 2023-11-24T06:10:08.959 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:08.955+0000 7ffba359e640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffba40aff90 con 0x7ffba406e5a0 2023-11-24T06:10:08.959 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:08.955+0000 7ffba259c640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ffba406e5a0 0x7ffba4145ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:08.960 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:08.955+0000 7ffba259c640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ffba406e5a0 0x7ffba4145ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3301/0 says I am v2:172.21.15.118:32826/0 (socket says 172.21.15.118:32826) 2023-11-24T06:10:08.960 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:08.955+0000 7ffba259c640 1 -- 172.21.15.118:0/2006813029 learned_addr learned my addr 172.21.15.118:0/2006813029 (peer_addr_for_me v2:172.21.15.118:0/0) 2023-11-24T06:10:08.960 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:08.955+0000 7ffba2d9d640 1 --2- 172.21.15.118:0/2006813029 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ffba4160770 0x7ffba415a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:08.960 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:08.956+0000 7ffba1d9b640 1 --2- 172.21.15.118:0/2006813029 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffba40714a0 0x7ffba415c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:08.960 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:08.956+0000 7ffba259c640 1 -- 172.21.15.118:0/2006813029 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ffba4160770 msgr2=0x7ffba415a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:08.960 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:08.956+0000 7ffba259c640 1 --2- 172.21.15.118:0/2006813029 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ffba4160770 0x7ffba415a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:08.960 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:08.956+0000 7ffba259c640 1 -- 172.21.15.118:0/2006813029 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffba40714a0 msgr2=0x7ffba415c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:08.960 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:08.956+0000 7ffba259c640 1 --2- 172.21.15.118:0/2006813029 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffba40714a0 0x7ffba415c6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:08.960 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:08.956+0000 7ffba259c640 1 -- 172.21.15.118:0/2006813029 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb940094d0 con 0x7ffba406e5a0 2023-11-24T06:10:08.960 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:08.956+0000 7ffba2d9d640 1 --2- 172.21.15.118:0/2006813029 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ffba4160770 0x7ffba415a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:10:08.960 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:08.956+0000 7ffba259c640 1 --2- 172.21.15.118:0/2006813029 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ffba406e5a0 0x7ffba4145ce0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7ffb9402ba00 tx=0x7ffb94004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:08.960 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:08.957+0000 7ffb92ffd640 1 -- 172.21.15.118:0/2006813029 <== mon.2 v2:172.21.15.118:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb94030850 con 0x7ffba406e5a0 2023-11-24T06:10:08.961 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:08.957+0000 7ffb92ffd640 1 -- 172.21.15.118:0/2006813029 <== mon.2 v2:172.21.15.118:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb94043400 con 0x7ffba406e5a0 2023-11-24T06:10:08.961 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:08.957+0000 7ffb92ffd640 1 -- 172.21.15.118:0/2006813029 <== mon.2 v2:172.21.15.118:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb940437b0 con 0x7ffba406e5a0 2023-11-24T06:10:08.961 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:08.957+0000 7ffba359e640 1 -- 172.21.15.118:0/2006813029 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffba40b0270 con 0x7ffba406e5a0 2023-11-24T06:10:08.961 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:08.957+0000 7ffba359e640 1 -- 172.21.15.118:0/2006813029 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffba415cc10 con 0x7ffba406e5a0 2023-11-24T06:10:08.961 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:08.957+0000 7ffb92ffd640 1 -- 172.21.15.118:0/2006813029 <== mon.2 v2:172.21.15.118:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7ffb94043c10 con 0x7ffba406e5a0 2023-11-24T06:10:08.961 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:08.957+0000 7ffb92ffd640 1 -- 172.21.15.118:0/2006813029 <== mon.2 v2:172.21.15.118:3301/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1530+0+0 (secure 0 0 0) 0x7ffb9403e070 con 0x7ffba406e5a0 2023-11-24T06:10:08.961 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:08.958+0000 7ffba359e640 1 -- 172.21.15.118:0/2006813029 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffba4067b60 con 0x7ffba406e5a0 2023-11-24T06:10:08.962 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:08.960+0000 7ffb92ffd640 1 -- 172.21.15.118:0/2006813029 <== mon.2 v2:172.21.15.118:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7ffb94034070 con 0x7ffba406e5a0 2023-11-24T06:10:09.041 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:09.039+0000 7ffba359e640 1 -- 172.21.15.118:0/2006813029 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_command({"prefix": "osd new", "uuid": "4ed4c624-c88f-4d35-bdb1-bf209b5dffb1", "id": 7} v 0) v1 -- 0x7ffba406b8b0 con 0x7ffba406e5a0 2023-11-24T06:10:09.046 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:09.044+0000 7ffb92ffd640 1 -- 172.21.15.118:0/2006813029 <== mon.2 v2:172.21.15.118:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "4ed4c624-c88f-4d35-bdb1-bf209b5dffb1", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7ffb9405c0b0 con 0x7ffba406e5a0 2023-11-24T06:10:09.047 INFO:teuthology.orchestra.run.smithi118.stdout:7 2023-11-24T06:10:09.047 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:09.045+0000 7ffba359e640 1 -- 172.21.15.118:0/2006813029 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ffba406e5a0 msgr2=0x7ffba4145ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:09.048 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:09.045+0000 7ffba359e640 1 --2- 172.21.15.118:0/2006813029 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ffba406e5a0 0x7ffba4145ce0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7ffb9402ba00 tx=0x7ffb94004730 comp rx=0 tx=0).stop 2023-11-24T06:10:09.048 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:09.045+0000 7ffba359e640 1 -- 172.21.15.118:0/2006813029 shutdown_connections 2023-11-24T06:10:09.048 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:09.045+0000 7ffba359e640 1 --2- 172.21.15.118:0/2006813029 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ffba4160770 0x7ffba415a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:09.048 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:09.045+0000 7ffba359e640 1 --2- 172.21.15.118:0/2006813029 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffba40714a0 0x7ffba415c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:09.048 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:09.045+0000 7ffba359e640 1 --2- 172.21.15.118:0/2006813029 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ffba406e5a0 0x7ffba4145ce0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:09.048 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:09.045+0000 7ffba359e640 1 -- 172.21.15.118:0/2006813029 >> 172.21.15.118:0/2006813029 conn(0x7ffba40b15b0 msgr2=0x7ffba415f5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:09.048 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:09.045+0000 7ffba359e640 1 -- 172.21.15.118:0/2006813029 shutdown_connections 2023-11-24T06:10:09.048 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:09.045+0000 7ffba359e640 1 -- 172.21.15.118:0/2006813029 wait complete. 2023-11-24T06:10:09.064 INFO:tasks.ceph.osd.0:Restarting daemon 2023-11-24T06:10:09.064 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2023-11-24T06:10:09.072 INFO:tasks.ceph.osd.0:Started 2023-11-24T06:10:09.072 INFO:tasks.ceph.osd.1:Restarting daemon 2023-11-24T06:10:09.073 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2023-11-24T06:10:09.081 INFO:tasks.ceph.osd.1:Started 2023-11-24T06:10:09.081 INFO:tasks.ceph.osd.2:Restarting daemon 2023-11-24T06:10:09.081 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2023-11-24T06:10:09.082 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-11-24T06:10:09.078+0000 7ff6ac62e040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-24T06:10:09.089 INFO:tasks.ceph.osd.2:Started 2023-11-24T06:10:09.090 INFO:tasks.ceph.osd.3:Restarting daemon 2023-11-24T06:10:09.090 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2023-11-24T06:10:09.095 INFO:tasks.ceph.osd.3:Started 2023-11-24T06:10:09.096 INFO:tasks.ceph.osd.4:Restarting daemon 2023-11-24T06:10:09.096 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2023-11-24T06:10:09.104 INFO:tasks.ceph.osd.4:Started 2023-11-24T06:10:09.104 INFO:tasks.ceph.osd.5:Restarting daemon 2023-11-24T06:10:09.104 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2023-11-24T06:10:09.112 INFO:tasks.ceph.osd.5:Started 2023-11-24T06:10:09.112 INFO:tasks.ceph.osd.6:Restarting daemon 2023-11-24T06:10:09.112 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2023-11-24T06:10:09.118 INFO:tasks.ceph.osd.6:Started 2023-11-24T06:10:09.118 INFO:tasks.ceph.osd.7:Restarting daemon 2023-11-24T06:10:09.118 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2023-11-24T06:10:09.123 INFO:tasks.ceph.osd.7:Started 2023-11-24T06:10:09.124 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:10:09.341 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:09.337+0000 7fd9bf23d640 1 Processor -- start 2023-11-24T06:10:09.341 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:09.338+0000 7fd9bf23d640 1 -- start start 2023-11-24T06:10:09.342 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:09.339+0000 7fd9bf23d640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd9b813df80 0x7fd9b815bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:09.342 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:09.339+0000 7fd9bf23d640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fd9b80cf9a0 0x7fd9b80cd840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:09.343 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:09.339+0000 7fd9bf23d640 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9b806c470 con 0x7fd9b813e360 2023-11-24T06:10:09.343 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:09.339+0000 7fd9bf23d640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9b806c750 con 0x7fd9b80cf9a0 2023-11-24T06:10:09.343 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:09.339+0000 7fd9bf23d640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9b806ca30 con 0x7fd9b813df80 2023-11-24T06:10:09.343 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:09.339+0000 7fd9bd7da640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fd9b80cf9a0 0x7fd9b80cd840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:09.343 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:09.339+0000 7fd9bd7da640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fd9b80cf9a0 0x7fd9b80cd840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:42480/0 (socket says 172.21.15.92:42480) 2023-11-24T06:10:09.343 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:09.339+0000 7fd9bcfd9640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd9b813df80 0x7fd9b815bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:09.343 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:09.339+0000 7fd9bd7da640 1 -- 172.21.15.92:0/1753321681 learned_addr learned my addr 172.21.15.92:0/1753321681 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:10:09.344 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:09.340+0000 7fd9bd7da640 1 -- 172.21.15.92:0/1753321681 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd9b813df80 msgr2=0x7fd9b815bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:09.344 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:09.340+0000 7fd9bd7da640 1 --2- 172.21.15.92:0/1753321681 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd9b813df80 0x7fd9b815bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:09.344 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:09.340+0000 7fd9bd7da640 1 -- 172.21.15.92:0/1753321681 >> v1:172.21.15.118:6789/0 conn(0x7fd9b813e360 legacy=0x7fd9b80cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:09.344 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:09.340+0000 7fd9bd7da640 1 -- 172.21.15.92:0/1753321681 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9b806cd10 con 0x7fd9b80cf9a0 2023-11-24T06:10:09.344 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:09.340+0000 7fd9bcfd9640 1 --2- 172.21.15.92:0/1753321681 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd9b813df80 0x7fd9b815bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:10:09.344 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:09.341+0000 7fd9bd7da640 1 --2- 172.21.15.92:0/1753321681 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd9b80cf9a0 0x7fd9b80cd840 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fd9b0009e50 tx=0x7fd9b002bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6c15fe747fbf812 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:09.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:09.341+0000 7fd9b77fe640 1 -- 172.21.15.92:0/1753321681 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9b002caf0 con 0x7fd9b80cf9a0 2023-11-24T06:10:09.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:09.341+0000 7fd9b77fe640 1 -- 172.21.15.92:0/1753321681 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9b002cc50 con 0x7fd9b80cf9a0 2023-11-24T06:10:09.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:09.341+0000 7fd9b77fe640 1 -- 172.21.15.92:0/1753321681 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9b00343f0 con 0x7fd9b80cf9a0 2023-11-24T06:10:09.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:09.342+0000 7fd9bf23d640 1 -- 172.21.15.92:0/1753321681 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd9b80cf9a0 msgr2=0x7fd9b80cd840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:09.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:09.342+0000 7fd9bf23d640 1 --2- 172.21.15.92:0/1753321681 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd9b80cf9a0 0x7fd9b80cd840 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fd9b0009e50 tx=0x7fd9b002bf50 comp rx=0 tx=0).stop 2023-11-24T06:10:09.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:09.342+0000 7fd9bf23d640 1 -- 172.21.15.92:0/1753321681 shutdown_connections 2023-11-24T06:10:09.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:09.342+0000 7fd9bf23d640 1 --2- 172.21.15.92:0/1753321681 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd9b80cf9a0 0x7fd9b80cd840 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:09.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:09.342+0000 7fd9bf23d640 1 --2- 172.21.15.92:0/1753321681 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd9b813df80 0x7fd9b815bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:09.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:09.342+0000 7fd9bf23d640 1 -- 172.21.15.92:0/1753321681 >> 172.21.15.92:0/1753321681 conn(0x7fd9b806a800 msgr2=0x7fd9b806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:09.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:09.342+0000 7fd9bf23d640 1 -- 172.21.15.92:0/1753321681 shutdown_connections 2023-11-24T06:10:09.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:09.342+0000 7fd9bf23d640 1 -- 172.21.15.92:0/1753321681 wait complete. 2023-11-24T06:10:09.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:09.344+0000 7fd9bf23d640 1 Processor -- start 2023-11-24T06:10:09.348 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:09.344+0000 7fd9bf23d640 1 -- start start 2023-11-24T06:10:09.349 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:09.344+0000 7fd9bf23d640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd9b80cf9a0 0x7fd9b8071ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:09.349 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:09.344+0000 7fd9bf23d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd9b813df80 0x7fd9b8151630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:09.349 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:09.345+0000 7fd9bf23d640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd9b813e360 0x7fd9b81ef330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:09.349 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:09.345+0000 7fd9bf23d640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9b8071a50 con 0x7fd9b813df80 2023-11-24T06:10:09.349 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:09.345+0000 7fd9bf23d640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9b81ef870 con 0x7fd9b80cf9a0 2023-11-24T06:10:09.349 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:09.345+0000 7fd9bf23d640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9b8070e40 con 0x7fd9b813e360 2023-11-24T06:10:09.349 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:09.345+0000 7fd9b7fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd9b813df80 0x7fd9b8151630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:09.349 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:09.345+0000 7fd9bcfd9640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd9b80cf9a0 0x7fd9b8071ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:09.349 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:09.345+0000 7fd9bcfd9640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd9b80cf9a0 0x7fd9b8071ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.92:51316/0 (socket says 172.21.15.92:51316) 2023-11-24T06:10:09.349 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:09.345+0000 7fd9bcfd9640 1 -- 172.21.15.92:0/894912927 learned_addr learned my addr 172.21.15.92:0/894912927 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:10:09.349 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:09.345+0000 7fd9bd7da640 1 --2- 172.21.15.92:0/894912927 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd9b813e360 0x7fd9b81ef330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:09.349 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:09.345+0000 7fd9bcfd9640 1 -- 172.21.15.92:0/894912927 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd9b813e360 msgr2=0x7fd9b81ef330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:09.349 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:09.345+0000 7fd9bcfd9640 1 --2- 172.21.15.92:0/894912927 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd9b813e360 0x7fd9b81ef330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:09.350 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:09.345+0000 7fd9bcfd9640 1 -- 172.21.15.92:0/894912927 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd9b813df80 msgr2=0x7fd9b8151630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:09.350 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:09.345+0000 7fd9bcfd9640 1 --2- 172.21.15.92:0/894912927 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd9b813df80 0x7fd9b8151630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:09.350 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:09.345+0000 7fd9bcfd9640 1 -- 172.21.15.92:0/894912927 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9ac00d8a0 con 0x7fd9b80cf9a0 2023-11-24T06:10:09.350 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:09.345+0000 7fd9b7fff640 1 --2- 172.21.15.92:0/894912927 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd9b813df80 0x7fd9b8151630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:10:09.350 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:09.346+0000 7fd9bcfd9640 1 --2- 172.21.15.92:0/894912927 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd9b80cf9a0 0x7fd9b8071ee0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fd9ac012fc0 tx=0x7fd9ac010490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:09.351 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:09.346+0000 7fd9b57fa640 1 -- 172.21.15.92:0/894912927 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9ac00dc00 con 0x7fd9b80cf9a0 2023-11-24T06:10:09.351 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:09.346+0000 7fd9b57fa640 1 -- 172.21.15.92:0/894912927 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9ac013040 con 0x7fd9b80cf9a0 2023-11-24T06:10:09.351 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:09.346+0000 7fd9bf23d640 1 -- 172.21.15.92:0/894912927 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9b0009b30 con 0x7fd9b80cf9a0 2023-11-24T06:10:09.351 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:09.346+0000 7fd9b57fa640 1 -- 172.21.15.92:0/894912927 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9ac00d2e0 con 0x7fd9b80cf9a0 2023-11-24T06:10:09.351 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:09.346+0000 7fd9bf23d640 1 -- 172.21.15.92:0/894912927 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9b8071400 con 0x7fd9b80cf9a0 2023-11-24T06:10:09.351 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:09.346+0000 7fd9b57fa640 1 -- 172.21.15.92:0/894912927 <== mon.1 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fd9ac005c90 con 0x7fd9b80cf9a0 2023-11-24T06:10:09.351 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:09.347+0000 7fd9b57fa640 1 -- 172.21.15.92:0/894912927 <== mon.1 v2:172.21.15.118:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7fd9ac018480 con 0x7fd9b80cf9a0 2023-11-24T06:10:09.351 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:09.347+0000 7fd9bf23d640 1 -- 172.21.15.92:0/894912927 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9b80675d0 con 0x7fd9b80cf9a0 2023-11-24T06:10:09.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:09.350+0000 7fd9b57fa640 1 -- 172.21.15.92:0/894912927 <== mon.1 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7fd9ac005f40 con 0x7fd9b80cf9a0 2023-11-24T06:10:09.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:09.413+0000 7fd9bf23d640 1 -- 172.21.15.92:0/894912927 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd9b806df00 con 0x7fd9b80cf9a0 2023-11-24T06:10:09.417 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:09.414+0000 7fd9b57fa640 1 -- 172.21.15.92:0/894912927 <== mon.1 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7fd9ac0060e0 con 0x7fd9b80cf9a0 2023-11-24T06:10:09.418 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-24T06:10:09.418 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":10,"fsid":"2d6dee5f-8aa3-470f-931a-50ade963b1cb","created":"2023-11-24T06:10:04.707204+0000","modified":"2023-11-24T06:10:09.040793+0000","last_up_change":"0.000000","last_in_change":"2023-11-24T06:10:09.040793+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"ff37a9fb-135f-471f-b4b0-f7b5772006c7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"03cdd353-dcf1-4f38-94fe-a795137c06b1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"a4a20d80-5d57-484b-becd-9aa8070dd3aa","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"4648d7ce-9b29-4583-9425-4ba8fd2b1903","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"3bc9d7eb-c6fa-4fcf-a3ec-ade604e05aac","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"dce9ba6d-59ed-46d1-84fc-a84b13de10c5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"82aec504-ae58-4709-8d30-46f2b12081b8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"4ed4c624-c88f-4d35-bdb1-bf209b5dffb1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:10:09.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:09.415+0000 7fd9bf23d640 1 -- 172.21.15.92:0/894912927 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd9b80cf9a0 msgr2=0x7fd9b8071ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:09.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:09.415+0000 7fd9bf23d640 1 --2- 172.21.15.92:0/894912927 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd9b80cf9a0 0x7fd9b8071ee0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fd9ac012fc0 tx=0x7fd9ac010490 comp rx=0 tx=0).stop 2023-11-24T06:10:09.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:09.415+0000 7fd9bf23d640 1 -- 172.21.15.92:0/894912927 shutdown_connections 2023-11-24T06:10:09.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:09.415+0000 7fd9bf23d640 1 --2- 172.21.15.92:0/894912927 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd9b813e360 0x7fd9b81ef330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:09.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:09.415+0000 7fd9bf23d640 1 --2- 172.21.15.92:0/894912927 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd9b813df80 0x7fd9b8151630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:09.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:09.415+0000 7fd9bf23d640 1 --2- 172.21.15.92:0/894912927 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd9b80cf9a0 0x7fd9b8071ee0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:09.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:09.415+0000 7fd9bf23d640 1 -- 172.21.15.92:0/894912927 >> 172.21.15.92:0/894912927 conn(0x7fd9b806a800 msgr2=0x7fd9b8155350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:09.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:09.415+0000 7fd9bf23d640 1 -- 172.21.15.92:0/894912927 shutdown_connections 2023-11-24T06:10:09.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:09.415+0000 7fd9bf23d640 1 -- 172.21.15.92:0/894912927 wait complete. 2023-11-24T06:10:09.435 INFO:tasks.ceph.ceph_manager.ceph:[] 2023-11-24T06:10:09.435 INFO:tasks.ceph:Waiting for OSDs to come up 2023-11-24T06:10:09.472 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-11-24T06:10:09.468+0000 7ff6ac62e040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-24T06:10:09.532 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-11-24T06:10:09.529+0000 7ff6ac62e040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-24T06:10:09.648 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-11-24T06:10:09.645+0000 7ff6ac62e040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-24T06:10:09.776 INFO:tasks.ceph.osd.0.smithi092.stderr:2023-11-24T06:10:09.773+0000 7f6927216500 -1 Falling back to public interface 2023-11-24T06:10:09.796 INFO:tasks.ceph.osd.2.smithi092.stderr:2023-11-24T06:10:09.793+0000 7f69b6e22500 -1 Falling back to public interface 2023-11-24T06:10:09.796 INFO:tasks.ceph.osd.5.smithi118.stderr:2023-11-24T06:10:09.793+0000 7f5e1a421500 -1 Falling back to public interface 2023-11-24T06:10:09.803 INFO:tasks.ceph.osd.6.smithi118.stderr:2023-11-24T06:10:09.801+0000 7f0f2bac0500 -1 Falling back to public interface 2023-11-24T06:10:09.805 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-24T06:10:09.802+0000 7f0e35011500 -1 Falling back to public interface 2023-11-24T06:10:09.836 INFO:tasks.ceph.osd.4.smithi118.stderr:2023-11-24T06:10:09.834+0000 7fecb2402500 -1 Falling back to public interface 2023-11-24T06:10:09.837 INFO:tasks.ceph.osd.7.smithi118.stderr:2023-11-24T06:10:09.834+0000 7fd0fc10e500 -1 Falling back to public interface 2023-11-24T06:10:09.839 INFO:tasks.ceph.osd.3.smithi092.stderr:2023-11-24T06:10:09.836+0000 7f5b308d8500 -1 Falling back to public interface 2023-11-24T06:10:09.986 INFO:tasks.ceph.mgr.x.smithi118.stderr:2023-11-24T06:10:09.983+0000 7f538f62c040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-24T06:10:10.240 DEBUG:teuthology.orchestra.run.smithi092:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-11-24T06:10:10.310 INFO:tasks.ceph.mgr.x.smithi118.stderr:2023-11-24T06:10:10.308+0000 7f538f62c040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-24T06:10:10.370 INFO:tasks.ceph.mgr.x.smithi118.stderr:2023-11-24T06:10:10.368+0000 7f538f62c040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-24T06:10:10.421 INFO:teuthology.misc.health.smithi092.stderr:2023-11-24T06:10:10.418+0000 7f258af01640 1 Processor -- start 2023-11-24T06:10:10.421 INFO:teuthology.misc.health.smithi092.stderr:2023-11-24T06:10:10.418+0000 7f258af01640 1 -- start start 2023-11-24T06:10:10.423 INFO:teuthology.misc.health.smithi092.stderr:2023-11-24T06:10:10.419+0000 7f258af01640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f258415d550 0x7f25840d11d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:10.423 INFO:teuthology.misc.health.smithi092.stderr:2023-11-24T06:10:10.419+0000 7f258af01640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f25840d38e0 0x7f25840b2fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:10.423 INFO:teuthology.misc.health.smithi092.stderr:2023-11-24T06:10:10.419+0000 7f258af01640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25840ce0a0 con 0x7f25840b2640 2023-11-24T06:10:10.423 INFO:teuthology.misc.health.smithi092.stderr:2023-11-24T06:10:10.419+0000 7f258af01640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25840ce340 con 0x7f258415d550 2023-11-24T06:10:10.423 INFO:teuthology.misc.health.smithi092.stderr:2023-11-24T06:10:10.419+0000 7f258af01640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25840ce620 con 0x7f25840d38e0 2023-11-24T06:10:10.423 INFO:teuthology.misc.health.smithi092.stderr:2023-11-24T06:10:10.419+0000 7f2588c9d640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f258415d550 0x7f25840d11d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:10.423 INFO:teuthology.misc.health.smithi092.stderr:2023-11-24T06:10:10.419+0000 7f2583fff640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f25840b2640 0x7f25840d31d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:35286/0 (socket says 172.21.15.92:35286) 2023-11-24T06:10:10.424 INFO:teuthology.misc.health.smithi092.stderr:2023-11-24T06:10:10.419+0000 7f2583fff640 1 -- 172.21.15.92:0/3005224890 learned_addr learned my addr 172.21.15.92:0/3005224890 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-24T06:10:10.424 INFO:teuthology.misc.health.smithi092.stderr:2023-11-24T06:10:10.419+0000 7f258949e640 1 --2- 172.21.15.92:0/3005224890 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f25840d38e0 0x7f25840b2fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:10.424 INFO:teuthology.misc.health.smithi092.stderr:2023-11-24T06:10:10.420+0000 7f25837fe640 1 -- 172.21.15.92:0/3005224890 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1936205236 0 0) 0x7f25840ce0a0 con 0x7f25840b2640 2023-11-24T06:10:10.424 INFO:teuthology.misc.health.smithi092.stderr:2023-11-24T06:10:10.420+0000 7f25837fe640 1 -- 172.21.15.92:0/3005224890 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2570003520 con 0x7f25840b2640 2023-11-24T06:10:10.424 INFO:teuthology.misc.health.smithi092.stderr:2023-11-24T06:10:10.420+0000 7f2588c9d640 1 -- 172.21.15.92:0/3005224890 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f25840d38e0 msgr2=0x7f25840b2fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:10.424 INFO:teuthology.misc.health.smithi092.stderr:2023-11-24T06:10:10.420+0000 7f2588c9d640 1 --2- 172.21.15.92:0/3005224890 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f25840d38e0 0x7f25840b2fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:10.424 INFO:teuthology.misc.health.smithi092.stderr:2023-11-24T06:10:10.420+0000 7f2588c9d640 1 -- 172.21.15.92:0/3005224890 >> v1:172.21.15.92:6789/0 conn(0x7f25840b2640 legacy=0x7f25840d31d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:10.424 INFO:teuthology.misc.health.smithi092.stderr:2023-11-24T06:10:10.420+0000 7f2588c9d640 1 -- 172.21.15.92:0/3005224890 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25840ce900 con 0x7f258415d550 2023-11-24T06:10:10.425 INFO:teuthology.misc.health.smithi092.stderr:2023-11-24T06:10:10.421+0000 7f2588c9d640 1 --2- 172.21.15.92:0/3005224890 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f258415d550 0x7f25840d11d0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f2574009d90 tx=0x7f257402bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d644e02b2ec30cc server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:10.425 INFO:teuthology.misc.health.smithi092.stderr:2023-11-24T06:10:10.421+0000 7f25837fe640 1 -- 172.21.15.92:0/3005224890 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f257402cc50 con 0x7f258415d550 2023-11-24T06:10:10.425 INFO:teuthology.misc.health.smithi092.stderr:2023-11-24T06:10:10.421+0000 7f25837fe640 1 -- 172.21.15.92:0/3005224890 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f257402cdb0 con 0x7f258415d550 2023-11-24T06:10:10.426 INFO:teuthology.misc.health.smithi092.stderr:2023-11-24T06:10:10.421+0000 7f25837fe640 1 -- 172.21.15.92:0/3005224890 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2574034660 con 0x7f258415d550 2023-11-24T06:10:10.426 INFO:teuthology.misc.health.smithi092.stderr:2023-11-24T06:10:10.421+0000 7f258af01640 1 -- 172.21.15.92:0/3005224890 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f258415d550 msgr2=0x7f25840d11d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:10.426 INFO:teuthology.misc.health.smithi092.stderr:2023-11-24T06:10:10.421+0000 7f258af01640 1 --2- 172.21.15.92:0/3005224890 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f258415d550 0x7f25840d11d0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f2574009d90 tx=0x7f257402bea0 comp rx=0 tx=0).stop 2023-11-24T06:10:10.426 INFO:teuthology.misc.health.smithi092.stderr:2023-11-24T06:10:10.422+0000 7f258af01640 1 -- 172.21.15.92:0/3005224890 shutdown_connections 2023-11-24T06:10:10.426 INFO:teuthology.misc.health.smithi092.stderr:2023-11-24T06:10:10.422+0000 7f258af01640 1 --2- 172.21.15.92:0/3005224890 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f25840d38e0 0x7f25840b2fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:10.426 INFO:teuthology.misc.health.smithi092.stderr:2023-11-24T06:10:10.422+0000 7f258af01640 1 --2- 172.21.15.92:0/3005224890 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f258415d550 0x7f25840d11d0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:10.426 INFO:teuthology.misc.health.smithi092.stderr:2023-11-24T06:10:10.422+0000 7f258af01640 1 -- 172.21.15.92:0/3005224890 >> 172.21.15.92:0/3005224890 conn(0x7f258408cc50 msgr2=0x7f258409c570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:10.426 INFO:teuthology.misc.health.smithi092.stderr:2023-11-24T06:10:10.422+0000 7f258af01640 1 -- 172.21.15.92:0/3005224890 shutdown_connections 2023-11-24T06:10:10.426 INFO:teuthology.misc.health.smithi092.stderr:2023-11-24T06:10:10.422+0000 7f258af01640 1 -- 172.21.15.92:0/3005224890 wait complete. 2023-11-24T06:10:10.427 INFO:teuthology.misc.health.smithi092.stderr:2023-11-24T06:10:10.423+0000 7f258af01640 1 Processor -- start 2023-11-24T06:10:10.427 INFO:teuthology.misc.health.smithi092.stderr:2023-11-24T06:10:10.423+0000 7f258af01640 1 -- start start 2023-11-24T06:10:10.428 INFO:teuthology.misc.health.smithi092.stderr:2023-11-24T06:10:10.424+0000 7f258af01640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f25840b2640 0x7f25841605c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:10.428 INFO:teuthology.misc.health.smithi092.stderr:2023-11-24T06:10:10.424+0000 7f258af01640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f25840d38e0 0x7f25841f2610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:10.428 INFO:teuthology.misc.health.smithi092.stderr:2023-11-24T06:10:10.424+0000 7f258af01640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f258415d550 0x7f25840c9530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:10.429 INFO:teuthology.misc.health.smithi092.stderr:2023-11-24T06:10:10.424+0000 7f258af01640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25841f2b50 con 0x7f25840b2640 2023-11-24T06:10:10.429 INFO:teuthology.misc.health.smithi092.stderr:2023-11-24T06:10:10.424+0000 7f258af01640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f258414a120 con 0x7f25840d38e0 2023-11-24T06:10:10.429 INFO:teuthology.misc.health.smithi092.stderr:2023-11-24T06:10:10.424+0000 7f258af01640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f258415e230 con 0x7f258415d550 2023-11-24T06:10:10.429 INFO:teuthology.misc.health.smithi092.stderr:2023-11-24T06:10:10.425+0000 7f2588c9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f25840b2640 0x7f25841605c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:10.429 INFO:teuthology.misc.health.smithi092.stderr:2023-11-24T06:10:10.425+0000 7f2588c9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f25840b2640 0x7f25841605c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:42556/0 (socket says 172.21.15.92:42556) 2023-11-24T06:10:10.429 INFO:teuthology.misc.health.smithi092.stderr:2023-11-24T06:10:10.425+0000 7f2588c9d640 1 -- 172.21.15.92:0/2405005074 learned_addr learned my addr 172.21.15.92:0/2405005074 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:10:10.429 INFO:teuthology.misc.health.smithi092.stderr:2023-11-24T06:10:10.425+0000 7f258949e640 1 --2- 172.21.15.92:0/2405005074 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f258415d550 0x7f25840c9530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:10.429 INFO:teuthology.misc.health.smithi092.stderr:2023-11-24T06:10:10.425+0000 7f2583fff640 1 --2- 172.21.15.92:0/2405005074 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f25840d38e0 0x7f25841f2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:10.429 INFO:teuthology.misc.health.smithi092.stderr:2023-11-24T06:10:10.425+0000 7f2588c9d640 1 -- 172.21.15.92:0/2405005074 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f258415d550 msgr2=0x7f25840c9530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:10.429 INFO:teuthology.misc.health.smithi092.stderr:2023-11-24T06:10:10.425+0000 7f2588c9d640 1 --2- 172.21.15.92:0/2405005074 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f258415d550 0x7f25840c9530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:10.429 INFO:teuthology.misc.health.smithi092.stderr:2023-11-24T06:10:10.425+0000 7f2588c9d640 1 -- 172.21.15.92:0/2405005074 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f25840d38e0 msgr2=0x7f25841f2610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:10.430 INFO:teuthology.misc.health.smithi092.stderr:2023-11-24T06:10:10.425+0000 7f2588c9d640 1 --2- 172.21.15.92:0/2405005074 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f25840d38e0 0x7f25841f2610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:10.430 INFO:teuthology.misc.health.smithi092.stderr:2023-11-24T06:10:10.425+0000 7f2588c9d640 1 -- 172.21.15.92:0/2405005074 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2574009a70 con 0x7f25840b2640 2023-11-24T06:10:10.430 INFO:teuthology.misc.health.smithi092.stderr:2023-11-24T06:10:10.425+0000 7f2588c9d640 1 --2- 172.21.15.92:0/2405005074 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f25840b2640 0x7f25841605c0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f2574009d90 tx=0x7f2574004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:10.431 INFO:teuthology.misc.health.smithi092.stderr:2023-11-24T06:10:10.426+0000 7f25817fa640 1 -- 172.21.15.92:0/2405005074 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2574040650 con 0x7f25840b2640 2023-11-24T06:10:10.431 INFO:teuthology.misc.health.smithi092.stderr:2023-11-24T06:10:10.426+0000 7f25817fa640 1 -- 172.21.15.92:0/2405005074 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25740407b0 con 0x7f25840b2640 2023-11-24T06:10:10.431 INFO:teuthology.misc.health.smithi092.stderr:2023-11-24T06:10:10.426+0000 7f258af01640 1 -- 172.21.15.92:0/2405005074 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25840c6f10 con 0x7f25840b2640 2023-11-24T06:10:10.431 INFO:teuthology.misc.health.smithi092.stderr:2023-11-24T06:10:10.426+0000 7f25817fa640 1 -- 172.21.15.92:0/2405005074 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2574040b60 con 0x7f25840b2640 2023-11-24T06:10:10.431 INFO:teuthology.misc.health.smithi092.stderr:2023-11-24T06:10:10.426+0000 7f258af01640 1 -- 172.21.15.92:0/2405005074 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f258415df50 con 0x7f25840b2640 2023-11-24T06:10:10.431 INFO:teuthology.misc.health.smithi092.stderr:2023-11-24T06:10:10.427+0000 7f25817fa640 1 -- 172.21.15.92:0/2405005074 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 2) v1 ==== 44567+0+0 (secure 0 0 0) 0x7f2574040cc0 con 0x7f25840b2640 2023-11-24T06:10:10.431 INFO:teuthology.misc.health.smithi092.stderr:2023-11-24T06:10:10.427+0000 7f258af01640 1 -- 172.21.15.92:0/2405005074 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25840675d0 con 0x7f25840b2640 2023-11-24T06:10:10.431 INFO:teuthology.misc.health.smithi092.stderr:2023-11-24T06:10:10.427+0000 7f25817fa640 1 -- 172.21.15.92:0/2405005074 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f257406c380 con 0x7f25840b2640 2023-11-24T06:10:10.432 INFO:teuthology.misc.health.smithi092.stderr:2023-11-24T06:10:10.429+0000 7f25817fa640 1 -- 172.21.15.92:0/2405005074 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7f2574009450 con 0x7f25840b2640 2023-11-24T06:10:10.486 INFO:tasks.ceph.mgr.x.smithi118.stderr:2023-11-24T06:10:10.483+0000 7f538f62c040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-24T06:10:10.494 INFO:teuthology.misc.health.smithi092.stderr:2023-11-24T06:10:10.490+0000 7f258af01640 1 -- 172.21.15.92:0/2405005074 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f258409c7e0 con 0x7f25840b2640 2023-11-24T06:10:10.495 INFO:teuthology.misc.health.smithi092.stderr:2023-11-24T06:10:10.492+0000 7f25817fa640 1 -- 172.21.15.92:0/2405005074 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7f2574039070 con 0x7f25840b2640 2023-11-24T06:10:10.495 INFO:teuthology.misc.health.smithi092.stdout: 2023-11-24T06:10:10.495 INFO:teuthology.misc.health.smithi092.stdout:{"epoch":10,"fsid":"2d6dee5f-8aa3-470f-931a-50ade963b1cb","created":"2023-11-24T06:10:04.707204+0000","modified":"2023-11-24T06:10:09.040793+0000","last_up_change":"0.000000","last_in_change":"2023-11-24T06:10:09.040793+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"ff37a9fb-135f-471f-b4b0-f7b5772006c7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"03cdd353-dcf1-4f38-94fe-a795137c06b1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"a4a20d80-5d57-484b-becd-9aa8070dd3aa","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"4648d7ce-9b29-4583-9425-4ba8fd2b1903","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"3bc9d7eb-c6fa-4fcf-a3ec-ade604e05aac","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"dce9ba6d-59ed-46d1-84fc-a84b13de10c5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"82aec504-ae58-4709-8d30-46f2b12081b8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"4ed4c624-c88f-4d35-bdb1-bf209b5dffb1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:10:10.496 INFO:teuthology.misc.health.smithi092.stderr:2023-11-24T06:10:10.493+0000 7f258af01640 1 -- 172.21.15.92:0/2405005074 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f25840b2640 msgr2=0x7f25841605c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:10.496 INFO:teuthology.misc.health.smithi092.stderr:2023-11-24T06:10:10.493+0000 7f258af01640 1 --2- 172.21.15.92:0/2405005074 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f25840b2640 0x7f25841605c0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f2574009d90 tx=0x7f2574004a40 comp rx=0 tx=0).stop 2023-11-24T06:10:10.496 INFO:teuthology.misc.health.smithi092.stderr:2023-11-24T06:10:10.493+0000 7f258af01640 1 -- 172.21.15.92:0/2405005074 shutdown_connections 2023-11-24T06:10:10.496 INFO:teuthology.misc.health.smithi092.stderr:2023-11-24T06:10:10.493+0000 7f258af01640 1 --2- 172.21.15.92:0/2405005074 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f258415d550 0x7f25840c9530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:10.496 INFO:teuthology.misc.health.smithi092.stderr:2023-11-24T06:10:10.493+0000 7f258af01640 1 --2- 172.21.15.92:0/2405005074 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f25840d38e0 0x7f25841f2610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:10.497 INFO:teuthology.misc.health.smithi092.stderr:2023-11-24T06:10:10.493+0000 7f258af01640 1 --2- 172.21.15.92:0/2405005074 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f25840b2640 0x7f25841605c0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:10.497 INFO:teuthology.misc.health.smithi092.stderr:2023-11-24T06:10:10.493+0000 7f258af01640 1 -- 172.21.15.92:0/2405005074 >> 172.21.15.92:0/2405005074 conn(0x7f258408cc50 msgr2=0x7f258415c840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:10.497 INFO:teuthology.misc.health.smithi092.stderr:2023-11-24T06:10:10.493+0000 7f258af01640 1 -- 172.21.15.92:0/2405005074 shutdown_connections 2023-11-24T06:10:10.497 INFO:teuthology.misc.health.smithi092.stderr:2023-11-24T06:10:10.493+0000 7f258af01640 1 -- 172.21.15.92:0/2405005074 wait complete. 2023-11-24T06:10:10.508 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-11-24T06:10:10.678 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-11-24T06:10:10.674+0000 7ff64d230640 -1 mgr.server handle_report got status from non-daemon mon.c 2023-11-24T06:10:10.683 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-11-24T06:10:10.680+0000 7ff64d230640 -1 mgr.server handle_report got status from non-daemon mon.b 2023-11-24T06:10:13.319 INFO:tasks.ceph.osd.2.smithi092.stderr:2023-11-24T06:10:13.317+0000 7f69b6e22500 -1 osd.2 0 log_to_monitors true 2023-11-24T06:10:13.355 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-24T06:10:13.353+0000 7f0e35011500 -1 osd.1 0 log_to_monitors true 2023-11-24T06:10:13.356 INFO:tasks.ceph.osd.5.smithi118.stderr:2023-11-24T06:10:13.354+0000 7f5e1a421500 -1 osd.5 0 log_to_monitors true 2023-11-24T06:10:13.377 INFO:tasks.ceph.osd.3.smithi092.stderr:2023-11-24T06:10:13.375+0000 7f5b308d8500 -1 osd.3 0 log_to_monitors true 2023-11-24T06:10:13.398 INFO:tasks.ceph.osd.7.smithi118.stderr:2023-11-24T06:10:13.396+0000 7fd0fc10e500 -1 osd.7 0 log_to_monitors true 2023-11-24T06:10:13.558 INFO:tasks.ceph.osd.0.smithi092.stderr:2023-11-24T06:10:13.556+0000 7f6927216500 -1 osd.0 0 log_to_monitors true 2023-11-24T06:10:13.615 INFO:tasks.ceph.osd.6.smithi118.stderr:2023-11-24T06:10:13.612+0000 7f0f2bac0500 -1 osd.6 0 log_to_monitors true 2023-11-24T06:10:13.667 INFO:tasks.ceph.osd.4.smithi118.stderr:2023-11-24T06:10:13.665+0000 7fecb2402500 -1 osd.4 0 log_to_monitors true 2023-11-24T06:10:16.127 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T06:10:16.133 DEBUG:teuthology.orchestra.run.smithi118:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T06:10:16.603 INFO:tasks.ceph.osd.6.smithi118.stderr:2023-11-24T06:10:16.601+0000 7f0f28a63640 -1 osd.6 0 waiting for initial osdmap 2023-11-24T06:10:16.611 INFO:tasks.ceph.osd.6.smithi118.stderr:2023-11-24T06:10:16.609+0000 7f0f1f864640 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-24T06:10:16.621 INFO:tasks.ceph.osd.5.smithi118.stderr:2023-11-24T06:10:16.619+0000 7f5e16e25640 -1 osd.5 0 waiting for initial osdmap 2023-11-24T06:10:16.625 INFO:tasks.ceph.osd.4.smithi118.stderr:2023-11-24T06:10:16.623+0000 7fecadb6d640 -1 osd.4 0 waiting for initial osdmap 2023-11-24T06:10:16.630 INFO:tasks.ceph.osd.5.smithi118.stderr:2023-11-24T06:10:16.628+0000 7f5e0dc26640 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-24T06:10:16.640 INFO:tasks.ceph.osd.4.smithi118.stderr:2023-11-24T06:10:16.638+0000 7feca61a6640 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-24T06:10:16.643 INFO:tasks.ceph.osd.7.smithi118.stderr:2023-11-24T06:10:16.641+0000 7fd0f7879640 -1 osd.7 0 waiting for initial osdmap 2023-11-24T06:10:16.653 INFO:tasks.ceph.osd.7.smithi118.stderr:2023-11-24T06:10:16.651+0000 7fd0efeb2640 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-24T06:10:17.184 INFO:tasks.ceph.osd.0.smithi092.stderr:2023-11-24T06:10:17.182+0000 7f692235b640 -1 osd.0 0 waiting for initial osdmap 2023-11-24T06:10:17.186 INFO:tasks.ceph.osd.2.smithi092.stderr:2023-11-24T06:10:17.184+0000 7f69b377a640 -1 osd.2 0 waiting for initial osdmap 2023-11-24T06:10:17.193 INFO:tasks.ceph.osd.2.smithi092.stderr:2023-11-24T06:10:17.191+0000 7f69aa57b640 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-24T06:10:17.203 INFO:tasks.ceph.osd.0.smithi092.stderr:2023-11-24T06:10:17.201+0000 7f691a994640 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-24T06:10:17.226 INFO:tasks.ceph.osd.3.smithi092.stderr:2023-11-24T06:10:17.224+0000 7f5b2c043640 -1 osd.3 0 waiting for initial osdmap 2023-11-24T06:10:17.232 INFO:tasks.ceph.osd.3.smithi092.stderr:2023-11-24T06:10:17.230+0000 7f5b2467c640 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-24T06:10:17.235 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-24T06:10:17.233+0000 7f0e30130640 -1 osd.1 0 waiting for initial osdmap 2023-11-24T06:10:17.244 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-24T06:10:17.242+0000 7f0e28769640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-24T06:10:17.314 DEBUG:teuthology.orchestra.run.smithi092:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-11-24T06:10:17.478 INFO:teuthology.misc.health.smithi092.stderr:2023-11-24T06:10:17.476+0000 7f91d711d640 1 Processor -- start 2023-11-24T06:10:17.478 INFO:teuthology.misc.health.smithi092.stderr:2023-11-24T06:10:17.476+0000 7f91d711d640 1 -- start start 2023-11-24T06:10:17.480 INFO:teuthology.misc.health.smithi092.stderr:2023-11-24T06:10:17.477+0000 7f91d711d640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f91d0144a00 0x7f91d00cc780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:17.480 INFO:teuthology.misc.health.smithi092.stderr:2023-11-24T06:10:17.477+0000 7f91d711d640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f91d00cccc0 0x7f91d006c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:17.480 INFO:teuthology.misc.health.smithi092.stderr:2023-11-24T06:10:17.477+0000 7f91d711d640 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91d0070b30 con 0x7f91d006c720 2023-11-24T06:10:17.480 INFO:teuthology.misc.health.smithi092.stderr:2023-11-24T06:10:17.477+0000 7f91d711d640 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91d006cbb0 con 0x7f91d0144a00 2023-11-24T06:10:17.480 INFO:teuthology.misc.health.smithi092.stderr:2023-11-24T06:10:17.477+0000 7f91d711d640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91d0070520 con 0x7f91d00cccc0 2023-11-24T06:10:17.481 INFO:teuthology.misc.health.smithi092.stderr:2023-11-24T06:10:17.478+0000 7f91cffff640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f91d00cccc0 0x7f91d006c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:17.481 INFO:teuthology.misc.health.smithi092.stderr:2023-11-24T06:10:17.478+0000 7f91d4eb9640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f91d0144a00 0x7f91d00cc780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:17.481 INFO:teuthology.misc.health.smithi092.stderr:2023-11-24T06:10:17.478+0000 7f91d4eb9640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f91d0144a00 0x7f91d00cc780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.92:51426/0 (socket says 172.21.15.92:51426) 2023-11-24T06:10:17.481 INFO:teuthology.misc.health.smithi092.stderr:2023-11-24T06:10:17.478+0000 7f91d4eb9640 1 -- 172.21.15.92:0/1824939843 learned_addr learned my addr 172.21.15.92:0/1824939843 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:10:17.481 INFO:teuthology.misc.health.smithi092.stderr:2023-11-24T06:10:17.479+0000 7f91d4eb9640 1 -- 172.21.15.92:0/1824939843 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f91d00cccc0 msgr2=0x7f91d006c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:17.481 INFO:teuthology.misc.health.smithi092.stderr:2023-11-24T06:10:17.479+0000 7f91d4eb9640 1 --2- 172.21.15.92:0/1824939843 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f91d00cccc0 0x7f91d006c1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:17.482 INFO:teuthology.misc.health.smithi092.stderr:2023-11-24T06:10:17.479+0000 7f91d4eb9640 1 -- 172.21.15.92:0/1824939843 >> v1:172.21.15.118:6789/0 conn(0x7f91d006c720 legacy=0x7f91d015ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:17.482 INFO:teuthology.misc.health.smithi092.stderr:2023-11-24T06:10:17.479+0000 7f91d4eb9640 1 -- 172.21.15.92:0/1824939843 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91d00cd470 con 0x7f91d0144a00 2023-11-24T06:10:17.482 INFO:teuthology.misc.health.smithi092.stderr:2023-11-24T06:10:17.480+0000 7f91d4eb9640 1 --2- 172.21.15.92:0/1824939843 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f91d0144a00 0x7f91d00cc780 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f91c0009e50 tx=0x7f91c002bf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5ca5e65cacd79384 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:17.483 INFO:teuthology.misc.health.smithi092.stderr:2023-11-24T06:10:17.481+0000 7f91cf7fe640 1 -- 172.21.15.92:0/1824939843 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91c002cb00 con 0x7f91d0144a00 2023-11-24T06:10:17.483 INFO:teuthology.misc.health.smithi092.stderr:2023-11-24T06:10:17.481+0000 7f91cf7fe640 1 -- 172.21.15.92:0/1824939843 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91c002cc60 con 0x7f91d0144a00 2023-11-24T06:10:17.483 INFO:teuthology.misc.health.smithi092.stderr:2023-11-24T06:10:17.481+0000 7f91cf7fe640 1 -- 172.21.15.92:0/1824939843 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91c00343f0 con 0x7f91d0144a00 2023-11-24T06:10:17.484 INFO:teuthology.misc.health.smithi092.stderr:2023-11-24T06:10:17.481+0000 7f91d711d640 1 -- 172.21.15.92:0/1824939843 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f91d0144a00 msgr2=0x7f91d00cc780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:17.484 INFO:teuthology.misc.health.smithi092.stderr:2023-11-24T06:10:17.481+0000 7f91d711d640 1 --2- 172.21.15.92:0/1824939843 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f91d0144a00 0x7f91d00cc780 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f91c0009e50 tx=0x7f91c002bf60 comp rx=0 tx=0).stop 2023-11-24T06:10:17.484 INFO:teuthology.misc.health.smithi092.stderr:2023-11-24T06:10:17.482+0000 7f91d711d640 1 -- 172.21.15.92:0/1824939843 shutdown_connections 2023-11-24T06:10:17.485 INFO:teuthology.misc.health.smithi092.stderr:2023-11-24T06:10:17.482+0000 7f91d711d640 1 --2- 172.21.15.92:0/1824939843 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f91d00cccc0 0x7f91d006c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:17.485 INFO:teuthology.misc.health.smithi092.stderr:2023-11-24T06:10:17.482+0000 7f91d711d640 1 --2- 172.21.15.92:0/1824939843 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f91d0144a00 0x7f91d00cc780 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:17.485 INFO:teuthology.misc.health.smithi092.stderr:2023-11-24T06:10:17.482+0000 7f91d711d640 1 -- 172.21.15.92:0/1824939843 >> 172.21.15.92:0/1824939843 conn(0x7f91d006dd50 msgr2=0x7f91d006a450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:17.485 INFO:teuthology.misc.health.smithi092.stderr:2023-11-24T06:10:17.482+0000 7f91d711d640 1 -- 172.21.15.92:0/1824939843 shutdown_connections 2023-11-24T06:10:17.485 INFO:teuthology.misc.health.smithi092.stderr:2023-11-24T06:10:17.482+0000 7f91d711d640 1 -- 172.21.15.92:0/1824939843 wait complete. 2023-11-24T06:10:17.485 INFO:teuthology.misc.health.smithi092.stderr:2023-11-24T06:10:17.483+0000 7f91d711d640 1 Processor -- start 2023-11-24T06:10:17.486 INFO:teuthology.misc.health.smithi092.stderr:2023-11-24T06:10:17.484+0000 7f91d711d640 1 -- start start 2023-11-24T06:10:17.488 INFO:teuthology.misc.health.smithi092.stderr:2023-11-24T06:10:17.484+0000 7f91d711d640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f91d006c720 0x7f91d00b2030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:17.488 INFO:teuthology.misc.health.smithi092.stderr:2023-11-24T06:10:17.484+0000 7f91d711d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f91d00cccc0 0x7f91d00b2570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:17.488 INFO:teuthology.misc.health.smithi092.stderr:2023-11-24T06:10:17.484+0000 7f91d711d640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f91d0144a00 0x7f91d0130230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:17.488 INFO:teuthology.misc.health.smithi092.stderr:2023-11-24T06:10:17.485+0000 7f91d711d640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91d0158bb0 con 0x7f91d00cccc0 2023-11-24T06:10:17.488 INFO:teuthology.misc.health.smithi092.stderr:2023-11-24T06:10:17.485+0000 7f91d711d640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91d0157fa0 con 0x7f91d006c720 2023-11-24T06:10:17.488 INFO:teuthology.misc.health.smithi092.stderr:2023-11-24T06:10:17.485+0000 7f91d711d640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91d0158280 con 0x7f91d0144a00 2023-11-24T06:10:17.489 INFO:teuthology.misc.health.smithi092.stderr:2023-11-24T06:10:17.485+0000 7f91cffff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f91d00cccc0 0x7f91d00b2570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:17.489 INFO:teuthology.misc.health.smithi092.stderr:2023-11-24T06:10:17.485+0000 7f91cffff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f91d00cccc0 0x7f91d00b2570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:42624/0 (socket says 172.21.15.92:42624) 2023-11-24T06:10:17.489 INFO:teuthology.misc.health.smithi092.stderr:2023-11-24T06:10:17.485+0000 7f91cffff640 1 -- 172.21.15.92:0/3222331717 learned_addr learned my addr 172.21.15.92:0/3222331717 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:10:17.489 INFO:teuthology.misc.health.smithi092.stderr:2023-11-24T06:10:17.485+0000 7f91d4eb9640 1 --2- 172.21.15.92:0/3222331717 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f91d006c720 0x7f91d00b2030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:17.489 INFO:teuthology.misc.health.smithi092.stderr:2023-11-24T06:10:17.485+0000 7f91cffff640 1 -- 172.21.15.92:0/3222331717 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f91d0144a00 msgr2=0x7f91d0130230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:17.489 INFO:teuthology.misc.health.smithi092.stderr:2023-11-24T06:10:17.485+0000 7f91d56ba640 1 --2- 172.21.15.92:0/3222331717 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f91d0144a00 0x7f91d0130230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:17.489 INFO:teuthology.misc.health.smithi092.stderr:2023-11-24T06:10:17.485+0000 7f91cffff640 1 --2- 172.21.15.92:0/3222331717 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f91d0144a00 0x7f91d0130230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:17.489 INFO:teuthology.misc.health.smithi092.stderr:2023-11-24T06:10:17.485+0000 7f91cffff640 1 -- 172.21.15.92:0/3222331717 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f91d006c720 msgr2=0x7f91d00b2030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:17.489 INFO:teuthology.misc.health.smithi092.stderr:2023-11-24T06:10:17.485+0000 7f91cffff640 1 --2- 172.21.15.92:0/3222331717 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f91d006c720 0x7f91d00b2030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:17.489 INFO:teuthology.misc.health.smithi092.stderr:2023-11-24T06:10:17.485+0000 7f91cffff640 1 -- 172.21.15.92:0/3222331717 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91c0009b30 con 0x7f91d00cccc0 2023-11-24T06:10:17.489 INFO:teuthology.misc.health.smithi092.stderr:2023-11-24T06:10:17.485+0000 7f91d4eb9640 1 --2- 172.21.15.92:0/3222331717 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f91d006c720 0x7f91d00b2030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:10:17.490 INFO:teuthology.misc.health.smithi092.stderr:2023-11-24T06:10:17.485+0000 7f91d56ba640 1 --2- 172.21.15.92:0/3222331717 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f91d0144a00 0x7f91d0130230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:10:17.490 INFO:teuthology.misc.health.smithi092.stderr:2023-11-24T06:10:17.486+0000 7f91cffff640 1 --2- 172.21.15.92:0/3222331717 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f91d00cccc0 0x7f91d00b2570 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f91c400e980 tx=0x7f91c400ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:17.490 INFO:teuthology.misc.health.smithi092.stderr:2023-11-24T06:10:17.486+0000 7f91cd7fa640 1 -- 172.21.15.92:0/3222331717 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91c4009670 con 0x7f91d00cccc0 2023-11-24T06:10:17.490 INFO:teuthology.misc.health.smithi092.stderr:2023-11-24T06:10:17.486+0000 7f91cd7fa640 1 -- 172.21.15.92:0/3222331717 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91c400ca00 con 0x7f91d00cccc0 2023-11-24T06:10:17.490 INFO:teuthology.misc.health.smithi092.stderr:2023-11-24T06:10:17.486+0000 7f91d711d640 1 -- 172.21.15.92:0/3222331717 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91d0158560 con 0x7f91d00cccc0 2023-11-24T06:10:17.490 INFO:teuthology.misc.health.smithi092.stderr:2023-11-24T06:10:17.486+0000 7f91cd7fa640 1 -- 172.21.15.92:0/3222331717 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91c400cd90 con 0x7f91d00cccc0 2023-11-24T06:10:17.490 INFO:teuthology.misc.health.smithi092.stderr:2023-11-24T06:10:17.486+0000 7f91d711d640 1 -- 172.21.15.92:0/3222331717 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91d0157c00 con 0x7f91d00cccc0 2023-11-24T06:10:17.491 INFO:teuthology.misc.health.smithi092.stderr:2023-11-24T06:10:17.487+0000 7f91d711d640 1 -- 172.21.15.92:0/3222331717 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91d0158d20 con 0x7f91d00cccc0 2023-11-24T06:10:17.491 INFO:teuthology.misc.health.smithi092.stderr:2023-11-24T06:10:17.488+0000 7f91cd7fa640 1 -- 172.21.15.92:0/3222331717 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 4) v1 ==== 89175+0+0 (secure 0 0 0) 0x7f91c4013470 con 0x7f91d00cccc0 2023-11-24T06:10:17.491 INFO:teuthology.misc.health.smithi092.stderr:2023-11-24T06:10:17.489+0000 7f91cd7fa640 1 --2- 172.21.15.92:0/3222331717 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f91bc06adb0 0x7f91bc06d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:17.491 INFO:teuthology.misc.health.smithi092.stderr:2023-11-24T06:10:17.489+0000 7f91cd7fa640 1 -- 172.21.15.92:0/3222331717 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(12..12 src has 1..12) v4 ==== 2231+0+0 (secure 0 0 0) 0x7f91c40888e0 con 0x7f91d00cccc0 2023-11-24T06:10:17.491 INFO:teuthology.misc.health.smithi092.stderr:2023-11-24T06:10:17.489+0000 7f91d4eb9640 1 --2- 172.21.15.92:0/3222331717 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f91bc06adb0 0x7f91bc06d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:17.492 INFO:teuthology.misc.health.smithi092.stderr:2023-11-24T06:10:17.490+0000 7f91d4eb9640 1 --2- 172.21.15.92:0/3222331717 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f91bc06adb0 0x7f91bc06d270 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f91c0035000 tx=0x7f91c0030000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:17.497 INFO:teuthology.misc.health.smithi092.stderr:2023-11-24T06:10:17.495+0000 7f91cd7fa640 1 -- 172.21.15.92:0/3222331717 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f91c4054890 con 0x7f91d00cccc0 2023-11-24T06:10:17.693 INFO:teuthology.misc.health.smithi092.stderr:2023-11-24T06:10:17.691+0000 7f91d711d640 1 -- 172.21.15.92:0/3222331717 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f91d00675d0 con 0x7f91d00cccc0 2023-11-24T06:10:17.736 INFO:teuthology.misc.health.smithi092.stderr:2023-11-24T06:10:17.733+0000 7f91cd7fa640 1 -- 172.21.15.92:0/3222331717 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v13) v1 ==== 74+0+10860 (secure 0 0 0) 0x7f91c4021090 con 0x7f91d00cccc0 2023-11-24T06:10:17.736 INFO:teuthology.misc.health.smithi092.stdout: 2023-11-24T06:10:17.736 INFO:teuthology.misc.health.smithi092.stdout:{"epoch":13,"fsid":"2d6dee5f-8aa3-470f-931a-50ade963b1cb","created":"2023-11-24T06:10:04.707204+0000","modified":"2023-11-24T06:10:17.598352+0000","last_up_change":"2023-11-24T06:10:17.598352+0000","last_in_change":"2023-11-24T06:10:09.040793+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"ff37a9fb-135f-471f-b4b0-f7b5772006c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6803","nonce":20560714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6805","nonce":20560714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6809","nonce":20560714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6807","nonce":20560714}]},"public_addr":"172.21.15.92:6803/20560714","cluster_addr":"172.21.15.92:6805/20560714","heartbeat_back_addr":"172.21.15.92:6809/20560714","heartbeat_front_addr":"172.21.15.92:6807/20560714","state":["exists","up"]},{"osd":1,"uuid":"03cdd353-dcf1-4f38-94fe-a795137c06b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6819","nonce":2451239906}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6821","nonce":2451239906}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6825","nonce":2451239906}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6823","nonce":2451239906}]},"public_addr":"172.21.15.92:6819/2451239906","cluster_addr":"172.21.15.92:6821/2451239906","heartbeat_back_addr":"172.21.15.92:6825/2451239906","heartbeat_front_addr":"172.21.15.92:6823/2451239906","state":["exists","up"]},{"osd":2,"uuid":"a4a20d80-5d57-484b-becd-9aa8070dd3aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6811","nonce":1124979277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6813","nonce":1124979277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6817","nonce":1124979277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6815","nonce":1124979277}]},"public_addr":"172.21.15.92:6811/1124979277","cluster_addr":"172.21.15.92:6813/1124979277","heartbeat_back_addr":"172.21.15.92:6817/1124979277","heartbeat_front_addr":"172.21.15.92:6815/1124979277","state":["exists","up"]},{"osd":3,"uuid":"4648d7ce-9b29-4583-9425-4ba8fd2b1903","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6827","nonce":1734115782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6829","nonce":1734115782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6833","nonce":1734115782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6831","nonce":1734115782}]},"public_addr":"172.21.15.92:6827/1734115782","cluster_addr":"172.21.15.92:6829/1734115782","heartbeat_back_addr":"172.21.15.92:6833/1734115782","heartbeat_front_addr":"172.21.15.92:6831/1734115782","state":["exists","up"]},{"osd":4,"uuid":"3bc9d7eb-c6fa-4fcf-a3ec-ade604e05aac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6819","nonce":3425937229}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6822","nonce":3425937229}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6830","nonce":3425937229}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6826","nonce":3425937229}]},"public_addr":"172.21.15.118:6819/3425937229","cluster_addr":"172.21.15.118:6822/3425937229","heartbeat_back_addr":"172.21.15.118:6830/3425937229","heartbeat_front_addr":"172.21.15.118:6826/3425937229","state":["exists","up"]},{"osd":5,"uuid":"dce9ba6d-59ed-46d1-84fc-a84b13de10c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6801","nonce":827455984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6803","nonce":827455984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6807","nonce":827455984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6805","nonce":827455984}]},"public_addr":"172.21.15.118:6801/827455984","cluster_addr":"172.21.15.118:6803/827455984","heartbeat_back_addr":"172.21.15.118:6807/827455984","heartbeat_front_addr":"172.21.15.118:6805/827455984","state":["exists","up"]},{"osd":6,"uuid":"82aec504-ae58-4709-8d30-46f2b12081b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6809","nonce":2946986141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6811","nonce":2946986141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6815","nonce":2946986141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6813","nonce":2946986141}]},"public_addr":"172.21.15.118:6809/2946986141","cluster_addr":"172.21.15.118:6811/2946986141","heartbeat_back_addr":"172.21.15.118:6815/2946986141","heartbeat_front_addr":"172.21.15.118:6813/2946986141","state":["exists","up"]},{"osd":7,"uuid":"4ed4c624-c88f-4d35-bdb1-bf209b5dffb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6817","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6818","nonce":48122059}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6821","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6823","nonce":48122059}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6829","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6831","nonce":48122059}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6825","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6827","nonce":48122059}]},"public_addr":"172.21.15.118:6818/48122059","cluster_addr":"172.21.15.118:6823/48122059","heartbeat_back_addr":"172.21.15.118:6831/48122059","heartbeat_front_addr":"172.21.15.118:6827/48122059","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:10:17.739 INFO:teuthology.misc.health.smithi092.stderr:2023-11-24T06:10:17.736+0000 7f91d711d640 1 -- 172.21.15.92:0/3222331717 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f91bc06adb0 msgr2=0x7f91bc06d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:17.739 INFO:teuthology.misc.health.smithi092.stderr:2023-11-24T06:10:17.736+0000 7f91d711d640 1 --2- 172.21.15.92:0/3222331717 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f91bc06adb0 0x7f91bc06d270 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f91c0035000 tx=0x7f91c0030000 comp rx=0 tx=0).stop 2023-11-24T06:10:17.739 INFO:teuthology.misc.health.smithi092.stderr:2023-11-24T06:10:17.736+0000 7f91d711d640 1 -- 172.21.15.92:0/3222331717 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f91d00cccc0 msgr2=0x7f91d00b2570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:17.739 INFO:teuthology.misc.health.smithi092.stderr:2023-11-24T06:10:17.736+0000 7f91d711d640 1 --2- 172.21.15.92:0/3222331717 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f91d00cccc0 0x7f91d00b2570 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f91c400e980 tx=0x7f91c400ee40 comp rx=0 tx=0).stop 2023-11-24T06:10:17.739 INFO:teuthology.misc.health.smithi092.stderr:2023-11-24T06:10:17.737+0000 7f91d711d640 1 -- 172.21.15.92:0/3222331717 shutdown_connections 2023-11-24T06:10:17.739 INFO:teuthology.misc.health.smithi092.stderr:2023-11-24T06:10:17.737+0000 7f91d711d640 1 --2- 172.21.15.92:0/3222331717 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f91bc06adb0 0x7f91bc06d270 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:17.740 INFO:teuthology.misc.health.smithi092.stderr:2023-11-24T06:10:17.737+0000 7f91d711d640 1 --2- 172.21.15.92:0/3222331717 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f91d0144a00 0x7f91d0130230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:17.740 INFO:teuthology.misc.health.smithi092.stderr:2023-11-24T06:10:17.737+0000 7f91d711d640 1 --2- 172.21.15.92:0/3222331717 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f91d00cccc0 0x7f91d00b2570 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:17.740 INFO:teuthology.misc.health.smithi092.stderr:2023-11-24T06:10:17.737+0000 7f91d711d640 1 --2- 172.21.15.92:0/3222331717 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f91d006c720 0x7f91d00b2030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:17.740 INFO:teuthology.misc.health.smithi092.stderr:2023-11-24T06:10:17.737+0000 7f91d711d640 1 -- 172.21.15.92:0/3222331717 >> 172.21.15.92:0/3222331717 conn(0x7f91d006dd50 msgr2=0x7f91d015ebf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:17.740 INFO:teuthology.misc.health.smithi092.stderr:2023-11-24T06:10:17.737+0000 7f91d711d640 1 -- 172.21.15.92:0/3222331717 shutdown_connections 2023-11-24T06:10:17.740 INFO:teuthology.misc.health.smithi092.stderr:2023-11-24T06:10:17.737+0000 7f91d711d640 1 -- 172.21.15.92:0/3222331717 wait complete. 2023-11-24T06:10:17.754 DEBUG:teuthology.misc:8 of 8 OSDs are up 2023-11-24T06:10:17.754 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2023-11-24T06:10:17.754 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-24T06:10:17.754 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-24T06:10:17.762 INFO:tasks.ceph.mds.a:Started 2023-11-24T06:10:17.762 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-24T06:10:17.762 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-24T06:10:17.770 INFO:tasks.ceph.mds.c:Started 2023-11-24T06:10:17.770 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-24T06:10:17.770 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-24T06:10:17.779 INFO:tasks.ceph.mds.b:Started 2023-11-24T06:10:17.779 INFO:tasks.ceph.mds.d:Restarting daemon 2023-11-24T06:10:17.780 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-11-24T06:10:17.789 INFO:tasks.ceph.mds.d:Started 2023-11-24T06:10:17.789 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2023-11-24T06:10:17.789 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:10:17.912 INFO:tasks.ceph.mds.c.smithi092.stdout:starting mds.c at 2023-11-24T06:10:17.913 INFO:tasks.ceph.mds.a.smithi092.stdout:starting mds.a at 2023-11-24T06:10:17.941 INFO:tasks.ceph.mds.d.smithi118.stdout:starting mds.d at 2023-11-24T06:10:17.944 INFO:tasks.ceph.mds.b.smithi118.stdout:starting mds.b at 2023-11-24T06:10:17.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:17.965+0000 7f7bc9fdc640 1 Processor -- start 2023-11-24T06:10:17.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:17.965+0000 7f7bc9fdc640 1 -- start start 2023-11-24T06:10:17.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:17.966+0000 7f7bc9fdc640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f7bc413df80 0x7f7bc415bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:17.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:17.966+0000 7f7bc9fdc640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7bc413e360 0x7f7bc40cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:17.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:17.966+0000 7f7bc9fdc640 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7bc406c470 con 0x7f7bc415f9c0 2023-11-24T06:10:17.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:17.966+0000 7f7bc9fdc640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7bc406c750 con 0x7f7bc413df80 2023-11-24T06:10:17.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:17.966+0000 7f7bc9fdc640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bc406ca30 con 0x7f7bc413e360 2023-11-24T06:10:17.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:17.966+0000 7f7bc37fe640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f7bc413df80 0x7f7bc415bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:17.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:17.966+0000 7f7bc37fe640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f7bc413df80 0x7f7bc415bee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:42664/0 (socket says 172.21.15.92:42664) 2023-11-24T06:10:17.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:17.966+0000 7f7bc37fe640 1 -- 172.21.15.92:0/3611369136 learned_addr learned my addr 172.21.15.92:0/3611369136 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:10:17.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:17.966+0000 7f7bbaffd640 1 --2- 172.21.15.92:0/3611369136 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7bc413e360 0x7f7bc40cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:17.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:17.967+0000 7f7bc37fe640 1 -- 172.21.15.92:0/3611369136 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7bc413e360 msgr2=0x7f7bc40cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:17.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:17.967+0000 7f7bc37fe640 1 --2- 172.21.15.92:0/3611369136 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7bc413e360 0x7f7bc40cf290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:17.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:17.967+0000 7f7bc37fe640 1 -- 172.21.15.92:0/3611369136 >> v1:172.21.15.118:6789/0 conn(0x7f7bc415f9c0 legacy=0x7f7bc41408d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:17.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:17.967+0000 7f7bc37fe640 1 -- 172.21.15.92:0/3611369136 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bc406cd10 con 0x7f7bc413df80 2023-11-24T06:10:17.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:17.967+0000 7f7bc37fe640 1 --2- 172.21.15.92:0/3611369136 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7bc413df80 0x7f7bc415bee0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f7bb4009e50 tx=0x7f7bb402bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbc83191d93f6bb9 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:17.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:17.968+0000 7f7bc2ffd640 1 -- 172.21.15.92:0/3611369136 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bb402cb00 con 0x7f7bc413df80 2023-11-24T06:10:17.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:17.968+0000 7f7bc2ffd640 1 -- 172.21.15.92:0/3611369136 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bb402cc60 con 0x7f7bc413df80 2023-11-24T06:10:17.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:17.968+0000 7f7bc2ffd640 1 -- 172.21.15.92:0/3611369136 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bb40343f0 con 0x7f7bc413df80 2023-11-24T06:10:17.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:17.968+0000 7f7bc9fdc640 1 -- 172.21.15.92:0/3611369136 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7bc413df80 msgr2=0x7f7bc415bee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:17.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:17.968+0000 7f7bc9fdc640 1 --2- 172.21.15.92:0/3611369136 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7bc413df80 0x7f7bc415bee0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f7bb4009e50 tx=0x7f7bb402bf60 comp rx=0 tx=0).stop 2023-11-24T06:10:17.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:17.968+0000 7f7bc9fdc640 1 -- 172.21.15.92:0/3611369136 shutdown_connections 2023-11-24T06:10:17.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:17.968+0000 7f7bc9fdc640 1 --2- 172.21.15.92:0/3611369136 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7bc413e360 0x7f7bc40cf290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:17.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:17.968+0000 7f7bc9fdc640 1 --2- 172.21.15.92:0/3611369136 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7bc413df80 0x7f7bc415bee0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:17.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:17.968+0000 7f7bc9fdc640 1 -- 172.21.15.92:0/3611369136 >> 172.21.15.92:0/3611369136 conn(0x7f7bc406a800 msgr2=0x7f7bc406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:17.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:17.968+0000 7f7bc9fdc640 1 -- 172.21.15.92:0/3611369136 shutdown_connections 2023-11-24T06:10:17.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:17.968+0000 7f7bc9fdc640 1 -- 172.21.15.92:0/3611369136 wait complete. 2023-11-24T06:10:17.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:17.970+0000 7f7bc9fdc640 1 Processor -- start 2023-11-24T06:10:17.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:17.970+0000 7f7bc9fdc640 1 -- start start 2023-11-24T06:10:17.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:17.971+0000 7f7bc9fdc640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7bc413df80 0x7f7bc415a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:17.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:17.971+0000 7f7bc9fdc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7bc413e360 0x7f7bc4152640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:17.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:17.971+0000 7f7bc9fdc640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7bc415f9c0 0x7f7bc40ad2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:17.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:17.971+0000 7f7bc9fdc640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bc40ae2b0 con 0x7f7bc413e360 2023-11-24T06:10:17.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:17.971+0000 7f7bc9fdc640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bc40ad7f0 con 0x7f7bc413df80 2023-11-24T06:10:17.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:17.971+0000 7f7bc9fdc640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bc40704f0 con 0x7f7bc415f9c0 2023-11-24T06:10:17.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:17.971+0000 7f7bbaffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7bc413e360 0x7f7bc4152640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:17.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:17.971+0000 7f7bbaffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7bc413e360 0x7f7bc4152640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:42666/0 (socket says 172.21.15.92:42666) 2023-11-24T06:10:17.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:17.971+0000 7f7bbaffd640 1 -- 172.21.15.92:0/2659441474 learned_addr learned my addr 172.21.15.92:0/2659441474 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:10:17.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:17.971+0000 7f7bc37fe640 1 --2- 172.21.15.92:0/2659441474 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7bc413df80 0x7f7bc415a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:17.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:17.971+0000 7f7bc3fff640 1 --2- 172.21.15.92:0/2659441474 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7bc415f9c0 0x7f7bc40ad2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:17.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:17.972+0000 7f7bbaffd640 1 -- 172.21.15.92:0/2659441474 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7bc415f9c0 msgr2=0x7f7bc40ad2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:17.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:17.972+0000 7f7bbaffd640 1 --2- 172.21.15.92:0/2659441474 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7bc415f9c0 0x7f7bc40ad2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:17.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:17.972+0000 7f7bbaffd640 1 -- 172.21.15.92:0/2659441474 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7bc413df80 msgr2=0x7f7bc415a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:17.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:17.972+0000 7f7bbaffd640 1 --2- 172.21.15.92:0/2659441474 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7bc413df80 0x7f7bc415a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:17.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:17.972+0000 7f7bbaffd640 1 -- 172.21.15.92:0/2659441474 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bb4009b30 con 0x7f7bc413e360 2023-11-24T06:10:17.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:17.972+0000 7f7bbaffd640 1 --2- 172.21.15.92:0/2659441474 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7bc413e360 0x7f7bc4152640 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f7bac00ea10 tx=0x7f7bac00eed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:17.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:17.972+0000 7f7bc0ff9640 1 -- 172.21.15.92:0/2659441474 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bac009670 con 0x7f7bc413e360 2023-11-24T06:10:17.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:17.972+0000 7f7bc0ff9640 1 -- 172.21.15.92:0/2659441474 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bac00ca00 con 0x7f7bc413e360 2023-11-24T06:10:17.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:17.972+0000 7f7bc9fdc640 1 -- 172.21.15.92:0/2659441474 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bc40ada10 con 0x7f7bc413e360 2023-11-24T06:10:17.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:17.972+0000 7f7bc0ff9640 1 -- 172.21.15.92:0/2659441474 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bac00cdb0 con 0x7f7bc413e360 2023-11-24T06:10:17.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:17.972+0000 7f7bc9fdc640 1 -- 172.21.15.92:0/2659441474 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bc40adfd0 con 0x7f7bc413e360 2023-11-24T06:10:17.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:17.973+0000 7f7bc9fdc640 1 -- 172.21.15.92:0/2659441474 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7bc40675d0 con 0x7f7bc413e360 2023-11-24T06:10:17.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:17.974+0000 7f7bc0ff9640 1 -- 172.21.15.92:0/2659441474 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 4) v1 ==== 89175+0+0 (secure 0 0 0) 0x7f7bac013470 con 0x7f7bc413e360 2023-11-24T06:10:17.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:17.975+0000 7f7bc0ff9640 1 --2- 172.21.15.92:0/2659441474 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f7ba406adb0 0x7f7ba406d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:17.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:17.975+0000 7f7bc0ff9640 1 -- 172.21.15.92:0/2659441474 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 4471+0+0 (secure 0 0 0) 0x7f7bac0891f0 con 0x7f7bc413e360 2023-11-24T06:10:17.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:17.976+0000 7f7bc37fe640 1 --2- 172.21.15.92:0/2659441474 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f7ba406adb0 0x7f7ba406d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:17.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:17.976+0000 7f7bc37fe640 1 --2- 172.21.15.92:0/2659441474 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f7ba406adb0 0x7f7ba406d270 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f7bb4035000 tx=0x7f7bb4030000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:17.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:17.980+0000 7f7bc0ff9640 1 -- 172.21.15.92:0/2659441474 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7bac054900 con 0x7f7bc413e360 2023-11-24T06:10:18.158 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:18.156+0000 7f7bc9fdc640 1 -- 172.21.15.92:0/2659441474 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7bc40702c0 con 0x7f7bc413e360 2023-11-24T06:10:18.160 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:18.158+0000 7f7bc0ff9640 1 -- 172.21.15.92:0/2659441474 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v13) v1 ==== 74+0+10860 (secure 0 0 0) 0x7f7bac057f20 con 0x7f7bc413e360 2023-11-24T06:10:18.160 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-24T06:10:18.161 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":13,"fsid":"2d6dee5f-8aa3-470f-931a-50ade963b1cb","created":"2023-11-24T06:10:04.707204+0000","modified":"2023-11-24T06:10:17.598352+0000","last_up_change":"2023-11-24T06:10:17.598352+0000","last_in_change":"2023-11-24T06:10:09.040793+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"ff37a9fb-135f-471f-b4b0-f7b5772006c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6803","nonce":20560714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6805","nonce":20560714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6809","nonce":20560714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6807","nonce":20560714}]},"public_addr":"172.21.15.92:6803/20560714","cluster_addr":"172.21.15.92:6805/20560714","heartbeat_back_addr":"172.21.15.92:6809/20560714","heartbeat_front_addr":"172.21.15.92:6807/20560714","state":["exists","up"]},{"osd":1,"uuid":"03cdd353-dcf1-4f38-94fe-a795137c06b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6819","nonce":2451239906}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6821","nonce":2451239906}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6825","nonce":2451239906}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6823","nonce":2451239906}]},"public_addr":"172.21.15.92:6819/2451239906","cluster_addr":"172.21.15.92:6821/2451239906","heartbeat_back_addr":"172.21.15.92:6825/2451239906","heartbeat_front_addr":"172.21.15.92:6823/2451239906","state":["exists","up"]},{"osd":2,"uuid":"a4a20d80-5d57-484b-becd-9aa8070dd3aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6811","nonce":1124979277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6813","nonce":1124979277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6817","nonce":1124979277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6815","nonce":1124979277}]},"public_addr":"172.21.15.92:6811/1124979277","cluster_addr":"172.21.15.92:6813/1124979277","heartbeat_back_addr":"172.21.15.92:6817/1124979277","heartbeat_front_addr":"172.21.15.92:6815/1124979277","state":["exists","up"]},{"osd":3,"uuid":"4648d7ce-9b29-4583-9425-4ba8fd2b1903","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6827","nonce":1734115782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6829","nonce":1734115782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6833","nonce":1734115782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6831","nonce":1734115782}]},"public_addr":"172.21.15.92:6827/1734115782","cluster_addr":"172.21.15.92:6829/1734115782","heartbeat_back_addr":"172.21.15.92:6833/1734115782","heartbeat_front_addr":"172.21.15.92:6831/1734115782","state":["exists","up"]},{"osd":4,"uuid":"3bc9d7eb-c6fa-4fcf-a3ec-ade604e05aac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6819","nonce":3425937229}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6822","nonce":3425937229}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6830","nonce":3425937229}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6826","nonce":3425937229}]},"public_addr":"172.21.15.118:6819/3425937229","cluster_addr":"172.21.15.118:6822/3425937229","heartbeat_back_addr":"172.21.15.118:6830/3425937229","heartbeat_front_addr":"172.21.15.118:6826/3425937229","state":["exists","up"]},{"osd":5,"uuid":"dce9ba6d-59ed-46d1-84fc-a84b13de10c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6801","nonce":827455984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6803","nonce":827455984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6807","nonce":827455984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6805","nonce":827455984}]},"public_addr":"172.21.15.118:6801/827455984","cluster_addr":"172.21.15.118:6803/827455984","heartbeat_back_addr":"172.21.15.118:6807/827455984","heartbeat_front_addr":"172.21.15.118:6805/827455984","state":["exists","up"]},{"osd":6,"uuid":"82aec504-ae58-4709-8d30-46f2b12081b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6809","nonce":2946986141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6811","nonce":2946986141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6815","nonce":2946986141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6813","nonce":2946986141}]},"public_addr":"172.21.15.118:6809/2946986141","cluster_addr":"172.21.15.118:6811/2946986141","heartbeat_back_addr":"172.21.15.118:6815/2946986141","heartbeat_front_addr":"172.21.15.118:6813/2946986141","state":["exists","up"]},{"osd":7,"uuid":"4ed4c624-c88f-4d35-bdb1-bf209b5dffb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6817","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6818","nonce":48122059}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6821","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6823","nonce":48122059}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6829","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6831","nonce":48122059}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6825","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6827","nonce":48122059}]},"public_addr":"172.21.15.118:6818/48122059","cluster_addr":"172.21.15.118:6823/48122059","heartbeat_back_addr":"172.21.15.118:6831/48122059","heartbeat_front_addr":"172.21.15.118:6827/48122059","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:10:18.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:18.160+0000 7f7bc9fdc640 1 -- 172.21.15.92:0/2659441474 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f7ba406adb0 msgr2=0x7f7ba406d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:18.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:18.160+0000 7f7bc9fdc640 1 --2- 172.21.15.92:0/2659441474 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f7ba406adb0 0x7f7ba406d270 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f7bb4035000 tx=0x7f7bb4030000 comp rx=0 tx=0).stop 2023-11-24T06:10:18.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:18.160+0000 7f7bc9fdc640 1 -- 172.21.15.92:0/2659441474 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7bc413e360 msgr2=0x7f7bc4152640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:18.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:18.160+0000 7f7bc9fdc640 1 --2- 172.21.15.92:0/2659441474 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7bc413e360 0x7f7bc4152640 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f7bac00ea10 tx=0x7f7bac00eed0 comp rx=0 tx=0).stop 2023-11-24T06:10:18.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:18.160+0000 7f7bc9fdc640 1 -- 172.21.15.92:0/2659441474 shutdown_connections 2023-11-24T06:10:18.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:18.160+0000 7f7bc9fdc640 1 --2- 172.21.15.92:0/2659441474 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f7ba406adb0 0x7f7ba406d270 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:18.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:18.160+0000 7f7bc9fdc640 1 --2- 172.21.15.92:0/2659441474 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7bc415f9c0 0x7f7bc40ad2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:18.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:18.160+0000 7f7bc9fdc640 1 --2- 172.21.15.92:0/2659441474 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7bc413e360 0x7f7bc4152640 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:18.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:18.160+0000 7f7bc9fdc640 1 --2- 172.21.15.92:0/2659441474 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7bc413df80 0x7f7bc415a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:18.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:18.160+0000 7f7bc9fdc640 1 -- 172.21.15.92:0/2659441474 >> 172.21.15.92:0/2659441474 conn(0x7f7bc406a800 msgr2=0x7f7bc40af220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:18.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:18.161+0000 7f7bc9fdc640 1 -- 172.21.15.92:0/2659441474 shutdown_connections 2023-11-24T06:10:18.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:18.161+0000 7f7bc9fdc640 1 -- 172.21.15.92:0/2659441474 wait complete. 2023-11-24T06:10:18.179 INFO:tasks.cephfs.filesystem.ceph_manager:[] 2023-11-24T06:10:18.179 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T06:10:18.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:18.369+0000 7f37abfff640 1 Processor -- start 2023-11-24T06:10:18.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:18.369+0000 7f37abfff640 1 -- start start 2023-11-24T06:10:18.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:18.370+0000 7f37abfff640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f37ac143c40 0x7f37ac071290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:18.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:18.370+0000 7f37abfff640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37ac070e50 con 0x7f37ac0717d0 2023-11-24T06:10:18.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:18.370+0000 7f37abfff640 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37ac144020 con 0x7f37ac0ccc10 2023-11-24T06:10:18.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:18.370+0000 7f37abfff640 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37ac0723c0 con 0x7f37ac143c40 2023-11-24T06:10:18.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:18.371+0000 7f37aa7fc640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f37ac0717d0 0x7f37ac0cc500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:35308/0 (socket says 172.21.15.92:35308) 2023-11-24T06:10:18.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:18.371+0000 7f37aa7fc640 1 -- 172.21.15.92:0/3604762117 learned_addr learned my addr 172.21.15.92:0/3604762117 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-24T06:10:18.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:18.371+0000 7f37a9ffb640 1 -- 172.21.15.92:0/3604762117 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 735195951 0 0) 0x7f37ac070e50 con 0x7f37ac0717d0 2023-11-24T06:10:18.375 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:18.371+0000 7f37a9ffb640 1 -- 172.21.15.92:0/3604762117 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3794003520 con 0x7f37ac0717d0 2023-11-24T06:10:18.375 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:18.371+0000 7f37aaffd640 1 --2- 172.21.15.92:0/3604762117 >> v2:172.21.15.118:3300/0 conn(0x7f37ac143c40 0x7f37ac071290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:18.375 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:18.371+0000 7f37a9ffb640 1 -- 172.21.15.92:0/3604762117 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4156632724 0 0) 0x7f3794003520 con 0x7f37ac0717d0 2023-11-24T06:10:18.376 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:18.371+0000 7f37a9ffb640 1 -- 172.21.15.92:0/3604762117 >> v2:172.21.15.118:3300/0 conn(0x7f37ac143c40 msgr2=0x7f37ac071290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:18.376 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:18.371+0000 7f37a9ffb640 1 --2- 172.21.15.92:0/3604762117 >> v2:172.21.15.118:3300/0 conn(0x7f37ac143c40 0x7f37ac071290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:18.376 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:18.371+0000 7f37a9ffb640 1 -- 172.21.15.92:0/3604762117 >> v1:172.21.15.118:6789/0 conn(0x7f37ac0ccc10 legacy=0x7f37ac06d160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:18.376 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:18.371+0000 7f37a9ffb640 1 -- 172.21.15.92:0/3604762117 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37ac071e00 con 0x7f37ac0717d0 2023-11-24T06:10:18.376 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:18.371+0000 7f37a9ffb640 1 -- 172.21.15.92:0/3604762117 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2607661350 0 0) 0x7f37a0001670 con 0x7f37ac0717d0 2023-11-24T06:10:18.376 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:18.371+0000 7f37a9ffb640 1 -- 172.21.15.92:0/3604762117 >> v1:172.21.15.92:6789/0 conn(0x7f37ac0717d0 legacy=0x7f37ac0cc500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:18.376 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:18.371+0000 7f37a9ffb640 1 --2- 172.21.15.92:0/3604762117 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f37940039c0 0x7f3794003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:18.376 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:18.371+0000 7f37a9ffb640 1 --2- 172.21.15.92:0/3604762117 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f37940047c0 0x7f3794004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:18.376 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:18.371+0000 7f37a9ffb640 1 --2- 172.21.15.92:0/3604762117 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f37940052a0 0x7f379400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:18.376 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:18.372+0000 7f37a9ffb640 1 -- 172.21.15.92:0/3604762117 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37ac0723c0 con 0x7f37940047c0 2023-11-24T06:10:18.376 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:18.372+0000 7f37a9ffb640 1 -- 172.21.15.92:0/3604762117 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f379400c1c0 con 0x7f37940052a0 2023-11-24T06:10:18.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:18.372+0000 7f37a9ffb640 1 -- 172.21.15.92:0/3604762117 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f379400c380 con 0x7f37940039c0 2023-11-24T06:10:18.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:18.372+0000 7f37aa7fc640 1 --2- 172.21.15.92:0/3604762117 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f37940047c0 0x7f3794004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:18.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:18.372+0000 7f37aaffd640 1 --2- 172.21.15.92:0/3604762117 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f37940039c0 0x7f3794003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:18.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:18.372+0000 7f37aa7fc640 1 -- 172.21.15.92:0/3604762117 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f37940039c0 msgr2=0x7f3794003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:18.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:18.372+0000 7f37aa7fc640 1 --2- 172.21.15.92:0/3604762117 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f37940039c0 0x7f3794003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:18.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:18.372+0000 7f37aa7fc640 1 -- 172.21.15.92:0/3604762117 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f37940052a0 msgr2=0x7f379400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:18.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:18.372+0000 7f37aa7fc640 1 --2- 172.21.15.92:0/3604762117 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f37940052a0 0x7f379400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:18.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:18.372+0000 7f37aa7fc640 1 -- 172.21.15.92:0/3604762117 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3794003520 con 0x7f37940047c0 2023-11-24T06:10:18.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:18.372+0000 7f37ab7fe640 1 -- 172.21.15.92:0/3604762117 reap_dead start 2023-11-24T06:10:18.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:18.372+0000 7f37aaffd640 1 --2- 172.21.15.92:0/3604762117 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f37940039c0 0x7f3794003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:10:18.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:18.373+0000 7f37aa7fc640 1 --2- 172.21.15.92:0/3604762117 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f37940047c0 0x7f3794004c40 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f37a000c0e0 tx=0x7f37940057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8601fcfec10bb211 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:18.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:18.373+0000 7f37a9ffb640 1 -- 172.21.15.92:0/3604762117 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37a0005ad0 con 0x7f37940047c0 2023-11-24T06:10:18.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:18.373+0000 7f37a9ffb640 1 -- 172.21.15.92:0/3604762117 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37a0005c30 con 0x7f37940047c0 2023-11-24T06:10:18.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:18.373+0000 7f37a9ffb640 1 -- 172.21.15.92:0/3604762117 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37a002ca30 con 0x7f37940047c0 2023-11-24T06:10:18.380 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:18.374+0000 7f37abfff640 1 -- 172.21.15.92:0/3604762117 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f37940047c0 msgr2=0x7f3794004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:18.380 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:18.374+0000 7f37abfff640 1 --2- 172.21.15.92:0/3604762117 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f37940047c0 0x7f3794004c40 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f37a000c0e0 tx=0x7f37940057f0 comp rx=0 tx=0).stop 2023-11-24T06:10:18.380 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:18.374+0000 7f37abfff640 1 -- 172.21.15.92:0/3604762117 shutdown_connections 2023-11-24T06:10:18.380 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:18.374+0000 7f37abfff640 1 --2- 172.21.15.92:0/3604762117 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f37940047c0 0x7f3794004c40 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:18.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:18.374+0000 7f37abfff640 1 -- 172.21.15.92:0/3604762117 >> 172.21.15.92:0/3604762117 conn(0x7f37ac06a800 msgr2=0x7f37ac06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:18.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:18.374+0000 7f37abfff640 1 -- 172.21.15.92:0/3604762117 shutdown_connections 2023-11-24T06:10:18.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:18.374+0000 7f37abfff640 1 -- 172.21.15.92:0/3604762117 wait complete. 2023-11-24T06:10:18.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:18.376+0000 7f37abfff640 1 Processor -- start 2023-11-24T06:10:18.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:18.376+0000 7f37abfff640 1 -- start start 2023-11-24T06:10:18.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:18.377+0000 7f37abfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f37940047c0 0x7f37ac15b980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:18.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:18.377+0000 7f37abfff640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f37ac15e700 0x7f37ac0b3850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:18.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:18.377+0000 7f37abfff640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f37ac13b6b0 0x7f37ac13bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:18.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:18.377+0000 7f37abfff640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37ac15bec0 con 0x7f37940047c0 2023-11-24T06:10:18.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:18.377+0000 7f37abfff640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37ac13c050 con 0x7f37ac15e700 2023-11-24T06:10:18.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:18.377+0000 7f37abfff640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37ac15ee70 con 0x7f37ac13b6b0 2023-11-24T06:10:18.382 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:18.377+0000 7f37aaffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f37940047c0 0x7f37ac15b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:18.382 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:18.377+0000 7f37aaffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f37940047c0 0x7f37ac15b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:42680/0 (socket says 172.21.15.92:42680) 2023-11-24T06:10:18.382 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:18.377+0000 7f37aaffd640 1 -- 172.21.15.92:0/3785047111 learned_addr learned my addr 172.21.15.92:0/3785047111 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:10:18.382 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:18.377+0000 7f37aa7fc640 1 --2- 172.21.15.92:0/3785047111 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f37ac15e700 0x7f37ac0b3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:18.382 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:18.377+0000 7f37aaffd640 1 -- 172.21.15.92:0/3785047111 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f37ac13b6b0 msgr2=0x7f37ac13bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:18.382 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:18.377+0000 7f37ab7fe640 1 --2- 172.21.15.92:0/3785047111 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f37ac13b6b0 0x7f37ac13bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:18.382 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:18.377+0000 7f37aaffd640 1 --2- 172.21.15.92:0/3785047111 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f37ac13b6b0 0x7f37ac13bb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:18.382 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:18.377+0000 7f37aaffd640 1 -- 172.21.15.92:0/3785047111 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f37ac15e700 msgr2=0x7f37ac0b3850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:18.382 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:18.377+0000 7f37aaffd640 1 --2- 172.21.15.92:0/3785047111 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f37ac15e700 0x7f37ac0b3850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:18.382 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:18.377+0000 7f37aaffd640 1 -- 172.21.15.92:0/3785047111 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37ac15f150 con 0x7f37940047c0 2023-11-24T06:10:18.382 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:18.377+0000 7f37aa7fc640 1 --2- 172.21.15.92:0/3785047111 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f37ac15e700 0x7f37ac0b3850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:10:18.383 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:18.378+0000 7f37aaffd640 1 --2- 172.21.15.92:0/3785047111 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f37940047c0 0x7f37ac15b980 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f379c00e5a0 tx=0x7f379c00ea60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:18.383 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:18.378+0000 7f3793fff640 1 -- 172.21.15.92:0/3785047111 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f379c013610 con 0x7f37940047c0 2023-11-24T06:10:18.383 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:18.378+0000 7f3793fff640 1 -- 172.21.15.92:0/3785047111 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f379c013770 con 0x7f37940047c0 2023-11-24T06:10:18.383 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:18.378+0000 7f3793fff640 1 -- 172.21.15.92:0/3785047111 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f379c013b20 con 0x7f37940047c0 2023-11-24T06:10:18.383 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:18.378+0000 7f37abfff640 1 -- 172.21.15.92:0/3785047111 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37ac15f430 con 0x7f37940047c0 2023-11-24T06:10:18.383 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:18.378+0000 7f37abfff640 1 -- 172.21.15.92:0/3785047111 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37ac15eb60 con 0x7f37940047c0 2023-11-24T06:10:18.384 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:18.379+0000 7f37abfff640 1 -- 172.21.15.92:0/3785047111 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37ac066e70 con 0x7f37940047c0 2023-11-24T06:10:18.384 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:18.380+0000 7f3793fff640 1 -- 172.21.15.92:0/3785047111 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 4) v1 ==== 89175+0+0 (secure 0 0 0) 0x7f379c013c80 con 0x7f37940047c0 2023-11-24T06:10:18.384 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:18.381+0000 7f3793fff640 1 --2- 172.21.15.92:0/3785047111 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f379806adb0 0x7f379806d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:18.384 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:18.381+0000 7f3793fff640 1 -- 172.21.15.92:0/3785047111 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 4471+0+0 (secure 0 0 0) 0x7f379c088cb0 con 0x7f37940047c0 2023-11-24T06:10:18.384 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:18.381+0000 7f37aa7fc640 1 --2- 172.21.15.92:0/3785047111 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f379806adb0 0x7f379806d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:18.384 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:18.382+0000 7f37aa7fc640 1 --2- 172.21.15.92:0/3785047111 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f379806adb0 0x7f379806d270 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f37a0034000 tx=0x7f37a0009030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:18.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:18.385+0000 7f3793fff640 1 -- 172.21.15.92:0/3785047111 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f379c054310 con 0x7f37940047c0 2023-11-24T06:10:18.601 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:18.598+0000 7f37abfff640 1 -- 172.21.15.92:0/3785047111 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f37ac067450 con 0x7f37940047c0 2023-11-24T06:10:18.601 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:18.599+0000 7f3793fff640 1 -- 172.21.15.92:0/3785047111 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1321 (secure 0 0 0) 0x7f379c05ce60 con 0x7f37940047c0 2023-11-24T06:10:18.602 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-24T06:10:18.602 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4236,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6836/437604699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":437604699},{"type":"v1","addr":"172.21.15.92:6836","nonce":437604699}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2023-11-24T06:10:18.602 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 2 2023-11-24T06:10:18.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:18.601+0000 7f37abfff640 1 -- 172.21.15.92:0/3785047111 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f379806adb0 msgr2=0x7f379806d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:18.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:18.601+0000 7f37abfff640 1 --2- 172.21.15.92:0/3785047111 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f379806adb0 0x7f379806d270 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f37a0034000 tx=0x7f37a0009030 comp rx=0 tx=0).stop 2023-11-24T06:10:18.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:18.601+0000 7f37abfff640 1 -- 172.21.15.92:0/3785047111 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f37940047c0 msgr2=0x7f37ac15b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:18.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:18.601+0000 7f37abfff640 1 --2- 172.21.15.92:0/3785047111 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f37940047c0 0x7f37ac15b980 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f379c00e5a0 tx=0x7f379c00ea60 comp rx=0 tx=0).stop 2023-11-24T06:10:18.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:18.602+0000 7f37abfff640 1 -- 172.21.15.92:0/3785047111 shutdown_connections 2023-11-24T06:10:18.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:18.602+0000 7f37abfff640 1 --2- 172.21.15.92:0/3785047111 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f379806adb0 0x7f379806d270 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:18.605 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:18.602+0000 7f37abfff640 1 --2- 172.21.15.92:0/3785047111 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f37ac13b6b0 0x7f37ac13bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:18.605 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:18.602+0000 7f37abfff640 1 --2- 172.21.15.92:0/3785047111 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f37ac15e700 0x7f37ac0b3850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:18.605 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:18.602+0000 7f37abfff640 1 --2- 172.21.15.92:0/3785047111 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f37940047c0 0x7f37ac15b980 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:18.605 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:18.602+0000 7f37abfff640 1 -- 172.21.15.92:0/3785047111 >> 172.21.15.92:0/3785047111 conn(0x7f37ac06a800 msgr2=0x7f37ac06abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:18.605 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:18.602+0000 7f37abfff640 1 -- 172.21.15.92:0/3785047111 shutdown_connections 2023-11-24T06:10:18.605 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:18.602+0000 7f37abfff640 1 -- 172.21.15.92:0/3785047111 wait complete. 2023-11-24T06:10:20.624 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T06:10:20.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:20.819+0000 7f8c813a9640 1 Processor -- start 2023-11-24T06:10:20.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:20.819+0000 7f8c813a9640 1 -- start start 2023-11-24T06:10:20.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:20.820+0000 7f8c813a9640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f8c7c0ce8f0 0x7f8c7c0ced30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:20.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:20.820+0000 7f8c813a9640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8c7c0b2a00 0x7f8c7c0b2e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:20.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:20.820+0000 7f8c813a9640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c7c0d0de0 con 0x7f8c7c160f80 2023-11-24T06:10:20.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:20.820+0000 7f8c813a9640 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c7c0d10c0 con 0x7f8c7c0ce8f0 2023-11-24T06:10:20.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:20.820+0000 7f8c813a9640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c7c0d13a0 con 0x7f8c7c0b2a00 2023-11-24T06:10:20.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:20.820+0000 7f8c7affd640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f8c7c160f80 0x7f8c7c161360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:40624/0 (socket says 172.21.15.92:40624) 2023-11-24T06:10:20.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:20.821+0000 7f8c7affd640 1 -- 172.21.15.92:0/2589038264 learned_addr learned my addr 172.21.15.92:0/2589038264 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-24T06:10:20.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:20.821+0000 7f8c7a7fc640 1 -- 172.21.15.92:0/2589038264 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3382604611 0 0) 0x7f8c7c0d0de0 con 0x7f8c7c160f80 2023-11-24T06:10:20.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:20.821+0000 7f8c7a7fc640 1 -- 172.21.15.92:0/2589038264 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8c68003520 con 0x7f8c7c160f80 2023-11-24T06:10:20.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:20.821+0000 7f8c7b7fe640 1 --2- 172.21.15.92:0/2589038264 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8c7c0b2a00 0x7f8c7c0b2e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:20.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:20.821+0000 7f8c73fff640 1 --2- 172.21.15.92:0/2589038264 >> v2:172.21.15.118:3300/0 conn(0x7f8c7c0ce8f0 0x7f8c7c0ced30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:20.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:20.821+0000 7f8c7a7fc640 1 -- 172.21.15.92:0/2589038264 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1873143615 0 0) 0x7f8c68003520 con 0x7f8c7c160f80 2023-11-24T06:10:20.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:20.821+0000 7f8c7a7fc640 1 -- 172.21.15.92:0/2589038264 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8c7c0b2a00 msgr2=0x7f8c7c0b2e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:20.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:20.821+0000 7f8c7a7fc640 1 --2- 172.21.15.92:0/2589038264 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8c7c0b2a00 0x7f8c7c0b2e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:20.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:20.821+0000 7f8c7a7fc640 1 -- 172.21.15.92:0/2589038264 >> v2:172.21.15.118:3300/0 conn(0x7f8c7c0ce8f0 msgr2=0x7f8c7c0ced30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:20.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:20.821+0000 7f8c7a7fc640 1 --2- 172.21.15.92:0/2589038264 >> v2:172.21.15.118:3300/0 conn(0x7f8c7c0ce8f0 0x7f8c7c0ced30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:20.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:20.821+0000 7f8c7a7fc640 1 -- 172.21.15.92:0/2589038264 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c7c0d1680 con 0x7f8c7c160f80 2023-11-24T06:10:20.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:20.821+0000 7f8c7a7fc640 1 -- 172.21.15.92:0/2589038264 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2607661350 0 0) 0x7f8c6c001670 con 0x7f8c7c160f80 2023-11-24T06:10:20.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:20.821+0000 7f8c7a7fc640 1 -- 172.21.15.92:0/2589038264 >> v1:172.21.15.92:6789/0 conn(0x7f8c7c160f80 legacy=0x7f8c7c161360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:20.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:20.821+0000 7f8c7a7fc640 1 --2- 172.21.15.92:0/2589038264 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8c68003970 0x7f8c68003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:20.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:20.821+0000 7f8c7a7fc640 1 --2- 172.21.15.92:0/2589038264 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8c680047c0 0x7f8c68004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:20.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:20.822+0000 7f8c7a7fc640 1 --2- 172.21.15.92:0/2589038264 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8c68005180 0x7f8c6800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:20.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:20.822+0000 7f8c7a7fc640 1 -- 172.21.15.92:0/2589038264 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c7c0d10c0 con 0x7f8c68003970 2023-11-24T06:10:20.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:20.822+0000 7f8c7a7fc640 1 -- 172.21.15.92:0/2589038264 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c7c0d13a0 con 0x7f8c68005180 2023-11-24T06:10:20.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:20.822+0000 7f8c7a7fc640 1 -- 172.21.15.92:0/2589038264 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c6800c020 con 0x7f8c680047c0 2023-11-24T06:10:20.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:20.822+0000 7f8c7b7fe640 1 --2- 172.21.15.92:0/2589038264 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8c7c0b2a00 0x7f8c7c0b2e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:10:20.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:20.822+0000 7f8c7affd640 1 --2- 172.21.15.92:0/2589038264 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8c68003970 0x7f8c68003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:20.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:20.822+0000 7f8c7affd640 1 -- 172.21.15.92:0/2589038264 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8c680047c0 msgr2=0x7f8c68004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:20.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:20.822+0000 7f8c7affd640 1 --2- 172.21.15.92:0/2589038264 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8c680047c0 0x7f8c68004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:20.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:20.822+0000 7f8c7affd640 1 -- 172.21.15.92:0/2589038264 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8c68005180 msgr2=0x7f8c6800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:20.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:20.822+0000 7f8c7affd640 1 --2- 172.21.15.92:0/2589038264 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8c68005180 0x7f8c6800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:20.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:20.822+0000 7f8c7affd640 1 -- 172.21.15.92:0/2589038264 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c68003520 con 0x7f8c68003970 2023-11-24T06:10:20.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:20.823+0000 7f8c7affd640 1 --2- 172.21.15.92:0/2589038264 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8c68003970 0x7f8c68003d90 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f8c6c00c110 tx=0x7f8c680057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee9598350424e52f server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:20.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:20.823+0000 7f8c7a7fc640 1 -- 172.21.15.92:0/2589038264 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c6c005ad0 con 0x7f8c68003970 2023-11-24T06:10:20.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:20.823+0000 7f8c7a7fc640 1 -- 172.21.15.92:0/2589038264 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c6c005c50 con 0x7f8c68003970 2023-11-24T06:10:20.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:20.823+0000 7f8c7a7fc640 1 -- 172.21.15.92:0/2589038264 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c6c02c830 con 0x7f8c68003970 2023-11-24T06:10:20.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:20.824+0000 7f8c813a9640 1 -- 172.21.15.92:0/2589038264 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8c68003970 msgr2=0x7f8c68003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:20.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:20.824+0000 7f8c813a9640 1 --2- 172.21.15.92:0/2589038264 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8c68003970 0x7f8c68003d90 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f8c6c00c110 tx=0x7f8c680057f0 comp rx=0 tx=0).stop 2023-11-24T06:10:20.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:20.824+0000 7f8c7b7fe640 1 -- 172.21.15.92:0/2589038264 reap_dead start 2023-11-24T06:10:20.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:20.824+0000 7f8c813a9640 1 -- 172.21.15.92:0/2589038264 shutdown_connections 2023-11-24T06:10:20.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:20.824+0000 7f8c813a9640 1 -- 172.21.15.92:0/2589038264 >> 172.21.15.92:0/2589038264 conn(0x7f8c7c09c550 msgr2=0x7f8c7c0d3210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:20.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:20.824+0000 7f8c813a9640 1 -- 172.21.15.92:0/2589038264 shutdown_connections 2023-11-24T06:10:20.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:20.824+0000 7f8c813a9640 1 -- 172.21.15.92:0/2589038264 wait complete. 2023-11-24T06:10:20.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:20.827+0000 7f8c813a9640 1 Processor -- start 2023-11-24T06:10:20.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:20.827+0000 7f8c813a9640 1 -- start start 2023-11-24T06:10:20.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:20.828+0000 7f8c813a9640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8c68003970 0x7f8c7c1586e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:20.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:20.828+0000 7f8c813a9640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8c7c15d6c0 0x7f8c7c15db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:20.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:20.829+0000 7f8c813a9640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8c7c0cc8f0 0x7f8c7c0ccd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:20.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:20.829+0000 7f8c813a9640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c7c0c7eb0 con 0x7f8c68003970 2023-11-24T06:10:20.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:20.829+0000 7f8c813a9640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c7c0c8100 con 0x7f8c7c0cc8f0 2023-11-24T06:10:20.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:20.829+0000 7f8c813a9640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c7c0c83e0 con 0x7f8c7c15d6c0 2023-11-24T06:10:20.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:20.829+0000 7f8c7affd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8c68003970 0x7f8c7c1586e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:20.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:20.829+0000 7f8c7affd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8c68003970 0x7f8c7c1586e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41760/0 (socket says 172.21.15.92:41760) 2023-11-24T06:10:20.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:20.829+0000 7f8c7affd640 1 -- 172.21.15.92:0/239138551 learned_addr learned my addr 172.21.15.92:0/239138551 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:10:20.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:20.829+0000 7f8c7affd640 1 -- 172.21.15.92:0/239138551 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8c7c15d6c0 msgr2=0x7f8c7c15db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:20.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:20.829+0000 7f8c7b7fe640 1 --2- 172.21.15.92:0/239138551 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8c7c0cc8f0 0x7f8c7c0ccd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:20.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:20.829+0000 7f8c73fff640 1 --2- 172.21.15.92:0/239138551 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8c7c15d6c0 0x7f8c7c15db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:20.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:20.829+0000 7f8c7affd640 1 --2- 172.21.15.92:0/239138551 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8c7c15d6c0 0x7f8c7c15db20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:20.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:20.829+0000 7f8c7affd640 1 -- 172.21.15.92:0/239138551 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8c7c0cc8f0 msgr2=0x7f8c7c0ccd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:20.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:20.829+0000 7f8c7affd640 1 --2- 172.21.15.92:0/239138551 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8c7c0cc8f0 0x7f8c7c0ccd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:20.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:20.829+0000 7f8c7affd640 1 -- 172.21.15.92:0/239138551 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c7c0c86c0 con 0x7f8c68003970 2023-11-24T06:10:20.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:20.829+0000 7f8c73fff640 1 --2- 172.21.15.92:0/239138551 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8c7c15d6c0 0x7f8c7c15db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:10:20.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:20.830+0000 7f8c7affd640 1 --2- 172.21.15.92:0/239138551 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8c68003970 0x7f8c7c1586e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f8c6c02c3d0 tx=0x7f8c6c001930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:20.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:20.830+0000 7f8c737fe640 1 -- 172.21.15.92:0/239138551 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c6c03c040 con 0x7f8c68003970 2023-11-24T06:10:20.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:20.830+0000 7f8c737fe640 1 -- 172.21.15.92:0/239138551 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c6c02dcd0 con 0x7f8c68003970 2023-11-24T06:10:20.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:20.830+0000 7f8c737fe640 1 -- 172.21.15.92:0/239138551 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c6c0072b0 con 0x7f8c68003970 2023-11-24T06:10:20.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:20.830+0000 7f8c813a9640 1 -- 172.21.15.92:0/239138551 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c7c0c89a0 con 0x7f8c68003970 2023-11-24T06:10:20.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:20.830+0000 7f8c813a9640 1 -- 172.21.15.92:0/239138551 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c7c0c9b90 con 0x7f8c68003970 2023-11-24T06:10:20.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:20.831+0000 7f8c813a9640 1 -- 172.21.15.92:0/239138551 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c7c15c4c0 con 0x7f8c68003970 2023-11-24T06:10:20.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:20.832+0000 7f8c737fe640 1 -- 172.21.15.92:0/239138551 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 4) v1 ==== 89175+0+0 (secure 0 0 0) 0x7f8c6c032070 con 0x7f8c68003970 2023-11-24T06:10:20.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:20.833+0000 7f8c737fe640 1 --2- 172.21.15.92:0/239138551 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f8c5c06ae30 0x7f8c5c06d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:20.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:20.833+0000 7f8c737fe640 1 -- 172.21.15.92:0/239138551 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f8c6c0aaab0 con 0x7f8c68003970 2023-11-24T06:10:20.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:20.838+0000 7f8c737fe640 1 -- 172.21.15.92:0/239138551 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8c6c076100 con 0x7f8c68003970 2023-11-24T06:10:20.846 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:20.843+0000 7f8c73fff640 1 --2- 172.21.15.92:0/239138551 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f8c5c06ae30 0x7f8c5c06d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:20.846 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:20.843+0000 7f8c73fff640 1 --2- 172.21.15.92:0/239138551 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f8c5c06ae30 0x7f8c5c06d2f0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f8c64006a90 tx=0x7f8c64008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:21.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:21.086+0000 7f8c813a9640 1 -- 172.21.15.92:0/239138551 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8c7c0675d0 con 0x7f8c68003970 2023-11-24T06:10:21.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:21.087+0000 7f8c737fe640 1 -- 172.21.15.92:0/239138551 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+3697 (secure 0 0 0) 0x7f8c6c079720 con 0x7f8c68003970 2023-11-24T06:10:21.090 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-24T06:10:21.090 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4236,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6836/437604699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":437604699},{"type":"v1","addr":"172.21.15.92:6836","nonce":437604699}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6835/3368711442","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":3368711442},{"type":"v1","addr":"172.21.15.118:6835","nonce":3368711442}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4255,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/2767290380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":2767290380},{"type":"v1","addr":"172.21.15.92:6837","nonce":2767290380}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4264,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6833/1146052262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":1146052262},{"type":"v1","addr":"172.21.15.118:6833","nonce":1146052262}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[]} 2023-11-24T06:10:21.091 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 3 2023-11-24T06:10:21.093 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:21.090+0000 7f8c813a9640 1 -- 172.21.15.92:0/239138551 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f8c5c06ae30 msgr2=0x7f8c5c06d2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:21.094 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:21.090+0000 7f8c813a9640 1 --2- 172.21.15.92:0/239138551 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f8c5c06ae30 0x7f8c5c06d2f0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f8c64006a90 tx=0x7f8c64008040 comp rx=0 tx=0).stop 2023-11-24T06:10:21.094 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:21.090+0000 7f8c813a9640 1 -- 172.21.15.92:0/239138551 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8c68003970 msgr2=0x7f8c7c1586e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:21.094 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:21.090+0000 7f8c813a9640 1 --2- 172.21.15.92:0/239138551 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8c68003970 0x7f8c7c1586e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f8c6c02c3d0 tx=0x7f8c6c001930 comp rx=0 tx=0).stop 2023-11-24T06:10:21.094 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:21.090+0000 7f8c813a9640 1 -- 172.21.15.92:0/239138551 shutdown_connections 2023-11-24T06:10:21.094 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:21.090+0000 7f8c813a9640 1 --2- 172.21.15.92:0/239138551 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f8c5c06ae30 0x7f8c5c06d2f0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:21.094 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:21.090+0000 7f8c813a9640 1 --2- 172.21.15.92:0/239138551 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8c7c0cc8f0 0x7f8c7c0ccd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:21.094 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:21.090+0000 7f8c813a9640 1 --2- 172.21.15.92:0/239138551 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8c7c15d6c0 0x7f8c7c15db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:21.094 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:21.090+0000 7f8c813a9640 1 --2- 172.21.15.92:0/239138551 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8c68003970 0x7f8c7c1586e0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:21.094 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:21.090+0000 7f8c813a9640 1 -- 172.21.15.92:0/239138551 >> 172.21.15.92:0/239138551 conn(0x7f8c7c09c550 msgr2=0x7f8c7c15b390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:21.095 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:21.091+0000 7f8c813a9640 1 -- 172.21.15.92:0/239138551 shutdown_connections 2023-11-24T06:10:21.095 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:21.091+0000 7f8c813a9640 1 -- 172.21.15.92:0/239138551 wait complete. 2023-11-24T06:10:21.114 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:10:21.323 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:21.320+0000 7f828359e640 1 Processor -- start 2023-11-24T06:10:21.323 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:21.320+0000 7f828359e640 1 -- start start 2023-11-24T06:10:21.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:21.321+0000 7f828359e640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f8284140300 0x7f828406a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:21.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:21.321+0000 7f828359e640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f828406a920 0x7f8284141c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:21.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:21.321+0000 7f828359e640 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f828406ad70 con 0x7f828413ff20 2023-11-24T06:10:21.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:21.321+0000 7f828359e640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82840b5440 con 0x7f828406a920 2023-11-24T06:10:21.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:21.321+0000 7f828359e640 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8284161c80 con 0x7f8284140300 2023-11-24T06:10:21.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:21.321+0000 7f8282d9d640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f828406a920 0x7f8284141c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:21.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:21.322+0000 7f8282d9d640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f828406a920 0x7f8284141c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41764/0 (socket says 172.21.15.92:41764) 2023-11-24T06:10:21.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:21.322+0000 7f8282d9d640 1 -- 172.21.15.92:0/1371295255 learned_addr learned my addr 172.21.15.92:0/1371295255 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:10:21.326 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:21.322+0000 7f8282d9d640 1 -- 172.21.15.92:0/1371295255 >> v2:172.21.15.118:3300/0 conn(0x7f8284140300 msgr2=0x7f828406a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:21.326 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:21.322+0000 7f8282d9d640 1 --2- 172.21.15.92:0/1371295255 >> v2:172.21.15.118:3300/0 conn(0x7f8284140300 0x7f828406a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:21.326 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:21.322+0000 7f8282d9d640 1 -- 172.21.15.92:0/1371295255 >> v1:172.21.15.118:6789/0 conn(0x7f828413ff20 legacy=0x7f828415e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:21.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:21.322+0000 7f8282d9d640 1 -- 172.21.15.92:0/1371295255 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82840b4830 con 0x7f828406a920 2023-11-24T06:10:21.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:21.323+0000 7f8282d9d640 1 --2- 172.21.15.92:0/1371295255 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f828406a920 0x7f8284141c50 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f82840b2c50 tx=0x7f827c02be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3da315f44562810 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:21.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:21.323+0000 7f8281d9b640 1 -- 172.21.15.92:0/1371295255 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f827c02c9e0 con 0x7f828406a920 2023-11-24T06:10:21.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:21.323+0000 7f8281d9b640 1 -- 172.21.15.92:0/1371295255 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f827c02cb40 con 0x7f828406a920 2023-11-24T06:10:21.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:21.323+0000 7f8281d9b640 1 -- 172.21.15.92:0/1371295255 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f827c0343f0 con 0x7f828406a920 2023-11-24T06:10:21.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:21.324+0000 7f828359e640 1 -- 172.21.15.92:0/1371295255 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f828406a920 msgr2=0x7f8284141c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:21.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:21.324+0000 7f828359e640 1 --2- 172.21.15.92:0/1371295255 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f828406a920 0x7f8284141c50 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f82840b2c50 tx=0x7f827c02be40 comp rx=0 tx=0).stop 2023-11-24T06:10:21.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:21.324+0000 7f828359e640 1 -- 172.21.15.92:0/1371295255 shutdown_connections 2023-11-24T06:10:21.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:21.324+0000 7f828359e640 1 --2- 172.21.15.92:0/1371295255 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f828406a920 0x7f8284141c50 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:21.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:21.324+0000 7f828359e640 1 --2- 172.21.15.92:0/1371295255 >> v2:172.21.15.118:3300/0 conn(0x7f8284140300 0x7f828406a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:21.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:21.324+0000 7f828359e640 1 -- 172.21.15.92:0/1371295255 >> 172.21.15.92:0/1371295255 conn(0x7f82840b1cd0 msgr2=0x7f82840b20b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:21.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:21.324+0000 7f828359e640 1 -- 172.21.15.92:0/1371295255 shutdown_connections 2023-11-24T06:10:21.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:21.324+0000 7f828359e640 1 -- 172.21.15.92:0/1371295255 wait complete. 2023-11-24T06:10:21.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:21.327+0000 7f828359e640 1 Processor -- start 2023-11-24T06:10:21.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:21.327+0000 7f828359e640 1 -- start start 2023-11-24T06:10:21.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:21.328+0000 7f828359e640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f828406a920 0x7f828413cee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:21.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:21.329+0000 7f828359e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f828413ff20 0x7f828414fd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:21.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:21.329+0000 7f828359e640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8284140300 0x7f82841502d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:21.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:21.329+0000 7f828359e640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82841386a0 con 0x7f828413ff20 2023-11-24T06:10:21.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:21.329+0000 7f828359e640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82841f7120 con 0x7f8284140300 2023-11-24T06:10:21.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:21.329+0000 7f828359e640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8284138960 con 0x7f828406a920 2023-11-24T06:10:21.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:21.329+0000 7f827bfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f828413ff20 0x7f828414fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:21.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:21.329+0000 7f827bfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f828413ff20 0x7f828414fd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41768/0 (socket says 172.21.15.92:41768) 2023-11-24T06:10:21.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:21.329+0000 7f827bfff640 1 -- 172.21.15.92:0/1480814518 learned_addr learned my addr 172.21.15.92:0/1480814518 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:10:21.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:21.329+0000 7f828259c640 1 --2- 172.21.15.92:0/1480814518 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f828406a920 0x7f828413cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:21.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:21.329+0000 7f8282d9d640 1 --2- 172.21.15.92:0/1480814518 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8284140300 0x7f82841502d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:21.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:21.329+0000 7f827bfff640 1 -- 172.21.15.92:0/1480814518 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f828406a920 msgr2=0x7f828413cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:21.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:21.329+0000 7f827bfff640 1 --2- 172.21.15.92:0/1480814518 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f828406a920 0x7f828413cee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:21.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:21.329+0000 7f827bfff640 1 -- 172.21.15.92:0/1480814518 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8284140300 msgr2=0x7f82841502d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:21.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:21.329+0000 7f827bfff640 1 --2- 172.21.15.92:0/1480814518 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8284140300 0x7f82841502d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:21.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:21.329+0000 7f827bfff640 1 -- 172.21.15.92:0/1480814518 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f827c009b30 con 0x7f828413ff20 2023-11-24T06:10:21.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:21.330+0000 7f827bfff640 1 --2- 172.21.15.92:0/1480814518 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f828413ff20 0x7f828414fd90 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f826c00b410 tx=0x7f826c00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:21.335 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:21.330+0000 7f827affd640 1 -- 172.21.15.92:0/1480814518 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f826c011830 con 0x7f828413ff20 2023-11-24T06:10:21.335 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:21.330+0000 7f827affd640 1 -- 172.21.15.92:0/1480814518 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f826c0119b0 con 0x7f828413ff20 2023-11-24T06:10:21.335 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:21.330+0000 7f828359e640 1 -- 172.21.15.92:0/1480814518 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8284138c40 con 0x7f828413ff20 2023-11-24T06:10:21.335 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:21.330+0000 7f827affd640 1 -- 172.21.15.92:0/1480814518 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f826c011d60 con 0x7f828413ff20 2023-11-24T06:10:21.335 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:21.330+0000 7f828359e640 1 -- 172.21.15.92:0/1480814518 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8284139440 con 0x7f828413ff20 2023-11-24T06:10:21.335 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:21.331+0000 7f828359e640 1 -- 172.21.15.92:0/1480814518 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8284067450 con 0x7f828413ff20 2023-11-24T06:10:21.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:21.332+0000 7f827affd640 1 -- 172.21.15.92:0/1480814518 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 4) v1 ==== 89175+0+0 (secure 0 0 0) 0x7f826c01a400 con 0x7f828413ff20 2023-11-24T06:10:21.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:21.333+0000 7f827affd640 1 --2- 172.21.15.92:0/1480814518 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f826406adb0 0x7f826406d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:21.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:21.333+0000 7f827affd640 1 -- 172.21.15.92:0/1480814518 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f826c089300 con 0x7f828413ff20 2023-11-24T06:10:21.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:21.334+0000 7f828259c640 1 --2- 172.21.15.92:0/1480814518 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f826406adb0 0x7f826406d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:21.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:21.334+0000 7f828259c640 1 --2- 172.21.15.92:0/1480814518 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f826406adb0 0x7f826406d270 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f8274007fd0 tx=0x7f8274009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:21.341 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:21.338+0000 7f827affd640 1 -- 172.21.15.92:0/1480814518 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f826c0547e0 con 0x7f828413ff20 2023-11-24T06:10:21.538 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:21.534+0000 7f828359e640 1 -- 172.21.15.92:0/1480814518 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8284161480 con 0x7f828413ff20 2023-11-24T06:10:21.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:21.536+0000 7f827affd640 1 -- 172.21.15.92:0/1480814518 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+12802 (secure 0 0 0) 0x7f826c057e00 con 0x7f828413ff20 2023-11-24T06:10:21.540 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-24T06:10:21.540 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":16,"fsid":"2d6dee5f-8aa3-470f-931a-50ade963b1cb","created":"2023-11-24T06:10:04.707204+0000","modified":"2023-11-24T06:10:20.609319+0000","last_up_change":"2023-11-24T06:10:17.598352+0000","last_in_change":"2023-11-24T06:10:09.040793+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:10:17.733098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ff37a9fb-135f-471f-b4b0-f7b5772006c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6803","nonce":20560714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6805","nonce":20560714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6809","nonce":20560714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6807","nonce":20560714}]},"public_addr":"172.21.15.92:6803/20560714","cluster_addr":"172.21.15.92:6805/20560714","heartbeat_back_addr":"172.21.15.92:6809/20560714","heartbeat_front_addr":"172.21.15.92:6807/20560714","state":["exists","up"]},{"osd":1,"uuid":"03cdd353-dcf1-4f38-94fe-a795137c06b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6819","nonce":2451239906}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6821","nonce":2451239906}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6825","nonce":2451239906}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6823","nonce":2451239906}]},"public_addr":"172.21.15.92:6819/2451239906","cluster_addr":"172.21.15.92:6821/2451239906","heartbeat_back_addr":"172.21.15.92:6825/2451239906","heartbeat_front_addr":"172.21.15.92:6823/2451239906","state":["exists","up"]},{"osd":2,"uuid":"a4a20d80-5d57-484b-becd-9aa8070dd3aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6811","nonce":1124979277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6813","nonce":1124979277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6817","nonce":1124979277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6815","nonce":1124979277}]},"public_addr":"172.21.15.92:6811/1124979277","cluster_addr":"172.21.15.92:6813/1124979277","heartbeat_back_addr":"172.21.15.92:6817/1124979277","heartbeat_front_addr":"172.21.15.92:6815/1124979277","state":["exists","up"]},{"osd":3,"uuid":"4648d7ce-9b29-4583-9425-4ba8fd2b1903","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6827","nonce":1734115782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6829","nonce":1734115782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6833","nonce":1734115782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6831","nonce":1734115782}]},"public_addr":"172.21.15.92:6827/1734115782","cluster_addr":"172.21.15.92:6829/1734115782","heartbeat_back_addr":"172.21.15.92:6833/1734115782","heartbeat_front_addr":"172.21.15.92:6831/1734115782","state":["exists","up"]},{"osd":4,"uuid":"3bc9d7eb-c6fa-4fcf-a3ec-ade604e05aac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6819","nonce":3425937229}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6822","nonce":3425937229}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6830","nonce":3425937229}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6826","nonce":3425937229}]},"public_addr":"172.21.15.118:6819/3425937229","cluster_addr":"172.21.15.118:6822/3425937229","heartbeat_back_addr":"172.21.15.118:6830/3425937229","heartbeat_front_addr":"172.21.15.118:6826/3425937229","state":["exists","up"]},{"osd":5,"uuid":"dce9ba6d-59ed-46d1-84fc-a84b13de10c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6801","nonce":827455984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6803","nonce":827455984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6807","nonce":827455984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6805","nonce":827455984}]},"public_addr":"172.21.15.118:6801/827455984","cluster_addr":"172.21.15.118:6803/827455984","heartbeat_back_addr":"172.21.15.118:6807/827455984","heartbeat_front_addr":"172.21.15.118:6805/827455984","state":["exists","up"]},{"osd":6,"uuid":"82aec504-ae58-4709-8d30-46f2b12081b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6809","nonce":2946986141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6811","nonce":2946986141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6815","nonce":2946986141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6813","nonce":2946986141}]},"public_addr":"172.21.15.118:6809/2946986141","cluster_addr":"172.21.15.118:6811/2946986141","heartbeat_back_addr":"172.21.15.118:6815/2946986141","heartbeat_front_addr":"172.21.15.118:6813/2946986141","state":["exists","up"]},{"osd":7,"uuid":"4ed4c624-c88f-4d35-bdb1-bf209b5dffb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6817","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6818","nonce":48122059}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6821","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6823","nonce":48122059}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6829","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6831","nonce":48122059}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6825","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6827","nonce":48122059}]},"public_addr":"172.21.15.118:6818/48122059","cluster_addr":"172.21.15.118:6823/48122059","heartbeat_back_addr":"172.21.15.118:6831/48122059","heartbeat_front_addr":"172.21.15.118:6827/48122059","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.536664+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.368324+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.271201+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.342971+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.621832+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.387110+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.632946+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.416032+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:10:21.542 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:21.539+0000 7f828359e640 1 -- 172.21.15.92:0/1480814518 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f826406adb0 msgr2=0x7f826406d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:21.543 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:21.539+0000 7f828359e640 1 --2- 172.21.15.92:0/1480814518 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f826406adb0 0x7f826406d270 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f8274007fd0 tx=0x7f8274009040 comp rx=0 tx=0).stop 2023-11-24T06:10:21.543 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:21.539+0000 7f828359e640 1 -- 172.21.15.92:0/1480814518 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f828413ff20 msgr2=0x7f828414fd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:21.543 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:21.539+0000 7f828359e640 1 --2- 172.21.15.92:0/1480814518 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f828413ff20 0x7f828414fd90 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f826c00b410 tx=0x7f826c00b8d0 comp rx=0 tx=0).stop 2023-11-24T06:10:21.543 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:21.539+0000 7f828359e640 1 -- 172.21.15.92:0/1480814518 shutdown_connections 2023-11-24T06:10:21.543 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:21.539+0000 7f828359e640 1 --2- 172.21.15.92:0/1480814518 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f826406adb0 0x7f826406d270 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:21.543 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:21.539+0000 7f828359e640 1 --2- 172.21.15.92:0/1480814518 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8284140300 0x7f82841502d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:21.543 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:21.539+0000 7f828359e640 1 --2- 172.21.15.92:0/1480814518 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f828413ff20 0x7f828414fd90 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:21.543 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:21.539+0000 7f828359e640 1 --2- 172.21.15.92:0/1480814518 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f828406a920 0x7f828413cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:21.543 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:21.539+0000 7f828359e640 1 -- 172.21.15.92:0/1480814518 >> 172.21.15.92:0/1480814518 conn(0x7f82840b1cd0 msgr2=0x7f828406d570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:21.543 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:21.539+0000 7f828359e640 1 -- 172.21.15.92:0/1480814518 shutdown_connections 2023-11-24T06:10:21.544 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:21.539+0000 7f828359e640 1 -- 172.21.15.92:0/1480814518 wait complete. 2023-11-24T06:10:21.560 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T06:10:17.733098+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T06:10:21.560 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T06:10:21.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:21.755+0000 7f12a6d89640 1 Processor -- start 2023-11-24T06:10:21.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:21.756+0000 7f12a6d89640 1 -- start start 2023-11-24T06:10:21.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:21.757+0000 7f12a6d89640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f12a00cfcf0 0x7f12a0160700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:21.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:21.757+0000 7f12a6d89640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f12a00d00d0 0x7f12a0160c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:21.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:21.757+0000 7f12a6d89640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12a00d4160 con 0x7f12a00d3910 2023-11-24T06:10:21.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:21.757+0000 7f12a6d89640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12a00ceff0 con 0x7f12a00d00d0 2023-11-24T06:10:21.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:21.757+0000 7f12a6d89640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12a00cf410 con 0x7f12a00cfcf0 2023-11-24T06:10:21.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:21.757+0000 7f129ffff640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f12a00d3910 0x7f12a00b2a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:40632/0 (socket says 172.21.15.92:40632) 2023-11-24T06:10:21.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:21.757+0000 7f129ffff640 1 -- 172.21.15.92:0/4243771782 learned_addr learned my addr 172.21.15.92:0/4243771782 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-24T06:10:21.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:21.757+0000 7f12a5326640 1 --2- 172.21.15.92:0/4243771782 >> v2:172.21.15.92:3300/0 conn(0x7f12a00d00d0 0x7f12a0160c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:21.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:21.758+0000 7f12a4b25640 1 --2- 172.21.15.92:0/4243771782 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f12a00cfcf0 0x7f12a0160700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:21.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:21.758+0000 7f129f7fe640 1 -- 172.21.15.92:0/4243771782 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3490387553 0 0) 0x7f12a00d4160 con 0x7f12a00d3910 2023-11-24T06:10:21.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:21.758+0000 7f129f7fe640 1 -- 172.21.15.92:0/4243771782 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f128c003520 con 0x7f12a00d3910 2023-11-24T06:10:21.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:21.758+0000 7f12a5326640 1 -- 172.21.15.92:0/4243771782 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f12a00cfcf0 msgr2=0x7f12a0160700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:21.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:21.758+0000 7f12a5326640 1 --2- 172.21.15.92:0/4243771782 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f12a00cfcf0 0x7f12a0160700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:21.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:21.758+0000 7f12a5326640 1 -- 172.21.15.92:0/4243771782 >> v1:172.21.15.92:6789/0 conn(0x7f12a00d3910 legacy=0x7f12a00b2a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:21.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:21.758+0000 7f12a5326640 1 -- 172.21.15.92:0/4243771782 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12a00b2420 con 0x7f12a00d00d0 2023-11-24T06:10:21.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:21.759+0000 7f12a5326640 1 --2- 172.21.15.92:0/4243771782 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f12a00d00d0 0x7f12a0160c40 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f12980097d0 tx=0x7f129802b810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad4055e680908708 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:21.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:21.759+0000 7f129f7fe640 1 -- 172.21.15.92:0/4243771782 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1298031770 con 0x7f12a00d00d0 2023-11-24T06:10:21.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:21.760+0000 7f129f7fe640 1 -- 172.21.15.92:0/4243771782 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12980318d0 con 0x7f12a00d00d0 2023-11-24T06:10:21.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:21.760+0000 7f129f7fe640 1 -- 172.21.15.92:0/4243771782 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1298031c60 con 0x7f12a00d00d0 2023-11-24T06:10:21.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:21.760+0000 7f12a6d89640 1 -- 172.21.15.92:0/4243771782 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f12a00d00d0 msgr2=0x7f12a0160c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:21.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:21.760+0000 7f12a6d89640 1 --2- 172.21.15.92:0/4243771782 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f12a00d00d0 0x7f12a0160c40 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f12980097d0 tx=0x7f129802b810 comp rx=0 tx=0).stop 2023-11-24T06:10:21.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:21.760+0000 7f12a6d89640 1 -- 172.21.15.92:0/4243771782 shutdown_connections 2023-11-24T06:10:21.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:21.760+0000 7f12a6d89640 1 --2- 172.21.15.92:0/4243771782 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f12a00d00d0 0x7f12a0160c40 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:21.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:21.760+0000 7f12a6d89640 1 --2- 172.21.15.92:0/4243771782 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f12a00cfcf0 0x7f12a0160700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:21.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:21.760+0000 7f12a6d89640 1 -- 172.21.15.92:0/4243771782 >> 172.21.15.92:0/4243771782 conn(0x7f12a009c530 msgr2=0x7f12a00d3200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:21.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:21.760+0000 7f12a6d89640 1 -- 172.21.15.92:0/4243771782 shutdown_connections 2023-11-24T06:10:21.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:21.760+0000 7f12a6d89640 1 -- 172.21.15.92:0/4243771782 wait complete. 2023-11-24T06:10:21.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:21.762+0000 7f12a6d89640 1 Processor -- start 2023-11-24T06:10:21.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:21.762+0000 7f12a6d89640 1 -- start start 2023-11-24T06:10:21.766 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:21.763+0000 7f12a6d89640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f12a00cfcf0 0x7f12a014cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:21.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:21.763+0000 7f12a6d89640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f12a00d00d0 0x7f12a00c6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:21.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:21.763+0000 7f12a6d89640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f12a00d3910 0x7f12a01f6610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:21.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:21.763+0000 7f12a6d89640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12a014d500 con 0x7f12a00d00d0 2023-11-24T06:10:21.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:21.763+0000 7f12a6d89640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12a00ca0b0 con 0x7f12a00cfcf0 2023-11-24T06:10:21.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:21.763+0000 7f12a6d89640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12a0153630 con 0x7f12a00d3910 2023-11-24T06:10:21.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:21.763+0000 7f129ffff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f12a00d00d0 0x7f12a00c6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:21.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:21.763+0000 7f129ffff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f12a00d00d0 0x7f12a00c6cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41790/0 (socket says 172.21.15.92:41790) 2023-11-24T06:10:21.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:21.763+0000 7f129ffff640 1 -- 172.21.15.92:0/2992484842 learned_addr learned my addr 172.21.15.92:0/2992484842 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:10:21.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:21.764+0000 7f12a4b25640 1 --2- 172.21.15.92:0/2992484842 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f12a00cfcf0 0x7f12a014cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:21.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:21.764+0000 7f129ffff640 1 -- 172.21.15.92:0/2992484842 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f12a00d3910 msgr2=0x7f12a01f6610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:21.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:21.764+0000 7f12a5326640 1 --2- 172.21.15.92:0/2992484842 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f12a00d3910 0x7f12a01f6610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:21.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:21.764+0000 7f129ffff640 1 --2- 172.21.15.92:0/2992484842 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f12a00d3910 0x7f12a01f6610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:21.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:21.764+0000 7f129ffff640 1 -- 172.21.15.92:0/2992484842 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f12a00cfcf0 msgr2=0x7f12a014cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:21.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:21.764+0000 7f129ffff640 1 --2- 172.21.15.92:0/2992484842 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f12a00cfcf0 0x7f12a014cfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:21.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:21.764+0000 7f129ffff640 1 -- 172.21.15.92:0/2992484842 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1298009400 con 0x7f12a00d00d0 2023-11-24T06:10:21.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:21.764+0000 7f129ffff640 1 --2- 172.21.15.92:0/2992484842 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f12a00d00d0 0x7f12a00c6cc0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f128800c530 tx=0x7f128800c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:21.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:21.764+0000 7f129d7fa640 1 -- 172.21.15.92:0/2992484842 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1288005ce0 con 0x7f12a00d00d0 2023-11-24T06:10:21.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:21.764+0000 7f129d7fa640 1 -- 172.21.15.92:0/2992484842 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1288005e40 con 0x7f12a00d00d0 2023-11-24T06:10:21.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:21.765+0000 7f12a6d89640 1 -- 172.21.15.92:0/2992484842 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12a00c9810 con 0x7f12a00d00d0 2023-11-24T06:10:21.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:21.765+0000 7f129d7fa640 1 -- 172.21.15.92:0/2992484842 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12880072b0 con 0x7f12a00d00d0 2023-11-24T06:10:21.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:21.765+0000 7f12a6d89640 1 -- 172.21.15.92:0/2992484842 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12a00c9dd0 con 0x7f12a00d00d0 2023-11-24T06:10:21.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:21.766+0000 7f12a6d89640 1 -- 172.21.15.92:0/2992484842 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12a00b06f0 con 0x7f12a00d00d0 2023-11-24T06:10:21.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:21.767+0000 7f129d7fa640 1 -- 172.21.15.92:0/2992484842 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 4) v1 ==== 89175+0+0 (secure 0 0 0) 0x7f1288007410 con 0x7f12a00d00d0 2023-11-24T06:10:21.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:21.767+0000 7f129d7fa640 1 --2- 172.21.15.92:0/2992484842 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f128006adb0 0x7f128006d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:21.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:21.768+0000 7f129d7fa640 1 -- 172.21.15.92:0/2992484842 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f12880887b0 con 0x7f12a00d00d0 2023-11-24T06:10:21.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:21.768+0000 7f12a4b25640 1 --2- 172.21.15.92:0/2992484842 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f128006adb0 0x7f128006d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:21.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:21.768+0000 7f12a4b25640 1 --2- 172.21.15.92:0/2992484842 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f128006adb0 0x7f128006d270 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f1290009e90 tx=0x7f129000b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:21.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:21.772+0000 7f129d7fa640 1 -- 172.21.15.92:0/2992484842 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1288053d40 con 0x7f12a00d00d0 2023-11-24T06:10:21.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:21.967+0000 7f12a6d89640 1 -- 172.21.15.92:0/2992484842 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f12a00675d0 con 0x7f12a00d00d0 2023-11-24T06:10:21.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:21.967+0000 7f129d7fa640 1 -- 172.21.15.92:0/2992484842 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1288017020 con 0x7f12a00d00d0 2023-11-24T06:10:21.971 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 1 2023-11-24T06:10:21.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:21.970+0000 7f12a6d89640 1 -- 172.21.15.92:0/2992484842 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f128006adb0 msgr2=0x7f128006d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:21.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:21.970+0000 7f12a6d89640 1 --2- 172.21.15.92:0/2992484842 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f128006adb0 0x7f128006d270 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f1290009e90 tx=0x7f129000b040 comp rx=0 tx=0).stop 2023-11-24T06:10:21.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:21.970+0000 7f12a6d89640 1 -- 172.21.15.92:0/2992484842 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f12a00d00d0 msgr2=0x7f12a00c6cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:21.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:21.970+0000 7f12a6d89640 1 --2- 172.21.15.92:0/2992484842 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f12a00d00d0 0x7f12a00c6cc0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f128800c530 tx=0x7f128800c9f0 comp rx=0 tx=0).stop 2023-11-24T06:10:21.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:21.970+0000 7f12a6d89640 1 -- 172.21.15.92:0/2992484842 shutdown_connections 2023-11-24T06:10:21.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:21.970+0000 7f12a6d89640 1 --2- 172.21.15.92:0/2992484842 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f128006adb0 0x7f128006d270 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:21.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:21.970+0000 7f12a6d89640 1 --2- 172.21.15.92:0/2992484842 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f12a00d3910 0x7f12a01f6610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:21.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:21.970+0000 7f12a6d89640 1 --2- 172.21.15.92:0/2992484842 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f12a00d00d0 0x7f12a00c6cc0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:21.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:21.970+0000 7f12a6d89640 1 --2- 172.21.15.92:0/2992484842 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f12a00cfcf0 0x7f12a014cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:21.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:21.970+0000 7f12a6d89640 1 -- 172.21.15.92:0/2992484842 >> 172.21.15.92:0/2992484842 conn(0x7f12a009c530 msgr2=0x7f12a00afe90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:21.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:21.970+0000 7f12a6d89640 1 -- 172.21.15.92:0/2992484842 shutdown_connections 2023-11-24T06:10:21.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:21.970+0000 7f12a6d89640 1 -- 172.21.15.92:0/2992484842 wait complete. 2023-11-24T06:10:21.992 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-11-24T06:10:22.201 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:22.198+0000 7f935d823640 1 Processor -- start 2023-11-24T06:10:22.201 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:22.198+0000 7f935d823640 1 -- start start 2023-11-24T06:10:22.203 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:22.200+0000 7f935d823640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f935813e360 0x7f93580cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:22.203 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:22.200+0000 7f935d823640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f935815f9c0 0x7f93580cf7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:22.203 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:22.200+0000 7f935d823640 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f935815c5f0 con 0x7f935813df80 2023-11-24T06:10:22.204 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:22.200+0000 7f935d823640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f935806c470 con 0x7f935815f9c0 2023-11-24T06:10:22.204 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:22.200+0000 7f935d823640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f935806c750 con 0x7f935813e360 2023-11-24T06:10:22.204 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:22.200+0000 7f93577fe640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f935815f9c0 0x7f93580cf7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:22.204 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:22.200+0000 7f93577fe640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f935815f9c0 0x7f93580cf7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41792/0 (socket says 172.21.15.92:41792) 2023-11-24T06:10:22.204 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:22.200+0000 7f93577fe640 1 -- 172.21.15.92:0/1938453829 learned_addr learned my addr 172.21.15.92:0/1938453829 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:10:22.204 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:22.200+0000 7f93567fc640 1 --2- 172.21.15.92:0/1938453829 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f935813e360 0x7f93580cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:22.205 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:22.201+0000 7f93577fe640 1 -- 172.21.15.92:0/1938453829 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f935813e360 msgr2=0x7f93580cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:22.205 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:22.201+0000 7f93577fe640 1 --2- 172.21.15.92:0/1938453829 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f935813e360 0x7f93580cf290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:22.205 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:22.201+0000 7f93577fe640 1 -- 172.21.15.92:0/1938453829 >> v1:172.21.15.118:6789/0 conn(0x7f935813df80 legacy=0x7f935815bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:22.205 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:22.201+0000 7f93577fe640 1 -- 172.21.15.92:0/1938453829 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f935806ca30 con 0x7f935815f9c0 2023-11-24T06:10:22.206 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:22.202+0000 7f93577fe640 1 --2- 172.21.15.92:0/1938453829 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f935815f9c0 0x7f93580cf7d0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f9350009e50 tx=0x7f935002bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b6f9bf14b116f2b server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:22.206 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:22.202+0000 7f9355ffb640 1 -- 172.21.15.92:0/1938453829 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f935002cb00 con 0x7f935815f9c0 2023-11-24T06:10:22.206 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:22.202+0000 7f9355ffb640 1 -- 172.21.15.92:0/1938453829 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f935002cc60 con 0x7f935815f9c0 2023-11-24T06:10:22.206 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:22.202+0000 7f9355ffb640 1 -- 172.21.15.92:0/1938453829 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93500343f0 con 0x7f935815f9c0 2023-11-24T06:10:22.206 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:22.203+0000 7f935d823640 1 -- 172.21.15.92:0/1938453829 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f935815f9c0 msgr2=0x7f93580cf7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:22.206 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:22.203+0000 7f935d823640 1 --2- 172.21.15.92:0/1938453829 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f935815f9c0 0x7f93580cf7d0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f9350009e50 tx=0x7f935002bf60 comp rx=0 tx=0).stop 2023-11-24T06:10:22.207 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:22.203+0000 7f935d823640 1 -- 172.21.15.92:0/1938453829 shutdown_connections 2023-11-24T06:10:22.207 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:22.203+0000 7f935d823640 1 --2- 172.21.15.92:0/1938453829 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f935815f9c0 0x7f93580cf7d0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:22.207 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:22.203+0000 7f935d823640 1 --2- 172.21.15.92:0/1938453829 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f935813e360 0x7f93580cf290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:22.207 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:22.203+0000 7f935d823640 1 -- 172.21.15.92:0/1938453829 >> 172.21.15.92:0/1938453829 conn(0x7f935806a800 msgr2=0x7f935806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:22.207 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:22.203+0000 7f935d823640 1 -- 172.21.15.92:0/1938453829 shutdown_connections 2023-11-24T06:10:22.207 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:22.203+0000 7f935d823640 1 -- 172.21.15.92:0/1938453829 wait complete. 2023-11-24T06:10:22.208 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:22.205+0000 7f935d823640 1 Processor -- start 2023-11-24T06:10:22.208 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:22.205+0000 7f935d823640 1 -- start start 2023-11-24T06:10:22.210 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:22.206+0000 7f935d823640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f935813df80 0x7f93581f5d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:22.210 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:22.206+0000 7f935d823640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f935813e360 0x7f93580b3470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:22.211 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:22.206+0000 7f935d823640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f935815f9c0 0x7f935813cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:22.211 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:22.206+0000 7f935d823640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93581f4cd0 con 0x7f935813df80 2023-11-24T06:10:22.211 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:22.206+0000 7f935d823640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93581f4fb0 con 0x7f935815f9c0 2023-11-24T06:10:22.211 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:22.206+0000 7f935d823640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93581f5290 con 0x7f935813e360 2023-11-24T06:10:22.211 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:22.206+0000 7f9356ffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f935813df80 0x7f93581f5d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:22.211 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:22.206+0000 7f9356ffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f935813df80 0x7f93581f5d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41804/0 (socket says 172.21.15.92:41804) 2023-11-24T06:10:22.211 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:22.206+0000 7f9356ffd640 1 -- 172.21.15.92:0/1096355258 learned_addr learned my addr 172.21.15.92:0/1096355258 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:10:22.211 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:22.206+0000 7f93577fe640 1 --2- 172.21.15.92:0/1096355258 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f935815f9c0 0x7f935813cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:22.211 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:22.206+0000 7f9356ffd640 1 -- 172.21.15.92:0/1096355258 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f935813e360 msgr2=0x7f93580b3470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:22.211 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:22.206+0000 7f93567fc640 1 --2- 172.21.15.92:0/1096355258 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f935813e360 0x7f93580b3470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:22.211 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:22.206+0000 7f9356ffd640 1 --2- 172.21.15.92:0/1096355258 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f935813e360 0x7f93580b3470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:22.212 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:22.206+0000 7f9356ffd640 1 -- 172.21.15.92:0/1096355258 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f935815f9c0 msgr2=0x7f935813cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:22.212 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:22.206+0000 7f9356ffd640 1 --2- 172.21.15.92:0/1096355258 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f935815f9c0 0x7f935813cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:22.212 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:22.206+0000 7f9356ffd640 1 -- 172.21.15.92:0/1096355258 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9350009b30 con 0x7f935813df80 2023-11-24T06:10:22.212 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:22.207+0000 7f9356ffd640 1 --2- 172.21.15.92:0/1096355258 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f935813df80 0x7f93581f5d70 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f934800c400 tx=0x7f934800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:22.212 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:22.207+0000 7f933ffff640 1 -- 172.21.15.92:0/1096355258 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93480052d0 con 0x7f935813df80 2023-11-24T06:10:22.212 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:22.207+0000 7f933ffff640 1 -- 172.21.15.92:0/1096355258 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9348005430 con 0x7f935813df80 2023-11-24T06:10:22.212 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:22.207+0000 7f933ffff640 1 -- 172.21.15.92:0/1096355258 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9348003830 con 0x7f935813df80 2023-11-24T06:10:22.212 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:22.207+0000 7f935d823640 1 -- 172.21.15.92:0/1096355258 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93581f5570 con 0x7f935813df80 2023-11-24T06:10:22.212 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:22.207+0000 7f935d823640 1 -- 172.21.15.92:0/1096355258 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93581f62b0 con 0x7f935813df80 2023-11-24T06:10:22.213 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:22.208+0000 7f935d823640 1 -- 172.21.15.92:0/1096355258 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f935806d920 con 0x7f935813df80 2023-11-24T06:10:22.213 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:22.209+0000 7f933ffff640 1 -- 172.21.15.92:0/1096355258 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 4) v1 ==== 89175+0+0 (secure 0 0 0) 0x7f9348003990 con 0x7f935813df80 2023-11-24T06:10:22.213 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:22.210+0000 7f933ffff640 1 --2- 172.21.15.92:0/1096355258 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f934406adb0 0x7f934406d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:22.213 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:22.210+0000 7f933ffff640 1 -- 172.21.15.92:0/1096355258 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f93480887d0 con 0x7f935813df80 2023-11-24T06:10:22.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:22.210+0000 7f93567fc640 1 --2- 172.21.15.92:0/1096355258 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f934406adb0 0x7f934406d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:22.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:22.211+0000 7f93567fc640 1 --2- 172.21.15.92:0/1096355258 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f934406adb0 0x7f934406d270 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f935813df50 tx=0x7f934c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:22.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:22.215+0000 7f933ffff640 1 -- 172.21.15.92:0/1096355258 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9348053c10 con 0x7f935813df80 2023-11-24T06:10:22.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:22.406+0000 7f935d823640 1 -- 172.21.15.92:0/1096355258 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f93580675d0 con 0x7f935813df80 2023-11-24T06:10:22.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:22.407+0000 7f933ffff640 1 -- 172.21.15.92:0/1096355258 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v16) v1 ==== 84+0+65 (secure 0 0 0) 0x7f9348017090 con 0x7f935813df80 2023-11-24T06:10:22.410 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-24T06:10:22.411 INFO:teuthology.orchestra.run.smithi092.stdout:[ 2023-11-24T06:10:22.411 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-24T06:10:22.411 INFO:teuthology.orchestra.run.smithi092.stdout: "poolnum": 1, 2023-11-24T06:10:22.411 INFO:teuthology.orchestra.run.smithi092.stdout: "poolname": ".mgr" 2023-11-24T06:10:22.411 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-24T06:10:22.411 INFO:teuthology.orchestra.run.smithi092.stdout:] 2023-11-24T06:10:22.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:22.409+0000 7f935d823640 1 -- 172.21.15.92:0/1096355258 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f934406adb0 msgr2=0x7f934406d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:22.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:22.409+0000 7f935d823640 1 --2- 172.21.15.92:0/1096355258 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f934406adb0 0x7f934406d270 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f935813df50 tx=0x7f934c008040 comp rx=0 tx=0).stop 2023-11-24T06:10:22.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:22.409+0000 7f935d823640 1 -- 172.21.15.92:0/1096355258 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f935813df80 msgr2=0x7f93581f5d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:22.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:22.409+0000 7f935d823640 1 --2- 172.21.15.92:0/1096355258 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f935813df80 0x7f93581f5d70 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f934800c400 tx=0x7f934800c8c0 comp rx=0 tx=0).stop 2023-11-24T06:10:22.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:22.410+0000 7f935d823640 1 -- 172.21.15.92:0/1096355258 shutdown_connections 2023-11-24T06:10:22.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:22.410+0000 7f935d823640 1 --2- 172.21.15.92:0/1096355258 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f934406adb0 0x7f934406d270 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:22.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:22.410+0000 7f935d823640 1 --2- 172.21.15.92:0/1096355258 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f935815f9c0 0x7f935813cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:22.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:22.410+0000 7f935d823640 1 --2- 172.21.15.92:0/1096355258 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f935813e360 0x7f93580b3470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:22.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:22.410+0000 7f935d823640 1 --2- 172.21.15.92:0/1096355258 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f935813df80 0x7f93581f5d70 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:22.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:22.410+0000 7f935d823640 1 -- 172.21.15.92:0/1096355258 >> 172.21.15.92:0/1096355258 conn(0x7f935806a800 msgr2=0x7f93580aff30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:22.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:22.410+0000 7f935d823640 1 -- 172.21.15.92:0/1096355258 shutdown_connections 2023-11-24T06:10:22.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:22.410+0000 7f935d823640 1 -- 172.21.15.92:0/1096355258 wait complete. 2023-11-24T06:10:22.431 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-11-24T06:10:22.431 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-11-24T06:10:22.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:22.647+0000 7f69e095c640 1 Processor -- start 2023-11-24T06:10:22.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:22.647+0000 7f69e095c640 1 -- start start 2023-11-24T06:10:22.652 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:22.648+0000 7f69e095c640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f69dc146650 0x7f69dc0b2830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:22.652 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:22.649+0000 7f69e095c640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f69dc15e030 0x7f69dc15e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:22.652 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:22.649+0000 7f69e095c640 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69dc15dba0 con 0x7f69dc0726f0 2023-11-24T06:10:22.652 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:22.649+0000 7f69e095c640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69dc0cd130 con 0x7f69dc146650 2023-11-24T06:10:22.652 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:22.649+0000 7f69e095c640 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69dc06f800 con 0x7f69dc15e030 2023-11-24T06:10:22.652 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:22.649+0000 7f69daffd640 1 --1- >> v1:172.21.15.118:6789/0 conn(0x7f69dc0726f0 0x7f69dc0cca20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.118:6789/0 says I am v1:172.21.15.92:45170/0 (socket says 172.21.15.92:45170) 2023-11-24T06:10:22.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:22.649+0000 7f69daffd640 1 -- 172.21.15.92:0/1403945604 learned_addr learned my addr 172.21.15.92:0/1403945604 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-24T06:10:22.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:22.649+0000 7f69dbfff640 1 --2- 172.21.15.92:0/1403945604 >> v2:172.21.15.118:3300/0 conn(0x7f69dc15e030 0x7f69dc15e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:22.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:22.649+0000 7f69da7fc640 1 -- 172.21.15.92:0/1403945604 <== mon.1 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2037383234 0 0) 0x7f69dc15dba0 con 0x7f69dc0726f0 2023-11-24T06:10:22.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:22.649+0000 7f69da7fc640 1 -- 172.21.15.92:0/1403945604 --> v1:172.21.15.118:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f69c4003520 con 0x7f69dc0726f0 2023-11-24T06:10:22.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:22.649+0000 7f69dbfff640 1 -- 172.21.15.92:0/1403945604 >> v2:172.21.15.92:3300/0 conn(0x7f69dc146650 msgr2=0x7f69dc0b2830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:22.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:22.649+0000 7f69dbfff640 1 --2- 172.21.15.92:0/1403945604 >> v2:172.21.15.92:3300/0 conn(0x7f69dc146650 0x7f69dc0b2830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:22.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:22.649+0000 7f69dbfff640 1 -- 172.21.15.92:0/1403945604 >> v1:172.21.15.118:6789/0 conn(0x7f69dc0726f0 legacy=0x7f69dc0cca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:22.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:22.650+0000 7f69dbfff640 1 -- 172.21.15.92:0/1403945604 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69dc06f240 con 0x7f69dc15e030 2023-11-24T06:10:22.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:22.651+0000 7f69dbfff640 1 --2- 172.21.15.92:0/1403945604 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f69dc15e030 0x7f69dc15e4e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f69dc073ff0 tx=0x7f69d402bed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3820edb413288ee7 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:22.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:22.651+0000 7f69da7fc640 1 -- 172.21.15.92:0/1403945604 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69d402cc80 con 0x7f69dc15e030 2023-11-24T06:10:22.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:22.651+0000 7f69da7fc640 1 -- 172.21.15.92:0/1403945604 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69d402cde0 con 0x7f69dc15e030 2023-11-24T06:10:22.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:22.651+0000 7f69da7fc640 1 -- 172.21.15.92:0/1403945604 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69d4034660 con 0x7f69dc15e030 2023-11-24T06:10:22.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:22.652+0000 7f69e095c640 1 -- 172.21.15.92:0/1403945604 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f69dc15e030 msgr2=0x7f69dc15e4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:22.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:22.652+0000 7f69e095c640 1 --2- 172.21.15.92:0/1403945604 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f69dc15e030 0x7f69dc15e4e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f69dc073ff0 tx=0x7f69d402bed0 comp rx=0 tx=0).stop 2023-11-24T06:10:22.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:22.652+0000 7f69e095c640 1 -- 172.21.15.92:0/1403945604 shutdown_connections 2023-11-24T06:10:22.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:22.652+0000 7f69e095c640 1 --2- 172.21.15.92:0/1403945604 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f69dc15e030 0x7f69dc15e4e0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:22.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:22.652+0000 7f69e095c640 1 --2- 172.21.15.92:0/1403945604 >> v2:172.21.15.92:3300/0 conn(0x7f69dc146650 0x7f69dc0b2830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:22.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:22.652+0000 7f69e095c640 1 -- 172.21.15.92:0/1403945604 >> 172.21.15.92:0/1403945604 conn(0x7f69dc0b1d40 msgr2=0x7f69dc0b2120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:22.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:22.652+0000 7f69e095c640 1 -- 172.21.15.92:0/1403945604 shutdown_connections 2023-11-24T06:10:22.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:22.652+0000 7f69e095c640 1 -- 172.21.15.92:0/1403945604 wait complete. 2023-11-24T06:10:22.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:22.654+0000 7f69e095c640 1 Processor -- start 2023-11-24T06:10:22.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:22.654+0000 7f69e095c640 1 -- start start 2023-11-24T06:10:22.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:22.655+0000 7f69e095c640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f69dc0726f0 0x7f69dc15cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:22.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:22.655+0000 7f69e095c640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f69dc146650 0x7f69dc0ac8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:22.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:22.655+0000 7f69e095c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f69dc15e030 0x7f69dc1343c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:22.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:22.655+0000 7f69e095c640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69dc0b47f0 con 0x7f69dc15e030 2023-11-24T06:10:22.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:22.655+0000 7f69e095c640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69dc141860 con 0x7f69dc146650 2023-11-24T06:10:22.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:22.655+0000 7f69e095c640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69dc15d290 con 0x7f69dc0726f0 2023-11-24T06:10:22.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:22.656+0000 7f69dbfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f69dc15e030 0x7f69dc1343c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:22.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:22.656+0000 7f69dbfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f69dc15e030 0x7f69dc1343c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41832/0 (socket says 172.21.15.92:41832) 2023-11-24T06:10:22.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:22.656+0000 7f69dbfff640 1 -- 172.21.15.92:0/3050555671 learned_addr learned my addr 172.21.15.92:0/3050555671 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:10:22.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:22.656+0000 7f69daffd640 1 --2- 172.21.15.92:0/3050555671 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f69dc146650 0x7f69dc0ac8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:22.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:22.656+0000 7f69db7fe640 1 --2- 172.21.15.92:0/3050555671 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f69dc0726f0 0x7f69dc15cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:22.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:22.656+0000 7f69daffd640 1 -- 172.21.15.92:0/3050555671 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f69dc0726f0 msgr2=0x7f69dc15cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:22.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:22.656+0000 7f69daffd640 1 --2- 172.21.15.92:0/3050555671 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f69dc0726f0 0x7f69dc15cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:22.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:22.656+0000 7f69daffd640 1 -- 172.21.15.92:0/3050555671 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f69dc15e030 msgr2=0x7f69dc1343c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:22.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:22.656+0000 7f69daffd640 1 --2- 172.21.15.92:0/3050555671 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f69dc15e030 0x7f69dc1343c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:22.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:22.656+0000 7f69daffd640 1 -- 172.21.15.92:0/3050555671 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69d4009a70 con 0x7f69dc146650 2023-11-24T06:10:22.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:22.657+0000 7f69daffd640 1 --2- 172.21.15.92:0/3050555671 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f69dc146650 0x7f69dc0ac8d0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f69d000c390 tx=0x7f69d0001e70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:22.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:22.657+0000 7f69c3fff640 1 -- 172.21.15.92:0/3050555671 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69d0005ce0 con 0x7f69dc146650 2023-11-24T06:10:22.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:22.657+0000 7f69c3fff640 1 -- 172.21.15.92:0/3050555671 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69d0005e40 con 0x7f69dc146650 2023-11-24T06:10:22.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:22.657+0000 7f69c3fff640 1 -- 172.21.15.92:0/3050555671 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69d00072d0 con 0x7f69dc146650 2023-11-24T06:10:22.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:22.657+0000 7f69e095c640 1 -- 172.21.15.92:0/3050555671 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69dc0b3900 con 0x7f69dc146650 2023-11-24T06:10:22.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:22.657+0000 7f69e095c640 1 -- 172.21.15.92:0/3050555671 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69dc0b3ec0 con 0x7f69dc146650 2023-11-24T06:10:22.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:22.658+0000 7f69e095c640 1 -- 172.21.15.92:0/3050555671 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69dc067b60 con 0x7f69dc146650 2023-11-24T06:10:22.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:22.660+0000 7f69c3fff640 1 -- 172.21.15.92:0/3050555671 <== mon.1 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f69d00074f0 con 0x7f69dc146650 2023-11-24T06:10:22.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:22.661+0000 7f69c3fff640 1 --2- 172.21.15.92:0/3050555671 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f69c806ae90 0x7f69c806d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:22.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:22.661+0000 7f69c3fff640 1 -- 172.21.15.92:0/3050555671 <== mon.1 v2:172.21.15.118:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f69d0088b50 con 0x7f69dc146650 2023-11-24T06:10:22.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:22.661+0000 7f69db7fe640 1 --2- 172.21.15.92:0/3050555671 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f69c806ae90 0x7f69c806d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:22.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:22.662+0000 7f69db7fe640 1 --2- 172.21.15.92:0/3050555671 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f69c806ae90 0x7f69c806d350 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f69cc0098e0 tx=0x7f69cc009060 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:22.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:22.665+0000 7f69c3fff640 1 -- 172.21.15.92:0/3050555671 <== mon.1 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f69d00540e0 con 0x7f69dc146650 2023-11-24T06:10:22.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:22.856+0000 7f69e095c640 1 -- 172.21.15.92:0/3050555671 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f69dc06fb80 con 0x7f69dc146650 2023-11-24T06:10:23.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:23.659+0000 7f69c3fff640 1 -- 172.21.15.92:0/3050555671 <== mon.1 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v17) v1 ==== 138+0+0 (secure 0 0 0) 0x7f69d0057700 con 0x7f69dc146650 2023-11-24T06:10:23.664 INFO:teuthology.orchestra.run.smithi092.stderr:pool 'cephfs_metadata' created 2023-11-24T06:10:23.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:23.664+0000 7f69e095c640 1 -- 172.21.15.92:0/3050555671 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f69c806ae90 msgr2=0x7f69c806d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:23.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:23.664+0000 7f69e095c640 1 --2- 172.21.15.92:0/3050555671 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f69c806ae90 0x7f69c806d350 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f69cc0098e0 tx=0x7f69cc009060 comp rx=0 tx=0).stop 2023-11-24T06:10:23.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:23.664+0000 7f69e095c640 1 -- 172.21.15.92:0/3050555671 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f69dc146650 msgr2=0x7f69dc0ac8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:23.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:23.664+0000 7f69e095c640 1 --2- 172.21.15.92:0/3050555671 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f69dc146650 0x7f69dc0ac8d0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f69d000c390 tx=0x7f69d0001e70 comp rx=0 tx=0).stop 2023-11-24T06:10:23.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:23.664+0000 7f69e095c640 1 -- 172.21.15.92:0/3050555671 shutdown_connections 2023-11-24T06:10:23.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:23.664+0000 7f69e095c640 1 --2- 172.21.15.92:0/3050555671 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f69c806ae90 0x7f69c806d350 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:23.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:23.664+0000 7f69e095c640 1 --2- 172.21.15.92:0/3050555671 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f69dc15e030 0x7f69dc1343c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:23.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:23.664+0000 7f69e095c640 1 --2- 172.21.15.92:0/3050555671 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f69dc146650 0x7f69dc0ac8d0 secure :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f69d000c390 tx=0x7f69d0001e70 comp rx=0 tx=0).stop 2023-11-24T06:10:23.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:23.664+0000 7f69e095c640 1 --2- 172.21.15.92:0/3050555671 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f69dc0726f0 0x7f69dc15cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:23.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:23.664+0000 7f69e095c640 1 -- 172.21.15.92:0/3050555671 >> 172.21.15.92:0/3050555671 conn(0x7f69dc0b1d40 msgr2=0x7f69dc145cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:23.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:23.665+0000 7f69e095c640 1 -- 172.21.15.92:0/3050555671 shutdown_connections 2023-11-24T06:10:23.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:23.666+0000 7f69e095c640 1 -- 172.21.15.92:0/3050555671 wait complete. 2023-11-24T06:10:23.692 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-11-24T06:10:23.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:23.893+0000 7f525a7ce640 1 Processor -- start 2023-11-24T06:10:23.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:23.893+0000 7f525a7ce640 1 -- start start 2023-11-24T06:10:23.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:23.895+0000 7f525a7ce640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f525415dd40 0x7f525415e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:23.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:23.895+0000 7f525a7ce640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f525415bed0 0x7f525415c360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:23.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:23.895+0000 7f525a7ce640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f525415e6e0 con 0x7f5254157dc0 2023-11-24T06:10:23.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:23.895+0000 7f525a7ce640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52541581a0 con 0x7f525415bed0 2023-11-24T06:10:23.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:23.895+0000 7f525a7ce640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f525415ba40 con 0x7f525415dd40 2023-11-24T06:10:23.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:23.895+0000 7f5253fff640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f5254157dc0 0x7f5254070cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:40640/0 (socket says 172.21.15.92:40640) 2023-11-24T06:10:23.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:23.895+0000 7f5253fff640 1 -- 172.21.15.92:0/4009040916 learned_addr learned my addr 172.21.15.92:0/4009040916 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-24T06:10:23.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:23.895+0000 7f5258d6b640 1 --2- 172.21.15.92:0/4009040916 >> v2:172.21.15.92:3300/0 conn(0x7f525415bed0 0x7f525415c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:23.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:23.895+0000 7f52537fe640 1 --2- 172.21.15.92:0/4009040916 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f525415dd40 0x7f525415e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:23.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:23.895+0000 7f5252ffd640 1 -- 172.21.15.92:0/4009040916 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1195860096 0 0) 0x7f525415e6e0 con 0x7f5254157dc0 2023-11-24T06:10:23.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:23.895+0000 7f5252ffd640 1 -- 172.21.15.92:0/4009040916 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f523c003520 con 0x7f5254157dc0 2023-11-24T06:10:23.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:23.896+0000 7f5258d6b640 1 -- 172.21.15.92:0/4009040916 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f525415dd40 msgr2=0x7f525415e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:23.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:23.896+0000 7f5258d6b640 1 --2- 172.21.15.92:0/4009040916 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f525415dd40 0x7f525415e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:23.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:23.896+0000 7f5258d6b640 1 -- 172.21.15.92:0/4009040916 >> v1:172.21.15.92:6789/0 conn(0x7f5254157dc0 legacy=0x7f5254070cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:23.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:23.896+0000 7f5258d6b640 1 -- 172.21.15.92:0/4009040916 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f525415eb10 con 0x7f525415bed0 2023-11-24T06:10:23.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:23.897+0000 7f5258d6b640 1 --2- 172.21.15.92:0/4009040916 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f525415bed0 0x7f525415c360 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f524c009dc0 tx=0x7f524c02be90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=362462b9c50a729a server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:23.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:23.897+0000 7f5252ffd640 1 -- 172.21.15.92:0/4009040916 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f524c02cc40 con 0x7f525415bed0 2023-11-24T06:10:23.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:23.897+0000 7f5252ffd640 1 -- 172.21.15.92:0/4009040916 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f524c02cda0 con 0x7f525415bed0 2023-11-24T06:10:23.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:23.897+0000 7f5252ffd640 1 -- 172.21.15.92:0/4009040916 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f524c034640 con 0x7f525415bed0 2023-11-24T06:10:23.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:23.898+0000 7f525a7ce640 1 -- 172.21.15.92:0/4009040916 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f525415bed0 msgr2=0x7f525415c360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:23.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:23.898+0000 7f525a7ce640 1 --2- 172.21.15.92:0/4009040916 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f525415bed0 0x7f525415c360 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f524c009dc0 tx=0x7f524c02be90 comp rx=0 tx=0).stop 2023-11-24T06:10:23.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:23.898+0000 7f525a7ce640 1 -- 172.21.15.92:0/4009040916 shutdown_connections 2023-11-24T06:10:23.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:23.898+0000 7f525a7ce640 1 --2- 172.21.15.92:0/4009040916 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f525415bed0 0x7f525415c360 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:23.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:23.898+0000 7f525a7ce640 1 --2- 172.21.15.92:0/4009040916 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f525415dd40 0x7f525415e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:23.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:23.898+0000 7f525a7ce640 1 -- 172.21.15.92:0/4009040916 >> 172.21.15.92:0/4009040916 conn(0x7f52540b0be0 msgr2=0x7f52540b0fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:23.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:23.898+0000 7f525a7ce640 1 -- 172.21.15.92:0/4009040916 shutdown_connections 2023-11-24T06:10:23.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:23.898+0000 7f525a7ce640 1 -- 172.21.15.92:0/4009040916 wait complete. 2023-11-24T06:10:23.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:23.900+0000 7f525a7ce640 1 Processor -- start 2023-11-24T06:10:23.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:23.900+0000 7f525a7ce640 1 -- start start 2023-11-24T06:10:23.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:23.901+0000 7f525a7ce640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5254157dc0 0x7f52540cd630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:23.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:23.901+0000 7f525a7ce640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f525415bed0 0x7f52541e47c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:23.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:23.901+0000 7f525a7ce640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f525415dd40 0x7f52540ceeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:23.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:23.901+0000 7f525a7ce640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52540cea90 con 0x7f525415bed0 2023-11-24T06:10:23.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:23.901+0000 7f525a7ce640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52540cdb70 con 0x7f525415dd40 2023-11-24T06:10:23.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:23.901+0000 7f525a7ce640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52540cde10 con 0x7f5254157dc0 2023-11-24T06:10:23.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:23.901+0000 7f52537fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f525415bed0 0x7f52541e47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:23.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:23.901+0000 7f52537fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f525415bed0 0x7f52541e47c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41844/0 (socket says 172.21.15.92:41844) 2023-11-24T06:10:23.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:23.901+0000 7f52537fe640 1 -- 172.21.15.92:0/3159373473 learned_addr learned my addr 172.21.15.92:0/3159373473 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:10:23.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:23.901+0000 7f52537fe640 1 -- 172.21.15.92:0/3159373473 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5254157dc0 msgr2=0x7f52540cd630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:23.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:23.901+0000 7f52537fe640 1 --2- 172.21.15.92:0/3159373473 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5254157dc0 0x7f52540cd630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:23.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:23.901+0000 7f52537fe640 1 -- 172.21.15.92:0/3159373473 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f525415dd40 msgr2=0x7f52540ceeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:23.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:23.901+0000 7f52537fe640 1 --2- 172.21.15.92:0/3159373473 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f525415dd40 0x7f52540ceeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:23.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:23.901+0000 7f52537fe640 1 -- 172.21.15.92:0/3159373473 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f524c009a70 con 0x7f525415bed0 2023-11-24T06:10:23.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:23.901+0000 7f52537fe640 1 --2- 172.21.15.92:0/3159373473 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f525415bed0 0x7f52541e47c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f524800e870 tx=0x7f524800ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:23.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:23.902+0000 7f5250ff9640 1 -- 172.21.15.92:0/3159373473 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52480094f0 con 0x7f525415bed0 2023-11-24T06:10:23.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:23.902+0000 7f5250ff9640 1 -- 172.21.15.92:0/3159373473 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f524800c940 con 0x7f525415bed0 2023-11-24T06:10:23.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:23.902+0000 7f5250ff9640 1 -- 172.21.15.92:0/3159373473 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f524800ccf0 con 0x7f525415bed0 2023-11-24T06:10:23.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:23.902+0000 7f525a7ce640 1 -- 172.21.15.92:0/3159373473 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52540ce0f0 con 0x7f525415bed0 2023-11-24T06:10:23.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:23.902+0000 7f525a7ce640 1 -- 172.21.15.92:0/3159373473 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52540ce6b0 con 0x7f525415bed0 2023-11-24T06:10:23.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:23.903+0000 7f525a7ce640 1 -- 172.21.15.92:0/3159373473 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5254067670 con 0x7f525415bed0 2023-11-24T06:10:23.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:23.904+0000 7f5250ff9640 1 -- 172.21.15.92:0/3159373473 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5248013470 con 0x7f525415bed0 2023-11-24T06:10:23.910 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:23.905+0000 7f5250ff9640 1 --2- 172.21.15.92:0/3159373473 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f524006ae40 0x7f524006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:23.910 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:23.905+0000 7f5250ff9640 1 -- 172.21.15.92:0/3159373473 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f52480894c0 con 0x7f525415bed0 2023-11-24T06:10:23.910 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:23.905+0000 7f5253fff640 1 --2- 172.21.15.92:0/3159373473 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f524006ae40 0x7f524006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:23.910 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:23.906+0000 7f5253fff640 1 --2- 172.21.15.92:0/3159373473 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f524006ae40 0x7f524006d300 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f5244000f60 tx=0x7f52440078f0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:23.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:23.910+0000 7f5250ff9640 1 -- 172.21.15.92:0/3159373473 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f52480547e0 con 0x7f525415bed0 2023-11-24T06:10:24.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:24.101+0000 7f525a7ce640 1 -- 172.21.15.92:0/3159373473 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f52540b1760 con 0x7f525415bed0 2023-11-24T06:10:24.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:24.657+0000 7f5250ff9640 1 -- 172.21.15.92:0/3159373473 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v18) v1 ==== 170+0+0 (secure 0 0 0) 0x7f5248057e00 con 0x7f525415bed0 2023-11-24T06:10:24.660 INFO:teuthology.orchestra.run.smithi092.stderr:pool 'cephfs_data' created 2023-11-24T06:10:24.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:24.660+0000 7f525a7ce640 1 -- 172.21.15.92:0/3159373473 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f524006ae40 msgr2=0x7f524006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:24.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:24.660+0000 7f525a7ce640 1 --2- 172.21.15.92:0/3159373473 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f524006ae40 0x7f524006d300 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f5244000f60 tx=0x7f52440078f0 comp rx=0 tx=0).stop 2023-11-24T06:10:24.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:24.660+0000 7f525a7ce640 1 -- 172.21.15.92:0/3159373473 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f525415bed0 msgr2=0x7f52541e47c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:24.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:24.660+0000 7f525a7ce640 1 --2- 172.21.15.92:0/3159373473 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f525415bed0 0x7f52541e47c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f524800e870 tx=0x7f524800ed30 comp rx=0 tx=0).stop 2023-11-24T06:10:24.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:24.660+0000 7f525a7ce640 1 -- 172.21.15.92:0/3159373473 shutdown_connections 2023-11-24T06:10:24.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:24.661+0000 7f525a7ce640 1 --2- 172.21.15.92:0/3159373473 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f524006ae40 0x7f524006d300 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:24.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:24.661+0000 7f525a7ce640 1 --2- 172.21.15.92:0/3159373473 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f525415dd40 0x7f52540ceeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:24.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:24.661+0000 7f525a7ce640 1 --2- 172.21.15.92:0/3159373473 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f525415bed0 0x7f52541e47c0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:24.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:24.661+0000 7f525a7ce640 1 --2- 172.21.15.92:0/3159373473 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5254157dc0 0x7f52540cd630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:24.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:24.661+0000 7f525a7ce640 1 -- 172.21.15.92:0/3159373473 >> 172.21.15.92:0/3159373473 conn(0x7f52540b0be0 msgr2=0x7f5254158f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:24.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:24.661+0000 7f525a7ce640 1 -- 172.21.15.92:0/3159373473 shutdown_connections 2023-11-24T06:10:24.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:24.661+0000 7f525a7ce640 1 -- 172.21.15.92:0/3159373473 wait complete. 2023-11-24T06:10:24.697 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-11-24T06:10:24.879 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:24.876+0000 7fee76d7a640 1 Processor -- start 2023-11-24T06:10:24.879 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:24.876+0000 7fee76d7a640 1 -- start start 2023-11-24T06:10:24.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:24.877+0000 7fee76d7a640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fee70070a30 0x7fee7015bf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:24.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:24.877+0000 7fee76d7a640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee7015f4e0 con 0x7fee700ccb50 2023-11-24T06:10:24.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:24.877+0000 7fee76d7a640 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee70070ee0 con 0x7fee700adf20 2023-11-24T06:10:24.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:24.877+0000 7fee76d7a640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee7015c450 con 0x7fee70070a30 2023-11-24T06:10:24.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:24.878+0000 7fee75577640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fee700ccb50 0x7fee70143d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:40654/0 (socket says 172.21.15.92:40654) 2023-11-24T06:10:24.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:24.878+0000 7fee75577640 1 -- 172.21.15.92:0/2039527438 learned_addr learned my addr 172.21.15.92:0/2039527438 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-24T06:10:24.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:24.878+0000 7fee74d76640 1 -- 172.21.15.92:0/2039527438 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 667970752 0 0) 0x7fee7015f4e0 con 0x7fee700ccb50 2023-11-24T06:10:24.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:24.878+0000 7fee74d76640 1 -- 172.21.15.92:0/2039527438 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fee5c003520 con 0x7fee700ccb50 2023-11-24T06:10:24.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:24.878+0000 7fee76579640 1 --2- 172.21.15.92:0/2039527438 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fee70070a30 0x7fee7015bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:24.884 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:24.878+0000 7fee74d76640 1 -- 172.21.15.92:0/2039527438 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 80861879 0 0) 0x7fee5c003520 con 0x7fee700ccb50 2023-11-24T06:10:24.884 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:24.878+0000 7fee74d76640 1 -- 172.21.15.92:0/2039527438 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fee70070a30 msgr2=0x7fee7015bf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:24.884 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:24.878+0000 7fee74d76640 1 --2- 172.21.15.92:0/2039527438 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fee70070a30 0x7fee7015bf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:24.884 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:24.878+0000 7fee74d76640 1 -- 172.21.15.92:0/2039527438 >> v1:172.21.15.118:6789/0 conn(0x7fee700adf20 legacy=0x7fee700cc440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:24.884 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:24.878+0000 7fee74d76640 1 -- 172.21.15.92:0/2039527438 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee70071130 con 0x7fee700ccb50 2023-11-24T06:10:24.884 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:24.878+0000 7fee74d76640 1 -- 172.21.15.92:0/2039527438 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2607661350 0 0) 0x7fee60001670 con 0x7fee700ccb50 2023-11-24T06:10:24.884 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:24.879+0000 7fee74d76640 1 -- 172.21.15.92:0/2039527438 >> v1:172.21.15.92:6789/0 conn(0x7fee700ccb50 legacy=0x7fee70143d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:24.884 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:24.879+0000 7fee74d76640 1 --2- 172.21.15.92:0/2039527438 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fee70070a30 0x7fee5c0039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:24.884 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:24.879+0000 7fee74d76640 1 --2- 172.21.15.92:0/2039527438 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fee5c003f90 0x7fee5c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:24.884 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:24.879+0000 7fee74d76640 1 --2- 172.21.15.92:0/2039527438 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fee5c004d00 0x7fee5c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:24.884 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:24.879+0000 7fee74d76640 1 -- 172.21.15.92:0/2039527438 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee7015c450 con 0x7fee5c003f90 2023-11-24T06:10:24.884 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:24.879+0000 7fee74d76640 1 -- 172.21.15.92:0/2039527438 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee5c00bb70 con 0x7fee5c004d00 2023-11-24T06:10:24.885 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:24.879+0000 7fee74d76640 1 -- 172.21.15.92:0/2039527438 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee5c00bd30 con 0x7fee70070a30 2023-11-24T06:10:24.885 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:24.879+0000 7fee75577640 1 --2- 172.21.15.92:0/2039527438 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fee5c003f90 0x7fee5c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:24.885 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:24.879+0000 7fee75577640 1 -- 172.21.15.92:0/2039527438 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fee70070a30 msgr2=0x7fee5c0039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:24.885 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:24.879+0000 7fee75577640 1 --2- 172.21.15.92:0/2039527438 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fee70070a30 0x7fee5c0039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:24.885 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:24.879+0000 7fee75577640 1 -- 172.21.15.92:0/2039527438 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fee5c004d00 msgr2=0x7fee5c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:24.885 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:24.879+0000 7fee76579640 1 --2- 172.21.15.92:0/2039527438 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fee5c004d00 0x7fee5c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:24.885 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:24.879+0000 7fee75577640 1 --2- 172.21.15.92:0/2039527438 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fee5c004d00 0x7fee5c005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:24.885 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:24.879+0000 7fee75577640 1 -- 172.21.15.92:0/2039527438 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee5c003520 con 0x7fee5c003f90 2023-11-24T06:10:24.885 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:24.880+0000 7fee75577640 1 --2- 172.21.15.92:0/2039527438 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fee5c003f90 0x7fee5c0047c0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fee60007fb0 tx=0x7fee5c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9977db25b1cc33f3 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:24.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:24.881+0000 7fee74d76640 1 -- 172.21.15.92:0/2039527438 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee60005ad0 con 0x7fee5c003f90 2023-11-24T06:10:24.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:24.881+0000 7fee74d76640 1 -- 172.21.15.92:0/2039527438 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee60005c30 con 0x7fee5c003f90 2023-11-24T06:10:24.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:24.881+0000 7fee74d76640 1 -- 172.21.15.92:0/2039527438 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee6002c790 con 0x7fee5c003f90 2023-11-24T06:10:24.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:24.881+0000 7fee76d7a640 1 -- 172.21.15.92:0/2039527438 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fee5c003f90 msgr2=0x7fee5c0047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:24.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:24.881+0000 7fee76d7a640 1 --2- 172.21.15.92:0/2039527438 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fee5c003f90 0x7fee5c0047c0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fee60007fb0 tx=0x7fee5c0057f0 comp rx=0 tx=0).stop 2023-11-24T06:10:24.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:24.881+0000 7fee76579640 1 -- 172.21.15.92:0/2039527438 reap_dead start 2023-11-24T06:10:24.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:24.881+0000 7fee76d7a640 1 -- 172.21.15.92:0/2039527438 shutdown_connections 2023-11-24T06:10:24.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:24.881+0000 7fee76d7a640 1 -- 172.21.15.92:0/2039527438 >> 172.21.15.92:0/2039527438 conn(0x7fee7006a7d0 msgr2=0x7fee7006abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:24.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:24.881+0000 7fee76d7a640 1 -- 172.21.15.92:0/2039527438 shutdown_connections 2023-11-24T06:10:24.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:24.881+0000 7fee76d7a640 1 -- 172.21.15.92:0/2039527438 wait complete. 2023-11-24T06:10:24.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:24.883+0000 7fee76d7a640 1 Processor -- start 2023-11-24T06:10:24.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:24.883+0000 7fee76d7a640 1 -- start start 2023-11-24T06:10:24.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:24.884+0000 7fee76d7a640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fee5c003f90 0x7fee7014f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:24.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:24.884+0000 7fee76d7a640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fee700cbea0 0x7fee700cc320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:24.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:24.884+0000 7fee76d7a640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fee701522c0 0x7fee70152740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:24.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:24.884+0000 7fee76d7a640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee70153cf0 con 0x7fee5c003f90 2023-11-24T06:10:24.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:24.884+0000 7fee76d7a640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee700cc860 con 0x7fee700cbea0 2023-11-24T06:10:24.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:24.884+0000 7fee76d7a640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee70152c80 con 0x7fee701522c0 2023-11-24T06:10:24.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:24.884+0000 7fee75d78640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fee5c003f90 0x7fee7014f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:24.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:24.884+0000 7fee75d78640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fee5c003f90 0x7fee7014f740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41852/0 (socket says 172.21.15.92:41852) 2023-11-24T06:10:24.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:24.884+0000 7fee75d78640 1 -- 172.21.15.92:0/868420946 learned_addr learned my addr 172.21.15.92:0/868420946 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:10:24.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:24.885+0000 7fee75577640 1 --2- 172.21.15.92:0/868420946 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fee700cbea0 0x7fee700cc320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:24.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:24.885+0000 7fee76579640 1 --2- 172.21.15.92:0/868420946 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fee701522c0 0x7fee70152740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:24.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:24.885+0000 7fee75d78640 1 -- 172.21.15.92:0/868420946 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fee701522c0 msgr2=0x7fee70152740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:24.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:24.885+0000 7fee75d78640 1 --2- 172.21.15.92:0/868420946 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fee701522c0 0x7fee70152740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:24.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:24.885+0000 7fee75d78640 1 -- 172.21.15.92:0/868420946 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fee700cbea0 msgr2=0x7fee700cc320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:24.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:24.885+0000 7fee75d78640 1 --2- 172.21.15.92:0/868420946 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fee700cbea0 0x7fee700cc320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:24.891 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:24.885+0000 7fee75d78640 1 -- 172.21.15.92:0/868420946 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee701f42d0 con 0x7fee5c003f90 2023-11-24T06:10:24.891 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:24.885+0000 7fee75d78640 1 --2- 172.21.15.92:0/868420946 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fee5c003f90 0x7fee7014f740 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fee6c00c7b0 tx=0x7fee6c00cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:24.891 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:24.885+0000 7fee667fc640 1 -- 172.21.15.92:0/868420946 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee6c005270 con 0x7fee5c003f90 2023-11-24T06:10:24.891 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:24.885+0000 7fee667fc640 1 -- 172.21.15.92:0/868420946 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee6c0053d0 con 0x7fee5c003f90 2023-11-24T06:10:24.891 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:24.885+0000 7fee76d7a640 1 -- 172.21.15.92:0/868420946 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee70153740 con 0x7fee5c003f90 2023-11-24T06:10:24.891 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:24.885+0000 7fee667fc640 1 -- 172.21.15.92:0/868420946 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee6c003810 con 0x7fee5c003f90 2023-11-24T06:10:24.891 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:24.885+0000 7fee76d7a640 1 -- 172.21.15.92:0/868420946 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee70153180 con 0x7fee5c003f90 2023-11-24T06:10:24.892 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:24.886+0000 7fee76d7a640 1 -- 172.21.15.92:0/868420946 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee70153a50 con 0x7fee5c003f90 2023-11-24T06:10:24.892 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:24.888+0000 7fee667fc640 1 -- 172.21.15.92:0/868420946 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fee6c003970 con 0x7fee5c003f90 2023-11-24T06:10:24.892 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:24.888+0000 7fee667fc640 1 --2- 172.21.15.92:0/868420946 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fee5406ae70 0x7fee5406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:24.892 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:24.889+0000 7fee667fc640 1 -- 172.21.15.92:0/868420946 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5609+0+0 (secure 0 0 0) 0x7fee6c0889f0 con 0x7fee5c003f90 2023-11-24T06:10:24.892 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:24.889+0000 7fee75577640 1 --2- 172.21.15.92:0/868420946 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fee5406ae70 0x7fee5406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:24.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:24.889+0000 7fee75577640 1 --2- 172.21.15.92:0/868420946 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fee5406ae70 0x7fee5406d330 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fee60034000 tx=0x7fee60037040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:24.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:24.893+0000 7fee667fc640 1 -- 172.21.15.92:0/868420946 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fee70153a50 con 0x7fee5c003f90 2023-11-24T06:10:25.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:25.112+0000 7fee76d7a640 1 -- 172.21.15.92:0/868420946 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fee70153a50 con 0x7fee5c003f90 2023-11-24T06:10:25.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:25.700+0000 7fee667fc640 1 -- 172.21.15.92:0/868420946 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) v1 ==== 172+0+0 (secure 0 0 0) 0x7fee70153a50 con 0x7fee5c003f90 2023-11-24T06:10:25.705 INFO:teuthology.orchestra.run.smithi092.stderr:new fs with metadata pool 2 and data pool 3 2023-11-24T06:10:25.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:25.708+0000 7fee76d7a640 1 -- 172.21.15.92:0/868420946 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fee5406ae70 msgr2=0x7fee5406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:25.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:25.708+0000 7fee76d7a640 1 --2- 172.21.15.92:0/868420946 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fee5406ae70 0x7fee5406d330 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fee60034000 tx=0x7fee60037040 comp rx=0 tx=0).stop 2023-11-24T06:10:25.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:25.708+0000 7fee76d7a640 1 -- 172.21.15.92:0/868420946 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fee5c003f90 msgr2=0x7fee7014f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:25.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:25.708+0000 7fee76d7a640 1 --2- 172.21.15.92:0/868420946 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fee5c003f90 0x7fee7014f740 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fee6c00c7b0 tx=0x7fee6c00cc70 comp rx=0 tx=0).stop 2023-11-24T06:10:25.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:25.708+0000 7fee76d7a640 1 -- 172.21.15.92:0/868420946 shutdown_connections 2023-11-24T06:10:25.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:25.708+0000 7fee76d7a640 1 --2- 172.21.15.92:0/868420946 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fee5406ae70 0x7fee5406d330 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:25.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:25.708+0000 7fee76d7a640 1 --2- 172.21.15.92:0/868420946 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fee701522c0 0x7fee70152740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:25.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:25.708+0000 7fee76d7a640 1 --2- 172.21.15.92:0/868420946 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fee700cbea0 0x7fee700cc320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:25.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:25.708+0000 7fee76d7a640 1 --2- 172.21.15.92:0/868420946 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fee5c003f90 0x7fee7014f740 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:25.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:25.708+0000 7fee76d7a640 1 -- 172.21.15.92:0/868420946 >> 172.21.15.92:0/868420946 conn(0x7fee7006a7d0 msgr2=0x7fee7015d000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:25.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:25.711+0000 7fee76d7a640 1 -- 172.21.15.92:0/868420946 shutdown_connections 2023-11-24T06:10:25.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:25.711+0000 7fee76d7a640 1 -- 172.21.15.92:0/868420946 wait complete. 2023-11-24T06:10:25.743 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:10:25.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:25.910+0000 7f3d6339e640 1 Processor -- start 2023-11-24T06:10:25.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:25.910+0000 7f3d6339e640 1 -- start start 2023-11-24T06:10:25.915 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:25.911+0000 7f3d6339e640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f3d5c0cd240 0x7f3d5c0b3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:25.915 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:25.911+0000 7f3d6339e640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f3d5c09a170 0x7f3d5c0cf640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:25.915 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:25.911+0000 7f3d6339e640 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d5c06cea0 con 0x7f3d5c06c6b0 2023-11-24T06:10:25.915 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:25.911+0000 7f3d6339e640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d5c06cbc0 con 0x7f3d5c0cd240 2023-11-24T06:10:25.915 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:25.911+0000 7f3d6339e640 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d5c06f350 con 0x7f3d5c09a170 2023-11-24T06:10:25.915 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:25.912+0000 7f3d6113a640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f3d5c0cd240 0x7f3d5c0b3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:25.916 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:25.912+0000 7f3d6113a640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f3d5c0cd240 0x7f3d5c0b3820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41864/0 (socket says 172.21.15.92:41864) 2023-11-24T06:10:25.916 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:25.912+0000 7f3d6113a640 1 -- 172.21.15.92:0/1081340873 learned_addr learned my addr 172.21.15.92:0/1081340873 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:10:25.916 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:25.912+0000 7f3d6113a640 1 -- 172.21.15.92:0/1081340873 >> v2:172.21.15.118:3300/0 conn(0x7f3d5c09a170 msgr2=0x7f3d5c0cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:25.916 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:25.912+0000 7f3d6113a640 1 --2- 172.21.15.92:0/1081340873 >> v2:172.21.15.118:3300/0 conn(0x7f3d5c09a170 0x7f3d5c0cf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:25.916 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:25.912+0000 7f3d6113a640 1 -- 172.21.15.92:0/1081340873 >> v1:172.21.15.118:6789/0 conn(0x7f3d5c06c6b0 legacy=0x7f3d5c099a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:25.916 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:25.912+0000 7f3d6113a640 1 -- 172.21.15.92:0/1081340873 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d5c06f630 con 0x7f3d5c0cd240 2023-11-24T06:10:25.917 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:25.913+0000 7f3d6113a640 1 --2- 172.21.15.92:0/1081340873 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3d5c0cd240 0x7f3d5c0b3820 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f3d5c0cb7f0 tx=0x7f3d5802b930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=857d551138c49e62 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:25.917 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:25.913+0000 7f3d53fff640 1 -- 172.21.15.92:0/1081340873 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d58031890 con 0x7f3d5c0cd240 2023-11-24T06:10:25.917 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:25.913+0000 7f3d53fff640 1 -- 172.21.15.92:0/1081340873 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d580319f0 con 0x7f3d5c0cd240 2023-11-24T06:10:25.917 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:25.913+0000 7f3d53fff640 1 -- 172.21.15.92:0/1081340873 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d58031d80 con 0x7f3d5c0cd240 2023-11-24T06:10:25.918 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:25.914+0000 7f3d6339e640 1 -- 172.21.15.92:0/1081340873 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3d5c0cd240 msgr2=0x7f3d5c0b3820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:25.918 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:25.914+0000 7f3d6339e640 1 --2- 172.21.15.92:0/1081340873 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3d5c0cd240 0x7f3d5c0b3820 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f3d5c0cb7f0 tx=0x7f3d5802b930 comp rx=0 tx=0).stop 2023-11-24T06:10:25.918 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:25.914+0000 7f3d6339e640 1 -- 172.21.15.92:0/1081340873 shutdown_connections 2023-11-24T06:10:25.918 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:25.914+0000 7f3d6339e640 1 --2- 172.21.15.92:0/1081340873 >> v2:172.21.15.118:3300/0 conn(0x7f3d5c09a170 0x7f3d5c0cf640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:25.918 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:25.914+0000 7f3d6339e640 1 --2- 172.21.15.92:0/1081340873 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3d5c0cd240 0x7f3d5c0b3820 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:25.918 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:25.914+0000 7f3d6339e640 1 -- 172.21.15.92:0/1081340873 >> 172.21.15.92:0/1081340873 conn(0x7f3d5c0b2d00 msgr2=0x7f3d5c0b3110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:25.918 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:25.914+0000 7f3d6339e640 1 -- 172.21.15.92:0/1081340873 shutdown_connections 2023-11-24T06:10:25.918 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:25.914+0000 7f3d6339e640 1 -- 172.21.15.92:0/1081340873 wait complete. 2023-11-24T06:10:25.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:25.916+0000 7f3d6339e640 1 Processor -- start 2023-11-24T06:10:25.920 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:25.916+0000 7f3d6339e640 1 -- start start 2023-11-24T06:10:25.922 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:25.917+0000 7f3d6339e640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3d5c06c6b0 0x7f3d5c15aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:25.922 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:25.918+0000 7f3d6339e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3d5c09a170 0x7f3d5c0c5710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:25.922 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:25.918+0000 7f3d6339e640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3d5c0cd240 0x7f3d5c0b5c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:25.922 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:25.918+0000 7f3d6339e640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d5c0c6080 con 0x7f3d5c09a170 2023-11-24T06:10:25.922 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:25.918+0000 7f3d6339e640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d5c0c6360 con 0x7f3d5c0cd240 2023-11-24T06:10:25.922 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:25.918+0000 7f3d6339e640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d5c0c6640 con 0x7f3d5c06c6b0 2023-11-24T06:10:25.922 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:25.918+0000 7f3d60939640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3d5c09a170 0x7f3d5c0c5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:25.923 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:25.918+0000 7f3d60939640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3d5c09a170 0x7f3d5c0c5710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41874/0 (socket says 172.21.15.92:41874) 2023-11-24T06:10:25.923 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:25.918+0000 7f3d60939640 1 -- 172.21.15.92:0/3031696456 learned_addr learned my addr 172.21.15.92:0/3031696456 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:10:25.923 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:25.918+0000 7f3d6113a640 1 --2- 172.21.15.92:0/3031696456 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3d5c06c6b0 0x7f3d5c15aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:25.923 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:25.918+0000 7f3d60939640 1 -- 172.21.15.92:0/3031696456 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3d5c06c6b0 msgr2=0x7f3d5c15aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:25.923 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:25.918+0000 7f3d60939640 1 --2- 172.21.15.92:0/3031696456 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3d5c06c6b0 0x7f3d5c15aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:25.923 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:25.918+0000 7f3d60939640 1 -- 172.21.15.92:0/3031696456 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3d5c0cd240 msgr2=0x7f3d5c0b5c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:25.923 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:25.918+0000 7f3d60939640 1 --2- 172.21.15.92:0/3031696456 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3d5c0cd240 0x7f3d5c0b5c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:25.923 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:25.918+0000 7f3d60939640 1 -- 172.21.15.92:0/3031696456 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d580094d0 con 0x7f3d5c09a170 2023-11-24T06:10:25.923 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:25.919+0000 7f3d60939640 1 --2- 172.21.15.92:0/3031696456 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3d5c09a170 0x7f3d5c0c5710 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f3d4c00c400 tx=0x7f3d4c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:25.923 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:25.919+0000 7f3d51ffb640 1 -- 172.21.15.92:0/3031696456 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d4c0035f0 con 0x7f3d5c09a170 2023-11-24T06:10:25.923 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:25.919+0000 7f3d6339e640 1 -- 172.21.15.92:0/3031696456 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d5c0c6920 con 0x7f3d5c09a170 2023-11-24T06:10:25.923 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:25.919+0000 7f3d51ffb640 1 -- 172.21.15.92:0/3031696456 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d4c003750 con 0x7f3d5c09a170 2023-11-24T06:10:25.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:25.919+0000 7f3d51ffb640 1 -- 172.21.15.92:0/3031696456 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d4c003b40 con 0x7f3d5c09a170 2023-11-24T06:10:25.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:25.919+0000 7f3d6339e640 1 -- 172.21.15.92:0/3031696456 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d5c0c5c50 con 0x7f3d5c09a170 2023-11-24T06:10:25.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:25.920+0000 7f3d6339e640 1 -- 172.21.15.92:0/3031696456 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d5c0675d0 con 0x7f3d5c09a170 2023-11-24T06:10:25.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:25.921+0000 7f3d51ffb640 1 -- 172.21.15.92:0/3031696456 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f3d4c003ca0 con 0x7f3d5c09a170 2023-11-24T06:10:25.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:25.922+0000 7f3d51ffb640 1 --2- 172.21.15.92:0/3031696456 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f3d4006ae40 0x7f3d4006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:25.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:25.922+0000 7f3d51ffb640 1 -- 172.21.15.92:0/3031696456 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3d4c088800 con 0x7f3d5c09a170 2023-11-24T06:10:25.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:25.922+0000 7f3d6113a640 1 --2- 172.21.15.92:0/3031696456 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f3d4006ae40 0x7f3d4006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:25.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:25.923+0000 7f3d6113a640 1 --2- 172.21.15.92:0/3031696456 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f3d4006ae40 0x7f3d4006d300 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f3d58036000 tx=0x7f3d5803c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:25.930 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:25.927+0000 7f3d51ffb640 1 -- 172.21.15.92:0/3031696456 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3d4c053950 con 0x7f3d5c09a170 2023-11-24T06:10:26.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:26.103+0000 7f3d6339e640 1 -- 172.21.15.92:0/3031696456 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3d5c0737d0 con 0x7f3d5c09a170 2023-11-24T06:10:26.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:26.106+0000 7f3d51ffb640 1 -- 172.21.15.92:0/3031696456 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) v1 ==== 74+0+16470 (secure 0 0 0) 0x7f3d4c056f70 con 0x7f3d5c09a170 2023-11-24T06:10:26.110 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-24T06:10:26.110 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":19,"fsid":"2d6dee5f-8aa3-470f-931a-50ade963b1cb","created":"2023-11-24T06:10:04.707204+0000","modified":"2023-11-24T06:10:25.114735+0000","last_up_change":"2023-11-24T06:10:17.598352+0000","last_in_change":"2023-11-24T06:10:09.040793+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:10:17.733098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:10:22.926445+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T06:10:24.151288+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ff37a9fb-135f-471f-b4b0-f7b5772006c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6803","nonce":20560714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6805","nonce":20560714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6809","nonce":20560714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6807","nonce":20560714}]},"public_addr":"172.21.15.92:6803/20560714","cluster_addr":"172.21.15.92:6805/20560714","heartbeat_back_addr":"172.21.15.92:6809/20560714","heartbeat_front_addr":"172.21.15.92:6807/20560714","state":["exists","up"]},{"osd":1,"uuid":"03cdd353-dcf1-4f38-94fe-a795137c06b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6819","nonce":2451239906}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6821","nonce":2451239906}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6825","nonce":2451239906}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6823","nonce":2451239906}]},"public_addr":"172.21.15.92:6819/2451239906","cluster_addr":"172.21.15.92:6821/2451239906","heartbeat_back_addr":"172.21.15.92:6825/2451239906","heartbeat_front_addr":"172.21.15.92:6823/2451239906","state":["exists","up"]},{"osd":2,"uuid":"a4a20d80-5d57-484b-becd-9aa8070dd3aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6811","nonce":1124979277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6813","nonce":1124979277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6817","nonce":1124979277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6815","nonce":1124979277}]},"public_addr":"172.21.15.92:6811/1124979277","cluster_addr":"172.21.15.92:6813/1124979277","heartbeat_back_addr":"172.21.15.92:6817/1124979277","heartbeat_front_addr":"172.21.15.92:6815/1124979277","state":["exists","up"]},{"osd":3,"uuid":"4648d7ce-9b29-4583-9425-4ba8fd2b1903","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6827","nonce":1734115782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6829","nonce":1734115782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6833","nonce":1734115782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6831","nonce":1734115782}]},"public_addr":"172.21.15.92:6827/1734115782","cluster_addr":"172.21.15.92:6829/1734115782","heartbeat_back_addr":"172.21.15.92:6833/1734115782","heartbeat_front_addr":"172.21.15.92:6831/1734115782","state":["exists","up"]},{"osd":4,"uuid":"3bc9d7eb-c6fa-4fcf-a3ec-ade604e05aac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6819","nonce":3425937229}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6822","nonce":3425937229}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6830","nonce":3425937229}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6826","nonce":3425937229}]},"public_addr":"172.21.15.118:6819/3425937229","cluster_addr":"172.21.15.118:6822/3425937229","heartbeat_back_addr":"172.21.15.118:6830/3425937229","heartbeat_front_addr":"172.21.15.118:6826/3425937229","state":["exists","up"]},{"osd":5,"uuid":"dce9ba6d-59ed-46d1-84fc-a84b13de10c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6801","nonce":827455984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6803","nonce":827455984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6807","nonce":827455984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6805","nonce":827455984}]},"public_addr":"172.21.15.118:6801/827455984","cluster_addr":"172.21.15.118:6803/827455984","heartbeat_back_addr":"172.21.15.118:6807/827455984","heartbeat_front_addr":"172.21.15.118:6805/827455984","state":["exists","up"]},{"osd":6,"uuid":"82aec504-ae58-4709-8d30-46f2b12081b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6809","nonce":2946986141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6811","nonce":2946986141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6815","nonce":2946986141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6813","nonce":2946986141}]},"public_addr":"172.21.15.118:6809/2946986141","cluster_addr":"172.21.15.118:6811/2946986141","heartbeat_back_addr":"172.21.15.118:6815/2946986141","heartbeat_front_addr":"172.21.15.118:6813/2946986141","state":["exists","up"]},{"osd":7,"uuid":"4ed4c624-c88f-4d35-bdb1-bf209b5dffb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6817","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6818","nonce":48122059}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6821","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6823","nonce":48122059}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6829","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6831","nonce":48122059}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6825","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6827","nonce":48122059}]},"public_addr":"172.21.15.118:6818/48122059","cluster_addr":"172.21.15.118:6823/48122059","heartbeat_back_addr":"172.21.15.118:6831/48122059","heartbeat_front_addr":"172.21.15.118:6827/48122059","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.536664+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.368324+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.271201+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.342971+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.621832+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.387110+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.632946+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.416032+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:10:26.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:26.109+0000 7f3d6339e640 1 -- 172.21.15.92:0/3031696456 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f3d4006ae40 msgr2=0x7f3d4006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:26.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:26.109+0000 7f3d6339e640 1 --2- 172.21.15.92:0/3031696456 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f3d4006ae40 0x7f3d4006d300 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f3d58036000 tx=0x7f3d5803c040 comp rx=0 tx=0).stop 2023-11-24T06:10:26.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:26.109+0000 7f3d6339e640 1 -- 172.21.15.92:0/3031696456 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3d5c09a170 msgr2=0x7f3d5c0c5710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:26.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:26.109+0000 7f3d6339e640 1 --2- 172.21.15.92:0/3031696456 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3d5c09a170 0x7f3d5c0c5710 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f3d4c00c400 tx=0x7f3d4c00c8c0 comp rx=0 tx=0).stop 2023-11-24T06:10:26.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:26.109+0000 7f3d6339e640 1 -- 172.21.15.92:0/3031696456 shutdown_connections 2023-11-24T06:10:26.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:26.109+0000 7f3d6339e640 1 --2- 172.21.15.92:0/3031696456 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f3d4006ae40 0x7f3d4006d300 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:26.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:26.109+0000 7f3d6339e640 1 --2- 172.21.15.92:0/3031696456 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3d5c0cd240 0x7f3d5c0b5c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:26.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:26.109+0000 7f3d6339e640 1 --2- 172.21.15.92:0/3031696456 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3d5c09a170 0x7f3d5c0c5710 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:26.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:26.109+0000 7f3d6339e640 1 --2- 172.21.15.92:0/3031696456 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3d5c06c6b0 0x7f3d5c15aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:26.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:26.109+0000 7f3d6339e640 1 -- 172.21.15.92:0/3031696456 >> 172.21.15.92:0/3031696456 conn(0x7f3d5c0b2d00 msgr2=0x7f3d5c099a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:26.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:26.109+0000 7f3d6339e640 1 -- 172.21.15.92:0/3031696456 shutdown_connections 2023-11-24T06:10:26.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:26.109+0000 7f3d6339e640 1 -- 172.21.15.92:0/3031696456 wait complete. 2023-11-24T06:10:26.129 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:10:26.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:26.331+0000 7f6e03fff640 1 Processor -- start 2023-11-24T06:10:26.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:26.331+0000 7f6e03fff640 1 -- start start 2023-11-24T06:10:26.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:26.332+0000 7f6e03fff640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6e04143c40 0x7f6e04071290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:26.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:26.332+0000 7f6e03fff640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f6e040717d0 0x7f6e040cc500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:26.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:26.333+0000 7f6e03fff640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f6e040cca40 0x7f6e0415eab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:26.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:26.333+0000 7f6e03fff640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e040723c0 con 0x7f6e040cca40 2023-11-24T06:10:26.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:26.333+0000 7f6e03fff640 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e04070e50 con 0x7f6e040717d0 2023-11-24T06:10:26.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:26.333+0000 7f6e03fff640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e040cce70 con 0x7f6e04143c40 2023-11-24T06:10:26.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:26.333+0000 7f6e037fe640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f6e040cca40 0x7f6e0415eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:26.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:26.333+0000 7f6e037fe640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f6e040cca40 0x7f6e0415eab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41886/0 (socket says 172.21.15.92:41886) 2023-11-24T06:10:26.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:26.333+0000 7f6e037fe640 1 -- 172.21.15.92:0/427826485 learned_addr learned my addr 172.21.15.92:0/427826485 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:10:26.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:26.333+0000 7f6e02ffd640 1 --2- 172.21.15.92:0/427826485 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6e04143c40 0x7f6e04071290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:26.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:26.333+0000 7f6dfa7fc640 1 --2- 172.21.15.92:0/427826485 >> v2:172.21.15.118:3300/0 conn(0x7f6e040717d0 0x7f6e040cc500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:26.338 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:26.333+0000 7f6e037fe640 1 -- 172.21.15.92:0/427826485 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6e04143c40 msgr2=0x7f6e04071290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:26.338 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:26.333+0000 7f6e037fe640 1 --2- 172.21.15.92:0/427826485 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6e04143c40 0x7f6e04071290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:26.338 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:26.333+0000 7f6e037fe640 1 -- 172.21.15.92:0/427826485 >> v2:172.21.15.118:3300/0 conn(0x7f6e040717d0 msgr2=0x7f6e040cc500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:26.338 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:26.333+0000 7f6e037fe640 1 --2- 172.21.15.92:0/427826485 >> v2:172.21.15.118:3300/0 conn(0x7f6e040717d0 0x7f6e040cc500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:26.338 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:26.333+0000 7f6e037fe640 1 -- 172.21.15.92:0/427826485 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e04144020 con 0x7f6e040cca40 2023-11-24T06:10:26.338 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:26.333+0000 7f6dfa7fc640 1 --2- 172.21.15.92:0/427826485 >> v2:172.21.15.118:3300/0 conn(0x7f6e040717d0 0x7f6e040cc500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:10:26.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:26.334+0000 7f6e037fe640 1 --2- 172.21.15.92:0/427826485 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6e040cca40 0x7f6e0415eab0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f6e0406e440 tx=0x7f6dfc02be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2dba2cc77cf6f73 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:26.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:26.335+0000 7f6e027fc640 1 -- 172.21.15.92:0/427826485 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dfc02c9e0 con 0x7f6e040cca40 2023-11-24T06:10:26.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:26.335+0000 7f6e027fc640 1 -- 172.21.15.92:0/427826485 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6dfc02cb40 con 0x7f6e040cca40 2023-11-24T06:10:26.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:26.335+0000 7f6e027fc640 1 -- 172.21.15.92:0/427826485 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dfc0343f0 con 0x7f6e040cca40 2023-11-24T06:10:26.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:26.335+0000 7f6e03fff640 1 -- 172.21.15.92:0/427826485 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6e040cca40 msgr2=0x7f6e0415eab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:26.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:26.335+0000 7f6e03fff640 1 --2- 172.21.15.92:0/427826485 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6e040cca40 0x7f6e0415eab0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f6e0406e440 tx=0x7f6dfc02be40 comp rx=0 tx=0).stop 2023-11-24T06:10:26.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:26.335+0000 7f6e03fff640 1 -- 172.21.15.92:0/427826485 shutdown_connections 2023-11-24T06:10:26.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:26.335+0000 7f6e03fff640 1 --2- 172.21.15.92:0/427826485 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6e040cca40 0x7f6e0415eab0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:26.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:26.335+0000 7f6e03fff640 1 --2- 172.21.15.92:0/427826485 >> v2:172.21.15.118:3300/0 conn(0x7f6e040717d0 0x7f6e040cc500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:26.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:26.335+0000 7f6e03fff640 1 --2- 172.21.15.92:0/427826485 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6e04143c40 0x7f6e04071290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:26.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:26.335+0000 7f6e03fff640 1 -- 172.21.15.92:0/427826485 >> 172.21.15.92:0/427826485 conn(0x7f6e0406a800 msgr2=0x7f6e0406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:26.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:26.335+0000 7f6e03fff640 1 -- 172.21.15.92:0/427826485 shutdown_connections 2023-11-24T06:10:26.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:26.335+0000 7f6e03fff640 1 -- 172.21.15.92:0/427826485 wait complete. 2023-11-24T06:10:26.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:26.337+0000 7f6e03fff640 1 Processor -- start 2023-11-24T06:10:26.341 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:26.337+0000 7f6e03fff640 1 -- start start 2023-11-24T06:10:26.342 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:26.338+0000 7f6e03fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6e040717d0 0x7f6e04159da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:26.342 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:26.338+0000 7f6e03fff640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6e040cca40 0x7f6e041427d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:26.342 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:26.338+0000 7f6e03fff640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6e04143c40 0x7f6e040b2be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:26.342 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:26.338+0000 7f6e03fff640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e04141730 con 0x7f6e040717d0 2023-11-24T06:10:26.342 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:26.338+0000 7f6e03fff640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e04141a10 con 0x7f6e040cca40 2023-11-24T06:10:26.342 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:26.338+0000 7f6e03fff640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e04141cf0 con 0x7f6e04143c40 2023-11-24T06:10:26.342 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:26.338+0000 7f6e02ffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6e040717d0 0x7f6e04159da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:26.342 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:26.338+0000 7f6e02ffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6e040717d0 0x7f6e04159da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41900/0 (socket says 172.21.15.92:41900) 2023-11-24T06:10:26.342 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:26.338+0000 7f6e02ffd640 1 -- 172.21.15.92:0/3287331268 learned_addr learned my addr 172.21.15.92:0/3287331268 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:10:26.343 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:26.339+0000 7f6e037fe640 1 --2- 172.21.15.92:0/3287331268 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6e04143c40 0x7f6e040b2be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:26.343 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:26.339+0000 7f6e02ffd640 1 -- 172.21.15.92:0/3287331268 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6e04143c40 msgr2=0x7f6e040b2be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:26.343 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:26.339+0000 7f6e02ffd640 1 --2- 172.21.15.92:0/3287331268 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6e04143c40 0x7f6e040b2be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:26.343 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:26.339+0000 7f6e02ffd640 1 -- 172.21.15.92:0/3287331268 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6e040cca40 msgr2=0x7f6e041427d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:26.343 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:26.339+0000 7f6e02ffd640 1 --2- 172.21.15.92:0/3287331268 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6e040cca40 0x7f6e041427d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:26.343 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:26.339+0000 7f6e02ffd640 1 -- 172.21.15.92:0/3287331268 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dfc009b30 con 0x7f6e040717d0 2023-11-24T06:10:26.343 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:26.339+0000 7f6e02ffd640 1 --2- 172.21.15.92:0/3287331268 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6e040717d0 0x7f6e04159da0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f6df400d800 tx=0x7f6df400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:26.344 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:26.339+0000 7f6e00ff9640 1 -- 172.21.15.92:0/3287331268 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6df400b6c0 con 0x7f6e040717d0 2023-11-24T06:10:26.344 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:26.339+0000 7f6e00ff9640 1 -- 172.21.15.92:0/3287331268 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6df400b820 con 0x7f6e040717d0 2023-11-24T06:10:26.344 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:26.339+0000 7f6e00ff9640 1 -- 172.21.15.92:0/3287331268 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6df400bb90 con 0x7f6e040717d0 2023-11-24T06:10:26.344 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:26.339+0000 7f6e03fff640 1 -- 172.21.15.92:0/3287331268 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e04141fd0 con 0x7f6e040717d0 2023-11-24T06:10:26.344 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:26.339+0000 7f6e03fff640 1 -- 172.21.15.92:0/3287331268 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e0415abf0 con 0x7f6e040717d0 2023-11-24T06:10:26.344 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:26.340+0000 7f6e03fff640 1 -- 172.21.15.92:0/3287331268 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e04067450 con 0x7f6e040717d0 2023-11-24T06:10:26.344 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:26.341+0000 7f6e00ff9640 1 -- 172.21.15.92:0/3287331268 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6df400bcf0 con 0x7f6e040717d0 2023-11-24T06:10:26.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:26.342+0000 7f6e00ff9640 1 --2- 172.21.15.92:0/3287331268 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f6de806ae40 0x7f6de806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:26.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:26.342+0000 7f6e00ff9640 1 -- 172.21.15.92:0/3287331268 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6df4089bd0 con 0x7f6e040717d0 2023-11-24T06:10:26.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:26.343+0000 7f6dfa7fc640 1 --2- 172.21.15.92:0/3287331268 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f6de806ae40 0x7f6de806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:26.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:26.343+0000 7f6dfa7fc640 1 --2- 172.21.15.92:0/3287331268 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f6de806ae40 0x7f6de806d300 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f6dec006fd0 tx=0x7f6dec008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:26.350 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:26.347+0000 7f6e00ff9640 1 -- 172.21.15.92:0/3287331268 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6df4054be0 con 0x7f6e040717d0 2023-11-24T06:10:26.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:26.538+0000 7f6e03fff640 1 -- 172.21.15.92:0/3287331268 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6e040ceb30 con 0x7f6e040717d0 2023-11-24T06:10:26.543 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:26.539+0000 7f6e00ff9640 1 -- 172.21.15.92:0/3287331268 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) v1 ==== 74+0+16470 (secure 0 0 0) 0x7f6df4058200 con 0x7f6e040717d0 2023-11-24T06:10:26.543 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-24T06:10:26.544 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":19,"fsid":"2d6dee5f-8aa3-470f-931a-50ade963b1cb","created":"2023-11-24T06:10:04.707204+0000","modified":"2023-11-24T06:10:25.114735+0000","last_up_change":"2023-11-24T06:10:17.598352+0000","last_in_change":"2023-11-24T06:10:09.040793+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:10:17.733098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:10:22.926445+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T06:10:24.151288+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ff37a9fb-135f-471f-b4b0-f7b5772006c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6803","nonce":20560714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6805","nonce":20560714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6809","nonce":20560714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6807","nonce":20560714}]},"public_addr":"172.21.15.92:6803/20560714","cluster_addr":"172.21.15.92:6805/20560714","heartbeat_back_addr":"172.21.15.92:6809/20560714","heartbeat_front_addr":"172.21.15.92:6807/20560714","state":["exists","up"]},{"osd":1,"uuid":"03cdd353-dcf1-4f38-94fe-a795137c06b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6819","nonce":2451239906}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6821","nonce":2451239906}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6825","nonce":2451239906}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6823","nonce":2451239906}]},"public_addr":"172.21.15.92:6819/2451239906","cluster_addr":"172.21.15.92:6821/2451239906","heartbeat_back_addr":"172.21.15.92:6825/2451239906","heartbeat_front_addr":"172.21.15.92:6823/2451239906","state":["exists","up"]},{"osd":2,"uuid":"a4a20d80-5d57-484b-becd-9aa8070dd3aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6811","nonce":1124979277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6813","nonce":1124979277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6817","nonce":1124979277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6815","nonce":1124979277}]},"public_addr":"172.21.15.92:6811/1124979277","cluster_addr":"172.21.15.92:6813/1124979277","heartbeat_back_addr":"172.21.15.92:6817/1124979277","heartbeat_front_addr":"172.21.15.92:6815/1124979277","state":["exists","up"]},{"osd":3,"uuid":"4648d7ce-9b29-4583-9425-4ba8fd2b1903","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6827","nonce":1734115782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6829","nonce":1734115782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6833","nonce":1734115782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6831","nonce":1734115782}]},"public_addr":"172.21.15.92:6827/1734115782","cluster_addr":"172.21.15.92:6829/1734115782","heartbeat_back_addr":"172.21.15.92:6833/1734115782","heartbeat_front_addr":"172.21.15.92:6831/1734115782","state":["exists","up"]},{"osd":4,"uuid":"3bc9d7eb-c6fa-4fcf-a3ec-ade604e05aac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6819","nonce":3425937229}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6822","nonce":3425937229}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6830","nonce":3425937229}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6826","nonce":3425937229}]},"public_addr":"172.21.15.118:6819/3425937229","cluster_addr":"172.21.15.118:6822/3425937229","heartbeat_back_addr":"172.21.15.118:6830/3425937229","heartbeat_front_addr":"172.21.15.118:6826/3425937229","state":["exists","up"]},{"osd":5,"uuid":"dce9ba6d-59ed-46d1-84fc-a84b13de10c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6801","nonce":827455984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6803","nonce":827455984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6807","nonce":827455984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6805","nonce":827455984}]},"public_addr":"172.21.15.118:6801/827455984","cluster_addr":"172.21.15.118:6803/827455984","heartbeat_back_addr":"172.21.15.118:6807/827455984","heartbeat_front_addr":"172.21.15.118:6805/827455984","state":["exists","up"]},{"osd":6,"uuid":"82aec504-ae58-4709-8d30-46f2b12081b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6809","nonce":2946986141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6811","nonce":2946986141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6815","nonce":2946986141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6813","nonce":2946986141}]},"public_addr":"172.21.15.118:6809/2946986141","cluster_addr":"172.21.15.118:6811/2946986141","heartbeat_back_addr":"172.21.15.118:6815/2946986141","heartbeat_front_addr":"172.21.15.118:6813/2946986141","state":["exists","up"]},{"osd":7,"uuid":"4ed4c624-c88f-4d35-bdb1-bf209b5dffb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6817","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6818","nonce":48122059}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6821","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6823","nonce":48122059}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6829","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6831","nonce":48122059}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6825","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6827","nonce":48122059}]},"public_addr":"172.21.15.118:6818/48122059","cluster_addr":"172.21.15.118:6823/48122059","heartbeat_back_addr":"172.21.15.118:6831/48122059","heartbeat_front_addr":"172.21.15.118:6827/48122059","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.536664+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.368324+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.271201+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.342971+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.621832+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.387110+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.632946+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.416032+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:10:26.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:26.542+0000 7f6e03fff640 1 -- 172.21.15.92:0/3287331268 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f6de806ae40 msgr2=0x7f6de806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:26.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:26.542+0000 7f6e03fff640 1 --2- 172.21.15.92:0/3287331268 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f6de806ae40 0x7f6de806d300 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f6dec006fd0 tx=0x7f6dec008040 comp rx=0 tx=0).stop 2023-11-24T06:10:26.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:26.542+0000 7f6e03fff640 1 -- 172.21.15.92:0/3287331268 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6e040717d0 msgr2=0x7f6e04159da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:26.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:26.542+0000 7f6e03fff640 1 --2- 172.21.15.92:0/3287331268 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6e040717d0 0x7f6e04159da0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f6df400d800 tx=0x7f6df400dcc0 comp rx=0 tx=0).stop 2023-11-24T06:10:26.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:26.542+0000 7f6e03fff640 1 -- 172.21.15.92:0/3287331268 shutdown_connections 2023-11-24T06:10:26.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:26.542+0000 7f6e03fff640 1 --2- 172.21.15.92:0/3287331268 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f6de806ae40 0x7f6de806d300 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:26.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:26.542+0000 7f6e03fff640 1 --2- 172.21.15.92:0/3287331268 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6e04143c40 0x7f6e040b2be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:26.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:26.542+0000 7f6e03fff640 1 --2- 172.21.15.92:0/3287331268 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6e040cca40 0x7f6e041427d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:26.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:26.542+0000 7f6e03fff640 1 --2- 172.21.15.92:0/3287331268 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6e040717d0 0x7f6e04159da0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:26.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:26.542+0000 7f6e03fff640 1 -- 172.21.15.92:0/3287331268 >> 172.21.15.92:0/3287331268 conn(0x7f6e0406a800 msgr2=0x7f6e040ac8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:26.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:26.542+0000 7f6e03fff640 1 -- 172.21.15.92:0/3287331268 shutdown_connections 2023-11-24T06:10:26.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:26.542+0000 7f6e03fff640 1 -- 172.21.15.92:0/3287331268 wait complete. 2023-11-24T06:10:26.565 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-11-24T06:10:26.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:26.768+0000 7f579b0d3640 1 Processor -- start 2023-11-24T06:10:26.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:26.768+0000 7f579b0d3640 1 -- start start 2023-11-24T06:10:26.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:26.770+0000 7f579b0d3640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f57940cf9a0 0x7f5794070960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:26.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:26.770+0000 7f579b0d3640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f5794070ea0 0x7f57940cc160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:26.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:26.770+0000 7f579b0d3640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5794071690 con 0x7f579414b2c0 2023-11-24T06:10:26.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:26.770+0000 7f579b0d3640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f579414b6a0 con 0x7f5794070ea0 2023-11-24T06:10:26.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:26.770+0000 7f579b0d3640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5794071970 con 0x7f57940cf9a0 2023-11-24T06:10:26.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:26.770+0000 7f5799670640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f5794070ea0 0x7f57940cc160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:26.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:26.770+0000 7f5798e6f640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f579414b2c0 0x7f57940cf290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:40658/0 (socket says 172.21.15.92:40658) 2023-11-24T06:10:26.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:26.770+0000 7f5798e6f640 1 -- 172.21.15.92:0/1472762319 learned_addr learned my addr 172.21.15.92:0/1472762319 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-24T06:10:26.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:26.770+0000 7f5793fff640 1 --2- 172.21.15.92:0/1472762319 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f57940cf9a0 0x7f5794070960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:26.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:26.771+0000 7f5799670640 1 -- 172.21.15.92:0/1472762319 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f57940cf9a0 msgr2=0x7f5794070960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:26.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:26.771+0000 7f5799670640 1 --2- 172.21.15.92:0/1472762319 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f57940cf9a0 0x7f5794070960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:26.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:26.771+0000 7f5799670640 1 -- 172.21.15.92:0/1472762319 >> v1:172.21.15.92:6789/0 conn(0x7f579414b2c0 legacy=0x7f57940cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:26.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:26.771+0000 7f5799670640 1 -- 172.21.15.92:0/1472762319 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5794071c50 con 0x7f5794070ea0 2023-11-24T06:10:26.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:26.771+0000 7f57937fe640 1 -- 172.21.15.92:0/1472762319 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4210207215 0 0) 0x7f5794071690 con 0x7f579414b2c0 2023-11-24T06:10:26.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:26.772+0000 7f5799670640 1 --2- 172.21.15.92:0/1472762319 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5794070ea0 0x7f57940cc160 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f578c009d90 tx=0x7f578c02bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba6d9ada79ef5f14 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:26.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:26.772+0000 7f57937fe640 1 -- 172.21.15.92:0/1472762319 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f578c02cc50 con 0x7f5794070ea0 2023-11-24T06:10:26.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:26.772+0000 7f57937fe640 1 -- 172.21.15.92:0/1472762319 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f578c02cdb0 con 0x7f5794070ea0 2023-11-24T06:10:26.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:26.772+0000 7f57937fe640 1 -- 172.21.15.92:0/1472762319 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f578c034660 con 0x7f5794070ea0 2023-11-24T06:10:26.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:26.773+0000 7f579b0d3640 1 -- 172.21.15.92:0/1472762319 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5794070ea0 msgr2=0x7f57940cc160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:26.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:26.773+0000 7f579b0d3640 1 --2- 172.21.15.92:0/1472762319 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5794070ea0 0x7f57940cc160 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f578c009d90 tx=0x7f578c02bea0 comp rx=0 tx=0).stop 2023-11-24T06:10:26.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:26.773+0000 7f579b0d3640 1 -- 172.21.15.92:0/1472762319 shutdown_connections 2023-11-24T06:10:26.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:26.773+0000 7f579b0d3640 1 --2- 172.21.15.92:0/1472762319 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5794070ea0 0x7f57940cc160 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:26.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:26.773+0000 7f579b0d3640 1 --2- 172.21.15.92:0/1472762319 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f57940cf9a0 0x7f5794070960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:26.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:26.773+0000 7f579b0d3640 1 -- 172.21.15.92:0/1472762319 >> 172.21.15.92:0/1472762319 conn(0x7f579406a800 msgr2=0x7f579406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:26.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:26.773+0000 7f579b0d3640 1 -- 172.21.15.92:0/1472762319 shutdown_connections 2023-11-24T06:10:26.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:26.773+0000 7f579b0d3640 1 -- 172.21.15.92:0/1472762319 wait complete. 2023-11-24T06:10:26.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:26.775+0000 7f579b0d3640 1 Processor -- start 2023-11-24T06:10:26.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:26.775+0000 7f579b0d3640 1 -- start start 2023-11-24T06:10:26.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:26.776+0000 7f579b0d3640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5794070ea0 0x7f5794143c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:26.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:26.776+0000 7f579b0d3640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f57940cf9a0 0x7f57940cdb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:26.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:26.776+0000 7f579b0d3640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f579414b2c0 0x7f57940ce060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:26.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:26.776+0000 7f579b0d3640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5794162910 con 0x7f5794070ea0 2023-11-24T06:10:26.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:26.776+0000 7f579b0d3640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5794161fe0 con 0x7f57940cf9a0 2023-11-24T06:10:26.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:26.776+0000 7f579b0d3640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57941622c0 con 0x7f579414b2c0 2023-11-24T06:10:26.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:26.776+0000 7f5798e6f640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5794070ea0 0x7f5794143c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:26.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:26.776+0000 7f5798e6f640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5794070ea0 0x7f5794143c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41918/0 (socket says 172.21.15.92:41918) 2023-11-24T06:10:26.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:26.776+0000 7f5798e6f640 1 -- 172.21.15.92:0/66716681 learned_addr learned my addr 172.21.15.92:0/66716681 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:10:26.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:26.776+0000 7f5798e6f640 1 -- 172.21.15.92:0/66716681 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f579414b2c0 msgr2=0x7f57940ce060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:26.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:26.776+0000 7f5798e6f640 1 --2- 172.21.15.92:0/66716681 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f579414b2c0 0x7f57940ce060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:26.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:26.776+0000 7f5798e6f640 1 -- 172.21.15.92:0/66716681 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f57940cf9a0 msgr2=0x7f57940cdb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:26.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:26.776+0000 7f5793fff640 1 --2- 172.21.15.92:0/66716681 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f57940cf9a0 0x7f57940cdb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:26.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:26.776+0000 7f5798e6f640 1 --2- 172.21.15.92:0/66716681 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f57940cf9a0 0x7f57940cdb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:26.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:26.776+0000 7f5798e6f640 1 -- 172.21.15.92:0/66716681 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f578c009a70 con 0x7f5794070ea0 2023-11-24T06:10:26.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:26.777+0000 7f5798e6f640 1 --2- 172.21.15.92:0/66716681 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5794070ea0 0x7f5794143c70 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f578800c430 tx=0x7f578800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:26.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:26.777+0000 7f57917fa640 1 -- 172.21.15.92:0/66716681 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57880074e0 con 0x7f5794070ea0 2023-11-24T06:10:26.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:26.777+0000 7f57917fa640 1 -- 172.21.15.92:0/66716681 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5788007660 con 0x7f5794070ea0 2023-11-24T06:10:26.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:26.777+0000 7f57917fa640 1 -- 172.21.15.92:0/66716681 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5788004a20 con 0x7f5794070ea0 2023-11-24T06:10:26.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:26.777+0000 7f579b0d3640 1 -- 172.21.15.92:0/66716681 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57941625a0 con 0x7f5794070ea0 2023-11-24T06:10:26.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:26.777+0000 7f579b0d3640 1 -- 172.21.15.92:0/66716681 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57941439d0 con 0x7f5794070ea0 2023-11-24T06:10:26.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:26.778+0000 7f579b0d3640 1 -- 172.21.15.92:0/66716681 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57940675d0 con 0x7f5794070ea0 2023-11-24T06:10:26.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:26.779+0000 7f57917fa640 1 -- 172.21.15.92:0/66716681 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5788004b80 con 0x7f5794070ea0 2023-11-24T06:10:26.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:26.780+0000 7f57917fa640 1 --2- 172.21.15.92:0/66716681 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f577806ae40 0x7f577806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:26.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:26.780+0000 7f57917fa640 1 -- 172.21.15.92:0/66716681 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5788089430 con 0x7f5794070ea0 2023-11-24T06:10:26.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:26.781+0000 7f5793fff640 1 --2- 172.21.15.92:0/66716681 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f577806ae40 0x7f577806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:26.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:26.781+0000 7f5793fff640 1 --2- 172.21.15.92:0/66716681 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f577806ae40 0x7f577806d300 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f57941496e0 tx=0x7f5780008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:26.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:26.785+0000 7f57917fa640 1 -- 172.21.15.92:0/66716681 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f57880544d0 con 0x7f5794070ea0 2023-11-24T06:10:27.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:27.015+0000 7f579b0d3640 1 -- 172.21.15.92:0/66716681 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f57940cb5a0 con 0x7f5794070ea0 2023-11-24T06:10:27.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:27.717+0000 7f57917fa640 1 -- 172.21.15.92:0/66716681 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v7) v1 ==== 118+0+0 (secure 0 0 0) 0x7f5788057af0 con 0x7f5794070ea0 2023-11-24T06:10:27.726 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:27.722+0000 7f579b0d3640 1 -- 172.21.15.92:0/66716681 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f577806ae40 msgr2=0x7f577806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:27.726 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:27.722+0000 7f579b0d3640 1 --2- 172.21.15.92:0/66716681 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f577806ae40 0x7f577806d300 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f57941496e0 tx=0x7f5780008040 comp rx=0 tx=0).stop 2023-11-24T06:10:27.726 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:27.722+0000 7f579b0d3640 1 -- 172.21.15.92:0/66716681 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5794070ea0 msgr2=0x7f5794143c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:27.726 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:27.722+0000 7f579b0d3640 1 --2- 172.21.15.92:0/66716681 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5794070ea0 0x7f5794143c70 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f578800c430 tx=0x7f578800c8f0 comp rx=0 tx=0).stop 2023-11-24T06:10:27.726 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:27.722+0000 7f579b0d3640 1 -- 172.21.15.92:0/66716681 shutdown_connections 2023-11-24T06:10:27.726 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:27.722+0000 7f579b0d3640 1 --2- 172.21.15.92:0/66716681 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f577806ae40 0x7f577806d300 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:27.726 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:27.722+0000 7f579b0d3640 1 --2- 172.21.15.92:0/66716681 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f579414b2c0 0x7f57940ce060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:27.726 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:27.722+0000 7f579b0d3640 1 --2- 172.21.15.92:0/66716681 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f57940cf9a0 0x7f57940cdb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:27.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:27.722+0000 7f579b0d3640 1 --2- 172.21.15.92:0/66716681 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5794070ea0 0x7f5794143c70 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:27.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:27.722+0000 7f579b0d3640 1 -- 172.21.15.92:0/66716681 >> 172.21.15.92:0/66716681 conn(0x7f579406a800 msgr2=0x7f57940cf260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:27.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:27.722+0000 7f579b0d3640 1 -- 172.21.15.92:0/66716681 shutdown_connections 2023-11-24T06:10:27.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:27.722+0000 7f579b0d3640 1 -- 172.21.15.92:0/66716681 wait complete. 2023-11-24T06:10:27.744 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-11-24T06:10:27.744 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-11-24T06:10:27.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:27.962+0000 7f59c1b3e640 1 Processor -- start 2023-11-24T06:10:27.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:27.962+0000 7f59c1b3e640 1 -- start start 2023-11-24T06:10:27.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:27.963+0000 7f59c1b3e640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f59bc15e3b0 0x7f59bc164de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:27.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:27.963+0000 7f59c1b3e640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59bc0b1ee0 con 0x7f59bc143ae0 2023-11-24T06:10:27.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:27.963+0000 7f59c1b3e640 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59bc0b2780 con 0x7f59bc0cf9a0 2023-11-24T06:10:27.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:27.963+0000 7f59c1b3e640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59bc15e990 con 0x7f59bc15e3b0 2023-11-24T06:10:27.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:27.964+0000 7f59bb7fe640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f59bc143ae0 0x7f59bc0cf290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:40664/0 (socket says 172.21.15.92:40664) 2023-11-24T06:10:27.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:27.964+0000 7f59bb7fe640 1 -- 172.21.15.92:0/1591734015 learned_addr learned my addr 172.21.15.92:0/1591734015 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-24T06:10:27.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:27.964+0000 7f59bbfff640 1 --2- 172.21.15.92:0/1591734015 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f59bc15e3b0 0x7f59bc164de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:27.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:27.964+0000 7f59baffd640 1 -- 172.21.15.92:0/1591734015 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3114091372 0 0) 0x7f59bc0b1ee0 con 0x7f59bc143ae0 2023-11-24T06:10:27.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:27.964+0000 7f59baffd640 1 -- 172.21.15.92:0/1591734015 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f59a8005730 con 0x7f59bc143ae0 2023-11-24T06:10:27.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:27.964+0000 7f59baffd640 1 -- 172.21.15.92:0/1591734015 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2692104863 0 0) 0x7f59a8005730 con 0x7f59bc143ae0 2023-11-24T06:10:27.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:27.964+0000 7f59baffd640 1 -- 172.21.15.92:0/1591734015 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f59bc15e3b0 msgr2=0x7f59bc164de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:27.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:27.964+0000 7f59baffd640 1 --2- 172.21.15.92:0/1591734015 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f59bc15e3b0 0x7f59bc164de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:27.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:27.964+0000 7f59baffd640 1 -- 172.21.15.92:0/1591734015 >> v1:172.21.15.118:6789/0 conn(0x7f59bc0cf9a0 legacy=0x7f59bc15dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:27.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:27.964+0000 7f59baffd640 1 -- 172.21.15.92:0/1591734015 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59bc0b2130 con 0x7f59bc143ae0 2023-11-24T06:10:27.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:27.964+0000 7f59baffd640 1 -- 172.21.15.92:0/1591734015 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2607661350 0 0) 0x7f59ac001670 con 0x7f59bc143ae0 2023-11-24T06:10:27.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:27.965+0000 7f59baffd640 1 -- 172.21.15.92:0/1591734015 >> v1:172.21.15.92:6789/0 conn(0x7f59bc143ae0 legacy=0x7f59bc0cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:27.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:27.965+0000 7f59baffd640 1 --2- 172.21.15.92:0/1591734015 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f59bc15e3b0 0x7f59a8005bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:27.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:27.965+0000 7f59baffd640 1 --2- 172.21.15.92:0/1591734015 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f59a8006130 0x7f59a80069d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:27.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:27.965+0000 7f59baffd640 1 --2- 172.21.15.92:0/1591734015 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f59a8006f10 0x7f59a8007390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:27.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:27.965+0000 7f59baffd640 1 -- 172.21.15.92:0/1591734015 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59bc15e990 con 0x7f59a8006130 2023-11-24T06:10:27.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:27.965+0000 7f59baffd640 1 -- 172.21.15.92:0/1591734015 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59a800dc70 con 0x7f59a8006f10 2023-11-24T06:10:27.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:27.965+0000 7f59baffd640 1 -- 172.21.15.92:0/1591734015 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59a800de30 con 0x7f59bc15e3b0 2023-11-24T06:10:27.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:27.965+0000 7f59b3fff640 1 --2- 172.21.15.92:0/1591734015 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f59a8006130 0x7f59a80069d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:27.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:27.965+0000 7f59bb7fe640 1 --2- 172.21.15.92:0/1591734015 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f59bc15e3b0 0x7f59a8005bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:27.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:27.965+0000 7f59bbfff640 1 --2- 172.21.15.92:0/1591734015 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f59a8006f10 0x7f59a8007390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:27.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:27.965+0000 7f59b3fff640 1 -- 172.21.15.92:0/1591734015 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f59bc15e3b0 msgr2=0x7f59a8005bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:27.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:27.965+0000 7f59b3fff640 1 --2- 172.21.15.92:0/1591734015 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f59bc15e3b0 0x7f59a8005bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:27.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:27.965+0000 7f59b3fff640 1 -- 172.21.15.92:0/1591734015 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f59a8006f10 msgr2=0x7f59a8007390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:27.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:27.965+0000 7f59b3fff640 1 --2- 172.21.15.92:0/1591734015 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f59a8006f10 0x7f59a8007390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:27.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:27.965+0000 7f59b3fff640 1 -- 172.21.15.92:0/1591734015 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59a8005730 con 0x7f59a8006130 2023-11-24T06:10:27.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:27.965+0000 7f59bbfff640 1 --2- 172.21.15.92:0/1591734015 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f59a8006f10 0x7f59a8007390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:10:27.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:27.965+0000 7f59bb7fe640 1 --2- 172.21.15.92:0/1591734015 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f59bc15e3b0 0x7f59a8005bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:10:27.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:27.967+0000 7f59b3fff640 1 --2- 172.21.15.92:0/1591734015 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f59a8006130 0x7f59a80069d0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f59a4001750 tx=0x7f59a8007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13f1e79f8d73e8b7 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:27.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:27.967+0000 7f59baffd640 1 -- 172.21.15.92:0/1591734015 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59a40052d0 con 0x7f59a8006130 2023-11-24T06:10:27.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:27.967+0000 7f59baffd640 1 -- 172.21.15.92:0/1591734015 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59a402e050 con 0x7f59a8006130 2023-11-24T06:10:27.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:27.967+0000 7f59baffd640 1 -- 172.21.15.92:0/1591734015 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59a4003830 con 0x7f59a8006130 2023-11-24T06:10:27.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:27.967+0000 7f59c1b3e640 1 -- 172.21.15.92:0/1591734015 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f59a8006130 msgr2=0x7f59a80069d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:27.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:27.967+0000 7f59c1b3e640 1 --2- 172.21.15.92:0/1591734015 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f59a8006130 0x7f59a80069d0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f59a4001750 tx=0x7f59a8007a00 comp rx=0 tx=0).stop 2023-11-24T06:10:27.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:27.967+0000 7f59bbfff640 1 -- 172.21.15.92:0/1591734015 reap_dead start 2023-11-24T06:10:27.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:27.967+0000 7f59c1b3e640 1 -- 172.21.15.92:0/1591734015 shutdown_connections 2023-11-24T06:10:27.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:27.967+0000 7f59c1b3e640 1 -- 172.21.15.92:0/1591734015 >> 172.21.15.92:0/1591734015 conn(0x7f59bc06a800 msgr2=0x7f59bc06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:27.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:27.967+0000 7f59c1b3e640 1 -- 172.21.15.92:0/1591734015 shutdown_connections 2023-11-24T06:10:27.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:27.968+0000 7f59c1b3e640 1 -- 172.21.15.92:0/1591734015 wait complete. 2023-11-24T06:10:27.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:27.969+0000 7f59c1b3e640 1 Processor -- start 2023-11-24T06:10:27.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:27.969+0000 7f59c1b3e640 1 -- start start 2023-11-24T06:10:27.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:27.970+0000 7f59c1b3e640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f59a8006130 0x7f59bc15d120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:27.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:27.970+0000 7f59c1b3e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f59bc13d230 0x7f59bc13b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:27.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:27.970+0000 7f59c1b3e640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f59bc13ab50 0x7f59bc13dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:27.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:27.970+0000 7f59c1b3e640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59bc13ce10 con 0x7f59bc13d230 2023-11-24T06:10:27.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:27.970+0000 7f59c1b3e640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59bc13bf10 con 0x7f59bc13ab50 2023-11-24T06:10:27.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:27.971+0000 7f59c1b3e640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59bc13c190 con 0x7f59a8006130 2023-11-24T06:10:27.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:27.971+0000 7f59b3fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f59bc13d230 0x7f59bc13b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:27.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:27.971+0000 7f59b3fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f59bc13d230 0x7f59bc13b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41946/0 (socket says 172.21.15.92:41946) 2023-11-24T06:10:27.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:27.971+0000 7f59b3fff640 1 -- 172.21.15.92:0/4120568055 learned_addr learned my addr 172.21.15.92:0/4120568055 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:10:27.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:27.971+0000 7f59bbfff640 1 --2- 172.21.15.92:0/4120568055 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f59bc13ab50 0x7f59bc13dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:27.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:27.971+0000 7f59b3fff640 1 -- 172.21.15.92:0/4120568055 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f59a8006130 msgr2=0x7f59bc15d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:27.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:27.971+0000 7f59b3fff640 1 --2- 172.21.15.92:0/4120568055 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f59a8006130 0x7f59bc15d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:27.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:27.971+0000 7f59b3fff640 1 -- 172.21.15.92:0/4120568055 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f59bc13ab50 msgr2=0x7f59bc13dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:27.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:27.971+0000 7f59b3fff640 1 --2- 172.21.15.92:0/4120568055 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f59bc13ab50 0x7f59bc13dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:27.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:27.971+0000 7f59b3fff640 1 -- 172.21.15.92:0/4120568055 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59bc13c470 con 0x7f59bc13d230 2023-11-24T06:10:27.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:27.971+0000 7f59bbfff640 1 --2- 172.21.15.92:0/4120568055 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f59bc13ab50 0x7f59bc13dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:10:27.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:27.971+0000 7f59b3fff640 1 --2- 172.21.15.92:0/4120568055 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f59bc13d230 0x7f59bc13b9d0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f59a4008fd0 tx=0x7f59a4001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:27.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:27.972+0000 7f59b8ff9640 1 -- 172.21.15.92:0/4120568055 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59a40076c0 con 0x7f59bc13d230 2023-11-24T06:10:27.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:27.972+0000 7f59b8ff9640 1 -- 172.21.15.92:0/4120568055 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59a4007a30 con 0x7f59bc13d230 2023-11-24T06:10:27.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:27.972+0000 7f59b8ff9640 1 -- 172.21.15.92:0/4120568055 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59a4006be0 con 0x7f59bc13d230 2023-11-24T06:10:27.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:27.972+0000 7f59c1b3e640 1 -- 172.21.15.92:0/4120568055 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59bc13c750 con 0x7f59bc13d230 2023-11-24T06:10:27.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:27.972+0000 7f59c1b3e640 1 -- 172.21.15.92:0/4120568055 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59bc13b000 con 0x7f59bc13d230 2023-11-24T06:10:27.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:27.973+0000 7f59c1b3e640 1 -- 172.21.15.92:0/4120568055 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59bc06e0d0 con 0x7f59bc13d230 2023-11-24T06:10:27.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:27.974+0000 7f59b8ff9640 1 -- 172.21.15.92:0/4120568055 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f59a4006d40 con 0x7f59bc13d230 2023-11-24T06:10:27.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:27.975+0000 7f59b8ff9640 1 --2- 172.21.15.92:0/4120568055 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f599c06ae40 0x7f599c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:27.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:27.975+0000 7f59b8ff9640 1 -- 172.21.15.92:0/4120568055 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f59a40ab790 con 0x7f59bc13d230 2023-11-24T06:10:27.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:27.975+0000 7f59bb7fe640 1 --2- 172.21.15.92:0/4120568055 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f599c06ae40 0x7f599c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:27.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:27.976+0000 7f59bb7fe640 1 --2- 172.21.15.92:0/4120568055 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f599c06ae40 0x7f599c06d300 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f59ac001c80 tx=0x7f59ac001150 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:27.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:27.979+0000 7f59b8ff9640 1 -- 172.21.15.92:0/4120568055 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f59a40767a0 con 0x7f59bc13d230 2023-11-24T06:10:28.211 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:28.208+0000 7f59c1b3e640 1 -- 172.21.15.92:0/4120568055 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f59bc06c3f0 con 0x7f59bc13d230 2023-11-24T06:10:28.724 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:28.720+0000 7f59b8ff9640 1 -- 172.21.15.92:0/4120568055 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v8) v1 ==== 122+0+0 (secure 0 0 0) 0x7f59a4079dc0 con 0x7f59bc13d230 2023-11-24T06:10:28.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:28.724+0000 7f59c1b3e640 1 -- 172.21.15.92:0/4120568055 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f599c06ae40 msgr2=0x7f599c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:28.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:28.724+0000 7f59c1b3e640 1 --2- 172.21.15.92:0/4120568055 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f599c06ae40 0x7f599c06d300 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f59ac001c80 tx=0x7f59ac001150 comp rx=0 tx=0).stop 2023-11-24T06:10:28.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:28.724+0000 7f59c1b3e640 1 -- 172.21.15.92:0/4120568055 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f59bc13d230 msgr2=0x7f59bc13b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:28.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:28.724+0000 7f59c1b3e640 1 --2- 172.21.15.92:0/4120568055 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f59bc13d230 0x7f59bc13b9d0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f59a4008fd0 tx=0x7f59a4001120 comp rx=0 tx=0).stop 2023-11-24T06:10:28.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:28.724+0000 7f59c1b3e640 1 -- 172.21.15.92:0/4120568055 shutdown_connections 2023-11-24T06:10:28.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:28.724+0000 7f59c1b3e640 1 --2- 172.21.15.92:0/4120568055 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f599c06ae40 0x7f599c06d300 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:28.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:28.724+0000 7f59c1b3e640 1 --2- 172.21.15.92:0/4120568055 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f59bc13ab50 0x7f59bc13dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:28.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:28.724+0000 7f59c1b3e640 1 --2- 172.21.15.92:0/4120568055 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f59bc13d230 0x7f59bc13b9d0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:28.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:28.724+0000 7f59c1b3e640 1 --2- 172.21.15.92:0/4120568055 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f59a8006130 0x7f59bc15d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:28.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:28.724+0000 7f59c1b3e640 1 -- 172.21.15.92:0/4120568055 >> 172.21.15.92:0/4120568055 conn(0x7f59bc06a800 msgr2=0x7f59bc0cc650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:28.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:28.724+0000 7f59c1b3e640 1 -- 172.21.15.92:0/4120568055 shutdown_connections 2023-11-24T06:10:28.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:28.724+0000 7f59c1b3e640 1 -- 172.21.15.92:0/4120568055 wait complete. 2023-11-24T06:10:28.747 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-11-24T06:10:28.747 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T06:10:28.939 INFO:tasks.ceph.mds.d.smithi118.stderr:2023-11-24T06:10:28.936+0000 7fb1594bf640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-24T06:10:28.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:28.959+0000 7fd34f2da640 1 Processor -- start 2023-11-24T06:10:28.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:28.959+0000 7fd34f2da640 1 -- start start 2023-11-24T06:10:28.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:28.960+0000 7fd34f2da640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fd34813df80 0x7fd34815bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:28.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:28.960+0000 7fd34f2da640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fd34813e360 0x7fd3480cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:28.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:28.961+0000 7fd34f2da640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd34806c470 con 0x7fd34815f9c0 2023-11-24T06:10:28.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:28.961+0000 7fd34f2da640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd34806c750 con 0x7fd34813df80 2023-11-24T06:10:28.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:28.961+0000 7fd34f2da640 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd34806ca30 con 0x7fd34813e360 2023-11-24T06:10:28.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:28.961+0000 7fd34d877640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fd34815f9c0 0x7fd3481408d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:55734/0 (socket says 172.21.15.92:55734) 2023-11-24T06:10:28.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:28.961+0000 7fd34d076640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fd34813df80 0x7fd34815bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:28.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:28.961+0000 7fd34d877640 1 -- 172.21.15.92:0/852362228 learned_addr learned my addr 172.21.15.92:0/852362228 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-24T06:10:28.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:28.961+0000 7fd34c875640 1 --2- 172.21.15.92:0/852362228 >> v2:172.21.15.118:3300/0 conn(0x7fd34813e360 0x7fd3480cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:28.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:28.961+0000 7fd34d076640 1 -- 172.21.15.92:0/852362228 >> v2:172.21.15.118:3300/0 conn(0x7fd34813e360 msgr2=0x7fd3480cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:28.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:28.961+0000 7fd34d076640 1 --2- 172.21.15.92:0/852362228 >> v2:172.21.15.118:3300/0 conn(0x7fd34813e360 0x7fd3480cf290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:28.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:28.961+0000 7fd34d076640 1 -- 172.21.15.92:0/852362228 >> v1:172.21.15.92:6789/0 conn(0x7fd34815f9c0 legacy=0x7fd3481408d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:28.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:28.961+0000 7fd33bfff640 1 -- 172.21.15.92:0/852362228 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1419500339 0 0) 0x7fd34806c470 con 0x7fd34815f9c0 2023-11-24T06:10:28.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:28.961+0000 7fd34d076640 1 -- 172.21.15.92:0/852362228 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd34806cd10 con 0x7fd34813df80 2023-11-24T06:10:28.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:28.963+0000 7fd34d076640 1 --2- 172.21.15.92:0/852362228 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd34813df80 0x7fd34815bee0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd34806ded0 tx=0x7fd34402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0a63931df5484f7 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:28.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:28.963+0000 7fd33bfff640 1 -- 172.21.15.92:0/852362228 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd344031830 con 0x7fd34813df80 2023-11-24T06:10:28.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:28.963+0000 7fd33bfff640 1 -- 172.21.15.92:0/852362228 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd344031990 con 0x7fd34813df80 2023-11-24T06:10:28.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:28.963+0000 7fd33bfff640 1 -- 172.21.15.92:0/852362228 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd344031d20 con 0x7fd34813df80 2023-11-24T06:10:28.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:28.963+0000 7fd34f2da640 1 -- 172.21.15.92:0/852362228 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd34813df80 msgr2=0x7fd34815bee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:28.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:28.963+0000 7fd34f2da640 1 --2- 172.21.15.92:0/852362228 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd34813df80 0x7fd34815bee0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd34806ded0 tx=0x7fd34402b8d0 comp rx=0 tx=0).stop 2023-11-24T06:10:28.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:28.964+0000 7fd34f2da640 1 -- 172.21.15.92:0/852362228 shutdown_connections 2023-11-24T06:10:28.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:28.964+0000 7fd34f2da640 1 --2- 172.21.15.92:0/852362228 >> v2:172.21.15.118:3300/0 conn(0x7fd34813e360 0x7fd3480cf290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:28.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:28.964+0000 7fd34f2da640 1 --2- 172.21.15.92:0/852362228 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd34813df80 0x7fd34815bee0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:28.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:28.964+0000 7fd34f2da640 1 -- 172.21.15.92:0/852362228 >> 172.21.15.92:0/852362228 conn(0x7fd34806a800 msgr2=0x7fd34806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:28.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:28.964+0000 7fd34f2da640 1 -- 172.21.15.92:0/852362228 shutdown_connections 2023-11-24T06:10:28.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:28.964+0000 7fd34f2da640 1 -- 172.21.15.92:0/852362228 wait complete. 2023-11-24T06:10:28.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:28.966+0000 7fd34f2da640 1 Processor -- start 2023-11-24T06:10:28.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:28.966+0000 7fd34f2da640 1 -- start start 2023-11-24T06:10:28.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:28.967+0000 7fd34f2da640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd34813df80 0x7fd348152fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:28.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:28.967+0000 7fd34f2da640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd34813e360 0x7fd34813d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:28.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:28.967+0000 7fd34f2da640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd34815f9c0 0x7fd34814f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:28.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:28.967+0000 7fd34f2da640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3481534e0 con 0x7fd34815f9c0 2023-11-24T06:10:28.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:28.967+0000 7fd34f2da640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd348151f00 con 0x7fd34813e360 2023-11-24T06:10:28.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:28.967+0000 7fd34f2da640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3481521e0 con 0x7fd34813df80 2023-11-24T06:10:28.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:28.967+0000 7fd34d877640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd34815f9c0 0x7fd34814f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:28.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:28.967+0000 7fd34d877640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd34815f9c0 0x7fd34814f6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47156/0 (socket says 172.21.15.92:47156) 2023-11-24T06:10:28.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:28.967+0000 7fd34d877640 1 -- 172.21.15.92:0/818587363 learned_addr learned my addr 172.21.15.92:0/818587363 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:10:28.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:28.967+0000 7fd34d076640 1 --2- 172.21.15.92:0/818587363 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd34813df80 0x7fd348152fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:28.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:28.967+0000 7fd34c875640 1 --2- 172.21.15.92:0/818587363 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd34813e360 0x7fd34813d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:28.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:28.967+0000 7fd34d877640 1 -- 172.21.15.92:0/818587363 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd34813df80 msgr2=0x7fd348152fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:28.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:28.967+0000 7fd34d877640 1 --2- 172.21.15.92:0/818587363 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd34813df80 0x7fd348152fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:28.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:28.967+0000 7fd34d877640 1 -- 172.21.15.92:0/818587363 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd34813e360 msgr2=0x7fd34813d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:28.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:28.967+0000 7fd34d877640 1 --2- 172.21.15.92:0/818587363 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd34813e360 0x7fd34813d5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:28.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:28.967+0000 7fd34d877640 1 -- 172.21.15.92:0/818587363 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd344009400 con 0x7fd34815f9c0 2023-11-24T06:10:28.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:28.967+0000 7fd34c875640 1 --2- 172.21.15.92:0/818587363 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd34813e360 0x7fd34813d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:10:28.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:28.968+0000 7fd34d877640 1 --2- 172.21.15.92:0/818587363 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd34815f9c0 0x7fd34814f6f0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fd340001560 tx=0x7fd34000cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:28.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:28.968+0000 7fd339ffb640 1 -- 172.21.15.92:0/818587363 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd34000a640 con 0x7fd34815f9c0 2023-11-24T06:10:28.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:28.968+0000 7fd339ffb640 1 -- 172.21.15.92:0/818587363 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd34000a7a0 con 0x7fd34815f9c0 2023-11-24T06:10:28.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:28.968+0000 7fd34f2da640 1 -- 172.21.15.92:0/818587363 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3481524c0 con 0x7fd34815f9c0 2023-11-24T06:10:28.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:28.968+0000 7fd339ffb640 1 -- 172.21.15.92:0/818587363 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd34000ab30 con 0x7fd34815f9c0 2023-11-24T06:10:28.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:28.968+0000 7fd34f2da640 1 -- 172.21.15.92:0/818587363 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd348152a80 con 0x7fd34815f9c0 2023-11-24T06:10:28.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:28.969+0000 7fd34f2da640 1 -- 172.21.15.92:0/818587363 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd348152c70 con 0x7fd34815f9c0 2023-11-24T06:10:28.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:28.970+0000 7fd339ffb640 1 -- 172.21.15.92:0/818587363 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd34000ac90 con 0x7fd34815f9c0 2023-11-24T06:10:28.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:28.971+0000 7fd339ffb640 1 --2- 172.21.15.92:0/818587363 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fd32c06ae70 0x7fd32c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:28.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:28.971+0000 7fd339ffb640 1 -- 172.21.15.92:0/818587363 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd3400889a0 con 0x7fd34815f9c0 2023-11-24T06:10:28.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:28.971+0000 7fd34d076640 1 --2- 172.21.15.92:0/818587363 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fd32c06ae70 0x7fd32c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:28.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:28.972+0000 7fd34d076640 1 --2- 172.21.15.92:0/818587363 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fd32c06ae70 0x7fd32c06d330 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fd344036000 tx=0x7fd34403c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:28.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:28.976+0000 7fd339ffb640 1 -- 172.21.15.92:0/818587363 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd340053ae0 con 0x7fd34815f9c0 2023-11-24T06:10:29.207 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:29.204+0000 7fd34f2da640 1 -- 172.21.15.92:0/818587363 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd3480675d0 con 0x7fd34815f9c0 2023-11-24T06:10:29.209 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:29.206+0000 7fd339ffb640 1 -- 172.21.15.92:0/818587363 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4910 (secure 0 0 0) 0x7fd340018020 con 0x7fd34815f9c0 2023-11-24T06:10:29.209 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-24T06:10:29.209 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4236,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6836/437604699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":437604699},{"type":"v1","addr":"172.21.15.92:6836","nonce":437604699}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6835/3368711442","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":3368711442},{"type":"v1","addr":"172.21.15.118:6835","nonce":3368711442}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4255,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/2767290380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":2767290380},{"type":"v1","addr":"172.21.15.92:6837","nonce":2767290380}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T06:10:25.115612+0000","modified":"2023-11-24T06:10:28.210438+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4264},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4264":{"gid":4264,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.118:6833/1146052262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":1146052262},{"type":"v1","addr":"172.21.15.118:6833","nonce":1146052262}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-24T06:10:29.209 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 8 2023-11-24T06:10:29.212 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:29.209+0000 7fd34f2da640 1 -- 172.21.15.92:0/818587363 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fd32c06ae70 msgr2=0x7fd32c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:29.212 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:29.209+0000 7fd34f2da640 1 --2- 172.21.15.92:0/818587363 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fd32c06ae70 0x7fd32c06d330 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fd344036000 tx=0x7fd34403c040 comp rx=0 tx=0).stop 2023-11-24T06:10:29.212 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:29.209+0000 7fd34f2da640 1 -- 172.21.15.92:0/818587363 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd34815f9c0 msgr2=0x7fd34814f6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:29.213 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:29.209+0000 7fd34f2da640 1 --2- 172.21.15.92:0/818587363 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd34815f9c0 0x7fd34814f6f0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fd340001560 tx=0x7fd34000cc70 comp rx=0 tx=0).stop 2023-11-24T06:10:29.213 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:29.209+0000 7fd34f2da640 1 -- 172.21.15.92:0/818587363 shutdown_connections 2023-11-24T06:10:29.213 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:29.209+0000 7fd34f2da640 1 --2- 172.21.15.92:0/818587363 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fd32c06ae70 0x7fd32c06d330 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:29.213 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:29.209+0000 7fd34f2da640 1 --2- 172.21.15.92:0/818587363 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd34815f9c0 0x7fd34814f6f0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:29.213 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:29.209+0000 7fd34f2da640 1 --2- 172.21.15.92:0/818587363 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd34813e360 0x7fd34813d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:29.213 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:29.209+0000 7fd34f2da640 1 --2- 172.21.15.92:0/818587363 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd34813df80 0x7fd348152fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:29.213 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:29.209+0000 7fd34f2da640 1 -- 172.21.15.92:0/818587363 >> 172.21.15.92:0/818587363 conn(0x7fd34806a800 msgr2=0x7fd3480affa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:29.213 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:29.209+0000 7fd34f2da640 1 -- 172.21.15.92:0/818587363 shutdown_connections 2023-11-24T06:10:29.213 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:29.209+0000 7fd34f2da640 1 -- 172.21.15.92:0/818587363 wait complete. 2023-11-24T06:10:29.229 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:10:29.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:29.440+0000 7f164273a640 1 Processor -- start 2023-11-24T06:10:29.444 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:29.440+0000 7f164273a640 1 -- start start 2023-11-24T06:10:29.445 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:29.442+0000 7f164273a640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f163c13df80 0x7f163c15bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:29.445 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:29.442+0000 7f164273a640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f163c0cf9a0 0x7f163c0cd840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:29.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:29.442+0000 7f164273a640 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f163c06c470 con 0x7f163c13e360 2023-11-24T06:10:29.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:29.442+0000 7f164273a640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f163c06c750 con 0x7f163c0cf9a0 2023-11-24T06:10:29.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:29.442+0000 7f164273a640 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f163c06ca30 con 0x7f163c13df80 2023-11-24T06:10:29.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:29.442+0000 7f1640cd7640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f163c0cf9a0 0x7f163c0cd840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:29.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:29.442+0000 7f1640cd7640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f163c0cf9a0 0x7f163c0cd840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47158/0 (socket says 172.21.15.92:47158) 2023-11-24T06:10:29.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:29.442+0000 7f1640cd7640 1 -- 172.21.15.92:0/588717359 learned_addr learned my addr 172.21.15.92:0/588717359 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:10:29.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:29.443+0000 7f163bfff640 1 --2- 172.21.15.92:0/588717359 >> v2:172.21.15.118:3300/0 conn(0x7f163c13df80 0x7f163c15bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:29.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:29.443+0000 7f1640cd7640 1 -- 172.21.15.92:0/588717359 >> v2:172.21.15.118:3300/0 conn(0x7f163c13df80 msgr2=0x7f163c15bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:29.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:29.443+0000 7f1640cd7640 1 --2- 172.21.15.92:0/588717359 >> v2:172.21.15.118:3300/0 conn(0x7f163c13df80 0x7f163c15bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:29.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:29.443+0000 7f1640cd7640 1 -- 172.21.15.92:0/588717359 >> v1:172.21.15.118:6789/0 conn(0x7f163c13e360 legacy=0x7f163c0cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:29.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:29.443+0000 7f1640cd7640 1 -- 172.21.15.92:0/588717359 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f163c06cd10 con 0x7f163c0cf9a0 2023-11-24T06:10:29.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:29.444+0000 7f1640cd7640 1 --2- 172.21.15.92:0/588717359 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f163c0cf9a0 0x7f163c0cd840 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f163c072770 tx=0x7f163402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=512e08a305bb41fc server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:29.448 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:29.444+0000 7f163b7fe640 1 -- 172.21.15.92:0/588717359 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1634031870 con 0x7f163c0cf9a0 2023-11-24T06:10:29.448 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:29.444+0000 7f163b7fe640 1 -- 172.21.15.92:0/588717359 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16340319d0 con 0x7f163c0cf9a0 2023-11-24T06:10:29.448 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:29.444+0000 7f163b7fe640 1 -- 172.21.15.92:0/588717359 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1634031d60 con 0x7f163c0cf9a0 2023-11-24T06:10:29.449 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:29.445+0000 7f164273a640 1 -- 172.21.15.92:0/588717359 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f163c0cf9a0 msgr2=0x7f163c0cd840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:29.449 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:29.445+0000 7f164273a640 1 --2- 172.21.15.92:0/588717359 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f163c0cf9a0 0x7f163c0cd840 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f163c072770 tx=0x7f163402b910 comp rx=0 tx=0).stop 2023-11-24T06:10:29.449 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:29.445+0000 7f164273a640 1 -- 172.21.15.92:0/588717359 shutdown_connections 2023-11-24T06:10:29.449 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:29.445+0000 7f164273a640 1 --2- 172.21.15.92:0/588717359 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f163c0cf9a0 0x7f163c0cd840 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:29.449 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:29.445+0000 7f164273a640 1 --2- 172.21.15.92:0/588717359 >> v2:172.21.15.118:3300/0 conn(0x7f163c13df80 0x7f163c15bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:29.449 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:29.445+0000 7f164273a640 1 -- 172.21.15.92:0/588717359 >> 172.21.15.92:0/588717359 conn(0x7f163c06a800 msgr2=0x7f163c06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:29.449 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:29.445+0000 7f164273a640 1 -- 172.21.15.92:0/588717359 shutdown_connections 2023-11-24T06:10:29.449 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:29.445+0000 7f164273a640 1 -- 172.21.15.92:0/588717359 wait complete. 2023-11-24T06:10:29.450 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:29.447+0000 7f164273a640 1 Processor -- start 2023-11-24T06:10:29.450 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:29.447+0000 7f164273a640 1 -- start start 2023-11-24T06:10:29.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:29.448+0000 7f164273a640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f163c0cf9a0 0x7f163c14bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:29.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:29.448+0000 7f164273a640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f163c13df80 0x7f163c152f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:29.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:29.448+0000 7f164273a640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f163c13e360 0x7f163c14f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:29.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:29.448+0000 7f164273a640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f163c14c0f0 con 0x7f163c13df80 2023-11-24T06:10:29.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:29.448+0000 7f164273a640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f163c14e720 con 0x7f163c0cf9a0 2023-11-24T06:10:29.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:29.448+0000 7f164273a640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f163c14ea00 con 0x7f163c13e360 2023-11-24T06:10:29.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:29.448+0000 7f16337fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f163c13df80 0x7f163c152f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:29.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:29.448+0000 7f16337fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f163c13df80 0x7f163c152f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47160/0 (socket says 172.21.15.92:47160) 2023-11-24T06:10:29.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:29.448+0000 7f16337fe640 1 -- 172.21.15.92:0/2090357845 learned_addr learned my addr 172.21.15.92:0/2090357845 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:10:29.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:29.448+0000 7f163bfff640 1 --2- 172.21.15.92:0/2090357845 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f163c0cf9a0 0x7f163c14bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:29.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:29.449+0000 7f16337fe640 1 -- 172.21.15.92:0/2090357845 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f163c13e360 msgr2=0x7f163c14f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:29.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:29.449+0000 7f1640cd7640 1 --2- 172.21.15.92:0/2090357845 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f163c13e360 0x7f163c14f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:29.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:29.449+0000 7f16337fe640 1 --2- 172.21.15.92:0/2090357845 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f163c13e360 0x7f163c14f7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:29.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:29.449+0000 7f16337fe640 1 -- 172.21.15.92:0/2090357845 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f163c0cf9a0 msgr2=0x7f163c14bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:29.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:29.449+0000 7f16337fe640 1 --2- 172.21.15.92:0/2090357845 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f163c0cf9a0 0x7f163c14bbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:29.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:29.449+0000 7f16337fe640 1 -- 172.21.15.92:0/2090357845 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16340094d0 con 0x7f163c13df80 2023-11-24T06:10:29.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:29.449+0000 7f16337fe640 1 --2- 172.21.15.92:0/2090357845 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f163c13df80 0x7f163c152f60 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f162400c400 tx=0x7f162400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:29.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:29.449+0000 7f16397fa640 1 -- 172.21.15.92:0/2090357845 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1624011790 con 0x7f163c13df80 2023-11-24T06:10:29.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:29.449+0000 7f16397fa640 1 -- 172.21.15.92:0/2090357845 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1624011930 con 0x7f163c13df80 2023-11-24T06:10:29.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:29.449+0000 7f16397fa640 1 -- 172.21.15.92:0/2090357845 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1624011ce0 con 0x7f163c13df80 2023-11-24T06:10:29.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:29.449+0000 7f164273a640 1 -- 172.21.15.92:0/2090357845 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f163c14ece0 con 0x7f163c13df80 2023-11-24T06:10:29.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:29.449+0000 7f164273a640 1 -- 172.21.15.92:0/2090357845 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f163c14f2a0 con 0x7f163c13df80 2023-11-24T06:10:29.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:29.451+0000 7f164273a640 1 -- 172.21.15.92:0/2090357845 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f163c0675d0 con 0x7f163c13df80 2023-11-24T06:10:29.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:29.452+0000 7f16397fa640 1 -- 172.21.15.92:0/2090357845 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f162401c460 con 0x7f163c13df80 2023-11-24T06:10:29.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:29.452+0000 7f16397fa640 1 --2- 172.21.15.92:0/2090357845 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f161c06ae40 0x7f161c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:29.456 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:29.453+0000 7f16397fa640 1 -- 172.21.15.92:0/2090357845 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1624089590 con 0x7f163c13df80 2023-11-24T06:10:29.456 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:29.453+0000 7f163bfff640 1 --2- 172.21.15.92:0/2090357845 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f161c06ae40 0x7f161c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:29.456 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:29.453+0000 7f163bfff640 1 --2- 172.21.15.92:0/2090357845 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f161c06ae40 0x7f161c06d300 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f162c009a60 tx=0x7f162c003990 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:29.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:29.458+0000 7f16397fa640 1 -- 172.21.15.92:0/2090357845 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1624054640 con 0x7f163c13df80 2023-11-24T06:10:29.652 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:29.648+0000 7f164273a640 1 -- 172.21.15.92:0/2090357845 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f163c06df00 con 0x7f163c13df80 2023-11-24T06:10:29.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:29.650+0000 7f16397fa640 1 -- 172.21.15.92:0/2090357845 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16457 (secure 0 0 0) 0x7f1624057c60 con 0x7f163c13df80 2023-11-24T06:10:29.654 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-24T06:10:29.654 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":20,"fsid":"2d6dee5f-8aa3-470f-931a-50ade963b1cb","created":"2023-11-24T06:10:04.707204+0000","modified":"2023-11-24T06:10:26.698733+0000","last_up_change":"2023-11-24T06:10:17.598352+0000","last_in_change":"2023-11-24T06:10:09.040793+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:10:17.733098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:10:22.926445+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T06:10:24.151288+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ff37a9fb-135f-471f-b4b0-f7b5772006c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6803","nonce":20560714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6805","nonce":20560714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6809","nonce":20560714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6807","nonce":20560714}]},"public_addr":"172.21.15.92:6803/20560714","cluster_addr":"172.21.15.92:6805/20560714","heartbeat_back_addr":"172.21.15.92:6809/20560714","heartbeat_front_addr":"172.21.15.92:6807/20560714","state":["exists","up"]},{"osd":1,"uuid":"03cdd353-dcf1-4f38-94fe-a795137c06b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6819","nonce":2451239906}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6821","nonce":2451239906}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6825","nonce":2451239906}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6823","nonce":2451239906}]},"public_addr":"172.21.15.92:6819/2451239906","cluster_addr":"172.21.15.92:6821/2451239906","heartbeat_back_addr":"172.21.15.92:6825/2451239906","heartbeat_front_addr":"172.21.15.92:6823/2451239906","state":["exists","up"]},{"osd":2,"uuid":"a4a20d80-5d57-484b-becd-9aa8070dd3aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6811","nonce":1124979277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6813","nonce":1124979277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6817","nonce":1124979277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6815","nonce":1124979277}]},"public_addr":"172.21.15.92:6811/1124979277","cluster_addr":"172.21.15.92:6813/1124979277","heartbeat_back_addr":"172.21.15.92:6817/1124979277","heartbeat_front_addr":"172.21.15.92:6815/1124979277","state":["exists","up"]},{"osd":3,"uuid":"4648d7ce-9b29-4583-9425-4ba8fd2b1903","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6827","nonce":1734115782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6829","nonce":1734115782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6833","nonce":1734115782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6831","nonce":1734115782}]},"public_addr":"172.21.15.92:6827/1734115782","cluster_addr":"172.21.15.92:6829/1734115782","heartbeat_back_addr":"172.21.15.92:6833/1734115782","heartbeat_front_addr":"172.21.15.92:6831/1734115782","state":["exists","up"]},{"osd":4,"uuid":"3bc9d7eb-c6fa-4fcf-a3ec-ade604e05aac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6819","nonce":3425937229}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6822","nonce":3425937229}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6830","nonce":3425937229}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6826","nonce":3425937229}]},"public_addr":"172.21.15.118:6819/3425937229","cluster_addr":"172.21.15.118:6822/3425937229","heartbeat_back_addr":"172.21.15.118:6830/3425937229","heartbeat_front_addr":"172.21.15.118:6826/3425937229","state":["exists","up"]},{"osd":5,"uuid":"dce9ba6d-59ed-46d1-84fc-a84b13de10c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6801","nonce":827455984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6803","nonce":827455984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6807","nonce":827455984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6805","nonce":827455984}]},"public_addr":"172.21.15.118:6801/827455984","cluster_addr":"172.21.15.118:6803/827455984","heartbeat_back_addr":"172.21.15.118:6807/827455984","heartbeat_front_addr":"172.21.15.118:6805/827455984","state":["exists","up"]},{"osd":6,"uuid":"82aec504-ae58-4709-8d30-46f2b12081b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6809","nonce":2946986141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6811","nonce":2946986141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6815","nonce":2946986141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6813","nonce":2946986141}]},"public_addr":"172.21.15.118:6809/2946986141","cluster_addr":"172.21.15.118:6811/2946986141","heartbeat_back_addr":"172.21.15.118:6815/2946986141","heartbeat_front_addr":"172.21.15.118:6813/2946986141","state":["exists","up"]},{"osd":7,"uuid":"4ed4c624-c88f-4d35-bdb1-bf209b5dffb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6817","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6818","nonce":48122059}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6821","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6823","nonce":48122059}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6829","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6831","nonce":48122059}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6825","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6827","nonce":48122059}]},"public_addr":"172.21.15.118:6818/48122059","cluster_addr":"172.21.15.118:6823/48122059","heartbeat_back_addr":"172.21.15.118:6831/48122059","heartbeat_front_addr":"172.21.15.118:6827/48122059","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.536664+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.368324+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.271201+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.342971+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.621832+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.387110+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.632946+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.416032+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:10:29.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:29.652+0000 7f164273a640 1 -- 172.21.15.92:0/2090357845 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f161c06ae40 msgr2=0x7f161c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:29.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:29.652+0000 7f164273a640 1 --2- 172.21.15.92:0/2090357845 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f161c06ae40 0x7f161c06d300 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f162c009a60 tx=0x7f162c003990 comp rx=0 tx=0).stop 2023-11-24T06:10:29.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:29.652+0000 7f164273a640 1 -- 172.21.15.92:0/2090357845 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f163c13df80 msgr2=0x7f163c152f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:29.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:29.652+0000 7f164273a640 1 --2- 172.21.15.92:0/2090357845 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f163c13df80 0x7f163c152f60 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f162400c400 tx=0x7f162400c8c0 comp rx=0 tx=0).stop 2023-11-24T06:10:29.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:29.652+0000 7f164273a640 1 -- 172.21.15.92:0/2090357845 shutdown_connections 2023-11-24T06:10:29.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:29.652+0000 7f164273a640 1 --2- 172.21.15.92:0/2090357845 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f161c06ae40 0x7f161c06d300 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:29.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:29.652+0000 7f164273a640 1 --2- 172.21.15.92:0/2090357845 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f163c13e360 0x7f163c14f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:29.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:29.652+0000 7f164273a640 1 --2- 172.21.15.92:0/2090357845 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f163c13df80 0x7f163c152f60 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:29.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:29.652+0000 7f164273a640 1 --2- 172.21.15.92:0/2090357845 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f163c0cf9a0 0x7f163c14bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:29.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:29.652+0000 7f164273a640 1 -- 172.21.15.92:0/2090357845 >> 172.21.15.92:0/2090357845 conn(0x7f163c06a800 msgr2=0x7f163c1540f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:29.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:29.653+0000 7f164273a640 1 -- 172.21.15.92:0/2090357845 shutdown_connections 2023-11-24T06:10:29.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:29.653+0000 7f164273a640 1 -- 172.21.15.92:0/2090357845 wait complete. 2023-11-24T06:10:29.674 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-11-24T06:10:29.674 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-24T06:10:29.878 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:29.874+0000 7f0c35fdc640 1 Processor -- start 2023-11-24T06:10:29.878 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:29.875+0000 7f0c35fdc640 1 -- start start 2023-11-24T06:10:29.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:29.876+0000 7f0c35fdc640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0c3015d610 0x7f0c300ce820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:29.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:29.876+0000 7f0c35fdc640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f0c300ced60 0x7f0c300d17e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:29.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:29.876+0000 7f0c35fdc640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c300b3400 con 0x7f0c300d1d20 2023-11-24T06:10:29.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:29.876+0000 7f0c35fdc640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c300d1390 con 0x7f0c300ced60 2023-11-24T06:10:29.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:29.876+0000 7f0c35fdc640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c300d21b0 con 0x7f0c3015d610 2023-11-24T06:10:29.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:29.876+0000 7f0c27fff640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f0c300ced60 0x7f0c300d17e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:29.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:29.876+0000 7f0c2ffff640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f0c300d1d20 0x7f0c300cf830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:55750/0 (socket says 172.21.15.92:55750) 2023-11-24T06:10:29.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:29.876+0000 7f0c2ffff640 1 -- 172.21.15.92:0/2267512119 learned_addr learned my addr 172.21.15.92:0/2267512119 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-24T06:10:29.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:29.877+0000 7f0c2f7fe640 1 --2- 172.21.15.92:0/2267512119 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0c3015d610 0x7f0c300ce820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:29.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:29.877+0000 7f0c2effd640 1 -- 172.21.15.92:0/2267512119 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2455269592 0 0) 0x7f0c300b3400 con 0x7f0c300d1d20 2023-11-24T06:10:29.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:29.877+0000 7f0c2effd640 1 -- 172.21.15.92:0/2267512119 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0c1c003520 con 0x7f0c300d1d20 2023-11-24T06:10:29.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:29.877+0000 7f0c27fff640 1 -- 172.21.15.92:0/2267512119 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0c3015d610 msgr2=0x7f0c300ce820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:29.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:29.877+0000 7f0c27fff640 1 --2- 172.21.15.92:0/2267512119 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0c3015d610 0x7f0c300ce820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:29.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:29.877+0000 7f0c27fff640 1 -- 172.21.15.92:0/2267512119 >> v1:172.21.15.92:6789/0 conn(0x7f0c300d1d20 legacy=0x7f0c300cf830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:29.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:29.877+0000 7f0c27fff640 1 -- 172.21.15.92:0/2267512119 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c300b27f0 con 0x7f0c300ced60 2023-11-24T06:10:29.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:29.878+0000 7f0c27fff640 1 --2- 172.21.15.92:0/2267512119 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0c300ced60 0x7f0c300d17e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f0c180097a0 tx=0x7f0c1802b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d20b4c267f59051 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:29.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:29.878+0000 7f0c2effd640 1 -- 172.21.15.92:0/2267512119 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c1802d040 con 0x7f0c300ced60 2023-11-24T06:10:29.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:29.878+0000 7f0c2effd640 1 -- 172.21.15.92:0/2267512119 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c18031810 con 0x7f0c300ced60 2023-11-24T06:10:29.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:29.878+0000 7f0c2effd640 1 -- 172.21.15.92:0/2267512119 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c18031ba0 con 0x7f0c300ced60 2023-11-24T06:10:29.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:29.879+0000 7f0c35fdc640 1 -- 172.21.15.92:0/2267512119 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0c300ced60 msgr2=0x7f0c300d17e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:29.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:29.879+0000 7f0c35fdc640 1 --2- 172.21.15.92:0/2267512119 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0c300ced60 0x7f0c300d17e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f0c180097a0 tx=0x7f0c1802b8b0 comp rx=0 tx=0).stop 2023-11-24T06:10:29.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:29.879+0000 7f0c35fdc640 1 -- 172.21.15.92:0/2267512119 shutdown_connections 2023-11-24T06:10:29.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:29.879+0000 7f0c35fdc640 1 --2- 172.21.15.92:0/2267512119 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0c300ced60 0x7f0c300d17e0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:29.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:29.879+0000 7f0c35fdc640 1 --2- 172.21.15.92:0/2267512119 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0c3015d610 0x7f0c300ce820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:29.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:29.879+0000 7f0c35fdc640 1 -- 172.21.15.92:0/2267512119 >> 172.21.15.92:0/2267512119 conn(0x7f0c3009c540 msgr2=0x7f0c300d3200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:29.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:29.879+0000 7f0c35fdc640 1 -- 172.21.15.92:0/2267512119 shutdown_connections 2023-11-24T06:10:29.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:29.879+0000 7f0c35fdc640 1 -- 172.21.15.92:0/2267512119 wait complete. 2023-11-24T06:10:29.884 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:29.881+0000 7f0c35fdc640 1 Processor -- start 2023-11-24T06:10:29.884 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:29.881+0000 7f0c35fdc640 1 -- start start 2023-11-24T06:10:29.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:29.882+0000 7f0c35fdc640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0c300ced60 0x7f0c301600d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:29.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:29.882+0000 7f0c35fdc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0c300d1d20 0x7f0c301f6420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:29.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:29.882+0000 7f0c35fdc640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0c3015d610 0x7f0c300b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:29.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:29.882+0000 7f0c35fdc640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c3015feb0 con 0x7f0c300d1d20 2023-11-24T06:10:29.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:29.882+0000 7f0c35fdc640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c301f2e00 con 0x7f0c3015d610 2023-11-24T06:10:29.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:29.882+0000 7f0c35fdc640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c3015e030 con 0x7f0c300ced60 2023-11-24T06:10:29.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:29.882+0000 7f0c27fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0c300d1d20 0x7f0c301f6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:29.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:29.882+0000 7f0c27fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0c300d1d20 0x7f0c301f6420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47184/0 (socket says 172.21.15.92:47184) 2023-11-24T06:10:29.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:29.882+0000 7f0c27fff640 1 -- 172.21.15.92:0/2080697489 learned_addr learned my addr 172.21.15.92:0/2080697489 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:10:29.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:29.882+0000 7f0c2f7fe640 1 --2- 172.21.15.92:0/2080697489 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0c300ced60 0x7f0c301600d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:29.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:29.882+0000 7f0c2ffff640 1 --2- 172.21.15.92:0/2080697489 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0c3015d610 0x7f0c300b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:29.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:29.882+0000 7f0c27fff640 1 -- 172.21.15.92:0/2080697489 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0c300ced60 msgr2=0x7f0c301600d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:29.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:29.882+0000 7f0c27fff640 1 --2- 172.21.15.92:0/2080697489 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0c300ced60 0x7f0c301600d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:29.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:29.882+0000 7f0c27fff640 1 -- 172.21.15.92:0/2080697489 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0c3015d610 msgr2=0x7f0c300b4e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:29.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:29.882+0000 7f0c27fff640 1 --2- 172.21.15.92:0/2080697489 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0c3015d610 0x7f0c300b4e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:29.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:29.882+0000 7f0c27fff640 1 -- 172.21.15.92:0/2080697489 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c18009400 con 0x7f0c300d1d20 2023-11-24T06:10:29.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:29.883+0000 7f0c27fff640 1 --2- 172.21.15.92:0/2080697489 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0c300d1d20 0x7f0c301f6420 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f0c18000c00 tx=0x7f0c180046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:29.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:29.883+0000 7f0c2cff9640 1 -- 172.21.15.92:0/2080697489 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c18006e10 con 0x7f0c300d1d20 2023-11-24T06:10:29.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:29.883+0000 7f0c2cff9640 1 -- 172.21.15.92:0/2080697489 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c1803ad60 con 0x7f0c300d1d20 2023-11-24T06:10:29.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:29.883+0000 7f0c35fdc640 1 -- 172.21.15.92:0/2080697489 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c3015e310 con 0x7f0c300d1d20 2023-11-24T06:10:29.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:29.883+0000 7f0c2cff9640 1 -- 172.21.15.92:0/2080697489 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c18044610 con 0x7f0c300d1d20 2023-11-24T06:10:29.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:29.883+0000 7f0c35fdc640 1 -- 172.21.15.92:0/2080697489 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c3015e8d0 con 0x7f0c300d1d20 2023-11-24T06:10:29.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:29.884+0000 7f0c35fdc640 1 -- 172.21.15.92:0/2080697489 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c300675d0 con 0x7f0c300d1d20 2023-11-24T06:10:29.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:29.885+0000 7f0c2cff9640 1 -- 172.21.15.92:0/2080697489 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f0c1804d450 con 0x7f0c300d1d20 2023-11-24T06:10:29.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:29.886+0000 7f0c2cff9640 1 --2- 172.21.15.92:0/2080697489 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f0c1006ae40 0x7f0c1006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:29.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:29.886+0000 7f0c2cff9640 1 -- 172.21.15.92:0/2080697489 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0c180ab960 con 0x7f0c300d1d20 2023-11-24T06:10:29.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:29.886+0000 7f0c2f7fe640 1 --2- 172.21.15.92:0/2080697489 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f0c1006ae40 0x7f0c1006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:29.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:29.887+0000 7f0c2f7fe640 1 --2- 172.21.15.92:0/2080697489 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f0c1006ae40 0x7f0c1006d300 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f0c20006fd0 tx=0x7f0c20008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:29.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:29.890+0000 7f0c2cff9640 1 -- 172.21.15.92:0/2080697489 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0c18076970 con 0x7f0c300d1d20 2023-11-24T06:10:30.067 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:30.064+0000 7f0c35fdc640 1 -- 172.21.15.92:0/2080697489 --> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0c300d1850 con 0x7f0c1006ae40 2023-11-24T06:10:30.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:30.088+0000 7f0c2cff9640 1 -- 172.21.15.92:0/2080697489 <== mgr.4101 v2:172.21.15.92:6800/784027401 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264474 (secure 0 0 0) 0x7f0c300d1850 con 0x7f0c1006ae40 2023-11-24T06:10:30.095 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-24T06:10:30.097 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2023-11-24T06:10:30.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:30.094+0000 7f0c35fdc640 1 -- 172.21.15.92:0/2080697489 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f0c1006ae40 msgr2=0x7f0c1006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:30.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:30.094+0000 7f0c35fdc640 1 --2- 172.21.15.92:0/2080697489 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f0c1006ae40 0x7f0c1006d300 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f0c20006fd0 tx=0x7f0c20008040 comp rx=0 tx=0).stop 2023-11-24T06:10:30.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:30.094+0000 7f0c35fdc640 1 -- 172.21.15.92:0/2080697489 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0c300d1d20 msgr2=0x7f0c301f6420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:30.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:30.094+0000 7f0c35fdc640 1 --2- 172.21.15.92:0/2080697489 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0c300d1d20 0x7f0c301f6420 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f0c18000c00 tx=0x7f0c180046c0 comp rx=0 tx=0).stop 2023-11-24T06:10:30.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:30.094+0000 7f0c35fdc640 1 -- 172.21.15.92:0/2080697489 shutdown_connections 2023-11-24T06:10:30.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:30.094+0000 7f0c35fdc640 1 --2- 172.21.15.92:0/2080697489 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f0c1006ae40 0x7f0c1006d300 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:30.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:30.094+0000 7f0c35fdc640 1 --2- 172.21.15.92:0/2080697489 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0c3015d610 0x7f0c300b4e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:30.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:30.094+0000 7f0c35fdc640 1 --2- 172.21.15.92:0/2080697489 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0c300d1d20 0x7f0c301f6420 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:30.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:30.094+0000 7f0c35fdc640 1 --2- 172.21.15.92:0/2080697489 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0c300ced60 0x7f0c301600d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:30.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:30.094+0000 7f0c35fdc640 1 -- 172.21.15.92:0/2080697489 >> 172.21.15.92:0/2080697489 conn(0x7f0c3009c540 msgr2=0x7f0c3006ca10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:30.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:30.094+0000 7f0c35fdc640 1 -- 172.21.15.92:0/2080697489 shutdown_connections 2023-11-24T06:10:30.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:30.094+0000 7f0c35fdc640 1 -- 172.21.15.92:0/2080697489 wait complete. 2023-11-24T06:10:30.115 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":19,"stamp":"2023-11-24T06:10:29.667971+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":156,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235656,"kb_used_data":2816,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754739064,"statfs":{"total":773094113280,"available":772852801536,"internally_reserved":0,"allocated":2883584,"data_stored":1751608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":10,"num_read_kb":9,"num_write":58,"num_write_kb":276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001621"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.887289+0000","last_change":"2023-11-24T06:10:25.718900+0000","last_active":"2023-11-24T06:10:26.887289+0000","last_peered":"2023-11-24T06:10:26.887289+0000","last_clean":"2023-11-24T06:10:26.887289+0000","last_became_active":"2023-11-24T06:10:25.718569+0000","last_became_peered":"2023-11-24T06:10:25.718569+0000","last_unstale":"2023-11-24T06:10:26.887289+0000","last_undegraded":"2023-11-24T06:10:26.887289+0000","last_fullsized":"2023-11-24T06:10:26.887289+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:56:03.801033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.885991+0000","last_change":"2023-11-24T06:10:25.711437+0000","last_active":"2023-11-24T06:10:26.885991+0000","last_peered":"2023-11-24T06:10:26.885991+0000","last_clean":"2023-11-24T06:10:26.885991+0000","last_became_active":"2023-11-24T06:10:25.710555+0000","last_became_peered":"2023-11-24T06:10:25.710555+0000","last_unstale":"2023-11-24T06:10:26.885991+0000","last_undegraded":"2023-11-24T06:10:26.885991+0000","last_fullsized":"2023-11-24T06:10:26.885991+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:37:01.293292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.886876+0000","last_change":"2023-11-24T06:10:25.716423+0000","last_active":"2023-11-24T06:10:26.886876+0000","last_peered":"2023-11-24T06:10:26.886876+0000","last_clean":"2023-11-24T06:10:26.886876+0000","last_became_active":"2023-11-24T06:10:25.716112+0000","last_became_peered":"2023-11-24T06:10:25.716112+0000","last_unstale":"2023-11-24T06:10:26.886876+0000","last_undegraded":"2023-11-24T06:10:26.886876+0000","last_fullsized":"2023-11-24T06:10:26.886876+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:43:50.924468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.888341+0000","last_change":"2023-11-24T06:10:25.733338+0000","last_active":"2023-11-24T06:10:26.888341+0000","last_peered":"2023-11-24T06:10:26.888341+0000","last_clean":"2023-11-24T06:10:26.888341+0000","last_became_active":"2023-11-24T06:10:25.733087+0000","last_became_peered":"2023-11-24T06:10:25.733087+0000","last_unstale":"2023-11-24T06:10:26.888341+0000","last_undegraded":"2023-11-24T06:10:26.888341+0000","last_fullsized":"2023-11-24T06:10:26.888341+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:00:56.115984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.887727+0000","last_change":"2023-11-24T06:10:25.721479+0000","last_active":"2023-11-24T06:10:26.887727+0000","last_peered":"2023-11-24T06:10:26.887727+0000","last_clean":"2023-11-24T06:10:26.887727+0000","last_became_active":"2023-11-24T06:10:25.721135+0000","last_became_peered":"2023-11-24T06:10:25.721135+0000","last_unstale":"2023-11-24T06:10:26.887727+0000","last_undegraded":"2023-11-24T06:10:26.887727+0000","last_fullsized":"2023-11-24T06:10:26.887727+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:14:05.090678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.886536+0000","last_change":"2023-11-24T06:10:25.731299+0000","last_active":"2023-11-24T06:10:26.886536+0000","last_peered":"2023-11-24T06:10:26.886536+0000","last_clean":"2023-11-24T06:10:26.886536+0000","last_became_active":"2023-11-24T06:10:25.730627+0000","last_became_peered":"2023-11-24T06:10:25.730627+0000","last_unstale":"2023-11-24T06:10:26.886536+0000","last_undegraded":"2023-11-24T06:10:26.886536+0000","last_fullsized":"2023-11-24T06:10:26.886536+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:53:47.062623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.885972+0000","last_change":"2023-11-24T06:10:25.710797+0000","last_active":"2023-11-24T06:10:26.885972+0000","last_peered":"2023-11-24T06:10:26.885972+0000","last_clean":"2023-11-24T06:10:26.885972+0000","last_became_active":"2023-11-24T06:10:25.710509+0000","last_became_peered":"2023-11-24T06:10:25.710509+0000","last_unstale":"2023-11-24T06:10:26.885972+0000","last_undegraded":"2023-11-24T06:10:26.885972+0000","last_fullsized":"2023-11-24T06:10:26.885972+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:44:12.726705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.714692+0000","last_change":"2023-11-24T06:10:25.721521+0000","last_active":"2023-11-24T06:10:26.714692+0000","last_peered":"2023-11-24T06:10:26.714692+0000","last_clean":"2023-11-24T06:10:26.714692+0000","last_became_active":"2023-11-24T06:10:25.721206+0000","last_became_peered":"2023-11-24T06:10:25.721206+0000","last_unstale":"2023-11-24T06:10:26.714692+0000","last_undegraded":"2023-11-24T06:10:26.714692+0000","last_fullsized":"2023-11-24T06:10:26.714692+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:52:32.288074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.887339+0000","last_change":"2023-11-24T06:10:25.743664+0000","last_active":"2023-11-24T06:10:26.887339+0000","last_peered":"2023-11-24T06:10:26.887339+0000","last_clean":"2023-11-24T06:10:26.887339+0000","last_became_active":"2023-11-24T06:10:25.743346+0000","last_became_peered":"2023-11-24T06:10:25.743346+0000","last_unstale":"2023-11-24T06:10:26.887339+0000","last_undegraded":"2023-11-24T06:10:26.887339+0000","last_fullsized":"2023-11-24T06:10:26.887339+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:59:37.010936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.716729+0000","last_change":"2023-11-24T06:10:25.722633+0000","last_active":"2023-11-24T06:10:26.716729+0000","last_peered":"2023-11-24T06:10:26.716729+0000","last_clean":"2023-11-24T06:10:26.716729+0000","last_became_active":"2023-11-24T06:10:25.722320+0000","last_became_peered":"2023-11-24T06:10:25.722320+0000","last_unstale":"2023-11-24T06:10:26.716729+0000","last_undegraded":"2023-11-24T06:10:26.716729+0000","last_fullsized":"2023-11-24T06:10:26.716729+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:37:01.293292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.716342+0000","last_change":"2023-11-24T06:10:25.721320+0000","last_active":"2023-11-24T06:10:26.716342+0000","last_peered":"2023-11-24T06:10:26.716342+0000","last_clean":"2023-11-24T06:10:26.716342+0000","last_became_active":"2023-11-24T06:10:25.720925+0000","last_became_peered":"2023-11-24T06:10:25.720925+0000","last_unstale":"2023-11-24T06:10:26.716342+0000","last_undegraded":"2023-11-24T06:10:26.716342+0000","last_fullsized":"2023-11-24T06:10:26.716342+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:40:01.407324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.716059+0000","last_change":"2023-11-24T06:10:25.700499+0000","last_active":"2023-11-24T06:10:26.716059+0000","last_peered":"2023-11-24T06:10:26.716059+0000","last_clean":"2023-11-24T06:10:26.716059+0000","last_became_active":"2023-11-24T06:10:25.700070+0000","last_became_peered":"2023-11-24T06:10:25.700070+0000","last_unstale":"2023-11-24T06:10:26.716059+0000","last_undegraded":"2023-11-24T06:10:26.716059+0000","last_fullsized":"2023-11-24T06:10:26.716059+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:22:20.935204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.716045+0000","last_change":"2023-11-24T06:10:25.714833+0000","last_active":"2023-11-24T06:10:26.716045+0000","last_peered":"2023-11-24T06:10:26.716045+0000","last_clean":"2023-11-24T06:10:26.716045+0000","last_became_active":"2023-11-24T06:10:25.714265+0000","last_became_peered":"2023-11-24T06:10:25.714265+0000","last_unstale":"2023-11-24T06:10:26.716045+0000","last_undegraded":"2023-11-24T06:10:26.716045+0000","last_fullsized":"2023-11-24T06:10:26.716045+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:40:01.407324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.885149+0000","last_change":"2023-11-24T06:10:25.710049+0000","last_active":"2023-11-24T06:10:26.885149+0000","last_peered":"2023-11-24T06:10:26.885149+0000","last_clean":"2023-11-24T06:10:26.885149+0000","last_became_active":"2023-11-24T06:10:25.709681+0000","last_became_peered":"2023-11-24T06:10:25.709681+0000","last_unstale":"2023-11-24T06:10:26.885149+0000","last_undegraded":"2023-11-24T06:10:26.885149+0000","last_fullsized":"2023-11-24T06:10:26.885149+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:52:32.288074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.885255+0000","last_change":"2023-11-24T06:10:25.713948+0000","last_active":"2023-11-24T06:10:26.885255+0000","last_peered":"2023-11-24T06:10:26.885255+0000","last_clean":"2023-11-24T06:10:26.885255+0000","last_became_active":"2023-11-24T06:10:25.713633+0000","last_became_peered":"2023-11-24T06:10:25.713633+0000","last_unstale":"2023-11-24T06:10:26.885255+0000","last_undegraded":"2023-11-24T06:10:26.885255+0000","last_fullsized":"2023-11-24T06:10:26.885255+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:18:29.626191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.885915+0000","last_change":"2023-11-24T06:10:25.733647+0000","last_active":"2023-11-24T06:10:26.885915+0000","last_peered":"2023-11-24T06:10:26.885915+0000","last_clean":"2023-11-24T06:10:26.885915+0000","last_became_active":"2023-11-24T06:10:25.733342+0000","last_became_peered":"2023-11-24T06:10:25.733342+0000","last_unstale":"2023-11-24T06:10:26.885915+0000","last_undegraded":"2023-11-24T06:10:26.885915+0000","last_fullsized":"2023-11-24T06:10:26.885915+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:43:50.924468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.889359+0000","last_change":"2023-11-24T06:10:25.737712+0000","last_active":"2023-11-24T06:10:26.889359+0000","last_peered":"2023-11-24T06:10:26.889359+0000","last_clean":"2023-11-24T06:10:26.889359+0000","last_became_active":"2023-11-24T06:10:25.735939+0000","last_became_peered":"2023-11-24T06:10:25.735939+0000","last_unstale":"2023-11-24T06:10:26.889359+0000","last_undegraded":"2023-11-24T06:10:26.889359+0000","last_fullsized":"2023-11-24T06:10:26.889359+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:05:14.621899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.826109+0000","last_change":"2023-11-24T06:10:25.717816+0000","last_active":"2023-11-24T06:10:26.826109+0000","last_peered":"2023-11-24T06:10:26.826109+0000","last_clean":"2023-11-24T06:10:26.826109+0000","last_became_active":"2023-11-24T06:10:25.717132+0000","last_became_peered":"2023-11-24T06:10:25.717132+0000","last_unstale":"2023-11-24T06:10:26.826109+0000","last_undegraded":"2023-11-24T06:10:26.826109+0000","last_fullsized":"2023-11-24T06:10:26.826109+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:17:03.098818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.888164+0000","last_change":"2023-11-24T06:10:25.724997+0000","last_active":"2023-11-24T06:10:26.888164+0000","last_peered":"2023-11-24T06:10:26.888164+0000","last_clean":"2023-11-24T06:10:26.888164+0000","last_became_active":"2023-11-24T06:10:25.716385+0000","last_became_peered":"2023-11-24T06:10:25.716385+0000","last_unstale":"2023-11-24T06:10:26.888164+0000","last_undegraded":"2023-11-24T06:10:26.888164+0000","last_fullsized":"2023-11-24T06:10:26.888164+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:01:41.923534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.716353+0000","last_change":"2023-11-24T06:10:25.717325+0000","last_active":"2023-11-24T06:10:26.716353+0000","last_peered":"2023-11-24T06:10:26.716353+0000","last_clean":"2023-11-24T06:10:26.716353+0000","last_became_active":"2023-11-24T06:10:25.716998+0000","last_became_peered":"2023-11-24T06:10:25.716998+0000","last_unstale":"2023-11-24T06:10:26.716353+0000","last_undegraded":"2023-11-24T06:10:26.716353+0000","last_fullsized":"2023-11-24T06:10:26.716353+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:43:50.924468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.886459+0000","last_change":"2023-11-24T06:10:25.729153+0000","last_active":"2023-11-24T06:10:26.886459+0000","last_peered":"2023-11-24T06:10:26.886459+0000","last_clean":"2023-11-24T06:10:26.886459+0000","last_became_active":"2023-11-24T06:10:25.724982+0000","last_became_peered":"2023-11-24T06:10:25.724982+0000","last_unstale":"2023-11-24T06:10:26.886459+0000","last_undegraded":"2023-11-24T06:10:26.886459+0000","last_fullsized":"2023-11-24T06:10:26.886459+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:46:00.341282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.886867+0000","last_change":"2023-11-24T06:10:25.734081+0000","last_active":"2023-11-24T06:10:26.886867+0000","last_peered":"2023-11-24T06:10:26.886867+0000","last_clean":"2023-11-24T06:10:26.886867+0000","last_became_active":"2023-11-24T06:10:25.733760+0000","last_became_peered":"2023-11-24T06:10:25.733760+0000","last_unstale":"2023-11-24T06:10:26.886867+0000","last_undegraded":"2023-11-24T06:10:26.886867+0000","last_fullsized":"2023-11-24T06:10:26.886867+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:26:40.207866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.715619+0000","last_change":"2023-11-24T06:10:25.720936+0000","last_active":"2023-11-24T06:10:26.715619+0000","last_peered":"2023-11-24T06:10:26.715619+0000","last_clean":"2023-11-24T06:10:26.715619+0000","last_became_active":"2023-11-24T06:10:25.720682+0000","last_became_peered":"2023-11-24T06:10:25.720682+0000","last_unstale":"2023-11-24T06:10:26.715619+0000","last_undegraded":"2023-11-24T06:10:26.715619+0000","last_fullsized":"2023-11-24T06:10:26.715619+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:18:29.626191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.886110+0000","last_change":"2023-11-24T06:10:25.730235+0000","last_active":"2023-11-24T06:10:26.886110+0000","last_peered":"2023-11-24T06:10:26.886110+0000","last_clean":"2023-11-24T06:10:26.886110+0000","last_became_active":"2023-11-24T06:10:25.729972+0000","last_became_peered":"2023-11-24T06:10:25.729972+0000","last_unstale":"2023-11-24T06:10:26.886110+0000","last_undegraded":"2023-11-24T06:10:26.886110+0000","last_fullsized":"2023-11-24T06:10:26.886110+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:37:01.293292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.716777+0000","last_change":"2023-11-24T06:10:25.719429+0000","last_active":"2023-11-24T06:10:26.716777+0000","last_peered":"2023-11-24T06:10:26.716777+0000","last_clean":"2023-11-24T06:10:26.716777+0000","last_became_active":"2023-11-24T06:10:25.719099+0000","last_became_peered":"2023-11-24T06:10:25.719099+0000","last_unstale":"2023-11-24T06:10:26.716777+0000","last_undegraded":"2023-11-24T06:10:26.716777+0000","last_fullsized":"2023-11-24T06:10:26.716777+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:53:47.062623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.716868+0000","last_change":"2023-11-24T06:10:25.710010+0000","last_active":"2023-11-24T06:10:26.716868+0000","last_peered":"2023-11-24T06:10:26.716868+0000","last_clean":"2023-11-24T06:10:26.716868+0000","last_became_active":"2023-11-24T06:10:25.709727+0000","last_became_peered":"2023-11-24T06:10:25.709727+0000","last_unstale":"2023-11-24T06:10:26.716868+0000","last_undegraded":"2023-11-24T06:10:26.716868+0000","last_fullsized":"2023-11-24T06:10:26.716868+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:53:47.062623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.885531+0000","last_change":"2023-11-24T06:10:25.713669+0000","last_active":"2023-11-24T06:10:26.885531+0000","last_peered":"2023-11-24T06:10:26.885531+0000","last_clean":"2023-11-24T06:10:26.885531+0000","last_became_active":"2023-11-24T06:10:25.713339+0000","last_became_peered":"2023-11-24T06:10:25.713339+0000","last_unstale":"2023-11-24T06:10:26.885531+0000","last_undegraded":"2023-11-24T06:10:26.885531+0000","last_fullsized":"2023-11-24T06:10:26.885531+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:40:01.407324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.716342+0000","last_change":"2023-11-24T06:10:25.721615+0000","last_active":"2023-11-24T06:10:26.716342+0000","last_peered":"2023-11-24T06:10:26.716342+0000","last_clean":"2023-11-24T06:10:26.716342+0000","last_became_active":"2023-11-24T06:10:25.721241+0000","last_became_peered":"2023-11-24T06:10:25.721241+0000","last_unstale":"2023-11-24T06:10:26.716342+0000","last_undegraded":"2023-11-24T06:10:26.716342+0000","last_fullsized":"2023-11-24T06:10:26.716342+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:37:01.293292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.886687+0000","last_change":"2023-11-24T06:10:25.713056+0000","last_active":"2023-11-24T06:10:26.886687+0000","last_peered":"2023-11-24T06:10:26.886687+0000","last_clean":"2023-11-24T06:10:26.886687+0000","last_became_active":"2023-11-24T06:10:25.712467+0000","last_became_peered":"2023-11-24T06:10:25.712467+0000","last_unstale":"2023-11-24T06:10:26.886687+0000","last_undegraded":"2023-11-24T06:10:26.886687+0000","last_fullsized":"2023-11-24T06:10:26.886687+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:18:29.626191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.887172+0000","last_change":"2023-11-24T06:10:25.733233+0000","last_active":"2023-11-24T06:10:26.887172+0000","last_peered":"2023-11-24T06:10:26.887172+0000","last_clean":"2023-11-24T06:10:26.887172+0000","last_became_active":"2023-11-24T06:10:25.732933+0000","last_became_peered":"2023-11-24T06:10:25.732933+0000","last_unstale":"2023-11-24T06:10:26.887172+0000","last_undegraded":"2023-11-24T06:10:26.887172+0000","last_fullsized":"2023-11-24T06:10:26.887172+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:44:12.726705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.826224+0000","last_change":"2023-11-24T06:10:25.722324+0000","last_active":"2023-11-24T06:10:26.826224+0000","last_peered":"2023-11-24T06:10:26.826224+0000","last_clean":"2023-11-24T06:10:26.826224+0000","last_became_active":"2023-11-24T06:10:25.722036+0000","last_became_peered":"2023-11-24T06:10:25.722036+0000","last_unstale":"2023-11-24T06:10:26.826224+0000","last_undegraded":"2023-11-24T06:10:26.826224+0000","last_fullsized":"2023-11-24T06:10:26.826224+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:09:06.300144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.715780+0000","last_change":"2023-11-24T06:10:25.703106+0000","last_active":"2023-11-24T06:10:26.715780+0000","last_peered":"2023-11-24T06:10:26.715780+0000","last_clean":"2023-11-24T06:10:26.715780+0000","last_became_active":"2023-11-24T06:10:25.701998+0000","last_became_peered":"2023-11-24T06:10:25.701998+0000","last_unstale":"2023-11-24T06:10:26.715780+0000","last_undegraded":"2023-11-24T06:10:26.715780+0000","last_fullsized":"2023-11-24T06:10:26.715780+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:46:00.341282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.715676+0000","last_change":"2023-11-24T06:10:25.708408+0000","last_active":"2023-11-24T06:10:26.715676+0000","last_peered":"2023-11-24T06:10:26.715676+0000","last_clean":"2023-11-24T06:10:26.715676+0000","last_became_active":"2023-11-24T06:10:25.708102+0000","last_became_peered":"2023-11-24T06:10:25.708102+0000","last_unstale":"2023-11-24T06:10:26.715676+0000","last_undegraded":"2023-11-24T06:10:26.715676+0000","last_fullsized":"2023-11-24T06:10:26.715676+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:38:11.967627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.716406+0000","last_change":"2023-11-24T06:10:24.696915+0000","last_active":"2023-11-24T06:10:26.716406+0000","last_peered":"2023-11-24T06:10:26.716406+0000","last_clean":"2023-11-24T06:10:26.716406+0000","last_became_active":"2023-11-24T06:10:24.696608+0000","last_became_peered":"2023-11-24T06:10:24.696608+0000","last_unstale":"2023-11-24T06:10:26.716406+0000","last_undegraded":"2023-11-24T06:10:26.716406+0000","last_fullsized":"2023-11-24T06:10:26.716406+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_clean_scrub_stamp":"2023-11-24T06:10:23.635255+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:14:19.359955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.886766+0000","last_change":"2023-11-24T06:10:25.713232+0000","last_active":"2023-11-24T06:10:26.886766+0000","last_peered":"2023-11-24T06:10:26.886766+0000","last_clean":"2023-11-24T06:10:26.886766+0000","last_became_active":"2023-11-24T06:10:25.712813+0000","last_became_peered":"2023-11-24T06:10:25.712813+0000","last_unstale":"2023-11-24T06:10:26.886766+0000","last_undegraded":"2023-11-24T06:10:26.886766+0000","last_fullsized":"2023-11-24T06:10:26.886766+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:40:01.407324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.716067+0000","last_change":"2023-11-24T06:10:24.689472+0000","last_active":"2023-11-24T06:10:26.716067+0000","last_peered":"2023-11-24T06:10:26.716067+0000","last_clean":"2023-11-24T06:10:26.716067+0000","last_became_active":"2023-11-24T06:10:24.689143+0000","last_became_peered":"2023-11-24T06:10:24.689143+0000","last_unstale":"2023-11-24T06:10:26.716067+0000","last_undegraded":"2023-11-24T06:10:26.716067+0000","last_fullsized":"2023-11-24T06:10:26.716067+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_clean_scrub_stamp":"2023-11-24T06:10:23.635255+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:18:53.146509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.715380+0000","last_change":"2023-11-24T06:10:24.694706+0000","last_active":"2023-11-24T06:10:26.715380+0000","last_peered":"2023-11-24T06:10:26.715380+0000","last_clean":"2023-11-24T06:10:26.715380+0000","last_became_active":"2023-11-24T06:10:24.693990+0000","last_became_peered":"2023-11-24T06:10:24.693990+0000","last_unstale":"2023-11-24T06:10:26.715380+0000","last_undegraded":"2023-11-24T06:10:26.715380+0000","last_fullsized":"2023-11-24T06:10:26.715380+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_clean_scrub_stamp":"2023-11-24T06:10:23.635255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:17:02.081734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.714912+0000","last_change":"2023-11-24T06:10:25.715897+0000","last_active":"2023-11-24T06:10:26.714912+0000","last_peered":"2023-11-24T06:10:26.714912+0000","last_clean":"2023-11-24T06:10:26.714912+0000","last_became_active":"2023-11-24T06:10:25.715596+0000","last_became_peered":"2023-11-24T06:10:25.715596+0000","last_unstale":"2023-11-24T06:10:26.714912+0000","last_undegraded":"2023-11-24T06:10:26.714912+0000","last_fullsized":"2023-11-24T06:10:26.714912+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:54:02.837130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.886161+0000","last_change":"2023-11-24T06:10:24.692669+0000","last_active":"2023-11-24T06:10:26.886161+0000","last_peered":"2023-11-24T06:10:26.886161+0000","last_clean":"2023-11-24T06:10:26.886161+0000","last_became_active":"2023-11-24T06:10:24.692341+0000","last_became_peered":"2023-11-24T06:10:24.692341+0000","last_unstale":"2023-11-24T06:10:26.886161+0000","last_undegraded":"2023-11-24T06:10:26.886161+0000","last_fullsized":"2023-11-24T06:10:26.886161+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_clean_scrub_stamp":"2023-11-24T06:10:23.635255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:09:05.283059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.885105+0000","last_change":"2023-11-24T06:10:25.713815+0000","last_active":"2023-11-24T06:10:26.885105+0000","last_peered":"2023-11-24T06:10:26.885105+0000","last_clean":"2023-11-24T06:10:26.885105+0000","last_became_active":"2023-11-24T06:10:25.713207+0000","last_became_peered":"2023-11-24T06:10:25.713207+0000","last_unstale":"2023-11-24T06:10:26.885105+0000","last_undegraded":"2023-11-24T06:10:26.885105+0000","last_fullsized":"2023-11-24T06:10:26.885105+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:46:00.341282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.716681+0000","last_change":"2023-11-24T06:10:24.700771+0000","last_active":"2023-11-24T06:10:26.716681+0000","last_peered":"2023-11-24T06:10:26.716681+0000","last_clean":"2023-11-24T06:10:26.716681+0000","last_became_active":"2023-11-24T06:10:24.699736+0000","last_became_peered":"2023-11-24T06:10:24.699736+0000","last_unstale":"2023-11-24T06:10:26.716681+0000","last_undegraded":"2023-11-24T06:10:26.716681+0000","last_fullsized":"2023-11-24T06:10:26.716681+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_clean_scrub_stamp":"2023-11-24T06:10:23.635255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:10:14.268052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.888828+0000","last_change":"2023-11-24T06:10:25.735798+0000","last_active":"2023-11-24T06:10:26.888828+0000","last_peered":"2023-11-24T06:10:26.888828+0000","last_clean":"2023-11-24T06:10:26.888828+0000","last_became_active":"2023-11-24T06:10:25.735161+0000","last_became_peered":"2023-11-24T06:10:25.735161+0000","last_unstale":"2023-11-24T06:10:26.888828+0000","last_undegraded":"2023-11-24T06:10:26.888828+0000","last_fullsized":"2023-11-24T06:10:26.888828+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:56:03.801033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.888836+0000","last_change":"2023-11-24T06:10:24.694962+0000","last_active":"2023-11-24T06:10:26.888836+0000","last_peered":"2023-11-24T06:10:26.888836+0000","last_clean":"2023-11-24T06:10:26.888836+0000","last_became_active":"2023-11-24T06:10:24.694740+0000","last_became_peered":"2023-11-24T06:10:24.694740+0000","last_unstale":"2023-11-24T06:10:26.888836+0000","last_undegraded":"2023-11-24T06:10:26.888836+0000","last_fullsized":"2023-11-24T06:10:26.888836+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_clean_scrub_stamp":"2023-11-24T06:10:23.635255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:45:59.324198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.887375+0000","last_change":"2023-11-24T06:10:25.732152+0000","last_active":"2023-11-24T06:10:26.887375+0000","last_peered":"2023-11-24T06:10:26.887375+0000","last_clean":"2023-11-24T06:10:26.887375+0000","last_became_active":"2023-11-24T06:10:25.731460+0000","last_became_peered":"2023-11-24T06:10:25.731460+0000","last_unstale":"2023-11-24T06:10:26.887375+0000","last_undegraded":"2023-11-24T06:10:26.887375+0000","last_fullsized":"2023-11-24T06:10:26.887375+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:53:47.062623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.885628+0000","last_change":"2023-11-24T06:10:24.692129+0000","last_active":"2023-11-24T06:10:26.885628+0000","last_peered":"2023-11-24T06:10:26.885628+0000","last_clean":"2023-11-24T06:10:26.885628+0000","last_became_active":"2023-11-24T06:10:24.691871+0000","last_became_peered":"2023-11-24T06:10:24.691871+0000","last_unstale":"2023-11-24T06:10:26.885628+0000","last_undegraded":"2023-11-24T06:10:26.885628+0000","last_fullsized":"2023-11-24T06:10:26.885628+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_clean_scrub_stamp":"2023-11-24T06:10:23.635255+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:14:19.359955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.717432+0000","last_change":"2023-11-24T06:10:25.718217+0000","last_active":"2023-11-24T06:10:26.717432+0000","last_peered":"2023-11-24T06:10:26.717432+0000","last_clean":"2023-11-24T06:10:26.717432+0000","last_became_active":"2023-11-24T06:10:25.717956+0000","last_became_peered":"2023-11-24T06:10:25.717956+0000","last_unstale":"2023-11-24T06:10:26.717432+0000","last_undegraded":"2023-11-24T06:10:26.717432+0000","last_fullsized":"2023-11-24T06:10:26.717432+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:09:06.300144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.716773+0000","last_change":"2023-11-24T06:10:24.694293+0000","last_active":"2023-11-24T06:10:26.716773+0000","last_peered":"2023-11-24T06:10:26.716773+0000","last_clean":"2023-11-24T06:10:26.716773+0000","last_became_active":"2023-11-24T06:10:24.694005+0000","last_became_peered":"2023-11-24T06:10:24.694005+0000","last_unstale":"2023-11-24T06:10:26.716773+0000","last_undegraded":"2023-11-24T06:10:26.716773+0000","last_fullsized":"2023-11-24T06:10:26.716773+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_clean_scrub_stamp":"2023-11-24T06:10:23.635255+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:54:01.820045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.887235+0000","last_change":"2023-11-24T06:10:25.716082+0000","last_active":"2023-11-24T06:10:26.887235+0000","last_peered":"2023-11-24T06:10:26.887235+0000","last_clean":"2023-11-24T06:10:26.887235+0000","last_became_active":"2023-11-24T06:10:25.713813+0000","last_became_peered":"2023-11-24T06:10:25.713813+0000","last_unstale":"2023-11-24T06:10:26.887235+0000","last_undegraded":"2023-11-24T06:10:26.887235+0000","last_fullsized":"2023-11-24T06:10:26.887235+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:42:25.852028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.886925+0000","last_change":"2023-11-24T06:10:24.687076+0000","last_active":"2023-11-24T06:10:26.886925+0000","last_peered":"2023-11-24T06:10:26.886925+0000","last_clean":"2023-11-24T06:10:26.886925+0000","last_became_active":"2023-11-24T06:10:24.686789+0000","last_became_peered":"2023-11-24T06:10:24.686789+0000","last_unstale":"2023-11-24T06:10:26.886925+0000","last_undegraded":"2023-11-24T06:10:26.886925+0000","last_fullsized":"2023-11-24T06:10:26.886925+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_clean_scrub_stamp":"2023-11-24T06:10:23.635255+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:38:10.950543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.885206+0000","last_change":"2023-11-24T06:10:25.704389+0000","last_active":"2023-11-24T06:10:26.885206+0000","last_peered":"2023-11-24T06:10:26.885206+0000","last_clean":"2023-11-24T06:10:26.885206+0000","last_became_active":"2023-11-24T06:10:25.704101+0000","last_became_peered":"2023-11-24T06:10:25.704101+0000","last_unstale":"2023-11-24T06:10:26.885206+0000","last_undegraded":"2023-11-24T06:10:26.885206+0000","last_fullsized":"2023-11-24T06:10:26.885206+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:54:02.837130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.887386+0000","last_change":"2023-11-24T06:10:24.691369+0000","last_active":"2023-11-24T06:10:26.887386+0000","last_peered":"2023-11-24T06:10:26.887386+0000","last_clean":"2023-11-24T06:10:26.887386+0000","last_became_active":"2023-11-24T06:10:24.691053+0000","last_became_peered":"2023-11-24T06:10:24.691053+0000","last_unstale":"2023-11-24T06:10:26.887386+0000","last_undegraded":"2023-11-24T06:10:26.887386+0000","last_fullsized":"2023-11-24T06:10:26.887386+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_clean_scrub_stamp":"2023-11-24T06:10:23.635255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:18:28.609106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.887207+0000","last_change":"2023-11-24T06:10:25.730194+0000","last_active":"2023-11-24T06:10:26.887207+0000","last_peered":"2023-11-24T06:10:26.887207+0000","last_clean":"2023-11-24T06:10:26.887207+0000","last_became_active":"2023-11-24T06:10:25.729811+0000","last_became_peered":"2023-11-24T06:10:25.729811+0000","last_unstale":"2023-11-24T06:10:26.887207+0000","last_undegraded":"2023-11-24T06:10:26.887207+0000","last_fullsized":"2023-11-24T06:10:26.887207+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:03:41.511367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.716013+0000","last_change":"2023-11-24T06:10:24.691658+0000","last_active":"2023-11-24T06:10:26.716013+0000","last_peered":"2023-11-24T06:10:26.716013+0000","last_clean":"2023-11-24T06:10:26.716013+0000","last_became_active":"2023-11-24T06:10:24.691365+0000","last_became_peered":"2023-11-24T06:10:24.691365+0000","last_unstale":"2023-11-24T06:10:26.716013+0000","last_undegraded":"2023-11-24T06:10:26.716013+0000","last_fullsized":"2023-11-24T06:10:26.716013+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_clean_scrub_stamp":"2023-11-24T06:10:23.635255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:17:02.081734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.715937+0000","last_change":"2023-11-24T06:10:25.703380+0000","last_active":"2023-11-24T06:10:26.715937+0000","last_peered":"2023-11-24T06:10:26.715937+0000","last_clean":"2023-11-24T06:10:26.715937+0000","last_became_active":"2023-11-24T06:10:25.702696+0000","last_became_peered":"2023-11-24T06:10:25.702696+0000","last_unstale":"2023-11-24T06:10:26.715937+0000","last_undegraded":"2023-11-24T06:10:26.715937+0000","last_fullsized":"2023-11-24T06:10:26.715937+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:18:29.626191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.825875+0000","last_change":"2023-11-24T06:10:24.683519+0000","last_active":"2023-11-24T06:10:26.825875+0000","last_peered":"2023-11-24T06:10:26.825875+0000","last_clean":"2023-11-24T06:10:26.825875+0000","last_became_active":"2023-11-24T06:10:24.682405+0000","last_became_peered":"2023-11-24T06:10:24.682405+0000","last_unstale":"2023-11-24T06:10:26.825875+0000","last_undegraded":"2023-11-24T06:10:26.825875+0000","last_fullsized":"2023-11-24T06:10:26.825875+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_clean_scrub_stamp":"2023-11-24T06:10:23.635255+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:20:39.581356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.886486+0000","last_change":"2023-11-24T06:10:25.726307+0000","last_active":"2023-11-24T06:10:26.886486+0000","last_peered":"2023-11-24T06:10:26.886486+0000","last_clean":"2023-11-24T06:10:26.886486+0000","last_became_active":"2023-11-24T06:10:25.726038+0000","last_became_peered":"2023-11-24T06:10:25.726038+0000","last_unstale":"2023-11-24T06:10:26.886486+0000","last_undegraded":"2023-11-24T06:10:26.886486+0000","last_fullsized":"2023-11-24T06:10:26.886486+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:00:56.115984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.717432+0000","last_change":"2023-11-24T06:10:24.692338+0000","last_active":"2023-11-24T06:10:26.717432+0000","last_peered":"2023-11-24T06:10:26.717432+0000","last_clean":"2023-11-24T06:10:26.717432+0000","last_became_active":"2023-11-24T06:10:24.692135+0000","last_became_peered":"2023-11-24T06:10:24.692135+0000","last_unstale":"2023-11-24T06:10:26.717432+0000","last_undegraded":"2023-11-24T06:10:26.717432+0000","last_fullsized":"2023-11-24T06:10:26.717432+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_clean_scrub_stamp":"2023-11-24T06:10:23.635255+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:57:48.799543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.887337+0000","last_change":"2023-11-24T06:10:25.729383+0000","last_active":"2023-11-24T06:10:26.887337+0000","last_peered":"2023-11-24T06:10:26.887337+0000","last_clean":"2023-11-24T06:10:26.887337+0000","last_became_active":"2023-11-24T06:10:25.729090+0000","last_became_peered":"2023-11-24T06:10:25.729090+0000","last_unstale":"2023-11-24T06:10:26.887337+0000","last_undegraded":"2023-11-24T06:10:26.887337+0000","last_fullsized":"2023-11-24T06:10:26.887337+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:40:47.125372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.826232+0000","last_change":"2023-11-24T06:10:24.689238+0000","last_active":"2023-11-24T06:10:26.826232+0000","last_peered":"2023-11-24T06:10:26.826232+0000","last_clean":"2023-11-24T06:10:26.826232+0000","last_became_active":"2023-11-24T06:10:24.686650+0000","last_became_peered":"2023-11-24T06:10:24.686650+0000","last_unstale":"2023-11-24T06:10:26.826232+0000","last_undegraded":"2023-11-24T06:10:26.826232+0000","last_fullsized":"2023-11-24T06:10:26.826232+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_clean_scrub_stamp":"2023-11-24T06:10:23.635255+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:06:50.936889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.887656+0000","last_change":"2023-11-24T06:10:25.726737+0000","last_active":"2023-11-24T06:10:26.887656+0000","last_peered":"2023-11-24T06:10:26.887656+0000","last_clean":"2023-11-24T06:10:26.887656+0000","last_became_active":"2023-11-24T06:10:25.725070+0000","last_became_peered":"2023-11-24T06:10:25.725070+0000","last_unstale":"2023-11-24T06:10:26.887656+0000","last_undegraded":"2023-11-24T06:10:26.887656+0000","last_fullsized":"2023-11-24T06:10:26.887656+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:22:18.598271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.885715+0000","last_change":"2023-11-24T06:10:24.681713+0000","last_active":"2023-11-24T06:10:26.885715+0000","last_peered":"2023-11-24T06:10:26.885715+0000","last_clean":"2023-11-24T06:10:26.885715+0000","last_became_active":"2023-11-24T06:10:24.681336+0000","last_became_peered":"2023-11-24T06:10:24.681336+0000","last_unstale":"2023-11-24T06:10:26.885715+0000","last_undegraded":"2023-11-24T06:10:26.885715+0000","last_fullsized":"2023-11-24T06:10:26.885715+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_clean_scrub_stamp":"2023-11-24T06:10:23.635255+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:10:14.268052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.888736+0000","last_change":"2023-11-24T06:10:25.733821+0000","last_active":"2023-11-24T06:10:26.888736+0000","last_peered":"2023-11-24T06:10:26.888736+0000","last_clean":"2023-11-24T06:10:26.888736+0000","last_became_active":"2023-11-24T06:10:25.733510+0000","last_became_peered":"2023-11-24T06:10:25.733510+0000","last_unstale":"2023-11-24T06:10:26.888736+0000","last_undegraded":"2023-11-24T06:10:26.888736+0000","last_fullsized":"2023-11-24T06:10:26.888736+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:42:25.852028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.826174+0000","last_change":"2023-11-24T06:10:24.690450+0000","last_active":"2023-11-24T06:10:26.826174+0000","last_peered":"2023-11-24T06:10:26.826174+0000","last_clean":"2023-11-24T06:10:26.826174+0000","last_became_active":"2023-11-24T06:10:24.690024+0000","last_became_peered":"2023-11-24T06:10:24.690024+0000","last_unstale":"2023-11-24T06:10:26.826174+0000","last_undegraded":"2023-11-24T06:10:26.826174+0000","last_fullsized":"2023-11-24T06:10:26.826174+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_clean_scrub_stamp":"2023-11-24T06:10:23.635255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:51:55.108153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.887639+0000","last_change":"2023-11-24T06:10:25.734784+0000","last_active":"2023-11-24T06:10:26.887639+0000","last_peered":"2023-11-24T06:10:26.887639+0000","last_clean":"2023-11-24T06:10:26.887639+0000","last_became_active":"2023-11-24T06:10:25.734478+0000","last_became_peered":"2023-11-24T06:10:25.734478+0000","last_unstale":"2023-11-24T06:10:26.887639+0000","last_undegraded":"2023-11-24T06:10:26.887639+0000","last_fullsized":"2023-11-24T06:10:26.887639+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:05:14.621899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":165,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:10:25.700259+0000","last_change":"2023-11-24T06:10:20.082677+0000","last_active":"2023-11-24T06:10:25.700259+0000","last_peered":"2023-11-24T06:10:25.700259+0000","last_clean":"2023-11-24T06:10:25.700259+0000","last_became_active":"2023-11-24T06:10:20.080551+0000","last_became_peered":"2023-11-24T06:10:20.080551+0000","last_unstale":"2023-11-24T06:10:25.700259+0000","last_undegraded":"2023-11-24T06:10:25.700259+0000","last_fullsized":"2023-11-24T06:10:25.700259+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:18.602209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:18.602209+0000","last_clean_scrub_stamp":"2023-11-24T06:10:18.602209+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:47:24.216235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.887865+0000","last_change":"2023-11-24T06:10:24.684957+0000","last_active":"2023-11-24T06:10:26.887865+0000","last_peered":"2023-11-24T06:10:26.887865+0000","last_clean":"2023-11-24T06:10:26.887865+0000","last_became_active":"2023-11-24T06:10:24.684507+0000","last_became_peered":"2023-11-24T06:10:24.684507+0000","last_unstale":"2023-11-24T06:10:26.887865+0000","last_undegraded":"2023-11-24T06:10:26.887865+0000","last_fullsized":"2023-11-24T06:10:26.887865+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_clean_scrub_stamp":"2023-11-24T06:10:23.635255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:52:31.270989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.715709+0000","last_change":"2023-11-24T06:10:25.735264+0000","last_active":"2023-11-24T06:10:26.715709+0000","last_peered":"2023-11-24T06:10:26.715709+0000","last_clean":"2023-11-24T06:10:26.715709+0000","last_became_active":"2023-11-24T06:10:25.734163+0000","last_became_peered":"2023-11-24T06:10:25.734163+0000","last_unstale":"2023-11-24T06:10:26.715709+0000","last_undegraded":"2023-11-24T06:10:26.715709+0000","last_fullsized":"2023-11-24T06:10:26.715709+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:45:21.592844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.885683+0000","last_change":"2023-11-24T06:10:24.699469+0000","last_active":"2023-11-24T06:10:26.885683+0000","last_peered":"2023-11-24T06:10:26.885683+0000","last_clean":"2023-11-24T06:10:26.885683+0000","last_became_active":"2023-11-24T06:10:24.699153+0000","last_became_peered":"2023-11-24T06:10:24.699153+0000","last_unstale":"2023-11-24T06:10:26.885683+0000","last_undegraded":"2023-11-24T06:10:26.885683+0000","last_fullsized":"2023-11-24T06:10:26.885683+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_clean_scrub_stamp":"2023-11-24T06:10:23.635255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:09:05.283059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.884898+0000","last_change":"2023-11-24T06:10:25.716501+0000","last_active":"2023-11-24T06:10:26.884898+0000","last_peered":"2023-11-24T06:10:26.884898+0000","last_clean":"2023-11-24T06:10:26.884898+0000","last_became_active":"2023-11-24T06:10:25.712127+0000","last_became_peered":"2023-11-24T06:10:25.712127+0000","last_unstale":"2023-11-24T06:10:26.884898+0000","last_undegraded":"2023-11-24T06:10:26.884898+0000","last_fullsized":"2023-11-24T06:10:26.884898+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:10:15.285136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.826108+0000","last_change":"2023-11-24T06:10:24.686855+0000","last_active":"2023-11-24T06:10:26.826108+0000","last_peered":"2023-11-24T06:10:26.826108+0000","last_clean":"2023-11-24T06:10:26.826108+0000","last_became_active":"2023-11-24T06:10:24.685894+0000","last_became_peered":"2023-11-24T06:10:24.685894+0000","last_unstale":"2023-11-24T06:10:26.826108+0000","last_undegraded":"2023-11-24T06:10:26.826108+0000","last_fullsized":"2023-11-24T06:10:26.826108+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_clean_scrub_stamp":"2023-11-24T06:10:23.635255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:38:10.950543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.885731+0000","last_change":"2023-11-24T06:10:25.715833+0000","last_active":"2023-11-24T06:10:26.885731+0000","last_peered":"2023-11-24T06:10:26.885731+0000","last_clean":"2023-11-24T06:10:26.885731+0000","last_became_active":"2023-11-24T06:10:25.715500+0000","last_became_peered":"2023-11-24T06:10:25.715500+0000","last_unstale":"2023-11-24T06:10:26.885731+0000","last_undegraded":"2023-11-24T06:10:26.885731+0000","last_fullsized":"2023-11-24T06:10:26.885731+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:43:50.924468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.887610+0000","last_change":"2023-11-24T06:10:24.690139+0000","last_active":"2023-11-24T06:10:26.887610+0000","last_peered":"2023-11-24T06:10:26.887610+0000","last_clean":"2023-11-24T06:10:26.887610+0000","last_became_active":"2023-11-24T06:10:24.689734+0000","last_became_peered":"2023-11-24T06:10:24.689734+0000","last_unstale":"2023-11-24T06:10:26.887610+0000","last_undegraded":"2023-11-24T06:10:26.887610+0000","last_fullsized":"2023-11-24T06:10:26.887610+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_clean_scrub_stamp":"2023-11-24T06:10:23.635255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:45:20.575759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.717184+0000","last_change":"2023-11-24T06:10:25.710787+0000","last_active":"2023-11-24T06:10:26.717184+0000","last_peered":"2023-11-24T06:10:26.717184+0000","last_clean":"2023-11-24T06:10:26.717184+0000","last_became_active":"2023-11-24T06:10:25.710522+0000","last_became_peered":"2023-11-24T06:10:25.710522+0000","last_unstale":"2023-11-24T06:10:26.717184+0000","last_undegraded":"2023-11-24T06:10:26.717184+0000","last_fullsized":"2023-11-24T06:10:26.717184+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:17:03.098818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.885579+0000","last_change":"2023-11-24T06:10:24.683405+0000","last_active":"2023-11-24T06:10:26.885579+0000","last_peered":"2023-11-24T06:10:26.885579+0000","last_clean":"2023-11-24T06:10:26.885579+0000","last_became_active":"2023-11-24T06:10:24.683144+0000","last_became_peered":"2023-11-24T06:10:24.683144+0000","last_unstale":"2023-11-24T06:10:26.885579+0000","last_undegraded":"2023-11-24T06:10:26.885579+0000","last_fullsized":"2023-11-24T06:10:26.885579+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_clean_scrub_stamp":"2023-11-24T06:10:23.635255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:18:53.146509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.888979+0000","last_change":"2023-11-24T06:10:25.734738+0000","last_active":"2023-11-24T06:10:26.888979+0000","last_peered":"2023-11-24T06:10:26.888979+0000","last_clean":"2023-11-24T06:10:26.888979+0000","last_became_active":"2023-11-24T06:10:25.733409+0000","last_became_peered":"2023-11-24T06:10:25.733409+0000","last_unstale":"2023-11-24T06:10:26.888979+0000","last_undegraded":"2023-11-24T06:10:26.888979+0000","last_fullsized":"2023-11-24T06:10:26.888979+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:59:37.010936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.888082+0000","last_change":"2023-11-24T06:10:24.701942+0000","last_active":"2023-11-24T06:10:26.888082+0000","last_peered":"2023-11-24T06:10:26.888082+0000","last_clean":"2023-11-24T06:10:26.888082+0000","last_became_active":"2023-11-24T06:10:24.701385+0000","last_became_peered":"2023-11-24T06:10:24.701385+0000","last_unstale":"2023-11-24T06:10:26.888082+0000","last_undegraded":"2023-11-24T06:10:26.888082+0000","last_fullsized":"2023-11-24T06:10:26.888082+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_clean_scrub_stamp":"2023-11-24T06:10:23.635255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:26:39.190781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.885358+0000","last_change":"2023-11-24T06:10:25.711825+0000","last_active":"2023-11-24T06:10:26.885358+0000","last_peered":"2023-11-24T06:10:26.885358+0000","last_clean":"2023-11-24T06:10:26.885358+0000","last_became_active":"2023-11-24T06:10:25.711161+0000","last_became_peered":"2023-11-24T06:10:25.711161+0000","last_unstale":"2023-11-24T06:10:26.885358+0000","last_undegraded":"2023-11-24T06:10:26.885358+0000","last_fullsized":"2023-11-24T06:10:26.885358+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:45:21.592844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.886084+0000","last_change":"2023-11-24T06:10:24.695283+0000","last_active":"2023-11-24T06:10:26.886084+0000","last_peered":"2023-11-24T06:10:26.886084+0000","last_clean":"2023-11-24T06:10:26.886084+0000","last_became_active":"2023-11-24T06:10:24.694723+0000","last_became_peered":"2023-11-24T06:10:24.694723+0000","last_unstale":"2023-11-24T06:10:26.886084+0000","last_undegraded":"2023-11-24T06:10:26.886084+0000","last_fullsized":"2023-11-24T06:10:26.886084+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_clean_scrub_stamp":"2023-11-24T06:10:23.635255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:52:31.270989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.716963+0000","last_change":"2023-11-24T06:10:25.730440+0000","last_active":"2023-11-24T06:10:26.716963+0000","last_peered":"2023-11-24T06:10:26.716963+0000","last_clean":"2023-11-24T06:10:26.716963+0000","last_became_active":"2023-11-24T06:10:25.730097+0000","last_became_peered":"2023-11-24T06:10:25.730097+0000","last_unstale":"2023-11-24T06:10:26.716963+0000","last_undegraded":"2023-11-24T06:10:26.716963+0000","last_fullsized":"2023-11-24T06:10:26.716963+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:44:12.726705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.716893+0000","last_change":"2023-11-24T06:10:24.702923+0000","last_active":"2023-11-24T06:10:26.716893+0000","last_peered":"2023-11-24T06:10:26.716893+0000","last_clean":"2023-11-24T06:10:26.716893+0000","last_became_active":"2023-11-24T06:10:24.702428+0000","last_became_peered":"2023-11-24T06:10:24.702428+0000","last_unstale":"2023-11-24T06:10:26.716893+0000","last_undegraded":"2023-11-24T06:10:26.716893+0000","last_fullsized":"2023-11-24T06:10:26.716893+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_clean_scrub_stamp":"2023-11-24T06:10:23.635255+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:52:31.270989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.886761+0000","last_change":"2023-11-24T06:10:25.709739+0000","last_active":"2023-11-24T06:10:26.886761+0000","last_peered":"2023-11-24T06:10:26.886761+0000","last_clean":"2023-11-24T06:10:26.886761+0000","last_became_active":"2023-11-24T06:10:25.709392+0000","last_became_peered":"2023-11-24T06:10:25.709392+0000","last_unstale":"2023-11-24T06:10:26.886761+0000","last_undegraded":"2023-11-24T06:10:26.886761+0000","last_fullsized":"2023-11-24T06:10:26.886761+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:45:21.592844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.886005+0000","last_change":"2023-11-24T06:10:24.698896+0000","last_active":"2023-11-24T06:10:26.886005+0000","last_peered":"2023-11-24T06:10:26.886005+0000","last_clean":"2023-11-24T06:10:26.886005+0000","last_became_active":"2023-11-24T06:10:24.698632+0000","last_became_peered":"2023-11-24T06:10:24.698632+0000","last_unstale":"2023-11-24T06:10:26.886005+0000","last_undegraded":"2023-11-24T06:10:26.886005+0000","last_fullsized":"2023-11-24T06:10:26.886005+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_clean_scrub_stamp":"2023-11-24T06:10:23.635255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:22:19.918119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.715380+0000","last_change":"2023-11-24T06:10:25.716284+0000","last_active":"2023-11-24T06:10:26.715380+0000","last_peered":"2023-11-24T06:10:26.715380+0000","last_clean":"2023-11-24T06:10:26.715380+0000","last_became_active":"2023-11-24T06:10:25.715948+0000","last_became_peered":"2023-11-24T06:10:25.715948+0000","last_unstale":"2023-11-24T06:10:26.715380+0000","last_undegraded":"2023-11-24T06:10:26.715380+0000","last_fullsized":"2023-11-24T06:10:26.715380+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:20:40.598440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.885366+0000","last_change":"2023-11-24T06:10:24.692784+0000","last_active":"2023-11-24T06:10:26.885366+0000","last_peered":"2023-11-24T06:10:26.885366+0000","last_clean":"2023-11-24T06:10:26.885366+0000","last_became_active":"2023-11-24T06:10:24.692514+0000","last_became_peered":"2023-11-24T06:10:24.692514+0000","last_unstale":"2023-11-24T06:10:26.885366+0000","last_undegraded":"2023-11-24T06:10:26.885366+0000","last_fullsized":"2023-11-24T06:10:26.885366+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_clean_scrub_stamp":"2023-11-24T06:10:23.635255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:38:10.950543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.715214+0000","last_change":"2023-11-24T06:10:25.712156+0000","last_active":"2023-11-24T06:10:26.715214+0000","last_peered":"2023-11-24T06:10:26.715214+0000","last_clean":"2023-11-24T06:10:26.715214+0000","last_became_active":"2023-11-24T06:10:25.711924+0000","last_became_peered":"2023-11-24T06:10:25.711924+0000","last_unstale":"2023-11-24T06:10:26.715214+0000","last_undegraded":"2023-11-24T06:10:26.715214+0000","last_fullsized":"2023-11-24T06:10:26.715214+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:46:00.341282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.885937+0000","last_change":"2023-11-24T06:10:24.695919+0000","last_active":"2023-11-24T06:10:26.885937+0000","last_peered":"2023-11-24T06:10:26.885937+0000","last_clean":"2023-11-24T06:10:26.885937+0000","last_became_active":"2023-11-24T06:10:24.695650+0000","last_became_peered":"2023-11-24T06:10:24.695650+0000","last_unstale":"2023-11-24T06:10:26.885937+0000","last_undegraded":"2023-11-24T06:10:26.885937+0000","last_fullsized":"2023-11-24T06:10:26.885937+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_clean_scrub_stamp":"2023-11-24T06:10:23.635255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:09:05.283059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.716052+0000","last_change":"2023-11-24T06:10:25.719797+0000","last_active":"2023-11-24T06:10:26.716052+0000","last_peered":"2023-11-24T06:10:26.716052+0000","last_clean":"2023-11-24T06:10:26.716052+0000","last_became_active":"2023-11-24T06:10:25.719483+0000","last_became_peered":"2023-11-24T06:10:25.719483+0000","last_unstale":"2023-11-24T06:10:26.716052+0000","last_undegraded":"2023-11-24T06:10:26.716052+0000","last_fullsized":"2023-11-24T06:10:26.716052+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:43:50.924468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.887695+0000","last_change":"2023-11-24T06:10:24.699236+0000","last_active":"2023-11-24T06:10:26.887695+0000","last_peered":"2023-11-24T06:10:26.887695+0000","last_clean":"2023-11-24T06:10:26.887695+0000","last_became_active":"2023-11-24T06:10:24.698709+0000","last_became_peered":"2023-11-24T06:10:24.698709+0000","last_unstale":"2023-11-24T06:10:26.887695+0000","last_undegraded":"2023-11-24T06:10:26.887695+0000","last_fullsized":"2023-11-24T06:10:26.887695+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_clean_scrub_stamp":"2023-11-24T06:10:23.635255+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:37:00.276208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.716147+0000","last_change":"2023-11-24T06:10:25.714752+0000","last_active":"2023-11-24T06:10:26.716147+0000","last_peered":"2023-11-24T06:10:26.716147+0000","last_clean":"2023-11-24T06:10:26.716147+0000","last_became_active":"2023-11-24T06:10:25.714468+0000","last_became_peered":"2023-11-24T06:10:25.714468+0000","last_unstale":"2023-11-24T06:10:26.716147+0000","last_undegraded":"2023-11-24T06:10:26.716147+0000","last_fullsized":"2023-11-24T06:10:26.716147+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:45:21.592844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.888957+0000","last_change":"2023-11-24T06:10:25.719973+0000","last_active":"2023-11-24T06:10:26.888957+0000","last_peered":"2023-11-24T06:10:26.888957+0000","last_clean":"2023-11-24T06:10:26.888957+0000","last_became_active":"2023-11-24T06:10:25.719716+0000","last_became_peered":"2023-11-24T06:10:25.719716+0000","last_unstale":"2023-11-24T06:10:26.888957+0000","last_undegraded":"2023-11-24T06:10:26.888957+0000","last_fullsized":"2023-11-24T06:10:26.888957+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:40:47.125372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.886889+0000","last_change":"2023-11-24T06:10:24.701875+0000","last_active":"2023-11-24T06:10:26.886889+0000","last_peered":"2023-11-24T06:10:26.886889+0000","last_clean":"2023-11-24T06:10:26.886889+0000","last_became_active":"2023-11-24T06:10:24.701520+0000","last_became_peered":"2023-11-24T06:10:24.701520+0000","last_unstale":"2023-11-24T06:10:26.886889+0000","last_undegraded":"2023-11-24T06:10:26.886889+0000","last_fullsized":"2023-11-24T06:10:26.886889+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_clean_scrub_stamp":"2023-11-24T06:10:23.635255+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:18:53.146509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.888401+0000","last_change":"2023-11-24T06:10:25.729759+0000","last_active":"2023-11-24T06:10:26.888401+0000","last_peered":"2023-11-24T06:10:26.888401+0000","last_clean":"2023-11-24T06:10:26.888401+0000","last_became_active":"2023-11-24T06:10:25.721916+0000","last_became_peered":"2023-11-24T06:10:25.721916+0000","last_unstale":"2023-11-24T06:10:26.888401+0000","last_undegraded":"2023-11-24T06:10:26.888401+0000","last_fullsized":"2023-11-24T06:10:26.888401+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:03:41.511367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.887901+0000","last_change":"2023-11-24T06:10:24.699400+0000","last_active":"2023-11-24T06:10:26.887901+0000","last_peered":"2023-11-24T06:10:26.887901+0000","last_clean":"2023-11-24T06:10:26.887901+0000","last_became_active":"2023-11-24T06:10:24.699090+0000","last_became_peered":"2023-11-24T06:10:24.699090+0000","last_unstale":"2023-11-24T06:10:26.887901+0000","last_undegraded":"2023-11-24T06:10:26.887901+0000","last_fullsized":"2023-11-24T06:10:26.887901+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_clean_scrub_stamp":"2023-11-24T06:10:23.635255+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:14:19.359955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.887209+0000","last_change":"2023-11-24T06:10:25.732187+0000","last_active":"2023-11-24T06:10:26.887209+0000","last_peered":"2023-11-24T06:10:26.887209+0000","last_clean":"2023-11-24T06:10:26.887209+0000","last_became_active":"2023-11-24T06:10:25.731130+0000","last_became_peered":"2023-11-24T06:10:25.731130+0000","last_unstale":"2023-11-24T06:10:26.887209+0000","last_undegraded":"2023-11-24T06:10:26.887209+0000","last_fullsized":"2023-11-24T06:10:26.887209+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:37:01.293292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.716804+0000","last_change":"2023-11-24T06:10:24.694116+0000","last_active":"2023-11-24T06:10:26.716804+0000","last_peered":"2023-11-24T06:10:26.716804+0000","last_clean":"2023-11-24T06:10:26.716804+0000","last_became_active":"2023-11-24T06:10:24.693832+0000","last_became_peered":"2023-11-24T06:10:24.693832+0000","last_unstale":"2023-11-24T06:10:26.716804+0000","last_undegraded":"2023-11-24T06:10:26.716804+0000","last_fullsized":"2023-11-24T06:10:26.716804+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_clean_scrub_stamp":"2023-11-24T06:10:23.635255+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:38:10.950543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.716865+0000","last_change":"2023-11-24T06:10:25.707741+0000","last_active":"2023-11-24T06:10:26.716865+0000","last_peered":"2023-11-24T06:10:26.716865+0000","last_clean":"2023-11-24T06:10:26.716865+0000","last_became_active":"2023-11-24T06:10:25.707453+0000","last_became_peered":"2023-11-24T06:10:25.707453+0000","last_unstale":"2023-11-24T06:10:26.716865+0000","last_undegraded":"2023-11-24T06:10:26.716865+0000","last_fullsized":"2023-11-24T06:10:26.716865+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:26:40.207866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.887375+0000","last_change":"2023-11-24T06:10:24.699025+0000","last_active":"2023-11-24T06:10:26.887375+0000","last_peered":"2023-11-24T06:10:26.887375+0000","last_clean":"2023-11-24T06:10:26.887375+0000","last_became_active":"2023-11-24T06:10:24.698788+0000","last_became_peered":"2023-11-24T06:10:24.698788+0000","last_unstale":"2023-11-24T06:10:26.887375+0000","last_undegraded":"2023-11-24T06:10:26.887375+0000","last_fullsized":"2023-11-24T06:10:26.887375+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_clean_scrub_stamp":"2023-11-24T06:10:23.635255+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:18:53.146509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574851,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31964,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339876,"statfs":{"total":96636764160,"available":96604033024,"internally_reserved":0,"allocated":802816,"data_stored":661860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89600000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78100000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2350000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0469999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2809999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4330000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6890000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574852,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31400,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340440,"statfs":{"total":96636764160,"available":96604610560,"internally_reserved":0,"allocated":217088,"data_stored":72522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.389}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.008}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2050000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.45}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.532}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.607}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.72}]}]},{"osd":5,"up_from":13,"seq":55834574852,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31388,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340452,"statfs":{"total":96636764160,"available":96604622848,"internally_reserved":0,"allocated":204800,"data_stored":70386,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.292}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.407}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4910000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.008}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1669999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.3109999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0790000000000002}]}]},{"osd":4,"up_from":13,"seq":55834574852,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31396,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340444,"statfs":{"total":96636764160,"available":96604614656,"internally_reserved":0,"allocated":217088,"data_stored":71036,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.177}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0129999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4039999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.708}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.556}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8340000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.895}]}]},{"osd":3,"up_from":13,"seq":55834574852,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":217088,"data_stored":72522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3100000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.107}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.516}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2030000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9910000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.821}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8640000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574852,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":196,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":200704,"data_stored":70296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91800000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80800000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1200000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8049999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2490000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2270000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6379999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574851,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":212992,"data_stored":71070,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0649999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2729999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1719999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6870000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.726}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0009999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574851,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27812,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344028,"statfs":{"total":96636764160,"available":96608284672,"internally_reserved":0,"allocated":811008,"data_stored":661916,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68700000000000006}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81499999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.276}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3049999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3660000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4510000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-24T06:10:30.120 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-24T06:10:30.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:30.370+0000 7fe61252c640 1 Processor -- start 2023-11-24T06:10:30.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:30.370+0000 7fe61252c640 1 -- start start 2023-11-24T06:10:30.375 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:30.371+0000 7fe61252c640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fe60c15da10 0x7fe60c072230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:30.375 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:30.371+0000 7fe61252c640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe60c0b3730 con 0x7fe60c0b2bd0 2023-11-24T06:10:30.375 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:30.371+0000 7fe61252c640 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe60c071190 con 0x7fe60c150760 2023-11-24T06:10:30.375 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:30.371+0000 7fe61252c640 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe60c071470 con 0x7fe60c15da10 2023-11-24T06:10:30.375 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:30.372+0000 7fe610ac9640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fe60c0b2bd0 0x7fe60c0b3020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:55762/0 (socket says 172.21.15.92:55762) 2023-11-24T06:10:30.375 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:30.372+0000 7fe610ac9640 1 -- 172.21.15.92:0/3551551682 learned_addr learned my addr 172.21.15.92:0/3551551682 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-24T06:10:30.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:30.372+0000 7fe60b7fe640 1 -- 172.21.15.92:0/3551551682 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2270528938 0 0) 0x7fe60c0b3730 con 0x7fe60c0b2bd0 2023-11-24T06:10:30.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:30.372+0000 7fe603fff640 1 --2- 172.21.15.92:0/3551551682 >> v2:172.21.15.118:3300/0 conn(0x7fe60c15da10 0x7fe60c072230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:30.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:30.372+0000 7fe60b7fe640 1 -- 172.21.15.92:0/3551551682 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe5f8003440 con 0x7fe60c0b2bd0 2023-11-24T06:10:30.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:30.372+0000 7fe60b7fe640 1 -- 172.21.15.92:0/3551551682 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 624586687 0 0) 0x7fe5f8003440 con 0x7fe60c0b2bd0 2023-11-24T06:10:30.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:30.372+0000 7fe60b7fe640 1 -- 172.21.15.92:0/3551551682 >> v2:172.21.15.118:3300/0 conn(0x7fe60c15da10 msgr2=0x7fe60c072230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:30.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:30.372+0000 7fe60b7fe640 1 --2- 172.21.15.92:0/3551551682 >> v2:172.21.15.118:3300/0 conn(0x7fe60c15da10 0x7fe60c072230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:30.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:30.372+0000 7fe60b7fe640 1 -- 172.21.15.92:0/3551551682 >> v1:172.21.15.118:6789/0 conn(0x7fe60c150760 legacy=0x7fe60c15d300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:30.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:30.372+0000 7fe60b7fe640 1 -- 172.21.15.92:0/3551551682 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe60c071750 con 0x7fe60c0b2bd0 2023-11-24T06:10:30.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:30.372+0000 7fe60b7fe640 1 -- 172.21.15.92:0/3551551682 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2607661350 0 0) 0x7fe6040019f0 con 0x7fe60c0b2bd0 2023-11-24T06:10:30.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:30.372+0000 7fe60b7fe640 1 -- 172.21.15.92:0/3551551682 >> v1:172.21.15.92:6789/0 conn(0x7fe60c0b2bd0 legacy=0x7fe60c0b3020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:30.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:30.372+0000 7fe60b7fe640 1 --2- 172.21.15.92:0/3551551682 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe5f8003a60 0x7fe5f8003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:30.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:30.372+0000 7fe60b7fe640 1 --2- 172.21.15.92:0/3551551682 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe5f8008850 0x7fe5f800ac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:30.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:30.372+0000 7fe60b7fe640 1 --2- 172.21.15.92:0/3551551682 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe5f800b480 0x7fe5f800d940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:30.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:30.373+0000 7fe60b7fe640 1 -- 172.21.15.92:0/3551551682 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe60c071470 con 0x7fe5f8003a60 2023-11-24T06:10:30.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:30.373+0000 7fe60b7fe640 1 -- 172.21.15.92:0/3551551682 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5f800e220 con 0x7fe5f8008850 2023-11-24T06:10:30.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:30.373+0000 7fe60b7fe640 1 -- 172.21.15.92:0/3551551682 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5f800e3e0 con 0x7fe5f800b480 2023-11-24T06:10:30.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:30.373+0000 7fe60bfff640 1 --2- 172.21.15.92:0/3551551682 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe5f8003a60 0x7fe5f8003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:30.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:30.373+0000 7fe60bfff640 1 -- 172.21.15.92:0/3551551682 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe5f800b480 msgr2=0x7fe5f800d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:30.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:30.373+0000 7fe60bfff640 1 --2- 172.21.15.92:0/3551551682 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe5f800b480 0x7fe5f800d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:30.379 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:30.373+0000 7fe60bfff640 1 -- 172.21.15.92:0/3551551682 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe5f8008850 msgr2=0x7fe5f800ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:30.379 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:30.373+0000 7fe60bfff640 1 --2- 172.21.15.92:0/3551551682 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe5f8008850 0x7fe5f800ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:30.379 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:30.373+0000 7fe60bfff640 1 -- 172.21.15.92:0/3551551682 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5f8003440 con 0x7fe5f8003a60 2023-11-24T06:10:30.379 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:30.373+0000 7fe610ac9640 1 -- 172.21.15.92:0/3551551682 reap_dead start 2023-11-24T06:10:30.380 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:30.374+0000 7fe60bfff640 1 --2- 172.21.15.92:0/3551551682 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe5f8003a60 0x7fe5f8003e80 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fe5fc001750 tx=0x7fe5f80036f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d06a8e5f45ab6c61 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:30.380 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:30.374+0000 7fe60b7fe640 1 -- 172.21.15.92:0/3551551682 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5fc0052d0 con 0x7fe5f8003a60 2023-11-24T06:10:30.380 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:30.374+0000 7fe60b7fe640 1 -- 172.21.15.92:0/3551551682 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5fc02e050 con 0x7fe5f8003a60 2023-11-24T06:10:30.380 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:30.374+0000 7fe60b7fe640 1 -- 172.21.15.92:0/3551551682 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5fc003830 con 0x7fe5f8003a60 2023-11-24T06:10:30.380 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:30.375+0000 7fe61252c640 1 -- 172.21.15.92:0/3551551682 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe5f8003a60 msgr2=0x7fe5f8003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:30.380 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:30.375+0000 7fe61252c640 1 --2- 172.21.15.92:0/3551551682 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe5f8003a60 0x7fe5f8003e80 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fe5fc001750 tx=0x7fe5f80036f0 comp rx=0 tx=0).stop 2023-11-24T06:10:30.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:30.375+0000 7fe61252c640 1 -- 172.21.15.92:0/3551551682 shutdown_connections 2023-11-24T06:10:30.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:30.375+0000 7fe61252c640 1 --2- 172.21.15.92:0/3551551682 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe5f8003a60 0x7fe5f8003e80 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:30.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:30.375+0000 7fe61252c640 1 -- 172.21.15.92:0/3551551682 >> 172.21.15.92:0/3551551682 conn(0x7fe60c06a800 msgr2=0x7fe60c06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:30.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:30.375+0000 7fe61252c640 1 -- 172.21.15.92:0/3551551682 shutdown_connections 2023-11-24T06:10:30.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:30.375+0000 7fe61252c640 1 -- 172.21.15.92:0/3551551682 wait complete. 2023-11-24T06:10:30.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:30.377+0000 7fe61252c640 1 Processor -- start 2023-11-24T06:10:30.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:30.377+0000 7fe61252c640 1 -- start start 2023-11-24T06:10:30.383 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:30.378+0000 7fe61252c640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe5f8003a60 0x7fe60c14cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:30.383 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:30.378+0000 7fe61252c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe60c0ad2e0 0x7fe60c0ad720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:30.383 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:30.378+0000 7fe61252c640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe60c147460 0x7fe60c1478a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:30.383 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:30.378+0000 7fe61252c640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe60c1e3120 con 0x7fe60c0ad2e0 2023-11-24T06:10:30.383 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:30.378+0000 7fe61252c640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe60c0add80 con 0x7fe60c147460 2023-11-24T06:10:30.383 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:30.378+0000 7fe61252c640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe60c0ae060 con 0x7fe5f8003a60 2023-11-24T06:10:30.384 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:30.378+0000 7fe603fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe60c0ad2e0 0x7fe60c0ad720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:30.384 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:30.378+0000 7fe603fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe60c0ad2e0 0x7fe60c0ad720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47192/0 (socket says 172.21.15.92:47192) 2023-11-24T06:10:30.384 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:30.378+0000 7fe603fff640 1 -- 172.21.15.92:0/4085927639 learned_addr learned my addr 172.21.15.92:0/4085927639 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:10:30.384 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:30.378+0000 7fe60bfff640 1 --2- 172.21.15.92:0/4085927639 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe5f8003a60 0x7fe60c14cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:30.384 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:30.378+0000 7fe610ac9640 1 --2- 172.21.15.92:0/4085927639 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe60c147460 0x7fe60c1478a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:30.384 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:30.378+0000 7fe603fff640 1 -- 172.21.15.92:0/4085927639 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe5f8003a60 msgr2=0x7fe60c14cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:30.384 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:30.378+0000 7fe603fff640 1 --2- 172.21.15.92:0/4085927639 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe5f8003a60 0x7fe60c14cea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:30.384 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:30.378+0000 7fe603fff640 1 -- 172.21.15.92:0/4085927639 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe60c147460 msgr2=0x7fe60c1478a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:30.384 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:30.378+0000 7fe603fff640 1 --2- 172.21.15.92:0/4085927639 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe60c147460 0x7fe60c1478a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:30.384 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:30.378+0000 7fe603fff640 1 -- 172.21.15.92:0/4085927639 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe60c1e3330 con 0x7fe60c0ad2e0 2023-11-24T06:10:30.384 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:30.378+0000 7fe610ac9640 1 --2- 172.21.15.92:0/4085927639 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe60c147460 0x7fe60c1478a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:10:30.384 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:30.379+0000 7fe603fff640 1 --2- 172.21.15.92:0/4085927639 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe60c0ad2e0 0x7fe60c0ad720 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fe5f400e860 tx=0x7fe5f400ed20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:30.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:30.379+0000 7fe6097fa640 1 -- 172.21.15.92:0/4085927639 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5f4009600 con 0x7fe60c0ad2e0 2023-11-24T06:10:30.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:30.379+0000 7fe6097fa640 1 -- 172.21.15.92:0/4085927639 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5f400c940 con 0x7fe60c0ad2e0 2023-11-24T06:10:30.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:30.379+0000 7fe6097fa640 1 -- 172.21.15.92:0/4085927639 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5f400ccd0 con 0x7fe60c0ad2e0 2023-11-24T06:10:30.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:30.379+0000 7fe61252c640 1 -- 172.21.15.92:0/4085927639 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe60c1e3610 con 0x7fe60c0ad2e0 2023-11-24T06:10:30.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:30.379+0000 7fe61252c640 1 -- 172.21.15.92:0/4085927639 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe60c14c4a0 con 0x7fe60c0ad2e0 2023-11-24T06:10:30.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:30.380+0000 7fe61252c640 1 -- 172.21.15.92:0/4085927639 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe60c0ae280 con 0x7fe60c0ad2e0 2023-11-24T06:10:30.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:30.381+0000 7fe6097fa640 1 -- 172.21.15.92:0/4085927639 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe5f4013470 con 0x7fe60c0ad2e0 2023-11-24T06:10:30.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:30.382+0000 7fe6097fa640 1 --2- 172.21.15.92:0/4085927639 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fe5ec06ae40 0x7fe5ec06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:30.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:30.382+0000 7fe6097fa640 1 -- 172.21.15.92:0/4085927639 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe5f40898b0 con 0x7fe60c0ad2e0 2023-11-24T06:10:30.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:30.382+0000 7fe60bfff640 1 --2- 172.21.15.92:0/4085927639 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fe5ec06ae40 0x7fe5ec06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:30.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:30.383+0000 7fe60bfff640 1 --2- 172.21.15.92:0/4085927639 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fe5ec06ae40 0x7fe5ec06d300 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fe5fc033000 tx=0x7fe5fc009030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:30.389 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:30.386+0000 7fe6097fa640 1 -- 172.21.15.92:0/4085927639 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe5f4054830 con 0x7fe60c0ad2e0 2023-11-24T06:10:30.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:30.570+0000 7fe61252c640 1 -- 172.21.15.92:0/4085927639 --> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe60c150b40 con 0x7fe5ec06ae40 2023-11-24T06:10:30.592 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:30.589+0000 7fe6097fa640 1 -- 172.21.15.92:0/4085927639 <== mgr.4101 v2:172.21.15.92:6800/784027401 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264474 (secure 0 0 0) 0x7fe5fc005410 con 0x7fe5ec06ae40 2023-11-24T06:10:30.596 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-24T06:10:30.597 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2023-11-24T06:10:30.599 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:30.594+0000 7fe61252c640 1 -- 172.21.15.92:0/4085927639 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fe5ec06ae40 msgr2=0x7fe5ec06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:30.599 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:30.594+0000 7fe61252c640 1 --2- 172.21.15.92:0/4085927639 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fe5ec06ae40 0x7fe5ec06d300 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fe5fc033000 tx=0x7fe5fc009030 comp rx=0 tx=0).stop 2023-11-24T06:10:30.600 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:30.594+0000 7fe61252c640 1 -- 172.21.15.92:0/4085927639 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe60c0ad2e0 msgr2=0x7fe60c0ad720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:30.600 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:30.594+0000 7fe61252c640 1 --2- 172.21.15.92:0/4085927639 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe60c0ad2e0 0x7fe60c0ad720 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fe5f400e860 tx=0x7fe5f400ed20 comp rx=0 tx=0).stop 2023-11-24T06:10:30.600 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:30.594+0000 7fe61252c640 1 -- 172.21.15.92:0/4085927639 shutdown_connections 2023-11-24T06:10:30.600 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:30.594+0000 7fe61252c640 1 --2- 172.21.15.92:0/4085927639 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fe5ec06ae40 0x7fe5ec06d300 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:30.600 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:30.594+0000 7fe61252c640 1 --2- 172.21.15.92:0/4085927639 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe60c147460 0x7fe60c1478a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:30.600 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:30.594+0000 7fe61252c640 1 --2- 172.21.15.92:0/4085927639 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe60c0ad2e0 0x7fe60c0ad720 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:30.600 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:30.594+0000 7fe61252c640 1 --2- 172.21.15.92:0/4085927639 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe5f8003a60 0x7fe60c14cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:30.600 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:30.594+0000 7fe61252c640 1 -- 172.21.15.92:0/4085927639 >> 172.21.15.92:0/4085927639 conn(0x7fe60c06a800 msgr2=0x7fe60c14f9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:30.601 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:30.594+0000 7fe61252c640 1 -- 172.21.15.92:0/4085927639 shutdown_connections 2023-11-24T06:10:30.601 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:30.594+0000 7fe61252c640 1 -- 172.21.15.92:0/4085927639 wait complete. 2023-11-24T06:10:30.615 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":19,"stamp":"2023-11-24T06:10:29.667971+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":156,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235656,"kb_used_data":2816,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754739064,"statfs":{"total":773094113280,"available":772852801536,"internally_reserved":0,"allocated":2883584,"data_stored":1751608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":10,"num_read_kb":9,"num_write":58,"num_write_kb":276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001621"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.887289+0000","last_change":"2023-11-24T06:10:25.718900+0000","last_active":"2023-11-24T06:10:26.887289+0000","last_peered":"2023-11-24T06:10:26.887289+0000","last_clean":"2023-11-24T06:10:26.887289+0000","last_became_active":"2023-11-24T06:10:25.718569+0000","last_became_peered":"2023-11-24T06:10:25.718569+0000","last_unstale":"2023-11-24T06:10:26.887289+0000","last_undegraded":"2023-11-24T06:10:26.887289+0000","last_fullsized":"2023-11-24T06:10:26.887289+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:56:03.801033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.885991+0000","last_change":"2023-11-24T06:10:25.711437+0000","last_active":"2023-11-24T06:10:26.885991+0000","last_peered":"2023-11-24T06:10:26.885991+0000","last_clean":"2023-11-24T06:10:26.885991+0000","last_became_active":"2023-11-24T06:10:25.710555+0000","last_became_peered":"2023-11-24T06:10:25.710555+0000","last_unstale":"2023-11-24T06:10:26.885991+0000","last_undegraded":"2023-11-24T06:10:26.885991+0000","last_fullsized":"2023-11-24T06:10:26.885991+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:37:01.293292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.886876+0000","last_change":"2023-11-24T06:10:25.716423+0000","last_active":"2023-11-24T06:10:26.886876+0000","last_peered":"2023-11-24T06:10:26.886876+0000","last_clean":"2023-11-24T06:10:26.886876+0000","last_became_active":"2023-11-24T06:10:25.716112+0000","last_became_peered":"2023-11-24T06:10:25.716112+0000","last_unstale":"2023-11-24T06:10:26.886876+0000","last_undegraded":"2023-11-24T06:10:26.886876+0000","last_fullsized":"2023-11-24T06:10:26.886876+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:43:50.924468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.888341+0000","last_change":"2023-11-24T06:10:25.733338+0000","last_active":"2023-11-24T06:10:26.888341+0000","last_peered":"2023-11-24T06:10:26.888341+0000","last_clean":"2023-11-24T06:10:26.888341+0000","last_became_active":"2023-11-24T06:10:25.733087+0000","last_became_peered":"2023-11-24T06:10:25.733087+0000","last_unstale":"2023-11-24T06:10:26.888341+0000","last_undegraded":"2023-11-24T06:10:26.888341+0000","last_fullsized":"2023-11-24T06:10:26.888341+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:00:56.115984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.887727+0000","last_change":"2023-11-24T06:10:25.721479+0000","last_active":"2023-11-24T06:10:26.887727+0000","last_peered":"2023-11-24T06:10:26.887727+0000","last_clean":"2023-11-24T06:10:26.887727+0000","last_became_active":"2023-11-24T06:10:25.721135+0000","last_became_peered":"2023-11-24T06:10:25.721135+0000","last_unstale":"2023-11-24T06:10:26.887727+0000","last_undegraded":"2023-11-24T06:10:26.887727+0000","last_fullsized":"2023-11-24T06:10:26.887727+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:14:05.090678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.886536+0000","last_change":"2023-11-24T06:10:25.731299+0000","last_active":"2023-11-24T06:10:26.886536+0000","last_peered":"2023-11-24T06:10:26.886536+0000","last_clean":"2023-11-24T06:10:26.886536+0000","last_became_active":"2023-11-24T06:10:25.730627+0000","last_became_peered":"2023-11-24T06:10:25.730627+0000","last_unstale":"2023-11-24T06:10:26.886536+0000","last_undegraded":"2023-11-24T06:10:26.886536+0000","last_fullsized":"2023-11-24T06:10:26.886536+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:53:47.062623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.885972+0000","last_change":"2023-11-24T06:10:25.710797+0000","last_active":"2023-11-24T06:10:26.885972+0000","last_peered":"2023-11-24T06:10:26.885972+0000","last_clean":"2023-11-24T06:10:26.885972+0000","last_became_active":"2023-11-24T06:10:25.710509+0000","last_became_peered":"2023-11-24T06:10:25.710509+0000","last_unstale":"2023-11-24T06:10:26.885972+0000","last_undegraded":"2023-11-24T06:10:26.885972+0000","last_fullsized":"2023-11-24T06:10:26.885972+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:44:12.726705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.714692+0000","last_change":"2023-11-24T06:10:25.721521+0000","last_active":"2023-11-24T06:10:26.714692+0000","last_peered":"2023-11-24T06:10:26.714692+0000","last_clean":"2023-11-24T06:10:26.714692+0000","last_became_active":"2023-11-24T06:10:25.721206+0000","last_became_peered":"2023-11-24T06:10:25.721206+0000","last_unstale":"2023-11-24T06:10:26.714692+0000","last_undegraded":"2023-11-24T06:10:26.714692+0000","last_fullsized":"2023-11-24T06:10:26.714692+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:52:32.288074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.887339+0000","last_change":"2023-11-24T06:10:25.743664+0000","last_active":"2023-11-24T06:10:26.887339+0000","last_peered":"2023-11-24T06:10:26.887339+0000","last_clean":"2023-11-24T06:10:26.887339+0000","last_became_active":"2023-11-24T06:10:25.743346+0000","last_became_peered":"2023-11-24T06:10:25.743346+0000","last_unstale":"2023-11-24T06:10:26.887339+0000","last_undegraded":"2023-11-24T06:10:26.887339+0000","last_fullsized":"2023-11-24T06:10:26.887339+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:59:37.010936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.716729+0000","last_change":"2023-11-24T06:10:25.722633+0000","last_active":"2023-11-24T06:10:26.716729+0000","last_peered":"2023-11-24T06:10:26.716729+0000","last_clean":"2023-11-24T06:10:26.716729+0000","last_became_active":"2023-11-24T06:10:25.722320+0000","last_became_peered":"2023-11-24T06:10:25.722320+0000","last_unstale":"2023-11-24T06:10:26.716729+0000","last_undegraded":"2023-11-24T06:10:26.716729+0000","last_fullsized":"2023-11-24T06:10:26.716729+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:37:01.293292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.716342+0000","last_change":"2023-11-24T06:10:25.721320+0000","last_active":"2023-11-24T06:10:26.716342+0000","last_peered":"2023-11-24T06:10:26.716342+0000","last_clean":"2023-11-24T06:10:26.716342+0000","last_became_active":"2023-11-24T06:10:25.720925+0000","last_became_peered":"2023-11-24T06:10:25.720925+0000","last_unstale":"2023-11-24T06:10:26.716342+0000","last_undegraded":"2023-11-24T06:10:26.716342+0000","last_fullsized":"2023-11-24T06:10:26.716342+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:40:01.407324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.716059+0000","last_change":"2023-11-24T06:10:25.700499+0000","last_active":"2023-11-24T06:10:26.716059+0000","last_peered":"2023-11-24T06:10:26.716059+0000","last_clean":"2023-11-24T06:10:26.716059+0000","last_became_active":"2023-11-24T06:10:25.700070+0000","last_became_peered":"2023-11-24T06:10:25.700070+0000","last_unstale":"2023-11-24T06:10:26.716059+0000","last_undegraded":"2023-11-24T06:10:26.716059+0000","last_fullsized":"2023-11-24T06:10:26.716059+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:22:20.935204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.716045+0000","last_change":"2023-11-24T06:10:25.714833+0000","last_active":"2023-11-24T06:10:26.716045+0000","last_peered":"2023-11-24T06:10:26.716045+0000","last_clean":"2023-11-24T06:10:26.716045+0000","last_became_active":"2023-11-24T06:10:25.714265+0000","last_became_peered":"2023-11-24T06:10:25.714265+0000","last_unstale":"2023-11-24T06:10:26.716045+0000","last_undegraded":"2023-11-24T06:10:26.716045+0000","last_fullsized":"2023-11-24T06:10:26.716045+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:40:01.407324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.885149+0000","last_change":"2023-11-24T06:10:25.710049+0000","last_active":"2023-11-24T06:10:26.885149+0000","last_peered":"2023-11-24T06:10:26.885149+0000","last_clean":"2023-11-24T06:10:26.885149+0000","last_became_active":"2023-11-24T06:10:25.709681+0000","last_became_peered":"2023-11-24T06:10:25.709681+0000","last_unstale":"2023-11-24T06:10:26.885149+0000","last_undegraded":"2023-11-24T06:10:26.885149+0000","last_fullsized":"2023-11-24T06:10:26.885149+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:52:32.288074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.885255+0000","last_change":"2023-11-24T06:10:25.713948+0000","last_active":"2023-11-24T06:10:26.885255+0000","last_peered":"2023-11-24T06:10:26.885255+0000","last_clean":"2023-11-24T06:10:26.885255+0000","last_became_active":"2023-11-24T06:10:25.713633+0000","last_became_peered":"2023-11-24T06:10:25.713633+0000","last_unstale":"2023-11-24T06:10:26.885255+0000","last_undegraded":"2023-11-24T06:10:26.885255+0000","last_fullsized":"2023-11-24T06:10:26.885255+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:18:29.626191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.885915+0000","last_change":"2023-11-24T06:10:25.733647+0000","last_active":"2023-11-24T06:10:26.885915+0000","last_peered":"2023-11-24T06:10:26.885915+0000","last_clean":"2023-11-24T06:10:26.885915+0000","last_became_active":"2023-11-24T06:10:25.733342+0000","last_became_peered":"2023-11-24T06:10:25.733342+0000","last_unstale":"2023-11-24T06:10:26.885915+0000","last_undegraded":"2023-11-24T06:10:26.885915+0000","last_fullsized":"2023-11-24T06:10:26.885915+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:43:50.924468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.889359+0000","last_change":"2023-11-24T06:10:25.737712+0000","last_active":"2023-11-24T06:10:26.889359+0000","last_peered":"2023-11-24T06:10:26.889359+0000","last_clean":"2023-11-24T06:10:26.889359+0000","last_became_active":"2023-11-24T06:10:25.735939+0000","last_became_peered":"2023-11-24T06:10:25.735939+0000","last_unstale":"2023-11-24T06:10:26.889359+0000","last_undegraded":"2023-11-24T06:10:26.889359+0000","last_fullsized":"2023-11-24T06:10:26.889359+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:05:14.621899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.826109+0000","last_change":"2023-11-24T06:10:25.717816+0000","last_active":"2023-11-24T06:10:26.826109+0000","last_peered":"2023-11-24T06:10:26.826109+0000","last_clean":"2023-11-24T06:10:26.826109+0000","last_became_active":"2023-11-24T06:10:25.717132+0000","last_became_peered":"2023-11-24T06:10:25.717132+0000","last_unstale":"2023-11-24T06:10:26.826109+0000","last_undegraded":"2023-11-24T06:10:26.826109+0000","last_fullsized":"2023-11-24T06:10:26.826109+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:17:03.098818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.888164+0000","last_change":"2023-11-24T06:10:25.724997+0000","last_active":"2023-11-24T06:10:26.888164+0000","last_peered":"2023-11-24T06:10:26.888164+0000","last_clean":"2023-11-24T06:10:26.888164+0000","last_became_active":"2023-11-24T06:10:25.716385+0000","last_became_peered":"2023-11-24T06:10:25.716385+0000","last_unstale":"2023-11-24T06:10:26.888164+0000","last_undegraded":"2023-11-24T06:10:26.888164+0000","last_fullsized":"2023-11-24T06:10:26.888164+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:01:41.923534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.716353+0000","last_change":"2023-11-24T06:10:25.717325+0000","last_active":"2023-11-24T06:10:26.716353+0000","last_peered":"2023-11-24T06:10:26.716353+0000","last_clean":"2023-11-24T06:10:26.716353+0000","last_became_active":"2023-11-24T06:10:25.716998+0000","last_became_peered":"2023-11-24T06:10:25.716998+0000","last_unstale":"2023-11-24T06:10:26.716353+0000","last_undegraded":"2023-11-24T06:10:26.716353+0000","last_fullsized":"2023-11-24T06:10:26.716353+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:43:50.924468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.886459+0000","last_change":"2023-11-24T06:10:25.729153+0000","last_active":"2023-11-24T06:10:26.886459+0000","last_peered":"2023-11-24T06:10:26.886459+0000","last_clean":"2023-11-24T06:10:26.886459+0000","last_became_active":"2023-11-24T06:10:25.724982+0000","last_became_peered":"2023-11-24T06:10:25.724982+0000","last_unstale":"2023-11-24T06:10:26.886459+0000","last_undegraded":"2023-11-24T06:10:26.886459+0000","last_fullsized":"2023-11-24T06:10:26.886459+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:46:00.341282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.886867+0000","last_change":"2023-11-24T06:10:25.734081+0000","last_active":"2023-11-24T06:10:26.886867+0000","last_peered":"2023-11-24T06:10:26.886867+0000","last_clean":"2023-11-24T06:10:26.886867+0000","last_became_active":"2023-11-24T06:10:25.733760+0000","last_became_peered":"2023-11-24T06:10:25.733760+0000","last_unstale":"2023-11-24T06:10:26.886867+0000","last_undegraded":"2023-11-24T06:10:26.886867+0000","last_fullsized":"2023-11-24T06:10:26.886867+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:26:40.207866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.715619+0000","last_change":"2023-11-24T06:10:25.720936+0000","last_active":"2023-11-24T06:10:26.715619+0000","last_peered":"2023-11-24T06:10:26.715619+0000","last_clean":"2023-11-24T06:10:26.715619+0000","last_became_active":"2023-11-24T06:10:25.720682+0000","last_became_peered":"2023-11-24T06:10:25.720682+0000","last_unstale":"2023-11-24T06:10:26.715619+0000","last_undegraded":"2023-11-24T06:10:26.715619+0000","last_fullsized":"2023-11-24T06:10:26.715619+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:18:29.626191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.886110+0000","last_change":"2023-11-24T06:10:25.730235+0000","last_active":"2023-11-24T06:10:26.886110+0000","last_peered":"2023-11-24T06:10:26.886110+0000","last_clean":"2023-11-24T06:10:26.886110+0000","last_became_active":"2023-11-24T06:10:25.729972+0000","last_became_peered":"2023-11-24T06:10:25.729972+0000","last_unstale":"2023-11-24T06:10:26.886110+0000","last_undegraded":"2023-11-24T06:10:26.886110+0000","last_fullsized":"2023-11-24T06:10:26.886110+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:37:01.293292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.716777+0000","last_change":"2023-11-24T06:10:25.719429+0000","last_active":"2023-11-24T06:10:26.716777+0000","last_peered":"2023-11-24T06:10:26.716777+0000","last_clean":"2023-11-24T06:10:26.716777+0000","last_became_active":"2023-11-24T06:10:25.719099+0000","last_became_peered":"2023-11-24T06:10:25.719099+0000","last_unstale":"2023-11-24T06:10:26.716777+0000","last_undegraded":"2023-11-24T06:10:26.716777+0000","last_fullsized":"2023-11-24T06:10:26.716777+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:53:47.062623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.716868+0000","last_change":"2023-11-24T06:10:25.710010+0000","last_active":"2023-11-24T06:10:26.716868+0000","last_peered":"2023-11-24T06:10:26.716868+0000","last_clean":"2023-11-24T06:10:26.716868+0000","last_became_active":"2023-11-24T06:10:25.709727+0000","last_became_peered":"2023-11-24T06:10:25.709727+0000","last_unstale":"2023-11-24T06:10:26.716868+0000","last_undegraded":"2023-11-24T06:10:26.716868+0000","last_fullsized":"2023-11-24T06:10:26.716868+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:53:47.062623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.885531+0000","last_change":"2023-11-24T06:10:25.713669+0000","last_active":"2023-11-24T06:10:26.885531+0000","last_peered":"2023-11-24T06:10:26.885531+0000","last_clean":"2023-11-24T06:10:26.885531+0000","last_became_active":"2023-11-24T06:10:25.713339+0000","last_became_peered":"2023-11-24T06:10:25.713339+0000","last_unstale":"2023-11-24T06:10:26.885531+0000","last_undegraded":"2023-11-24T06:10:26.885531+0000","last_fullsized":"2023-11-24T06:10:26.885531+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:40:01.407324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.716342+0000","last_change":"2023-11-24T06:10:25.721615+0000","last_active":"2023-11-24T06:10:26.716342+0000","last_peered":"2023-11-24T06:10:26.716342+0000","last_clean":"2023-11-24T06:10:26.716342+0000","last_became_active":"2023-11-24T06:10:25.721241+0000","last_became_peered":"2023-11-24T06:10:25.721241+0000","last_unstale":"2023-11-24T06:10:26.716342+0000","last_undegraded":"2023-11-24T06:10:26.716342+0000","last_fullsized":"2023-11-24T06:10:26.716342+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:37:01.293292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.886687+0000","last_change":"2023-11-24T06:10:25.713056+0000","last_active":"2023-11-24T06:10:26.886687+0000","last_peered":"2023-11-24T06:10:26.886687+0000","last_clean":"2023-11-24T06:10:26.886687+0000","last_became_active":"2023-11-24T06:10:25.712467+0000","last_became_peered":"2023-11-24T06:10:25.712467+0000","last_unstale":"2023-11-24T06:10:26.886687+0000","last_undegraded":"2023-11-24T06:10:26.886687+0000","last_fullsized":"2023-11-24T06:10:26.886687+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:18:29.626191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.887172+0000","last_change":"2023-11-24T06:10:25.733233+0000","last_active":"2023-11-24T06:10:26.887172+0000","last_peered":"2023-11-24T06:10:26.887172+0000","last_clean":"2023-11-24T06:10:26.887172+0000","last_became_active":"2023-11-24T06:10:25.732933+0000","last_became_peered":"2023-11-24T06:10:25.732933+0000","last_unstale":"2023-11-24T06:10:26.887172+0000","last_undegraded":"2023-11-24T06:10:26.887172+0000","last_fullsized":"2023-11-24T06:10:26.887172+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:44:12.726705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.826224+0000","last_change":"2023-11-24T06:10:25.722324+0000","last_active":"2023-11-24T06:10:26.826224+0000","last_peered":"2023-11-24T06:10:26.826224+0000","last_clean":"2023-11-24T06:10:26.826224+0000","last_became_active":"2023-11-24T06:10:25.722036+0000","last_became_peered":"2023-11-24T06:10:25.722036+0000","last_unstale":"2023-11-24T06:10:26.826224+0000","last_undegraded":"2023-11-24T06:10:26.826224+0000","last_fullsized":"2023-11-24T06:10:26.826224+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:09:06.300144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.715780+0000","last_change":"2023-11-24T06:10:25.703106+0000","last_active":"2023-11-24T06:10:26.715780+0000","last_peered":"2023-11-24T06:10:26.715780+0000","last_clean":"2023-11-24T06:10:26.715780+0000","last_became_active":"2023-11-24T06:10:25.701998+0000","last_became_peered":"2023-11-24T06:10:25.701998+0000","last_unstale":"2023-11-24T06:10:26.715780+0000","last_undegraded":"2023-11-24T06:10:26.715780+0000","last_fullsized":"2023-11-24T06:10:26.715780+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:46:00.341282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.715676+0000","last_change":"2023-11-24T06:10:25.708408+0000","last_active":"2023-11-24T06:10:26.715676+0000","last_peered":"2023-11-24T06:10:26.715676+0000","last_clean":"2023-11-24T06:10:26.715676+0000","last_became_active":"2023-11-24T06:10:25.708102+0000","last_became_peered":"2023-11-24T06:10:25.708102+0000","last_unstale":"2023-11-24T06:10:26.715676+0000","last_undegraded":"2023-11-24T06:10:26.715676+0000","last_fullsized":"2023-11-24T06:10:26.715676+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:38:11.967627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.716406+0000","last_change":"2023-11-24T06:10:24.696915+0000","last_active":"2023-11-24T06:10:26.716406+0000","last_peered":"2023-11-24T06:10:26.716406+0000","last_clean":"2023-11-24T06:10:26.716406+0000","last_became_active":"2023-11-24T06:10:24.696608+0000","last_became_peered":"2023-11-24T06:10:24.696608+0000","last_unstale":"2023-11-24T06:10:26.716406+0000","last_undegraded":"2023-11-24T06:10:26.716406+0000","last_fullsized":"2023-11-24T06:10:26.716406+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_clean_scrub_stamp":"2023-11-24T06:10:23.635255+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:14:19.359955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.886766+0000","last_change":"2023-11-24T06:10:25.713232+0000","last_active":"2023-11-24T06:10:26.886766+0000","last_peered":"2023-11-24T06:10:26.886766+0000","last_clean":"2023-11-24T06:10:26.886766+0000","last_became_active":"2023-11-24T06:10:25.712813+0000","last_became_peered":"2023-11-24T06:10:25.712813+0000","last_unstale":"2023-11-24T06:10:26.886766+0000","last_undegraded":"2023-11-24T06:10:26.886766+0000","last_fullsized":"2023-11-24T06:10:26.886766+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:40:01.407324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.716067+0000","last_change":"2023-11-24T06:10:24.689472+0000","last_active":"2023-11-24T06:10:26.716067+0000","last_peered":"2023-11-24T06:10:26.716067+0000","last_clean":"2023-11-24T06:10:26.716067+0000","last_became_active":"2023-11-24T06:10:24.689143+0000","last_became_peered":"2023-11-24T06:10:24.689143+0000","last_unstale":"2023-11-24T06:10:26.716067+0000","last_undegraded":"2023-11-24T06:10:26.716067+0000","last_fullsized":"2023-11-24T06:10:26.716067+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_clean_scrub_stamp":"2023-11-24T06:10:23.635255+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:18:53.146509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.715380+0000","last_change":"2023-11-24T06:10:24.694706+0000","last_active":"2023-11-24T06:10:26.715380+0000","last_peered":"2023-11-24T06:10:26.715380+0000","last_clean":"2023-11-24T06:10:26.715380+0000","last_became_active":"2023-11-24T06:10:24.693990+0000","last_became_peered":"2023-11-24T06:10:24.693990+0000","last_unstale":"2023-11-24T06:10:26.715380+0000","last_undegraded":"2023-11-24T06:10:26.715380+0000","last_fullsized":"2023-11-24T06:10:26.715380+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_clean_scrub_stamp":"2023-11-24T06:10:23.635255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:17:02.081734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.714912+0000","last_change":"2023-11-24T06:10:25.715897+0000","last_active":"2023-11-24T06:10:26.714912+0000","last_peered":"2023-11-24T06:10:26.714912+0000","last_clean":"2023-11-24T06:10:26.714912+0000","last_became_active":"2023-11-24T06:10:25.715596+0000","last_became_peered":"2023-11-24T06:10:25.715596+0000","last_unstale":"2023-11-24T06:10:26.714912+0000","last_undegraded":"2023-11-24T06:10:26.714912+0000","last_fullsized":"2023-11-24T06:10:26.714912+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:54:02.837130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.886161+0000","last_change":"2023-11-24T06:10:24.692669+0000","last_active":"2023-11-24T06:10:26.886161+0000","last_peered":"2023-11-24T06:10:26.886161+0000","last_clean":"2023-11-24T06:10:26.886161+0000","last_became_active":"2023-11-24T06:10:24.692341+0000","last_became_peered":"2023-11-24T06:10:24.692341+0000","last_unstale":"2023-11-24T06:10:26.886161+0000","last_undegraded":"2023-11-24T06:10:26.886161+0000","last_fullsized":"2023-11-24T06:10:26.886161+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_clean_scrub_stamp":"2023-11-24T06:10:23.635255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:09:05.283059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.885105+0000","last_change":"2023-11-24T06:10:25.713815+0000","last_active":"2023-11-24T06:10:26.885105+0000","last_peered":"2023-11-24T06:10:26.885105+0000","last_clean":"2023-11-24T06:10:26.885105+0000","last_became_active":"2023-11-24T06:10:25.713207+0000","last_became_peered":"2023-11-24T06:10:25.713207+0000","last_unstale":"2023-11-24T06:10:26.885105+0000","last_undegraded":"2023-11-24T06:10:26.885105+0000","last_fullsized":"2023-11-24T06:10:26.885105+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:46:00.341282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.716681+0000","last_change":"2023-11-24T06:10:24.700771+0000","last_active":"2023-11-24T06:10:26.716681+0000","last_peered":"2023-11-24T06:10:26.716681+0000","last_clean":"2023-11-24T06:10:26.716681+0000","last_became_active":"2023-11-24T06:10:24.699736+0000","last_became_peered":"2023-11-24T06:10:24.699736+0000","last_unstale":"2023-11-24T06:10:26.716681+0000","last_undegraded":"2023-11-24T06:10:26.716681+0000","last_fullsized":"2023-11-24T06:10:26.716681+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_clean_scrub_stamp":"2023-11-24T06:10:23.635255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:10:14.268052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.888828+0000","last_change":"2023-11-24T06:10:25.735798+0000","last_active":"2023-11-24T06:10:26.888828+0000","last_peered":"2023-11-24T06:10:26.888828+0000","last_clean":"2023-11-24T06:10:26.888828+0000","last_became_active":"2023-11-24T06:10:25.735161+0000","last_became_peered":"2023-11-24T06:10:25.735161+0000","last_unstale":"2023-11-24T06:10:26.888828+0000","last_undegraded":"2023-11-24T06:10:26.888828+0000","last_fullsized":"2023-11-24T06:10:26.888828+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:56:03.801033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.888836+0000","last_change":"2023-11-24T06:10:24.694962+0000","last_active":"2023-11-24T06:10:26.888836+0000","last_peered":"2023-11-24T06:10:26.888836+0000","last_clean":"2023-11-24T06:10:26.888836+0000","last_became_active":"2023-11-24T06:10:24.694740+0000","last_became_peered":"2023-11-24T06:10:24.694740+0000","last_unstale":"2023-11-24T06:10:26.888836+0000","last_undegraded":"2023-11-24T06:10:26.888836+0000","last_fullsized":"2023-11-24T06:10:26.888836+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_clean_scrub_stamp":"2023-11-24T06:10:23.635255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:45:59.324198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.887375+0000","last_change":"2023-11-24T06:10:25.732152+0000","last_active":"2023-11-24T06:10:26.887375+0000","last_peered":"2023-11-24T06:10:26.887375+0000","last_clean":"2023-11-24T06:10:26.887375+0000","last_became_active":"2023-11-24T06:10:25.731460+0000","last_became_peered":"2023-11-24T06:10:25.731460+0000","last_unstale":"2023-11-24T06:10:26.887375+0000","last_undegraded":"2023-11-24T06:10:26.887375+0000","last_fullsized":"2023-11-24T06:10:26.887375+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:53:47.062623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.885628+0000","last_change":"2023-11-24T06:10:24.692129+0000","last_active":"2023-11-24T06:10:26.885628+0000","last_peered":"2023-11-24T06:10:26.885628+0000","last_clean":"2023-11-24T06:10:26.885628+0000","last_became_active":"2023-11-24T06:10:24.691871+0000","last_became_peered":"2023-11-24T06:10:24.691871+0000","last_unstale":"2023-11-24T06:10:26.885628+0000","last_undegraded":"2023-11-24T06:10:26.885628+0000","last_fullsized":"2023-11-24T06:10:26.885628+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_clean_scrub_stamp":"2023-11-24T06:10:23.635255+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:14:19.359955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.717432+0000","last_change":"2023-11-24T06:10:25.718217+0000","last_active":"2023-11-24T06:10:26.717432+0000","last_peered":"2023-11-24T06:10:26.717432+0000","last_clean":"2023-11-24T06:10:26.717432+0000","last_became_active":"2023-11-24T06:10:25.717956+0000","last_became_peered":"2023-11-24T06:10:25.717956+0000","last_unstale":"2023-11-24T06:10:26.717432+0000","last_undegraded":"2023-11-24T06:10:26.717432+0000","last_fullsized":"2023-11-24T06:10:26.717432+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:09:06.300144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.716773+0000","last_change":"2023-11-24T06:10:24.694293+0000","last_active":"2023-11-24T06:10:26.716773+0000","last_peered":"2023-11-24T06:10:26.716773+0000","last_clean":"2023-11-24T06:10:26.716773+0000","last_became_active":"2023-11-24T06:10:24.694005+0000","last_became_peered":"2023-11-24T06:10:24.694005+0000","last_unstale":"2023-11-24T06:10:26.716773+0000","last_undegraded":"2023-11-24T06:10:26.716773+0000","last_fullsized":"2023-11-24T06:10:26.716773+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_clean_scrub_stamp":"2023-11-24T06:10:23.635255+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:54:01.820045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.887235+0000","last_change":"2023-11-24T06:10:25.716082+0000","last_active":"2023-11-24T06:10:26.887235+0000","last_peered":"2023-11-24T06:10:26.887235+0000","last_clean":"2023-11-24T06:10:26.887235+0000","last_became_active":"2023-11-24T06:10:25.713813+0000","last_became_peered":"2023-11-24T06:10:25.713813+0000","last_unstale":"2023-11-24T06:10:26.887235+0000","last_undegraded":"2023-11-24T06:10:26.887235+0000","last_fullsized":"2023-11-24T06:10:26.887235+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:42:25.852028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.886925+0000","last_change":"2023-11-24T06:10:24.687076+0000","last_active":"2023-11-24T06:10:26.886925+0000","last_peered":"2023-11-24T06:10:26.886925+0000","last_clean":"2023-11-24T06:10:26.886925+0000","last_became_active":"2023-11-24T06:10:24.686789+0000","last_became_peered":"2023-11-24T06:10:24.686789+0000","last_unstale":"2023-11-24T06:10:26.886925+0000","last_undegraded":"2023-11-24T06:10:26.886925+0000","last_fullsized":"2023-11-24T06:10:26.886925+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_clean_scrub_stamp":"2023-11-24T06:10:23.635255+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:38:10.950543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.885206+0000","last_change":"2023-11-24T06:10:25.704389+0000","last_active":"2023-11-24T06:10:26.885206+0000","last_peered":"2023-11-24T06:10:26.885206+0000","last_clean":"2023-11-24T06:10:26.885206+0000","last_became_active":"2023-11-24T06:10:25.704101+0000","last_became_peered":"2023-11-24T06:10:25.704101+0000","last_unstale":"2023-11-24T06:10:26.885206+0000","last_undegraded":"2023-11-24T06:10:26.885206+0000","last_fullsized":"2023-11-24T06:10:26.885206+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:54:02.837130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.887386+0000","last_change":"2023-11-24T06:10:24.691369+0000","last_active":"2023-11-24T06:10:26.887386+0000","last_peered":"2023-11-24T06:10:26.887386+0000","last_clean":"2023-11-24T06:10:26.887386+0000","last_became_active":"2023-11-24T06:10:24.691053+0000","last_became_peered":"2023-11-24T06:10:24.691053+0000","last_unstale":"2023-11-24T06:10:26.887386+0000","last_undegraded":"2023-11-24T06:10:26.887386+0000","last_fullsized":"2023-11-24T06:10:26.887386+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_clean_scrub_stamp":"2023-11-24T06:10:23.635255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:18:28.609106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.887207+0000","last_change":"2023-11-24T06:10:25.730194+0000","last_active":"2023-11-24T06:10:26.887207+0000","last_peered":"2023-11-24T06:10:26.887207+0000","last_clean":"2023-11-24T06:10:26.887207+0000","last_became_active":"2023-11-24T06:10:25.729811+0000","last_became_peered":"2023-11-24T06:10:25.729811+0000","last_unstale":"2023-11-24T06:10:26.887207+0000","last_undegraded":"2023-11-24T06:10:26.887207+0000","last_fullsized":"2023-11-24T06:10:26.887207+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:03:41.511367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.716013+0000","last_change":"2023-11-24T06:10:24.691658+0000","last_active":"2023-11-24T06:10:26.716013+0000","last_peered":"2023-11-24T06:10:26.716013+0000","last_clean":"2023-11-24T06:10:26.716013+0000","last_became_active":"2023-11-24T06:10:24.691365+0000","last_became_peered":"2023-11-24T06:10:24.691365+0000","last_unstale":"2023-11-24T06:10:26.716013+0000","last_undegraded":"2023-11-24T06:10:26.716013+0000","last_fullsized":"2023-11-24T06:10:26.716013+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_clean_scrub_stamp":"2023-11-24T06:10:23.635255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:17:02.081734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.715937+0000","last_change":"2023-11-24T06:10:25.703380+0000","last_active":"2023-11-24T06:10:26.715937+0000","last_peered":"2023-11-24T06:10:26.715937+0000","last_clean":"2023-11-24T06:10:26.715937+0000","last_became_active":"2023-11-24T06:10:25.702696+0000","last_became_peered":"2023-11-24T06:10:25.702696+0000","last_unstale":"2023-11-24T06:10:26.715937+0000","last_undegraded":"2023-11-24T06:10:26.715937+0000","last_fullsized":"2023-11-24T06:10:26.715937+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:18:29.626191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.825875+0000","last_change":"2023-11-24T06:10:24.683519+0000","last_active":"2023-11-24T06:10:26.825875+0000","last_peered":"2023-11-24T06:10:26.825875+0000","last_clean":"2023-11-24T06:10:26.825875+0000","last_became_active":"2023-11-24T06:10:24.682405+0000","last_became_peered":"2023-11-24T06:10:24.682405+0000","last_unstale":"2023-11-24T06:10:26.825875+0000","last_undegraded":"2023-11-24T06:10:26.825875+0000","last_fullsized":"2023-11-24T06:10:26.825875+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_clean_scrub_stamp":"2023-11-24T06:10:23.635255+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:20:39.581356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.886486+0000","last_change":"2023-11-24T06:10:25.726307+0000","last_active":"2023-11-24T06:10:26.886486+0000","last_peered":"2023-11-24T06:10:26.886486+0000","last_clean":"2023-11-24T06:10:26.886486+0000","last_became_active":"2023-11-24T06:10:25.726038+0000","last_became_peered":"2023-11-24T06:10:25.726038+0000","last_unstale":"2023-11-24T06:10:26.886486+0000","last_undegraded":"2023-11-24T06:10:26.886486+0000","last_fullsized":"2023-11-24T06:10:26.886486+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:00:56.115984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.717432+0000","last_change":"2023-11-24T06:10:24.692338+0000","last_active":"2023-11-24T06:10:26.717432+0000","last_peered":"2023-11-24T06:10:26.717432+0000","last_clean":"2023-11-24T06:10:26.717432+0000","last_became_active":"2023-11-24T06:10:24.692135+0000","last_became_peered":"2023-11-24T06:10:24.692135+0000","last_unstale":"2023-11-24T06:10:26.717432+0000","last_undegraded":"2023-11-24T06:10:26.717432+0000","last_fullsized":"2023-11-24T06:10:26.717432+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_clean_scrub_stamp":"2023-11-24T06:10:23.635255+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:57:48.799543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.887337+0000","last_change":"2023-11-24T06:10:25.729383+0000","last_active":"2023-11-24T06:10:26.887337+0000","last_peered":"2023-11-24T06:10:26.887337+0000","last_clean":"2023-11-24T06:10:26.887337+0000","last_became_active":"2023-11-24T06:10:25.729090+0000","last_became_peered":"2023-11-24T06:10:25.729090+0000","last_unstale":"2023-11-24T06:10:26.887337+0000","last_undegraded":"2023-11-24T06:10:26.887337+0000","last_fullsized":"2023-11-24T06:10:26.887337+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:40:47.125372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.826232+0000","last_change":"2023-11-24T06:10:24.689238+0000","last_active":"2023-11-24T06:10:26.826232+0000","last_peered":"2023-11-24T06:10:26.826232+0000","last_clean":"2023-11-24T06:10:26.826232+0000","last_became_active":"2023-11-24T06:10:24.686650+0000","last_became_peered":"2023-11-24T06:10:24.686650+0000","last_unstale":"2023-11-24T06:10:26.826232+0000","last_undegraded":"2023-11-24T06:10:26.826232+0000","last_fullsized":"2023-11-24T06:10:26.826232+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_clean_scrub_stamp":"2023-11-24T06:10:23.635255+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:06:50.936889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.887656+0000","last_change":"2023-11-24T06:10:25.726737+0000","last_active":"2023-11-24T06:10:26.887656+0000","last_peered":"2023-11-24T06:10:26.887656+0000","last_clean":"2023-11-24T06:10:26.887656+0000","last_became_active":"2023-11-24T06:10:25.725070+0000","last_became_peered":"2023-11-24T06:10:25.725070+0000","last_unstale":"2023-11-24T06:10:26.887656+0000","last_undegraded":"2023-11-24T06:10:26.887656+0000","last_fullsized":"2023-11-24T06:10:26.887656+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:22:18.598271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.885715+0000","last_change":"2023-11-24T06:10:24.681713+0000","last_active":"2023-11-24T06:10:26.885715+0000","last_peered":"2023-11-24T06:10:26.885715+0000","last_clean":"2023-11-24T06:10:26.885715+0000","last_became_active":"2023-11-24T06:10:24.681336+0000","last_became_peered":"2023-11-24T06:10:24.681336+0000","last_unstale":"2023-11-24T06:10:26.885715+0000","last_undegraded":"2023-11-24T06:10:26.885715+0000","last_fullsized":"2023-11-24T06:10:26.885715+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_clean_scrub_stamp":"2023-11-24T06:10:23.635255+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:10:14.268052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.888736+0000","last_change":"2023-11-24T06:10:25.733821+0000","last_active":"2023-11-24T06:10:26.888736+0000","last_peered":"2023-11-24T06:10:26.888736+0000","last_clean":"2023-11-24T06:10:26.888736+0000","last_became_active":"2023-11-24T06:10:25.733510+0000","last_became_peered":"2023-11-24T06:10:25.733510+0000","last_unstale":"2023-11-24T06:10:26.888736+0000","last_undegraded":"2023-11-24T06:10:26.888736+0000","last_fullsized":"2023-11-24T06:10:26.888736+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:42:25.852028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.826174+0000","last_change":"2023-11-24T06:10:24.690450+0000","last_active":"2023-11-24T06:10:26.826174+0000","last_peered":"2023-11-24T06:10:26.826174+0000","last_clean":"2023-11-24T06:10:26.826174+0000","last_became_active":"2023-11-24T06:10:24.690024+0000","last_became_peered":"2023-11-24T06:10:24.690024+0000","last_unstale":"2023-11-24T06:10:26.826174+0000","last_undegraded":"2023-11-24T06:10:26.826174+0000","last_fullsized":"2023-11-24T06:10:26.826174+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_clean_scrub_stamp":"2023-11-24T06:10:23.635255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:51:55.108153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.887639+0000","last_change":"2023-11-24T06:10:25.734784+0000","last_active":"2023-11-24T06:10:26.887639+0000","last_peered":"2023-11-24T06:10:26.887639+0000","last_clean":"2023-11-24T06:10:26.887639+0000","last_became_active":"2023-11-24T06:10:25.734478+0000","last_became_peered":"2023-11-24T06:10:25.734478+0000","last_unstale":"2023-11-24T06:10:26.887639+0000","last_undegraded":"2023-11-24T06:10:26.887639+0000","last_fullsized":"2023-11-24T06:10:26.887639+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:05:14.621899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":165,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:10:25.700259+0000","last_change":"2023-11-24T06:10:20.082677+0000","last_active":"2023-11-24T06:10:25.700259+0000","last_peered":"2023-11-24T06:10:25.700259+0000","last_clean":"2023-11-24T06:10:25.700259+0000","last_became_active":"2023-11-24T06:10:20.080551+0000","last_became_peered":"2023-11-24T06:10:20.080551+0000","last_unstale":"2023-11-24T06:10:25.700259+0000","last_undegraded":"2023-11-24T06:10:25.700259+0000","last_fullsized":"2023-11-24T06:10:25.700259+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:18.602209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:18.602209+0000","last_clean_scrub_stamp":"2023-11-24T06:10:18.602209+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:47:24.216235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.887865+0000","last_change":"2023-11-24T06:10:24.684957+0000","last_active":"2023-11-24T06:10:26.887865+0000","last_peered":"2023-11-24T06:10:26.887865+0000","last_clean":"2023-11-24T06:10:26.887865+0000","last_became_active":"2023-11-24T06:10:24.684507+0000","last_became_peered":"2023-11-24T06:10:24.684507+0000","last_unstale":"2023-11-24T06:10:26.887865+0000","last_undegraded":"2023-11-24T06:10:26.887865+0000","last_fullsized":"2023-11-24T06:10:26.887865+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_clean_scrub_stamp":"2023-11-24T06:10:23.635255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:52:31.270989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.715709+0000","last_change":"2023-11-24T06:10:25.735264+0000","last_active":"2023-11-24T06:10:26.715709+0000","last_peered":"2023-11-24T06:10:26.715709+0000","last_clean":"2023-11-24T06:10:26.715709+0000","last_became_active":"2023-11-24T06:10:25.734163+0000","last_became_peered":"2023-11-24T06:10:25.734163+0000","last_unstale":"2023-11-24T06:10:26.715709+0000","last_undegraded":"2023-11-24T06:10:26.715709+0000","last_fullsized":"2023-11-24T06:10:26.715709+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:45:21.592844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.885683+0000","last_change":"2023-11-24T06:10:24.699469+0000","last_active":"2023-11-24T06:10:26.885683+0000","last_peered":"2023-11-24T06:10:26.885683+0000","last_clean":"2023-11-24T06:10:26.885683+0000","last_became_active":"2023-11-24T06:10:24.699153+0000","last_became_peered":"2023-11-24T06:10:24.699153+0000","last_unstale":"2023-11-24T06:10:26.885683+0000","last_undegraded":"2023-11-24T06:10:26.885683+0000","last_fullsized":"2023-11-24T06:10:26.885683+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_clean_scrub_stamp":"2023-11-24T06:10:23.635255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:09:05.283059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.884898+0000","last_change":"2023-11-24T06:10:25.716501+0000","last_active":"2023-11-24T06:10:26.884898+0000","last_peered":"2023-11-24T06:10:26.884898+0000","last_clean":"2023-11-24T06:10:26.884898+0000","last_became_active":"2023-11-24T06:10:25.712127+0000","last_became_peered":"2023-11-24T06:10:25.712127+0000","last_unstale":"2023-11-24T06:10:26.884898+0000","last_undegraded":"2023-11-24T06:10:26.884898+0000","last_fullsized":"2023-11-24T06:10:26.884898+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:10:15.285136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.826108+0000","last_change":"2023-11-24T06:10:24.686855+0000","last_active":"2023-11-24T06:10:26.826108+0000","last_peered":"2023-11-24T06:10:26.826108+0000","last_clean":"2023-11-24T06:10:26.826108+0000","last_became_active":"2023-11-24T06:10:24.685894+0000","last_became_peered":"2023-11-24T06:10:24.685894+0000","last_unstale":"2023-11-24T06:10:26.826108+0000","last_undegraded":"2023-11-24T06:10:26.826108+0000","last_fullsized":"2023-11-24T06:10:26.826108+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_clean_scrub_stamp":"2023-11-24T06:10:23.635255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:38:10.950543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.885731+0000","last_change":"2023-11-24T06:10:25.715833+0000","last_active":"2023-11-24T06:10:26.885731+0000","last_peered":"2023-11-24T06:10:26.885731+0000","last_clean":"2023-11-24T06:10:26.885731+0000","last_became_active":"2023-11-24T06:10:25.715500+0000","last_became_peered":"2023-11-24T06:10:25.715500+0000","last_unstale":"2023-11-24T06:10:26.885731+0000","last_undegraded":"2023-11-24T06:10:26.885731+0000","last_fullsized":"2023-11-24T06:10:26.885731+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:43:50.924468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.887610+0000","last_change":"2023-11-24T06:10:24.690139+0000","last_active":"2023-11-24T06:10:26.887610+0000","last_peered":"2023-11-24T06:10:26.887610+0000","last_clean":"2023-11-24T06:10:26.887610+0000","last_became_active":"2023-11-24T06:10:24.689734+0000","last_became_peered":"2023-11-24T06:10:24.689734+0000","last_unstale":"2023-11-24T06:10:26.887610+0000","last_undegraded":"2023-11-24T06:10:26.887610+0000","last_fullsized":"2023-11-24T06:10:26.887610+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_clean_scrub_stamp":"2023-11-24T06:10:23.635255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:45:20.575759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.717184+0000","last_change":"2023-11-24T06:10:25.710787+0000","last_active":"2023-11-24T06:10:26.717184+0000","last_peered":"2023-11-24T06:10:26.717184+0000","last_clean":"2023-11-24T06:10:26.717184+0000","last_became_active":"2023-11-24T06:10:25.710522+0000","last_became_peered":"2023-11-24T06:10:25.710522+0000","last_unstale":"2023-11-24T06:10:26.717184+0000","last_undegraded":"2023-11-24T06:10:26.717184+0000","last_fullsized":"2023-11-24T06:10:26.717184+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:17:03.098818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.885579+0000","last_change":"2023-11-24T06:10:24.683405+0000","last_active":"2023-11-24T06:10:26.885579+0000","last_peered":"2023-11-24T06:10:26.885579+0000","last_clean":"2023-11-24T06:10:26.885579+0000","last_became_active":"2023-11-24T06:10:24.683144+0000","last_became_peered":"2023-11-24T06:10:24.683144+0000","last_unstale":"2023-11-24T06:10:26.885579+0000","last_undegraded":"2023-11-24T06:10:26.885579+0000","last_fullsized":"2023-11-24T06:10:26.885579+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_clean_scrub_stamp":"2023-11-24T06:10:23.635255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:18:53.146509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.888979+0000","last_change":"2023-11-24T06:10:25.734738+0000","last_active":"2023-11-24T06:10:26.888979+0000","last_peered":"2023-11-24T06:10:26.888979+0000","last_clean":"2023-11-24T06:10:26.888979+0000","last_became_active":"2023-11-24T06:10:25.733409+0000","last_became_peered":"2023-11-24T06:10:25.733409+0000","last_unstale":"2023-11-24T06:10:26.888979+0000","last_undegraded":"2023-11-24T06:10:26.888979+0000","last_fullsized":"2023-11-24T06:10:26.888979+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:59:37.010936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.888082+0000","last_change":"2023-11-24T06:10:24.701942+0000","last_active":"2023-11-24T06:10:26.888082+0000","last_peered":"2023-11-24T06:10:26.888082+0000","last_clean":"2023-11-24T06:10:26.888082+0000","last_became_active":"2023-11-24T06:10:24.701385+0000","last_became_peered":"2023-11-24T06:10:24.701385+0000","last_unstale":"2023-11-24T06:10:26.888082+0000","last_undegraded":"2023-11-24T06:10:26.888082+0000","last_fullsized":"2023-11-24T06:10:26.888082+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_clean_scrub_stamp":"2023-11-24T06:10:23.635255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:26:39.190781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.885358+0000","last_change":"2023-11-24T06:10:25.711825+0000","last_active":"2023-11-24T06:10:26.885358+0000","last_peered":"2023-11-24T06:10:26.885358+0000","last_clean":"2023-11-24T06:10:26.885358+0000","last_became_active":"2023-11-24T06:10:25.711161+0000","last_became_peered":"2023-11-24T06:10:25.711161+0000","last_unstale":"2023-11-24T06:10:26.885358+0000","last_undegraded":"2023-11-24T06:10:26.885358+0000","last_fullsized":"2023-11-24T06:10:26.885358+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:45:21.592844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.886084+0000","last_change":"2023-11-24T06:10:24.695283+0000","last_active":"2023-11-24T06:10:26.886084+0000","last_peered":"2023-11-24T06:10:26.886084+0000","last_clean":"2023-11-24T06:10:26.886084+0000","last_became_active":"2023-11-24T06:10:24.694723+0000","last_became_peered":"2023-11-24T06:10:24.694723+0000","last_unstale":"2023-11-24T06:10:26.886084+0000","last_undegraded":"2023-11-24T06:10:26.886084+0000","last_fullsized":"2023-11-24T06:10:26.886084+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_clean_scrub_stamp":"2023-11-24T06:10:23.635255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:52:31.270989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.716963+0000","last_change":"2023-11-24T06:10:25.730440+0000","last_active":"2023-11-24T06:10:26.716963+0000","last_peered":"2023-11-24T06:10:26.716963+0000","last_clean":"2023-11-24T06:10:26.716963+0000","last_became_active":"2023-11-24T06:10:25.730097+0000","last_became_peered":"2023-11-24T06:10:25.730097+0000","last_unstale":"2023-11-24T06:10:26.716963+0000","last_undegraded":"2023-11-24T06:10:26.716963+0000","last_fullsized":"2023-11-24T06:10:26.716963+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:44:12.726705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.716893+0000","last_change":"2023-11-24T06:10:24.702923+0000","last_active":"2023-11-24T06:10:26.716893+0000","last_peered":"2023-11-24T06:10:26.716893+0000","last_clean":"2023-11-24T06:10:26.716893+0000","last_became_active":"2023-11-24T06:10:24.702428+0000","last_became_peered":"2023-11-24T06:10:24.702428+0000","last_unstale":"2023-11-24T06:10:26.716893+0000","last_undegraded":"2023-11-24T06:10:26.716893+0000","last_fullsized":"2023-11-24T06:10:26.716893+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_clean_scrub_stamp":"2023-11-24T06:10:23.635255+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:52:31.270989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.886761+0000","last_change":"2023-11-24T06:10:25.709739+0000","last_active":"2023-11-24T06:10:26.886761+0000","last_peered":"2023-11-24T06:10:26.886761+0000","last_clean":"2023-11-24T06:10:26.886761+0000","last_became_active":"2023-11-24T06:10:25.709392+0000","last_became_peered":"2023-11-24T06:10:25.709392+0000","last_unstale":"2023-11-24T06:10:26.886761+0000","last_undegraded":"2023-11-24T06:10:26.886761+0000","last_fullsized":"2023-11-24T06:10:26.886761+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:45:21.592844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.886005+0000","last_change":"2023-11-24T06:10:24.698896+0000","last_active":"2023-11-24T06:10:26.886005+0000","last_peered":"2023-11-24T06:10:26.886005+0000","last_clean":"2023-11-24T06:10:26.886005+0000","last_became_active":"2023-11-24T06:10:24.698632+0000","last_became_peered":"2023-11-24T06:10:24.698632+0000","last_unstale":"2023-11-24T06:10:26.886005+0000","last_undegraded":"2023-11-24T06:10:26.886005+0000","last_fullsized":"2023-11-24T06:10:26.886005+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_clean_scrub_stamp":"2023-11-24T06:10:23.635255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:22:19.918119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.715380+0000","last_change":"2023-11-24T06:10:25.716284+0000","last_active":"2023-11-24T06:10:26.715380+0000","last_peered":"2023-11-24T06:10:26.715380+0000","last_clean":"2023-11-24T06:10:26.715380+0000","last_became_active":"2023-11-24T06:10:25.715948+0000","last_became_peered":"2023-11-24T06:10:25.715948+0000","last_unstale":"2023-11-24T06:10:26.715380+0000","last_undegraded":"2023-11-24T06:10:26.715380+0000","last_fullsized":"2023-11-24T06:10:26.715380+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:20:40.598440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.885366+0000","last_change":"2023-11-24T06:10:24.692784+0000","last_active":"2023-11-24T06:10:26.885366+0000","last_peered":"2023-11-24T06:10:26.885366+0000","last_clean":"2023-11-24T06:10:26.885366+0000","last_became_active":"2023-11-24T06:10:24.692514+0000","last_became_peered":"2023-11-24T06:10:24.692514+0000","last_unstale":"2023-11-24T06:10:26.885366+0000","last_undegraded":"2023-11-24T06:10:26.885366+0000","last_fullsized":"2023-11-24T06:10:26.885366+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_clean_scrub_stamp":"2023-11-24T06:10:23.635255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:38:10.950543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.715214+0000","last_change":"2023-11-24T06:10:25.712156+0000","last_active":"2023-11-24T06:10:26.715214+0000","last_peered":"2023-11-24T06:10:26.715214+0000","last_clean":"2023-11-24T06:10:26.715214+0000","last_became_active":"2023-11-24T06:10:25.711924+0000","last_became_peered":"2023-11-24T06:10:25.711924+0000","last_unstale":"2023-11-24T06:10:26.715214+0000","last_undegraded":"2023-11-24T06:10:26.715214+0000","last_fullsized":"2023-11-24T06:10:26.715214+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:46:00.341282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.885937+0000","last_change":"2023-11-24T06:10:24.695919+0000","last_active":"2023-11-24T06:10:26.885937+0000","last_peered":"2023-11-24T06:10:26.885937+0000","last_clean":"2023-11-24T06:10:26.885937+0000","last_became_active":"2023-11-24T06:10:24.695650+0000","last_became_peered":"2023-11-24T06:10:24.695650+0000","last_unstale":"2023-11-24T06:10:26.885937+0000","last_undegraded":"2023-11-24T06:10:26.885937+0000","last_fullsized":"2023-11-24T06:10:26.885937+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_clean_scrub_stamp":"2023-11-24T06:10:23.635255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:09:05.283059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.716052+0000","last_change":"2023-11-24T06:10:25.719797+0000","last_active":"2023-11-24T06:10:26.716052+0000","last_peered":"2023-11-24T06:10:26.716052+0000","last_clean":"2023-11-24T06:10:26.716052+0000","last_became_active":"2023-11-24T06:10:25.719483+0000","last_became_peered":"2023-11-24T06:10:25.719483+0000","last_unstale":"2023-11-24T06:10:26.716052+0000","last_undegraded":"2023-11-24T06:10:26.716052+0000","last_fullsized":"2023-11-24T06:10:26.716052+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:43:50.924468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.887695+0000","last_change":"2023-11-24T06:10:24.699236+0000","last_active":"2023-11-24T06:10:26.887695+0000","last_peered":"2023-11-24T06:10:26.887695+0000","last_clean":"2023-11-24T06:10:26.887695+0000","last_became_active":"2023-11-24T06:10:24.698709+0000","last_became_peered":"2023-11-24T06:10:24.698709+0000","last_unstale":"2023-11-24T06:10:26.887695+0000","last_undegraded":"2023-11-24T06:10:26.887695+0000","last_fullsized":"2023-11-24T06:10:26.887695+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_clean_scrub_stamp":"2023-11-24T06:10:23.635255+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:37:00.276208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.716147+0000","last_change":"2023-11-24T06:10:25.714752+0000","last_active":"2023-11-24T06:10:26.716147+0000","last_peered":"2023-11-24T06:10:26.716147+0000","last_clean":"2023-11-24T06:10:26.716147+0000","last_became_active":"2023-11-24T06:10:25.714468+0000","last_became_peered":"2023-11-24T06:10:25.714468+0000","last_unstale":"2023-11-24T06:10:26.716147+0000","last_undegraded":"2023-11-24T06:10:26.716147+0000","last_fullsized":"2023-11-24T06:10:26.716147+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:45:21.592844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.888957+0000","last_change":"2023-11-24T06:10:25.719973+0000","last_active":"2023-11-24T06:10:26.888957+0000","last_peered":"2023-11-24T06:10:26.888957+0000","last_clean":"2023-11-24T06:10:26.888957+0000","last_became_active":"2023-11-24T06:10:25.719716+0000","last_became_peered":"2023-11-24T06:10:25.719716+0000","last_unstale":"2023-11-24T06:10:26.888957+0000","last_undegraded":"2023-11-24T06:10:26.888957+0000","last_fullsized":"2023-11-24T06:10:26.888957+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:40:47.125372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.886889+0000","last_change":"2023-11-24T06:10:24.701875+0000","last_active":"2023-11-24T06:10:26.886889+0000","last_peered":"2023-11-24T06:10:26.886889+0000","last_clean":"2023-11-24T06:10:26.886889+0000","last_became_active":"2023-11-24T06:10:24.701520+0000","last_became_peered":"2023-11-24T06:10:24.701520+0000","last_unstale":"2023-11-24T06:10:26.886889+0000","last_undegraded":"2023-11-24T06:10:26.886889+0000","last_fullsized":"2023-11-24T06:10:26.886889+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_clean_scrub_stamp":"2023-11-24T06:10:23.635255+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:18:53.146509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.888401+0000","last_change":"2023-11-24T06:10:25.729759+0000","last_active":"2023-11-24T06:10:26.888401+0000","last_peered":"2023-11-24T06:10:26.888401+0000","last_clean":"2023-11-24T06:10:26.888401+0000","last_became_active":"2023-11-24T06:10:25.721916+0000","last_became_peered":"2023-11-24T06:10:25.721916+0000","last_unstale":"2023-11-24T06:10:26.888401+0000","last_undegraded":"2023-11-24T06:10:26.888401+0000","last_fullsized":"2023-11-24T06:10:26.888401+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:03:41.511367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.887901+0000","last_change":"2023-11-24T06:10:24.699400+0000","last_active":"2023-11-24T06:10:26.887901+0000","last_peered":"2023-11-24T06:10:26.887901+0000","last_clean":"2023-11-24T06:10:26.887901+0000","last_became_active":"2023-11-24T06:10:24.699090+0000","last_became_peered":"2023-11-24T06:10:24.699090+0000","last_unstale":"2023-11-24T06:10:26.887901+0000","last_undegraded":"2023-11-24T06:10:26.887901+0000","last_fullsized":"2023-11-24T06:10:26.887901+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_clean_scrub_stamp":"2023-11-24T06:10:23.635255+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:14:19.359955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.887209+0000","last_change":"2023-11-24T06:10:25.732187+0000","last_active":"2023-11-24T06:10:26.887209+0000","last_peered":"2023-11-24T06:10:26.887209+0000","last_clean":"2023-11-24T06:10:26.887209+0000","last_became_active":"2023-11-24T06:10:25.731130+0000","last_became_peered":"2023-11-24T06:10:25.731130+0000","last_unstale":"2023-11-24T06:10:26.887209+0000","last_undegraded":"2023-11-24T06:10:26.887209+0000","last_fullsized":"2023-11-24T06:10:26.887209+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:37:01.293292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.716804+0000","last_change":"2023-11-24T06:10:24.694116+0000","last_active":"2023-11-24T06:10:26.716804+0000","last_peered":"2023-11-24T06:10:26.716804+0000","last_clean":"2023-11-24T06:10:26.716804+0000","last_became_active":"2023-11-24T06:10:24.693832+0000","last_became_peered":"2023-11-24T06:10:24.693832+0000","last_unstale":"2023-11-24T06:10:26.716804+0000","last_undegraded":"2023-11-24T06:10:26.716804+0000","last_fullsized":"2023-11-24T06:10:26.716804+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_clean_scrub_stamp":"2023-11-24T06:10:23.635255+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:38:10.950543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.716865+0000","last_change":"2023-11-24T06:10:25.707741+0000","last_active":"2023-11-24T06:10:26.716865+0000","last_peered":"2023-11-24T06:10:26.716865+0000","last_clean":"2023-11-24T06:10:26.716865+0000","last_became_active":"2023-11-24T06:10:25.707453+0000","last_became_peered":"2023-11-24T06:10:25.707453+0000","last_unstale":"2023-11-24T06:10:26.716865+0000","last_undegraded":"2023-11-24T06:10:26.716865+0000","last_fullsized":"2023-11-24T06:10:26.716865+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:26:40.207866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.887375+0000","last_change":"2023-11-24T06:10:24.699025+0000","last_active":"2023-11-24T06:10:26.887375+0000","last_peered":"2023-11-24T06:10:26.887375+0000","last_clean":"2023-11-24T06:10:26.887375+0000","last_became_active":"2023-11-24T06:10:24.698788+0000","last_became_peered":"2023-11-24T06:10:24.698788+0000","last_unstale":"2023-11-24T06:10:26.887375+0000","last_undegraded":"2023-11-24T06:10:26.887375+0000","last_fullsized":"2023-11-24T06:10:26.887375+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_clean_scrub_stamp":"2023-11-24T06:10:23.635255+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:18:53.146509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574851,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31964,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339876,"statfs":{"total":96636764160,"available":96604033024,"internally_reserved":0,"allocated":802816,"data_stored":661860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89600000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78100000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2350000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0469999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2809999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4330000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6890000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574852,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31400,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340440,"statfs":{"total":96636764160,"available":96604610560,"internally_reserved":0,"allocated":217088,"data_stored":72522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.389}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.008}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2050000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.45}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.532}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.607}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.72}]}]},{"osd":5,"up_from":13,"seq":55834574852,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31388,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340452,"statfs":{"total":96636764160,"available":96604622848,"internally_reserved":0,"allocated":204800,"data_stored":70386,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.292}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.407}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4910000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.008}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1669999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.3109999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0790000000000002}]}]},{"osd":4,"up_from":13,"seq":55834574852,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31396,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340444,"statfs":{"total":96636764160,"available":96604614656,"internally_reserved":0,"allocated":217088,"data_stored":71036,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.177}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0129999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4039999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.708}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.556}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8340000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.895}]}]},{"osd":3,"up_from":13,"seq":55834574852,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":217088,"data_stored":72522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3100000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.107}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.516}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2030000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9910000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.821}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8640000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574852,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":196,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":200704,"data_stored":70296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91800000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80800000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1200000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8049999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2490000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2270000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6379999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574851,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":212992,"data_stored":71070,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0649999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2729999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1719999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6870000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.726}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0009999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574851,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27812,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344028,"statfs":{"total":96636764160,"available":96608284672,"internally_reserved":0,"allocated":811008,"data_stored":661916,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68700000000000006}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81499999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.276}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3049999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3660000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4510000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-24T06:10:30.620 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-11-24T06:10:30.620 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-11-24T06:10:30.620 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2023-11-24T06:10:30.621 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2023-11-24T06:10:30.621 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2023-11-24T06:10:30.870 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:30.867+0000 7f3123651640 1 Processor -- start 2023-11-24T06:10:30.870 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:30.867+0000 7f3123651640 1 -- start start 2023-11-24T06:10:30.872 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:30.868+0000 7f3123651640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f311c143ae0 0x7f311c0cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:30.872 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:30.868+0000 7f3123651640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f311c0cf7d0 0x7f311c15dca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:30.872 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:30.869+0000 7f3123651640 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f311c0cfc30 con 0x7f311c15e1e0 2023-11-24T06:10:30.872 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:30.869+0000 7f3123651640 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f311c0b1ee0 con 0x7f311c143ae0 2023-11-24T06:10:30.872 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:30.869+0000 7f3123651640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f311c0b2780 con 0x7f311c0cf7d0 2023-11-24T06:10:30.873 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:30.869+0000 7f3120bec640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f311c0cf7d0 0x7f311c15dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:30.873 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:30.869+0000 7f3121bee640 1 --1- >> v1:172.21.15.118:6789/0 conn(0x7f311c15e1e0 0x7f311c15a670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.118:6789/0 says I am v1:172.21.15.92:56770/0 (socket says 172.21.15.92:56770) 2023-11-24T06:10:30.873 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:30.869+0000 7f3121bee640 1 -- 172.21.15.92:0/866679932 learned_addr learned my addr 172.21.15.92:0/866679932 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-24T06:10:30.873 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:30.869+0000 7f31213ed640 1 --2- 172.21.15.92:0/866679932 >> v2:172.21.15.118:3300/0 conn(0x7f311c143ae0 0x7f311c0cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:30.874 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:30.870+0000 7f3120bec640 1 -- 172.21.15.92:0/866679932 >> v2:172.21.15.118:3300/0 conn(0x7f311c143ae0 msgr2=0x7f311c0cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:30.874 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:30.870+0000 7f3120bec640 1 --2- 172.21.15.92:0/866679932 >> v2:172.21.15.118:3300/0 conn(0x7f311c143ae0 0x7f311c0cf290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:30.874 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:30.870+0000 7f3120bec640 1 -- 172.21.15.92:0/866679932 >> v1:172.21.15.118:6789/0 conn(0x7f311c15e1e0 legacy=0x7f311c15a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:30.874 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:30.870+0000 7f3120bec640 1 -- 172.21.15.92:0/866679932 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f311c15e990 con 0x7f311c0cf7d0 2023-11-24T06:10:30.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:30.872+0000 7f3120bec640 1 --2- 172.21.15.92:0/866679932 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f311c0cf7d0 0x7f311c15dca0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f310c0097a0 tx=0x7f310c02dae0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=75b9e53da80c7d47 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:30.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:30.872+0000 7f3113fff640 1 -- 172.21.15.92:0/866679932 <== mon.2 v2:172.21.15.118:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f310c02f8a0 con 0x7f311c0cf7d0 2023-11-24T06:10:30.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:30.872+0000 7f3113fff640 1 -- 172.21.15.92:0/866679932 <== mon.2 v2:172.21.15.118:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f310c02fa00 con 0x7f311c0cf7d0 2023-11-24T06:10:30.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:30.872+0000 7f3113fff640 1 -- 172.21.15.92:0/866679932 <== mon.2 v2:172.21.15.118:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f310c02fd90 con 0x7f311c0cf7d0 2023-11-24T06:10:30.876 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:30.873+0000 7f3123651640 1 -- 172.21.15.92:0/866679932 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f311c0cf7d0 msgr2=0x7f311c15dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:30.876 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:30.873+0000 7f3123651640 1 --2- 172.21.15.92:0/866679932 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f311c0cf7d0 0x7f311c15dca0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f310c0097a0 tx=0x7f310c02dae0 comp rx=0 tx=0).stop 2023-11-24T06:10:30.876 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:30.873+0000 7f3123651640 1 -- 172.21.15.92:0/866679932 shutdown_connections 2023-11-24T06:10:30.876 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:30.873+0000 7f3123651640 1 --2- 172.21.15.92:0/866679932 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f311c0cf7d0 0x7f311c15dca0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:30.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:30.873+0000 7f3123651640 1 --2- 172.21.15.92:0/866679932 >> v2:172.21.15.118:3300/0 conn(0x7f311c143ae0 0x7f311c0cf290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:30.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:30.873+0000 7f3123651640 1 -- 172.21.15.92:0/866679932 >> 172.21.15.92:0/866679932 conn(0x7f311c06a800 msgr2=0x7f311c06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:30.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:30.873+0000 7f3123651640 1 -- 172.21.15.92:0/866679932 shutdown_connections 2023-11-24T06:10:30.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:30.873+0000 7f3123651640 1 -- 172.21.15.92:0/866679932 wait complete. 2023-11-24T06:10:30.878 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:30.875+0000 7f3123651640 1 Processor -- start 2023-11-24T06:10:30.878 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:30.875+0000 7f3123651640 1 -- start start 2023-11-24T06:10:30.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:30.876+0000 7f3123651640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f311c0cf7d0 0x7f311c13f430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:30.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:30.876+0000 7f3123651640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f311c143ae0 0x7f311c1e44b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:30.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:30.876+0000 7f3123651640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f311c15e1e0 0x7f311c1e49f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:30.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:30.876+0000 7f3123651640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f311c1422d0 con 0x7f311c143ae0 2023-11-24T06:10:30.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:30.876+0000 7f3123651640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f311c1416c0 con 0x7f311c0cf7d0 2023-11-24T06:10:30.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:30.876+0000 7f3123651640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f311c1419a0 con 0x7f311c15e1e0 2023-11-24T06:10:30.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:30.876+0000 7f3120bec640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f311c143ae0 0x7f311c1e44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:30.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:30.876+0000 7f3120bec640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f311c143ae0 0x7f311c1e44b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47198/0 (socket says 172.21.15.92:47198) 2023-11-24T06:10:30.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:30.876+0000 7f3120bec640 1 -- 172.21.15.92:0/3608534851 learned_addr learned my addr 172.21.15.92:0/3608534851 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:10:30.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:30.877+0000 7f31213ed640 1 --2- 172.21.15.92:0/3608534851 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f311c0cf7d0 0x7f311c13f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:30.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:30.877+0000 7f3121bee640 1 --2- 172.21.15.92:0/3608534851 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f311c15e1e0 0x7f311c1e49f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:30.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:30.877+0000 7f3120bec640 1 -- 172.21.15.92:0/3608534851 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f311c15e1e0 msgr2=0x7f311c1e49f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:30.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:30.877+0000 7f3120bec640 1 --2- 172.21.15.92:0/3608534851 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f311c15e1e0 0x7f311c1e49f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:30.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:30.877+0000 7f3120bec640 1 -- 172.21.15.92:0/3608534851 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f311c0cf7d0 msgr2=0x7f311c13f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:30.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:30.877+0000 7f3120bec640 1 --2- 172.21.15.92:0/3608534851 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f311c0cf7d0 0x7f311c13f430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:30.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:30.877+0000 7f3120bec640 1 -- 172.21.15.92:0/3608534851 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f310c009400 con 0x7f311c143ae0 2023-11-24T06:10:30.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:30.877+0000 7f31213ed640 1 --2- 172.21.15.92:0/3608534851 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f311c0cf7d0 0x7f311c13f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:10:30.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:30.877+0000 7f3120bec640 1 --2- 172.21.15.92:0/3608534851 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f311c143ae0 0x7f311c1e44b0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f310c02e0f0 tx=0x7f310c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:30.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:30.877+0000 7f3111ffb640 1 -- 172.21.15.92:0/3608534851 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f310c030ab0 con 0x7f311c143ae0 2023-11-24T06:10:30.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:30.877+0000 7f3111ffb640 1 -- 172.21.15.92:0/3608534851 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f310c030c30 con 0x7f311c143ae0 2023-11-24T06:10:30.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:30.877+0000 7f3123651640 1 -- 172.21.15.92:0/3608534851 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f311c141c80 con 0x7f311c143ae0 2023-11-24T06:10:30.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:30.877+0000 7f3111ffb640 1 -- 172.21.15.92:0/3608534851 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f310c042410 con 0x7f311c143ae0 2023-11-24T06:10:30.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:30.878+0000 7f3123651640 1 -- 172.21.15.92:0/3608534851 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f311c13f970 con 0x7f311c143ae0 2023-11-24T06:10:30.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:30.879+0000 7f3123651640 1 -- 172.21.15.92:0/3608534851 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f311c142480 con 0x7f311c143ae0 2023-11-24T06:10:30.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:30.879+0000 7f3111ffb640 1 -- 172.21.15.92:0/3608534851 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f310c039400 con 0x7f311c143ae0 2023-11-24T06:10:30.884 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:30.880+0000 7f3111ffb640 1 --2- 172.21.15.92:0/3608534851 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f310006ae40 0x7f310006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:30.884 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:30.880+0000 7f3111ffb640 1 -- 172.21.15.92:0/3608534851 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f310c0ab380 con 0x7f311c143ae0 2023-11-24T06:10:30.884 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:30.881+0000 7f31213ed640 1 --2- 172.21.15.92:0/3608534851 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f310006ae40 0x7f310006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:30.884 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:30.881+0000 7f31213ed640 1 --2- 172.21.15.92:0/3608534851 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f310006ae40 0x7f310006d300 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f3118009ac0 tx=0x7f3118008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:30.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:30.885+0000 7f3111ffb640 1 -- 172.21.15.92:0/3608534851 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f311c142480 con 0x7f311c143ae0 2023-11-24T06:10:31.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:31.108+0000 7f3123651640 1 -- 172.21.15.92:0/3608534851 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f311c0675d0 con 0x7f311c143ae0 2023-11-24T06:10:31.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:31.117+0000 7f3111ffb640 1 -- 172.21.15.92:0/3608534851 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+171028 (secure 0 0 0) 0x7f310c076540 con 0x7f311c143ae0 2023-11-24T06:10:31.122 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-24T06:10:31.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:31.121+0000 7f3123651640 1 -- 172.21.15.92:0/3608534851 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f310006ae40 msgr2=0x7f310006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:31.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:31.121+0000 7f3123651640 1 --2- 172.21.15.92:0/3608534851 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f310006ae40 0x7f310006d300 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f3118009ac0 tx=0x7f3118008040 comp rx=0 tx=0).stop 2023-11-24T06:10:31.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:31.121+0000 7f3123651640 1 -- 172.21.15.92:0/3608534851 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f311c143ae0 msgr2=0x7f311c1e44b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:31.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:31.121+0000 7f3123651640 1 --2- 172.21.15.92:0/3608534851 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f311c143ae0 0x7f311c1e44b0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f310c02e0f0 tx=0x7f310c0046c0 comp rx=0 tx=0).stop 2023-11-24T06:10:31.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:31.121+0000 7f3123651640 1 -- 172.21.15.92:0/3608534851 shutdown_connections 2023-11-24T06:10:31.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:31.121+0000 7f3123651640 1 --2- 172.21.15.92:0/3608534851 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f310006ae40 0x7f310006d300 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:31.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:31.121+0000 7f3123651640 1 --2- 172.21.15.92:0/3608534851 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f311c15e1e0 0x7f311c1e49f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:31.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:31.121+0000 7f3123651640 1 --2- 172.21.15.92:0/3608534851 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f311c143ae0 0x7f311c1e44b0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:31.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:31.121+0000 7f3123651640 1 --2- 172.21.15.92:0/3608534851 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f311c0cf7d0 0x7f311c13f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:31.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:31.121+0000 7f3123651640 1 -- 172.21.15.92:0/3608534851 >> 172.21.15.92:0/3608534851 conn(0x7f311c06a800 msgr2=0x7f311c0b3690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:31.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:31.122+0000 7f3123651640 1 -- 172.21.15.92:0/3608534851 shutdown_connections 2023-11-24T06:10:31.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:31.122+0000 7f3123651640 1 -- 172.21.15.92:0/3608534851 wait complete. 2023-11-24T06:10:31.143 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":5,"active_gid":4101,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":784027401},{"type":"v1","addr":"172.21.15.92:6801","nonce":784027401}]},"active_addr":"172.21.15.92:6801/784027401","active_change":"2023-11-24T06:10:09.649020+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":4108,"name":"x","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.92:0","nonce":370754313}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.92:0","nonce":3623001361}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.92:0","nonce":3505134739}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.92:0","nonce":4056701907}]}]} 2023-11-24T06:10:31.147 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2023-11-24T06:10:31.147 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2023-11-24T06:10:31.147 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:10:31.355 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:31.352+0000 7f49e1ead640 1 Processor -- start 2023-11-24T06:10:31.355 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:31.352+0000 7f49e1ead640 1 -- start start 2023-11-24T06:10:31.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:31.354+0000 7f49e1ead640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f49dc143ae0 0x7f49dc0cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:31.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:31.354+0000 7f49e1ead640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f49dc0cf7d0 0x7f49dc15dca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:31.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:31.354+0000 7f49e1ead640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49dc0cfc30 con 0x7f49dc15e1e0 2023-11-24T06:10:31.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:31.354+0000 7f49e1ead640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49dc0b1ee0 con 0x7f49dc0cf7d0 2023-11-24T06:10:31.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:31.354+0000 7f49e1ead640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49dc0b2780 con 0x7f49dc143ae0 2023-11-24T06:10:31.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:31.354+0000 7f49dbfff640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f49dc15e1e0 0x7f49dc15a670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:55766/0 (socket says 172.21.15.92:55766) 2023-11-24T06:10:31.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:31.354+0000 7f49d3fff640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f49dc0cf7d0 0x7f49dc15dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:31.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:31.354+0000 7f49dbfff640 1 -- 172.21.15.92:0/1034066380 learned_addr learned my addr 172.21.15.92:0/1034066380 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-24T06:10:31.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:31.354+0000 7f49db7fe640 1 --2- 172.21.15.92:0/1034066380 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f49dc143ae0 0x7f49dc0cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:31.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:31.354+0000 7f49d3fff640 1 -- 172.21.15.92:0/1034066380 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f49dc143ae0 msgr2=0x7f49dc0cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:31.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:31.354+0000 7f49d3fff640 1 --2- 172.21.15.92:0/1034066380 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f49dc143ae0 0x7f49dc0cf290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:31.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:31.354+0000 7f49d3fff640 1 -- 172.21.15.92:0/1034066380 >> v1:172.21.15.92:6789/0 conn(0x7f49dc15e1e0 legacy=0x7f49dc15a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:31.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:31.354+0000 7f49d3fff640 1 -- 172.21.15.92:0/1034066380 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49dc15e990 con 0x7f49dc0cf7d0 2023-11-24T06:10:31.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:31.356+0000 7f49d3fff640 1 --2- 172.21.15.92:0/1034066380 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f49dc0cf7d0 0x7f49dc15dca0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f49c40097a0 tx=0x7f49c402b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=feeb7cc930f1065e server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:31.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:31.356+0000 7f49daffd640 1 -- 172.21.15.92:0/1034066380 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49c402d040 con 0x7f49dc0cf7d0 2023-11-24T06:10:31.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:31.356+0000 7f49daffd640 1 -- 172.21.15.92:0/1034066380 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49c4031810 con 0x7f49dc0cf7d0 2023-11-24T06:10:31.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:31.356+0000 7f49daffd640 1 -- 172.21.15.92:0/1034066380 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49c4031ba0 con 0x7f49dc0cf7d0 2023-11-24T06:10:31.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:31.356+0000 7f49e1ead640 1 -- 172.21.15.92:0/1034066380 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f49dc0cf7d0 msgr2=0x7f49dc15dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:31.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:31.356+0000 7f49e1ead640 1 --2- 172.21.15.92:0/1034066380 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f49dc0cf7d0 0x7f49dc15dca0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f49c40097a0 tx=0x7f49c402b8b0 comp rx=0 tx=0).stop 2023-11-24T06:10:31.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:31.356+0000 7f49e1ead640 1 -- 172.21.15.92:0/1034066380 shutdown_connections 2023-11-24T06:10:31.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:31.356+0000 7f49e1ead640 1 --2- 172.21.15.92:0/1034066380 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f49dc0cf7d0 0x7f49dc15dca0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:31.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:31.356+0000 7f49e1ead640 1 --2- 172.21.15.92:0/1034066380 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f49dc143ae0 0x7f49dc0cf290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:31.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:31.356+0000 7f49e1ead640 1 -- 172.21.15.92:0/1034066380 >> 172.21.15.92:0/1034066380 conn(0x7f49dc06a800 msgr2=0x7f49dc06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:31.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:31.356+0000 7f49e1ead640 1 -- 172.21.15.92:0/1034066380 shutdown_connections 2023-11-24T06:10:31.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:31.357+0000 7f49e1ead640 1 -- 172.21.15.92:0/1034066380 wait complete. 2023-11-24T06:10:31.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:31.358+0000 7f49e1ead640 1 Processor -- start 2023-11-24T06:10:31.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:31.358+0000 7f49e1ead640 1 -- start start 2023-11-24T06:10:31.363 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:31.359+0000 7f49e1ead640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f49dc0cf7d0 0x7f49dc0c5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:31.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:31.359+0000 7f49e1ead640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f49dc143ae0 0x7f49dc0afda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:31.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:31.359+0000 7f49e1ead640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f49dc15e1e0 0x7f49dc0c8b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:31.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:31.359+0000 7f49e1ead640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49dc0c5c40 con 0x7f49dc143ae0 2023-11-24T06:10:31.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:31.359+0000 7f49e1ead640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49dc0c4660 con 0x7f49dc0cf7d0 2023-11-24T06:10:31.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:31.359+0000 7f49e1ead640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49dc0c4940 con 0x7f49dc15e1e0 2023-11-24T06:10:31.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:31.359+0000 7f49d3fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f49dc143ae0 0x7f49dc0afda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:31.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:31.359+0000 7f49d3fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f49dc143ae0 0x7f49dc0afda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47212/0 (socket says 172.21.15.92:47212) 2023-11-24T06:10:31.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:31.359+0000 7f49d3fff640 1 -- 172.21.15.92:0/2070465653 learned_addr learned my addr 172.21.15.92:0/2070465653 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:10:31.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:31.360+0000 7f49db7fe640 1 --2- 172.21.15.92:0/2070465653 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f49dc0cf7d0 0x7f49dc0c5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:31.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:31.360+0000 7f49d3fff640 1 -- 172.21.15.92:0/2070465653 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f49dc15e1e0 msgr2=0x7f49dc0c8b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:31.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:31.360+0000 7f49d3fff640 1 --2- 172.21.15.92:0/2070465653 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f49dc15e1e0 0x7f49dc0c8b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:31.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:31.360+0000 7f49d3fff640 1 -- 172.21.15.92:0/2070465653 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f49dc0cf7d0 msgr2=0x7f49dc0c5700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:31.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:31.360+0000 7f49d3fff640 1 --2- 172.21.15.92:0/2070465653 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f49dc0cf7d0 0x7f49dc0c5700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:31.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:31.360+0000 7f49d3fff640 1 -- 172.21.15.92:0/2070465653 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49c4009400 con 0x7f49dc143ae0 2023-11-24T06:10:31.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:31.360+0000 7f49d3fff640 1 --2- 172.21.15.92:0/2070465653 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f49dc143ae0 0x7f49dc0afda0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f49c40098c0 tx=0x7f49c40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:31.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:31.360+0000 7f49d8ff9640 1 -- 172.21.15.92:0/2070465653 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49c403ad60 con 0x7f49dc143ae0 2023-11-24T06:10:31.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:31.360+0000 7f49d8ff9640 1 -- 172.21.15.92:0/2070465653 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49c4044400 con 0x7f49dc143ae0 2023-11-24T06:10:31.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:31.360+0000 7f49d8ff9640 1 -- 172.21.15.92:0/2070465653 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49c40447b0 con 0x7f49dc143ae0 2023-11-24T06:10:31.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:31.360+0000 7f49e1ead640 1 -- 172.21.15.92:0/2070465653 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49dc0c4c20 con 0x7f49dc143ae0 2023-11-24T06:10:31.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:31.360+0000 7f49e1ead640 1 -- 172.21.15.92:0/2070465653 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49dc0c51e0 con 0x7f49dc143ae0 2023-11-24T06:10:31.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:31.362+0000 7f49e1ead640 1 -- 172.21.15.92:0/2070465653 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49dc0b4420 con 0x7f49dc143ae0 2023-11-24T06:10:31.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:31.362+0000 7f49d8ff9640 1 -- 172.21.15.92:0/2070465653 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f49c404d440 con 0x7f49dc143ae0 2023-11-24T06:10:31.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:31.363+0000 7f49d8ff9640 1 --2- 172.21.15.92:0/2070465653 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f49bc06ae40 0x7f49bc06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:31.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:31.363+0000 7f49d8ff9640 1 -- 172.21.15.92:0/2070465653 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f49c40ab970 con 0x7f49dc143ae0 2023-11-24T06:10:31.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:31.363+0000 7f49db7fe640 1 --2- 172.21.15.92:0/2070465653 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f49bc06ae40 0x7f49bc06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:31.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:31.364+0000 7f49db7fe640 1 --2- 172.21.15.92:0/2070465653 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f49bc06ae40 0x7f49bc06d300 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f49cc006fd0 tx=0x7f49cc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:31.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:31.368+0000 7f49d8ff9640 1 -- 172.21.15.92:0/2070465653 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f49c4076b30 con 0x7f49dc143ae0 2023-11-24T06:10:31.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:31.544+0000 7f49e1ead640 1 -- 172.21.15.92:0/2070465653 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f49dc0cbf50 con 0x7f49dc143ae0 2023-11-24T06:10:31.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:31.546+0000 7f49d8ff9640 1 -- 172.21.15.92:0/2070465653 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16457 (secure 0 0 0) 0x7f49c4035090 con 0x7f49dc143ae0 2023-11-24T06:10:31.550 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-24T06:10:31.550 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":20,"fsid":"2d6dee5f-8aa3-470f-931a-50ade963b1cb","created":"2023-11-24T06:10:04.707204+0000","modified":"2023-11-24T06:10:26.698733+0000","last_up_change":"2023-11-24T06:10:17.598352+0000","last_in_change":"2023-11-24T06:10:09.040793+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:10:17.733098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:10:22.926445+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T06:10:24.151288+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ff37a9fb-135f-471f-b4b0-f7b5772006c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6803","nonce":20560714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6805","nonce":20560714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6809","nonce":20560714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6807","nonce":20560714}]},"public_addr":"172.21.15.92:6803/20560714","cluster_addr":"172.21.15.92:6805/20560714","heartbeat_back_addr":"172.21.15.92:6809/20560714","heartbeat_front_addr":"172.21.15.92:6807/20560714","state":["exists","up"]},{"osd":1,"uuid":"03cdd353-dcf1-4f38-94fe-a795137c06b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6819","nonce":2451239906}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6821","nonce":2451239906}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6825","nonce":2451239906}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6823","nonce":2451239906}]},"public_addr":"172.21.15.92:6819/2451239906","cluster_addr":"172.21.15.92:6821/2451239906","heartbeat_back_addr":"172.21.15.92:6825/2451239906","heartbeat_front_addr":"172.21.15.92:6823/2451239906","state":["exists","up"]},{"osd":2,"uuid":"a4a20d80-5d57-484b-becd-9aa8070dd3aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6811","nonce":1124979277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6813","nonce":1124979277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6817","nonce":1124979277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6815","nonce":1124979277}]},"public_addr":"172.21.15.92:6811/1124979277","cluster_addr":"172.21.15.92:6813/1124979277","heartbeat_back_addr":"172.21.15.92:6817/1124979277","heartbeat_front_addr":"172.21.15.92:6815/1124979277","state":["exists","up"]},{"osd":3,"uuid":"4648d7ce-9b29-4583-9425-4ba8fd2b1903","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6827","nonce":1734115782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6829","nonce":1734115782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6833","nonce":1734115782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6831","nonce":1734115782}]},"public_addr":"172.21.15.92:6827/1734115782","cluster_addr":"172.21.15.92:6829/1734115782","heartbeat_back_addr":"172.21.15.92:6833/1734115782","heartbeat_front_addr":"172.21.15.92:6831/1734115782","state":["exists","up"]},{"osd":4,"uuid":"3bc9d7eb-c6fa-4fcf-a3ec-ade604e05aac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6819","nonce":3425937229}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6822","nonce":3425937229}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6830","nonce":3425937229}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6826","nonce":3425937229}]},"public_addr":"172.21.15.118:6819/3425937229","cluster_addr":"172.21.15.118:6822/3425937229","heartbeat_back_addr":"172.21.15.118:6830/3425937229","heartbeat_front_addr":"172.21.15.118:6826/3425937229","state":["exists","up"]},{"osd":5,"uuid":"dce9ba6d-59ed-46d1-84fc-a84b13de10c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6801","nonce":827455984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6803","nonce":827455984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6807","nonce":827455984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6805","nonce":827455984}]},"public_addr":"172.21.15.118:6801/827455984","cluster_addr":"172.21.15.118:6803/827455984","heartbeat_back_addr":"172.21.15.118:6807/827455984","heartbeat_front_addr":"172.21.15.118:6805/827455984","state":["exists","up"]},{"osd":6,"uuid":"82aec504-ae58-4709-8d30-46f2b12081b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6809","nonce":2946986141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6811","nonce":2946986141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6815","nonce":2946986141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6813","nonce":2946986141}]},"public_addr":"172.21.15.118:6809/2946986141","cluster_addr":"172.21.15.118:6811/2946986141","heartbeat_back_addr":"172.21.15.118:6815/2946986141","heartbeat_front_addr":"172.21.15.118:6813/2946986141","state":["exists","up"]},{"osd":7,"uuid":"4ed4c624-c88f-4d35-bdb1-bf209b5dffb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6817","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6818","nonce":48122059}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6821","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6823","nonce":48122059}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6829","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6831","nonce":48122059}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6825","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6827","nonce":48122059}]},"public_addr":"172.21.15.118:6818/48122059","cluster_addr":"172.21.15.118:6823/48122059","heartbeat_back_addr":"172.21.15.118:6831/48122059","heartbeat_front_addr":"172.21.15.118:6827/48122059","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.536664+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.368324+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.271201+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.342971+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.621832+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.387110+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.632946+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.416032+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:10:31.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:31.549+0000 7f49e1ead640 1 -- 172.21.15.92:0/2070465653 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f49bc06ae40 msgr2=0x7f49bc06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:31.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:31.549+0000 7f49e1ead640 1 --2- 172.21.15.92:0/2070465653 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f49bc06ae40 0x7f49bc06d300 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f49cc006fd0 tx=0x7f49cc008040 comp rx=0 tx=0).stop 2023-11-24T06:10:31.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:31.549+0000 7f49e1ead640 1 -- 172.21.15.92:0/2070465653 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f49dc143ae0 msgr2=0x7f49dc0afda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:31.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:31.549+0000 7f49e1ead640 1 --2- 172.21.15.92:0/2070465653 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f49dc143ae0 0x7f49dc0afda0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f49c40098c0 tx=0x7f49c40046c0 comp rx=0 tx=0).stop 2023-11-24T06:10:31.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:31.549+0000 7f49e1ead640 1 -- 172.21.15.92:0/2070465653 shutdown_connections 2023-11-24T06:10:31.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:31.549+0000 7f49e1ead640 1 --2- 172.21.15.92:0/2070465653 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f49bc06ae40 0x7f49bc06d300 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:31.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:31.549+0000 7f49e1ead640 1 --2- 172.21.15.92:0/2070465653 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f49dc15e1e0 0x7f49dc0c8b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:31.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:31.549+0000 7f49e1ead640 1 --2- 172.21.15.92:0/2070465653 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f49dc143ae0 0x7f49dc0afda0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:31.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:31.549+0000 7f49e1ead640 1 --2- 172.21.15.92:0/2070465653 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f49dc0cf7d0 0x7f49dc0c5700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:31.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:31.549+0000 7f49e1ead640 1 -- 172.21.15.92:0/2070465653 >> 172.21.15.92:0/2070465653 conn(0x7f49dc06a800 msgr2=0x7f49dc15aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:31.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:31.549+0000 7f49e1ead640 1 -- 172.21.15.92:0/2070465653 shutdown_connections 2023-11-24T06:10:31.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:31.549+0000 7f49e1ead640 1 -- 172.21.15.92:0/2070465653 wait complete. 2023-11-24T06:10:31.571 INFO:tasks.ceph.ceph_manager.ceph:all up! 2023-11-24T06:10:31.571 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:10:31.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:31.781+0000 7f179eeda640 1 Processor -- start 2023-11-24T06:10:31.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:31.782+0000 7f179eeda640 1 -- start start 2023-11-24T06:10:31.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:31.783+0000 7f179eeda640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1798071410 0x7f1798146770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:31.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:31.783+0000 7f179eeda640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f179806c7c0 0x7f17980717f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:31.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:31.783+0000 7f179eeda640 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f179806cc20 con 0x7f1798071d30 2023-11-24T06:10:31.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:31.783+0000 7f179eeda640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f179806cec0 con 0x7f179806c7c0 2023-11-24T06:10:31.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:31.783+0000 7f179eeda640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f179806d1a0 con 0x7f1798071410 2023-11-24T06:10:31.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:31.784+0000 7f179d6d7640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f179806c7c0 0x7f17980717f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:31.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:31.784+0000 7f179d6d7640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f179806c7c0 0x7f17980717f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47220/0 (socket says 172.21.15.92:47220) 2023-11-24T06:10:31.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:31.784+0000 7f179d6d7640 1 -- 172.21.15.92:0/871004665 learned_addr learned my addr 172.21.15.92:0/871004665 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:10:31.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:31.784+0000 7f179d6d7640 1 -- 172.21.15.92:0/871004665 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1798071410 msgr2=0x7f1798146770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:31.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:31.784+0000 7f179ded8640 1 --2- 172.21.15.92:0/871004665 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1798071410 0x7f1798146770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:31.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:31.784+0000 7f179d6d7640 1 --2- 172.21.15.92:0/871004665 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1798071410 0x7f1798146770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:31.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:31.784+0000 7f179d6d7640 1 -- 172.21.15.92:0/871004665 >> v1:172.21.15.118:6789/0 conn(0x7f1798071d30 legacy=0x7f17980b1e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:31.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:31.784+0000 7f179d6d7640 1 -- 172.21.15.92:0/871004665 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f179806d480 con 0x7f179806c7c0 2023-11-24T06:10:31.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:31.784+0000 7f179ded8640 1 --2- 172.21.15.92:0/871004665 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1798071410 0x7f1798146770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:10:31.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:31.785+0000 7f179d6d7640 1 --2- 172.21.15.92:0/871004665 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f179806c7c0 0x7f17980717f0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f1788009870 tx=0x7f178802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6b4ee97c667367b server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:31.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:31.785+0000 7f179ced6640 1 -- 172.21.15.92:0/871004665 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1788031640 con 0x7f179806c7c0 2023-11-24T06:10:31.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:31.786+0000 7f179ced6640 1 -- 172.21.15.92:0/871004665 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17880317a0 con 0x7f179806c7c0 2023-11-24T06:10:31.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:31.786+0000 7f179ced6640 1 -- 172.21.15.92:0/871004665 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1788031b30 con 0x7f179806c7c0 2023-11-24T06:10:31.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:31.786+0000 7f179eeda640 1 -- 172.21.15.92:0/871004665 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f179806c7c0 msgr2=0x7f17980717f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:31.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:31.786+0000 7f179eeda640 1 --2- 172.21.15.92:0/871004665 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f179806c7c0 0x7f17980717f0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f1788009870 tx=0x7f178802b8f0 comp rx=0 tx=0).stop 2023-11-24T06:10:31.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:31.786+0000 7f179eeda640 1 -- 172.21.15.92:0/871004665 shutdown_connections 2023-11-24T06:10:31.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:31.786+0000 7f179eeda640 1 --2- 172.21.15.92:0/871004665 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f179806c7c0 0x7f17980717f0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:31.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:31.786+0000 7f179eeda640 1 --2- 172.21.15.92:0/871004665 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1798071410 0x7f1798146770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:31.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:31.786+0000 7f179eeda640 1 -- 172.21.15.92:0/871004665 >> 172.21.15.92:0/871004665 conn(0x7f179806a800 msgr2=0x7f179806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:31.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:31.786+0000 7f179eeda640 1 -- 172.21.15.92:0/871004665 shutdown_connections 2023-11-24T06:10:31.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:31.786+0000 7f179eeda640 1 -- 172.21.15.92:0/871004665 wait complete. 2023-11-24T06:10:31.794 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:31.788+0000 7f179eeda640 1 Processor -- start 2023-11-24T06:10:31.794 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:31.788+0000 7f179eeda640 1 -- start start 2023-11-24T06:10:31.794 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:31.789+0000 7f179eeda640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f179806c7c0 0x7f179815c730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:31.794 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:31.789+0000 7f179eeda640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1798071410 0x7f17981e38e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:31.794 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:31.789+0000 7f179eeda640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1798071d30 0x7f17981e8540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:31.794 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:31.789+0000 7f179eeda640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17981e6920 con 0x7f179806c7c0 2023-11-24T06:10:31.794 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:31.789+0000 7f179eeda640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17981e4230 con 0x7f1798071410 2023-11-24T06:10:31.794 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:31.789+0000 7f179eeda640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17981e4510 con 0x7f1798071d30 2023-11-24T06:10:31.794 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:31.789+0000 7f179ded8640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f179806c7c0 0x7f179815c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:31.794 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:31.790+0000 7f179ded8640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f179806c7c0 0x7f179815c730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47224/0 (socket says 172.21.15.92:47224) 2023-11-24T06:10:31.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:31.790+0000 7f179ded8640 1 -- 172.21.15.92:0/41857336 learned_addr learned my addr 172.21.15.92:0/41857336 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:10:31.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:31.790+0000 7f179ded8640 1 -- 172.21.15.92:0/41857336 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1798071d30 msgr2=0x7f17981e8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:31.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:31.790+0000 7f179ded8640 1 --2- 172.21.15.92:0/41857336 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1798071d30 0x7f17981e8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:31.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:31.790+0000 7f179ded8640 1 -- 172.21.15.92:0/41857336 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1798071410 msgr2=0x7f17981e38e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:31.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:31.790+0000 7f179ded8640 1 --2- 172.21.15.92:0/41857336 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1798071410 0x7f17981e38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:31.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:31.790+0000 7f179ded8640 1 -- 172.21.15.92:0/41857336 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17880094d0 con 0x7f179806c7c0 2023-11-24T06:10:31.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:31.790+0000 7f179ded8640 1 --2- 172.21.15.92:0/41857336 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f179806c7c0 0x7f179815c730 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f179400c7f0 tx=0x7f179400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:31.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:31.790+0000 7f178e7fc640 1 -- 172.21.15.92:0/41857336 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1794007a30 con 0x7f179806c7c0 2023-11-24T06:10:31.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:31.790+0000 7f178e7fc640 1 -- 172.21.15.92:0/41857336 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1794007b90 con 0x7f179806c7c0 2023-11-24T06:10:31.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:31.790+0000 7f178e7fc640 1 -- 172.21.15.92:0/41857336 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f179400ce40 con 0x7f179806c7c0 2023-11-24T06:10:31.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:31.790+0000 7f179eeda640 1 -- 172.21.15.92:0/41857336 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17981e47f0 con 0x7f179806c7c0 2023-11-24T06:10:31.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:31.791+0000 7f179eeda640 1 -- 172.21.15.92:0/41857336 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17981e3e20 con 0x7f179806c7c0 2023-11-24T06:10:31.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:31.792+0000 7f179eeda640 1 -- 172.21.15.92:0/41857336 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1798067450 con 0x7f179806c7c0 2023-11-24T06:10:31.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:31.792+0000 7f178e7fc640 1 -- 172.21.15.92:0/41857336 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f17940113f0 con 0x7f179806c7c0 2023-11-24T06:10:31.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:31.793+0000 7f178e7fc640 1 --2- 172.21.15.92:0/41857336 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f177c06ae40 0x7f177c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:31.797 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:31.793+0000 7f178e7fc640 1 -- 172.21.15.92:0/41857336 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f17940897a0 con 0x7f179806c7c0 2023-11-24T06:10:31.797 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:31.794+0000 7f179d6d7640 1 --2- 172.21.15.92:0/41857336 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f177c06ae40 0x7f177c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:31.797 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:31.794+0000 7f179d6d7640 1 --2- 172.21.15.92:0/41857336 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f177c06ae40 0x7f177c06d300 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f1788036000 tx=0x7f1788030000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:31.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:31.798+0000 7f178e7fc640 1 -- 172.21.15.92:0/41857336 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1794054850 con 0x7f179806c7c0 2023-11-24T06:10:31.988 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:31.985+0000 7f179eeda640 1 -- 172.21.15.92:0/41857336 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1798142210 con 0x7f179806c7c0 2023-11-24T06:10:31.990 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:31.987+0000 7f178e7fc640 1 -- 172.21.15.92:0/41857336 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16457 (secure 0 0 0) 0x7f1794057e70 con 0x7f179806c7c0 2023-11-24T06:10:31.991 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-24T06:10:31.992 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":20,"fsid":"2d6dee5f-8aa3-470f-931a-50ade963b1cb","created":"2023-11-24T06:10:04.707204+0000","modified":"2023-11-24T06:10:26.698733+0000","last_up_change":"2023-11-24T06:10:17.598352+0000","last_in_change":"2023-11-24T06:10:09.040793+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:10:17.733098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:10:22.926445+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T06:10:24.151288+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ff37a9fb-135f-471f-b4b0-f7b5772006c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6803","nonce":20560714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6805","nonce":20560714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6809","nonce":20560714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6807","nonce":20560714}]},"public_addr":"172.21.15.92:6803/20560714","cluster_addr":"172.21.15.92:6805/20560714","heartbeat_back_addr":"172.21.15.92:6809/20560714","heartbeat_front_addr":"172.21.15.92:6807/20560714","state":["exists","up"]},{"osd":1,"uuid":"03cdd353-dcf1-4f38-94fe-a795137c06b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6819","nonce":2451239906}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6821","nonce":2451239906}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6825","nonce":2451239906}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6823","nonce":2451239906}]},"public_addr":"172.21.15.92:6819/2451239906","cluster_addr":"172.21.15.92:6821/2451239906","heartbeat_back_addr":"172.21.15.92:6825/2451239906","heartbeat_front_addr":"172.21.15.92:6823/2451239906","state":["exists","up"]},{"osd":2,"uuid":"a4a20d80-5d57-484b-becd-9aa8070dd3aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6811","nonce":1124979277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6813","nonce":1124979277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6817","nonce":1124979277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6815","nonce":1124979277}]},"public_addr":"172.21.15.92:6811/1124979277","cluster_addr":"172.21.15.92:6813/1124979277","heartbeat_back_addr":"172.21.15.92:6817/1124979277","heartbeat_front_addr":"172.21.15.92:6815/1124979277","state":["exists","up"]},{"osd":3,"uuid":"4648d7ce-9b29-4583-9425-4ba8fd2b1903","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6827","nonce":1734115782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6829","nonce":1734115782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6833","nonce":1734115782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6831","nonce":1734115782}]},"public_addr":"172.21.15.92:6827/1734115782","cluster_addr":"172.21.15.92:6829/1734115782","heartbeat_back_addr":"172.21.15.92:6833/1734115782","heartbeat_front_addr":"172.21.15.92:6831/1734115782","state":["exists","up"]},{"osd":4,"uuid":"3bc9d7eb-c6fa-4fcf-a3ec-ade604e05aac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6819","nonce":3425937229}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6822","nonce":3425937229}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6830","nonce":3425937229}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6826","nonce":3425937229}]},"public_addr":"172.21.15.118:6819/3425937229","cluster_addr":"172.21.15.118:6822/3425937229","heartbeat_back_addr":"172.21.15.118:6830/3425937229","heartbeat_front_addr":"172.21.15.118:6826/3425937229","state":["exists","up"]},{"osd":5,"uuid":"dce9ba6d-59ed-46d1-84fc-a84b13de10c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6801","nonce":827455984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6803","nonce":827455984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6807","nonce":827455984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6805","nonce":827455984}]},"public_addr":"172.21.15.118:6801/827455984","cluster_addr":"172.21.15.118:6803/827455984","heartbeat_back_addr":"172.21.15.118:6807/827455984","heartbeat_front_addr":"172.21.15.118:6805/827455984","state":["exists","up"]},{"osd":6,"uuid":"82aec504-ae58-4709-8d30-46f2b12081b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6809","nonce":2946986141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6811","nonce":2946986141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6815","nonce":2946986141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6813","nonce":2946986141}]},"public_addr":"172.21.15.118:6809/2946986141","cluster_addr":"172.21.15.118:6811/2946986141","heartbeat_back_addr":"172.21.15.118:6815/2946986141","heartbeat_front_addr":"172.21.15.118:6813/2946986141","state":["exists","up"]},{"osd":7,"uuid":"4ed4c624-c88f-4d35-bdb1-bf209b5dffb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6817","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6818","nonce":48122059}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6821","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6823","nonce":48122059}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6829","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6831","nonce":48122059}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6825","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6827","nonce":48122059}]},"public_addr":"172.21.15.118:6818/48122059","cluster_addr":"172.21.15.118:6823/48122059","heartbeat_back_addr":"172.21.15.118:6831/48122059","heartbeat_front_addr":"172.21.15.118:6827/48122059","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.536664+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.368324+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.271201+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.342971+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.621832+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.387110+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.632946+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.416032+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:10:31.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:31.990+0000 7f179eeda640 1 -- 172.21.15.92:0/41857336 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f177c06ae40 msgr2=0x7f177c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:31.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:31.990+0000 7f179eeda640 1 --2- 172.21.15.92:0/41857336 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f177c06ae40 0x7f177c06d300 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f1788036000 tx=0x7f1788030000 comp rx=0 tx=0).stop 2023-11-24T06:10:31.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:31.990+0000 7f179eeda640 1 -- 172.21.15.92:0/41857336 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f179806c7c0 msgr2=0x7f179815c730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:31.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:31.990+0000 7f179eeda640 1 --2- 172.21.15.92:0/41857336 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f179806c7c0 0x7f179815c730 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f179400c7f0 tx=0x7f179400cbf0 comp rx=0 tx=0).stop 2023-11-24T06:10:31.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:31.990+0000 7f179eeda640 1 -- 172.21.15.92:0/41857336 shutdown_connections 2023-11-24T06:10:31.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:31.990+0000 7f179eeda640 1 --2- 172.21.15.92:0/41857336 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f177c06ae40 0x7f177c06d300 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:31.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:31.990+0000 7f179eeda640 1 --2- 172.21.15.92:0/41857336 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1798071d30 0x7f17981e8540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:31.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:31.990+0000 7f179eeda640 1 --2- 172.21.15.92:0/41857336 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1798071410 0x7f17981e38e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:31.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:31.990+0000 7f179eeda640 1 --2- 172.21.15.92:0/41857336 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f179806c7c0 0x7f179815c730 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:31.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:31.990+0000 7f179eeda640 1 -- 172.21.15.92:0/41857336 >> 172.21.15.92:0/41857336 conn(0x7f179806a800 msgr2=0x7f17980b1230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:31.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:31.990+0000 7f179eeda640 1 -- 172.21.15.92:0/41857336 shutdown_connections 2023-11-24T06:10:31.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:31.990+0000 7f179eeda640 1 -- 172.21.15.92:0/41857336 wait complete. 2023-11-24T06:10:32.013 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2023-11-24T06:10:32.013 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2023-11-24T06:10:32.014 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2023-11-24T06:10:32.014 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2023-11-24T06:10:32.015 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2023-11-24T06:10:32.015 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2023-11-24T06:10:32.015 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2023-11-24T06:10:32.016 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2023-11-24T06:10:32.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.307+0000 7f84811ac640 1 Processor -- start 2023-11-24T06:10:32.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.307+0000 7f84811ac640 1 -- start start 2023-11-24T06:10:32.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.308+0000 7f7ccd16c640 1 Processor -- start 2023-11-24T06:10:32.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.309+0000 7f84811ac640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f84740e76e0 0x7f847404ca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:32.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.309+0000 7f84811ac640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8474058990 0x7f8474053500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:32.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.308+0000 7f7ccd16c640 1 -- start start 2023-11-24T06:10:32.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.310+0000 7f7ccd16c640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7cc00ebb20 0x7f7cc00552d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:32.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.310+0000 7f7ccd16c640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f7cc0006170 0x7f7cc0006600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:32.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.310+0000 7f847bfff640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f84740e76e0 0x7f847404ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:32.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.310+0000 7f847bfff640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f84740e76e0 0x7f847404ca00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.92:58204/0 (socket says 172.21.15.92:58204) 2023-11-24T06:10:32.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.310+0000 7f847bfff640 1 -- 172.21.15.92:0/4146867646 learned_addr learned my addr 172.21.15.92:0/4146867646 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:10:32.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.310+0000 7f7ccd16c640 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7cc0057e00 con 0x7f7cc00eb030 2023-11-24T06:10:32.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.310+0000 7f7ccd16c640 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7cc0057970 con 0x7f7cc0006170 2023-11-24T06:10:32.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.310+0000 7f7ccd16c640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cc00387b0 con 0x7f7cc00ebb20 2023-11-24T06:10:32.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.311+0000 7f847b7fe640 1 --2- 172.21.15.92:0/4146867646 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8474058990 0x7f8474053500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:32.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.311+0000 7f84811ac640 1 -- 172.21.15.92:0/4146867646 --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8474057d60 con 0x7f8474056a80 2023-11-24T06:10:32.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.311+0000 7f7cc77fe640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7cc00ebb20 0x7f7cc00552d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:32.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.311+0000 7f7cc77fe640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7cc00ebb20 0x7f7cc00552d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3301/0 says I am v2:172.21.15.92:39454/0 (socket says 172.21.15.92:39454) 2023-11-24T06:10:32.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.311+0000 7f7cc77fe640 1 -- 172.21.15.92:0/2315547990 learned_addr learned my addr 172.21.15.92:0/2315547990 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:10:32.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.311+0000 7f84811ac640 1 -- 172.21.15.92:0/4146867646 --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8474006230 con 0x7f84740e76e0 2023-11-24T06:10:32.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.311+0000 7f84811ac640 1 -- 172.21.15.92:0/4146867646 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8474006510 con 0x7f8474058990 2023-11-24T06:10:32.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.311+0000 7f7ccc96b640 1 --2- 172.21.15.92:0/2315547990 >> v2:172.21.15.118:3300/0 conn(0x7f7cc0006170 0x7f7cc0006600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:32.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.312+0000 7f7ccc96b640 1 -- 172.21.15.92:0/2315547990 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7cc00ebb20 msgr2=0x7f7cc00552d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:32.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.312+0000 7f7ccc96b640 1 --2- 172.21.15.92:0/2315547990 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7cc00ebb20 0x7f7cc00552d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:32.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.312+0000 7f7ccc96b640 1 -- 172.21.15.92:0/2315547990 >> v1:172.21.15.118:6789/0 conn(0x7f7cc00eb030 legacy=0x7f7cc00eb410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:32.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.312+0000 7f7ccc96b640 1 -- 172.21.15.92:0/2315547990 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cc00ebfa0 con 0x7f7cc0006170 2023-11-24T06:10:32.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.313+0000 7f847affd640 1 -- 172.21.15.92:0/4146867646 <== mon.1 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2244547142 0 0) 0x7f8474057d60 con 0x7f8474056a80 2023-11-24T06:10:32.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.313+0000 7f847bfff640 1 -- 172.21.15.92:0/4146867646 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8474058990 msgr2=0x7f8474053500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:32.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.313+0000 7f847bfff640 1 --2- 172.21.15.92:0/4146867646 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8474058990 0x7f8474053500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:32.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.313+0000 7f847bfff640 1 -- 172.21.15.92:0/4146867646 >> v1:172.21.15.118:6789/0 conn(0x7f8474056a80 legacy=0x7f8474006ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:32.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.313+0000 7f847bfff640 1 -- 172.21.15.92:0/4146867646 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84740067f0 con 0x7f84740e76e0 2023-11-24T06:10:32.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.313+0000 7f7ccc96b640 1 --2- 172.21.15.92:0/2315547990 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7cc0006170 0x7f7cc0006600 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f7cc804fab0 tx=0x7f7cc80a5df0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1fa235fba683f0f2 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:32.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.314+0000 7f7cc6ffd640 1 -- 172.21.15.92:0/2315547990 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cc80a6ba0 con 0x7f7cc0006170 2023-11-24T06:10:32.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.315+0000 7f847bfff640 1 --2- 172.21.15.92:0/4146867646 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f84740e76e0 0x7f847404ca00 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f8468009870 tx=0x7f846802b910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=40100fc3bcb6e152 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:32.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.315+0000 7f7cc6ffd640 1 -- 172.21.15.92:0/2315547990 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cc80a6d40 con 0x7f7cc0006170 2023-11-24T06:10:32.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.315+0000 7f7cc6ffd640 1 -- 172.21.15.92:0/2315547990 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cc80ae640 con 0x7f7cc0006170 2023-11-24T06:10:32.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.316+0000 7f7ccd16c640 1 -- 172.21.15.92:0/2315547990 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7cc0006170 msgr2=0x7f7cc0006600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:32.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.316+0000 7f7ccd16c640 1 --2- 172.21.15.92:0/2315547990 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7cc0006170 0x7f7cc0006600 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f7cc804fab0 tx=0x7f7cc80a5df0 comp rx=0 tx=0).stop 2023-11-24T06:10:32.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.316+0000 7f847affd640 1 -- 172.21.15.92:0/4146867646 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8468031870 con 0x7f84740e76e0 2023-11-24T06:10:32.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.316+0000 7f7ccd16c640 1 -- 172.21.15.92:0/2315547990 shutdown_connections 2023-11-24T06:10:32.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.316+0000 7f7ccd16c640 1 --2- 172.21.15.92:0/2315547990 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7cc0006170 0x7f7cc0006600 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:32.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.316+0000 7f7ccd16c640 1 --2- 172.21.15.92:0/2315547990 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7cc00ebb20 0x7f7cc00552d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:32.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.317+0000 7f847affd640 1 -- 172.21.15.92:0/4146867646 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84680319d0 con 0x7f84740e76e0 2023-11-24T06:10:32.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.316+0000 7f7ccd16c640 1 -- 172.21.15.92:0/2315547990 >> 172.21.15.92:0/2315547990 conn(0x7f7cc00559b0 msgr2=0x7f7cc0055dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:32.323 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.317+0000 7f847affd640 1 -- 172.21.15.92:0/4146867646 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8468031d60 con 0x7f84740e76e0 2023-11-24T06:10:32.323 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.317+0000 7f7ccd16c640 1 -- 172.21.15.92:0/2315547990 shutdown_connections 2023-11-24T06:10:32.323 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.318+0000 7f7ccd16c640 1 -- 172.21.15.92:0/2315547990 wait complete. 2023-11-24T06:10:32.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.319+0000 7f84811ac640 1 -- 172.21.15.92:0/4146867646 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f84740e76e0 msgr2=0x7f847404ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:32.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.319+0000 7f84811ac640 1 --2- 172.21.15.92:0/4146867646 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f84740e76e0 0x7f847404ca00 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f8468009870 tx=0x7f846802b910 comp rx=0 tx=0).stop 2023-11-24T06:10:32.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.319+0000 7f84811ac640 1 -- 172.21.15.92:0/4146867646 shutdown_connections 2023-11-24T06:10:32.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.319+0000 7f84811ac640 1 --2- 172.21.15.92:0/4146867646 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8474058990 0x7f8474053500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:32.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.319+0000 7f84811ac640 1 --2- 172.21.15.92:0/4146867646 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f84740e76e0 0x7f847404ca00 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:32.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.319+0000 7f84811ac640 1 -- 172.21.15.92:0/4146867646 >> 172.21.15.92:0/4146867646 conn(0x7f8474058eb0 msgr2=0x7f8474059290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:32.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.319+0000 7f84811ac640 1 -- 172.21.15.92:0/4146867646 shutdown_connections 2023-11-24T06:10:32.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.320+0000 7f84811ac640 1 -- 172.21.15.92:0/4146867646 wait complete. 2023-11-24T06:10:32.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.320+0000 7f37503b0640 1 Processor -- start 2023-11-24T06:10:32.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.322+0000 7f7ccd16c640 1 Processor -- start 2023-11-24T06:10:32.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.321+0000 7f37503b0640 1 -- start start 2023-11-24T06:10:32.326 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.322+0000 7f84811ac640 1 Processor -- start 2023-11-24T06:10:32.326 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.322+0000 7f84811ac640 1 -- start start 2023-11-24T06:10:32.326 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.322+0000 7f7ccd16c640 1 -- start start 2023-11-24T06:10:32.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.323+0000 7f37503b0640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f37480ce8f0 0x7f37480ced30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:32.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.323+0000 7f37503b0640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37480d0de0 con 0x7f37480b2a00 2023-11-24T06:10:32.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.323+0000 7f37503b0640 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37480d10c0 con 0x7f3748160f80 2023-11-24T06:10:32.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.323+0000 7f37503b0640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37480d13a0 con 0x7f37480ce8f0 2023-11-24T06:10:32.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.324+0000 7f7ccd16c640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7cc0006170 0x7f7cc0052700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:32.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.324+0000 7f84811ac640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8474056a80 0x7f84740e3c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:32.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.324+0000 7f7ccd16c640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7cc00eb030 0x7f7cc004ff70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:32.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.324+0000 7f7ccd16c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7cc00ebb20 0x7f7cc00504b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:32.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.324+0000 7f7ccd16c640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cc0052c40 con 0x7f7cc00ebb20 2023-11-24T06:10:32.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.324+0000 7f7ccd16c640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cc0171400 con 0x7f7cc0006170 2023-11-24T06:10:32.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.324+0000 7f7ccd16c640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cc01716e0 con 0x7f7cc00eb030 2023-11-24T06:10:32.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.324+0000 7f374e14c640 1 --1- >> v1:172.21.15.118:6789/0 conn(0x7f3748160f80 0x7f3748161360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.118:6789/0 says I am v1:172.21.15.92:56804/0 (socket says 172.21.15.92:56804) 2023-11-24T06:10:32.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.324+0000 7f374d94b640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f37480ce8f0 0x7f37480ced30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:32.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.324+0000 7f84811ac640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8474058990 0x7f84740e1340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:32.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.324+0000 7f374e14c640 1 -- 172.21.15.92:0/3104945923 learned_addr learned my addr 172.21.15.92:0/3104945923 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-24T06:10:32.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.325+0000 7f84811ac640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f84740e76e0 0x7f847404ff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:32.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.325+0000 7f84811ac640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f847400a1b0 con 0x7f84740e76e0 2023-11-24T06:10:32.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.325+0000 7f84811ac640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8474050d40 con 0x7f8474058990 2023-11-24T06:10:32.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.325+0000 7f84811ac640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84740e4310 con 0x7f8474056a80 2023-11-24T06:10:32.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.324+0000 7f374e94d640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f37480b2a00 0x7f37480b2e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:55780/0 (socket says 172.21.15.92:55780) 2023-11-24T06:10:32.331 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.325+0000 7f7cc77fe640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7cc00eb030 0x7f7cc004ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:32.331 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.325+0000 7f7cc7fff640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7cc0006170 0x7f7cc0052700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:32.331 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.325+0000 7f7cc77fe640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7cc00eb030 0x7f7cc004ff70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3301/0 says I am v2:172.21.15.92:39470/0 (socket says 172.21.15.92:39470) 2023-11-24T06:10:32.331 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.325+0000 7f7cc77fe640 1 -- 172.21.15.92:0/3667755503 learned_addr learned my addr 172.21.15.92:0/3667755503 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:10:32.331 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.325+0000 7f7ccc96b640 1 --2- 172.21.15.92:0/3667755503 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7cc00ebb20 0x7f7cc00504b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:32.331 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.325+0000 7f847bfff640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8474056a80 0x7f84740e3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:32.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.325+0000 7f860ab0f640 1 Processor -- start 2023-11-24T06:10:32.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.325+0000 7f847bfff640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8474056a80 0x7f84740e3c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3301/0 says I am v2:172.21.15.92:39472/0 (socket says 172.21.15.92:39472) 2023-11-24T06:10:32.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.325+0000 7f847bfff640 1 -- 172.21.15.92:0/3101676020 learned_addr learned my addr 172.21.15.92:0/3101676020 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:10:32.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.326+0000 7f847b7fe640 1 --2- 172.21.15.92:0/3101676020 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8474058990 0x7f84740e1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:32.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.326+0000 7f84809ab640 1 --2- 172.21.15.92:0/3101676020 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f84740e76e0 0x7f847404ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:32.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.326+0000 7f860ab0f640 1 -- start start 2023-11-24T06:10:32.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.326+0000 7f7cc77fe640 1 -- 172.21.15.92:0/3667755503 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7cc0006170 msgr2=0x7f7cc0052700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:32.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.326+0000 7f7cc77fe640 1 --2- 172.21.15.92:0/3667755503 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7cc0006170 0x7f7cc0052700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:32.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.326+0000 7f7cc77fe640 1 -- 172.21.15.92:0/3667755503 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7cc00ebb20 msgr2=0x7f7cc00504b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:32.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.326+0000 7f7cc77fe640 1 --2- 172.21.15.92:0/3667755503 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7cc00ebb20 0x7f7cc00504b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:32.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.326+0000 7f7cc77fe640 1 -- 172.21.15.92:0/3667755503 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cc804edc0 con 0x7f7cc00eb030 2023-11-24T06:10:32.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.326+0000 7f374d94b640 1 -- 172.21.15.92:0/3104945923 >> v1:172.21.15.118:6789/0 conn(0x7f3748160f80 legacy=0x7f3748161360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:32.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.326+0000 7f374d94b640 1 -- 172.21.15.92:0/3104945923 >> v1:172.21.15.92:6789/0 conn(0x7f37480b2a00 legacy=0x7f37480b2e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:32.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.326+0000 7f374d94b640 1 -- 172.21.15.92:0/3104945923 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37480d1680 con 0x7f37480ce8f0 2023-11-24T06:10:32.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.327+0000 7f847b7fe640 1 -- 172.21.15.92:0/3101676020 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8474056a80 msgr2=0x7f84740e3c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:32.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.327+0000 7f847b7fe640 1 --2- 172.21.15.92:0/3101676020 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8474056a80 0x7f84740e3c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:32.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.327+0000 7f847b7fe640 1 -- 172.21.15.92:0/3101676020 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f84740e76e0 msgr2=0x7f847404ff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:32.335 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.327+0000 7f7cc77fe640 1 --2- 172.21.15.92:0/3667755503 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7cc00eb030 0x7f7cc004ff70 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f7cb4009fd0 tx=0x7f7cb400ed40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:32.335 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.327+0000 7f847b7fe640 1 --2- 172.21.15.92:0/3101676020 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f84740e76e0 0x7f847404ff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:32.335 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.327+0000 7f847b7fe640 1 -- 172.21.15.92:0/3101676020 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84680094d0 con 0x7f8474058990 2023-11-24T06:10:32.335 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.327+0000 7f84809ab640 1 --2- 172.21.15.92:0/3101676020 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f84740e76e0 0x7f847404ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:10:32.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.328+0000 7f374d94b640 1 --2- 172.21.15.92:0/3104945923 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f37480ce8f0 0x7f37480ced30 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f37440097a0 tx=0x7f374402b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=818bdd8884cd3362 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:32.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.328+0000 7f7cc4ff9640 1 -- 172.21.15.92:0/3667755503 <== mon.2 v2:172.21.15.118:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cb400c940 con 0x7f7cc00eb030 2023-11-24T06:10:32.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.328+0000 7f7cc4ff9640 1 -- 172.21.15.92:0/3667755503 <== mon.2 v2:172.21.15.118:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cb400caa0 con 0x7f7cc00eb030 2023-11-24T06:10:32.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.328+0000 7f7cc4ff9640 1 -- 172.21.15.92:0/3667755503 <== mon.2 v2:172.21.15.118:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cb400ce30 con 0x7f7cc00eb030 2023-11-24T06:10:32.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.328+0000 7f847b7fe640 1 --2- 172.21.15.92:0/3101676020 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8474058990 0x7f84740e1340 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f847000e820 tx=0x7f847000ec20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:32.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.329+0000 7f8478ff9640 1 -- 172.21.15.92:0/3101676020 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f847000ee70 con 0x7f8474058990 2023-11-24T06:10:32.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.328+0000 7f7ccd16c640 1 -- 172.21.15.92:0/3667755503 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7cc01719c0 con 0x7f7cc00eb030 2023-11-24T06:10:32.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.328+0000 7f7ccd16c640 1 -- 172.21.15.92:0/3667755503 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7cc0171f80 con 0x7f7cc00eb030 2023-11-24T06:10:32.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.329+0000 7f84811ac640 1 -- 172.21.15.92:0/3101676020 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84740e45f0 con 0x7f8474058990 2023-11-24T06:10:32.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.329+0000 7f8478ff9640 1 -- 172.21.15.92:0/3101676020 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f847000c940 con 0x7f8474058990 2023-11-24T06:10:32.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.329+0000 7f84811ac640 1 -- 172.21.15.92:0/3101676020 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84740e4bb0 con 0x7f8474058990 2023-11-24T06:10:32.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.329+0000 7f8478ff9640 1 -- 172.21.15.92:0/3101676020 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f847000ccf0 con 0x7f8474058990 2023-11-24T06:10:32.338 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.329+0000 7f7ccd16c640 1 -- 172.21.15.92:0/3667755503 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f7cc0057320 con 0x7f7cc00eb030 2023-11-24T06:10:32.338 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.329+0000 7f374d14a640 1 -- 172.21.15.92:0/3104945923 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3744030610 con 0x7f37480ce8f0 2023-11-24T06:10:32.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.329+0000 7f374d14a640 1 -- 172.21.15.92:0/3104945923 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3744030770 con 0x7f37480ce8f0 2023-11-24T06:10:32.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.329+0000 7f374d14a640 1 -- 172.21.15.92:0/3104945923 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3744030b00 con 0x7f37480ce8f0 2023-11-24T06:10:32.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.330+0000 7f37503b0640 1 -- 172.21.15.92:0/3104945923 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f37480ce8f0 msgr2=0x7f37480ced30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:32.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.330+0000 7f37503b0640 1 --2- 172.21.15.92:0/3104945923 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f37480ce8f0 0x7f37480ced30 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f37440097a0 tx=0x7f374402b960 comp rx=0 tx=0).stop 2023-11-24T06:10:32.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.330+0000 7f37503b0640 1 -- 172.21.15.92:0/3104945923 shutdown_connections 2023-11-24T06:10:32.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.330+0000 7f37503b0640 1 --2- 172.21.15.92:0/3104945923 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f37480ce8f0 0x7f37480ced30 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:32.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.330+0000 7f37503b0640 1 -- 172.21.15.92:0/3104945923 >> 172.21.15.92:0/3104945923 conn(0x7f374809c550 msgr2=0x7f37480d3210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:32.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.330+0000 7f37503b0640 1 -- 172.21.15.92:0/3104945923 shutdown_connections 2023-11-24T06:10:32.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.331+0000 7f37503b0640 1 -- 172.21.15.92:0/3104945923 wait complete. 2023-11-24T06:10:32.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.331+0000 7f84667fc640 1 -- 172.21.15.92:0/3101676020 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f845c000fc0 con 0x7f8474058990 2023-11-24T06:10:32.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.333+0000 7f37503b0640 1 Processor -- start 2023-11-24T06:10:32.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.333+0000 7f37503b0640 1 -- start start 2023-11-24T06:10:32.341 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.333+0000 7f860ab0f640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f86040cfad0 0x7f86041539c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:32.341 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.333+0000 7f860ab0f640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8604159e20 con 0x7f86041598d0 2023-11-24T06:10:32.341 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.333+0000 7f860ab0f640 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f860415a100 con 0x7f860406e3b0 2023-11-24T06:10:32.341 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.333+0000 7f860ab0f640 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86040b31c0 con 0x7f86040cfad0 2023-11-24T06:10:32.341 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.334+0000 7f8603fff640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f86041598d0 0x7f86040cf3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:55786/0 (socket says 172.21.15.92:55786) 2023-11-24T06:10:32.341 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.334+0000 7f8603fff640 1 -- 172.21.15.92:0/3868495226 learned_addr learned my addr 172.21.15.92:0/3868495226 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-24T06:10:32.341 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.335+0000 7f86090ac640 1 --2- 172.21.15.92:0/3868495226 >> v2:172.21.15.118:3300/0 conn(0x7f86040cfad0 0x7f86041539c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:32.342 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.335+0000 7f37503b0640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f37480b2a00 0x7f374815d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:32.342 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.335+0000 7f37503b0640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3748160f80 0x7f37480b3f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:32.342 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.335+0000 7f86037fe640 1 -- 172.21.15.92:0/3868495226 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3262224866 0 0) 0x7f8604159e20 con 0x7f86041598d0 2023-11-24T06:10:32.342 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.335+0000 7f86037fe640 1 -- 172.21.15.92:0/3868495226 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f85ec003520 con 0x7f86041598d0 2023-11-24T06:10:32.343 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.335+0000 7f8478ff9640 1 -- 172.21.15.92:0/3101676020 <== mon.1 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f8470005990 con 0x7f8474058990 2023-11-24T06:10:32.343 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.335+0000 7f37503b0640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f37480b4460 0x7f37481fb180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:32.343 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.335+0000 7f37503b0640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37480b48f0 con 0x7f3748160f80 2023-11-24T06:10:32.343 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.335+0000 7f86090ac640 1 -- 172.21.15.92:0/3868495226 >> v1:172.21.15.118:6789/0 conn(0x7f860406e3b0 legacy=0x7f86040b2ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:32.343 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.335+0000 7f86090ac640 1 -- 172.21.15.92:0/3868495226 >> v1:172.21.15.92:6789/0 conn(0x7f86041598d0 legacy=0x7f86040cf3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:32.344 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.335+0000 7f86090ac640 1 -- 172.21.15.92:0/3868495226 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f860415a3e0 con 0x7f86040cfad0 2023-11-24T06:10:32.344 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.335+0000 7f37503b0640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37481499f0 con 0x7f37480b4460 2023-11-24T06:10:32.344 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.335+0000 7f37503b0640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f374814a770 con 0x7f37480b2a00 2023-11-24T06:10:32.344 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.336+0000 7f374e14c640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f37480b2a00 0x7f374815d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:32.344 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.335+0000 7f7cc4ff9640 1 -- 172.21.15.92:0/3667755503 <== mon.2 v2:172.21.15.118:3301/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f7cb4013470 con 0x7f7cc00eb030 2023-11-24T06:10:32.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.336+0000 7f374e94d640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f37480b4460 0x7f37481fb180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:32.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.336+0000 7f374e94d640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f37480b4460 0x7f37481fb180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.92:58230/0 (socket says 172.21.15.92:58230) 2023-11-24T06:10:32.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.336+0000 7f374e94d640 1 -- 172.21.15.92:0/1125600430 learned_addr learned my addr 172.21.15.92:0/1125600430 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:10:32.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.337+0000 7f374d94b640 1 --2- 172.21.15.92:0/1125600430 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3748160f80 0x7f37480b3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:32.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.337+0000 7f7cc4ff9640 1 --2- 172.21.15.92:0/3667755503 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f7cac06ae90 0x7f7cac06d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:32.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.337+0000 7f7cc4ff9640 1 -- 172.21.15.92:0/3667755503 <== mon.2 v2:172.21.15.118:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7cb40899d0 con 0x7f7cc00eb030 2023-11-24T06:10:32.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.337+0000 7f7cc4ff9640 1 --2- 172.21.15.92:0/3667755503 >> [v2:172.21.15.92:6818/2451239906,v1:172.21.15.92:6819/2451239906] conn(0x7f7cac0730b0 0x7f7cac0754d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:32.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.337+0000 7f7cc4ff9640 1 -- 172.21.15.92:0/3667755503 --> [v2:172.21.15.92:6818/2451239906,v1:172.21.15.92:6819/2451239906] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7cac075b80 con 0x7f7cac0730b0 2023-11-24T06:10:32.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.337+0000 7f7cc4ff9640 1 -- 172.21.15.92:0/3667755503 <== mon.2 v2:172.21.15.118:3301/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7cb4089e70 con 0x7f7cc00eb030 2023-11-24T06:10:32.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.337+0000 7f374d94b640 1 -- 172.21.15.92:0/1125600430 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f37480b2a00 msgr2=0x7f374815d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:32.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.337+0000 7f374d94b640 1 --2- 172.21.15.92:0/1125600430 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f37480b2a00 0x7f374815d6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:32.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.337+0000 7f374d94b640 1 -- 172.21.15.92:0/1125600430 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f37480b4460 msgr2=0x7f37481fb180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:32.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.337+0000 7f374d94b640 1 --2- 172.21.15.92:0/1125600430 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f37480b4460 0x7f37481fb180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:32.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.337+0000 7f374d94b640 1 -- 172.21.15.92:0/1125600430 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3744009400 con 0x7f3748160f80 2023-11-24T06:10:32.348 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.337+0000 7f8478ff9640 1 --2- 172.21.15.92:0/3101676020 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f846006ae40 0x7f846006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:32.348 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.338+0000 7f8478ff9640 1 -- 172.21.15.92:0/3101676020 <== mon.1 v2:172.21.15.118:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8470089820 con 0x7f8474058990 2023-11-24T06:10:32.348 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.338+0000 7f8478ff9640 1 --2- 172.21.15.92:0/3101676020 >> [v2:172.21.15.92:6802/20560714,v1:172.21.15.92:6803/20560714] conn(0x7f8460072fe0 0x7f8460075400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:32.348 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.338+0000 7f8478ff9640 1 -- 172.21.15.92:0/3101676020 --> [v2:172.21.15.92:6802/20560714,v1:172.21.15.92:6803/20560714] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8460075ab0 con 0x7f8460072fe0 2023-11-24T06:10:32.348 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.338+0000 7f8478ff9640 1 -- 172.21.15.92:0/3101676020 <== mon.1 v2:172.21.15.118:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8470089c70 con 0x7f8474058990 2023-11-24T06:10:32.348 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.338+0000 7f847bfff640 1 --2- 172.21.15.92:0/3101676020 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f846006ae40 0x7f846006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:32.349 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.338+0000 7f374d94b640 1 --2- 172.21.15.92:0/1125600430 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3748160f80 0x7f37480b3f20 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f3744033000 tx=0x7f3744004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:32.349 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.338+0000 7f373effd640 1 -- 172.21.15.92:0/1125600430 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f374403e040 con 0x7f3748160f80 2023-11-24T06:10:32.349 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.338+0000 7f373effd640 1 -- 172.21.15.92:0/1125600430 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3744039c30 con 0x7f3748160f80 2023-11-24T06:10:32.349 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.338+0000 7f373effd640 1 -- 172.21.15.92:0/1125600430 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37440443f0 con 0x7f3748160f80 2023-11-24T06:10:32.349 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.338+0000 7f37503b0640 1 -- 172.21.15.92:0/1125600430 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f374815dca0 con 0x7f3748160f80 2023-11-24T06:10:32.349 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.338+0000 7f7cc7fff640 1 --2- 172.21.15.92:0/3667755503 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f7cac06ae90 0x7f7cac06d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:32.349 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.339+0000 7f7ccc96b640 1 --2- 172.21.15.92:0/3667755503 >> [v2:172.21.15.92:6818/2451239906,v1:172.21.15.92:6819/2451239906] conn(0x7f7cac0730b0 0x7f7cac0754d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:32.350 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.340+0000 7f86090ac640 1 --2- 172.21.15.92:0/3868495226 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f86040cfad0 0x7f86041539c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f85fc009d90 tx=0x7f85fc02bf40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3b638d581c1be54d server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:32.350 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.339+0000 7f84809ab640 1 --2- 172.21.15.92:0/3101676020 >> [v2:172.21.15.92:6802/20560714,v1:172.21.15.92:6803/20560714] conn(0x7f8460072fe0 0x7f8460075400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:32.350 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.340+0000 7f7ccc96b640 1 --2- 172.21.15.92:0/3667755503 >> [v2:172.21.15.92:6818/2451239906,v1:172.21.15.92:6819/2451239906] conn(0x7f7cac0730b0 0x7f7cac0754d0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:32.350 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.340+0000 7f86037fe640 1 -- 172.21.15.92:0/3868495226 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85fc02ccf0 con 0x7f86040cfad0 2023-11-24T06:10:32.351 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.340+0000 7f7cc7fff640 1 --2- 172.21.15.92:0/3667755503 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f7cac06ae90 0x7f7cac06d350 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f7cbc000f60 tx=0x7f7cbc009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:32.351 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.340+0000 7f847bfff640 1 --2- 172.21.15.92:0/3101676020 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f846006ae40 0x7f846006d300 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f846802b950 tx=0x7f846803c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:32.351 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.340+0000 7f86037fe640 1 -- 172.21.15.92:0/3868495226 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85fc02ce50 con 0x7f86040cfad0 2023-11-24T06:10:32.352 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.340+0000 7f86037fe640 1 -- 172.21.15.92:0/3868495226 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85fc034640 con 0x7f86040cfad0 2023-11-24T06:10:32.352 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.341+0000 7f860ab0f640 1 -- 172.21.15.92:0/3868495226 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f86040cfad0 msgr2=0x7f86041539c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:32.352 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.341+0000 7f860ab0f640 1 --2- 172.21.15.92:0/3868495226 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f86040cfad0 0x7f86041539c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f85fc009d90 tx=0x7f85fc02bf40 comp rx=0 tx=0).stop 2023-11-24T06:10:32.352 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.341+0000 7f84809ab640 1 --2- 172.21.15.92:0/3101676020 >> [v2:172.21.15.92:6802/20560714,v1:172.21.15.92:6803/20560714] conn(0x7f8460072fe0 0x7f8460075400 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:32.352 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.341+0000 7f37503b0640 1 -- 172.21.15.92:0/1125600430 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3748149c90 con 0x7f3748160f80 2023-11-24T06:10:32.352 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.342+0000 7f7cc4ff9640 1 -- 172.21.15.92:0/3667755503 <== osd.1 v2:172.21.15.92:6818/2451239906 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f7cac075b80 con 0x7f7cac0730b0 2023-11-24T06:10:32.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.342+0000 7f860ab0f640 1 -- 172.21.15.92:0/3868495226 shutdown_connections 2023-11-24T06:10:32.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.342+0000 7f860ab0f640 1 --2- 172.21.15.92:0/3868495226 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f86040cfad0 0x7f86041539c0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:32.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.342+0000 7f860ab0f640 1 -- 172.21.15.92:0/3868495226 >> 172.21.15.92:0/3868495226 conn(0x7f860406a800 msgr2=0x7f860406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:32.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.342+0000 7f616a8e0640 1 Processor -- start 2023-11-24T06:10:32.354 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.343+0000 7f860ab0f640 1 -- 172.21.15.92:0/3868495226 shutdown_connections 2023-11-24T06:10:32.354 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.343+0000 7f860ab0f640 1 -- 172.21.15.92:0/3868495226 wait complete. 2023-11-24T06:10:32.354 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.343+0000 7f616a8e0640 1 -- start start 2023-11-24T06:10:32.355 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.343+0000 7f8478ff9640 1 -- 172.21.15.92:0/3101676020 <== osd.0 v2:172.21.15.92:6802/20560714 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f8460075ab0 con 0x7f8460072fe0 2023-11-24T06:10:32.355 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.343+0000 7f37503b0640 1 -- 172.21.15.92:0/1125600430 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f37480d1c40 con 0x7f3748160f80 2023-11-24T06:10:32.355 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.345+0000 7f616a8e0640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f61640ce8f0 0x7f61640ced30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:32.355 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.345+0000 7f616a8e0640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61640d0de0 con 0x7f6164160f80 2023-11-24T06:10:32.355 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.345+0000 7f616a8e0640 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61640d10c0 con 0x7f61640b2a00 2023-11-24T06:10:32.355 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.345+0000 7f616a8e0640 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61640d13a0 con 0x7f61640ce8f0 2023-11-24T06:10:32.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.345+0000 7fe215f97640 1 Processor -- start 2023-11-24T06:10:32.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.345+0000 7f860ab0f640 1 Processor -- start 2023-11-24T06:10:32.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.345+0000 7f6163fff640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f6164160f80 0x7f6164161360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:55792/0 (socket says 172.21.15.92:55792) 2023-11-24T06:10:32.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.345+0000 7f6163fff640 1 -- 172.21.15.92:0/592422164 learned_addr learned my addr 172.21.15.92:0/592422164 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-24T06:10:32.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.345+0000 7fe215f97640 1 -- start start 2023-11-24T06:10:32.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.346+0000 7f860ab0f640 1 -- start start 2023-11-24T06:10:32.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.346+0000 7f373effd640 1 -- 172.21.15.92:0/1125600430 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f374404d450 con 0x7f3748160f80 2023-11-24T06:10:32.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.346+0000 7f615bfff640 1 --2- 172.21.15.92:0/592422164 >> v2:172.21.15.118:3300/0 conn(0x7f61640ce8f0 0x7f61640ced30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:32.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.347+0000 7f61637fe640 1 -- 172.21.15.92:0/592422164 <== mon.1 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3522526830 0 0) 0x7f61640d10c0 con 0x7f61640b2a00 2023-11-24T06:10:32.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.347+0000 7fe215f97640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe210160f80 0x7fe210161360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:32.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.347+0000 7f61637fe640 1 -- 172.21.15.92:0/592422164 --> v1:172.21.15.118:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6144003520 con 0x7f61640b2a00 2023-11-24T06:10:32.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.347+0000 7f373effd640 1 --2- 172.21.15.92:0/1125600430 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f372c06af10 0x7f372c06d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:32.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.347+0000 7f61637fe640 1 -- 172.21.15.92:0/592422164 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2442741109 0 0) 0x7f61640d0de0 con 0x7f6164160f80 2023-11-24T06:10:32.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.347+0000 7f1e62c56640 1 Processor -- start 2023-11-24T06:10:32.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.347+0000 7f373effd640 1 -- 172.21.15.92:0/1125600430 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f37440ab7f0 con 0x7f3748160f80 2023-11-24T06:10:32.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.347+0000 7f61637fe640 1 -- 172.21.15.92:0/592422164 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f61640d10c0 con 0x7f6164160f80 2023-11-24T06:10:32.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.348+0000 7f860ab0f640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f860406e3b0 0x7f86041f7750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:32.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.348+0000 7f373effd640 1 --2- 172.21.15.92:0/1125600430 >> [v2:172.21.15.92:6810/1124979277,v1:172.21.15.92:6811/1124979277] conn(0x7f372c073130 0x7f372c075550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:32.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.348+0000 7f373effd640 1 -- 172.21.15.92:0/1125600430 --> [v2:172.21.15.92:6810/1124979277,v1:172.21.15.92:6811/1124979277] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f372c075c00 con 0x7f372c073130 2023-11-24T06:10:32.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.348+0000 7f373effd640 1 -- 172.21.15.92:0/1125600430 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f37440ade10 con 0x7f3748160f80 2023-11-24T06:10:32.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.348+0000 7f1e62c56640 1 -- start start 2023-11-24T06:10:32.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.347+0000 7fe215f97640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fe2100ced50 0x7fe2100d02f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:32.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.347+0000 7fe215f97640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2100cf1b0 con 0x7fe2100ce8f0 2023-11-24T06:10:32.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.347+0000 7fe215f97640 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2101618a0 con 0x7fe2100ced50 2023-11-24T06:10:32.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.348+0000 7f615bfff640 1 -- 172.21.15.92:0/592422164 >> v1:172.21.15.118:6789/0 conn(0x7f61640b2a00 legacy=0x7f61640b2e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:32.362 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.347+0000 7fe215f97640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2100d0de0 con 0x7fe210160f80 2023-11-24T06:10:32.362 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.348+0000 7f847e778640 1 Processor -- start 2023-11-24T06:10:32.362 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.348+0000 7f860ab0f640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f86040cfad0 0x7f860415c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:32.363 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.348+0000 7fe20f7fe640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe210160f80 0x7fe210161360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:32.363 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.348+0000 7fe20f7fe640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe210160f80 0x7fe210161360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3301/0 says I am v2:172.21.15.92:39496/0 (socket says 172.21.15.92:39496) 2023-11-24T06:10:32.363 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.348+0000 7f847e778640 1 -- start start 2023-11-24T06:10:32.363 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.348+0000 7fe20f7fe640 1 -- 172.21.15.92:0/667229210 learned_addr learned my addr 172.21.15.92:0/667229210 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:10:32.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.349+0000 7f860ab0f640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f86041598d0 0x7f86040b23d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:32.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.349+0000 7f860ab0f640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86041f7c90 con 0x7f860406e3b0 2023-11-24T06:10:32.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.349+0000 7f860ab0f640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8604149660 con 0x7f86040cfad0 2023-11-24T06:10:32.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.349+0000 7f860ab0f640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8604149940 con 0x7f86041598d0 2023-11-24T06:10:32.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.349+0000 7f86088ab640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f860406e3b0 0x7f86041f7750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:32.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.349+0000 7f86088ab640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f860406e3b0 0x7f86041f7750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47274/0 (socket says 172.21.15.92:47274) 2023-11-24T06:10:32.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.349+0000 7f86088ab640 1 -- 172.21.15.92:0/1255019544 learned_addr learned my addr 172.21.15.92:0/1255019544 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:10:32.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.349+0000 7f8603fff640 1 --2- 172.21.15.92:0/1255019544 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f86040cfad0 0x7f860415c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:32.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.348+0000 7f615bfff640 1 -- 172.21.15.92:0/592422164 >> v1:172.21.15.92:6789/0 conn(0x7f6164160f80 legacy=0x7f6164161360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:32.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.348+0000 7f615bfff640 1 -- 172.21.15.92:0/592422164 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61640d1680 con 0x7f61640ce8f0 2023-11-24T06:10:32.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.350+0000 7f374e14c640 1 --2- 172.21.15.92:0/1125600430 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f372c06af10 0x7f372c06d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:32.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.350+0000 7f86090ac640 1 --2- 172.21.15.92:0/1255019544 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f86041598d0 0x7f86040b23d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:32.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.350+0000 7fe20f7fe640 1 -- 172.21.15.92:0/667229210 >> v2:172.21.15.118:3300/0 conn(0x7fe2100ced50 msgr2=0x7fe2100d02f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:32.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.350+0000 7fe20f7fe640 1 --2- 172.21.15.92:0/667229210 >> v2:172.21.15.118:3300/0 conn(0x7fe2100ced50 0x7fe2100d02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:32.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.350+0000 7fe20f7fe640 1 -- 172.21.15.92:0/667229210 >> v1:172.21.15.92:6789/0 conn(0x7fe2100ce8f0 legacy=0x7fe2100cfbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:32.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.350+0000 7f374e94d640 1 --2- 172.21.15.92:0/1125600430 >> [v2:172.21.15.92:6810/1124979277,v1:172.21.15.92:6811/1124979277] conn(0x7f372c073130 0x7f372c075550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:32.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.350+0000 7f8603fff640 1 -- 172.21.15.92:0/1255019544 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f86041598d0 msgr2=0x7f86040b23d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:32.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.350+0000 7f8603fff640 1 --2- 172.21.15.92:0/1255019544 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f86041598d0 0x7f86040b23d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:32.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.350+0000 7f8603fff640 1 -- 172.21.15.92:0/1255019544 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f860406e3b0 msgr2=0x7f86041f7750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:32.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.350+0000 7f8603fff640 1 --2- 172.21.15.92:0/1255019544 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f860406e3b0 0x7f86041f7750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:32.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.350+0000 7f8603fff640 1 -- 172.21.15.92:0/1255019544 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85fc009a70 con 0x7f86040cfad0 2023-11-24T06:10:32.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.350+0000 7f1e62c56640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f1e5c153950 0x7f1e5c0cb0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:32.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.350+0000 7f1e62c56640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1e5c15ce90 0x7f1e5c15d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:32.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.350+0000 7f1e62c56640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e5c15d830 con 0x7f1e5c1598a0 2023-11-24T06:10:32.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.350+0000 7f1e62c56640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e5c153d30 con 0x7f1e5c153950 2023-11-24T06:10:32.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.350+0000 7f1e62c56640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e5c0b2b70 con 0x7f1e5c15ce90 2023-11-24T06:10:32.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.350+0000 7f847e778640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f847814b2c0 0x7f84780cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:32.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.350+0000 7f847e778640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f84780cf7d0 0x7f8478070960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:32.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.350+0000 7fe20f7fe640 1 -- 172.21.15.92:0/667229210 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2100d10c0 con 0x7fe210160f80 2023-11-24T06:10:32.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.350+0000 7f847e778640 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84780cfbf0 con 0x7f8478070ea0 2023-11-24T06:10:32.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.350+0000 7f847e778640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f847814b6a0 con 0x7f847814b2c0 2023-11-24T06:10:32.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.350+0000 7f847e778640 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8478071690 con 0x7f84780cf7d0 2023-11-24T06:10:32.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.351+0000 7f8603fff640 1 --2- 172.21.15.92:0/1255019544 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f86040cfad0 0x7f860415c1e0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f85f800c430 tx=0x7f85f800c8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:32.369 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.351+0000 7f1e5bfff640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1e5c15ce90 0x7f1e5c15d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:32.369 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.351+0000 7f1e611f3640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f1e5c1598a0 0x7f1e5c159d10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:55808/0 (socket says 172.21.15.92:55808) 2023-11-24T06:10:32.369 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.351+0000 7f1e611f3640 1 -- 172.21.15.92:0/3567978216 learned_addr learned my addr 172.21.15.92:0/3567978216 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-24T06:10:32.369 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.351+0000 7f1e609f2640 1 --2- 172.21.15.92:0/3567978216 >> v2:172.21.15.92:3300/0 conn(0x7f1e5c153950 0x7f1e5c0cb0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:32.369 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.350+0000 7f615bfff640 1 --2- 172.21.15.92:0/592422164 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f61640ce8f0 0x7f61640ced30 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f614c0097a0 tx=0x7f614c02b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bde9a6fa75ff43e7 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:32.370 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.351+0000 7f847cd15640 1 --1- >> v1:172.21.15.118:6789/0 conn(0x7f8478070ea0 0x7f84780b1ee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.118:6789/0 says I am v1:172.21.15.92:56816/0 (socket says 172.21.15.92:56816) 2023-11-24T06:10:32.370 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.351+0000 7f847cd15640 1 -- 172.21.15.92:0/1848913185 learned_addr learned my addr 172.21.15.92:0/1848913185 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-24T06:10:32.370 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.351+0000 7f86017fa640 1 -- 172.21.15.92:0/1255019544 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85f8005ce0 con 0x7f86040cfad0 2023-11-24T06:10:32.370 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.352+0000 7f86017fa640 1 -- 172.21.15.92:0/1255019544 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85f8005e40 con 0x7f86040cfad0 2023-11-24T06:10:32.370 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.352+0000 7f86017fa640 1 -- 172.21.15.92:0/1255019544 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85f80072d0 con 0x7f86040cfad0 2023-11-24T06:10:32.370 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.352+0000 7f860ab0f640 1 -- 172.21.15.92:0/1255019544 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8604149c20 con 0x7f86040cfad0 2023-11-24T06:10:32.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.352+0000 7f860ab0f640 1 -- 172.21.15.92:0/1255019544 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f860414a1e0 con 0x7f86040cfad0 2023-11-24T06:10:32.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.352+0000 7f1e609f2640 1 -- 172.21.15.92:0/3567978216 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1e5c15ce90 msgr2=0x7f1e5c15d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:32.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.352+0000 7f1e609f2640 1 --2- 172.21.15.92:0/3567978216 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1e5c15ce90 0x7f1e5c15d2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:32.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.352+0000 7f1e609f2640 1 -- 172.21.15.92:0/3567978216 >> v1:172.21.15.92:6789/0 conn(0x7f1e5c1598a0 legacy=0x7f1e5c159d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:32.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.352+0000 7f84777fe640 1 --2- 172.21.15.92:0/1848913185 >> v2:172.21.15.118:3300/0 conn(0x7f84780cf7d0 0x7f8478070960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:32.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.352+0000 7f374e94d640 1 --2- 172.21.15.92:0/1125600430 >> [v2:172.21.15.92:6810/1124979277,v1:172.21.15.92:6811/1124979277] conn(0x7f372c073130 0x7f372c075550 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:32.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.352+0000 7f8477fff640 1 --2- 172.21.15.92:0/1848913185 >> v2:172.21.15.92:3300/0 conn(0x7f847814b2c0 0x7f84780cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:32.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.352+0000 7f8476ffd640 1 -- 172.21.15.92:0/1848913185 <== mon.1 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2769175476 0 0) 0x7f84780cfbf0 con 0x7f8478070ea0 2023-11-24T06:10:32.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.353+0000 7f8476ffd640 1 -- 172.21.15.92:0/1848913185 --> v1:172.21.15.118:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8464003520 con 0x7f8478070ea0 2023-11-24T06:10:32.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.353+0000 7f374e14c640 1 --2- 172.21.15.92:0/1125600430 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f372c06af10 0x7f372c06d3d0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f3738005880 tx=0x7f3738009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:32.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.353+0000 7f8477fff640 1 -- 172.21.15.92:0/1848913185 >> v2:172.21.15.118:3300/0 conn(0x7f84780cf7d0 msgr2=0x7f8478070960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:32.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.353+0000 7f8477fff640 1 --2- 172.21.15.92:0/1848913185 >> v2:172.21.15.118:3300/0 conn(0x7f84780cf7d0 0x7f8478070960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:32.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.353+0000 7f8477fff640 1 -- 172.21.15.92:0/1848913185 >> v1:172.21.15.118:6789/0 conn(0x7f8478070ea0 legacy=0x7f84780b1ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:32.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.353+0000 7f8477fff640 1 -- 172.21.15.92:0/1848913185 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8478071970 con 0x7f847814b2c0 2023-11-24T06:10:32.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.353+0000 7fe20f7fe640 1 --2- 172.21.15.92:0/667229210 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe210160f80 0x7fe210161360 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fe204009870 tx=0x7fe20402b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a8e870f244ff3666 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:32.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.353+0000 7f61637fe640 1 -- 172.21.15.92:0/592422164 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f614c030610 con 0x7f61640ce8f0 2023-11-24T06:10:32.374 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.353+0000 7f61637fe640 1 -- 172.21.15.92:0/592422164 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f614c030770 con 0x7f61640ce8f0 2023-11-24T06:10:32.374 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.353+0000 7f61637fe640 1 -- 172.21.15.92:0/592422164 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f614c030b00 con 0x7f61640ce8f0 2023-11-24T06:10:32.374 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.352+0000 7f1e609f2640 1 -- 172.21.15.92:0/3567978216 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e5c0b2e50 con 0x7f1e5c153950 2023-11-24T06:10:32.374 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.354+0000 7fe20e7fc640 1 -- 172.21.15.92:0/667229210 <== mon.2 v2:172.21.15.118:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe204030840 con 0x7fe210160f80 2023-11-24T06:10:32.374 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.354+0000 7fe20e7fc640 1 -- 172.21.15.92:0/667229210 <== mon.2 v2:172.21.15.118:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2040309a0 con 0x7fe210160f80 2023-11-24T06:10:32.374 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.354+0000 7fe20e7fc640 1 -- 172.21.15.92:0/667229210 <== mon.2 v2:172.21.15.118:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe204030d30 con 0x7fe210160f80 2023-11-24T06:10:32.375 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.354+0000 7f373effd640 1 -- 172.21.15.92:0/1125600430 <== osd.2 v2:172.21.15.92:6810/1124979277 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f372c075c00 con 0x7f372c073130 2023-11-24T06:10:32.375 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.354+0000 7f616a8e0640 1 -- 172.21.15.92:0/592422164 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f61640ce8f0 msgr2=0x7f61640ced30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:32.375 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.354+0000 7f616a8e0640 1 --2- 172.21.15.92:0/592422164 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f61640ce8f0 0x7f61640ced30 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f614c0097a0 tx=0x7f614c02b960 comp rx=0 tx=0).stop 2023-11-24T06:10:32.375 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.354+0000 7f860ab0f640 1 -- 172.21.15.92:0/1255019544 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f85e4000fc0 con 0x7f86040cfad0 2023-11-24T06:10:32.376 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.355+0000 7f616a8e0640 1 -- 172.21.15.92:0/592422164 shutdown_connections 2023-11-24T06:10:32.376 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.355+0000 7f616a8e0640 1 --2- 172.21.15.92:0/592422164 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f61640ce8f0 0x7f61640ced30 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:32.376 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.355+0000 7f616a8e0640 1 -- 172.21.15.92:0/592422164 >> 172.21.15.92:0/592422164 conn(0x7f616409c550 msgr2=0x7f61640d3210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:32.376 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.355+0000 7f616a8e0640 1 -- 172.21.15.92:0/592422164 shutdown_connections 2023-11-24T06:10:32.376 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.355+0000 7f1e609f2640 1 --2- 172.21.15.92:0/3567978216 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1e5c153950 0x7f1e5c0cb0d0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f1e500097a0 tx=0x7f1e5002b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=840b79348a5939aa server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:32.376 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.355+0000 7f616a8e0640 1 -- 172.21.15.92:0/592422164 wait complete. 2023-11-24T06:10:32.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.355+0000 7fe215f97640 1 -- 172.21.15.92:0/667229210 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe210160f80 msgr2=0x7fe210161360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:32.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.355+0000 7fe215f97640 1 --2- 172.21.15.92:0/667229210 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe210160f80 0x7fe210161360 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fe204009870 tx=0x7fe20402b980 comp rx=0 tx=0).stop 2023-11-24T06:10:32.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.355+0000 7f8477fff640 1 --2- 172.21.15.92:0/1848913185 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f847814b2c0 0x7f84780cf290 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f84780cfea0 tx=0x7f846002bec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=398bfedd45e3c2db server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:32.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.355+0000 7fe215f97640 1 -- 172.21.15.92:0/667229210 shutdown_connections 2023-11-24T06:10:32.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.355+0000 7fe215f97640 1 --2- 172.21.15.92:0/667229210 >> v2:172.21.15.118:3300/0 conn(0x7fe2100ced50 0x7fe2100d02f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:32.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.355+0000 7fe215f97640 1 --2- 172.21.15.92:0/667229210 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe210160f80 0x7fe210161360 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:32.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.355+0000 7fe215f97640 1 -- 172.21.15.92:0/667229210 >> 172.21.15.92:0/667229210 conn(0x7fe21009c550 msgr2=0x7fe2100d3210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:32.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.355+0000 7fe215f97640 1 -- 172.21.15.92:0/667229210 shutdown_connections 2023-11-24T06:10:32.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.355+0000 7fe215f97640 1 -- 172.21.15.92:0/667229210 wait complete. 2023-11-24T06:10:32.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.355+0000 7f1e5b7fe640 1 -- 172.21.15.92:0/3567978216 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e5002d040 con 0x7f1e5c153950 2023-11-24T06:10:32.379 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.356+0000 7f1e5b7fe640 1 -- 172.21.15.92:0/3567978216 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e50031810 con 0x7f1e5c153950 2023-11-24T06:10:32.379 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.356+0000 7f1e5b7fe640 1 -- 172.21.15.92:0/3567978216 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e50031ba0 con 0x7f1e5c153950 2023-11-24T06:10:32.379 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.356+0000 7f8476ffd640 1 -- 172.21.15.92:0/1848913185 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f846002cc70 con 0x7f847814b2c0 2023-11-24T06:10:32.379 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.356+0000 7f8476ffd640 1 -- 172.21.15.92:0/1848913185 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f846002cdd0 con 0x7f847814b2c0 2023-11-24T06:10:32.379 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.356+0000 7f8476ffd640 1 -- 172.21.15.92:0/1848913185 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8460034660 con 0x7f847814b2c0 2023-11-24T06:10:32.379 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.357+0000 7f847e778640 1 -- 172.21.15.92:0/1848913185 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f847814b2c0 msgr2=0x7f84780cf290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:32.380 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.357+0000 7f847e778640 1 --2- 172.21.15.92:0/1848913185 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f847814b2c0 0x7f84780cf290 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f84780cfea0 tx=0x7f846002bec0 comp rx=0 tx=0).stop 2023-11-24T06:10:32.380 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.357+0000 7f1e62c56640 1 -- 172.21.15.92:0/3567978216 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1e5c153950 msgr2=0x7f1e5c0cb0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:32.380 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.357+0000 7f616a8e0640 1 Processor -- start 2023-11-24T06:10:32.380 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.357+0000 7f1e62c56640 1 --2- 172.21.15.92:0/3567978216 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1e5c153950 0x7f1e5c0cb0d0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f1e500097a0 tx=0x7f1e5002b8b0 comp rx=0 tx=0).stop 2023-11-24T06:10:32.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.357+0000 7f86017fa640 1 -- 172.21.15.92:0/1255019544 <== mon.1 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f85f8007430 con 0x7f86040cfad0 2023-11-24T06:10:32.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.358+0000 7f847e778640 1 -- 172.21.15.92:0/1848913185 shutdown_connections 2023-11-24T06:10:32.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.358+0000 7f847e778640 1 --2- 172.21.15.92:0/1848913185 >> v2:172.21.15.118:3300/0 conn(0x7f84780cf7d0 0x7f8478070960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:32.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.358+0000 7f847e778640 1 --2- 172.21.15.92:0/1848913185 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f847814b2c0 0x7f84780cf290 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:32.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.358+0000 7f847e778640 1 -- 172.21.15.92:0/1848913185 >> 172.21.15.92:0/1848913185 conn(0x7f847806a800 msgr2=0x7f847806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:32.382 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.358+0000 7f1e62c56640 1 -- 172.21.15.92:0/3567978216 shutdown_connections 2023-11-24T06:10:32.382 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.358+0000 7f1e62c56640 1 --2- 172.21.15.92:0/3567978216 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1e5c15ce90 0x7f1e5c15d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:32.382 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.358+0000 7f1e62c56640 1 --2- 172.21.15.92:0/3567978216 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1e5c153950 0x7f1e5c0cb0d0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:32.382 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.358+0000 7fe215f97640 1 Processor -- start 2023-11-24T06:10:32.382 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.358+0000 7f1e62c56640 1 -- 172.21.15.92:0/3567978216 >> 172.21.15.92:0/3567978216 conn(0x7f1e5c0b0be0 msgr2=0x7f1e5c0b0fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:32.383 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.359+0000 7f1e62c56640 1 -- 172.21.15.92:0/3567978216 shutdown_connections 2023-11-24T06:10:32.383 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.359+0000 7f847e778640 1 -- 172.21.15.92:0/1848913185 shutdown_connections 2023-11-24T06:10:32.383 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.357+0000 7f616a8e0640 1 -- start start 2023-11-24T06:10:32.383 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.359+0000 7f1e62c56640 1 -- 172.21.15.92:0/3567978216 wait complete. 2023-11-24T06:10:32.384 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.359+0000 7f847e778640 1 -- 172.21.15.92:0/1848913185 wait complete. 2023-11-24T06:10:32.384 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.358+0000 7f616a8e0640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f61640b2a00 0x7f61640b7e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:32.384 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.359+0000 7f616a8e0640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f61640ce8f0 0x7f61640b8390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:32.384 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.359+0000 7f616a8e0640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6164160f80 0x7f61640b88d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:32.384 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.359+0000 7f616a8e0640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61640c9c90 con 0x7f6164160f80 2023-11-24T06:10:32.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.359+0000 7f616a8e0640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6164160080 con 0x7f61640ce8f0 2023-11-24T06:10:32.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.359+0000 7f86017fa640 1 --2- 172.21.15.92:0/1255019544 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f85f006af40 0x7f85f006d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:32.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.359+0000 7f86017fa640 1 -- 172.21.15.92:0/1255019544 <== mon.1 v2:172.21.15.118:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f85f8088a80 con 0x7f86040cfad0 2023-11-24T06:10:32.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.359+0000 7f86017fa640 1 --2- 172.21.15.92:0/1255019544 >> [v2:172.21.15.92:6826/1734115782,v1:172.21.15.92:6827/1734115782] conn(0x7f85f00730e0 0x7f85f0075500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:32.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.359+0000 7f86017fa640 1 -- 172.21.15.92:0/1255019544 --> [v2:172.21.15.92:6826/1734115782,v1:172.21.15.92:6827/1734115782] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f85f0075bb0 con 0x7f85f00730e0 2023-11-24T06:10:32.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.359+0000 7f616a8e0640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f616415f2c0 con 0x7f61640b2a00 2023-11-24T06:10:32.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.359+0000 7f6163fff640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f61640b2a00 0x7f61640b7e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:32.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.359+0000 7f6163fff640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f61640b2a00 0x7f61640b7e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3301/0 says I am v2:172.21.15.92:39532/0 (socket says 172.21.15.92:39532) 2023-11-24T06:10:32.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.359+0000 7f6163fff640 1 -- 172.21.15.92:0/701523222 learned_addr learned my addr 172.21.15.92:0/701523222 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:10:32.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.360+0000 7f86017fa640 1 -- 172.21.15.92:0/1255019544 <== mon.1 v2:172.21.15.118:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f85f8053c50 con 0x7f86040cfad0 2023-11-24T06:10:32.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.360+0000 7f86088ab640 1 --2- 172.21.15.92:0/1255019544 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f85f006af40 0x7f85f006d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:32.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.360+0000 7f615bfff640 1 --2- 172.21.15.92:0/701523222 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f61640ce8f0 0x7f61640b8390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:32.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.360+0000 7f6163fff640 1 -- 172.21.15.92:0/701523222 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f61640ce8f0 msgr2=0x7f61640b8390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:32.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.360+0000 7f6163fff640 1 --2- 172.21.15.92:0/701523222 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f61640ce8f0 0x7f61640b8390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:32.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.360+0000 7f6163fff640 1 -- 172.21.15.92:0/701523222 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6164160f80 msgr2=0x7f61640b88d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:32.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.360+0000 7f6163fff640 1 --2- 172.21.15.92:0/701523222 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6164160f80 0x7f61640b88d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:32.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.360+0000 7f6163fff640 1 -- 172.21.15.92:0/701523222 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f614c009400 con 0x7f61640b2a00 2023-11-24T06:10:32.388 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.359+0000 7fe215f97640 1 -- start start 2023-11-24T06:10:32.388 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.361+0000 7fe215f97640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe2100ce8f0 0x7fe2100c8310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:32.388 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.361+0000 7f6163fff640 1 --2- 172.21.15.92:0/701523222 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f61640b2a00 0x7f61640b7e50 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f615400c410 tx=0x7f615400c8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:32.388 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.361+0000 7fe215f97640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe2100ced50 0x7fe21006bf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:32.389 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.361+0000 7f61617fa640 1 -- 172.21.15.92:0/701523222 <== mon.2 v2:172.21.15.118:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6154005ce0 con 0x7f61640b2a00 2023-11-24T06:10:32.389 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.361+0000 7f61617fa640 1 -- 172.21.15.92:0/701523222 <== mon.2 v2:172.21.15.118:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6154005e40 con 0x7f61640b2a00 2023-11-24T06:10:32.389 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.361+0000 7f61617fa640 1 -- 172.21.15.92:0/701523222 <== mon.2 v2:172.21.15.118:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61540072f0 con 0x7f61640b2a00 2023-11-24T06:10:32.389 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.361+0000 7fe215f97640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe210160f80 0x7fe2100c6fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:32.389 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.361+0000 7fe215f97640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe21006c480 con 0x7fe210160f80 2023-11-24T06:10:32.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.362+0000 7fe215f97640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2100cb490 con 0x7fe2100ced50 2023-11-24T06:10:32.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.362+0000 7fe20f7fe640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe2100ce8f0 0x7fe2100c8310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:32.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.361+0000 7f616a8e0640 1 -- 172.21.15.92:0/701523222 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f616415f5a0 con 0x7f61640b2a00 2023-11-24T06:10:32.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.362+0000 7f616a8e0640 1 -- 172.21.15.92:0/701523222 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f616415fb60 con 0x7f61640b2a00 2023-11-24T06:10:32.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.362+0000 7fe20effd640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe2100ced50 0x7fe21006bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:32.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.362+0000 7fe20f7fe640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe2100ce8f0 0x7fe2100c8310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3301/0 says I am v2:172.21.15.92:39538/0 (socket says 172.21.15.92:39538) 2023-11-24T06:10:32.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.361+0000 7f86090ac640 1 --2- 172.21.15.92:0/1255019544 >> [v2:172.21.15.92:6826/1734115782,v1:172.21.15.92:6827/1734115782] conn(0x7f85f00730e0 0x7f85f0075500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:32.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.362+0000 7f86090ac640 1 --2- 172.21.15.92:0/1255019544 >> [v2:172.21.15.92:6826/1734115782,v1:172.21.15.92:6827/1734115782] conn(0x7f85f00730e0 0x7f85f0075500 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:32.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.362+0000 7f86088ab640 1 --2- 172.21.15.92:0/1255019544 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f85f006af40 0x7f85f006d400 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f85f4007fd0 tx=0x7f85f4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:32.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.362+0000 7fe20f7fe640 1 -- 172.21.15.92:0/3101787087 learned_addr learned my addr 172.21.15.92:0/3101787087 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:10:32.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.363+0000 7f616a8e0640 1 -- 172.21.15.92:0/701523222 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f616409c9a0 con 0x7f61640b2a00 2023-11-24T06:10:32.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.363+0000 7fe215f97640 1 -- 172.21.15.92:0/3101787087 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2100cb770 con 0x7fe2100ce8f0 2023-11-24T06:10:32.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.363+0000 7f847e778640 1 Processor -- start 2023-11-24T06:10:32.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.363+0000 7fe20ffff640 1 --2- 172.21.15.92:0/3101787087 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe210160f80 0x7fe2100c6fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:32.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.362+0000 7f1e62c56640 1 Processor -- start 2023-11-24T06:10:32.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.362+0000 7f1e62c56640 1 -- start start 2023-11-24T06:10:32.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.363+0000 7f847e778640 1 -- start start 2023-11-24T06:10:32.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.364+0000 7fe20ffff640 1 -- 172.21.15.92:0/3101787087 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe2100ce8f0 msgr2=0x7fe2100c8310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:32.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.364+0000 7fe20ffff640 1 --2- 172.21.15.92:0/3101787087 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe2100ce8f0 0x7fe2100c8310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:32.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.364+0000 7fe20ffff640 1 -- 172.21.15.92:0/3101787087 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe2100ced50 msgr2=0x7fe21006bf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:32.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.364+0000 7fe20ffff640 1 --2- 172.21.15.92:0/3101787087 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe2100ced50 0x7fe21006bf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:32.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.364+0000 7fe20ffff640 1 -- 172.21.15.92:0/3101787087 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2040094d0 con 0x7fe210160f80 2023-11-24T06:10:32.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.364+0000 7f86017fa640 1 -- 172.21.15.92:0/1255019544 <== osd.3 v2:172.21.15.92:6826/1734115782 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f85f0075bb0 con 0x7f85f00730e0 2023-11-24T06:10:32.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.364+0000 7fe20ffff640 1 --2- 172.21.15.92:0/3101787087 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe210160f80 0x7fe2100c6fb0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fe20800be20 tx=0x7fe20800bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:32.395 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.364+0000 7f1e62c56640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1e5c153950 0x7f1e5c154880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:32.395 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.364+0000 7f1e62c56640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1e5c1598a0 0x7f1e5c13eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:32.395 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.364+0000 7f1e62c56640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1e5c15ce90 0x7f1e5c0cdeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:32.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.364+0000 7fe1f7fff640 1 -- 172.21.15.92:0/3101787087 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe20800ce70 con 0x7fe210160f80 2023-11-24T06:10:32.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.364+0000 7fe1f7fff640 1 -- 172.21.15.92:0/3101787087 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe208014440 con 0x7fe210160f80 2023-11-24T06:10:32.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.365+0000 7fe215f97640 1 -- 172.21.15.92:0/3101787087 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2100cba50 con 0x7fe210160f80 2023-11-24T06:10:32.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.365+0000 7fe215f97640 1 -- 172.21.15.92:0/3101787087 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2100cc010 con 0x7fe210160f80 2023-11-24T06:10:32.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.365+0000 7fe1f7fff640 1 -- 172.21.15.92:0/3101787087 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe208014810 con 0x7fe210160f80 2023-11-24T06:10:32.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.366+0000 7f847e778640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8478070ea0 0x7f8478158120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:32.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.366+0000 7f847e778640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f84780cf7d0 0x7f84781464b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:32.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.366+0000 7f847e778640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f847814b2c0 0x7f84780cd860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:32.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.366+0000 7f847e778640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8478158660 con 0x7f84780cf7d0 2023-11-24T06:10:32.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.366+0000 7f847e778640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84781469f0 con 0x7f8478070ea0 2023-11-24T06:10:32.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.366+0000 7f847e778640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8478146c90 con 0x7f847814b2c0 2023-11-24T06:10:32.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.366+0000 7f1e609f2640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1e5c153950 0x7f1e5c154880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:32.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.366+0000 7f1e609f2640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1e5c153950 0x7f1e5c154880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3301/0 says I am v2:172.21.15.92:39554/0 (socket says 172.21.15.92:39554) 2023-11-24T06:10:32.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.366+0000 7f1e62c56640 1 -- 172.21.15.92:0/2825822426 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e5c0cda90 con 0x7f1e5c1598a0 2023-11-24T06:10:32.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.366+0000 7f1e609f2640 1 -- 172.21.15.92:0/2825822426 learned_addr learned my addr 172.21.15.92:0/2825822426 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:10:32.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.366+0000 7f1e62c56640 1 -- 172.21.15.92:0/2825822426 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e5c0cce10 con 0x7f1e5c15ce90 2023-11-24T06:10:32.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.366+0000 7f1e62c56640 1 -- 172.21.15.92:0/2825822426 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e5c0cd0f0 con 0x7f1e5c153950 2023-11-24T06:10:32.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.366+0000 7f61617fa640 1 -- 172.21.15.92:0/701523222 <== mon.2 v2:172.21.15.118:3301/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6154007510 con 0x7f61640b2a00 2023-11-24T06:10:32.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.367+0000 7f1e609f2640 1 -- 172.21.15.92:0/2825822426 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1e5c15ce90 msgr2=0x7f1e5c0cdeb0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-11-24T06:10:32.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.367+0000 7f1e609f2640 1 --2- 172.21.15.92:0/2825822426 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1e5c15ce90 0x7f1e5c0cdeb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:32.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.367+0000 7f847cd15640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f847814b2c0 0x7f84780cd860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:32.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.367+0000 7f1e609f2640 1 -- 172.21.15.92:0/2825822426 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1e5c1598a0 msgr2=0x7f1e5c13eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:32.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.367+0000 7f1e609f2640 1 --2- 172.21.15.92:0/2825822426 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1e5c1598a0 0x7f1e5c13eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:32.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.367+0000 7f1e609f2640 1 -- 172.21.15.92:0/2825822426 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e50009400 con 0x7f1e5c153950 2023-11-24T06:10:32.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.367+0000 7f847cd15640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f847814b2c0 0x7f84780cd860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3301/0 says I am v2:172.21.15.92:39556/0 (socket says 172.21.15.92:39556) 2023-11-24T06:10:32.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.367+0000 7f847cd15640 1 -- 172.21.15.92:0/3378706504 learned_addr learned my addr 172.21.15.92:0/3378706504 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:10:32.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.368+0000 7f84777fe640 1 --2- 172.21.15.92:0/3378706504 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f84780cf7d0 0x7f84781464b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:32.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.368+0000 7fe215f97640 1 -- 172.21.15.92:0/3101787087 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fe1f0000fc0 con 0x7fe210160f80 2023-11-24T06:10:32.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.368+0000 7f8477fff640 1 --2- 172.21.15.92:0/3378706504 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8478070ea0 0x7f8478158120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:32.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.368+0000 7f847cd15640 1 -- 172.21.15.92:0/3378706504 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8478070ea0 msgr2=0x7f8478158120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:32.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.368+0000 7f61617fa640 1 --2- 172.21.15.92:0/701523222 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f614806ae40 0x7f614806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:32.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.368+0000 7f61617fa640 1 -- 172.21.15.92:0/701523222 <== mon.2 v2:172.21.15.118:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6154089550 con 0x7f61640b2a00 2023-11-24T06:10:32.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.368+0000 7f61617fa640 1 --2- 172.21.15.92:0/701523222 >> [v2:172.21.15.118:6800/827455984,v1:172.21.15.118:6801/827455984] conn(0x7f6148072fe0 0x7f6148075400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:32.406 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.368+0000 7f61617fa640 1 -- 172.21.15.92:0/701523222 --> [v2:172.21.15.118:6800/827455984,v1:172.21.15.118:6801/827455984] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6148075ab0 con 0x7f6148072fe0 2023-11-24T06:10:32.406 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.368+0000 7f61617fa640 1 -- 172.21.15.92:0/701523222 <== mon.2 v2:172.21.15.118:3301/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f61540898c0 con 0x7f61640b2a00 2023-11-24T06:10:32.406 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.369+0000 7f1e609f2640 1 --2- 172.21.15.92:0/2825822426 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1e5c153950 0x7f1e5c154880 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f1e500098c0 tx=0x7f1e500046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:32.406 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.369+0000 7f1e597fa640 1 -- 172.21.15.92:0/2825822426 <== mon.2 v2:172.21.15.118:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e50006e10 con 0x7f1e5c153950 2023-11-24T06:10:32.406 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.369+0000 7f1e597fa640 1 -- 172.21.15.92:0/2825822426 <== mon.2 v2:172.21.15.118:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e5003ad60 con 0x7f1e5c153950 2023-11-24T06:10:32.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.369+0000 7f1e597fa640 1 -- 172.21.15.92:0/2825822426 <== mon.2 v2:172.21.15.118:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e50044620 con 0x7f1e5c153950 2023-11-24T06:10:32.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.369+0000 7f1e62c56640 1 -- 172.21.15.92:0/2825822426 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e5c0cd3d0 con 0x7f1e5c153950 2023-11-24T06:10:32.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.370+0000 7f1e62c56640 1 -- 172.21.15.92:0/2825822426 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e5c13f410 con 0x7f1e5c153950 2023-11-24T06:10:32.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.369+0000 7f615bfff640 1 --2- 172.21.15.92:0/701523222 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f614806ae40 0x7f614806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:32.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.368+0000 7f847cd15640 1 --2- 172.21.15.92:0/3378706504 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8478070ea0 0x7f8478158120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:32.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.368+0000 7f847cd15640 1 -- 172.21.15.92:0/3378706504 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f84780cf7d0 msgr2=0x7f84781464b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:32.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.368+0000 7f847cd15640 1 --2- 172.21.15.92:0/3378706504 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f84780cf7d0 0x7f84781464b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:32.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.368+0000 7f847cd15640 1 -- 172.21.15.92:0/3378706504 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8460009a70 con 0x7f847814b2c0 2023-11-24T06:10:32.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.369+0000 7f847cd15640 1 --2- 172.21.15.92:0/3378706504 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f847814b2c0 0x7f84780cd860 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f847000c430 tx=0x7f847000c8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:32.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.370+0000 7f8474ff9640 1 -- 172.21.15.92:0/3378706504 <== mon.2 v2:172.21.15.118:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8470005ce0 con 0x7f847814b2c0 2023-11-24T06:10:32.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.370+0000 7f847e778640 1 -- 172.21.15.92:0/3378706504 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8478146f70 con 0x7f847814b2c0 2023-11-24T06:10:32.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.371+0000 7f847e778640 1 -- 172.21.15.92:0/3378706504 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8478147530 con 0x7f847814b2c0 2023-11-24T06:10:32.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.371+0000 7f8474ff9640 1 -- 172.21.15.92:0/3378706504 <== mon.2 v2:172.21.15.118:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8470005e40 con 0x7f847814b2c0 2023-11-24T06:10:32.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.371+0000 7f8474ff9640 1 -- 172.21.15.92:0/3378706504 <== mon.2 v2:172.21.15.118:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84700072d0 con 0x7f847814b2c0 2023-11-24T06:10:32.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.371+0000 7f615bfff640 1 --2- 172.21.15.92:0/701523222 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f614806ae40 0x7f614806d300 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f614c036000 tx=0x7f614c03b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:32.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.372+0000 7f6168e7d640 1 --2- 172.21.15.92:0/701523222 >> [v2:172.21.15.118:6800/827455984,v1:172.21.15.118:6801/827455984] conn(0x7f6148072fe0 0x7f6148075400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:32.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.372+0000 7f1e62c56640 1 -- 172.21.15.92:0/2825822426 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f1e5c13f5a0 con 0x7f1e5c153950 2023-11-24T06:10:32.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.372+0000 7f847e778640 1 -- 172.21.15.92:0/3378706504 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f845c000fc0 con 0x7f847814b2c0 2023-11-24T06:10:32.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.372+0000 7fe1f7fff640 1 -- 172.21.15.92:0/3101787087 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe20801d3b0 con 0x7fe210160f80 2023-11-24T06:10:32.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.372+0000 7f84667fc640 1 -- 172.21.15.92:0/3101676020 --> [v2:172.21.15.92:6802/20560714,v1:172.21.15.92:6803/20560714] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f845c002cb0 con 0x7f8460072fe0 2023-11-24T06:10:32.412 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.373+0000 7f6168e7d640 1 --2- 172.21.15.92:0/701523222 >> [v2:172.21.15.118:6800/827455984,v1:172.21.15.118:6801/827455984] conn(0x7f6148072fe0 0x7f6148075400 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:32.412 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.373+0000 7f8478ff9640 1 -- 172.21.15.92:0/3101676020 <== osd.0 v2:172.21.15.92:6802/20560714 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f845c002cb0 con 0x7f8460072fe0 2023-11-24T06:10:32.412 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.373+0000 7f1e597fa640 1 -- 172.21.15.92:0/2825822426 <== mon.2 v2:172.21.15.118:3301/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f1e5004d440 con 0x7f1e5c153950 2023-11-24T06:10:32.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.374+0000 7f84667fc640 1 -- 172.21.15.92:0/3101676020 >> [v2:172.21.15.92:6802/20560714,v1:172.21.15.92:6803/20560714] conn(0x7f8460072fe0 msgr2=0x7f8460075400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:32.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.374+0000 7f84667fc640 1 --2- 172.21.15.92:0/3101676020 >> [v2:172.21.15.92:6802/20560714,v1:172.21.15.92:6803/20560714] conn(0x7f8460072fe0 0x7f8460075400 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:32.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.374+0000 7f84667fc640 1 -- 172.21.15.92:0/3101676020 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f846006ae40 msgr2=0x7f846006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:32.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.374+0000 7f84667fc640 1 --2- 172.21.15.92:0/3101676020 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f846006ae40 0x7f846006d300 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f846802b950 tx=0x7f846803c040 comp rx=0 tx=0).stop 2023-11-24T06:10:32.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.374+0000 7f84667fc640 1 -- 172.21.15.92:0/3101676020 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8474058990 msgr2=0x7f84740e1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:32.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.374+0000 7f84667fc640 1 --2- 172.21.15.92:0/3101676020 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8474058990 0x7f84740e1340 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f847000e820 tx=0x7f847000ec20 comp rx=0 tx=0).stop 2023-11-24T06:10:32.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.374+0000 7f84809ab640 1 -- 172.21.15.92:0/3101676020 reap_dead start 2023-11-24T06:10:32.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.374+0000 7f84667fc640 1 -- 172.21.15.92:0/3101676020 shutdown_connections 2023-11-24T06:10:32.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.374+0000 7f84667fc640 1 -- 172.21.15.92:0/3101676020 >> 172.21.15.92:0/3101676020 conn(0x7f8474058eb0 msgr2=0x7f847404d9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:32.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.374+0000 7fe1f7fff640 1 --2- 172.21.15.92:0/3101787087 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fe1fc06ae40 0x7fe1fc06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:32.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.374+0000 7fe1f7fff640 1 -- 172.21.15.92:0/3101787087 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe208089780 con 0x7fe210160f80 2023-11-24T06:10:32.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.375+0000 7fe1f7fff640 1 --2- 172.21.15.92:0/3101787087 >> [v2:172.21.15.118:6808/2946986141,v1:172.21.15.118:6809/2946986141] conn(0x7fe1fc073060 0x7fe1fc075480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:32.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.375+0000 7fe1f7fff640 1 -- 172.21.15.92:0/3101787087 --> [v2:172.21.15.118:6808/2946986141,v1:172.21.15.118:6809/2946986141] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe1fc075b30 con 0x7fe1fc073060 2023-11-24T06:10:32.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.375+0000 7fe1f7fff640 1 -- 172.21.15.92:0/3101787087 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe20808bda0 con 0x7fe210160f80 2023-11-24T06:10:32.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.375+0000 7f8474ff9640 1 -- 172.21.15.92:0/3378706504 <== mon.2 v2:172.21.15.118:3301/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f8470007430 con 0x7f847814b2c0 2023-11-24T06:10:32.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.375+0000 7fe20f7fe640 1 --2- 172.21.15.92:0/3101787087 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fe1fc06ae40 0x7fe1fc06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:32.417 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.375+0000 7fe20effd640 1 --2- 172.21.15.92:0/3101787087 >> [v2:172.21.15.118:6808/2946986141,v1:172.21.15.118:6809/2946986141] conn(0x7fe1fc073060 0x7fe1fc075480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:32.417 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.376+0000 7f61617fa640 1 -- 172.21.15.92:0/701523222 <== osd.5 v2:172.21.15.118:6800/827455984 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f6148075ab0 con 0x7f6148072fe0 2023-11-24T06:10:32.417 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.376+0000 7f7ccd16c640 1 -- 172.21.15.92:0/3667755503 --> [v2:172.21.15.92:6818/2451239906,v1:172.21.15.92:6819/2451239906] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f7cc0057040 con 0x7f7cac0730b0 2023-11-24T06:10:32.418 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.376+0000 7f8474ff9640 1 --2- 172.21.15.92:0/3378706504 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f845806ae40 0x7f845806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:32.418 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.376+0000 7f8474ff9640 1 -- 172.21.15.92:0/3378706504 <== mon.2 v2:172.21.15.118:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8470089360 con 0x7f847814b2c0 2023-11-24T06:10:32.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.376+0000 7f84667fc640 1 -- 172.21.15.92:0/3101676020 shutdown_connections 2023-11-24T06:10:32.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.376+0000 7f84667fc640 1 -- 172.21.15.92:0/3101676020 wait complete. 2023-11-24T06:10:32.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.375+0000 7f1e597fa640 1 --2- 172.21.15.92:0/2825822426 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f1e4806ae40 0x7f1e4806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:32.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.376+0000 7f1e597fa640 1 -- 172.21.15.92:0/2825822426 <== mon.2 v2:172.21.15.118:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1e500ab8f0 con 0x7f1e5c153950 2023-11-24T06:10:32.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.376+0000 7f1e597fa640 1 --2- 172.21.15.92:0/2825822426 >> [v2:172.21.15.118:6816/3425937229,v1:172.21.15.118:6819/3425937229] conn(0x7f1e48072fe0 0x7f1e48075400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:32.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.376+0000 7f1e597fa640 1 -- 172.21.15.92:0/2825822426 --> [v2:172.21.15.118:6816/3425937229,v1:172.21.15.118:6819/3425937229] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1e48075ab0 con 0x7f1e48072fe0 2023-11-24T06:10:32.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.376+0000 7f1e597fa640 1 -- 172.21.15.92:0/2825822426 <== mon.2 v2:172.21.15.118:3301/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1e500b02c0 con 0x7f1e5c153950 2023-11-24T06:10:32.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.376+0000 7fe20effd640 1 --2- 172.21.15.92:0/3101787087 >> [v2:172.21.15.118:6808/2946986141,v1:172.21.15.118:6809/2946986141] conn(0x7fe1fc073060 0x7fe1fc075480 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:32.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.376+0000 7fe20f7fe640 1 --2- 172.21.15.92:0/3101787087 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fe1fc06ae40 0x7fe1fc06d300 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fe204000c00 tx=0x7fe20403b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:32.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.376+0000 7f8474ff9640 1 --2- 172.21.15.92:0/3378706504 >> [v2:172.21.15.118:6817/48122059,v1:172.21.15.118:6818/48122059] conn(0x7f8458072fe0 0x7f8458075400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:32.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.376+0000 7f8474ff9640 1 -- 172.21.15.92:0/3378706504 --> [v2:172.21.15.118:6817/48122059,v1:172.21.15.118:6818/48122059] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8458075ab0 con 0x7f8458072fe0 2023-11-24T06:10:32.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.376+0000 7f8474ff9640 1 -- 172.21.15.92:0/3378706504 <== mon.2 v2:172.21.15.118:3301/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f84700896d0 con 0x7f847814b2c0 2023-11-24T06:10:32.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.377+0000 7f7cc4ff9640 1 -- 172.21.15.92:0/3667755503 <== osd.1 v2:172.21.15.92:6818/2451239906 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f7cc80a6ba0 con 0x7f7cac0730b0 2023-11-24T06:10:32.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.377+0000 7f8477fff640 1 --2- 172.21.15.92:0/3378706504 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f845806ae40 0x7f845806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:32.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.377+0000 7f84777fe640 1 --2- 172.21.15.92:0/3378706504 >> [v2:172.21.15.118:6817/48122059,v1:172.21.15.118:6818/48122059] conn(0x7f8458072fe0 0x7f8458075400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:32.423 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.377+0000 7f7ccd16c640 1 -- 172.21.15.92:0/3667755503 >> [v2:172.21.15.92:6818/2451239906,v1:172.21.15.92:6819/2451239906] conn(0x7f7cac0730b0 msgr2=0x7f7cac0754d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:32.423 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.377+0000 7f7ccd16c640 1 --2- 172.21.15.92:0/3667755503 >> [v2:172.21.15.92:6818/2451239906,v1:172.21.15.92:6819/2451239906] conn(0x7f7cac0730b0 0x7f7cac0754d0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:32.423 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.377+0000 7f1e611f3640 1 --2- 172.21.15.92:0/2825822426 >> [v2:172.21.15.118:6816/3425937229,v1:172.21.15.118:6819/3425937229] conn(0x7f1e48072fe0 0x7f1e48075400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:32.424 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.378+0000 7f7ccd16c640 1 -- 172.21.15.92:0/3667755503 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f7cac06ae90 msgr2=0x7f7cac06d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:32.424 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.378+0000 7f7ccd16c640 1 --2- 172.21.15.92:0/3667755503 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f7cac06ae90 0x7f7cac06d350 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f7cbc000f60 tx=0x7f7cbc009040 comp rx=0 tx=0).stop 2023-11-24T06:10:32.424 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.378+0000 7f7ccd16c640 1 -- 172.21.15.92:0/3667755503 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7cc00eb030 msgr2=0x7f7cc004ff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:32.424 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.378+0000 7f7ccd16c640 1 --2- 172.21.15.92:0/3667755503 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7cc00eb030 0x7f7cc004ff70 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f7cb4009fd0 tx=0x7f7cb400ed40 comp rx=0 tx=0).stop 2023-11-24T06:10:32.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.378+0000 7f7ccc96b640 1 -- 172.21.15.92:0/3667755503 reap_dead start 2023-11-24T06:10:32.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.378+0000 7f1e5bfff640 1 --2- 172.21.15.92:0/2825822426 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f1e4806ae40 0x7f1e4806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:32.426 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.378+0000 7f7ccd16c640 1 -- 172.21.15.92:0/3667755503 shutdown_connections 2023-11-24T06:10:32.426 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.378+0000 7f7ccd16c640 1 -- 172.21.15.92:0/3667755503 >> 172.21.15.92:0/3667755503 conn(0x7f7cc00559b0 msgr2=0x7f7cc003c0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:32.426 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.378+0000 7f7ccd16c640 1 -- 172.21.15.92:0/3667755503 shutdown_connections 2023-11-24T06:10:32.426 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.379+0000 7f7ccd16c640 1 -- 172.21.15.92:0/3667755503 wait complete. 2023-11-24T06:10:32.427 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.379+0000 7f84777fe640 1 --2- 172.21.15.92:0/3378706504 >> [v2:172.21.15.118:6817/48122059,v1:172.21.15.118:6818/48122059] conn(0x7f8458072fe0 0x7f8458075400 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:32.427 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.379+0000 7f1e5bfff640 1 --2- 172.21.15.92:0/2825822426 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f1e4806ae40 0x7f1e4806d300 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f1e4c006a90 tx=0x7f1e4c006a20 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:32.427 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.379+0000 7fe1f7fff640 1 -- 172.21.15.92:0/3101787087 <== osd.6 v2:172.21.15.118:6808/2946986141 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7fe1fc075b30 con 0x7fe1fc073060 2023-11-24T06:10:32.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.379+0000 7f1e611f3640 1 --2- 172.21.15.92:0/2825822426 >> [v2:172.21.15.118:6816/3425937229,v1:172.21.15.118:6819/3425937229] conn(0x7f1e48072fe0 0x7f1e48075400 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:32.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.379+0000 7f8477fff640 1 --2- 172.21.15.92:0/3378706504 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f845806ae40 0x7f845806d300 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f8460036000 tx=0x7f8460030000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:32.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.382+0000 7f8474ff9640 1 -- 172.21.15.92:0/3378706504 <== osd.7 v2:172.21.15.118:6817/48122059 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f8458075ab0 con 0x7f8458072fe0 2023-11-24T06:10:32.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.385+0000 7f1e597fa640 1 -- 172.21.15.92:0/2825822426 <== osd.4 v2:172.21.15.118:6816/3425937229 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f1e48075ab0 con 0x7f1e48072fe0 2023-11-24T06:10:32.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.388+0000 7f37503b0640 1 -- 172.21.15.92:0/1125600430 --> [v2:172.21.15.92:6810/1124979277,v1:172.21.15.92:6811/1124979277] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f37480675d0 con 0x7f372c073130 2023-11-24T06:10:32.430 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.389+0000 7f373effd640 1 -- 172.21.15.92:0/1125600430 <== osd.2 v2:172.21.15.92:6810/1124979277 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f37480675d0 con 0x7f372c073130 2023-11-24T06:10:32.430 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.390+0000 7f37503b0640 1 -- 172.21.15.92:0/1125600430 >> [v2:172.21.15.92:6810/1124979277,v1:172.21.15.92:6811/1124979277] conn(0x7f372c073130 msgr2=0x7f372c075550 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:32.430 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.390+0000 7f37503b0640 1 --2- 172.21.15.92:0/1125600430 >> [v2:172.21.15.92:6810/1124979277,v1:172.21.15.92:6811/1124979277] conn(0x7f372c073130 0x7f372c075550 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:32.431 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.392+0000 7f37503b0640 1 -- 172.21.15.92:0/1125600430 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f372c06af10 msgr2=0x7f372c06d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:32.431 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.392+0000 7f37503b0640 1 --2- 172.21.15.92:0/1125600430 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f372c06af10 0x7f372c06d3d0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f3738005880 tx=0x7f3738009040 comp rx=0 tx=0).stop 2023-11-24T06:10:32.431 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.392+0000 7f37503b0640 1 -- 172.21.15.92:0/1125600430 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3748160f80 msgr2=0x7f37480b3f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:32.431 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.392+0000 7f37503b0640 1 --2- 172.21.15.92:0/1125600430 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3748160f80 0x7f37480b3f20 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f3744033000 tx=0x7f3744004730 comp rx=0 tx=0).stop 2023-11-24T06:10:32.431 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.392+0000 7f374e94d640 1 -- 172.21.15.92:0/1125600430 reap_dead start 2023-11-24T06:10:32.431 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.393+0000 7f37503b0640 1 -- 172.21.15.92:0/1125600430 shutdown_connections 2023-11-24T06:10:32.431 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.393+0000 7f37503b0640 1 -- 172.21.15.92:0/1125600430 >> 172.21.15.92:0/1125600430 conn(0x7f374809c550 msgr2=0x7f374806b030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:32.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.393+0000 7f37503b0640 1 -- 172.21.15.92:0/1125600430 shutdown_connections 2023-11-24T06:10:32.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.393+0000 7f37503b0640 1 -- 172.21.15.92:0/1125600430 wait complete. 2023-11-24T06:10:32.433 INFO:teuthology.orchestra.run.smithi092.stdout:55834574852 2023-11-24T06:10:32.433 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-11-24T06:10:32.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.401+0000 7f860ab0f640 1 -- 172.21.15.92:0/1255019544 --> [v2:172.21.15.92:6826/1734115782,v1:172.21.15.92:6827/1734115782] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f85e4002cb0 con 0x7f85f00730e0 2023-11-24T06:10:32.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.402+0000 7f86017fa640 1 -- 172.21.15.92:0/1255019544 <== osd.3 v2:172.21.15.92:6826/1734115782 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f85e4002cb0 con 0x7f85f00730e0 2023-11-24T06:10:32.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.403+0000 7f860ab0f640 1 -- 172.21.15.92:0/1255019544 >> [v2:172.21.15.92:6826/1734115782,v1:172.21.15.92:6827/1734115782] conn(0x7f85f00730e0 msgr2=0x7f85f0075500 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:32.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.403+0000 7f860ab0f640 1 --2- 172.21.15.92:0/1255019544 >> [v2:172.21.15.92:6826/1734115782,v1:172.21.15.92:6827/1734115782] conn(0x7f85f00730e0 0x7f85f0075500 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:32.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.403+0000 7f860ab0f640 1 -- 172.21.15.92:0/1255019544 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f85f006af40 msgr2=0x7f85f006d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:32.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.403+0000 7f860ab0f640 1 --2- 172.21.15.92:0/1255019544 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f85f006af40 0x7f85f006d400 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f85f4007fd0 tx=0x7f85f4009040 comp rx=0 tx=0).stop 2023-11-24T06:10:32.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.403+0000 7f860ab0f640 1 -- 172.21.15.92:0/1255019544 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f86040cfad0 msgr2=0x7f860415c1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:32.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.403+0000 7f860ab0f640 1 --2- 172.21.15.92:0/1255019544 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f86040cfad0 0x7f860415c1e0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f85f800c430 tx=0x7f85f800c8f0 comp rx=0 tx=0).stop 2023-11-24T06:10:32.436 INFO:teuthology.orchestra.run.smithi092.stdout:55834574852 2023-11-24T06:10:32.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.403+0000 7f86090ac640 1 -- 172.21.15.92:0/1255019544 reap_dead start 2023-11-24T06:10:32.437 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.404+0000 7f860ab0f640 1 -- 172.21.15.92:0/1255019544 shutdown_connections 2023-11-24T06:10:32.437 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.404+0000 7f860ab0f640 1 -- 172.21.15.92:0/1255019544 >> 172.21.15.92:0/1255019544 conn(0x7f860406a800 msgr2=0x7f860406abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:32.437 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-11-24T06:10:32.438 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.404+0000 7f860ab0f640 1 -- 172.21.15.92:0/1255019544 shutdown_connections 2023-11-24T06:10:32.438 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.404+0000 7f860ab0f640 1 -- 172.21.15.92:0/1255019544 wait complete. 2023-11-24T06:10:32.439 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.411+0000 7fe215f97640 1 -- 172.21.15.92:0/3101787087 --> [v2:172.21.15.118:6808/2946986141,v1:172.21.15.118:6809/2946986141] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fe1f0002cb0 con 0x7fe1fc073060 2023-11-24T06:10:32.439 INFO:teuthology.orchestra.run.smithi092.stdout:55834574853 2023-11-24T06:10:32.440 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-11-24T06:10:32.440 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.412+0000 7f847e778640 1 -- 172.21.15.92:0/3378706504 --> [v2:172.21.15.118:6817/48122059,v1:172.21.15.118:6818/48122059] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f845c002cb0 con 0x7f8458072fe0 2023-11-24T06:10:32.440 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.412+0000 7f616a8e0640 1 -- 172.21.15.92:0/701523222 --> [v2:172.21.15.118:6800/827455984,v1:172.21.15.118:6801/827455984] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f61640d1c40 con 0x7f6148072fe0 2023-11-24T06:10:32.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.412+0000 7fe1f7fff640 1 -- 172.21.15.92:0/3101787087 <== osd.6 v2:172.21.15.118:6808/2946986141 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fe1f0002cb0 con 0x7fe1fc073060 2023-11-24T06:10:32.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.413+0000 7fe215f97640 1 -- 172.21.15.92:0/3101787087 >> [v2:172.21.15.118:6808/2946986141,v1:172.21.15.118:6809/2946986141] conn(0x7fe1fc073060 msgr2=0x7fe1fc075480 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:32.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.413+0000 7fe215f97640 1 --2- 172.21.15.92:0/3101787087 >> [v2:172.21.15.118:6808/2946986141,v1:172.21.15.118:6809/2946986141] conn(0x7fe1fc073060 0x7fe1fc075480 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:32.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.413+0000 7fe215f97640 1 -- 172.21.15.92:0/3101787087 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fe1fc06ae40 msgr2=0x7fe1fc06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:32.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.413+0000 7fe215f97640 1 --2- 172.21.15.92:0/3101787087 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fe1fc06ae40 0x7fe1fc06d300 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fe204000c00 tx=0x7fe20403b040 comp rx=0 tx=0).stop 2023-11-24T06:10:32.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.413+0000 7fe215f97640 1 -- 172.21.15.92:0/3101787087 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe210160f80 msgr2=0x7fe2100c6fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:32.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.413+0000 7fe215f97640 1 --2- 172.21.15.92:0/3101787087 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe210160f80 0x7fe2100c6fb0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fe20800be20 tx=0x7fe20800bf10 comp rx=0 tx=0).stop 2023-11-24T06:10:32.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.413+0000 7fe20ffff640 1 -- 172.21.15.92:0/3101787087 reap_dead start 2023-11-24T06:10:32.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.413+0000 7fe215f97640 1 -- 172.21.15.92:0/3101787087 shutdown_connections 2023-11-24T06:10:32.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.413+0000 7fe215f97640 1 -- 172.21.15.92:0/3101787087 >> 172.21.15.92:0/3101787087 conn(0x7fe21009c550 msgr2=0x7fe2100d2160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:32.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.413+0000 7fe215f97640 1 -- 172.21.15.92:0/3101787087 shutdown_connections 2023-11-24T06:10:32.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.413+0000 7fe215f97640 1 -- 172.21.15.92:0/3101787087 wait complete. 2023-11-24T06:10:32.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.413+0000 7f8474ff9640 1 -- 172.21.15.92:0/3378706504 <== osd.7 v2:172.21.15.118:6817/48122059 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f845c002cb0 con 0x7f8458072fe0 2023-11-24T06:10:32.444 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.413+0000 7f61617fa640 1 -- 172.21.15.92:0/701523222 <== osd.5 v2:172.21.15.118:6800/827455984 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f6148075ab0 con 0x7f6148072fe0 2023-11-24T06:10:32.444 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.414+0000 7f847e778640 1 -- 172.21.15.92:0/3378706504 >> [v2:172.21.15.118:6817/48122059,v1:172.21.15.118:6818/48122059] conn(0x7f8458072fe0 msgr2=0x7f8458075400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:32.445 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.414+0000 7f616a8e0640 1 -- 172.21.15.92:0/701523222 >> [v2:172.21.15.118:6800/827455984,v1:172.21.15.118:6801/827455984] conn(0x7f6148072fe0 msgr2=0x7f6148075400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:32.445 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.414+0000 7f616a8e0640 1 --2- 172.21.15.92:0/701523222 >> [v2:172.21.15.118:6800/827455984,v1:172.21.15.118:6801/827455984] conn(0x7f6148072fe0 0x7f6148075400 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:32.445 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.414+0000 7f616a8e0640 1 -- 172.21.15.92:0/701523222 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f614806ae40 msgr2=0x7f614806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:32.445 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.414+0000 7f616a8e0640 1 --2- 172.21.15.92:0/701523222 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f614806ae40 0x7f614806d300 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f614c036000 tx=0x7f614c03b040 comp rx=0 tx=0).stop 2023-11-24T06:10:32.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.414+0000 7f616a8e0640 1 -- 172.21.15.92:0/701523222 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f61640b2a00 msgr2=0x7f61640b7e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:32.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.414+0000 7f616a8e0640 1 --2- 172.21.15.92:0/701523222 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f61640b2a00 0x7f61640b7e50 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f615400c410 tx=0x7f615400c8d0 comp rx=0 tx=0).stop 2023-11-24T06:10:32.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.414+0000 7f847e778640 1 --2- 172.21.15.92:0/3378706504 >> [v2:172.21.15.118:6817/48122059,v1:172.21.15.118:6818/48122059] conn(0x7f8458072fe0 0x7f8458075400 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:32.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.414+0000 7f847e778640 1 -- 172.21.15.92:0/3378706504 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f845806ae40 msgr2=0x7f845806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:32.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.414+0000 7f847e778640 1 --2- 172.21.15.92:0/3378706504 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f845806ae40 0x7f845806d300 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f8460036000 tx=0x7f8460030000 comp rx=0 tx=0).stop 2023-11-24T06:10:32.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.414+0000 7f847e778640 1 -- 172.21.15.92:0/3378706504 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f847814b2c0 msgr2=0x7f84780cd860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:32.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.414+0000 7f847e778640 1 --2- 172.21.15.92:0/3378706504 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f847814b2c0 0x7f84780cd860 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f847000c430 tx=0x7f847000c8f0 comp rx=0 tx=0).stop 2023-11-24T06:10:32.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.414+0000 7f847cd15640 1 -- 172.21.15.92:0/3378706504 reap_dead start 2023-11-24T06:10:32.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.414+0000 7f847e778640 1 -- 172.21.15.92:0/3378706504 shutdown_connections 2023-11-24T06:10:32.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.414+0000 7f6168e7d640 1 -- 172.21.15.92:0/701523222 reap_dead start 2023-11-24T06:10:32.448 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.414+0000 7f847e778640 1 -- 172.21.15.92:0/3378706504 >> 172.21.15.92:0/3378706504 conn(0x7f847806a800 msgr2=0x7f84780b22f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:32.448 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.415+0000 7f616a8e0640 1 -- 172.21.15.92:0/701523222 shutdown_connections 2023-11-24T06:10:32.448 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.415+0000 7f616a8e0640 1 -- 172.21.15.92:0/701523222 >> 172.21.15.92:0/701523222 conn(0x7f616409c550 msgr2=0x7f616406c950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:32.449 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.415+0000 7f616a8e0640 1 -- 172.21.15.92:0/701523222 shutdown_connections 2023-11-24T06:10:32.449 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.415+0000 7f616a8e0640 1 -- 172.21.15.92:0/701523222 wait complete. 2023-11-24T06:10:32.449 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.415+0000 7f847e778640 1 -- 172.21.15.92:0/3378706504 shutdown_connections 2023-11-24T06:10:32.449 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.415+0000 7f847e778640 1 -- 172.21.15.92:0/3378706504 wait complete. 2023-11-24T06:10:32.450 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.419+0000 7f1e62c56640 1 -- 172.21.15.92:0/2825822426 --> [v2:172.21.15.118:6816/3425937229,v1:172.21.15.118:6819/3425937229] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f1e5c0b36f0 con 0x7f1e48072fe0 2023-11-24T06:10:32.450 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.420+0000 7f1e597fa640 1 -- 172.21.15.92:0/2825822426 <== osd.4 v2:172.21.15.118:6816/3425937229 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f1e54007630 con 0x7f1e48072fe0 2023-11-24T06:10:32.450 INFO:teuthology.orchestra.run.smithi092.stdout:55834574853 2023-11-24T06:10:32.451 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-11-24T06:10:32.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.420+0000 7f1e62c56640 1 -- 172.21.15.92:0/2825822426 >> [v2:172.21.15.118:6816/3425937229,v1:172.21.15.118:6819/3425937229] conn(0x7f1e48072fe0 msgr2=0x7f1e48075400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:32.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.420+0000 7f1e62c56640 1 --2- 172.21.15.92:0/2825822426 >> [v2:172.21.15.118:6816/3425937229,v1:172.21.15.118:6819/3425937229] conn(0x7f1e48072fe0 0x7f1e48075400 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:32.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.420+0000 7f1e62c56640 1 -- 172.21.15.92:0/2825822426 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f1e4806ae40 msgr2=0x7f1e4806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:32.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.420+0000 7f1e62c56640 1 --2- 172.21.15.92:0/2825822426 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f1e4806ae40 0x7f1e4806d300 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f1e4c006a90 tx=0x7f1e4c006a20 comp rx=0 tx=0).stop 2023-11-24T06:10:32.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.421+0000 7f1e62c56640 1 -- 172.21.15.92:0/2825822426 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1e5c153950 msgr2=0x7f1e5c154880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:32.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.421+0000 7f1e62c56640 1 --2- 172.21.15.92:0/2825822426 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1e5c153950 0x7f1e5c154880 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f1e500098c0 tx=0x7f1e500046c0 comp rx=0 tx=0).stop 2023-11-24T06:10:32.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.421+0000 7f1e611f3640 1 -- 172.21.15.92:0/2825822426 reap_dead start 2023-11-24T06:10:32.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.421+0000 7f1e62c56640 1 -- 172.21.15.92:0/2825822426 shutdown_connections 2023-11-24T06:10:32.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.421+0000 7f1e62c56640 1 -- 172.21.15.92:0/2825822426 >> 172.21.15.92:0/2825822426 conn(0x7f1e5c0b0be0 msgr2=0x7f1e5c159cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:32.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.421+0000 7f1e62c56640 1 -- 172.21.15.92:0/2825822426 shutdown_connections 2023-11-24T06:10:32.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.421+0000 7f1e62c56640 1 -- 172.21.15.92:0/2825822426 wait complete. 2023-11-24T06:10:32.455 INFO:teuthology.orchestra.run.smithi092.stdout:55834574853 2023-11-24T06:10:32.455 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-11-24T06:10:32.456 INFO:teuthology.orchestra.run.smithi092.stdout:55834574853 2023-11-24T06:10:32.457 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-11-24T06:10:32.458 INFO:teuthology.orchestra.run.smithi092.stdout:55834574852 2023-11-24T06:10:32.458 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-11-24T06:10:32.459 INFO:teuthology.orchestra.run.smithi092.stdout:55834574853 2023-11-24T06:10:32.459 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-11-24T06:10:32.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.776+0000 7fad138d3640 1 Processor -- start 2023-11-24T06:10:32.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.776+0000 7fad138d3640 1 -- start start 2023-11-24T06:10:32.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.778+0000 7fad138d3640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fad0c0b1f10 0x7fad0c0d12f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:32.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.778+0000 7fad138d3640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad0c0afc60 con 0x7fad0c0d1830 2023-11-24T06:10:32.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.778+0000 7fad138d3640 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad0c0ce7d0 con 0x7fad0c15d3b0 2023-11-24T06:10:32.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.778+0000 7fad138d3640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad0c0d3f70 con 0x7fad0c0b1f10 2023-11-24T06:10:32.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.779+0000 7fad1166f640 1 --1- >> v1:172.21.15.118:6789/0 conn(0x7fad0c15d3b0 0x7fad0c0d3740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.118:6789/0 says I am v1:172.21.15.92:56824/0 (socket says 172.21.15.92:56824) 2023-11-24T06:10:32.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.779+0000 7fad1166f640 1 -- 172.21.15.92:0/1529875767 learned_addr learned my addr 172.21.15.92:0/1529875767 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-24T06:10:32.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.779+0000 7fad10e6e640 1 --2- 172.21.15.92:0/1529875767 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fad0c0b1f10 0x7fad0c0d12f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:32.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.781+0000 7fad03fff640 1 -- 172.21.15.92:0/1529875767 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 807185342 0 0) 0x7fad0c0afc60 con 0x7fad0c0d1830 2023-11-24T06:10:32.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.781+0000 7fad03fff640 1 -- 172.21.15.92:0/1529875767 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7facf8003520 con 0x7fad0c0d1830 2023-11-24T06:10:32.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.781+0000 7fad03fff640 1 -- 172.21.15.92:0/1529875767 <== mon.1 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3515710440 0 0) 0x7fad0c0ce7d0 con 0x7fad0c15d3b0 2023-11-24T06:10:32.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.781+0000 7fad03fff640 1 -- 172.21.15.92:0/1529875767 --> v1:172.21.15.118:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fad0c0afc60 con 0x7fad0c15d3b0 2023-11-24T06:10:32.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.781+0000 7fad10e6e640 1 -- 172.21.15.92:0/1529875767 >> v1:172.21.15.118:6789/0 conn(0x7fad0c15d3b0 legacy=0x7fad0c0d3740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:32.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.782+0000 7fad10e6e640 1 -- 172.21.15.92:0/1529875767 >> v1:172.21.15.92:6789/0 conn(0x7fad0c0d1830 legacy=0x7fad0c0b00f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:32.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.782+0000 7fad10e6e640 1 -- 172.21.15.92:0/1529875767 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad0c0ceab0 con 0x7fad0c0b1f10 2023-11-24T06:10:32.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.785+0000 7f21aa046640 1 Processor -- start 2023-11-24T06:10:32.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.785+0000 7f21aa046640 1 -- start start 2023-11-24T06:10:32.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.786+0000 7fad10e6e640 1 --2- 172.21.15.92:0/1529875767 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fad0c0b1f10 0x7fad0c0d12f0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7facfc0097d0 tx=0x7facfc02b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b5f1f63168ef46a1 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:32.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.787+0000 7f21aa046640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f21a40d0280 0x7f21a40ce1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:32.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.787+0000 7f21aa046640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f21a40ce6f0 0x7f21a40b6be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:32.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.788+0000 7f21aa046640 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21a40b6750 con 0x7f21a415d590 2023-11-24T06:10:32.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.788+0000 7f21aa046640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21a40cf1f0 con 0x7f21a40ce6f0 2023-11-24T06:10:32.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.788+0000 7f21aa046640 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21a40cef10 con 0x7f21a40d0280 2023-11-24T06:10:32.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.788+0000 7f21a37fe640 1 --1- >> v1:172.21.15.118:6789/0 conn(0x7f21a415d590 0x7f21a40cfb70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.118:6789/0 says I am v1:172.21.15.92:56830/0 (socket says 172.21.15.92:56830) 2023-11-24T06:10:32.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.788+0000 7f21a37fe640 1 -- 172.21.15.92:0/3086492601 learned_addr learned my addr 172.21.15.92:0/3086492601 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-24T06:10:32.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.788+0000 7f21a2ffd640 1 --2- 172.21.15.92:0/3086492601 >> v2:172.21.15.118:3300/0 conn(0x7f21a40d0280 0x7f21a40ce1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:32.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.788+0000 7f21a3fff640 1 --2- 172.21.15.92:0/3086492601 >> v2:172.21.15.92:3300/0 conn(0x7f21a40ce6f0 0x7f21a40b6be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:32.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.789+0000 7f21a2ffd640 1 -- 172.21.15.92:0/3086492601 >> v2:172.21.15.92:3300/0 conn(0x7f21a40ce6f0 msgr2=0x7f21a40b6be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:32.794 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.789+0000 7f21a2ffd640 1 --2- 172.21.15.92:0/3086492601 >> v2:172.21.15.92:3300/0 conn(0x7f21a40ce6f0 0x7f21a40b6be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:32.794 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.789+0000 7f21a2ffd640 1 -- 172.21.15.92:0/3086492601 >> v1:172.21.15.118:6789/0 conn(0x7f21a415d590 legacy=0x7f21a40cfb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:32.794 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.789+0000 7f21a27fc640 1 -- 172.21.15.92:0/3086492601 <== mon.1 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2247731152 0 0) 0x7f21a40b6750 con 0x7f21a415d590 2023-11-24T06:10:32.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.789+0000 7f21a2ffd640 1 -- 172.21.15.92:0/3086492601 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21a40b6100 con 0x7f21a40d0280 2023-11-24T06:10:32.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.790+0000 7f21a3fff640 1 --2- 172.21.15.92:0/3086492601 >> v2:172.21.15.92:3300/0 conn(0x7f21a40ce6f0 0x7f21a40b6be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:10:32.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.791+0000 7fad03fff640 1 -- 172.21.15.92:0/1529875767 <== mon.2 v2:172.21.15.118:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facfc031830 con 0x7fad0c0b1f10 2023-11-24T06:10:32.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.791+0000 7fad03fff640 1 -- 172.21.15.92:0/1529875767 <== mon.2 v2:172.21.15.118:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facfc031990 con 0x7fad0c0b1f10 2023-11-24T06:10:32.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.791+0000 7fad03fff640 1 -- 172.21.15.92:0/1529875767 <== mon.2 v2:172.21.15.118:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facfc031d20 con 0x7fad0c0b1f10 2023-11-24T06:10:32.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.791+0000 7f21a2ffd640 1 --2- 172.21.15.92:0/3086492601 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f21a40d0280 0x7f21a40ce1b0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f21a408d970 tx=0x7f219402b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9cfea7b3316fc0f8 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:32.797 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.792+0000 7fad138d3640 1 -- 172.21.15.92:0/1529875767 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fad0c0b1f10 msgr2=0x7fad0c0d12f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:32.797 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.792+0000 7fad138d3640 1 --2- 172.21.15.92:0/1529875767 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fad0c0b1f10 0x7fad0c0d12f0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7facfc0097d0 tx=0x7facfc02b8d0 comp rx=0 tx=0).stop 2023-11-24T06:10:32.797 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.792+0000 7f21a27fc640 1 -- 172.21.15.92:0/3086492601 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2194031830 con 0x7f21a40d0280 2023-11-24T06:10:32.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.792+0000 7fad138d3640 1 -- 172.21.15.92:0/1529875767 shutdown_connections 2023-11-24T06:10:32.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.792+0000 7fad138d3640 1 --2- 172.21.15.92:0/1529875767 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fad0c0b1f10 0x7fad0c0d12f0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:32.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.792+0000 7fad138d3640 1 -- 172.21.15.92:0/1529875767 >> 172.21.15.92:0/1529875767 conn(0x7fad0c09c370 msgr2=0x7fad0c0d3030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:32.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.792+0000 7fad138d3640 1 -- 172.21.15.92:0/1529875767 shutdown_connections 2023-11-24T06:10:32.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.792+0000 7f21a27fc640 1 -- 172.21.15.92:0/3086492601 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2194031990 con 0x7f21a40d0280 2023-11-24T06:10:32.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.792+0000 7f21a27fc640 1 -- 172.21.15.92:0/3086492601 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2194031d20 con 0x7f21a40d0280 2023-11-24T06:10:32.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.793+0000 7f21aa046640 1 -- 172.21.15.92:0/3086492601 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f21a40d0280 msgr2=0x7f21a40ce1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:32.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.793+0000 7f21aa046640 1 --2- 172.21.15.92:0/3086492601 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f21a40d0280 0x7f21a40ce1b0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f21a408d970 tx=0x7f219402b8d0 comp rx=0 tx=0).stop 2023-11-24T06:10:32.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.794+0000 7f21aa046640 1 -- 172.21.15.92:0/3086492601 shutdown_connections 2023-11-24T06:10:32.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.794+0000 7f21aa046640 1 --2- 172.21.15.92:0/3086492601 >> v2:172.21.15.92:3300/0 conn(0x7f21a40ce6f0 0x7f21a40b6be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:32.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.794+0000 7f21aa046640 1 --2- 172.21.15.92:0/3086492601 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f21a40d0280 0x7f21a40ce1b0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:32.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.794+0000 7f21aa046640 1 -- 172.21.15.92:0/3086492601 >> 172.21.15.92:0/3086492601 conn(0x7f21a409c540 msgr2=0x7f21a40d3210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:32.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.794+0000 7f21aa046640 1 -- 172.21.15.92:0/3086492601 shutdown_connections 2023-11-24T06:10:32.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.794+0000 7f21aa046640 1 -- 172.21.15.92:0/3086492601 wait complete. 2023-11-24T06:10:32.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.792+0000 7fad138d3640 1 -- 172.21.15.92:0/1529875767 wait complete. 2023-11-24T06:10:32.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.795+0000 7fad138d3640 1 Processor -- start 2023-11-24T06:10:32.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.796+0000 7fad138d3640 1 -- start start 2023-11-24T06:10:32.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.797+0000 7fad138d3640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fad0c0b1f10 0x7fad0c0c92b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:32.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.797+0000 7f21aa046640 1 Processor -- start 2023-11-24T06:10:32.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.797+0000 7fad138d3640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fad0c0d1830 0x7fad0c152870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:32.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.797+0000 7fad138d3640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fad0c15d3b0 0x7fad0c0cf6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:32.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.797+0000 7fad138d3640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad0c0ccaf0 con 0x7fad0c0d1830 2023-11-24T06:10:32.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.797+0000 7fad138d3640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad0c0ccf30 con 0x7fad0c15d3b0 2023-11-24T06:10:32.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.797+0000 7fad138d3640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad0c0cbf70 con 0x7fad0c0b1f10 2023-11-24T06:10:32.808 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.798+0000 7f21aa046640 1 -- start start 2023-11-24T06:10:32.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.798+0000 7fad1166f640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fad0c0b1f10 0x7fad0c0c92b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:32.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.798+0000 7fad1166f640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fad0c0b1f10 0x7fad0c0c92b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3301/0 says I am v2:172.21.15.92:39566/0 (socket says 172.21.15.92:39566) 2023-11-24T06:10:32.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.798+0000 7fad1166f640 1 -- 172.21.15.92:0/2810576455 learned_addr learned my addr 172.21.15.92:0/2810576455 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:10:32.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.798+0000 7fad11e70640 1 --2- 172.21.15.92:0/2810576455 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fad0c15d3b0 0x7fad0c0cf6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:32.811 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.798+0000 7fad10e6e640 1 --2- 172.21.15.92:0/2810576455 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fad0c0d1830 0x7fad0c152870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:32.811 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.799+0000 7fad1166f640 1 -- 172.21.15.92:0/2810576455 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fad0c15d3b0 msgr2=0x7fad0c0cf6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:32.811 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.799+0000 7fad1166f640 1 --2- 172.21.15.92:0/2810576455 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fad0c15d3b0 0x7fad0c0cf6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:32.811 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.799+0000 7fad1166f640 1 -- 172.21.15.92:0/2810576455 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fad0c0d1830 msgr2=0x7fad0c152870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:32.811 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.799+0000 7fad1166f640 1 --2- 172.21.15.92:0/2810576455 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fad0c0d1830 0x7fad0c152870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:32.811 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.799+0000 7fad1166f640 1 -- 172.21.15.92:0/2810576455 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facfc009400 con 0x7fad0c0b1f10 2023-11-24T06:10:32.812 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.800+0000 7fad1166f640 1 --2- 172.21.15.92:0/2810576455 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fad0c0b1f10 0x7fad0c0c92b0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fad0800c410 tx=0x7fad0800c8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:32.812 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.800+0000 7fad01ffb640 1 -- 172.21.15.92:0/2810576455 <== mon.2 v2:172.21.15.118:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad08005ce0 con 0x7fad0c0b1f10 2023-11-24T06:10:32.812 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.800+0000 7fad138d3640 1 -- 172.21.15.92:0/2810576455 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad0c0cc250 con 0x7fad0c0b1f10 2023-11-24T06:10:32.812 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.800+0000 7fad138d3640 1 -- 172.21.15.92:0/2810576455 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad0c0cc810 con 0x7fad0c0b1f10 2023-11-24T06:10:32.812 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.801+0000 7fad01ffb640 1 -- 172.21.15.92:0/2810576455 <== mon.2 v2:172.21.15.118:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad08005e40 con 0x7fad0c0b1f10 2023-11-24T06:10:32.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.801+0000 7f49e7bbe640 1 Processor -- start 2023-11-24T06:10:32.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.801+0000 7f49e7bbe640 1 -- start start 2023-11-24T06:10:32.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.801+0000 7f21aa046640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f21a40ce6f0 0x7f21a409d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:32.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.801+0000 7f21aa046640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f21a40d0280 0x7f21a40c7530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:32.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.802+0000 7f21aa046640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f21a415d590 0x7f21a41fcf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:32.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.802+0000 7f21aa046640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21a40c8d40 con 0x7f21a40ce6f0 2023-11-24T06:10:32.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.802+0000 7f21aa046640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21a409d820 con 0x7f21a415d590 2023-11-24T06:10:32.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.802+0000 7f21aa046640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21a40c7a70 con 0x7f21a40d0280 2023-11-24T06:10:32.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.802+0000 7f21a3fff640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f21a415d590 0x7f21a41fcf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:32.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.802+0000 7f21a37fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f21a40ce6f0 0x7f21a409d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:32.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.802+0000 7f21a3fff640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f21a415d590 0x7f21a41fcf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.92:58320/0 (socket says 172.21.15.92:58320) 2023-11-24T06:10:32.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.802+0000 7f21a3fff640 1 -- 172.21.15.92:0/1926531856 learned_addr learned my addr 172.21.15.92:0/1926531856 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:10:32.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.803+0000 7f21a3fff640 1 -- 172.21.15.92:0/1926531856 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f21a40d0280 msgr2=0x7f21a40c7530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:32.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.803+0000 7f49e7bbe640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f49e0072230 0x7f49e01506d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:32.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.803+0000 7f49e7bbe640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f49e00cf290 0x7f49e015d300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:32.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.803+0000 7f49e7bbe640 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49e0150c10 con 0x7f49e00cf6d0 2023-11-24T06:10:32.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.803+0000 7f49e7bbe640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49e0072610 con 0x7f49e0072230 2023-11-24T06:10:32.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.803+0000 7f49e7bbe640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49e0071190 con 0x7f49e00cf290 2023-11-24T06:10:32.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.803+0000 7f21a3fff640 1 --2- 172.21.15.92:0/1926531856 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f21a40d0280 0x7f21a40c7530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:32.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.803+0000 7f21a3fff640 1 -- 172.21.15.92:0/1926531856 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f21a40ce6f0 msgr2=0x7f21a409d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:32.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.803+0000 7f21a3fff640 1 --2- 172.21.15.92:0/1926531856 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f21a40ce6f0 0x7f21a409d2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:32.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.803+0000 7f21a3fff640 1 -- 172.21.15.92:0/1926531856 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f219c009a70 con 0x7f21a415d590 2023-11-24T06:10:32.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.804+0000 7f49e615b640 1 --1- >> v1:172.21.15.118:6789/0 conn(0x7f49e00cf6d0 0x7f49e00b2bd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.118:6789/0 says I am v1:172.21.15.92:56846/0 (socket says 172.21.15.92:56846) 2023-11-24T06:10:32.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.804+0000 7f49e615b640 1 -- 172.21.15.92:0/4157705810 learned_addr learned my addr 172.21.15.92:0/4157705810 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-24T06:10:32.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.801+0000 7fad01ffb640 1 -- 172.21.15.92:0/2810576455 <== mon.2 v2:172.21.15.118:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad080072d0 con 0x7fad0c0b1f10 2023-11-24T06:10:32.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.804+0000 7f49e595a640 1 --2- 172.21.15.92:0/4157705810 >> v2:172.21.15.92:3300/0 conn(0x7f49e0072230 0x7f49e01506d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:32.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.804+0000 7fad138d3640 1 -- 172.21.15.92:0/2810576455 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad0c15d970 con 0x7fad0c0b1f10 2023-11-24T06:10:32.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.805+0000 7f49e595a640 1 -- 172.21.15.92:0/4157705810 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f49e00cf290 msgr2=0x7f49e015d300 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-24T06:10:32.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.805+0000 7f49e595a640 1 --2- 172.21.15.92:0/4157705810 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f49e00cf290 0x7f49e015d300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:32.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.805+0000 7f49e595a640 1 -- 172.21.15.92:0/4157705810 >> v1:172.21.15.118:6789/0 conn(0x7f49e00cf6d0 legacy=0x7f49e00b2bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:32.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.805+0000 7f49e595a640 1 -- 172.21.15.92:0/4157705810 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49e0071470 con 0x7f49e0072230 2023-11-24T06:10:32.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.805+0000 7f21a3fff640 1 --2- 172.21.15.92:0/1926531856 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f21a415d590 0x7f21a41fcf00 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f219c009f00 tx=0x7f219c00c490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:32.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.806+0000 7f219bfff640 1 -- 172.21.15.92:0/1926531856 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f219c009d70 con 0x7f21a415d590 2023-11-24T06:10:32.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.806+0000 7f219bfff640 1 -- 172.21.15.92:0/1926531856 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f219c00f040 con 0x7f21a415d590 2023-11-24T06:10:32.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.806+0000 7f219bfff640 1 -- 172.21.15.92:0/1926531856 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f219c003bb0 con 0x7f21a415d590 2023-11-24T06:10:32.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.807+0000 7f49e595a640 1 --2- 172.21.15.92:0/4157705810 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f49e0072230 0x7f49e01506d0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f49d4009d90 tx=0x7f49d402bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6d8635d75695074 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:32.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.807+0000 7fad01ffb640 1 -- 172.21.15.92:0/2810576455 <== mon.2 v2:172.21.15.118:3301/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fad08007430 con 0x7fad0c0b1f10 2023-11-24T06:10:32.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.807+0000 7f21aa046640 1 -- 172.21.15.92:0/1926531856 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2194009400 con 0x7f21a415d590 2023-11-24T06:10:32.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.808+0000 7f49e4958640 1 -- 172.21.15.92:0/4157705810 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49d402cd30 con 0x7f49e0072230 2023-11-24T06:10:32.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.808+0000 7f49e4958640 1 -- 172.21.15.92:0/4157705810 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49d402ce90 con 0x7f49e0072230 2023-11-24T06:10:32.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.808+0000 7f49e4958640 1 -- 172.21.15.92:0/4157705810 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49d4034660 con 0x7f49e0072230 2023-11-24T06:10:32.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.808+0000 7f21aa046640 1 -- 172.21.15.92:0/1926531856 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21a40c88b0 con 0x7f21a415d590 2023-11-24T06:10:32.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.808+0000 7f24d74d1640 1 Processor -- start 2023-11-24T06:10:32.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.809+0000 7fad01ffb640 1 --2- 172.21.15.92:0/2810576455 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7facf006ae70 0x7facf006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:32.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.809+0000 7fad01ffb640 1 -- 172.21.15.92:0/2810576455 <== mon.2 v2:172.21.15.118:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fad08088bb0 con 0x7fad0c0b1f10 2023-11-24T06:10:32.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.809+0000 7f24d74d1640 1 -- start start 2023-11-24T06:10:32.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.809+0000 7fad10e6e640 1 --2- 172.21.15.92:0/2810576455 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7facf006ae70 0x7facf006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:32.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.809+0000 7f21aa046640 1 -- 172.21.15.92:0/1926531856 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21a40c8a40 con 0x7f21a415d590 2023-11-24T06:10:32.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.810+0000 7f49e7bbe640 1 -- 172.21.15.92:0/4157705810 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f49e0072230 msgr2=0x7f49e01506d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:32.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.810+0000 7f49e7bbe640 1 --2- 172.21.15.92:0/4157705810 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f49e0072230 0x7f49e01506d0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f49d4009d90 tx=0x7f49d402bf80 comp rx=0 tx=0).stop 2023-11-24T06:10:32.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.810+0000 7f49e7bbe640 1 -- 172.21.15.92:0/4157705810 shutdown_connections 2023-11-24T06:10:32.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.810+0000 7f49e7bbe640 1 --2- 172.21.15.92:0/4157705810 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f49e00cf290 0x7f49e015d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:32.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.810+0000 7f49e7bbe640 1 --2- 172.21.15.92:0/4157705810 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f49e0072230 0x7f49e01506d0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:32.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.810+0000 7f49e7bbe640 1 -- 172.21.15.92:0/4157705810 >> 172.21.15.92:0/4157705810 conn(0x7f49e006a800 msgr2=0x7f49e006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:32.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.811+0000 7f49e7bbe640 1 -- 172.21.15.92:0/4157705810 shutdown_connections 2023-11-24T06:10:32.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.811+0000 7fad10e6e640 1 --2- 172.21.15.92:0/2810576455 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7facf006ae70 0x7facf006d330 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7facfc036000 tx=0x7facfc03d040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:32.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.811+0000 7f49e7bbe640 1 -- 172.21.15.92:0/4157705810 wait complete. 2023-11-24T06:10:32.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.811+0000 7f24d74d1640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f24c80e3130 0x7f24c80e7750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:32.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.811+0000 7f24d74d1640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f24c803c070 0x7f24c8058020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:32.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.811+0000 7f24d74d1640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f24c8053c80 0x7f24c80540f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:32.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.811+0000 7f24d74d1640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24c8058560 con 0x7f24c80e3130 2023-11-24T06:10:32.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.811+0000 7f24d74d1640 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24c8054630 con 0x7f24c8053c80 2023-11-24T06:10:32.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.811+0000 7f24d74d1640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24c80587e0 con 0x7f24c803c070 2023-11-24T06:10:32.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.812+0000 7f24d5cce640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f24c803c070 0x7f24c8058020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:32.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.812+0000 7f24d64cf640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f24c80e3130 0x7f24c80e7750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:32.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.812+0000 7f24d5cce640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f24c803c070 0x7f24c8058020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3301/0 says I am v2:172.21.15.92:39582/0 (socket says 172.21.15.92:39582) 2023-11-24T06:10:32.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.812+0000 7f24d5cce640 1 -- 172.21.15.92:0/1957573920 learned_addr learned my addr 172.21.15.92:0/1957573920 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:10:32.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.812+0000 7f24d6cd0640 1 --2- 172.21.15.92:0/1957573920 >> v2:172.21.15.118:3300/0 conn(0x7f24c8053c80 0x7f24c80540f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:32.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.813+0000 7f24d6cd0640 1 -- 172.21.15.92:0/1957573920 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f24c803c070 msgr2=0x7f24c8058020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:32.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.813+0000 7f24d6cd0640 1 --2- 172.21.15.92:0/1957573920 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f24c803c070 0x7f24c8058020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:32.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.813+0000 7f24d6cd0640 1 -- 172.21.15.92:0/1957573920 >> v2:172.21.15.92:3300/0 conn(0x7f24c80e3130 msgr2=0x7f24c80e7750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:32.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.813+0000 7f24d6cd0640 1 --2- 172.21.15.92:0/1957573920 >> v2:172.21.15.92:3300/0 conn(0x7f24c80e3130 0x7f24c80e7750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:32.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.813+0000 7f24d6cd0640 1 -- 172.21.15.92:0/1957573920 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24c8058ac0 con 0x7f24c8053c80 2023-11-24T06:10:32.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.813+0000 7f24d5cce640 1 --2- 172.21.15.92:0/1957573920 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f24c803c070 0x7f24c8058020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:10:32.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.814+0000 7f24d6cd0640 1 --2- 172.21.15.92:0/1957573920 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f24c8053c80 0x7f24c80540f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f24d004fab0 tx=0x7f24d00a5e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=be88039b2d2c6e14 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:32.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.813+0000 7f219bfff640 1 -- 172.21.15.92:0/1926531856 <== mon.1 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f219c014480 con 0x7f21a415d590 2023-11-24T06:10:32.839 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.814+0000 7f219bfff640 1 --2- 172.21.15.92:0/1926531856 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f218406af10 0x7f218406d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:32.839 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.814+0000 7f219bfff640 1 -- 172.21.15.92:0/1926531856 <== mon.1 v2:172.21.15.118:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f219c08a9c0 con 0x7f21a415d590 2023-11-24T06:10:32.839 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.815+0000 7f21a37fe640 1 --2- 172.21.15.92:0/1926531856 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f218406af10 0x7f218406d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:32.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.815+0000 7f49e7bbe640 1 Processor -- start 2023-11-24T06:10:32.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.815+0000 7f49e7bbe640 1 -- start start 2023-11-24T06:10:32.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.815+0000 7f11fb63d640 1 Processor -- start 2023-11-24T06:10:32.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.815+0000 7fab43aad640 1 Processor -- start 2023-11-24T06:10:32.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.816+0000 7f21a37fe640 1 --2- 172.21.15.92:0/1926531856 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f218406af10 0x7f218406d3d0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f218c000f60 tx=0x7f218c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:32.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.816+0000 7fab43aad640 1 -- start start 2023-11-24T06:10:32.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.816+0000 7f11fb63d640 1 -- start start 2023-11-24T06:10:32.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.816+0000 7f49e7bbe640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f49e0072230 0x7f49e01538c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:32.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.816+0000 7f49e7bbe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f49e00cf290 0x7f49e01406b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:32.844 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.816+0000 7f49e7bbe640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f49e00cf6d0 0x7f49e0140bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:32.844 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.816+0000 7f49e7bbe640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49e01565d0 con 0x7f49e00cf290 2023-11-24T06:10:32.844 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.816+0000 7f49e7bbe640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49e0153370 con 0x7f49e00cf6d0 2023-11-24T06:10:32.844 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.816+0000 7f49e7bbe640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49e013eb00 con 0x7f49e0072230 2023-11-24T06:10:32.844 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.817+0000 7f49e5159640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f49e00cf290 0x7f49e01406b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:32.844 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.817+0000 7f49e615b640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f49e00cf6d0 0x7f49e0140bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:32.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.817+0000 7f49e615b640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f49e00cf6d0 0x7f49e0140bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.92:58332/0 (socket says 172.21.15.92:58332) 2023-11-24T06:10:32.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.817+0000 7f49e615b640 1 -- 172.21.15.92:0/504994497 learned_addr learned my addr 172.21.15.92:0/504994497 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:10:32.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.817+0000 7f49e5159640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f49e00cf290 0x7f49e01406b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47356/0 (socket says 172.21.15.92:47356) 2023-11-24T06:10:32.846 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.817+0000 7f11fb63d640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f11f415d590 0x7f11f40cfb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:32.846 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.817+0000 7f49e595a640 1 --2- 172.21.15.92:0/504994497 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f49e0072230 0x7f49e01538c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:32.847 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.817+0000 7f11fb63d640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f11f40d00b0 0x7f11f40ce1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:32.847 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.818+0000 7fab43aad640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fab3c15d320 0x7fab3c0c6d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:32.847 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.818+0000 7fab43aad640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fab3c0c72c0 0x7fab3c0ce890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:32.847 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.818+0000 7fab43aad640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab3c0d1c50 con 0x7fab3c15d700 2023-11-24T06:10:32.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.818+0000 7fab43aad640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab3c0d1f30 con 0x7fab3c15d320 2023-11-24T06:10:32.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.818+0000 7fab43aad640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab3c0ce090 con 0x7fab3c0c72c0 2023-11-24T06:10:32.849 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.817+0000 7f11fb63d640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11f40b6750 con 0x7f11f40ce6f0 2023-11-24T06:10:32.849 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.818+0000 7f11fb63d640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11f40d0530 con 0x7f11f415d590 2023-11-24T06:10:32.849 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.818+0000 7f11fb63d640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11f40cf1f0 con 0x7f11f40d00b0 2023-11-24T06:10:32.849 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.818+0000 7f11f93d9640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f11f415d590 0x7f11f40cfb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:32.849 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.818+0000 7f11f93d9640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f11f415d590 0x7f11f40cfb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47362/0 (socket says 172.21.15.92:47362) 2023-11-24T06:10:32.849 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.818+0000 7f11f93d9640 1 -- 172.21.15.92:0/506804865 learned_addr learned my addr 172.21.15.92:0/506804865 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:10:32.850 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.818+0000 7f11f8bd8640 1 --2- 172.21.15.92:0/506804865 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f11f40d00b0 0x7f11f40ce1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:32.850 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.819+0000 7f49e615b640 1 -- 172.21.15.92:0/504994497 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f49e0072230 msgr2=0x7f49e01538c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:32.850 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.819+0000 7f49e615b640 1 --2- 172.21.15.92:0/504994497 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f49e0072230 0x7f49e01538c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:32.850 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.819+0000 7f49e615b640 1 -- 172.21.15.92:0/504994497 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f49e00cf290 msgr2=0x7f49e01406b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:32.851 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.819+0000 7f49e615b640 1 --2- 172.21.15.92:0/504994497 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f49e00cf290 0x7f49e01406b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:32.851 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.819+0000 7fab41048640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fab3c0c72c0 0x7fab3c0ce890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:32.851 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.818+0000 7f11f93d9640 1 -- 172.21.15.92:0/506804865 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f11f40d00b0 msgr2=0x7f11f40ce1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:32.852 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.818+0000 7f11f93d9640 1 --2- 172.21.15.92:0/506804865 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f11f40d00b0 0x7f11f40ce1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:32.852 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.818+0000 7f11f93d9640 1 -- 172.21.15.92:0/506804865 >> v1:172.21.15.92:6789/0 conn(0x7f11f40ce6f0 legacy=0x7f11f40d0de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:32.852 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.819+0000 7fab41048640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fab3c0c72c0 0x7fab3c0ce890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3301/0 says I am v2:172.21.15.92:39604/0 (socket says 172.21.15.92:39604) 2023-11-24T06:10:32.852 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.819+0000 7fab41048640 1 -- 172.21.15.92:0/3639411776 learned_addr learned my addr 172.21.15.92:0/3639411776 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:10:32.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.819+0000 7efe47143640 1 Processor -- start 2023-11-24T06:10:32.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.819+0000 7fab41849640 1 --2- 172.21.15.92:0/3639411776 >> v2:172.21.15.92:3300/0 conn(0x7fab3c15d320 0x7fab3c0c6d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:32.854 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.819+0000 7efe47143640 1 -- start start 2023-11-24T06:10:32.854 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.819+0000 7f11f93d9640 1 -- 172.21.15.92:0/506804865 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11f40cef10 con 0x7f11f415d590 2023-11-24T06:10:32.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.819+0000 7f49e615b640 1 -- 172.21.15.92:0/504994497 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49d4009a70 con 0x7f49e00cf6d0 2023-11-24T06:10:32.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.819+0000 7f49e615b640 1 --2- 172.21.15.92:0/504994497 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f49e00cf6d0 0x7f49e0140bf0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f49d800c3c0 tx=0x7f49d800c880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:32.856 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.819+0000 7fab41048640 1 -- 172.21.15.92:0/3639411776 >> v2:172.21.15.92:3300/0 conn(0x7fab3c15d320 msgr2=0x7fab3c0c6d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:32.856 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.820+0000 7fab41048640 1 --2- 172.21.15.92:0/3639411776 >> v2:172.21.15.92:3300/0 conn(0x7fab3c15d320 0x7fab3c0c6d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:32.856 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.820+0000 7fab40847640 1 -- 172.21.15.92:0/3639411776 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4088327659 0 0) 0x7fab3c0d1c50 con 0x7fab3c15d700 2023-11-24T06:10:32.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.820+0000 7fab41048640 1 -- 172.21.15.92:0/3639411776 >> v1:172.21.15.92:6789/0 conn(0x7fab3c15d700 legacy=0x7fab3c0d2e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:32.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.820+0000 7fab41048640 1 -- 172.21.15.92:0/3639411776 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab3c0ce370 con 0x7fab3c0c72c0 2023-11-24T06:10:32.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.820+0000 7fab41849640 1 --2- 172.21.15.92:0/3639411776 >> v2:172.21.15.92:3300/0 conn(0x7fab3c15d320 0x7fab3c0c6d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:10:32.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.820+0000 7f11f93d9640 1 --2- 172.21.15.92:0/506804865 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f11f415d590 0x7f11f40cfb70 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f11f0009870 tx=0x7f11f002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=208810b007df878b server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:32.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.821+0000 7f11ebfff640 1 -- 172.21.15.92:0/506804865 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11f0031640 con 0x7f11f415d590 2023-11-24T06:10:32.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.821+0000 7f11ebfff640 1 -- 172.21.15.92:0/506804865 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11f00317a0 con 0x7f11f415d590 2023-11-24T06:10:32.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.821+0000 7f11ebfff640 1 -- 172.21.15.92:0/506804865 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11f0031b30 con 0x7f11f415d590 2023-11-24T06:10:32.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.821+0000 7efe47143640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7efe40159040 0x7efe400cf000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:32.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.821+0000 7efe47143640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7efe4015c300 0x7efe4015f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:32.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.821+0000 7efe47143640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe400c6060 con 0x7efe400b34e0 2023-11-24T06:10:32.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.821+0000 7efe47143640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe400c62d0 con 0x7efe40159040 2023-11-24T06:10:32.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.821+0000 7efe47143640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe400c65b0 con 0x7efe4015c300 2023-11-24T06:10:32.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.821+0000 7efe44edf640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7efe40159040 0x7efe400cf000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:32.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.821+0000 7efe3ffff640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7efe4015c300 0x7efe4015f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:32.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.821+0000 7efe44edf640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7efe40159040 0x7efe400cf000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47388/0 (socket says 172.21.15.92:47388) 2023-11-24T06:10:32.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.821+0000 7efe44edf640 1 -- 172.21.15.92:0/1731199960 learned_addr learned my addr 172.21.15.92:0/1731199960 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:10:32.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.822+0000 7f11fb63d640 1 -- 172.21.15.92:0/506804865 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f11f415d590 msgr2=0x7f11f40cfb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:32.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.822+0000 7efe44edf640 1 -- 172.21.15.92:0/1731199960 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7efe4015c300 msgr2=0x7efe4015f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:32.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.822+0000 7efe44edf640 1 --2- 172.21.15.92:0/1731199960 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7efe4015c300 0x7efe4015f3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:32.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.822+0000 7f11fb63d640 1 --2- 172.21.15.92:0/506804865 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f11f415d590 0x7f11f40cfb70 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f11f0009870 tx=0x7f11f002b8f0 comp rx=0 tx=0).stop 2023-11-24T06:10:32.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.822+0000 7efe44edf640 1 -- 172.21.15.92:0/1731199960 >> v1:172.21.15.92:6789/0 conn(0x7efe400b34e0 legacy=0x7efe400cdb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:32.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.822+0000 7efe44edf640 1 -- 172.21.15.92:0/1731199960 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe4015eec0 con 0x7efe40159040 2023-11-24T06:10:32.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.822+0000 7f11fb63d640 1 -- 172.21.15.92:0/506804865 shutdown_connections 2023-11-24T06:10:32.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.822+0000 7f11fb63d640 1 --2- 172.21.15.92:0/506804865 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f11f40d00b0 0x7f11f40ce1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:32.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.822+0000 7f11fb63d640 1 --2- 172.21.15.92:0/506804865 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f11f415d590 0x7f11f40cfb70 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:32.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.822+0000 7f11fb63d640 1 -- 172.21.15.92:0/506804865 >> 172.21.15.92:0/506804865 conn(0x7f11f409c540 msgr2=0x7f11f40d3210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:32.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.822+0000 7f11fb63d640 1 -- 172.21.15.92:0/506804865 shutdown_connections 2023-11-24T06:10:32.865 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.822+0000 7f11fb63d640 1 -- 172.21.15.92:0/506804865 wait complete. 2023-11-24T06:10:32.865 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.823+0000 7fab41048640 1 --2- 172.21.15.92:0/3639411776 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fab3c0c72c0 0x7fab3c0ce890 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fab28009870 tx=0x7fab2802b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9e820ab7dd15af52 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:32.865 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.823+0000 7efe44edf640 1 --2- 172.21.15.92:0/1731199960 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efe40159040 0x7efe400cf000 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7efe30009870 tx=0x7efe3002b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b243acb62918c1 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:32.866 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.823+0000 7efe3f7fe640 1 -- 172.21.15.92:0/1731199960 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe30030860 con 0x7efe40159040 2023-11-24T06:10:32.866 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.824+0000 7efe3f7fe640 1 -- 172.21.15.92:0/1731199960 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe300309c0 con 0x7efe40159040 2023-11-24T06:10:32.866 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.824+0000 7efe3f7fe640 1 -- 172.21.15.92:0/1731199960 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe30030d90 con 0x7efe40159040 2023-11-24T06:10:32.866 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.824+0000 7efe47143640 1 -- 172.21.15.92:0/1731199960 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efe40159040 msgr2=0x7efe400cf000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:32.866 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.824+0000 7efe47143640 1 --2- 172.21.15.92:0/1731199960 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efe40159040 0x7efe400cf000 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7efe30009870 tx=0x7efe3002b9a0 comp rx=0 tx=0).stop 2023-11-24T06:10:32.866 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.824+0000 7efe47143640 1 -- 172.21.15.92:0/1731199960 shutdown_connections 2023-11-24T06:10:32.866 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.824+0000 7efe47143640 1 --2- 172.21.15.92:0/1731199960 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7efe4015c300 0x7efe4015f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:32.866 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.824+0000 7efe47143640 1 --2- 172.21.15.92:0/1731199960 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efe40159040 0x7efe400cf000 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:32.866 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.824+0000 7efe47143640 1 -- 172.21.15.92:0/1731199960 >> 172.21.15.92:0/1731199960 conn(0x7efe40066690 msgr2=0x7efe4006a3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:32.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.824+0000 7f11fb63d640 1 Processor -- start 2023-11-24T06:10:32.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.825+0000 7fab40847640 1 -- 172.21.15.92:0/3639411776 <== mon.2 v2:172.21.15.118:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab28031640 con 0x7fab3c0c72c0 2023-11-24T06:10:32.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.825+0000 7fab40847640 1 -- 172.21.15.92:0/3639411776 <== mon.2 v2:172.21.15.118:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab280317a0 con 0x7fab3c0c72c0 2023-11-24T06:10:32.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.825+0000 7fab40847640 1 -- 172.21.15.92:0/3639411776 <== mon.2 v2:172.21.15.118:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab28031b30 con 0x7fab3c0c72c0 2023-11-24T06:10:32.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.825+0000 7efe47143640 1 -- 172.21.15.92:0/1731199960 shutdown_connections 2023-11-24T06:10:32.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.825+0000 7f11fb63d640 1 -- start start 2023-11-24T06:10:32.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.825+0000 7fad01ffb640 1 -- 172.21.15.92:0/2810576455 <== mon.2 v2:172.21.15.118:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fad08053d30 con 0x7fad0c0b1f10 2023-11-24T06:10:32.869 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.825+0000 7f6e1611d640 1 Processor -- start 2023-11-24T06:10:32.869 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.825+0000 7f6e1611d640 1 -- start start 2023-11-24T06:10:32.869 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.825+0000 7fab43aad640 1 -- 172.21.15.92:0/3639411776 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fab3c0c72c0 msgr2=0x7fab3c0ce890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:32.870 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.825+0000 7efe47143640 1 -- 172.21.15.92:0/1731199960 wait complete. 2023-11-24T06:10:32.870 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.826+0000 7f11fb63d640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f11f40ce6f0 0x7f11f40b86d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:32.871 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.826+0000 7f11fb63d640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f11f40d00b0 0x7f11f415b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:32.871 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.826+0000 7f11fb63d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f11f415bb50 0x7f11f40cc5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:32.872 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.825+0000 7fab43aad640 1 --2- 172.21.15.92:0/3639411776 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fab3c0c72c0 0x7fab3c0ce890 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fab28009870 tx=0x7fab2802b8f0 comp rx=0 tx=0).stop 2023-11-24T06:10:32.872 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.826+0000 7fab43aad640 1 -- 172.21.15.92:0/3639411776 shutdown_connections 2023-11-24T06:10:32.872 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.826+0000 7fab43aad640 1 --2- 172.21.15.92:0/3639411776 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fab3c0c72c0 0x7fab3c0ce890 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:32.872 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.826+0000 7fab43aad640 1 --2- 172.21.15.92:0/3639411776 >> v2:172.21.15.92:3300/0 conn(0x7fab3c15d320 0x7fab3c0c6d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:32.873 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.826+0000 7fab43aad640 1 -- 172.21.15.92:0/3639411776 >> 172.21.15.92:0/3639411776 conn(0x7fab3c08cc50 msgr2=0x7fab3c09c570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:32.873 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.826+0000 7fab43aad640 1 -- 172.21.15.92:0/3639411776 shutdown_connections 2023-11-24T06:10:32.873 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.826+0000 7fab43aad640 1 -- 172.21.15.92:0/3639411776 wait complete. 2023-11-24T06:10:32.874 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.826+0000 7f11fb63d640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11f415bfb0 con 0x7f11f415bb50 2023-11-24T06:10:32.874 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.826+0000 7f11fb63d640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11f406e980 con 0x7f11f40d00b0 2023-11-24T06:10:32.874 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.826+0000 7f11fb63d640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11f41fa9a0 con 0x7f11f40ce6f0 2023-11-24T06:10:32.874 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.826+0000 7f11f9bda640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f11f415bb50 0x7f11f40cc5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:32.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.826+0000 7f11f9bda640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f11f415bb50 0x7f11f40cc5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47396/0 (socket says 172.21.15.92:47396) 2023-11-24T06:10:32.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.826+0000 7f11f93d9640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f11f40ce6f0 0x7f11f40b86d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:32.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.826+0000 7f11f9bda640 1 -- 172.21.15.92:0/3859295851 learned_addr learned my addr 172.21.15.92:0/3859295851 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:10:32.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.827+0000 7f11f8bd8640 1 --2- 172.21.15.92:0/3859295851 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f11f40d00b0 0x7f11f415b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:32.876 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.827+0000 7efe47143640 1 Processor -- start 2023-11-24T06:10:32.876 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.827+0000 7f11f9bda640 1 -- 172.21.15.92:0/3859295851 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f11f40ce6f0 msgr2=0x7f11f40b86d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:32.876 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.827+0000 7f11f9bda640 1 --2- 172.21.15.92:0/3859295851 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f11f40ce6f0 0x7f11f40b86d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:32.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.827+0000 7f11f9bda640 1 -- 172.21.15.92:0/3859295851 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f11f40d00b0 msgr2=0x7f11f415b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:32.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.827+0000 7f11f9bda640 1 --2- 172.21.15.92:0/3859295851 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f11f40d00b0 0x7f11f415b610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:32.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.827+0000 7f11f9bda640 1 -- 172.21.15.92:0/3859295851 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11f00094d0 con 0x7f11f415bb50 2023-11-24T06:10:32.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.827+0000 7efe47143640 1 -- start start 2023-11-24T06:10:32.879 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.827+0000 7f6e1611d640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f6e100d0280 0x7f6e100ce1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:32.879 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.827+0000 7f6e1611d640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6e100ce6f0 0x7f6e100b6be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:32.879 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.827+0000 7f6e1611d640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e100b6750 con 0x7f6e1015d590 2023-11-24T06:10:32.879 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.827+0000 7f6e1611d640 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e100cf1f0 con 0x7f6e100d0280 2023-11-24T06:10:32.879 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.827+0000 7f6e1611d640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e100cef10 con 0x7f6e100ce6f0 2023-11-24T06:10:32.879 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.828+0000 7f6e0f7fe640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f6e1015d590 0x7f6e100cfb70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:55860/0 (socket says 172.21.15.92:55860) 2023-11-24T06:10:32.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.828+0000 7f6e0f7fe640 1 -- 172.21.15.92:0/2674364762 learned_addr learned my addr 172.21.15.92:0/2674364762 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-24T06:10:32.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.828+0000 7f11f9bda640 1 --2- 172.21.15.92:0/3859295851 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f11f415bb50 0x7f11f40cc5b0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f11ec00b970 tx=0x7f11ec00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:32.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.828+0000 7fab43aad640 1 Processor -- start 2023-11-24T06:10:32.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.828+0000 7f6e0ffff640 1 --2- 172.21.15.92:0/2674364762 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6e100ce6f0 0x7f6e100b6be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:32.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.828+0000 7fab43aad640 1 -- start start 2023-11-24T06:10:32.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.828+0000 7f24d54cd640 1 -- 172.21.15.92:0/1957573920 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24d00a6be0 con 0x7f24c8053c80 2023-11-24T06:10:32.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.828+0000 7f24d54cd640 1 -- 172.21.15.92:0/1957573920 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24d00a6dc0 con 0x7f24c8053c80 2023-11-24T06:10:32.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.829+0000 7efe47143640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7efe400b34e0 0x7efe4014aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:32.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.828+0000 7f6e07fff640 1 --2- 172.21.15.92:0/2674364762 >> v2:172.21.15.118:3300/0 conn(0x7f6e100d0280 0x7f6e100ce1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:32.884 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.829+0000 7efe47143640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7efe40159040 0x7efe400b24f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:32.884 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.828+0000 7f24d54cd640 1 -- 172.21.15.92:0/1957573920 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24d00ae640 con 0x7f24c8053c80 2023-11-24T06:10:32.885 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.829+0000 7f6e0effd640 1 -- 172.21.15.92:0/2674364762 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3986202792 0 0) 0x7f6e100b6750 con 0x7f6e1015d590 2023-11-24T06:10:32.885 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.829+0000 7f49de7fc640 1 -- 172.21.15.92:0/504994497 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49d8005ce0 con 0x7f49e00cf6d0 2023-11-24T06:10:32.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.829+0000 7f6e0effd640 1 -- 172.21.15.92:0/2674364762 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6dfc003520 con 0x7f6e1015d590 2023-11-24T06:10:32.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.829+0000 7f11e9ffb640 1 -- 172.21.15.92:0/3859295851 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11ec00cb40 con 0x7f11f415bb50 2023-11-24T06:10:32.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.829+0000 7f11e9ffb640 1 -- 172.21.15.92:0/3859295851 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11ec00cca0 con 0x7f11f415bb50 2023-11-24T06:10:32.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.829+0000 7f6e0ffff640 1 -- 172.21.15.92:0/2674364762 >> v2:172.21.15.118:3300/0 conn(0x7f6e100d0280 msgr2=0x7f6e100ce1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:32.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.829+0000 7f6e0ffff640 1 --2- 172.21.15.92:0/2674364762 >> v2:172.21.15.118:3300/0 conn(0x7f6e100d0280 0x7f6e100ce1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:32.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.829+0000 7f6e0ffff640 1 -- 172.21.15.92:0/2674364762 >> v1:172.21.15.92:6789/0 conn(0x7f6e1015d590 legacy=0x7f6e100cfb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:32.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.829+0000 7f6e0ffff640 1 -- 172.21.15.92:0/2674364762 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e100b6100 con 0x7f6e100ce6f0 2023-11-24T06:10:32.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.829+0000 7f6e07fff640 1 --2- 172.21.15.92:0/2674364762 >> v2:172.21.15.118:3300/0 conn(0x7f6e100d0280 0x7f6e100ce1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:10:32.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.829+0000 7efe47143640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efe4015c300 0x7efe4015aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:32.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.830+0000 7f219bfff640 1 -- 172.21.15.92:0/1926531856 <== mon.1 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f219c055a50 con 0x7f21a415d590 2023-11-24T06:10:32.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.829+0000 7f49de7fc640 1 -- 172.21.15.92:0/504994497 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49d8005e40 con 0x7f49e00cf6d0 2023-11-24T06:10:32.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.829+0000 7f49de7fc640 1 -- 172.21.15.92:0/504994497 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49d80072d0 con 0x7f49e00cf6d0 2023-11-24T06:10:32.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.829+0000 7f49e7bbe640 1 -- 172.21.15.92:0/504994497 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49e0153650 con 0x7f49e00cf6d0 2023-11-24T06:10:32.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.830+0000 7efe47143640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe400b46b0 con 0x7efe4015c300 2023-11-24T06:10:32.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.830+0000 7efe47143640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe4014afd0 con 0x7efe400b34e0 2023-11-24T06:10:32.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.830+0000 7efe47143640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe400b2f00 con 0x7efe40159040 2023-11-24T06:10:32.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.829+0000 7f49e7bbe640 1 -- 172.21.15.92:0/504994497 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49e006d090 con 0x7f49e00cf6d0 2023-11-24T06:10:32.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.830+0000 7efe3ffff640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7efe40159040 0x7efe400b24f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:32.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.830+0000 7efe3ffff640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7efe40159040 0x7efe400b24f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3301/0 says I am v2:172.21.15.92:39638/0 (socket says 172.21.15.92:39638) 2023-11-24T06:10:32.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.830+0000 7efe3ffff640 1 -- 172.21.15.92:0/4254925593 learned_addr learned my addr 172.21.15.92:0/4254925593 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:10:32.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.830+0000 7efe456e0640 1 --2- 172.21.15.92:0/4254925593 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efe4015c300 0x7efe4015aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:32.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.829+0000 7f11e9ffb640 1 -- 172.21.15.92:0/3859295851 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11ec014660 con 0x7f11f415bb50 2023-11-24T06:10:32.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.829+0000 7f11fb63d640 1 -- 172.21.15.92:0/3859295851 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11f41fac80 con 0x7f11f415bb50 2023-11-24T06:10:32.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.829+0000 7f11fb63d640 1 -- 172.21.15.92:0/3859295851 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11f41fb240 con 0x7f11f415bb50 2023-11-24T06:10:32.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.831+0000 7f6e0ffff640 1 --2- 172.21.15.92:0/2674364762 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6e100ce6f0 0x7f6e100b6be0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f6e08009d90 tx=0x7f6e0802beb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4f87a3c76605c658 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:32.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.831+0000 7f6e0effd640 1 -- 172.21.15.92:0/2674364762 <== mon.2 v2:172.21.15.118:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e0802cc60 con 0x7f6e100ce6f0 2023-11-24T06:10:32.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.831+0000 7f6e0effd640 1 -- 172.21.15.92:0/2674364762 <== mon.2 v2:172.21.15.118:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e0802cdc0 con 0x7f6e100ce6f0 2023-11-24T06:10:32.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.831+0000 7f6e0effd640 1 -- 172.21.15.92:0/2674364762 <== mon.2 v2:172.21.15.118:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e08034640 con 0x7f6e100ce6f0 2023-11-24T06:10:32.891 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.829+0000 7fab43aad640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fab3c0c72c0 0x7fab3c0b5f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:32.891 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.829+0000 7fab43aad640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fab3c15d320 0x7fab3c0b8760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:32.891 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.829+0000 7fab43aad640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fab3c15d700 0x7fab3c0ccae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:32.891 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.829+0000 7fab43aad640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab3c0b8ca0 con 0x7fab3c15d700 2023-11-24T06:10:32.891 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.829+0000 7fab43aad640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab3c14e0c0 con 0x7fab3c0c72c0 2023-11-24T06:10:32.891 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.830+0000 7fab43aad640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab3c14e3a0 con 0x7fab3c15d320 2023-11-24T06:10:32.891 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.830+0000 7fab41849640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fab3c0c72c0 0x7fab3c0b5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:32.891 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.830+0000 7fab41849640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fab3c0c72c0 0x7fab3c0b5f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.92:58374/0 (socket says 172.21.15.92:58374) 2023-11-24T06:10:32.891 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.830+0000 7fab41849640 1 -- 172.21.15.92:0/677833557 learned_addr learned my addr 172.21.15.92:0/677833557 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:10:32.891 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.830+0000 7fab4204a640 1 --2- 172.21.15.92:0/677833557 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fab3c15d700 0x7fab3c0ccae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:32.892 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.831+0000 7fab41849640 1 -- 172.21.15.92:0/677833557 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fab3c15d320 msgr2=0x7fab3c0b8760 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-11-24T06:10:32.892 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.831+0000 7fab41849640 1 --2- 172.21.15.92:0/677833557 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fab3c15d320 0x7fab3c0b8760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:32.892 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.831+0000 7fab41849640 1 -- 172.21.15.92:0/677833557 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fab3c15d700 msgr2=0x7fab3c0ccae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:32.892 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.831+0000 7fab41849640 1 --2- 172.21.15.92:0/677833557 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fab3c15d700 0x7fab3c0ccae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:32.892 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.831+0000 7fab41849640 1 -- 172.21.15.92:0/677833557 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab30009a70 con 0x7fab3c0c72c0 2023-11-24T06:10:32.892 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.830+0000 7f24d74d1640 1 -- 172.21.15.92:0/1957573920 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f24c8053c80 msgr2=0x7f24c80540f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:32.892 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.830+0000 7f24d74d1640 1 --2- 172.21.15.92:0/1957573920 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f24c8053c80 0x7f24c80540f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f24d004fab0 tx=0x7f24d00a5e30 comp rx=0 tx=0).stop 2023-11-24T06:10:32.892 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.831+0000 7f24d74d1640 1 -- 172.21.15.92:0/1957573920 shutdown_connections 2023-11-24T06:10:32.892 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.831+0000 7f24d74d1640 1 --2- 172.21.15.92:0/1957573920 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f24c8053c80 0x7f24c80540f0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:32.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.831+0000 7f24d74d1640 1 --2- 172.21.15.92:0/1957573920 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f24c803c070 0x7f24c8058020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:32.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.831+0000 7f24d74d1640 1 --2- 172.21.15.92:0/1957573920 >> v2:172.21.15.92:3300/0 conn(0x7f24c80e3130 0x7f24c80e7750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:32.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.831+0000 7f24d74d1640 1 -- 172.21.15.92:0/1957573920 >> 172.21.15.92:0/1957573920 conn(0x7f24c8058eb0 msgr2=0x7f24c8059290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:32.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.831+0000 7f24d74d1640 1 -- 172.21.15.92:0/1957573920 shutdown_connections 2023-11-24T06:10:32.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.831+0000 7f24d74d1640 1 -- 172.21.15.92:0/1957573920 wait complete. 2023-11-24T06:10:32.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.832+0000 7f11fb63d640 1 -- 172.21.15.92:0/3859295851 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11d4005250 con 0x7f11f415bb50 2023-11-24T06:10:32.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.832+0000 7f6e1611d640 1 -- 172.21.15.92:0/2674364762 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6e100ce6f0 msgr2=0x7f6e100b6be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:32.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.832+0000 7f6e1611d640 1 --2- 172.21.15.92:0/2674364762 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6e100ce6f0 0x7f6e100b6be0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f6e08009d90 tx=0x7f6e0802beb0 comp rx=0 tx=0).stop 2023-11-24T06:10:32.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.832+0000 7f49e7bbe640 1 -- 172.21.15.92:0/504994497 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49e0156710 con 0x7f49e00cf6d0 2023-11-24T06:10:32.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.833+0000 7f6e1611d640 1 -- 172.21.15.92:0/2674364762 shutdown_connections 2023-11-24T06:10:32.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.833+0000 7f6e1611d640 1 --2- 172.21.15.92:0/2674364762 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6e100ce6f0 0x7f6e100b6be0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:32.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.833+0000 7f6e1611d640 1 --2- 172.21.15.92:0/2674364762 >> v2:172.21.15.118:3300/0 conn(0x7f6e100d0280 0x7f6e100ce1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:32.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.833+0000 7f6e1611d640 1 -- 172.21.15.92:0/2674364762 >> 172.21.15.92:0/2674364762 conn(0x7f6e1009c540 msgr2=0x7f6e100d3210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:32.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.833+0000 7f6e1611d640 1 -- 172.21.15.92:0/2674364762 shutdown_connections 2023-11-24T06:10:32.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.833+0000 7fab41849640 1 --2- 172.21.15.92:0/677833557 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fab3c0c72c0 0x7fab3c0b5f30 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fab3000efc0 tx=0x7fab3000c490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:32.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.831+0000 7efe456e0640 1 -- 172.21.15.92:0/4254925593 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7efe40159040 msgr2=0x7efe400b24f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:32.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.831+0000 7efe456e0640 1 --2- 172.21.15.92:0/4254925593 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7efe40159040 0x7efe400b24f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:32.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.832+0000 7efe456e0640 1 -- 172.21.15.92:0/4254925593 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7efe400b34e0 msgr2=0x7efe4014aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:32.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.832+0000 7efe456e0640 1 --2- 172.21.15.92:0/4254925593 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7efe400b34e0 0x7efe4014aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:32.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.832+0000 7efe456e0640 1 -- 172.21.15.92:0/4254925593 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe300094d0 con 0x7efe4015c300 2023-11-24T06:10:32.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.833+0000 7f6e1611d640 1 -- 172.21.15.92:0/2674364762 wait complete. 2023-11-24T06:10:32.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.834+0000 7f24d74d1640 1 Processor -- start 2023-11-24T06:10:32.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.834+0000 7f24d74d1640 1 -- start start 2023-11-24T06:10:32.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.834+0000 7efe456e0640 1 --2- 172.21.15.92:0/4254925593 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efe4015c300 0x7efe4015aa10 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7efe3800c700 tx=0x7efe3800cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:32.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.835+0000 7efe3ffff640 1 --2- 172.21.15.92:0/4254925593 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7efe40159040 0x7efe400b24f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:10:32.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.836+0000 7fab3a7fc640 1 -- 172.21.15.92:0/677833557 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab30014680 con 0x7fab3c0c72c0 2023-11-24T06:10:32.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.836+0000 7fab3a7fc640 1 -- 172.21.15.92:0/677833557 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab3000f040 con 0x7fab3c0c72c0 2023-11-24T06:10:32.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.836+0000 7fab3a7fc640 1 -- 172.21.15.92:0/677833557 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab30014a00 con 0x7fab3c0c72c0 2023-11-24T06:10:32.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.836+0000 7fab43aad640 1 -- 172.21.15.92:0/677833557 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab280094d0 con 0x7fab3c0c72c0 2023-11-24T06:10:32.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.836+0000 7f6e1611d640 1 Processor -- start 2023-11-24T06:10:32.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.836+0000 7fab43aad640 1 -- 172.21.15.92:0/677833557 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab3c14e960 con 0x7fab3c0c72c0 2023-11-24T06:10:32.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.836+0000 7f49de7fc640 1 -- 172.21.15.92:0/504994497 <== mon.1 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f49d80074f0 con 0x7f49e00cf6d0 2023-11-24T06:10:32.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.837+0000 7f24d74d1640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f24c803c070 0x7f24c80e4e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:32.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.837+0000 7f24d74d1640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f24c8053c80 0x7f24c81899d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:32.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.837+0000 7fab43aad640 1 -- 172.21.15.92:0/677833557 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab3c0675d0 con 0x7fab3c0c72c0 2023-11-24T06:10:32.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.837+0000 7f6e1611d640 1 -- start start 2023-11-24T06:10:32.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.838+0000 7f24d64cf640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f24c803c070 0x7f24c80e4e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:32.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.838+0000 7f24d64cf640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f24c803c070 0x7f24c80e4e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.92:58388/0 (socket says 172.21.15.92:58388) 2023-11-24T06:10:32.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.838+0000 7f11e9ffb640 1 -- 172.21.15.92:0/3859295851 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f11ec003970 con 0x7f11f415bb50 2023-11-24T06:10:32.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.838+0000 7f24d64cf640 1 -- 172.21.15.92:0/4217055934 learned_addr learned my addr 172.21.15.92:0/4217055934 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:10:32.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.838+0000 7f24d74d1640 1 --2- 172.21.15.92:0/4217055934 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f24c80e3130 0x7f24c8185260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:32.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.838+0000 7f49de7fc640 1 --2- 172.21.15.92:0/504994497 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f49c406af10 0x7f49c406d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:32.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.838+0000 7f49de7fc640 1 -- 172.21.15.92:0/504994497 <== mon.1 v2:172.21.15.118:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f49d8088cf0 con 0x7f49e00cf6d0 2023-11-24T06:10:32.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.839+0000 7f49e595a640 1 --2- 172.21.15.92:0/504994497 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f49c406af10 0x7f49c406d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:32.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.839+0000 7f24d5cce640 1 --2- 172.21.15.92:0/4217055934 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f24c8053c80 0x7f24c81899d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:32.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.839+0000 7f24d74d1640 1 -- 172.21.15.92:0/4217055934 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24c8189f10 con 0x7f24c8053c80 2023-11-24T06:10:32.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.839+0000 7f24d74d1640 1 -- 172.21.15.92:0/4217055934 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24c8050020 con 0x7f24c803c070 2023-11-24T06:10:32.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.839+0000 7f24d6cd0640 1 --2- 172.21.15.92:0/4217055934 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f24c80e3130 0x7f24c8185260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:32.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.839+0000 7f24d74d1640 1 -- 172.21.15.92:0/4217055934 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24c80e53e0 con 0x7f24c80e3130 2023-11-24T06:10:32.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.840+0000 7f49e595a640 1 --2- 172.21.15.92:0/504994497 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f49c406af10 0x7f49c406d3d0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f49d4036000 tx=0x7f49d4030000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:32.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.840+0000 7f24d5cce640 1 -- 172.21.15.92:0/4217055934 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f24c80e3130 msgr2=0x7f24c8185260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:32.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.840+0000 7f24d5cce640 1 --2- 172.21.15.92:0/4217055934 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f24c80e3130 0x7f24c8185260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:32.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.840+0000 7f24d5cce640 1 -- 172.21.15.92:0/4217055934 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f24c803c070 msgr2=0x7f24c80e4e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:32.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.840+0000 7f24d5cce640 1 --2- 172.21.15.92:0/4217055934 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f24c803c070 0x7f24c80e4e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:32.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.840+0000 7f24d5cce640 1 -- 172.21.15.92:0/4217055934 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24c00094d0 con 0x7f24c8053c80 2023-11-24T06:10:32.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.840+0000 7f11e9ffb640 1 --2- 172.21.15.92:0/3859295851 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f11d806af10 0x7f11d806d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:32.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.840+0000 7f11e9ffb640 1 -- 172.21.15.92:0/3859295851 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f11ec089350 con 0x7f11f415bb50 2023-11-24T06:10:32.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.840+0000 7f11f93d9640 1 --2- 172.21.15.92:0/3859295851 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f11d806af10 0x7f11d806d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:32.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.838+0000 7f6e1611d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6e100ce6f0 0x7f6e1015bb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:32.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.838+0000 7f6e1611d640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6e100d0280 0x7f6e100b5190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:32.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.840+0000 7f6e1611d640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6e1015d590 0x7f6e100b56d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:32.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.840+0000 7f6e1611d640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e1015f7c0 con 0x7f6e100ce6f0 2023-11-24T06:10:32.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.840+0000 7f6e1611d640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e1015eb40 con 0x7f6e100d0280 2023-11-24T06:10:32.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.840+0000 7f6e1611d640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e1015ee20 con 0x7f6e1015d590 2023-11-24T06:10:32.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.841+0000 7f24d5cce640 1 --2- 172.21.15.92:0/4217055934 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f24c8053c80 0x7f24c81899d0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f24c000e860 tx=0x7f24c000ed20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:32.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.840+0000 7f6e0ffff640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6e1015d590 0x7f6e100b56d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:32.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.840+0000 7f6e07fff640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6e100d0280 0x7f6e100b5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:32.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.840+0000 7f6e0ffff640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6e1015d590 0x7f6e100b56d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3301/0 says I am v2:172.21.15.92:39674/0 (socket says 172.21.15.92:39674) 2023-11-24T06:10:32.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.840+0000 7f6e0ffff640 1 -- 172.21.15.92:0/1041297052 learned_addr learned my addr 172.21.15.92:0/1041297052 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:10:32.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.841+0000 7f11f93d9640 1 --2- 172.21.15.92:0/3859295851 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f11d806af10 0x7f11d806d3d0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f11f0036000 tx=0x7f11f003c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:32.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.841+0000 7f6e0f7fe640 1 --2- 172.21.15.92:0/1041297052 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6e100ce6f0 0x7f6e1015bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:32.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.841+0000 7f6e0ffff640 1 -- 172.21.15.92:0/1041297052 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6e100d0280 msgr2=0x7f6e100b5190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:32.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.841+0000 7f6e0ffff640 1 --2- 172.21.15.92:0/1041297052 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6e100d0280 0x7f6e100b5190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:32.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.841+0000 7f6e0ffff640 1 -- 172.21.15.92:0/1041297052 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6e100ce6f0 msgr2=0x7f6e1015bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:32.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.841+0000 7f6e0ffff640 1 --2- 172.21.15.92:0/1041297052 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6e100ce6f0 0x7f6e1015bb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:32.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.841+0000 7f6e0ffff640 1 -- 172.21.15.92:0/1041297052 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e08009a70 con 0x7f6e1015d590 2023-11-24T06:10:32.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.842+0000 7f6e0f7fe640 1 --2- 172.21.15.92:0/1041297052 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6e100ce6f0 0x7f6e1015bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:10:32.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.842+0000 7f6e0ffff640 1 --2- 172.21.15.92:0/1041297052 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6e1015d590 0x7f6e100b56d0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f6e08009eb0 tx=0x7f6e08004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:32.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.842+0000 7f6e0cff9640 1 -- 172.21.15.92:0/1041297052 <== mon.2 v2:172.21.15.118:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e08009450 con 0x7f6e1015d590 2023-11-24T06:10:32.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.842+0000 7f6e0cff9640 1 -- 172.21.15.92:0/1041297052 <== mon.2 v2:172.21.15.118:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e08003e10 con 0x7f6e1015d590 2023-11-24T06:10:32.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.842+0000 7f6e0cff9640 1 -- 172.21.15.92:0/1041297052 <== mon.2 v2:172.21.15.118:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e080047c0 con 0x7f6e1015d590 2023-11-24T06:10:32.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.842+0000 7f6e1611d640 1 -- 172.21.15.92:0/1041297052 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e1015f100 con 0x7f6e1015d590 2023-11-24T06:10:32.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.843+0000 7f6e1611d640 1 -- 172.21.15.92:0/1041297052 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e1015c210 con 0x7f6e1015d590 2023-11-24T06:10:32.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.843+0000 7f24c6ffd640 1 -- 172.21.15.92:0/4217055934 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24c000c940 con 0x7f24c8053c80 2023-11-24T06:10:32.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.844+0000 7f24c6ffd640 1 -- 172.21.15.92:0/4217055934 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24c000caa0 con 0x7f24c8053c80 2023-11-24T06:10:32.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.843+0000 7efe3d7fa640 1 -- 172.21.15.92:0/4254925593 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe38003b70 con 0x7efe4015c300 2023-11-24T06:10:32.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.843+0000 7efe47143640 1 -- 172.21.15.92:0/4254925593 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe400b3d80 con 0x7efe4015c300 2023-11-24T06:10:32.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.844+0000 7efe47143640 1 -- 172.21.15.92:0/4254925593 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe400b4340 con 0x7efe4015c300 2023-11-24T06:10:32.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.844+0000 7efe3d7fa640 1 -- 172.21.15.92:0/4254925593 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe38003cd0 con 0x7efe4015c300 2023-11-24T06:10:32.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.844+0000 7efe3d7fa640 1 -- 172.21.15.92:0/4254925593 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe38006d50 con 0x7efe4015c300 2023-11-24T06:10:32.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.844+0000 7f24d74d1640 1 -- 172.21.15.92:0/4217055934 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24d004edc0 con 0x7f24c8053c80 2023-11-24T06:10:32.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.844+0000 7f24d74d1640 1 -- 172.21.15.92:0/4217055934 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24c80e59a0 con 0x7f24c8053c80 2023-11-24T06:10:32.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.844+0000 7f6e1611d640 1 -- 172.21.15.92:0/1041297052 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e1015f900 con 0x7f6e1015d590 2023-11-24T06:10:32.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.846+0000 7f6e0cff9640 1 -- 172.21.15.92:0/1041297052 <== mon.2 v2:172.21.15.118:3301/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6e08049460 con 0x7f6e1015d590 2023-11-24T06:10:32.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.845+0000 7f24c6ffd640 1 -- 172.21.15.92:0/4217055934 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24c000ce10 con 0x7f24c8053c80 2023-11-24T06:10:32.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.846+0000 7f24d74d1640 1 -- 172.21.15.92:0/4217055934 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24c80501f0 con 0x7f24c8053c80 2023-11-24T06:10:32.910 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.846+0000 7efe47143640 1 -- 172.21.15.92:0/4254925593 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe20005250 con 0x7efe4015c300 2023-11-24T06:10:32.910 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.846+0000 7f11e9ffb640 1 -- 172.21.15.92:0/3859295851 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f11ec054520 con 0x7f11f415bb50 2023-11-24T06:10:32.910 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.847+0000 7f6e0cff9640 1 --2- 172.21.15.92:0/1041297052 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f6df006ae40 0x7f6df006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:32.910 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.847+0000 7f6e0cff9640 1 -- 172.21.15.92:0/1041297052 <== mon.2 v2:172.21.15.118:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6e080abc50 con 0x7f6e1015d590 2023-11-24T06:10:32.911 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.847+0000 7f6e0f7fe640 1 --2- 172.21.15.92:0/1041297052 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f6df006ae40 0x7f6df006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:32.911 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.847+0000 7efe3d7fa640 1 -- 172.21.15.92:0/4254925593 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7efe380050d0 con 0x7efe4015c300 2023-11-24T06:10:32.911 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.848+0000 7f6e0f7fe640 1 --2- 172.21.15.92:0/1041297052 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f6df006ae40 0x7f6df006d300 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f6e000011e0 tx=0x7f6e00009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:32.911 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.848+0000 7f24c6ffd640 1 -- 172.21.15.92:0/4217055934 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f24c0013470 con 0x7f24c8053c80 2023-11-24T06:10:32.912 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.849+0000 7efe3d7fa640 1 --2- 172.21.15.92:0/4254925593 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7efe2c06ae40 0x7efe2c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:32.912 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.849+0000 7efe3d7fa640 1 -- 172.21.15.92:0/4254925593 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7efe38089130 con 0x7efe4015c300 2023-11-24T06:10:32.912 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.850+0000 7f24c6ffd640 1 --2- 172.21.15.92:0/4217055934 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f24b406ae40 0x7f24b406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:32.912 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.850+0000 7f24c6ffd640 1 -- 172.21.15.92:0/4217055934 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f24c0089be0 con 0x7f24c8053c80 2023-11-24T06:10:32.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.850+0000 7efe44edf640 1 --2- 172.21.15.92:0/4254925593 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7efe2c06ae40 0x7efe2c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:32.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.850+0000 7f24d64cf640 1 --2- 172.21.15.92:0/4217055934 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f24b406ae40 0x7f24b406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:32.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.851+0000 7f49de7fc640 1 -- 172.21.15.92:0/504994497 <== mon.1 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f49d8053ec0 con 0x7f49e00cf6d0 2023-11-24T06:10:32.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.851+0000 7efe44edf640 1 --2- 172.21.15.92:0/4254925593 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7efe2c06ae40 0x7efe2c06d300 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7efe30036000 tx=0x7efe3003b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:32.914 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.852+0000 7f24d64cf640 1 --2- 172.21.15.92:0/4217055934 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f24b406ae40 0x7f24b406d300 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f24cc009b60 tx=0x7f24cc009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:32.914 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.853+0000 7fab3a7fc640 1 -- 172.21.15.92:0/677833557 <== mon.1 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fab30014b60 con 0x7fab3c0c72c0 2023-11-24T06:10:32.914 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.853+0000 7fab3a7fc640 1 --2- 172.21.15.92:0/677833557 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fab2006ae40 0x7fab2006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:32.915 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.854+0000 7fab3a7fc640 1 -- 172.21.15.92:0/677833557 <== mon.1 v2:172.21.15.118:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fab3008aa70 con 0x7fab3c0c72c0 2023-11-24T06:10:32.915 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.854+0000 7fab41048640 1 --2- 172.21.15.92:0/677833557 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fab2006ae40 0x7fab2006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:32.915 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.855+0000 7fab41048640 1 --2- 172.21.15.92:0/677833557 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fab2006ae40 0x7fab2006d300 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fab28036000 tx=0x7fab28008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:32.915 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.855+0000 7efe3d7fa640 1 -- 172.21.15.92:0/4254925593 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7efe380540a0 con 0x7efe4015c300 2023-11-24T06:10:32.915 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.859+0000 7f6e0cff9640 1 -- 172.21.15.92:0/1041297052 <== mon.2 v2:172.21.15.118:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6e08076d70 con 0x7f6e1015d590 2023-11-24T06:10:32.916 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.863+0000 7f24c6ffd640 1 -- 172.21.15.92:0/4217055934 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f24c80501f0 con 0x7f24c8053c80 2023-11-24T06:10:32.916 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:32.865+0000 7fab3a7fc640 1 -- 172.21.15.92:0/677833557 <== mon.1 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fab30055c90 con 0x7fab3c0c72c0 2023-11-24T06:10:33.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:33.232+0000 7f21aa046640 1 -- 172.21.15.92:0/1926531856 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f21a40675d0 con 0x7f21a415d590 2023-11-24T06:10:33.239 INFO:teuthology.orchestra.run.smithi092.stdout:55834574851 2023-11-24T06:10:33.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:33.233+0000 7f219bfff640 1 -- 172.21.15.92:0/1926531856 <== mon.1 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f219c018090 con 0x7f21a415d590 2023-11-24T06:10:33.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:33.236+0000 7fad138d3640 1 -- 172.21.15.92:0/2810576455 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fad0c067450 con 0x7fad0c0b1f10 2023-11-24T06:10:33.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:33.237+0000 7f21aa046640 1 -- 172.21.15.92:0/1926531856 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f218406af10 msgr2=0x7f218406d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:33.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:33.237+0000 7f21aa046640 1 --2- 172.21.15.92:0/1926531856 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f218406af10 0x7f218406d3d0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f218c000f60 tx=0x7f218c009040 comp rx=0 tx=0).stop 2023-11-24T06:10:33.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:33.237+0000 7f21aa046640 1 -- 172.21.15.92:0/1926531856 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f21a415d590 msgr2=0x7f21a41fcf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:33.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:33.237+0000 7f21aa046640 1 --2- 172.21.15.92:0/1926531856 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f21a415d590 0x7f21a41fcf00 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f219c009f00 tx=0x7f219c00c490 comp rx=0 tx=0).stop 2023-11-24T06:10:33.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:33.237+0000 7f21aa046640 1 -- 172.21.15.92:0/1926531856 shutdown_connections 2023-11-24T06:10:33.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:33.237+0000 7f21aa046640 1 --2- 172.21.15.92:0/1926531856 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f218406af10 0x7f218406d3d0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:33.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:33.237+0000 7f21aa046640 1 --2- 172.21.15.92:0/1926531856 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f21a415d590 0x7f21a41fcf00 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:33.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:33.237+0000 7f21aa046640 1 --2- 172.21.15.92:0/1926531856 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f21a40d0280 0x7f21a40c7530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:33.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:33.237+0000 7f21aa046640 1 --2- 172.21.15.92:0/1926531856 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f21a40ce6f0 0x7f21a409d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:33.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:33.237+0000 7f21aa046640 1 -- 172.21.15.92:0/1926531856 >> 172.21.15.92:0/1926531856 conn(0x7f21a409c540 msgr2=0x7f21a406b2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:33.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:33.238+0000 7fad01ffb640 1 -- 172.21.15.92:0/2810576455 <== mon.2 v2:172.21.15.118:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fad08057350 con 0x7fad0c0b1f10 2023-11-24T06:10:33.242 INFO:teuthology.orchestra.run.smithi092.stdout:55834574851 2023-11-24T06:10:33.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:33.239+0000 7f21aa046640 1 -- 172.21.15.92:0/1926531856 shutdown_connections 2023-11-24T06:10:33.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:33.239+0000 7f21aa046640 1 -- 172.21.15.92:0/1926531856 wait complete. 2023-11-24T06:10:33.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:33.241+0000 7fad138d3640 1 -- 172.21.15.92:0/2810576455 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7facf006ae70 msgr2=0x7facf006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:33.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:33.241+0000 7fad138d3640 1 --2- 172.21.15.92:0/2810576455 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7facf006ae70 0x7facf006d330 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7facfc036000 tx=0x7facfc03d040 comp rx=0 tx=0).stop 2023-11-24T06:10:33.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:33.241+0000 7fad138d3640 1 -- 172.21.15.92:0/2810576455 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fad0c0b1f10 msgr2=0x7fad0c0c92b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:33.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:33.241+0000 7fad138d3640 1 --2- 172.21.15.92:0/2810576455 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fad0c0b1f10 0x7fad0c0c92b0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fad0800c410 tx=0x7fad0800c8d0 comp rx=0 tx=0).stop 2023-11-24T06:10:33.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:33.242+0000 7fad138d3640 1 -- 172.21.15.92:0/2810576455 shutdown_connections 2023-11-24T06:10:33.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:33.242+0000 7fad138d3640 1 --2- 172.21.15.92:0/2810576455 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7facf006ae70 0x7facf006d330 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:33.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:33.242+0000 7fad138d3640 1 --2- 172.21.15.92:0/2810576455 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fad0c15d3b0 0x7fad0c0cf6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:33.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:33.242+0000 7fad138d3640 1 --2- 172.21.15.92:0/2810576455 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fad0c0d1830 0x7fad0c152870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:33.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:33.242+0000 7fad138d3640 1 --2- 172.21.15.92:0/2810576455 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fad0c0b1f10 0x7fad0c0c92b0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:33.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:33.242+0000 7fad138d3640 1 -- 172.21.15.92:0/2810576455 >> 172.21.15.92:0/2810576455 conn(0x7fad0c09c370 msgr2=0x7fad0c0d0bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:33.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:33.243+0000 7fad138d3640 1 -- 172.21.15.92:0/2810576455 shutdown_connections 2023-11-24T06:10:33.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:33.243+0000 7fad138d3640 1 -- 172.21.15.92:0/2810576455 wait complete. 2023-11-24T06:10:33.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:33.253+0000 7f11fb63d640 1 -- 172.21.15.92:0/3859295851 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f11d40057d0 con 0x7f11f415bb50 2023-11-24T06:10:33.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:33.254+0000 7f11e9ffb640 1 -- 172.21.15.92:0/3859295851 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f11ec019020 con 0x7f11f415bb50 2023-11-24T06:10:33.260 INFO:teuthology.orchestra.run.smithi092.stdout:55834574852 2023-11-24T06:10:33.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:33.258+0000 7f11fb63d640 1 -- 172.21.15.92:0/3859295851 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f11d806af10 msgr2=0x7f11d806d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:33.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:33.258+0000 7f11fb63d640 1 --2- 172.21.15.92:0/3859295851 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f11d806af10 0x7f11d806d3d0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f11f0036000 tx=0x7f11f003c040 comp rx=0 tx=0).stop 2023-11-24T06:10:33.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:33.258+0000 7f11fb63d640 1 -- 172.21.15.92:0/3859295851 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f11f415bb50 msgr2=0x7f11f40cc5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:33.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:33.258+0000 7f11fb63d640 1 --2- 172.21.15.92:0/3859295851 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f11f415bb50 0x7f11f40cc5b0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f11ec00b970 tx=0x7f11ec00be30 comp rx=0 tx=0).stop 2023-11-24T06:10:33.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:33.258+0000 7f11fb63d640 1 -- 172.21.15.92:0/3859295851 shutdown_connections 2023-11-24T06:10:33.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:33.258+0000 7f11fb63d640 1 --2- 172.21.15.92:0/3859295851 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f11d806af10 0x7f11d806d3d0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:33.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:33.258+0000 7f11fb63d640 1 --2- 172.21.15.92:0/3859295851 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f11f415bb50 0x7f11f40cc5b0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:33.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:33.258+0000 7f11fb63d640 1 --2- 172.21.15.92:0/3859295851 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f11f40d00b0 0x7f11f415b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:33.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:33.258+0000 7f11fb63d640 1 --2- 172.21.15.92:0/3859295851 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f11f40ce6f0 0x7f11f40b86d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:33.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:33.258+0000 7f11fb63d640 1 -- 172.21.15.92:0/3859295851 >> 172.21.15.92:0/3859295851 conn(0x7f11f409c540 msgr2=0x7f11f40b71d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:33.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:33.259+0000 7f11fb63d640 1 -- 172.21.15.92:0/3859295851 shutdown_connections 2023-11-24T06:10:33.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:33.259+0000 7f11fb63d640 1 -- 172.21.15.92:0/3859295851 wait complete. 2023-11-24T06:10:33.272 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574851 for osd.1 2023-11-24T06:10:33.272 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574851 for osd.0 2023-11-24T06:10:33.273 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:33.270+0000 7f49e7bbe640 1 -- 172.21.15.92:0/504994497 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f49e0156710 con 0x7f49e00cf6d0 2023-11-24T06:10:33.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:33.271+0000 7f24d74d1640 1 -- 172.21.15.92:0/4217055934 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f24c8005360 con 0x7f24c8053c80 2023-11-24T06:10:33.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:33.271+0000 7f24c6ffd640 1 -- 172.21.15.92:0/4217055934 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f24c0017090 con 0x7f24c8053c80 2023-11-24T06:10:33.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:33.271+0000 7f49de7fc640 1 -- 172.21.15.92:0/504994497 <== mon.1 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f49e0156710 con 0x7f49e00cf6d0 2023-11-24T06:10:33.276 INFO:teuthology.orchestra.run.smithi092.stdout:55834574852 2023-11-24T06:10:33.276 INFO:teuthology.orchestra.run.smithi092.stdout:55834574852 2023-11-24T06:10:33.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:33.274+0000 7f24d74d1640 1 -- 172.21.15.92:0/4217055934 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f24b406ae40 msgr2=0x7f24b406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:33.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:33.274+0000 7f24d74d1640 1 --2- 172.21.15.92:0/4217055934 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f24b406ae40 0x7f24b406d300 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f24cc009b60 tx=0x7f24cc009190 comp rx=0 tx=0).stop 2023-11-24T06:10:33.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:33.274+0000 7f24d74d1640 1 -- 172.21.15.92:0/4217055934 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f24c8053c80 msgr2=0x7f24c81899d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:33.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:33.274+0000 7f24d74d1640 1 --2- 172.21.15.92:0/4217055934 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f24c8053c80 0x7f24c81899d0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f24c000e860 tx=0x7f24c000ed20 comp rx=0 tx=0).stop 2023-11-24T06:10:33.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:33.274+0000 7f24d74d1640 1 -- 172.21.15.92:0/4217055934 shutdown_connections 2023-11-24T06:10:33.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:33.274+0000 7f24d74d1640 1 --2- 172.21.15.92:0/4217055934 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f24b406ae40 0x7f24b406d300 secure :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f24cc009b60 tx=0x7f24cc009190 comp rx=0 tx=0).stop 2023-11-24T06:10:33.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:33.274+0000 7f24d74d1640 1 --2- 172.21.15.92:0/4217055934 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f24c80e3130 0x7f24c8185260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:33.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:33.274+0000 7f24d74d1640 1 --2- 172.21.15.92:0/4217055934 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f24c8053c80 0x7f24c81899d0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:33.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:33.274+0000 7f24d74d1640 1 --2- 172.21.15.92:0/4217055934 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f24c803c070 0x7f24c80e4e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:33.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:33.274+0000 7f24d74d1640 1 -- 172.21.15.92:0/4217055934 >> 172.21.15.92:0/4217055934 conn(0x7f24c8058eb0 msgr2=0x7f24c8006980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:33.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:33.275+0000 7f49e7bbe640 1 -- 172.21.15.92:0/504994497 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f49c406af10 msgr2=0x7f49c406d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:33.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:33.275+0000 7f24d74d1640 1 -- 172.21.15.92:0/4217055934 shutdown_connections 2023-11-24T06:10:33.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:33.275+0000 7f49e7bbe640 1 --2- 172.21.15.92:0/504994497 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f49c406af10 0x7f49c406d3d0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f49d4036000 tx=0x7f49d4030000 comp rx=0 tx=0).stop 2023-11-24T06:10:33.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:33.275+0000 7f24d74d1640 1 -- 172.21.15.92:0/4217055934 wait complete. 2023-11-24T06:10:33.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:33.275+0000 7f49e7bbe640 1 -- 172.21.15.92:0/504994497 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f49e00cf6d0 msgr2=0x7f49e0140bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:33.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:33.275+0000 7f49e7bbe640 1 --2- 172.21.15.92:0/504994497 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f49e00cf6d0 0x7f49e0140bf0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f49d800c3c0 tx=0x7f49d800c880 comp rx=0 tx=0).stop 2023-11-24T06:10:33.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:33.275+0000 7f49e7bbe640 1 -- 172.21.15.92:0/504994497 shutdown_connections 2023-11-24T06:10:33.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:33.275+0000 7f49e7bbe640 1 --2- 172.21.15.92:0/504994497 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f49c406af10 0x7f49c406d3d0 secure :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f49d4036000 tx=0x7f49d4030000 comp rx=0 tx=0).stop 2023-11-24T06:10:33.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:33.275+0000 7f49e7bbe640 1 --2- 172.21.15.92:0/504994497 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f49e00cf6d0 0x7f49e0140bf0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:33.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:33.275+0000 7f49e7bbe640 1 --2- 172.21.15.92:0/504994497 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f49e00cf290 0x7f49e01406b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:33.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:33.275+0000 7f49e7bbe640 1 --2- 172.21.15.92:0/504994497 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f49e0072230 0x7f49e01538c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:33.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:33.275+0000 7f49e7bbe640 1 -- 172.21.15.92:0/504994497 >> 172.21.15.92:0/504994497 conn(0x7f49e006a800 msgr2=0x7f49e015bd50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:33.282 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:33.277+0000 7efe47143640 1 -- 172.21.15.92:0/4254925593 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7efe200057d0 con 0x7efe4015c300 2023-11-24T06:10:33.282 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:33.277+0000 7f49e7bbe640 1 -- 172.21.15.92:0/504994497 shutdown_connections 2023-11-24T06:10:33.282 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:33.277+0000 7efe3d7fa640 1 -- 172.21.15.92:0/4254925593 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7efe380576c0 con 0x7efe4015c300 2023-11-24T06:10:33.283 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:33.277+0000 7f49e7bbe640 1 -- 172.21.15.92:0/504994497 wait complete. 2023-11-24T06:10:33.283 INFO:teuthology.orchestra.run.smithi092.stdout:55834574851 2023-11-24T06:10:33.283 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:33.279+0000 7efe47143640 1 -- 172.21.15.92:0/4254925593 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7efe2c06ae40 msgr2=0x7efe2c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:33.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:33.279+0000 7efe47143640 1 --2- 172.21.15.92:0/4254925593 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7efe2c06ae40 0x7efe2c06d300 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7efe30036000 tx=0x7efe3003b040 comp rx=0 tx=0).stop 2023-11-24T06:10:33.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:33.279+0000 7efe47143640 1 -- 172.21.15.92:0/4254925593 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efe4015c300 msgr2=0x7efe4015aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:33.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:33.279+0000 7efe47143640 1 --2- 172.21.15.92:0/4254925593 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efe4015c300 0x7efe4015aa10 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7efe3800c700 tx=0x7efe3800cbc0 comp rx=0 tx=0).stop 2023-11-24T06:10:33.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:33.280+0000 7efe47143640 1 -- 172.21.15.92:0/4254925593 shutdown_connections 2023-11-24T06:10:33.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:33.280+0000 7efe47143640 1 --2- 172.21.15.92:0/4254925593 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7efe2c06ae40 0x7efe2c06d300 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:33.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:33.280+0000 7efe47143640 1 --2- 172.21.15.92:0/4254925593 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efe4015c300 0x7efe4015aa10 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:33.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:33.280+0000 7fab43aad640 1 -- 172.21.15.92:0/677833557 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7fab3c0d4140 con 0x7fab3c0c72c0 2023-11-24T06:10:33.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:33.280+0000 7efe47143640 1 --2- 172.21.15.92:0/4254925593 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7efe40159040 0x7efe400b24f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:33.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:33.280+0000 7efe47143640 1 --2- 172.21.15.92:0/4254925593 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7efe400b34e0 0x7efe4014aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:33.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:33.280+0000 7efe47143640 1 -- 172.21.15.92:0/4254925593 >> 172.21.15.92:0/4254925593 conn(0x7efe40066690 msgr2=0x7efe400ccc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:33.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:33.280+0000 7efe47143640 1 -- 172.21.15.92:0/4254925593 shutdown_connections 2023-11-24T06:10:33.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:33.280+0000 7efe47143640 1 -- 172.21.15.92:0/4254925593 wait complete. 2023-11-24T06:10:33.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:33.281+0000 7fab3a7fc640 1 -- 172.21.15.92:0/677833557 <== mon.1 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fab300592b0 con 0x7fab3c0c72c0 2023-11-24T06:10:33.286 INFO:teuthology.orchestra.run.smithi092.stdout:55834574852 2023-11-24T06:10:33.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:33.282+0000 7f6e1611d640 1 -- 172.21.15.92:0/1041297052 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f6e1015f900 con 0x7f6e1015d590 2023-11-24T06:10:33.287 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574852 for osd.6 2023-11-24T06:10:33.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:33.283+0000 7f6e0cff9640 1 -- 172.21.15.92:0/1041297052 <== mon.2 v2:172.21.15.118:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f6e1015f900 con 0x7f6e1015d590 2023-11-24T06:10:33.288 INFO:teuthology.orchestra.run.smithi092.stdout:55834574852 2023-11-24T06:10:33.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:33.284+0000 7fab43aad640 1 -- 172.21.15.92:0/677833557 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fab2006ae40 msgr2=0x7fab2006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:33.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:33.284+0000 7fab43aad640 1 --2- 172.21.15.92:0/677833557 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fab2006ae40 0x7fab2006d300 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fab28036000 tx=0x7fab28008030 comp rx=0 tx=0).stop 2023-11-24T06:10:33.289 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:33.284+0000 7fab43aad640 1 -- 172.21.15.92:0/677833557 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fab3c0c72c0 msgr2=0x7fab3c0b5f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:33.289 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:33.284+0000 7fab43aad640 1 --2- 172.21.15.92:0/677833557 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fab3c0c72c0 0x7fab3c0b5f30 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fab3000efc0 tx=0x7fab3000c490 comp rx=0 tx=0).stop 2023-11-24T06:10:33.289 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:33.284+0000 7fab43aad640 1 -- 172.21.15.92:0/677833557 shutdown_connections 2023-11-24T06:10:33.289 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:33.284+0000 7fab43aad640 1 --2- 172.21.15.92:0/677833557 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fab2006ae40 0x7fab2006d300 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:33.289 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:33.284+0000 7fab43aad640 1 --2- 172.21.15.92:0/677833557 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fab3c15d700 0x7fab3c0ccae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:33.289 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:33.284+0000 7fab43aad640 1 --2- 172.21.15.92:0/677833557 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fab3c15d320 0x7fab3c0b8760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:33.289 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:33.284+0000 7fab43aad640 1 --2- 172.21.15.92:0/677833557 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fab3c0c72c0 0x7fab3c0b5f30 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:33.289 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:33.284+0000 7fab43aad640 1 -- 172.21.15.92:0/677833557 >> 172.21.15.92:0/677833557 conn(0x7fab3c08cc50 msgr2=0x7fab3c15c8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:33.289 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:33.284+0000 7fab43aad640 1 -- 172.21.15.92:0/677833557 shutdown_connections 2023-11-24T06:10:33.289 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:33.284+0000 7fab43aad640 1 -- 172.21.15.92:0/677833557 wait complete. 2023-11-24T06:10:33.290 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:33.286+0000 7f6e1611d640 1 -- 172.21.15.92:0/1041297052 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f6df006ae40 msgr2=0x7f6df006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:33.291 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:33.286+0000 7f6e1611d640 1 --2- 172.21.15.92:0/1041297052 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f6df006ae40 0x7f6df006d300 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f6e000011e0 tx=0x7f6e00009040 comp rx=0 tx=0).stop 2023-11-24T06:10:33.291 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:33.286+0000 7f6e1611d640 1 -- 172.21.15.92:0/1041297052 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6e1015d590 msgr2=0x7f6e100b56d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:33.291 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:33.286+0000 7f6e1611d640 1 --2- 172.21.15.92:0/1041297052 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6e1015d590 0x7f6e100b56d0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f6e08009eb0 tx=0x7f6e08004a40 comp rx=0 tx=0).stop 2023-11-24T06:10:33.291 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:33.286+0000 7f6e1611d640 1 -- 172.21.15.92:0/1041297052 shutdown_connections 2023-11-24T06:10:33.291 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:33.286+0000 7f6e1611d640 1 --2- 172.21.15.92:0/1041297052 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f6df006ae40 0x7f6df006d300 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:33.291 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:33.286+0000 7f6e1611d640 1 --2- 172.21.15.92:0/1041297052 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6e1015d590 0x7f6e100b56d0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:33.291 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:33.286+0000 7f6e1611d640 1 --2- 172.21.15.92:0/1041297052 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6e100d0280 0x7f6e100b5190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:33.291 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:33.286+0000 7f6e1611d640 1 --2- 172.21.15.92:0/1041297052 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6e100ce6f0 0x7f6e1015bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:33.291 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:33.286+0000 7f6e1611d640 1 -- 172.21.15.92:0/1041297052 >> 172.21.15.92:0/1041297052 conn(0x7f6e1009c540 msgr2=0x7f6e100d3800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:33.291 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:33.287+0000 7f6e1611d640 1 -- 172.21.15.92:0/1041297052 shutdown_connections 2023-11-24T06:10:33.291 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:33.287+0000 7f6e1611d640 1 -- 172.21.15.92:0/1041297052 wait complete. 2023-11-24T06:10:33.302 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574852 for osd.3 2023-11-24T06:10:33.306 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574852 for osd.2 2023-11-24T06:10:33.310 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574852 for osd.4 2023-11-24T06:10:33.310 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574851 for osd.7 2023-11-24T06:10:33.313 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574852 for osd.5 2023-11-24T06:10:34.273 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-11-24T06:10:34.273 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-11-24T06:10:34.288 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-11-24T06:10:34.303 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-11-24T06:10:34.307 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-11-24T06:10:34.311 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-11-24T06:10:34.311 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-11-24T06:10:34.314 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-11-24T06:10:34.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.546+0000 7faed7015640 1 Processor -- start 2023-11-24T06:10:34.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.546+0000 7faed7015640 1 -- start start 2023-11-24T06:10:34.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.549+0000 7faed7015640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7faec80e3120 0x7faec80e7740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:34.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.549+0000 7faed7015640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7faec8053c70 0x7faec80540e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:34.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.550+0000 7faed7015640 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faec8054620 con 0x7faec803c060 2023-11-24T06:10:34.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.550+0000 7faecffff640 1 --1- >> v1:172.21.15.118:6789/0 conn(0x7faec803c060 0x7faec8058010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.118:6789/0 says I am v1:172.21.15.92:56862/0 (socket says 172.21.15.92:56862) 2023-11-24T06:10:34.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.550+0000 7faecffff640 1 -- 172.21.15.92:0/2494983708 learned_addr learned my addr 172.21.15.92:0/2494983708 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-24T06:10:34.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.551+0000 7faed7015640 1 -- 172.21.15.92:0/2494983708 --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faec80587d0 con 0x7faec8053c70 2023-11-24T06:10:34.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.551+0000 7faed7015640 1 -- 172.21.15.92:0/2494983708 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faec8058ab0 con 0x7faec80e3120 2023-11-24T06:10:34.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.550+0000 7faed4db1640 1 --2- 172.21.15.92:0/2494983708 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7faec80e3120 0x7faec80e7740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:34.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.550+0000 7faed55b2640 1 --2- 172.21.15.92:0/2494983708 >> v2:172.21.15.118:3300/0 conn(0x7faec8053c70 0x7faec80540e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:34.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.551+0000 7faecf7fe640 1 -- 172.21.15.92:0/2494983708 <== mon.1 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2091293248 0 0) 0x7faec8054620 con 0x7faec803c060 2023-11-24T06:10:34.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.551+0000 7faecf7fe640 1 -- 172.21.15.92:0/2494983708 --> v1:172.21.15.118:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faec0003520 con 0x7faec803c060 2023-11-24T06:10:34.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.552+0000 7faed4db1640 1 -- 172.21.15.92:0/2494983708 >> v2:172.21.15.118:3300/0 conn(0x7faec8053c70 msgr2=0x7faec80540e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:34.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.552+0000 7faed4db1640 1 --2- 172.21.15.92:0/2494983708 >> v2:172.21.15.118:3300/0 conn(0x7faec8053c70 0x7faec80540e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:34.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.552+0000 7faecf7fe640 1 -- 172.21.15.92:0/2494983708 <== mon.1 v1:172.21.15.118:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3051368554 0 0) 0x7faec0003520 con 0x7faec803c060 2023-11-24T06:10:34.556 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.552+0000 7faed4db1640 1 -- 172.21.15.92:0/2494983708 >> v1:172.21.15.118:6789/0 conn(0x7faec803c060 legacy=0x7faec8058010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:34.557 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.553+0000 7faed4db1640 1 -- 172.21.15.92:0/2494983708 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faec8006320 con 0x7faec80e3120 2023-11-24T06:10:34.557 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.554+0000 7faed4db1640 1 --2- 172.21.15.92:0/2494983708 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7faec80e3120 0x7faec80e7740 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7faebc0097a0 tx=0x7faebc02b790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b1c09b7e71417196 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:34.558 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.555+0000 7faecf7fe640 1 -- 172.21.15.92:0/2494983708 <== mon.2 v2:172.21.15.118:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faed004f090 con 0x7faec80e3120 2023-11-24T06:10:34.558 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.555+0000 7faecf7fe640 1 -- 172.21.15.92:0/2494983708 <== mon.2 v2:172.21.15.118:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faebc02e050 con 0x7faec80e3120 2023-11-24T06:10:34.559 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.556+0000 7faed7015640 1 -- 172.21.15.92:0/2494983708 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7faec80e3120 msgr2=0x7faec80e7740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:34.559 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.556+0000 7faed7015640 1 --2- 172.21.15.92:0/2494983708 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7faec80e3120 0x7faec80e7740 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7faebc0097a0 tx=0x7faebc02b790 comp rx=0 tx=0).stop 2023-11-24T06:10:34.560 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.557+0000 7faed7015640 1 -- 172.21.15.92:0/2494983708 shutdown_connections 2023-11-24T06:10:34.560 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.557+0000 7faed7015640 1 --2- 172.21.15.92:0/2494983708 >> v2:172.21.15.118:3300/0 conn(0x7faec8053c70 0x7faec80540e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:34.560 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.557+0000 7faed7015640 1 --2- 172.21.15.92:0/2494983708 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7faec80e3120 0x7faec80e7740 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:34.560 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.557+0000 7faed7015640 1 -- 172.21.15.92:0/2494983708 >> 172.21.15.92:0/2494983708 conn(0x7faec8058ea0 msgr2=0x7faec8059280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:34.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.558+0000 7faed7015640 1 -- 172.21.15.92:0/2494983708 shutdown_connections 2023-11-24T06:10:34.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.558+0000 7faed7015640 1 -- 172.21.15.92:0/2494983708 wait complete. 2023-11-24T06:10:34.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.560+0000 7f06d143d640 1 Processor -- start 2023-11-24T06:10:34.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.560+0000 7f06d143d640 1 -- start start 2023-11-24T06:10:34.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.561+0000 7faed7015640 1 Processor -- start 2023-11-24T06:10:34.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.561+0000 7faed7015640 1 -- start start 2023-11-24T06:10:34.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.563+0000 7f06d143d640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f06cc153a50 0x7f06cc0b2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:34.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.563+0000 7faed7015640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7faec803c060 0x7faec804ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:34.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.563+0000 7faed7015640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7faec8053c70 0x7faec804f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:34.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.563+0000 7faed7015640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faec80e3120 0x7faec817c040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:34.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.563+0000 7f06d143d640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06cc159e20 con 0x7f06cc1598d0 2023-11-24T06:10:34.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.564+0000 7f06d143d640 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06cc15a100 con 0x7f06cc0cfad0 2023-11-24T06:10:34.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.564+0000 7f06caffd640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f06cc153a50 0x7f06cc0b2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:34.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.564+0000 7faed55b2640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faec80e3120 0x7faec817c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:34.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.564+0000 7faed55b2640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faec80e3120 0x7faec817c040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47454/0 (socket says 172.21.15.92:47454) 2023-11-24T06:10:34.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.564+0000 7faecffff640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7faec8053c70 0x7faec804f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:34.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.565+0000 7faecffff640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7faec8053c70 0x7faec804f3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3301/0 says I am v2:172.21.15.92:39690/0 (socket says 172.21.15.92:39690) 2023-11-24T06:10:34.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.564+0000 7f06cb7fe640 1 --1- >> v1:172.21.15.118:6789/0 conn(0x7f06cc0cfad0 0x7f06cc070e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.118:6789/0 says I am v1:172.21.15.92:56870/0 (socket says 172.21.15.92:56870) 2023-11-24T06:10:34.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.564+0000 7f06cb7fe640 1 -- 172.21.15.92:0/1094986899 learned_addr learned my addr 172.21.15.92:0/1094986899 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-24T06:10:34.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.564+0000 7f06d143d640 1 -- 172.21.15.92:0/1094986899 --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06cc153e30 con 0x7f06cc153a50 2023-11-24T06:10:34.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.565+0000 7faed4db1640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7faec803c060 0x7faec804ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:34.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.565+0000 7f06c9ffb640 1 -- 172.21.15.92:0/1094986899 <== mon.1 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2168000063 0 0) 0x7f06cc15a100 con 0x7f06cc0cfad0 2023-11-24T06:10:34.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.565+0000 7f06c9ffb640 1 -- 172.21.15.92:0/1094986899 --> v1:172.21.15.118:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f06b4003520 con 0x7f06cc0cfad0 2023-11-24T06:10:34.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.565+0000 7faed55b2640 1 -- 172.21.15.92:0/2717557770 learned_addr learned my addr 172.21.15.92:0/2717557770 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:10:34.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.565+0000 7faed7015640 1 -- 172.21.15.92:0/2717557770 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faec804dc20 con 0x7faec80e3120 2023-11-24T06:10:34.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.565+0000 7faed7015640 1 -- 172.21.15.92:0/2717557770 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faec804df00 con 0x7faec803c060 2023-11-24T06:10:34.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.565+0000 7faed7015640 1 -- 172.21.15.92:0/2717557770 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faec804e1e0 con 0x7faec8053c70 2023-11-24T06:10:34.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.565+0000 7f06caffd640 1 -- 172.21.15.92:0/1094986899 >> v1:172.21.15.118:6789/0 conn(0x7f06cc0cfad0 legacy=0x7f06cc070e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:34.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.567+0000 7faed55b2640 1 -- 172.21.15.92:0/2717557770 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7faec8053c70 msgr2=0x7faec804f3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:34.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.567+0000 7faed55b2640 1 --2- 172.21.15.92:0/2717557770 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7faec8053c70 0x7faec804f3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:34.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.567+0000 7faed55b2640 1 -- 172.21.15.92:0/2717557770 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7faec803c060 msgr2=0x7faec804ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:34.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.567+0000 7faed55b2640 1 --2- 172.21.15.92:0/2717557770 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7faec803c060 0x7faec804ee90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:34.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.567+0000 7faed55b2640 1 -- 172.21.15.92:0/2717557770 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faebc009400 con 0x7faec80e3120 2023-11-24T06:10:34.572 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.566+0000 7f06caffd640 1 -- 172.21.15.92:0/1094986899 >> v1:172.21.15.92:6789/0 conn(0x7f06cc1598d0 legacy=0x7f06cc0cf3c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-24T06:10:34.572 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.566+0000 7f06caffd640 1 -- 172.21.15.92:0/1094986899 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06cc15a3e0 con 0x7f06cc153a50 2023-11-24T06:10:34.572 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.568+0000 7faed55b2640 1 --2- 172.21.15.92:0/2717557770 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faec80e3120 0x7faec817c040 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7faec400e830 tx=0x7faec400ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:34.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.567+0000 7f06caffd640 1 --2- 172.21.15.92:0/1094986899 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f06cc153a50 0x7f06cc0b2ab0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f06bc0071d0 tx=0x7f06bc02ca80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e13a0d4f4f9afeff server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:34.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.568+0000 7f06c9ffb640 1 -- 172.21.15.92:0/1094986899 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06bc008df0 con 0x7f06cc153a50 2023-11-24T06:10:34.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.568+0000 7f06c9ffb640 1 -- 172.21.15.92:0/1094986899 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06bc00a0d0 con 0x7f06cc153a50 2023-11-24T06:10:34.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.568+0000 7f06c9ffb640 1 -- 172.21.15.92:0/1094986899 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06bc00a440 con 0x7f06cc153a50 2023-11-24T06:10:34.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.568+0000 7faecd7fa640 1 -- 172.21.15.92:0/2717557770 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faec4009610 con 0x7faec80e3120 2023-11-24T06:10:34.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.569+0000 7f06d143d640 1 -- 172.21.15.92:0/1094986899 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f06cc153a50 msgr2=0x7f06cc0b2ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:34.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.569+0000 7f06d143d640 1 --2- 172.21.15.92:0/1094986899 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f06cc153a50 0x7f06cc0b2ab0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f06bc0071d0 tx=0x7f06bc02ca80 comp rx=0 tx=0).stop 2023-11-24T06:10:34.574 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.569+0000 7faed7015640 1 -- 172.21.15.92:0/2717557770 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faec804e4c0 con 0x7faec80e3120 2023-11-24T06:10:34.574 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.569+0000 7faed7015640 1 -- 172.21.15.92:0/2717557770 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faec80d4c80 con 0x7faec80e3120 2023-11-24T06:10:34.574 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.569+0000 7f06d143d640 1 -- 172.21.15.92:0/1094986899 shutdown_connections 2023-11-24T06:10:34.574 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.570+0000 7f06d143d640 1 --2- 172.21.15.92:0/1094986899 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f06cc153a50 0x7f06cc0b2ab0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:34.575 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.569+0000 7faecd7fa640 1 -- 172.21.15.92:0/2717557770 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faec4017c60 con 0x7faec80e3120 2023-11-24T06:10:34.575 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.570+0000 7f06d143d640 1 -- 172.21.15.92:0/1094986899 >> 172.21.15.92:0/1094986899 conn(0x7f06cc06a800 msgr2=0x7f06cc06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:34.575 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.569+0000 7faecd7fa640 1 -- 172.21.15.92:0/2717557770 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faec4009610 con 0x7faec80e3120 2023-11-24T06:10:34.576 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.570+0000 7f06d143d640 1 -- 172.21.15.92:0/1094986899 shutdown_connections 2023-11-24T06:10:34.576 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.570+0000 7f06d143d640 1 -- 172.21.15.92:0/1094986899 wait complete. 2023-11-24T06:10:34.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.572+0000 7faed7015640 1 -- 172.21.15.92:0/2717557770 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faec804e700 con 0x7faec80e3120 2023-11-24T06:10:34.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.574+0000 7f06d143d640 1 Processor -- start 2023-11-24T06:10:34.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.574+0000 7f06d143d640 1 -- start start 2023-11-24T06:10:34.579 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.576+0000 7f06d143d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f06cc0cfad0 0x7f06cc1569e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:34.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.577+0000 7f06d143d640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f06cc153a50 0x7f06cc1ee1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:34.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.577+0000 7f06d143d640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f06cc1598d0 0x7f06cc158630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:34.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.577+0000 7f06d143d640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06cc159440 con 0x7f06cc0cfad0 2023-11-24T06:10:34.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.578+0000 7f06d143d640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06cc1570e0 con 0x7f06cc1598d0 2023-11-24T06:10:34.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.578+0000 7f06d143d640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06cc1573c0 con 0x7f06cc153a50 2023-11-24T06:10:34.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.578+0000 7f06cb7fe640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f06cc1598d0 0x7f06cc158630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:34.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.578+0000 7f06cb7fe640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f06cc1598d0 0x7f06cc158630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.92:58438/0 (socket says 172.21.15.92:58438) 2023-11-24T06:10:34.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.578+0000 7f06cb7fe640 1 -- 172.21.15.92:0/2377317365 learned_addr learned my addr 172.21.15.92:0/2377317365 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:10:34.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.579+0000 7f06ca7fc640 1 --2- 172.21.15.92:0/2377317365 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f06cc153a50 0x7f06cc1ee1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:34.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.579+0000 7f06caffd640 1 --2- 172.21.15.92:0/2377317365 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f06cc0cfad0 0x7f06cc1569e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:34.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.579+0000 7f06cb7fe640 1 -- 172.21.15.92:0/2377317365 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f06cc153a50 msgr2=0x7f06cc1ee1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:34.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.579+0000 7f06cb7fe640 1 --2- 172.21.15.92:0/2377317365 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f06cc153a50 0x7f06cc1ee1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:34.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.579+0000 7f06cb7fe640 1 -- 172.21.15.92:0/2377317365 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f06cc0cfad0 msgr2=0x7f06cc1569e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:34.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.579+0000 7f06cb7fe640 1 --2- 172.21.15.92:0/2377317365 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f06cc0cfad0 0x7f06cc1569e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:34.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.579+0000 7f06cb7fe640 1 -- 172.21.15.92:0/2377317365 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06bc00ae30 con 0x7f06cc1598d0 2023-11-24T06:10:34.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.580+0000 7f06caffd640 1 --2- 172.21.15.92:0/2377317365 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f06cc0cfad0 0x7f06cc1569e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:10:34.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.580+0000 7faecd7fa640 1 -- 172.21.15.92:0/2717557770 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7faec400c6d0 con 0x7faec80e3120 2023-11-24T06:10:34.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.582+0000 7faecd7fa640 1 --2- 172.21.15.92:0/2717557770 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7faeb406aea0 0x7faeb406d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:34.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.582+0000 7faecd7fa640 1 -- 172.21.15.92:0/2717557770 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7faec4091880 con 0x7faec80e3120 2023-11-24T06:10:34.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.583+0000 7f06cb7fe640 1 --2- 172.21.15.92:0/2377317365 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f06cc1598d0 0x7f06cc158630 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f06c400cb40 tx=0x7f06c400a060 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:34.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.583+0000 7faed4db1640 1 --2- 172.21.15.92:0/2717557770 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7faeb406aea0 0x7faeb406d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:34.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.585+0000 7faed4db1640 1 --2- 172.21.15.92:0/2717557770 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7faeb406aea0 0x7faeb406d360 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7faebc034000 tx=0x7faebc02f000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:34.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.585+0000 7f06b37fe640 1 -- 172.21.15.92:0/2377317365 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06c400aca0 con 0x7f06cc1598d0 2023-11-24T06:10:34.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.585+0000 7f06d143d640 1 -- 172.21.15.92:0/2377317365 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06cc1576a0 con 0x7f06cc1598d0 2023-11-24T06:10:34.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.585+0000 7f06d143d640 1 -- 172.21.15.92:0/2377317365 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06cc1ee700 con 0x7f06cc1598d0 2023-11-24T06:10:34.589 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.585+0000 7f06b37fe640 1 -- 172.21.15.92:0/2377317365 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06c400ae00 con 0x7f06cc1598d0 2023-11-24T06:10:34.589 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.585+0000 7f06b37fe640 1 -- 172.21.15.92:0/2377317365 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06c400aca0 con 0x7f06cc1598d0 2023-11-24T06:10:34.590 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.587+0000 7f06d143d640 1 -- 172.21.15.92:0/2377317365 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06ac005250 con 0x7f06cc1598d0 2023-11-24T06:10:34.591 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.588+0000 7faecd7fa640 1 -- 172.21.15.92:0/2717557770 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7faec405ca50 con 0x7faec80e3120 2023-11-24T06:10:34.594 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.590+0000 7f06b37fe640 1 -- 172.21.15.92:0/2377317365 <== mon.1 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f06c4007070 con 0x7f06cc1598d0 2023-11-24T06:10:34.595 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.591+0000 7f06b37fe640 1 --2- 172.21.15.92:0/2377317365 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f06b806aea0 0x7f06b806d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:34.595 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.591+0000 7f06b37fe640 1 -- 172.21.15.92:0/2377317365 <== mon.1 v2:172.21.15.118:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f06c4089570 con 0x7f06cc1598d0 2023-11-24T06:10:34.595 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.592+0000 7f06caffd640 1 --2- 172.21.15.92:0/2377317365 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f06b806aea0 0x7f06b806d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:34.597 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.593+0000 7f06caffd640 1 --2- 172.21.15.92:0/2377317365 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f06b806aea0 0x7f06b806d360 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f06bc036000 tx=0x7f06bc00a000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:34.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.603+0000 7f06b37fe640 1 -- 172.21.15.92:0/2377317365 <== mon.1 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f06c4054740 con 0x7f06cc1598d0 2023-11-24T06:10:34.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.611+0000 7f0b19a15640 1 Processor -- start 2023-11-24T06:10:34.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.611+0000 7f0b19a15640 1 -- start start 2023-11-24T06:10:34.616 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.613+0000 7f0b19a15640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0b140d1c40 0x7f0b140b3fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:34.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.613+0000 7f0b19a15640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b140b3400 con 0x7f0b1415d610 2023-11-24T06:10:34.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.613+0000 7f0b19a15640 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b140b27f0 con 0x7f0b140d17e0 2023-11-24T06:10:34.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.613+0000 7f0b19a15640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b140d1390 con 0x7f0b140d1c40 2023-11-24T06:10:34.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.614+0000 7f0b12ffd640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f0b1415d610 0x7f0b140ce820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:55864/0 (socket says 172.21.15.92:55864) 2023-11-24T06:10:34.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.614+0000 7f0b12ffd640 1 -- 172.21.15.92:0/1684695517 learned_addr learned my addr 172.21.15.92:0/1684695517 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-24T06:10:34.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.615+0000 7f0b137fe640 1 --2- 172.21.15.92:0/1684695517 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0b140d1c40 0x7f0b140b3fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:34.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.615+0000 7f0b11ffb640 1 -- 172.21.15.92:0/1684695517 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 728745364 0 0) 0x7f0b140b3400 con 0x7f0b1415d610 2023-11-24T06:10:34.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.615+0000 7f0b11ffb640 1 -- 172.21.15.92:0/1684695517 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0b00003520 con 0x7f0b1415d610 2023-11-24T06:10:34.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.615+0000 7f0b11ffb640 1 -- 172.21.15.92:0/1684695517 <== mon.1 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3631510962 0 0) 0x7f0b140b27f0 con 0x7f0b140d17e0 2023-11-24T06:10:34.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.615+0000 7f0b11ffb640 1 -- 172.21.15.92:0/1684695517 --> v1:172.21.15.118:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0b140b3400 con 0x7f0b140d17e0 2023-11-24T06:10:34.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.615+0000 7f0b11ffb640 1 -- 172.21.15.92:0/1684695517 <== mon.1 v1:172.21.15.118:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 444697304 0 0) 0x7f0b140b3400 con 0x7f0b140d17e0 2023-11-24T06:10:34.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.615+0000 7f0b11ffb640 1 -- 172.21.15.92:0/1684695517 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0b140d1c40 msgr2=0x7f0b140b3fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:34.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.615+0000 7f0b11ffb640 1 --2- 172.21.15.92:0/1684695517 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0b140d1c40 0x7f0b140b3fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:34.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.615+0000 7f0b11ffb640 1 -- 172.21.15.92:0/1684695517 >> v1:172.21.15.92:6789/0 conn(0x7f0b1415d610 legacy=0x7f0b140ce820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:34.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.615+0000 7f0b11ffb640 1 -- 172.21.15.92:0/1684695517 --> v1:172.21.15.118:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b140b2ad0 con 0x7f0b140d17e0 2023-11-24T06:10:34.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.617+0000 7f0b11ffb640 1 -- 172.21.15.92:0/1684695517 <== mon.1 v1:172.21.15.118:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2607661350 0 0) 0x7f0afc0015f0 con 0x7f0b140d17e0 2023-11-24T06:10:34.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.617+0000 7f0b11ffb640 1 -- 172.21.15.92:0/1684695517 >> v1:172.21.15.118:6789/0 conn(0x7f0b140d17e0 legacy=0x7f0b140b3890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:34.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.617+0000 7f0b11ffb640 1 --2- 172.21.15.92:0/1684695517 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0b00004b60 0x7f0b00004f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:34.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.617+0000 7f0b11ffb640 1 --2- 172.21.15.92:0/1684695517 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0b140d1c40 0x7f0b00003760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:34.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.617+0000 7f0b11ffb640 1 --2- 172.21.15.92:0/1684695517 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0b00003ca0 0x7f0b00004120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:34.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.617+0000 7f0b11ffb640 1 -- 172.21.15.92:0/1684695517 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b140d1390 con 0x7f0b00003ca0 2023-11-24T06:10:34.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.617+0000 7f0b11ffb640 1 -- 172.21.15.92:0/1684695517 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b0000ba30 con 0x7f0b00004b60 2023-11-24T06:10:34.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.617+0000 7f0b11ffb640 1 -- 172.21.15.92:0/1684695517 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b0000bbf0 con 0x7f0b140d1c40 2023-11-24T06:10:34.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.617+0000 7f0b12ffd640 1 --2- 172.21.15.92:0/1684695517 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0b00004b60 0x7f0b00004f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:34.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.618+0000 7f0b137fe640 1 --2- 172.21.15.92:0/1684695517 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0b00003ca0 0x7f0b00004120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:34.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.618+0000 7f0b137fe640 1 -- 172.21.15.92:0/1684695517 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0b140d1c40 msgr2=0x7f0b00003760 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-24T06:10:34.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.618+0000 7f0b137fe640 1 --2- 172.21.15.92:0/1684695517 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0b140d1c40 0x7f0b00003760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:34.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.618+0000 7f0b137fe640 1 -- 172.21.15.92:0/1684695517 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0b00004b60 msgr2=0x7f0b00004f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:34.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.618+0000 7f0b137fe640 1 --2- 172.21.15.92:0/1684695517 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0b00004b60 0x7f0b00004f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:34.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.618+0000 7f0b137fe640 1 -- 172.21.15.92:0/1684695517 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b140b3400 con 0x7f0b00003ca0 2023-11-24T06:10:34.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.620+0000 7f0b137fe640 1 --2- 172.21.15.92:0/1684695517 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0b00003ca0 0x7f0b00004120 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f0b04009a90 tx=0x7f0b000057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc76668e10b865cb server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:34.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.621+0000 7f0b11ffb640 1 -- 172.21.15.92:0/1684695517 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b0402c6b0 con 0x7f0b00003ca0 2023-11-24T06:10:34.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.621+0000 7f0b11ffb640 1 -- 172.21.15.92:0/1684695517 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b0402c810 con 0x7f0b00003ca0 2023-11-24T06:10:34.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.622+0000 7f0b19a15640 1 -- 172.21.15.92:0/1684695517 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0b00003ca0 msgr2=0x7f0b00004120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:34.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.622+0000 7f0b19a15640 1 --2- 172.21.15.92:0/1684695517 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0b00003ca0 0x7f0b00004120 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f0b04009a90 tx=0x7f0b000057f0 comp rx=0 tx=0).stop 2023-11-24T06:10:34.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.622+0000 7f0b137fe640 1 -- 172.21.15.92:0/1684695517 reap_dead start 2023-11-24T06:10:34.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.623+0000 7f0b19a15640 1 -- 172.21.15.92:0/1684695517 shutdown_connections 2023-11-24T06:10:34.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.623+0000 7f0b19a15640 1 -- 172.21.15.92:0/1684695517 >> 172.21.15.92:0/1684695517 conn(0x7f0b1409c540 msgr2=0x7f0b140d3200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:34.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.623+0000 7f0b19a15640 1 -- 172.21.15.92:0/1684695517 shutdown_connections 2023-11-24T06:10:34.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.623+0000 7f0b19a15640 1 -- 172.21.15.92:0/1684695517 wait complete. 2023-11-24T06:10:34.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.628+0000 7f0b19a15640 1 Processor -- start 2023-11-24T06:10:34.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.628+0000 7f0b19a15640 1 -- start start 2023-11-24T06:10:34.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.628+0000 7ff4cb941640 1 Processor -- start 2023-11-24T06:10:34.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.628+0000 7ff4cb941640 1 -- start start 2023-11-24T06:10:34.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.629+0000 7f0b19a15640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0b140c89b0 0x7f0b140cca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:34.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.629+0000 7f0b19a15640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0b140ca500 0x7f0b142001c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:34.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.629+0000 7f0b19a15640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0b14200700 0x7f0b141e8bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:34.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.629+0000 7f0b19a15640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b140cd660 con 0x7f0b14200700 2023-11-24T06:10:34.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.629+0000 7f0b19a15640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b1415eda0 con 0x7f0b140ca500 2023-11-24T06:10:34.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.629+0000 7f0b19a15640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b140cd0a0 con 0x7f0b140c89b0 2023-11-24T06:10:34.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.629+0000 7f1ba2148640 1 Processor -- start 2023-11-24T06:10:34.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.629+0000 7f1ba2148640 1 -- start start 2023-11-24T06:10:34.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.630+0000 7f0b137fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0b14200700 0x7f0b141e8bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:34.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.630+0000 7f0b137fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0b14200700 0x7f0b141e8bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47484/0 (socket says 172.21.15.92:47484) 2023-11-24T06:10:34.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.630+0000 7f0b137fe640 1 -- 172.21.15.92:0/3462423445 learned_addr learned my addr 172.21.15.92:0/3462423445 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:10:34.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.630+0000 7ff4cb941640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff4c4162820 0x7ff4c416c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:34.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.631+0000 7f0b12ffd640 1 --2- 172.21.15.92:0/3462423445 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0b140c89b0 0x7f0b140cca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:34.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.631+0000 7f0b127fc640 1 --2- 172.21.15.92:0/3462423445 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0b140ca500 0x7f0b142001c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:34.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.631+0000 7ff4cb941640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4c4161b90 con 0x7ff4c406d3b0 2023-11-24T06:10:34.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.631+0000 7ff4cb941640 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4c4070840 con 0x7ff4c4161d30 2023-11-24T06:10:34.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.631+0000 7ff4cb941640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4c40702c0 con 0x7ff4c4162820 2023-11-24T06:10:34.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.631+0000 7ff4c96dd640 1 --1- >> v1:172.21.15.118:6789/0 conn(0x7ff4c4161d30 0x7ff4c40b3510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.118:6789/0 says I am v1:172.21.15.92:56888/0 (socket says 172.21.15.92:56888) 2023-11-24T06:10:34.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.631+0000 7ff4c96dd640 1 -- 172.21.15.92:0/1308963259 learned_addr learned my addr 172.21.15.92:0/1308963259 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-24T06:10:34.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.631+0000 7ff4c9ede640 1 --2- 172.21.15.92:0/1308963259 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff4c4162820 0x7ff4c416c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:34.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.632+0000 7f0b12ffd640 1 -- 172.21.15.92:0/3462423445 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0b140ca500 msgr2=0x7f0b142001c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:34.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.632+0000 7f0b12ffd640 1 --2- 172.21.15.92:0/3462423445 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0b140ca500 0x7f0b142001c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:34.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.632+0000 7f0b12ffd640 1 -- 172.21.15.92:0/3462423445 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0b14200700 msgr2=0x7f0b141e8bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:34.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.632+0000 7ff4bbfff640 1 -- 172.21.15.92:0/1308963259 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2041056043 0 0) 0x7ff4bc0013c0 con 0x7ff4c406d3b0 2023-11-24T06:10:34.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.632+0000 7ff4bbfff640 1 -- 172.21.15.92:0/1308963259 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff4b0003520 con 0x7ff4c406d3b0 2023-11-24T06:10:34.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.632+0000 7f0b12ffd640 1 --2- 172.21.15.92:0/3462423445 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0b14200700 0x7f0b141e8bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:34.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.632+0000 7ff4c9ede640 1 -- 172.21.15.92:0/1308963259 >> v1:172.21.15.118:6789/0 conn(0x7ff4c4161d30 legacy=0x7ff4c40b3510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:34.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.632+0000 7ff4c9ede640 1 -- 172.21.15.92:0/1308963259 >> v1:172.21.15.92:6789/0 conn(0x7ff4c406d3b0 legacy=0x7ff4c4162110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:34.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.632+0000 7ff4c9ede640 1 -- 172.21.15.92:0/1308963259 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4c4162e80 con 0x7ff4c4162820 2023-11-24T06:10:34.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.632+0000 7f0b12ffd640 1 -- 172.21.15.92:0/3462423445 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b140cd380 con 0x7f0b140c89b0 2023-11-24T06:10:34.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.632+0000 7f0b137fe640 1 --2- 172.21.15.92:0/3462423445 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0b14200700 0x7f0b141e8bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:10:34.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.633+0000 7f0b12ffd640 1 --2- 172.21.15.92:0/3462423445 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0b140c89b0 0x7f0b140cca50 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f0b0c002230 tx=0x7f0b0c00e220 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:34.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.634+0000 7f0b0bfff640 1 -- 172.21.15.92:0/3462423445 <== mon.2 v2:172.21.15.118:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b0c00ec60 con 0x7f0b140c89b0 2023-11-24T06:10:34.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.634+0000 7f1ba2148640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1b9c06aaf0 0x7f1b9c16c630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:34.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.634+0000 7f0b19a15640 1 -- 172.21.15.92:0/3462423445 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b1415f040 con 0x7f0b140c89b0 2023-11-24T06:10:34.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.634+0000 7f0b19a15640 1 -- 172.21.15.92:0/3462423445 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b14200bb0 con 0x7f0b140c89b0 2023-11-24T06:10:34.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.634+0000 7ff4c9ede640 1 --2- 172.21.15.92:0/1308963259 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff4c4162820 0x7ff4c416c2f0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7ff4b40097d0 tx=0x7ff4b402b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b736f6d7963669ea server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:34.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.635+0000 7f0b0bfff640 1 -- 172.21.15.92:0/3462423445 <== mon.2 v2:172.21.15.118:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b0c00ede0 con 0x7f0b140c89b0 2023-11-24T06:10:34.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.635+0000 7f0b0bfff640 1 -- 172.21.15.92:0/3462423445 <== mon.2 v2:172.21.15.118:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b0c013660 con 0x7f0b140c89b0 2023-11-24T06:10:34.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.634+0000 7f1ba2148640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b9c06e870 con 0x7f1b9c14b1a0 2023-11-24T06:10:34.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.636+0000 7ff4bbfff640 1 -- 172.21.15.92:0/1308963259 <== mon.2 v2:172.21.15.118:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4b402d040 con 0x7ff4c4162820 2023-11-24T06:10:34.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.636+0000 7ff4bbfff640 1 -- 172.21.15.92:0/1308963259 <== mon.2 v2:172.21.15.118:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4b4031810 con 0x7ff4c4162820 2023-11-24T06:10:34.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.636+0000 7ff4bbfff640 1 -- 172.21.15.92:0/1308963259 <== mon.2 v2:172.21.15.118:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4b4031ba0 con 0x7ff4c4162820 2023-11-24T06:10:34.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.635+0000 7f1ba2148640 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b9c14b580 con 0x7f1b9c1652a0 2023-11-24T06:10:34.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.635+0000 7f1ba2148640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b9c0d0d40 con 0x7f1b9c06aaf0 2023-11-24T06:10:34.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.636+0000 7f1b9affd640 1 --1- >> v1:172.21.15.118:6789/0 conn(0x7f1b9c1652a0 0x7f1b9c06a3e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.118:6789/0 says I am v1:172.21.15.92:56900/0 (socket says 172.21.15.92:56900) 2023-11-24T06:10:34.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.636+0000 7f1b9affd640 1 -- 172.21.15.92:0/2991064249 learned_addr learned my addr 172.21.15.92:0/2991064249 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-24T06:10:34.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.636+0000 7f0b19a15640 1 -- 172.21.15.92:0/3462423445 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0af8005250 con 0x7f0b140c89b0 2023-11-24T06:10:34.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.636+0000 7f1b9a7fc640 1 -- 172.21.15.92:0/2991064249 <== mon.1 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 433086529 0 0) 0x7f1b9c14b580 con 0x7f1b9c1652a0 2023-11-24T06:10:34.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.637+0000 7f1b9a7fc640 1 -- 172.21.15.92:0/2991064249 --> v1:172.21.15.118:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1b88003520 con 0x7f1b9c1652a0 2023-11-24T06:10:34.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.637+0000 7ff4cb941640 1 -- 172.21.15.92:0/1308963259 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff4c4162820 msgr2=0x7ff4c416c2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:34.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.637+0000 7ff4cb941640 1 --2- 172.21.15.92:0/1308963259 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff4c4162820 0x7ff4c416c2f0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7ff4b40097d0 tx=0x7ff4b402b8b0 comp rx=0 tx=0).stop 2023-11-24T06:10:34.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.637+0000 7f1b9a7fc640 1 -- 172.21.15.92:0/2991064249 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 155041802 0 0) 0x7f1b9c06e870 con 0x7f1b9c14b1a0 2023-11-24T06:10:34.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.637+0000 7f1b9a7fc640 1 -- 172.21.15.92:0/2991064249 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1b9c14b580 con 0x7f1b9c14b1a0 2023-11-24T06:10:34.645 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.637+0000 7ff4cb941640 1 -- 172.21.15.92:0/1308963259 shutdown_connections 2023-11-24T06:10:34.645 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.637+0000 7ff4cb941640 1 --2- 172.21.15.92:0/1308963259 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff4c4162820 0x7ff4c416c2f0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:34.645 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.637+0000 7ff4cb941640 1 -- 172.21.15.92:0/1308963259 >> 172.21.15.92:0/1308963259 conn(0x7ff4c40b0be0 msgr2=0x7ff4c40b0fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:34.645 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.637+0000 7ff4cb941640 1 -- 172.21.15.92:0/1308963259 shutdown_connections 2023-11-24T06:10:34.645 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.638+0000 7ff4cb941640 1 -- 172.21.15.92:0/1308963259 wait complete. 2023-11-24T06:10:34.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.637+0000 7f1b9a7fc640 1 -- 172.21.15.92:0/2991064249 <== mon.1 v1:172.21.15.118:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3001835699 0 0) 0x7f1b88003520 con 0x7f1b9c1652a0 2023-11-24T06:10:34.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.637+0000 7f1b9a7fc640 1 -- 172.21.15.92:0/2991064249 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1b9c06aaf0 msgr2=0x7f1b9c16c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:34.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.637+0000 7f1b9a7fc640 1 --2- 172.21.15.92:0/2991064249 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1b9c06aaf0 0x7f1b9c16c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:34.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.637+0000 7f1b9a7fc640 1 -- 172.21.15.92:0/2991064249 >> v1:172.21.15.92:6789/0 conn(0x7f1b9c14b1a0 legacy=0x7f1b9c0b3970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:34.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.637+0000 7f1b9a7fc640 1 -- 172.21.15.92:0/2991064249 --> v1:172.21.15.118:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b9c06fd10 con 0x7f1b9c1652a0 2023-11-24T06:10:34.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.638+0000 7f1b9a7fc640 1 -- 172.21.15.92:0/2991064249 <== mon.1 v1:172.21.15.118:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2607661350 0 0) 0x7f1b84001670 con 0x7f1b9c1652a0 2023-11-24T06:10:34.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.638+0000 7f1b9a7fc640 1 -- 172.21.15.92:0/2991064249 >> v1:172.21.15.118:6789/0 conn(0x7f1b9c1652a0 legacy=0x7f1b9c06a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:34.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.638+0000 7f1b9a7fc640 1 --2- 172.21.15.92:0/2991064249 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1b88003b00 0x7f1b88003f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:34.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.638+0000 7f1b9a7fc640 1 --2- 172.21.15.92:0/2991064249 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b88007960 0x7f1b88009d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:34.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.638+0000 7f1b9a7fc640 1 --2- 172.21.15.92:0/2991064249 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1b8800a420 0x7f1b8800c890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:34.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.638+0000 7f1b9a7fc640 1 -- 172.21.15.92:0/2991064249 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b9c0d0d40 con 0x7f1b88007960 2023-11-24T06:10:34.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.638+0000 7f1b9a7fc640 1 -- 172.21.15.92:0/2991064249 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b8800d120 con 0x7f1b88003b00 2023-11-24T06:10:34.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.638+0000 7f1b9a7fc640 1 -- 172.21.15.92:0/2991064249 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b8800d2e0 con 0x7f1b8800a420 2023-11-24T06:10:34.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.639+0000 7f1b9b7fe640 1 --2- 172.21.15.92:0/2991064249 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1b88003b00 0x7f1b88003f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:34.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.639+0000 7f1b9bfff640 1 --2- 172.21.15.92:0/2991064249 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1b8800a420 0x7f1b8800c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:34.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.640+0000 7f4fa096f640 1 Processor -- start 2023-11-24T06:10:34.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.640+0000 7f4fa096f640 1 -- start start 2023-11-24T06:10:34.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.640+0000 7f1b9bfff640 1 -- 172.21.15.92:0/2991064249 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1b88003b00 msgr2=0x7f1b88003f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:34.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.640+0000 7f1b9bfff640 1 --2- 172.21.15.92:0/2991064249 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1b88003b00 0x7f1b88003f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:34.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.640+0000 7f1b9bfff640 1 -- 172.21.15.92:0/2991064249 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b88007960 msgr2=0x7f1b88009d50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-24T06:10:34.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.640+0000 7f1b9affd640 1 --2- 172.21.15.92:0/2991064249 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b88007960 0x7f1b88009d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:34.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.640+0000 7f1b9bfff640 1 --2- 172.21.15.92:0/2991064249 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b88007960 0x7f1b88009d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:34.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.640+0000 7f1b9bfff640 1 -- 172.21.15.92:0/2991064249 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b88003520 con 0x7f1b8800a420 2023-11-24T06:10:34.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.640+0000 7ff4cb941640 1 Processor -- start 2023-11-24T06:10:34.652 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.640+0000 7f0b0bfff640 1 -- 172.21.15.92:0/3462423445 <== mon.2 v2:172.21.15.118:3301/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f0b0c0137c0 con 0x7f0b140c89b0 2023-11-24T06:10:34.652 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.641+0000 7ff4cb941640 1 -- start start 2023-11-24T06:10:34.652 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.641+0000 7fb5662ad640 1 Processor -- start 2023-11-24T06:10:34.652 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.642+0000 7fb5662ad640 1 -- start start 2023-11-24T06:10:34.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.642+0000 7f0b0bfff640 1 --2- 172.21.15.92:0/3462423445 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f0af406ae70 0x7f0af406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:34.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.642+0000 7f4fa096f640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4f9c15d590 0x7f4f9c0cfb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:34.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.642+0000 7f0b0bfff640 1 -- 172.21.15.92:0/3462423445 <== mon.2 v2:172.21.15.118:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0b0c08a580 con 0x7f0b140c89b0 2023-11-24T06:10:34.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.642+0000 7f1b9bfff640 1 --2- 172.21.15.92:0/2991064249 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1b8800a420 0x7f1b8800c890 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f1b9c0d0fd0 tx=0x7f1b880057f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e284ca496de98e96 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:34.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.643+0000 7f4fa096f640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f4f9c0d00b0 0x7f4f9c0ce1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:34.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.642+0000 7ff4cb941640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff4c406d3b0 0x7ff4c40adeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:34.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.642+0000 7ff4cb941640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff4c4161d30 0x7ff4c41fad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:34.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.642+0000 7ff4cb941640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff4c4162820 0x7ff4c4146f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:34.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.642+0000 7ff4cb941640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4c40a9050 con 0x7ff4c4162820 2023-11-24T06:10:34.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.642+0000 7ff4cb941640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4c41fb5e0 con 0x7ff4c4161d30 2023-11-24T06:10:34.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.642+0000 7ff4cb941640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4c41fb8c0 con 0x7ff4c406d3b0 2023-11-24T06:10:34.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.643+0000 7ff4c9ede640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff4c4162820 0x7ff4c4146f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:34.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.643+0000 7ff4c9ede640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff4c4162820 0x7ff4c4146f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47514/0 (socket says 172.21.15.92:47514) 2023-11-24T06:10:34.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.643+0000 7ff4c9ede640 1 -- 172.21.15.92:0/3750574147 learned_addr learned my addr 172.21.15.92:0/3750574147 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:10:34.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.643+0000 7ff4c8edc640 1 --2- 172.21.15.92:0/3750574147 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff4c4161d30 0x7ff4c41fad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:34.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.643+0000 7ff4c96dd640 1 --2- 172.21.15.92:0/3750574147 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff4c406d3b0 0x7ff4c40adeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:34.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.643+0000 7f4f99d9b640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f4f9c0d00b0 0x7f4f9c0ce1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:34.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.643+0000 7f4f9a59c640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4f9c15d590 0x7f4f9c0cfb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:34.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.643+0000 7f4f99d9b640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f4f9c0d00b0 0x7f4f9c0ce1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.92:58470/0 (socket says 172.21.15.92:58470) 2023-11-24T06:10:34.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.643+0000 7f4f99d9b640 1 -- 172.21.15.92:0/2379723035 learned_addr learned my addr 172.21.15.92:0/2379723035 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:10:34.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.644+0000 7ff4c9ede640 1 -- 172.21.15.92:0/3750574147 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff4c406d3b0 msgr2=0x7ff4c40adeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:34.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.644+0000 7ff4c9ede640 1 --2- 172.21.15.92:0/3750574147 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff4c406d3b0 0x7ff4c40adeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:34.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.643+0000 7f4fa096f640 1 -- 172.21.15.92:0/2379723035 --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f9c0b6750 con 0x7f4f9c0ce6f0 2023-11-24T06:10:34.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.644+0000 7ff4c9ede640 1 -- 172.21.15.92:0/3750574147 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff4c4161d30 msgr2=0x7ff4c41fad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:34.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.644+0000 7f4fa096f640 1 -- 172.21.15.92:0/2379723035 --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f9c0d0530 con 0x7f4f9c0d00b0 2023-11-24T06:10:34.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.644+0000 7f4fa096f640 1 -- 172.21.15.92:0/2379723035 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f9c0cf1f0 con 0x7f4f9c15d590 2023-11-24T06:10:34.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.644+0000 7f0b127fc640 1 --2- 172.21.15.92:0/3462423445 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f0af406ae70 0x7f0af406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:34.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.644+0000 7fb5662ad640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fb56006d3b0 0x7fb560162110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:34.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.644+0000 7ff4c9ede640 1 --2- 172.21.15.92:0/3750574147 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff4c4161d30 0x7ff4c41fad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:34.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.644+0000 7ff4c9ede640 1 -- 172.21.15.92:0/3750574147 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4b4009400 con 0x7ff4c4162820 2023-11-24T06:10:34.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.644+0000 7fb5662ad640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb560161b90 con 0x7fb560161d30 2023-11-24T06:10:34.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.644+0000 7fb5662ad640 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb560070840 con 0x7fb560162650 2023-11-24T06:10:34.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.644+0000 7fb5662ad640 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5600702c0 con 0x7fb56006d3b0 2023-11-24T06:10:34.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.644+0000 7fb55ffff640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fb560161d30 0x7fb5600b3510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:55878/0 (socket says 172.21.15.92:55878) 2023-11-24T06:10:34.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.644+0000 7fb55ffff640 1 -- 172.21.15.92:0/1471585567 learned_addr learned my addr 172.21.15.92:0/1471585567 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-24T06:10:34.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.645+0000 7f4f99d9b640 1 -- 172.21.15.92:0/2379723035 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4f9c15d590 msgr2=0x7f4f9c0cfb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:34.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.645+0000 7f4f99d9b640 1 --2- 172.21.15.92:0/2379723035 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4f9c15d590 0x7f4f9c0cfb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:34.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.645+0000 7ff4c9ede640 1 --2- 172.21.15.92:0/3750574147 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff4c4162820 0x7ff4c4146f50 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7ff4b402b8f0 tx=0x7ff4b40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:34.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.645+0000 7f0b127fc640 1 --2- 172.21.15.92:0/3462423445 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f0af406ae70 0x7f0af406d330 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f0afc001050 tx=0x7f0afc009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:34.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.645+0000 7fb5577fe640 1 --2- 172.21.15.92:0/1471585567 >> v2:172.21.15.118:3300/0 conn(0x7fb56006d3b0 0x7fb560162110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:34.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.645+0000 7f4f9959a640 1 -- 172.21.15.92:0/2379723035 <== mon.1 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3155370389 0 0) 0x7f4f9c0b6750 con 0x7f4f9c0ce6f0 2023-11-24T06:10:34.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.645+0000 7f4f99d9b640 1 -- 172.21.15.92:0/2379723035 >> v1:172.21.15.118:6789/0 conn(0x7f4f9c0ce6f0 legacy=0x7f4f9c0d0de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:34.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.645+0000 7f4f99d9b640 1 -- 172.21.15.92:0/2379723035 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f9c0cef10 con 0x7f4f9c0d00b0 2023-11-24T06:10:34.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.645+0000 7ff4b9ffb640 1 -- 172.21.15.92:0/3750574147 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4b403ac60 con 0x7ff4c4162820 2023-11-24T06:10:34.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.646+0000 7ff4b9ffb640 1 -- 172.21.15.92:0/3750574147 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4b403ade0 con 0x7ff4c4162820 2023-11-24T06:10:34.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.646+0000 7ff4b9ffb640 1 -- 172.21.15.92:0/3750574147 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4b4044630 con 0x7ff4c4162820 2023-11-24T06:10:34.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.646+0000 7f1bc780e640 1 Processor -- start 2023-11-24T06:10:34.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.646+0000 7f1bc780e640 1 -- start start 2023-11-24T06:10:34.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.646+0000 7fb5577fe640 1 -- 172.21.15.92:0/1471585567 >> v1:172.21.15.118:6789/0 conn(0x7fb560162650 legacy=0x7fb56016b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:34.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.646+0000 7fb5577fe640 1 -- 172.21.15.92:0/1471585567 >> v1:172.21.15.92:6789/0 conn(0x7fb560161d30 legacy=0x7fb5600b3510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:34.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.646+0000 7fb5577fe640 1 -- 172.21.15.92:0/1471585567 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb560162e80 con 0x7fb56006d3b0 2023-11-24T06:10:34.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.646+0000 7ff4cb941640 1 -- 172.21.15.92:0/3750574147 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4c41fbba0 con 0x7ff4c4162820 2023-11-24T06:10:34.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.646+0000 7ff4cb941640 1 -- 172.21.15.92:0/3750574147 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4c41fb2d0 con 0x7ff4c4162820 2023-11-24T06:10:34.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.647+0000 7f4f99d9b640 1 --2- 172.21.15.92:0/2379723035 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4f9c0d00b0 0x7f4f9c0ce1b0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f4f94009e50 tx=0x7f4f9402bf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3462e906c1d6aa0f server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:34.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.647+0000 7f4f9959a640 1 -- 172.21.15.92:0/2379723035 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f9402cb00 con 0x7f4f9c0d00b0 2023-11-24T06:10:34.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.648+0000 7fb5577fe640 1 --2- 172.21.15.92:0/1471585567 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb56006d3b0 0x7fb560162110 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fb550009720 tx=0x7fb55002b730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f45c7b4675d482ab server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:34.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.647+0000 7ff4cb941640 1 -- 172.21.15.92:0/3750574147 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4c40a9530 con 0x7ff4c4162820 2023-11-24T06:10:34.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.648+0000 7f1bc780e640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f1bc00b16d0 0x7f1bc00cb0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:34.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.648+0000 7f1bc780e640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f1bc01536e0 0x7f1bc015ce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:34.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.648+0000 7f1bc780e640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1bc0153b70 0x7f1bc015d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:34.666 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.648+0000 7f4f9959a640 1 -- 172.21.15.92:0/2379723035 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f9402cc60 con 0x7f4f9c0d00b0 2023-11-24T06:10:34.666 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.648+0000 7f4f9959a640 1 -- 172.21.15.92:0/2379723035 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f940343f0 con 0x7f4f9c0d00b0 2023-11-24T06:10:34.666 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.648+0000 7f4fa096f640 1 -- 172.21.15.92:0/2379723035 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4f9c0d00b0 msgr2=0x7f4f9c0ce1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:34.666 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.648+0000 7f4fa096f640 1 --2- 172.21.15.92:0/2379723035 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4f9c0d00b0 0x7f4f9c0ce1b0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f4f94009e50 tx=0x7f4f9402bf60 comp rx=0 tx=0).stop 2023-11-24T06:10:34.666 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.648+0000 7fb55f7fe640 1 -- 172.21.15.92:0/1471585567 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb550030710 con 0x7fb56006d3b0 2023-11-24T06:10:34.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.648+0000 7f4fa096f640 1 -- 172.21.15.92:0/2379723035 shutdown_connections 2023-11-24T06:10:34.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.648+0000 7f4fa096f640 1 --2- 172.21.15.92:0/2379723035 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4f9c0d00b0 0x7f4f9c0ce1b0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:34.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.649+0000 7f4fa096f640 1 --2- 172.21.15.92:0/2379723035 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4f9c15d590 0x7f4f9c0cfb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:34.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.649+0000 7f4fa096f640 1 -- 172.21.15.92:0/2379723035 >> 172.21.15.92:0/2379723035 conn(0x7f4f9c09c540 msgr2=0x7f4f9c0d3210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:34.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.648+0000 7fb55f7fe640 1 -- 172.21.15.92:0/1471585567 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb55002e050 con 0x7fb56006d3b0 2023-11-24T06:10:34.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.648+0000 7fb55f7fe640 1 -- 172.21.15.92:0/1471585567 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb550030a90 con 0x7fb56006d3b0 2023-11-24T06:10:34.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.648+0000 7f1bc780e640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1bc00b2e50 con 0x7f1bc00b16d0 2023-11-24T06:10:34.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.648+0000 7f1bc780e640 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1bc00b3130 con 0x7f1bc01536e0 2023-11-24T06:10:34.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.648+0000 7f1bc780e640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1bc00b3410 con 0x7f1bc0153b70 2023-11-24T06:10:34.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.648+0000 7f1bc55aa640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f1bc00b16d0 0x7f1bc00cb0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:34.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.648+0000 7f1bc55aa640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f1bc00b16d0 0x7f1bc00cb0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47520/0 (socket says 172.21.15.92:47520) 2023-11-24T06:10:34.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.648+0000 7f1bc55aa640 1 -- 172.21.15.92:0/3973845990 learned_addr learned my addr 172.21.15.92:0/3973845990 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:10:34.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.649+0000 7f1bc4da9640 1 --2- 172.21.15.92:0/3973845990 >> v2:172.21.15.118:3300/0 conn(0x7f1bc01536e0 0x7f1bc015ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:34.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.649+0000 7f1bc55aa640 1 -- 172.21.15.92:0/3973845990 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1bc0153b70 msgr2=0x7f1bc015d3d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-24T06:10:34.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.649+0000 7f1bc55aa640 1 --2- 172.21.15.92:0/3973845990 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1bc0153b70 0x7f1bc015d3d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:34.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.649+0000 7f1bc55aa640 1 -- 172.21.15.92:0/3973845990 >> v2:172.21.15.118:3300/0 conn(0x7f1bc01536e0 msgr2=0x7f1bc015ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:34.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.649+0000 7f1bc55aa640 1 --2- 172.21.15.92:0/3973845990 >> v2:172.21.15.118:3300/0 conn(0x7f1bc01536e0 0x7f1bc015ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:34.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.649+0000 7f1bc55aa640 1 -- 172.21.15.92:0/3973845990 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bc00b36f0 con 0x7f1bc00b16d0 2023-11-24T06:10:34.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.649+0000 7f1bc4da9640 1 --2- 172.21.15.92:0/3973845990 >> v2:172.21.15.118:3300/0 conn(0x7f1bc01536e0 0x7f1bc015ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:10:34.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.649+0000 7f4fa096f640 1 -- 172.21.15.92:0/2379723035 shutdown_connections 2023-11-24T06:10:34.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.649+0000 7f4fa096f640 1 -- 172.21.15.92:0/2379723035 wait complete. 2023-11-24T06:10:34.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.650+0000 7ff4b9ffb640 1 -- 172.21.15.92:0/3750574147 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ff4b404d450 con 0x7ff4c4162820 2023-11-24T06:10:34.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.650+0000 7f1bc55aa640 1 --2- 172.21.15.92:0/3973845990 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1bc00b16d0 0x7f1bc00cb0d0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f1bc0071ec0 tx=0x7f1bb402bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63dfa071cdc023ff server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:34.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.651+0000 7fb5662ad640 1 -- 172.21.15.92:0/1471585567 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb56006d3b0 msgr2=0x7fb560162110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:34.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.651+0000 7fb5662ad640 1 --2- 172.21.15.92:0/1471585567 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb56006d3b0 0x7fb560162110 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fb550009720 tx=0x7fb55002b730 comp rx=0 tx=0).stop 2023-11-24T06:10:34.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.651+0000 7ff4b9ffb640 1 --2- 172.21.15.92:0/3750574147 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7ff4a806ae40 0x7ff4a806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:34.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.651+0000 7fb5662ad640 1 -- 172.21.15.92:0/1471585567 shutdown_connections 2023-11-24T06:10:34.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.651+0000 7fb5662ad640 1 --2- 172.21.15.92:0/1471585567 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb56006d3b0 0x7fb560162110 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:34.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.651+0000 7fb5662ad640 1 -- 172.21.15.92:0/1471585567 >> 172.21.15.92:0/1471585567 conn(0x7fb5600b0be0 msgr2=0x7fb5600b0fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:34.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.651+0000 7ff4b9ffb640 1 -- 172.21.15.92:0/3750574147 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff4b40ab780 con 0x7ff4c4162820 2023-11-24T06:10:34.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.651+0000 7fb5662ad640 1 -- 172.21.15.92:0/1471585567 shutdown_connections 2023-11-24T06:10:34.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.651+0000 7fb5662ad640 1 -- 172.21.15.92:0/1471585567 wait complete. 2023-11-24T06:10:34.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.651+0000 7ff4c96dd640 1 --2- 172.21.15.92:0/3750574147 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7ff4a806ae40 0x7ff4a806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:34.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.652+0000 7ff4c96dd640 1 --2- 172.21.15.92:0/3750574147 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7ff4a806ae40 0x7ff4a806d300 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ff4c0005880 tx=0x7ff4c0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:34.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.652+0000 7f4fa096f640 1 Processor -- start 2023-11-24T06:10:34.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.652+0000 7f4fa096f640 1 -- start start 2023-11-24T06:10:34.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.653+0000 7fb5662ad640 1 Processor -- start 2023-11-24T06:10:34.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.653+0000 7fb5662ad640 1 -- start start 2023-11-24T06:10:34.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.654+0000 7fb5662ad640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb56006d3b0 0x7fb560143590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:34.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.654+0000 7fb5662ad640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb560161d30 0x7fb560159250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:34.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.654+0000 7fb5662ad640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb560162650 0x7fb560159790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:34.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.654+0000 7fb5662ad640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb560159dc0 con 0x7fb56006d3b0 2023-11-24T06:10:34.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.654+0000 7fb5662ad640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb56015a0a0 con 0x7fb560161d30 2023-11-24T06:10:34.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.654+0000 7fb5662ad640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb560143cd0 con 0x7fb560162650 2023-11-24T06:10:34.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.655+0000 7fb55ffff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb56006d3b0 0x7fb560143590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:34.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.655+0000 7fb55ffff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb56006d3b0 0x7fb560143590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47528/0 (socket says 172.21.15.92:47528) 2023-11-24T06:10:34.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.655+0000 7fb55ffff640 1 -- 172.21.15.92:0/4106248893 learned_addr learned my addr 172.21.15.92:0/4106248893 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:10:34.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.655+0000 7fb56484a640 1 --2- 172.21.15.92:0/4106248893 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb560162650 0x7fb560159790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:34.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.655+0000 7fb5577fe640 1 --2- 172.21.15.92:0/4106248893 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb560161d30 0x7fb560159250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:34.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.655+0000 7fb55ffff640 1 -- 172.21.15.92:0/4106248893 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb560162650 msgr2=0x7fb560159790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:34.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.655+0000 7fb55ffff640 1 --2- 172.21.15.92:0/4106248893 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb560162650 0x7fb560159790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:34.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.655+0000 7fb55ffff640 1 -- 172.21.15.92:0/4106248893 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb560161d30 msgr2=0x7fb560159250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:34.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.655+0000 7fb55ffff640 1 --2- 172.21.15.92:0/4106248893 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb560161d30 0x7fb560159250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:34.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.655+0000 7fb55ffff640 1 -- 172.21.15.92:0/4106248893 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb550009400 con 0x7fb56006d3b0 2023-11-24T06:10:34.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.656+0000 7f4fa096f640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4f9c0ce6f0 0x7f4f9c0c8090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:34.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.656+0000 7f4fa096f640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4f9c0d00b0 0x7f4f9c159810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:34.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.656+0000 7f4f9a59c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4f9c0ce6f0 0x7f4f9c0c8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:34.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.656+0000 7f4f9a59c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4f9c0ce6f0 0x7f4f9c0c8090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47540/0 (socket says 172.21.15.92:47540) 2023-11-24T06:10:34.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.656+0000 7f4f9a59c640 1 -- 172.21.15.92:0/659019952 learned_addr learned my addr 172.21.15.92:0/659019952 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:10:34.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.657+0000 7f4fa096f640 1 --2- 172.21.15.92:0/659019952 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4f9c15d590 0x7f4f9c0ca5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:34.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.657+0000 7f4f99d9b640 1 --2- 172.21.15.92:0/659019952 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4f9c0d00b0 0x7f4f9c159810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:34.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.657+0000 7fb55ffff640 1 --2- 172.21.15.92:0/4106248893 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb56006d3b0 0x7fb560143590 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fb54800cba0 tx=0x7fb54800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:34.679 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.657+0000 7f4fa096f640 1 -- 172.21.15.92:0/659019952 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f9c159d50 con 0x7f4f9c0ce6f0 2023-11-24T06:10:34.679 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.657+0000 7f4fa096f640 1 -- 172.21.15.92:0/659019952 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f9c1f3bb0 con 0x7f4f9c0d00b0 2023-11-24T06:10:34.679 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.657+0000 7f4fa096f640 1 -- 172.21.15.92:0/659019952 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f9c1f3030 con 0x7f4f9c15d590 2023-11-24T06:10:34.679 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.657+0000 7f4f9ad9d640 1 --2- 172.21.15.92:0/659019952 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4f9c15d590 0x7f4f9c0ca5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:34.680 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.658+0000 7f4f9a59c640 1 -- 172.21.15.92:0/659019952 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4f9c15d590 msgr2=0x7f4f9c0ca5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:34.680 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.658+0000 7f4f9a59c640 1 --2- 172.21.15.92:0/659019952 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4f9c15d590 0x7f4f9c0ca5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:34.680 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.658+0000 7f4f9a59c640 1 -- 172.21.15.92:0/659019952 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4f9c0d00b0 msgr2=0x7f4f9c159810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:34.680 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.658+0000 7f4f9a59c640 1 --2- 172.21.15.92:0/659019952 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4f9c0d00b0 0x7f4f9c159810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:34.680 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.658+0000 7f4f9a59c640 1 -- 172.21.15.92:0/659019952 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f94009b30 con 0x7f4f9c0ce6f0 2023-11-24T06:10:34.680 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.658+0000 7f4f9a59c640 1 --2- 172.21.15.92:0/659019952 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4f9c0ce6f0 0x7f4f9c0c8090 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f4f8c00e9b0 tx=0x7f4f8c00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:34.680 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.659+0000 7f1b9a7fc640 1 -- 172.21.15.92:0/2991064249 <== mon.2 v2:172.21.15.118:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b9402cc10 con 0x7f1b8800a420 2023-11-24T06:10:34.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.659+0000 7f1b9a7fc640 1 -- 172.21.15.92:0/2991064249 <== mon.2 v2:172.21.15.118:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b9402cd90 con 0x7f1b8800a420 2023-11-24T06:10:34.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.659+0000 7f4f8affd640 1 -- 172.21.15.92:0/659019952 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f8c009670 con 0x7f4f9c0ce6f0 2023-11-24T06:10:34.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.659+0000 7f1b9a7fc640 1 -- 172.21.15.92:0/2991064249 <== mon.2 v2:172.21.15.118:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b94034680 con 0x7f1b8800a420 2023-11-24T06:10:34.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.659+0000 7f4f8affd640 1 -- 172.21.15.92:0/659019952 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f8c00c9a0 con 0x7f4f9c0ce6f0 2023-11-24T06:10:34.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.659+0000 7f4f8affd640 1 -- 172.21.15.92:0/659019952 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f8c00cd50 con 0x7f4f9c0ce6f0 2023-11-24T06:10:34.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.659+0000 7ff4b9ffb640 1 -- 172.21.15.92:0/3750574147 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff4c40a9530 con 0x7ff4c4162820 2023-11-24T06:10:34.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.659+0000 7f4fa096f640 1 -- 172.21.15.92:0/659019952 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f9c1f3310 con 0x7f4f9c0ce6f0 2023-11-24T06:10:34.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.659+0000 7f1bbffff640 1 -- 172.21.15.92:0/3973845990 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bb402cb20 con 0x7f1bc00b16d0 2023-11-24T06:10:34.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.660+0000 7f1bbffff640 1 -- 172.21.15.92:0/3973845990 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bb402cc80 con 0x7f1bc00b16d0 2023-11-24T06:10:34.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.660+0000 7f1bbffff640 1 -- 172.21.15.92:0/3973845990 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bb40343f0 con 0x7f1bc00b16d0 2023-11-24T06:10:34.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.660+0000 7f4fa096f640 1 -- 172.21.15.92:0/659019952 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f9c1f38d0 con 0x7f4f9c0ce6f0 2023-11-24T06:10:34.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.660+0000 7fb55d7fa640 1 -- 172.21.15.92:0/4106248893 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb54800ad40 con 0x7fb56006d3b0 2023-11-24T06:10:34.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.660+0000 7f0b0bfff640 1 -- 172.21.15.92:0/3462423445 <== mon.2 v2:172.21.15.118:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0b0c055750 con 0x7f0b140c89b0 2023-11-24T06:10:34.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.660+0000 7fb55d7fa640 1 -- 172.21.15.92:0/4106248893 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb548012400 con 0x7fb56006d3b0 2023-11-24T06:10:34.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.660+0000 7fb55d7fa640 1 -- 172.21.15.92:0/4106248893 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb54800aea0 con 0x7fb56006d3b0 2023-11-24T06:10:34.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.660+0000 7fb5662ad640 1 -- 172.21.15.92:0/4106248893 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb560143fb0 con 0x7fb56006d3b0 2023-11-24T06:10:34.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.660+0000 7fb5662ad640 1 -- 172.21.15.92:0/4106248893 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb560157090 con 0x7fb56006d3b0 2023-11-24T06:10:34.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.661+0000 7f1ba2148640 1 -- 172.21.15.92:0/2991064249 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1b8800a420 msgr2=0x7f1b8800c890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:34.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.661+0000 7f1ba2148640 1 --2- 172.21.15.92:0/2991064249 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1b8800a420 0x7f1b8800c890 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f1b9c0d0fd0 tx=0x7f1b880057f0 comp rx=0 tx=0).stop 2023-11-24T06:10:34.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.661+0000 7f1bc780e640 1 -- 172.21.15.92:0/3973845990 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1bc00b16d0 msgr2=0x7f1bc00cb0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:34.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.661+0000 7f1bc780e640 1 --2- 172.21.15.92:0/3973845990 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1bc00b16d0 0x7f1bc00cb0d0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f1bc0071ec0 tx=0x7f1bb402bf80 comp rx=0 tx=0).stop 2023-11-24T06:10:34.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.661+0000 7f1b9bfff640 1 -- 172.21.15.92:0/2991064249 reap_dead start 2023-11-24T06:10:34.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.662+0000 7f1ba2148640 1 -- 172.21.15.92:0/2991064249 shutdown_connections 2023-11-24T06:10:34.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.662+0000 7f1ba2148640 1 -- 172.21.15.92:0/2991064249 >> 172.21.15.92:0/2991064249 conn(0x7f1b9c0cfef0 msgr2=0x7f1b9c0d0300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:34.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.662+0000 7f1ba2148640 1 -- 172.21.15.92:0/2991064249 shutdown_connections 2023-11-24T06:10:34.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.662+0000 7f4fa096f640 1 -- 172.21.15.92:0/659019952 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f80005250 con 0x7f4f9c0ce6f0 2023-11-24T06:10:34.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.662+0000 7f1ba2148640 1 -- 172.21.15.92:0/2991064249 wait complete. 2023-11-24T06:10:34.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.662+0000 7f1bc780e640 1 -- 172.21.15.92:0/3973845990 shutdown_connections 2023-11-24T06:10:34.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.662+0000 7f1bc780e640 1 --2- 172.21.15.92:0/3973845990 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1bc0153b70 0x7f1bc015d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:34.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.662+0000 7f1bc780e640 1 --2- 172.21.15.92:0/3973845990 >> v2:172.21.15.118:3300/0 conn(0x7f1bc01536e0 0x7f1bc015ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:34.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.662+0000 7f1bc780e640 1 --2- 172.21.15.92:0/3973845990 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1bc00b16d0 0x7f1bc00cb0d0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:34.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.662+0000 7f1bc780e640 1 -- 172.21.15.92:0/3973845990 >> 172.21.15.92:0/3973845990 conn(0x7f1bc00b0be0 msgr2=0x7f1bc00b0fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:34.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.663+0000 7f1bc780e640 1 -- 172.21.15.92:0/3973845990 shutdown_connections 2023-11-24T06:10:34.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.663+0000 7fb5662ad640 1 -- 172.21.15.92:0/4106248893 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5601444d0 con 0x7fb56006d3b0 2023-11-24T06:10:34.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.665+0000 7f1bc780e640 1 -- 172.21.15.92:0/3973845990 wait complete. 2023-11-24T06:10:34.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.665+0000 7f1ba2148640 1 Processor -- start 2023-11-24T06:10:34.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.667+0000 7f1bc780e640 1 Processor -- start 2023-11-24T06:10:34.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.668+0000 7f1ba2148640 1 -- start start 2023-11-24T06:10:34.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.666+0000 7fb55d7fa640 1 -- 172.21.15.92:0/4106248893 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb548007070 con 0x7fb56006d3b0 2023-11-24T06:10:34.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.668+0000 7fb55d7fa640 1 --2- 172.21.15.92:0/4106248893 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fb54406ae90 0x7fb54406d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:34.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.668+0000 7fb55d7fa640 1 -- 172.21.15.92:0/4106248893 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb548089750 con 0x7fb56006d3b0 2023-11-24T06:10:34.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.669+0000 7fb5577fe640 1 --2- 172.21.15.92:0/4106248893 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fb54406ae90 0x7fb54406d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:34.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.670+0000 7fb5577fe640 1 --2- 172.21.15.92:0/4106248893 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fb54406ae90 0x7fb54406d350 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fb550034000 tx=0x7fb550008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:34.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.669+0000 7f4f8affd640 1 -- 172.21.15.92:0/659019952 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f4f8c013470 con 0x7f4f9c0ce6f0 2023-11-24T06:10:34.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.670+0000 7f1ba2148640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1b8800a420 0x7f1b9c147110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:34.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.670+0000 7f1ba2148640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1b9c1fb270 0x7f1b9c1fa8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:34.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.670+0000 7f1ba2148640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b9c164100 0x7f1b9c0cd940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:34.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.671+0000 7f1ba2148640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b9c163c70 con 0x7f1b9c164100 2023-11-24T06:10:34.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.671+0000 7f1ba2148640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b9c147bd0 con 0x7f1b8800a420 2023-11-24T06:10:34.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.671+0000 7f1ba2148640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b9c147eb0 con 0x7f1b9c1fb270 2023-11-24T06:10:34.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.671+0000 7f1b9bfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b9c164100 0x7f1b9c0cd940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:34.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.671+0000 7f1b9bfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b9c164100 0x7f1b9c0cd940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47542/0 (socket says 172.21.15.92:47542) 2023-11-24T06:10:34.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.671+0000 7f1b9bfff640 1 -- 172.21.15.92:0/510264154 learned_addr learned my addr 172.21.15.92:0/510264154 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:10:34.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.671+0000 7f1b9b7fe640 1 --2- 172.21.15.92:0/510264154 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1b8800a420 0x7f1b9c147110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:34.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.672+0000 7f1b9bfff640 1 -- 172.21.15.92:0/510264154 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1b9c1fb270 msgr2=0x7f1b9c1fa8d0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-11-24T06:10:34.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.672+0000 7f1b9bfff640 1 --2- 172.21.15.92:0/510264154 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1b9c1fb270 0x7f1b9c1fa8d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:34.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.672+0000 7f1b9bfff640 1 -- 172.21.15.92:0/510264154 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1b8800a420 msgr2=0x7f1b9c147110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:34.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.672+0000 7f1b9bfff640 1 --2- 172.21.15.92:0/510264154 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1b8800a420 0x7f1b9c147110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:34.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.672+0000 7f4f8affd640 1 --2- 172.21.15.92:0/659019952 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f4f7c06af10 0x7f4f7c06d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:34.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.672+0000 7f1b9bfff640 1 -- 172.21.15.92:0/510264154 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b9c148190 con 0x7f1b9c164100 2023-11-24T06:10:34.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.673+0000 7f4f8affd640 1 -- 172.21.15.92:0/659019952 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4f8c089f70 con 0x7f4f9c0ce6f0 2023-11-24T06:10:34.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.673+0000 7f4f99d9b640 1 --2- 172.21.15.92:0/659019952 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f4f7c06af10 0x7f4f7c06d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:34.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.673+0000 7f1b9bfff640 1 --2- 172.21.15.92:0/510264154 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b9c164100 0x7f1b9c0cd940 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f1b94009e60 tx=0x7f1b94004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:34.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.668+0000 7f1bc780e640 1 -- start start 2023-11-24T06:10:34.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.669+0000 7f1bc780e640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1bc00b16d0 0x7f1bc014aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:34.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.669+0000 7f1bc780e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1bc01536e0 0x7f1bc0150250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:34.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.669+0000 7f1bc780e640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1bc0159ca0 0x7f1bc015a150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:34.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.669+0000 7f1bc780e640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bc015a690 con 0x7f1bc01536e0 2023-11-24T06:10:34.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.669+0000 7f1bc780e640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bc014af50 con 0x7f1bc00b16d0 2023-11-24T06:10:34.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.669+0000 7f1bc780e640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1bc014e890 con 0x7f1bc0159ca0 2023-11-24T06:10:34.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.674+0000 7f4f99d9b640 1 --2- 172.21.15.92:0/659019952 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f4f7c06af10 0x7f4f7c06d3d0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f4f94035000 tx=0x7f4f94030000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:34.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.675+0000 7f1bc4da9640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1bc01536e0 0x7f1bc0150250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:34.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.675+0000 7f1bc4da9640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1bc01536e0 0x7f1bc0150250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47554/0 (socket says 172.21.15.92:47554) 2023-11-24T06:10:34.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.675+0000 7f1bc4da9640 1 -- 172.21.15.92:0/3446645759 learned_addr learned my addr 172.21.15.92:0/3446645759 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:10:34.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.675+0000 7f1bc5dab640 1 --2- 172.21.15.92:0/3446645759 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1bc0159ca0 0x7f1bc015a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:34.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.675+0000 7f1bc55aa640 1 --2- 172.21.15.92:0/3446645759 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1bc00b16d0 0x7f1bc014aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:34.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.675+0000 7f1bc4da9640 1 -- 172.21.15.92:0/3446645759 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1bc0159ca0 msgr2=0x7f1bc015a150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:34.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.675+0000 7f1bc4da9640 1 --2- 172.21.15.92:0/3446645759 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1bc0159ca0 0x7f1bc015a150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:34.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.675+0000 7f1bc4da9640 1 -- 172.21.15.92:0/3446645759 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1bc00b16d0 msgr2=0x7f1bc014aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:34.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.675+0000 7f1bc4da9640 1 --2- 172.21.15.92:0/3446645759 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1bc00b16d0 0x7f1bc014aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:34.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.675+0000 7f1bc4da9640 1 -- 172.21.15.92:0/3446645759 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bb4009b30 con 0x7f1bc01536e0 2023-11-24T06:10:34.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.676+0000 7f1bc4da9640 1 --2- 172.21.15.92:0/3446645759 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1bc01536e0 0x7f1bc0150250 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f1bac00e810 tx=0x7f1bac00ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:34.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.676+0000 7f4f8affd640 1 -- 172.21.15.92:0/659019952 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4f8c055140 con 0x7f4f9c0ce6f0 2023-11-24T06:10:34.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.685+0000 7f1bbdffb640 1 -- 172.21.15.92:0/3446645759 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bac004030 con 0x7f1bc01536e0 2023-11-24T06:10:34.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.685+0000 7f1bbdffb640 1 -- 172.21.15.92:0/3446645759 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bac004190 con 0x7f1bc01536e0 2023-11-24T06:10:34.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.685+0000 7f1bbdffb640 1 -- 172.21.15.92:0/3446645759 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bac004520 con 0x7f1bc01536e0 2023-11-24T06:10:34.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.685+0000 7fb55d7fa640 1 -- 172.21.15.92:0/4106248893 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb548054890 con 0x7fb56006d3b0 2023-11-24T06:10:34.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.685+0000 7f1bc780e640 1 -- 172.21.15.92:0/3446645759 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1bc014eb30 con 0x7f1bc01536e0 2023-11-24T06:10:34.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.685+0000 7f1bc780e640 1 -- 172.21.15.92:0/3446645759 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1bc014f0f0 con 0x7f1bc01536e0 2023-11-24T06:10:34.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.685+0000 7f1b93fff640 1 -- 172.21.15.92:0/510264154 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b9403e610 con 0x7f1b9c164100 2023-11-24T06:10:34.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.686+0000 7f1b93fff640 1 -- 172.21.15.92:0/510264154 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b94035e90 con 0x7f1b9c164100 2023-11-24T06:10:34.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.686+0000 7f1ba2148640 1 -- 172.21.15.92:0/510264154 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b9c163900 con 0x7f1b9c164100 2023-11-24T06:10:34.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.686+0000 7f1b93fff640 1 -- 172.21.15.92:0/510264154 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b9403e9b0 con 0x7f1b9c164100 2023-11-24T06:10:34.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.686+0000 7f1ba2148640 1 -- 172.21.15.92:0/510264154 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b9c1fae10 con 0x7f1b9c164100 2023-11-24T06:10:34.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.688+0000 7f1ba2148640 1 -- 172.21.15.92:0/510264154 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b9c163de0 con 0x7f1b9c164100 2023-11-24T06:10:34.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.688+0000 7f1bc780e640 1 -- 172.21.15.92:0/3446645759 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1bc00675d0 con 0x7f1bc01536e0 2023-11-24T06:10:34.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.690+0000 7f1bbdffb640 1 -- 172.21.15.92:0/3446645759 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f1bac00c6d0 con 0x7f1bc01536e0 2023-11-24T06:10:34.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.691+0000 7f1bbdffb640 1 --2- 172.21.15.92:0/3446645759 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f1ba406ae90 0x7f1ba406d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:34.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.692+0000 7f1bbdffb640 1 -- 172.21.15.92:0/3446645759 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1bac058040 con 0x7f1bc01536e0 2023-11-24T06:10:34.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.691+0000 7f1b93fff640 1 -- 172.21.15.92:0/510264154 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f1b9403b050 con 0x7f1b9c164100 2023-11-24T06:10:34.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.693+0000 7f1b93fff640 1 --2- 172.21.15.92:0/510264154 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f1b7c06ae90 0x7f1b7c06d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:34.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.694+0000 7f1b93fff640 1 -- 172.21.15.92:0/510264154 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1b94008030 con 0x7f1b9c164100 2023-11-24T06:10:34.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.694+0000 7f1b9b7fe640 1 --2- 172.21.15.92:0/510264154 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f1b7c06ae90 0x7f1b7c06d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:34.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.695+0000 7f1bc55aa640 1 --2- 172.21.15.92:0/3446645759 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f1ba406ae90 0x7f1ba406d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:34.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.696+0000 7f1b9b7fe640 1 --2- 172.21.15.92:0/510264154 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f1b7c06ae90 0x7f1b7c06d350 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f1b8c005530 tx=0x7f1b8c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:34.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.696+0000 7f1bc55aa640 1 --2- 172.21.15.92:0/3446645759 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f1ba406ae90 0x7f1ba406d350 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f1bb4035000 tx=0x7f1bb4030000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:34.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.702+0000 7f1b93fff640 1 -- 172.21.15.92:0/510264154 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1b94076b30 con 0x7f1b9c164100 2023-11-24T06:10:34.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.711+0000 7f1bbdffb640 1 -- 172.21.15.92:0/3446645759 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1bac054b10 con 0x7f1bc01536e0 2023-11-24T06:10:34.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.986+0000 7f06d143d640 1 -- 172.21.15.92:0/2377317365 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f06ac0057d0 con 0x7f06cc1598d0 2023-11-24T06:10:34.990 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.987+0000 7f06b37fe640 1 -- 172.21.15.92:0/2377317365 <== mon.1 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f06c4021090 con 0x7f06cc1598d0 2023-11-24T06:10:34.990 INFO:teuthology.orchestra.run.smithi092.stdout:55834574853 2023-11-24T06:10:34.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.990+0000 7f06d143d640 1 -- 172.21.15.92:0/2377317365 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f06b806aea0 msgr2=0x7f06b806d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:34.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.990+0000 7f06d143d640 1 --2- 172.21.15.92:0/2377317365 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f06b806aea0 0x7f06b806d360 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f06bc036000 tx=0x7f06bc00a000 comp rx=0 tx=0).stop 2023-11-24T06:10:34.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.991+0000 7f06d143d640 1 -- 172.21.15.92:0/2377317365 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f06cc1598d0 msgr2=0x7f06cc158630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:34.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.991+0000 7f06d143d640 1 --2- 172.21.15.92:0/2377317365 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f06cc1598d0 0x7f06cc158630 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f06c400cb40 tx=0x7f06c400a060 comp rx=0 tx=0).stop 2023-11-24T06:10:34.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.991+0000 7f06d143d640 1 -- 172.21.15.92:0/2377317365 shutdown_connections 2023-11-24T06:10:34.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.991+0000 7f06d143d640 1 --2- 172.21.15.92:0/2377317365 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f06b806aea0 0x7f06b806d360 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:34.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.991+0000 7f06d143d640 1 --2- 172.21.15.92:0/2377317365 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f06cc1598d0 0x7f06cc158630 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:34.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.992+0000 7f06d143d640 1 --2- 172.21.15.92:0/2377317365 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f06cc153a50 0x7f06cc1ee1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:34.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.992+0000 7f06d143d640 1 --2- 172.21.15.92:0/2377317365 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f06cc0cfad0 0x7f06cc1569e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:34.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.992+0000 7f06d143d640 1 -- 172.21.15.92:0/2377317365 >> 172.21.15.92:0/2377317365 conn(0x7f06cc06a800 msgr2=0x7f06cc1519e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:34.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.992+0000 7f06d143d640 1 -- 172.21.15.92:0/2377317365 shutdown_connections 2023-11-24T06:10:34.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.992+0000 7f06d143d640 1 -- 172.21.15.92:0/2377317365 wait complete. 2023-11-24T06:10:34.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.994+0000 7faed7015640 1 -- 172.21.15.92:0/2717557770 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7faec804e700 con 0x7faec80e3120 2023-11-24T06:10:34.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.995+0000 7faecd7fa640 1 -- 172.21.15.92:0/2717557770 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7faec4024020 con 0x7faec80e3120 2023-11-24T06:10:34.999 INFO:teuthology.orchestra.run.smithi092.stdout:55834574853 2023-11-24T06:10:35.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.999+0000 7faed7015640 1 -- 172.21.15.92:0/2717557770 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7faeb406aea0 msgr2=0x7faeb406d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:35.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.999+0000 7faed7015640 1 --2- 172.21.15.92:0/2717557770 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7faeb406aea0 0x7faeb406d360 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7faebc034000 tx=0x7faebc02f000 comp rx=0 tx=0).stop 2023-11-24T06:10:35.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:34.999+0000 7faed7015640 1 -- 172.21.15.92:0/2717557770 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faec80e3120 msgr2=0x7faec817c040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:35.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.000+0000 7faed7015640 1 --2- 172.21.15.92:0/2717557770 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faec80e3120 0x7faec817c040 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7faec400e830 tx=0x7faec400ecf0 comp rx=0 tx=0).stop 2023-11-24T06:10:35.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.000+0000 7faed7015640 1 -- 172.21.15.92:0/2717557770 shutdown_connections 2023-11-24T06:10:35.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.001+0000 7faed7015640 1 --2- 172.21.15.92:0/2717557770 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7faeb406aea0 0x7faeb406d360 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:35.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.001+0000 7faed7015640 1 --2- 172.21.15.92:0/2717557770 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faec80e3120 0x7faec817c040 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:35.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.001+0000 7faed7015640 1 --2- 172.21.15.92:0/2717557770 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7faec8053c70 0x7faec804f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:35.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.001+0000 7faed7015640 1 --2- 172.21.15.92:0/2717557770 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7faec803c060 0x7faec804ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:35.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.001+0000 7faed7015640 1 -- 172.21.15.92:0/2717557770 >> 172.21.15.92:0/2717557770 conn(0x7faec8058ea0 msgr2=0x7faec80e6b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:35.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.001+0000 7faed7015640 1 -- 172.21.15.92:0/2717557770 shutdown_connections 2023-11-24T06:10:35.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.002+0000 7faed7015640 1 -- 172.21.15.92:0/2717557770 wait complete. 2023-11-24T06:10:35.024 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574853 for osd.0 2023-11-24T06:10:35.024 DEBUG:teuthology.parallel:result is None 2023-11-24T06:10:35.031 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574853 for osd.1 2023-11-24T06:10:35.031 DEBUG:teuthology.parallel:result is None 2023-11-24T06:10:35.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.042+0000 7f0b19a15640 1 -- 172.21.15.92:0/3462423445 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f0af80057d0 con 0x7f0b140c89b0 2023-11-24T06:10:35.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.042+0000 7ff4cb941640 1 -- 172.21.15.92:0/3750574147 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7ff4c40a9530 con 0x7ff4c4162820 2023-11-24T06:10:35.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.042+0000 7ff4b9ffb640 1 -- 172.21.15.92:0/3750574147 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7ff4c40a9530 con 0x7ff4c4162820 2023-11-24T06:10:35.046 INFO:teuthology.orchestra.run.smithi092.stdout:55834574854 2023-11-24T06:10:35.046 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.042+0000 7f0b0bfff640 1 -- 172.21.15.92:0/3462423445 <== mon.2 v2:172.21.15.118:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f0b0c058d70 con 0x7f0b140c89b0 2023-11-24T06:10:35.046 INFO:teuthology.orchestra.run.smithi092.stdout:55834574854 2023-11-24T06:10:35.047 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.044+0000 7ff4cb941640 1 -- 172.21.15.92:0/3750574147 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7ff4a806ae40 msgr2=0x7ff4a806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:35.048 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.044+0000 7ff4cb941640 1 --2- 172.21.15.92:0/3750574147 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7ff4a806ae40 0x7ff4a806d300 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ff4c0005880 tx=0x7ff4c0009040 comp rx=0 tx=0).stop 2023-11-24T06:10:35.048 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.045+0000 7ff4cb941640 1 -- 172.21.15.92:0/3750574147 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff4c4162820 msgr2=0x7ff4c4146f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:35.049 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.045+0000 7ff4cb941640 1 --2- 172.21.15.92:0/3750574147 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff4c4162820 0x7ff4c4146f50 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7ff4b402b8f0 tx=0x7ff4b40046c0 comp rx=0 tx=0).stop 2023-11-24T06:10:35.049 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.045+0000 7ff4cb941640 1 -- 172.21.15.92:0/3750574147 shutdown_connections 2023-11-24T06:10:35.049 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.045+0000 7ff4cb941640 1 --2- 172.21.15.92:0/3750574147 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7ff4a806ae40 0x7ff4a806d300 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:35.049 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.045+0000 7ff4cb941640 1 --2- 172.21.15.92:0/3750574147 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff4c4162820 0x7ff4c4146f50 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:35.049 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.045+0000 7ff4cb941640 1 --2- 172.21.15.92:0/3750574147 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff4c4161d30 0x7ff4c41fad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:35.049 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.045+0000 7ff4cb941640 1 --2- 172.21.15.92:0/3750574147 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff4c406d3b0 0x7ff4c40adeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:35.049 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.045+0000 7ff4cb941640 1 -- 172.21.15.92:0/3750574147 >> 172.21.15.92:0/3750574147 conn(0x7ff4c40b0be0 msgr2=0x7ff4c4160e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:35.049 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.045+0000 7ff4cb941640 1 -- 172.21.15.92:0/3750574147 shutdown_connections 2023-11-24T06:10:35.049 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.045+0000 7ff4cb941640 1 -- 172.21.15.92:0/3750574147 wait complete. 2023-11-24T06:10:35.050 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.046+0000 7f0b19a15640 1 -- 172.21.15.92:0/3462423445 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f0af406ae70 msgr2=0x7f0af406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:35.050 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.046+0000 7f0b19a15640 1 --2- 172.21.15.92:0/3462423445 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f0af406ae70 0x7f0af406d330 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f0afc001050 tx=0x7f0afc009040 comp rx=0 tx=0).stop 2023-11-24T06:10:35.051 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.046+0000 7f0b19a15640 1 -- 172.21.15.92:0/3462423445 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0b140c89b0 msgr2=0x7f0b140cca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:35.051 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.046+0000 7f0b19a15640 1 --2- 172.21.15.92:0/3462423445 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0b140c89b0 0x7f0b140cca50 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f0b0c002230 tx=0x7f0b0c00e220 comp rx=0 tx=0).stop 2023-11-24T06:10:35.051 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.046+0000 7f0b19a15640 1 -- 172.21.15.92:0/3462423445 shutdown_connections 2023-11-24T06:10:35.051 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.046+0000 7f0b19a15640 1 --2- 172.21.15.92:0/3462423445 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f0af406ae70 0x7f0af406d330 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:35.051 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.046+0000 7f0b19a15640 1 --2- 172.21.15.92:0/3462423445 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0b14200700 0x7f0b141e8bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:35.051 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.046+0000 7f0b19a15640 1 --2- 172.21.15.92:0/3462423445 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0b140ca500 0x7f0b142001c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:35.051 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.046+0000 7f0b19a15640 1 --2- 172.21.15.92:0/3462423445 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0b140c89b0 0x7f0b140cca50 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:35.051 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.046+0000 7f0b19a15640 1 -- 172.21.15.92:0/3462423445 >> 172.21.15.92:0/3462423445 conn(0x7f0b1409c540 msgr2=0x7f0b140b4f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:35.051 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.046+0000 7f0b19a15640 1 -- 172.21.15.92:0/3462423445 shutdown_connections 2023-11-24T06:10:35.051 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.046+0000 7f0b19a15640 1 -- 172.21.15.92:0/3462423445 wait complete. 2023-11-24T06:10:35.066 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574854 for osd.3 2023-11-24T06:10:35.067 DEBUG:teuthology.parallel:result is None 2023-11-24T06:10:35.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.071+0000 7f1bc780e640 1 -- 172.21.15.92:0/3446645759 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f1bc00b43c0 con 0x7f1bc01536e0 2023-11-24T06:10:35.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.071+0000 7f1bbdffb640 1 -- 172.21.15.92:0/3446645759 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f1bac054cb0 con 0x7f1bc01536e0 2023-11-24T06:10:35.075 INFO:teuthology.orchestra.run.smithi092.stdout:55834574854 2023-11-24T06:10:35.076 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574854 for osd.6 2023-11-24T06:10:35.076 DEBUG:teuthology.parallel:result is None 2023-11-24T06:10:35.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.074+0000 7f1bc780e640 1 -- 172.21.15.92:0/3446645759 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f1ba406ae90 msgr2=0x7f1ba406d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:35.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.074+0000 7f1bc780e640 1 --2- 172.21.15.92:0/3446645759 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f1ba406ae90 0x7f1ba406d350 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f1bb4035000 tx=0x7f1bb4030000 comp rx=0 tx=0).stop 2023-11-24T06:10:35.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.074+0000 7f1bc780e640 1 -- 172.21.15.92:0/3446645759 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1bc01536e0 msgr2=0x7f1bc0150250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:35.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.074+0000 7f1bc780e640 1 --2- 172.21.15.92:0/3446645759 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1bc01536e0 0x7f1bc0150250 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f1bac00e810 tx=0x7f1bac00ecd0 comp rx=0 tx=0).stop 2023-11-24T06:10:35.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.074+0000 7f1bc780e640 1 -- 172.21.15.92:0/3446645759 shutdown_connections 2023-11-24T06:10:35.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.074+0000 7f1bc780e640 1 --2- 172.21.15.92:0/3446645759 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f1ba406ae90 0x7f1ba406d350 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:35.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.074+0000 7f1bc780e640 1 --2- 172.21.15.92:0/3446645759 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1bc0159ca0 0x7f1bc015a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:35.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.074+0000 7f1bc780e640 1 --2- 172.21.15.92:0/3446645759 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1bc01536e0 0x7f1bc0150250 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:35.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.074+0000 7f1bc780e640 1 --2- 172.21.15.92:0/3446645759 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1bc00b16d0 0x7f1bc014aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:35.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.074+0000 7f1bc780e640 1 -- 172.21.15.92:0/3446645759 >> 172.21.15.92:0/3446645759 conn(0x7f1bc00b0be0 msgr2=0x7f1bc0152af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:35.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.074+0000 7f1bc780e640 1 -- 172.21.15.92:0/3446645759 shutdown_connections 2023-11-24T06:10:35.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.074+0000 7f1bc780e640 1 -- 172.21.15.92:0/3446645759 wait complete. 2023-11-24T06:10:35.093 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574854 for osd.5 2023-11-24T06:10:35.093 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.090+0000 7f1ba2148640 1 -- 172.21.15.92:0/510264154 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f1b9c148320 con 0x7f1b9c164100 2023-11-24T06:10:35.093 DEBUG:teuthology.parallel:result is None 2023-11-24T06:10:35.093 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.090+0000 7f1b93fff640 1 -- 172.21.15.92:0/510264154 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f1b94039020 con 0x7f1b9c164100 2023-11-24T06:10:35.094 INFO:teuthology.orchestra.run.smithi092.stdout:55834574854 2023-11-24T06:10:35.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.092+0000 7f1ba2148640 1 -- 172.21.15.92:0/510264154 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f1b7c06ae90 msgr2=0x7f1b7c06d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:35.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.092+0000 7f1ba2148640 1 --2- 172.21.15.92:0/510264154 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f1b7c06ae90 0x7f1b7c06d350 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f1b8c005530 tx=0x7f1b8c009040 comp rx=0 tx=0).stop 2023-11-24T06:10:35.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.092+0000 7f1ba2148640 1 -- 172.21.15.92:0/510264154 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b9c164100 msgr2=0x7f1b9c0cd940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:35.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.092+0000 7f1ba2148640 1 --2- 172.21.15.92:0/510264154 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b9c164100 0x7f1b9c0cd940 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f1b94009e60 tx=0x7f1b94004a40 comp rx=0 tx=0).stop 2023-11-24T06:10:35.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.092+0000 7f1ba2148640 1 -- 172.21.15.92:0/510264154 shutdown_connections 2023-11-24T06:10:35.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.092+0000 7f1ba2148640 1 --2- 172.21.15.92:0/510264154 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f1b7c06ae90 0x7f1b7c06d350 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:35.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.092+0000 7f1ba2148640 1 --2- 172.21.15.92:0/510264154 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b9c164100 0x7f1b9c0cd940 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:35.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.092+0000 7f1ba2148640 1 --2- 172.21.15.92:0/510264154 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1b9c1fb270 0x7f1b9c1fa8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:35.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.092+0000 7f1ba2148640 1 --2- 172.21.15.92:0/510264154 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1b8800a420 0x7f1b9c147110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:35.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.092+0000 7f1ba2148640 1 -- 172.21.15.92:0/510264154 >> 172.21.15.92:0/510264154 conn(0x7f1b9c0cfef0 msgr2=0x7f1b9c0b2d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:35.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.092+0000 7f1ba2148640 1 -- 172.21.15.92:0/510264154 shutdown_connections 2023-11-24T06:10:35.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.092+0000 7f1ba2148640 1 -- 172.21.15.92:0/510264154 wait complete. 2023-11-24T06:10:35.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.102+0000 7f4fa096f640 1 -- 172.21.15.92:0/659019952 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f4f800057d0 con 0x7f4f9c0ce6f0 2023-11-24T06:10:35.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.102+0000 7f4f8affd640 1 -- 172.21.15.92:0/659019952 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f4f8c058760 con 0x7f4f9c0ce6f0 2023-11-24T06:10:35.105 INFO:teuthology.orchestra.run.smithi092.stdout:55834574854 2023-11-24T06:10:35.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.104+0000 7fb5662ad640 1 -- 172.21.15.92:0/4106248893 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7fb560161b90 con 0x7fb56006d3b0 2023-11-24T06:10:35.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.104+0000 7fb55d7fa640 1 -- 172.21.15.92:0/4106248893 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fb548017020 con 0x7fb56006d3b0 2023-11-24T06:10:35.107 INFO:teuthology.orchestra.run.smithi092.stdout:55834574853 2023-11-24T06:10:35.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.105+0000 7f4fa096f640 1 -- 172.21.15.92:0/659019952 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f4f7c06af10 msgr2=0x7f4f7c06d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:35.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.105+0000 7f4fa096f640 1 --2- 172.21.15.92:0/659019952 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f4f7c06af10 0x7f4f7c06d3d0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f4f94035000 tx=0x7f4f94030000 comp rx=0 tx=0).stop 2023-11-24T06:10:35.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.105+0000 7f4fa096f640 1 -- 172.21.15.92:0/659019952 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4f9c0ce6f0 msgr2=0x7f4f9c0c8090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:35.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.105+0000 7f4fa096f640 1 --2- 172.21.15.92:0/659019952 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4f9c0ce6f0 0x7f4f9c0c8090 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f4f8c00e9b0 tx=0x7f4f8c00ee70 comp rx=0 tx=0).stop 2023-11-24T06:10:35.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.105+0000 7f4fa096f640 1 -- 172.21.15.92:0/659019952 shutdown_connections 2023-11-24T06:10:35.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.105+0000 7f4fa096f640 1 --2- 172.21.15.92:0/659019952 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f4f7c06af10 0x7f4f7c06d3d0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:35.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.105+0000 7f4fa096f640 1 --2- 172.21.15.92:0/659019952 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4f9c15d590 0x7f4f9c0ca5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:35.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.105+0000 7f4fa096f640 1 --2- 172.21.15.92:0/659019952 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4f9c0d00b0 0x7f4f9c159810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:35.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.105+0000 7f4fa096f640 1 --2- 172.21.15.92:0/659019952 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4f9c0ce6f0 0x7f4f9c0c8090 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:35.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.105+0000 7f4fa096f640 1 -- 172.21.15.92:0/659019952 >> 172.21.15.92:0/659019952 conn(0x7f4f9c09c540 msgr2=0x7f4f9c15fb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:35.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.105+0000 7f4fa096f640 1 -- 172.21.15.92:0/659019952 shutdown_connections 2023-11-24T06:10:35.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.105+0000 7f4fa096f640 1 -- 172.21.15.92:0/659019952 wait complete. 2023-11-24T06:10:35.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.106+0000 7fb5662ad640 1 -- 172.21.15.92:0/4106248893 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fb54406ae90 msgr2=0x7fb54406d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:35.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.106+0000 7fb5662ad640 1 --2- 172.21.15.92:0/4106248893 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fb54406ae90 0x7fb54406d350 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fb550034000 tx=0x7fb550008030 comp rx=0 tx=0).stop 2023-11-24T06:10:35.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.106+0000 7fb5662ad640 1 -- 172.21.15.92:0/4106248893 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb56006d3b0 msgr2=0x7fb560143590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:35.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.106+0000 7fb5662ad640 1 --2- 172.21.15.92:0/4106248893 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb56006d3b0 0x7fb560143590 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fb54800cba0 tx=0x7fb54800a430 comp rx=0 tx=0).stop 2023-11-24T06:10:35.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.107+0000 7fb5662ad640 1 -- 172.21.15.92:0/4106248893 shutdown_connections 2023-11-24T06:10:35.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.107+0000 7fb5662ad640 1 --2- 172.21.15.92:0/4106248893 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fb54406ae90 0x7fb54406d350 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:35.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.107+0000 7fb5662ad640 1 --2- 172.21.15.92:0/4106248893 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb560162650 0x7fb560159790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:35.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.107+0000 7fb5662ad640 1 --2- 172.21.15.92:0/4106248893 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb560161d30 0x7fb560159250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:35.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.107+0000 7fb5662ad640 1 --2- 172.21.15.92:0/4106248893 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb56006d3b0 0x7fb560143590 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:35.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.107+0000 7fb5662ad640 1 -- 172.21.15.92:0/4106248893 >> 172.21.15.92:0/4106248893 conn(0x7fb5600b0be0 msgr2=0x7fb5600b0fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:35.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.107+0000 7fb5662ad640 1 -- 172.21.15.92:0/4106248893 shutdown_connections 2023-11-24T06:10:35.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.107+0000 7fb5662ad640 1 -- 172.21.15.92:0/4106248893 wait complete. 2023-11-24T06:10:35.113 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574854 for osd.2 2023-11-24T06:10:35.113 DEBUG:teuthology.parallel:result is None 2023-11-24T06:10:35.126 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574854 for osd.4 2023-11-24T06:10:35.127 DEBUG:teuthology.parallel:result is None 2023-11-24T06:10:35.127 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574853 for osd.7 2023-11-24T06:10:35.127 DEBUG:teuthology.parallel:result is None 2023-11-24T06:10:35.128 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-11-24T06:10:35.128 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-24T06:10:35.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.368+0000 7f7d1a3a0640 1 Processor -- start 2023-11-24T06:10:35.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.368+0000 7f7d1a3a0640 1 -- start start 2023-11-24T06:10:35.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.370+0000 7f7d1a3a0640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f7d140d0100 0x7f7d140b3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:35.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.370+0000 7f7d1a3a0640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f7d140d0f40 0x7f7d140d1390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:35.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.370+0000 7f7d1a3a0640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d140d0a10 con 0x7f7d14160920 2023-11-24T06:10:35.374 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.370+0000 7f7d1a3a0640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d140cf080 con 0x7f7d140d0f40 2023-11-24T06:10:35.374 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.370+0000 7f7d1a3a0640 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d140c6be0 con 0x7f7d140d0100 2023-11-24T06:10:35.374 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.370+0000 7f7d1939e640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f7d14160920 0x7f7d140cf9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:55890/0 (socket says 172.21.15.92:55890) 2023-11-24T06:10:35.374 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.370+0000 7f7d1939e640 1 -- 172.21.15.92:0/1877869710 learned_addr learned my addr 172.21.15.92:0/1877869710 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-24T06:10:35.374 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.370+0000 7f7d19b9f640 1 --2- 172.21.15.92:0/1877869710 >> v2:172.21.15.92:3300/0 conn(0x7f7d140d0f40 0x7f7d140d1390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:35.376 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.370+0000 7f7d18b9d640 1 --2- 172.21.15.92:0/1877869710 >> v2:172.21.15.118:3300/0 conn(0x7f7d140d0100 0x7f7d140b3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:35.376 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.371+0000 7f7d07fff640 1 -- 172.21.15.92:0/1877869710 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2778213091 0 0) 0x7f7d140d0a10 con 0x7f7d14160920 2023-11-24T06:10:35.376 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.371+0000 7f7d07fff640 1 -- 172.21.15.92:0/1877869710 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7d00003520 con 0x7f7d14160920 2023-11-24T06:10:35.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.371+0000 7f7d19b9f640 1 -- 172.21.15.92:0/1877869710 >> v2:172.21.15.118:3300/0 conn(0x7f7d140d0100 msgr2=0x7f7d140b3f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:35.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.371+0000 7f7d19b9f640 1 --2- 172.21.15.92:0/1877869710 >> v2:172.21.15.118:3300/0 conn(0x7f7d140d0100 0x7f7d140b3f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:35.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.371+0000 7f7d19b9f640 1 -- 172.21.15.92:0/1877869710 >> v1:172.21.15.92:6789/0 conn(0x7f7d14160920 legacy=0x7f7d140cf9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:35.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.371+0000 7f7d19b9f640 1 -- 172.21.15.92:0/1877869710 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d140ce7e0 con 0x7f7d140d0f40 2023-11-24T06:10:35.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.371+0000 7f7d18b9d640 1 --2- 172.21.15.92:0/1877869710 >> v2:172.21.15.118:3300/0 conn(0x7f7d140d0100 0x7f7d140b3f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:10:35.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.372+0000 7f7d19b9f640 1 --2- 172.21.15.92:0/1877869710 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7d140d0f40 0x7f7d140d1390 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f7d1408c1f0 tx=0x7f7d0c02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=692ea9d648674dfe server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:35.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.372+0000 7f7d07fff640 1 -- 172.21.15.92:0/1877869710 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d0c02cc80 con 0x7f7d140d0f40 2023-11-24T06:10:35.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.372+0000 7f7d07fff640 1 -- 172.21.15.92:0/1877869710 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d0c02cde0 con 0x7f7d140d0f40 2023-11-24T06:10:35.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.372+0000 7f7d07fff640 1 -- 172.21.15.92:0/1877869710 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d0c034660 con 0x7f7d140d0f40 2023-11-24T06:10:35.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.372+0000 7f7d1a3a0640 1 -- 172.21.15.92:0/1877869710 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7d140d0f40 msgr2=0x7f7d140d1390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:35.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.372+0000 7f7d1a3a0640 1 --2- 172.21.15.92:0/1877869710 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7d140d0f40 0x7f7d140d1390 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f7d1408c1f0 tx=0x7f7d0c02bed0 comp rx=0 tx=0).stop 2023-11-24T06:10:35.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.373+0000 7f7d1a3a0640 1 -- 172.21.15.92:0/1877869710 shutdown_connections 2023-11-24T06:10:35.379 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.373+0000 7f7d1a3a0640 1 --2- 172.21.15.92:0/1877869710 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7d140d0f40 0x7f7d140d1390 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:35.379 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.373+0000 7f7d1a3a0640 1 --2- 172.21.15.92:0/1877869710 >> v2:172.21.15.118:3300/0 conn(0x7f7d140d0100 0x7f7d140b3f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:35.379 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.373+0000 7f7d1a3a0640 1 -- 172.21.15.92:0/1877869710 >> 172.21.15.92:0/1877869710 conn(0x7f7d140d3090 msgr2=0x7f7d140d34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:35.379 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.373+0000 7f7d1a3a0640 1 -- 172.21.15.92:0/1877869710 shutdown_connections 2023-11-24T06:10:35.379 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.373+0000 7f7d1a3a0640 1 -- 172.21.15.92:0/1877869710 wait complete. 2023-11-24T06:10:35.383 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.374+0000 7f7d1a3a0640 1 Processor -- start 2023-11-24T06:10:35.383 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.374+0000 7f7d1a3a0640 1 -- start start 2023-11-24T06:10:35.383 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.375+0000 7f7d1a3a0640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7d140d0100 0x7f7d1414d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:35.383 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.375+0000 7f7d1a3a0640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7d140d0f40 0x7f7d1415fa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:35.383 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.375+0000 7f7d1a3a0640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7d14160920 0x7f7d141f67a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:35.383 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.375+0000 7f7d1a3a0640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d140b7310 con 0x7f7d140d0100 2023-11-24T06:10:35.383 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.375+0000 7f7d1a3a0640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d140b6700 con 0x7f7d14160920 2023-11-24T06:10:35.383 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.375+0000 7f7d1a3a0640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d140b69e0 con 0x7f7d140d0f40 2023-11-24T06:10:35.384 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.375+0000 7f7d1939e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7d140d0100 0x7f7d1414d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:35.384 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.375+0000 7f7d1939e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7d140d0100 0x7f7d1414d3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47566/0 (socket says 172.21.15.92:47566) 2023-11-24T06:10:35.384 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.376+0000 7f7d1939e640 1 -- 172.21.15.92:0/1570265125 learned_addr learned my addr 172.21.15.92:0/1570265125 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:10:35.384 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.376+0000 7f7d1939e640 1 -- 172.21.15.92:0/1570265125 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7d140d0f40 msgr2=0x7f7d1415fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:35.384 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.376+0000 7f7d1939e640 1 --2- 172.21.15.92:0/1570265125 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7d140d0f40 0x7f7d1415fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:35.384 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.376+0000 7f7d1939e640 1 -- 172.21.15.92:0/1570265125 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7d14160920 msgr2=0x7f7d141f67a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:35.384 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.376+0000 7f7d1939e640 1 --2- 172.21.15.92:0/1570265125 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7d14160920 0x7f7d141f67a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:35.384 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.376+0000 7f7d1939e640 1 -- 172.21.15.92:0/1570265125 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d0c009a70 con 0x7f7d140d0100 2023-11-24T06:10:35.384 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.376+0000 7f7d1939e640 1 --2- 172.21.15.92:0/1570265125 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7d140d0100 0x7f7d1414d3a0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f7d1000c410 tx=0x7f7d1000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:35.384 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.376+0000 7f7d05ffb640 1 -- 172.21.15.92:0/1570265125 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d10005ce0 con 0x7f7d140d0100 2023-11-24T06:10:35.384 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.376+0000 7f7d05ffb640 1 -- 172.21.15.92:0/1570265125 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d10005e40 con 0x7f7d140d0100 2023-11-24T06:10:35.384 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.376+0000 7f7d05ffb640 1 -- 172.21.15.92:0/1570265125 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d100072f0 con 0x7f7d140d0100 2023-11-24T06:10:35.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.376+0000 7f7d1a3a0640 1 -- 172.21.15.92:0/1570265125 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d140b6cc0 con 0x7f7d140d0100 2023-11-24T06:10:35.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.377+0000 7f7d1a3a0640 1 -- 172.21.15.92:0/1570265125 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d1414d8e0 con 0x7f7d140d0100 2023-11-24T06:10:35.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.377+0000 7f7d1a3a0640 1 -- 172.21.15.92:0/1570265125 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d14067450 con 0x7f7d140d0100 2023-11-24T06:10:35.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.378+0000 7f7d05ffb640 1 -- 172.21.15.92:0/1570265125 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f7d10007450 con 0x7f7d140d0100 2023-11-24T06:10:35.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.379+0000 7f7d05ffb640 1 --2- 172.21.15.92:0/1570265125 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f7cf806ae40 0x7f7cf806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:35.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.379+0000 7f7d05ffb640 1 -- 172.21.15.92:0/1570265125 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7d10088cd0 con 0x7f7d140d0100 2023-11-24T06:10:35.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.380+0000 7f7d18b9d640 1 --2- 172.21.15.92:0/1570265125 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f7cf806ae40 0x7f7cf806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:35.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.380+0000 7f7d18b9d640 1 --2- 172.21.15.92:0/1570265125 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f7cf806ae40 0x7f7cf806d300 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f7d08006fd0 tx=0x7f7d08008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:35.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.384+0000 7f7d05ffb640 1 -- 172.21.15.92:0/1570265125 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7d100578c0 con 0x7f7d140d0100 2023-11-24T06:10:35.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.559+0000 7f7d1a3a0640 1 -- 172.21.15.92:0/1570265125 --> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f7d1414da70 con 0x7f7cf806ae40 2023-11-24T06:10:35.578 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.575+0000 7f7d05ffb640 1 -- 172.21.15.92:0/1570265125 <== mgr.4101 v2:172.21.15.92:6800/784027401 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264389 (secure 0 0 0) 0x7f7d08009d80 con 0x7f7cf806ae40 2023-11-24T06:10:35.583 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-24T06:10:35.585 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2023-11-24T06:10:35.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.580+0000 7f7d1a3a0640 1 -- 172.21.15.92:0/1570265125 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f7cf806ae40 msgr2=0x7f7cf806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:35.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.580+0000 7f7d1a3a0640 1 --2- 172.21.15.92:0/1570265125 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f7cf806ae40 0x7f7cf806d300 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f7d08006fd0 tx=0x7f7d08008040 comp rx=0 tx=0).stop 2023-11-24T06:10:35.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.580+0000 7f7d1a3a0640 1 -- 172.21.15.92:0/1570265125 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7d140d0100 msgr2=0x7f7d1414d3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:35.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.580+0000 7f7d1a3a0640 1 --2- 172.21.15.92:0/1570265125 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7d140d0100 0x7f7d1414d3a0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f7d1000c410 tx=0x7f7d1000c8d0 comp rx=0 tx=0).stop 2023-11-24T06:10:35.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.580+0000 7f7d1a3a0640 1 -- 172.21.15.92:0/1570265125 shutdown_connections 2023-11-24T06:10:35.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.580+0000 7f7d1a3a0640 1 --2- 172.21.15.92:0/1570265125 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f7cf806ae40 0x7f7cf806d300 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:35.589 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.580+0000 7f7d1a3a0640 1 --2- 172.21.15.92:0/1570265125 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7d14160920 0x7f7d141f67a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:35.590 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.580+0000 7f7d1a3a0640 1 --2- 172.21.15.92:0/1570265125 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7d140d0f40 0x7f7d1415fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:35.590 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.580+0000 7f7d1a3a0640 1 --2- 172.21.15.92:0/1570265125 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7d140d0100 0x7f7d1414d3a0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:35.590 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.580+0000 7f7d1a3a0640 1 -- 172.21.15.92:0/1570265125 >> 172.21.15.92:0/1570265125 conn(0x7f7d140d3090 msgr2=0x7f7d1406c0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:35.591 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.580+0000 7f7d1a3a0640 1 -- 172.21.15.92:0/1570265125 shutdown_connections 2023-11-24T06:10:35.591 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.580+0000 7f7d1a3a0640 1 -- 172.21.15.92:0/1570265125 wait complete. 2023-11-24T06:10:35.602 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2023-11-24T06:10:33.669332+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235656,"kb_used_data":2816,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754739064,"statfs":{"total":773094113280,"available":772852801536,"internally_reserved":0,"allocated":2883584,"data_stored":1751608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":10,"num_read_kb":9,"num_write":59,"num_write_kb":276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002569"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.887289+0000","last_change":"2023-11-24T06:10:25.718900+0000","last_active":"2023-11-24T06:10:26.887289+0000","last_peered":"2023-11-24T06:10:26.887289+0000","last_clean":"2023-11-24T06:10:26.887289+0000","last_became_active":"2023-11-24T06:10:25.718569+0000","last_became_peered":"2023-11-24T06:10:25.718569+0000","last_unstale":"2023-11-24T06:10:26.887289+0000","last_undegraded":"2023-11-24T06:10:26.887289+0000","last_fullsized":"2023-11-24T06:10:26.887289+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:56:03.801033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.885991+0000","last_change":"2023-11-24T06:10:25.711437+0000","last_active":"2023-11-24T06:10:26.885991+0000","last_peered":"2023-11-24T06:10:26.885991+0000","last_clean":"2023-11-24T06:10:26.885991+0000","last_became_active":"2023-11-24T06:10:25.710555+0000","last_became_peered":"2023-11-24T06:10:25.710555+0000","last_unstale":"2023-11-24T06:10:26.885991+0000","last_undegraded":"2023-11-24T06:10:26.885991+0000","last_fullsized":"2023-11-24T06:10:26.885991+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:37:01.293292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.886876+0000","last_change":"2023-11-24T06:10:25.716423+0000","last_active":"2023-11-24T06:10:26.886876+0000","last_peered":"2023-11-24T06:10:26.886876+0000","last_clean":"2023-11-24T06:10:26.886876+0000","last_became_active":"2023-11-24T06:10:25.716112+0000","last_became_peered":"2023-11-24T06:10:25.716112+0000","last_unstale":"2023-11-24T06:10:26.886876+0000","last_undegraded":"2023-11-24T06:10:26.886876+0000","last_fullsized":"2023-11-24T06:10:26.886876+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:43:50.924468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.888341+0000","last_change":"2023-11-24T06:10:25.733338+0000","last_active":"2023-11-24T06:10:26.888341+0000","last_peered":"2023-11-24T06:10:26.888341+0000","last_clean":"2023-11-24T06:10:26.888341+0000","last_became_active":"2023-11-24T06:10:25.733087+0000","last_became_peered":"2023-11-24T06:10:25.733087+0000","last_unstale":"2023-11-24T06:10:26.888341+0000","last_undegraded":"2023-11-24T06:10:26.888341+0000","last_fullsized":"2023-11-24T06:10:26.888341+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:00:56.115984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.887727+0000","last_change":"2023-11-24T06:10:25.721479+0000","last_active":"2023-11-24T06:10:26.887727+0000","last_peered":"2023-11-24T06:10:26.887727+0000","last_clean":"2023-11-24T06:10:26.887727+0000","last_became_active":"2023-11-24T06:10:25.721135+0000","last_became_peered":"2023-11-24T06:10:25.721135+0000","last_unstale":"2023-11-24T06:10:26.887727+0000","last_undegraded":"2023-11-24T06:10:26.887727+0000","last_fullsized":"2023-11-24T06:10:26.887727+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:14:05.090678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.886536+0000","last_change":"2023-11-24T06:10:25.731299+0000","last_active":"2023-11-24T06:10:26.886536+0000","last_peered":"2023-11-24T06:10:26.886536+0000","last_clean":"2023-11-24T06:10:26.886536+0000","last_became_active":"2023-11-24T06:10:25.730627+0000","last_became_peered":"2023-11-24T06:10:25.730627+0000","last_unstale":"2023-11-24T06:10:26.886536+0000","last_undegraded":"2023-11-24T06:10:26.886536+0000","last_fullsized":"2023-11-24T06:10:26.886536+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:53:47.062623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.885972+0000","last_change":"2023-11-24T06:10:25.710797+0000","last_active":"2023-11-24T06:10:26.885972+0000","last_peered":"2023-11-24T06:10:26.885972+0000","last_clean":"2023-11-24T06:10:26.885972+0000","last_became_active":"2023-11-24T06:10:25.710509+0000","last_became_peered":"2023-11-24T06:10:25.710509+0000","last_unstale":"2023-11-24T06:10:26.885972+0000","last_undegraded":"2023-11-24T06:10:26.885972+0000","last_fullsized":"2023-11-24T06:10:26.885972+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:44:12.726705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.714692+0000","last_change":"2023-11-24T06:10:25.721521+0000","last_active":"2023-11-24T06:10:26.714692+0000","last_peered":"2023-11-24T06:10:26.714692+0000","last_clean":"2023-11-24T06:10:26.714692+0000","last_became_active":"2023-11-24T06:10:25.721206+0000","last_became_peered":"2023-11-24T06:10:25.721206+0000","last_unstale":"2023-11-24T06:10:26.714692+0000","last_undegraded":"2023-11-24T06:10:26.714692+0000","last_fullsized":"2023-11-24T06:10:26.714692+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:52:32.288074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.887339+0000","last_change":"2023-11-24T06:10:25.743664+0000","last_active":"2023-11-24T06:10:26.887339+0000","last_peered":"2023-11-24T06:10:26.887339+0000","last_clean":"2023-11-24T06:10:26.887339+0000","last_became_active":"2023-11-24T06:10:25.743346+0000","last_became_peered":"2023-11-24T06:10:25.743346+0000","last_unstale":"2023-11-24T06:10:26.887339+0000","last_undegraded":"2023-11-24T06:10:26.887339+0000","last_fullsized":"2023-11-24T06:10:26.887339+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:59:37.010936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.716729+0000","last_change":"2023-11-24T06:10:25.722633+0000","last_active":"2023-11-24T06:10:26.716729+0000","last_peered":"2023-11-24T06:10:26.716729+0000","last_clean":"2023-11-24T06:10:26.716729+0000","last_became_active":"2023-11-24T06:10:25.722320+0000","last_became_peered":"2023-11-24T06:10:25.722320+0000","last_unstale":"2023-11-24T06:10:26.716729+0000","last_undegraded":"2023-11-24T06:10:26.716729+0000","last_fullsized":"2023-11-24T06:10:26.716729+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:37:01.293292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.716342+0000","last_change":"2023-11-24T06:10:25.721320+0000","last_active":"2023-11-24T06:10:26.716342+0000","last_peered":"2023-11-24T06:10:26.716342+0000","last_clean":"2023-11-24T06:10:26.716342+0000","last_became_active":"2023-11-24T06:10:25.720925+0000","last_became_peered":"2023-11-24T06:10:25.720925+0000","last_unstale":"2023-11-24T06:10:26.716342+0000","last_undegraded":"2023-11-24T06:10:26.716342+0000","last_fullsized":"2023-11-24T06:10:26.716342+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:40:01.407324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.716059+0000","last_change":"2023-11-24T06:10:25.700499+0000","last_active":"2023-11-24T06:10:26.716059+0000","last_peered":"2023-11-24T06:10:26.716059+0000","last_clean":"2023-11-24T06:10:26.716059+0000","last_became_active":"2023-11-24T06:10:25.700070+0000","last_became_peered":"2023-11-24T06:10:25.700070+0000","last_unstale":"2023-11-24T06:10:26.716059+0000","last_undegraded":"2023-11-24T06:10:26.716059+0000","last_fullsized":"2023-11-24T06:10:26.716059+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:22:20.935204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.716045+0000","last_change":"2023-11-24T06:10:25.714833+0000","last_active":"2023-11-24T06:10:26.716045+0000","last_peered":"2023-11-24T06:10:26.716045+0000","last_clean":"2023-11-24T06:10:26.716045+0000","last_became_active":"2023-11-24T06:10:25.714265+0000","last_became_peered":"2023-11-24T06:10:25.714265+0000","last_unstale":"2023-11-24T06:10:26.716045+0000","last_undegraded":"2023-11-24T06:10:26.716045+0000","last_fullsized":"2023-11-24T06:10:26.716045+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:40:01.407324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.885149+0000","last_change":"2023-11-24T06:10:25.710049+0000","last_active":"2023-11-24T06:10:26.885149+0000","last_peered":"2023-11-24T06:10:26.885149+0000","last_clean":"2023-11-24T06:10:26.885149+0000","last_became_active":"2023-11-24T06:10:25.709681+0000","last_became_peered":"2023-11-24T06:10:25.709681+0000","last_unstale":"2023-11-24T06:10:26.885149+0000","last_undegraded":"2023-11-24T06:10:26.885149+0000","last_fullsized":"2023-11-24T06:10:26.885149+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:52:32.288074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.885255+0000","last_change":"2023-11-24T06:10:25.713948+0000","last_active":"2023-11-24T06:10:26.885255+0000","last_peered":"2023-11-24T06:10:26.885255+0000","last_clean":"2023-11-24T06:10:26.885255+0000","last_became_active":"2023-11-24T06:10:25.713633+0000","last_became_peered":"2023-11-24T06:10:25.713633+0000","last_unstale":"2023-11-24T06:10:26.885255+0000","last_undegraded":"2023-11-24T06:10:26.885255+0000","last_fullsized":"2023-11-24T06:10:26.885255+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:18:29.626191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.885915+0000","last_change":"2023-11-24T06:10:25.733647+0000","last_active":"2023-11-24T06:10:26.885915+0000","last_peered":"2023-11-24T06:10:26.885915+0000","last_clean":"2023-11-24T06:10:26.885915+0000","last_became_active":"2023-11-24T06:10:25.733342+0000","last_became_peered":"2023-11-24T06:10:25.733342+0000","last_unstale":"2023-11-24T06:10:26.885915+0000","last_undegraded":"2023-11-24T06:10:26.885915+0000","last_fullsized":"2023-11-24T06:10:26.885915+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:43:50.924468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.889359+0000","last_change":"2023-11-24T06:10:25.737712+0000","last_active":"2023-11-24T06:10:26.889359+0000","last_peered":"2023-11-24T06:10:26.889359+0000","last_clean":"2023-11-24T06:10:26.889359+0000","last_became_active":"2023-11-24T06:10:25.735939+0000","last_became_peered":"2023-11-24T06:10:25.735939+0000","last_unstale":"2023-11-24T06:10:26.889359+0000","last_undegraded":"2023-11-24T06:10:26.889359+0000","last_fullsized":"2023-11-24T06:10:26.889359+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:05:14.621899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.826109+0000","last_change":"2023-11-24T06:10:25.717816+0000","last_active":"2023-11-24T06:10:26.826109+0000","last_peered":"2023-11-24T06:10:26.826109+0000","last_clean":"2023-11-24T06:10:26.826109+0000","last_became_active":"2023-11-24T06:10:25.717132+0000","last_became_peered":"2023-11-24T06:10:25.717132+0000","last_unstale":"2023-11-24T06:10:26.826109+0000","last_undegraded":"2023-11-24T06:10:26.826109+0000","last_fullsized":"2023-11-24T06:10:26.826109+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:17:03.098818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.888164+0000","last_change":"2023-11-24T06:10:25.724997+0000","last_active":"2023-11-24T06:10:26.888164+0000","last_peered":"2023-11-24T06:10:26.888164+0000","last_clean":"2023-11-24T06:10:26.888164+0000","last_became_active":"2023-11-24T06:10:25.716385+0000","last_became_peered":"2023-11-24T06:10:25.716385+0000","last_unstale":"2023-11-24T06:10:26.888164+0000","last_undegraded":"2023-11-24T06:10:26.888164+0000","last_fullsized":"2023-11-24T06:10:26.888164+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:01:41.923534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.716353+0000","last_change":"2023-11-24T06:10:25.717325+0000","last_active":"2023-11-24T06:10:26.716353+0000","last_peered":"2023-11-24T06:10:26.716353+0000","last_clean":"2023-11-24T06:10:26.716353+0000","last_became_active":"2023-11-24T06:10:25.716998+0000","last_became_peered":"2023-11-24T06:10:25.716998+0000","last_unstale":"2023-11-24T06:10:26.716353+0000","last_undegraded":"2023-11-24T06:10:26.716353+0000","last_fullsized":"2023-11-24T06:10:26.716353+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:43:50.924468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.886459+0000","last_change":"2023-11-24T06:10:25.729153+0000","last_active":"2023-11-24T06:10:26.886459+0000","last_peered":"2023-11-24T06:10:26.886459+0000","last_clean":"2023-11-24T06:10:26.886459+0000","last_became_active":"2023-11-24T06:10:25.724982+0000","last_became_peered":"2023-11-24T06:10:25.724982+0000","last_unstale":"2023-11-24T06:10:26.886459+0000","last_undegraded":"2023-11-24T06:10:26.886459+0000","last_fullsized":"2023-11-24T06:10:26.886459+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:46:00.341282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.886867+0000","last_change":"2023-11-24T06:10:25.734081+0000","last_active":"2023-11-24T06:10:26.886867+0000","last_peered":"2023-11-24T06:10:26.886867+0000","last_clean":"2023-11-24T06:10:26.886867+0000","last_became_active":"2023-11-24T06:10:25.733760+0000","last_became_peered":"2023-11-24T06:10:25.733760+0000","last_unstale":"2023-11-24T06:10:26.886867+0000","last_undegraded":"2023-11-24T06:10:26.886867+0000","last_fullsized":"2023-11-24T06:10:26.886867+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:26:40.207866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.715619+0000","last_change":"2023-11-24T06:10:25.720936+0000","last_active":"2023-11-24T06:10:26.715619+0000","last_peered":"2023-11-24T06:10:26.715619+0000","last_clean":"2023-11-24T06:10:26.715619+0000","last_became_active":"2023-11-24T06:10:25.720682+0000","last_became_peered":"2023-11-24T06:10:25.720682+0000","last_unstale":"2023-11-24T06:10:26.715619+0000","last_undegraded":"2023-11-24T06:10:26.715619+0000","last_fullsized":"2023-11-24T06:10:26.715619+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:18:29.626191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.886110+0000","last_change":"2023-11-24T06:10:25.730235+0000","last_active":"2023-11-24T06:10:26.886110+0000","last_peered":"2023-11-24T06:10:26.886110+0000","last_clean":"2023-11-24T06:10:26.886110+0000","last_became_active":"2023-11-24T06:10:25.729972+0000","last_became_peered":"2023-11-24T06:10:25.729972+0000","last_unstale":"2023-11-24T06:10:26.886110+0000","last_undegraded":"2023-11-24T06:10:26.886110+0000","last_fullsized":"2023-11-24T06:10:26.886110+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:37:01.293292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.716777+0000","last_change":"2023-11-24T06:10:25.719429+0000","last_active":"2023-11-24T06:10:26.716777+0000","last_peered":"2023-11-24T06:10:26.716777+0000","last_clean":"2023-11-24T06:10:26.716777+0000","last_became_active":"2023-11-24T06:10:25.719099+0000","last_became_peered":"2023-11-24T06:10:25.719099+0000","last_unstale":"2023-11-24T06:10:26.716777+0000","last_undegraded":"2023-11-24T06:10:26.716777+0000","last_fullsized":"2023-11-24T06:10:26.716777+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:53:47.062623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.716868+0000","last_change":"2023-11-24T06:10:25.710010+0000","last_active":"2023-11-24T06:10:26.716868+0000","last_peered":"2023-11-24T06:10:26.716868+0000","last_clean":"2023-11-24T06:10:26.716868+0000","last_became_active":"2023-11-24T06:10:25.709727+0000","last_became_peered":"2023-11-24T06:10:25.709727+0000","last_unstale":"2023-11-24T06:10:26.716868+0000","last_undegraded":"2023-11-24T06:10:26.716868+0000","last_fullsized":"2023-11-24T06:10:26.716868+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:53:47.062623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.885531+0000","last_change":"2023-11-24T06:10:25.713669+0000","last_active":"2023-11-24T06:10:26.885531+0000","last_peered":"2023-11-24T06:10:26.885531+0000","last_clean":"2023-11-24T06:10:26.885531+0000","last_became_active":"2023-11-24T06:10:25.713339+0000","last_became_peered":"2023-11-24T06:10:25.713339+0000","last_unstale":"2023-11-24T06:10:26.885531+0000","last_undegraded":"2023-11-24T06:10:26.885531+0000","last_fullsized":"2023-11-24T06:10:26.885531+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:40:01.407324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.716342+0000","last_change":"2023-11-24T06:10:25.721615+0000","last_active":"2023-11-24T06:10:26.716342+0000","last_peered":"2023-11-24T06:10:26.716342+0000","last_clean":"2023-11-24T06:10:26.716342+0000","last_became_active":"2023-11-24T06:10:25.721241+0000","last_became_peered":"2023-11-24T06:10:25.721241+0000","last_unstale":"2023-11-24T06:10:26.716342+0000","last_undegraded":"2023-11-24T06:10:26.716342+0000","last_fullsized":"2023-11-24T06:10:26.716342+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:37:01.293292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.886687+0000","last_change":"2023-11-24T06:10:25.713056+0000","last_active":"2023-11-24T06:10:26.886687+0000","last_peered":"2023-11-24T06:10:26.886687+0000","last_clean":"2023-11-24T06:10:26.886687+0000","last_became_active":"2023-11-24T06:10:25.712467+0000","last_became_peered":"2023-11-24T06:10:25.712467+0000","last_unstale":"2023-11-24T06:10:26.886687+0000","last_undegraded":"2023-11-24T06:10:26.886687+0000","last_fullsized":"2023-11-24T06:10:26.886687+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:18:29.626191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.887172+0000","last_change":"2023-11-24T06:10:25.733233+0000","last_active":"2023-11-24T06:10:26.887172+0000","last_peered":"2023-11-24T06:10:26.887172+0000","last_clean":"2023-11-24T06:10:26.887172+0000","last_became_active":"2023-11-24T06:10:25.732933+0000","last_became_peered":"2023-11-24T06:10:25.732933+0000","last_unstale":"2023-11-24T06:10:26.887172+0000","last_undegraded":"2023-11-24T06:10:26.887172+0000","last_fullsized":"2023-11-24T06:10:26.887172+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:44:12.726705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.826224+0000","last_change":"2023-11-24T06:10:25.722324+0000","last_active":"2023-11-24T06:10:26.826224+0000","last_peered":"2023-11-24T06:10:26.826224+0000","last_clean":"2023-11-24T06:10:26.826224+0000","last_became_active":"2023-11-24T06:10:25.722036+0000","last_became_peered":"2023-11-24T06:10:25.722036+0000","last_unstale":"2023-11-24T06:10:26.826224+0000","last_undegraded":"2023-11-24T06:10:26.826224+0000","last_fullsized":"2023-11-24T06:10:26.826224+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:09:06.300144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.715780+0000","last_change":"2023-11-24T06:10:25.703106+0000","last_active":"2023-11-24T06:10:26.715780+0000","last_peered":"2023-11-24T06:10:26.715780+0000","last_clean":"2023-11-24T06:10:26.715780+0000","last_became_active":"2023-11-24T06:10:25.701998+0000","last_became_peered":"2023-11-24T06:10:25.701998+0000","last_unstale":"2023-11-24T06:10:26.715780+0000","last_undegraded":"2023-11-24T06:10:26.715780+0000","last_fullsized":"2023-11-24T06:10:26.715780+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:46:00.341282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.715676+0000","last_change":"2023-11-24T06:10:25.708408+0000","last_active":"2023-11-24T06:10:26.715676+0000","last_peered":"2023-11-24T06:10:26.715676+0000","last_clean":"2023-11-24T06:10:26.715676+0000","last_became_active":"2023-11-24T06:10:25.708102+0000","last_became_peered":"2023-11-24T06:10:25.708102+0000","last_unstale":"2023-11-24T06:10:26.715676+0000","last_undegraded":"2023-11-24T06:10:26.715676+0000","last_fullsized":"2023-11-24T06:10:26.715676+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:38:11.967627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.716406+0000","last_change":"2023-11-24T06:10:24.696915+0000","last_active":"2023-11-24T06:10:26.716406+0000","last_peered":"2023-11-24T06:10:26.716406+0000","last_clean":"2023-11-24T06:10:26.716406+0000","last_became_active":"2023-11-24T06:10:24.696608+0000","last_became_peered":"2023-11-24T06:10:24.696608+0000","last_unstale":"2023-11-24T06:10:26.716406+0000","last_undegraded":"2023-11-24T06:10:26.716406+0000","last_fullsized":"2023-11-24T06:10:26.716406+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_clean_scrub_stamp":"2023-11-24T06:10:23.635255+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:14:19.359955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.886766+0000","last_change":"2023-11-24T06:10:25.713232+0000","last_active":"2023-11-24T06:10:26.886766+0000","last_peered":"2023-11-24T06:10:26.886766+0000","last_clean":"2023-11-24T06:10:26.886766+0000","last_became_active":"2023-11-24T06:10:25.712813+0000","last_became_peered":"2023-11-24T06:10:25.712813+0000","last_unstale":"2023-11-24T06:10:26.886766+0000","last_undegraded":"2023-11-24T06:10:26.886766+0000","last_fullsized":"2023-11-24T06:10:26.886766+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:40:01.407324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.716067+0000","last_change":"2023-11-24T06:10:24.689472+0000","last_active":"2023-11-24T06:10:26.716067+0000","last_peered":"2023-11-24T06:10:26.716067+0000","last_clean":"2023-11-24T06:10:26.716067+0000","last_became_active":"2023-11-24T06:10:24.689143+0000","last_became_peered":"2023-11-24T06:10:24.689143+0000","last_unstale":"2023-11-24T06:10:26.716067+0000","last_undegraded":"2023-11-24T06:10:26.716067+0000","last_fullsized":"2023-11-24T06:10:26.716067+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_clean_scrub_stamp":"2023-11-24T06:10:23.635255+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:18:53.146509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.715380+0000","last_change":"2023-11-24T06:10:24.694706+0000","last_active":"2023-11-24T06:10:26.715380+0000","last_peered":"2023-11-24T06:10:26.715380+0000","last_clean":"2023-11-24T06:10:26.715380+0000","last_became_active":"2023-11-24T06:10:24.693990+0000","last_became_peered":"2023-11-24T06:10:24.693990+0000","last_unstale":"2023-11-24T06:10:26.715380+0000","last_undegraded":"2023-11-24T06:10:26.715380+0000","last_fullsized":"2023-11-24T06:10:26.715380+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_clean_scrub_stamp":"2023-11-24T06:10:23.635255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:17:02.081734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.714912+0000","last_change":"2023-11-24T06:10:25.715897+0000","last_active":"2023-11-24T06:10:26.714912+0000","last_peered":"2023-11-24T06:10:26.714912+0000","last_clean":"2023-11-24T06:10:26.714912+0000","last_became_active":"2023-11-24T06:10:25.715596+0000","last_became_peered":"2023-11-24T06:10:25.715596+0000","last_unstale":"2023-11-24T06:10:26.714912+0000","last_undegraded":"2023-11-24T06:10:26.714912+0000","last_fullsized":"2023-11-24T06:10:26.714912+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:54:02.837130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.886161+0000","last_change":"2023-11-24T06:10:24.692669+0000","last_active":"2023-11-24T06:10:26.886161+0000","last_peered":"2023-11-24T06:10:26.886161+0000","last_clean":"2023-11-24T06:10:26.886161+0000","last_became_active":"2023-11-24T06:10:24.692341+0000","last_became_peered":"2023-11-24T06:10:24.692341+0000","last_unstale":"2023-11-24T06:10:26.886161+0000","last_undegraded":"2023-11-24T06:10:26.886161+0000","last_fullsized":"2023-11-24T06:10:26.886161+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_clean_scrub_stamp":"2023-11-24T06:10:23.635255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:09:05.283059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.885105+0000","last_change":"2023-11-24T06:10:25.713815+0000","last_active":"2023-11-24T06:10:26.885105+0000","last_peered":"2023-11-24T06:10:26.885105+0000","last_clean":"2023-11-24T06:10:26.885105+0000","last_became_active":"2023-11-24T06:10:25.713207+0000","last_became_peered":"2023-11-24T06:10:25.713207+0000","last_unstale":"2023-11-24T06:10:26.885105+0000","last_undegraded":"2023-11-24T06:10:26.885105+0000","last_fullsized":"2023-11-24T06:10:26.885105+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:46:00.341282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.716681+0000","last_change":"2023-11-24T06:10:24.700771+0000","last_active":"2023-11-24T06:10:26.716681+0000","last_peered":"2023-11-24T06:10:26.716681+0000","last_clean":"2023-11-24T06:10:26.716681+0000","last_became_active":"2023-11-24T06:10:24.699736+0000","last_became_peered":"2023-11-24T06:10:24.699736+0000","last_unstale":"2023-11-24T06:10:26.716681+0000","last_undegraded":"2023-11-24T06:10:26.716681+0000","last_fullsized":"2023-11-24T06:10:26.716681+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_clean_scrub_stamp":"2023-11-24T06:10:23.635255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:10:14.268052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.888828+0000","last_change":"2023-11-24T06:10:25.735798+0000","last_active":"2023-11-24T06:10:26.888828+0000","last_peered":"2023-11-24T06:10:26.888828+0000","last_clean":"2023-11-24T06:10:26.888828+0000","last_became_active":"2023-11-24T06:10:25.735161+0000","last_became_peered":"2023-11-24T06:10:25.735161+0000","last_unstale":"2023-11-24T06:10:26.888828+0000","last_undegraded":"2023-11-24T06:10:26.888828+0000","last_fullsized":"2023-11-24T06:10:26.888828+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:56:03.801033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.888836+0000","last_change":"2023-11-24T06:10:24.694962+0000","last_active":"2023-11-24T06:10:26.888836+0000","last_peered":"2023-11-24T06:10:26.888836+0000","last_clean":"2023-11-24T06:10:26.888836+0000","last_became_active":"2023-11-24T06:10:24.694740+0000","last_became_peered":"2023-11-24T06:10:24.694740+0000","last_unstale":"2023-11-24T06:10:26.888836+0000","last_undegraded":"2023-11-24T06:10:26.888836+0000","last_fullsized":"2023-11-24T06:10:26.888836+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_clean_scrub_stamp":"2023-11-24T06:10:23.635255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:45:59.324198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.887375+0000","last_change":"2023-11-24T06:10:25.732152+0000","last_active":"2023-11-24T06:10:26.887375+0000","last_peered":"2023-11-24T06:10:26.887375+0000","last_clean":"2023-11-24T06:10:26.887375+0000","last_became_active":"2023-11-24T06:10:25.731460+0000","last_became_peered":"2023-11-24T06:10:25.731460+0000","last_unstale":"2023-11-24T06:10:26.887375+0000","last_undegraded":"2023-11-24T06:10:26.887375+0000","last_fullsized":"2023-11-24T06:10:26.887375+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:53:47.062623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.885628+0000","last_change":"2023-11-24T06:10:24.692129+0000","last_active":"2023-11-24T06:10:26.885628+0000","last_peered":"2023-11-24T06:10:26.885628+0000","last_clean":"2023-11-24T06:10:26.885628+0000","last_became_active":"2023-11-24T06:10:24.691871+0000","last_became_peered":"2023-11-24T06:10:24.691871+0000","last_unstale":"2023-11-24T06:10:26.885628+0000","last_undegraded":"2023-11-24T06:10:26.885628+0000","last_fullsized":"2023-11-24T06:10:26.885628+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_clean_scrub_stamp":"2023-11-24T06:10:23.635255+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:14:19.359955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.717432+0000","last_change":"2023-11-24T06:10:25.718217+0000","last_active":"2023-11-24T06:10:26.717432+0000","last_peered":"2023-11-24T06:10:26.717432+0000","last_clean":"2023-11-24T06:10:26.717432+0000","last_became_active":"2023-11-24T06:10:25.717956+0000","last_became_peered":"2023-11-24T06:10:25.717956+0000","last_unstale":"2023-11-24T06:10:26.717432+0000","last_undegraded":"2023-11-24T06:10:26.717432+0000","last_fullsized":"2023-11-24T06:10:26.717432+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:09:06.300144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.716773+0000","last_change":"2023-11-24T06:10:24.694293+0000","last_active":"2023-11-24T06:10:26.716773+0000","last_peered":"2023-11-24T06:10:26.716773+0000","last_clean":"2023-11-24T06:10:26.716773+0000","last_became_active":"2023-11-24T06:10:24.694005+0000","last_became_peered":"2023-11-24T06:10:24.694005+0000","last_unstale":"2023-11-24T06:10:26.716773+0000","last_undegraded":"2023-11-24T06:10:26.716773+0000","last_fullsized":"2023-11-24T06:10:26.716773+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_clean_scrub_stamp":"2023-11-24T06:10:23.635255+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:54:01.820045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.887235+0000","last_change":"2023-11-24T06:10:25.716082+0000","last_active":"2023-11-24T06:10:26.887235+0000","last_peered":"2023-11-24T06:10:26.887235+0000","last_clean":"2023-11-24T06:10:26.887235+0000","last_became_active":"2023-11-24T06:10:25.713813+0000","last_became_peered":"2023-11-24T06:10:25.713813+0000","last_unstale":"2023-11-24T06:10:26.887235+0000","last_undegraded":"2023-11-24T06:10:26.887235+0000","last_fullsized":"2023-11-24T06:10:26.887235+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:42:25.852028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.886925+0000","last_change":"2023-11-24T06:10:24.687076+0000","last_active":"2023-11-24T06:10:26.886925+0000","last_peered":"2023-11-24T06:10:26.886925+0000","last_clean":"2023-11-24T06:10:26.886925+0000","last_became_active":"2023-11-24T06:10:24.686789+0000","last_became_peered":"2023-11-24T06:10:24.686789+0000","last_unstale":"2023-11-24T06:10:26.886925+0000","last_undegraded":"2023-11-24T06:10:26.886925+0000","last_fullsized":"2023-11-24T06:10:26.886925+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_clean_scrub_stamp":"2023-11-24T06:10:23.635255+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:38:10.950543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.885206+0000","last_change":"2023-11-24T06:10:25.704389+0000","last_active":"2023-11-24T06:10:26.885206+0000","last_peered":"2023-11-24T06:10:26.885206+0000","last_clean":"2023-11-24T06:10:26.885206+0000","last_became_active":"2023-11-24T06:10:25.704101+0000","last_became_peered":"2023-11-24T06:10:25.704101+0000","last_unstale":"2023-11-24T06:10:26.885206+0000","last_undegraded":"2023-11-24T06:10:26.885206+0000","last_fullsized":"2023-11-24T06:10:26.885206+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:54:02.837130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.887386+0000","last_change":"2023-11-24T06:10:24.691369+0000","last_active":"2023-11-24T06:10:26.887386+0000","last_peered":"2023-11-24T06:10:26.887386+0000","last_clean":"2023-11-24T06:10:26.887386+0000","last_became_active":"2023-11-24T06:10:24.691053+0000","last_became_peered":"2023-11-24T06:10:24.691053+0000","last_unstale":"2023-11-24T06:10:26.887386+0000","last_undegraded":"2023-11-24T06:10:26.887386+0000","last_fullsized":"2023-11-24T06:10:26.887386+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_clean_scrub_stamp":"2023-11-24T06:10:23.635255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:18:28.609106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.887207+0000","last_change":"2023-11-24T06:10:25.730194+0000","last_active":"2023-11-24T06:10:26.887207+0000","last_peered":"2023-11-24T06:10:26.887207+0000","last_clean":"2023-11-24T06:10:26.887207+0000","last_became_active":"2023-11-24T06:10:25.729811+0000","last_became_peered":"2023-11-24T06:10:25.729811+0000","last_unstale":"2023-11-24T06:10:26.887207+0000","last_undegraded":"2023-11-24T06:10:26.887207+0000","last_fullsized":"2023-11-24T06:10:26.887207+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:03:41.511367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.716013+0000","last_change":"2023-11-24T06:10:24.691658+0000","last_active":"2023-11-24T06:10:26.716013+0000","last_peered":"2023-11-24T06:10:26.716013+0000","last_clean":"2023-11-24T06:10:26.716013+0000","last_became_active":"2023-11-24T06:10:24.691365+0000","last_became_peered":"2023-11-24T06:10:24.691365+0000","last_unstale":"2023-11-24T06:10:26.716013+0000","last_undegraded":"2023-11-24T06:10:26.716013+0000","last_fullsized":"2023-11-24T06:10:26.716013+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_clean_scrub_stamp":"2023-11-24T06:10:23.635255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:17:02.081734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.715937+0000","last_change":"2023-11-24T06:10:25.703380+0000","last_active":"2023-11-24T06:10:26.715937+0000","last_peered":"2023-11-24T06:10:26.715937+0000","last_clean":"2023-11-24T06:10:26.715937+0000","last_became_active":"2023-11-24T06:10:25.702696+0000","last_became_peered":"2023-11-24T06:10:25.702696+0000","last_unstale":"2023-11-24T06:10:26.715937+0000","last_undegraded":"2023-11-24T06:10:26.715937+0000","last_fullsized":"2023-11-24T06:10:26.715937+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:18:29.626191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.825875+0000","last_change":"2023-11-24T06:10:24.683519+0000","last_active":"2023-11-24T06:10:26.825875+0000","last_peered":"2023-11-24T06:10:26.825875+0000","last_clean":"2023-11-24T06:10:26.825875+0000","last_became_active":"2023-11-24T06:10:24.682405+0000","last_became_peered":"2023-11-24T06:10:24.682405+0000","last_unstale":"2023-11-24T06:10:26.825875+0000","last_undegraded":"2023-11-24T06:10:26.825875+0000","last_fullsized":"2023-11-24T06:10:26.825875+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_clean_scrub_stamp":"2023-11-24T06:10:23.635255+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:20:39.581356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.886486+0000","last_change":"2023-11-24T06:10:25.726307+0000","last_active":"2023-11-24T06:10:26.886486+0000","last_peered":"2023-11-24T06:10:26.886486+0000","last_clean":"2023-11-24T06:10:26.886486+0000","last_became_active":"2023-11-24T06:10:25.726038+0000","last_became_peered":"2023-11-24T06:10:25.726038+0000","last_unstale":"2023-11-24T06:10:26.886486+0000","last_undegraded":"2023-11-24T06:10:26.886486+0000","last_fullsized":"2023-11-24T06:10:26.886486+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:00:56.115984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.717432+0000","last_change":"2023-11-24T06:10:24.692338+0000","last_active":"2023-11-24T06:10:26.717432+0000","last_peered":"2023-11-24T06:10:26.717432+0000","last_clean":"2023-11-24T06:10:26.717432+0000","last_became_active":"2023-11-24T06:10:24.692135+0000","last_became_peered":"2023-11-24T06:10:24.692135+0000","last_unstale":"2023-11-24T06:10:26.717432+0000","last_undegraded":"2023-11-24T06:10:26.717432+0000","last_fullsized":"2023-11-24T06:10:26.717432+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_clean_scrub_stamp":"2023-11-24T06:10:23.635255+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:57:48.799543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.887337+0000","last_change":"2023-11-24T06:10:25.729383+0000","last_active":"2023-11-24T06:10:26.887337+0000","last_peered":"2023-11-24T06:10:26.887337+0000","last_clean":"2023-11-24T06:10:26.887337+0000","last_became_active":"2023-11-24T06:10:25.729090+0000","last_became_peered":"2023-11-24T06:10:25.729090+0000","last_unstale":"2023-11-24T06:10:26.887337+0000","last_undegraded":"2023-11-24T06:10:26.887337+0000","last_fullsized":"2023-11-24T06:10:26.887337+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:40:47.125372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.826232+0000","last_change":"2023-11-24T06:10:24.689238+0000","last_active":"2023-11-24T06:10:26.826232+0000","last_peered":"2023-11-24T06:10:26.826232+0000","last_clean":"2023-11-24T06:10:26.826232+0000","last_became_active":"2023-11-24T06:10:24.686650+0000","last_became_peered":"2023-11-24T06:10:24.686650+0000","last_unstale":"2023-11-24T06:10:26.826232+0000","last_undegraded":"2023-11-24T06:10:26.826232+0000","last_fullsized":"2023-11-24T06:10:26.826232+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_clean_scrub_stamp":"2023-11-24T06:10:23.635255+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:06:50.936889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.887656+0000","last_change":"2023-11-24T06:10:25.726737+0000","last_active":"2023-11-24T06:10:26.887656+0000","last_peered":"2023-11-24T06:10:26.887656+0000","last_clean":"2023-11-24T06:10:26.887656+0000","last_became_active":"2023-11-24T06:10:25.725070+0000","last_became_peered":"2023-11-24T06:10:25.725070+0000","last_unstale":"2023-11-24T06:10:26.887656+0000","last_undegraded":"2023-11-24T06:10:26.887656+0000","last_fullsized":"2023-11-24T06:10:26.887656+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:22:18.598271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.885715+0000","last_change":"2023-11-24T06:10:24.681713+0000","last_active":"2023-11-24T06:10:26.885715+0000","last_peered":"2023-11-24T06:10:26.885715+0000","last_clean":"2023-11-24T06:10:26.885715+0000","last_became_active":"2023-11-24T06:10:24.681336+0000","last_became_peered":"2023-11-24T06:10:24.681336+0000","last_unstale":"2023-11-24T06:10:26.885715+0000","last_undegraded":"2023-11-24T06:10:26.885715+0000","last_fullsized":"2023-11-24T06:10:26.885715+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_clean_scrub_stamp":"2023-11-24T06:10:23.635255+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:10:14.268052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.888736+0000","last_change":"2023-11-24T06:10:25.733821+0000","last_active":"2023-11-24T06:10:26.888736+0000","last_peered":"2023-11-24T06:10:26.888736+0000","last_clean":"2023-11-24T06:10:26.888736+0000","last_became_active":"2023-11-24T06:10:25.733510+0000","last_became_peered":"2023-11-24T06:10:25.733510+0000","last_unstale":"2023-11-24T06:10:26.888736+0000","last_undegraded":"2023-11-24T06:10:26.888736+0000","last_fullsized":"2023-11-24T06:10:26.888736+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:42:25.852028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.826174+0000","last_change":"2023-11-24T06:10:24.690450+0000","last_active":"2023-11-24T06:10:26.826174+0000","last_peered":"2023-11-24T06:10:26.826174+0000","last_clean":"2023-11-24T06:10:26.826174+0000","last_became_active":"2023-11-24T06:10:24.690024+0000","last_became_peered":"2023-11-24T06:10:24.690024+0000","last_unstale":"2023-11-24T06:10:26.826174+0000","last_undegraded":"2023-11-24T06:10:26.826174+0000","last_fullsized":"2023-11-24T06:10:26.826174+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_clean_scrub_stamp":"2023-11-24T06:10:23.635255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:51:55.108153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.887639+0000","last_change":"2023-11-24T06:10:25.734784+0000","last_active":"2023-11-24T06:10:26.887639+0000","last_peered":"2023-11-24T06:10:26.887639+0000","last_clean":"2023-11-24T06:10:26.887639+0000","last_became_active":"2023-11-24T06:10:25.734478+0000","last_became_peered":"2023-11-24T06:10:25.734478+0000","last_unstale":"2023-11-24T06:10:26.887639+0000","last_undegraded":"2023-11-24T06:10:26.887639+0000","last_fullsized":"2023-11-24T06:10:26.887639+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:05:14.621899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":165,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:10:25.700259+0000","last_change":"2023-11-24T06:10:20.082677+0000","last_active":"2023-11-24T06:10:25.700259+0000","last_peered":"2023-11-24T06:10:25.700259+0000","last_clean":"2023-11-24T06:10:25.700259+0000","last_became_active":"2023-11-24T06:10:20.080551+0000","last_became_peered":"2023-11-24T06:10:20.080551+0000","last_unstale":"2023-11-24T06:10:25.700259+0000","last_undegraded":"2023-11-24T06:10:25.700259+0000","last_fullsized":"2023-11-24T06:10:25.700259+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:18.602209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:18.602209+0000","last_clean_scrub_stamp":"2023-11-24T06:10:18.602209+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:47:24.216235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.887865+0000","last_change":"2023-11-24T06:10:24.684957+0000","last_active":"2023-11-24T06:10:26.887865+0000","last_peered":"2023-11-24T06:10:26.887865+0000","last_clean":"2023-11-24T06:10:26.887865+0000","last_became_active":"2023-11-24T06:10:24.684507+0000","last_became_peered":"2023-11-24T06:10:24.684507+0000","last_unstale":"2023-11-24T06:10:26.887865+0000","last_undegraded":"2023-11-24T06:10:26.887865+0000","last_fullsized":"2023-11-24T06:10:26.887865+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_clean_scrub_stamp":"2023-11-24T06:10:23.635255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:52:31.270989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.715709+0000","last_change":"2023-11-24T06:10:25.735264+0000","last_active":"2023-11-24T06:10:26.715709+0000","last_peered":"2023-11-24T06:10:26.715709+0000","last_clean":"2023-11-24T06:10:26.715709+0000","last_became_active":"2023-11-24T06:10:25.734163+0000","last_became_peered":"2023-11-24T06:10:25.734163+0000","last_unstale":"2023-11-24T06:10:26.715709+0000","last_undegraded":"2023-11-24T06:10:26.715709+0000","last_fullsized":"2023-11-24T06:10:26.715709+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:45:21.592844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.885683+0000","last_change":"2023-11-24T06:10:24.699469+0000","last_active":"2023-11-24T06:10:26.885683+0000","last_peered":"2023-11-24T06:10:26.885683+0000","last_clean":"2023-11-24T06:10:26.885683+0000","last_became_active":"2023-11-24T06:10:24.699153+0000","last_became_peered":"2023-11-24T06:10:24.699153+0000","last_unstale":"2023-11-24T06:10:26.885683+0000","last_undegraded":"2023-11-24T06:10:26.885683+0000","last_fullsized":"2023-11-24T06:10:26.885683+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_clean_scrub_stamp":"2023-11-24T06:10:23.635255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:09:05.283059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.884898+0000","last_change":"2023-11-24T06:10:25.716501+0000","last_active":"2023-11-24T06:10:26.884898+0000","last_peered":"2023-11-24T06:10:26.884898+0000","last_clean":"2023-11-24T06:10:26.884898+0000","last_became_active":"2023-11-24T06:10:25.712127+0000","last_became_peered":"2023-11-24T06:10:25.712127+0000","last_unstale":"2023-11-24T06:10:26.884898+0000","last_undegraded":"2023-11-24T06:10:26.884898+0000","last_fullsized":"2023-11-24T06:10:26.884898+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:10:15.285136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.826108+0000","last_change":"2023-11-24T06:10:24.686855+0000","last_active":"2023-11-24T06:10:26.826108+0000","last_peered":"2023-11-24T06:10:26.826108+0000","last_clean":"2023-11-24T06:10:26.826108+0000","last_became_active":"2023-11-24T06:10:24.685894+0000","last_became_peered":"2023-11-24T06:10:24.685894+0000","last_unstale":"2023-11-24T06:10:26.826108+0000","last_undegraded":"2023-11-24T06:10:26.826108+0000","last_fullsized":"2023-11-24T06:10:26.826108+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_clean_scrub_stamp":"2023-11-24T06:10:23.635255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:38:10.950543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.885731+0000","last_change":"2023-11-24T06:10:25.715833+0000","last_active":"2023-11-24T06:10:26.885731+0000","last_peered":"2023-11-24T06:10:26.885731+0000","last_clean":"2023-11-24T06:10:26.885731+0000","last_became_active":"2023-11-24T06:10:25.715500+0000","last_became_peered":"2023-11-24T06:10:25.715500+0000","last_unstale":"2023-11-24T06:10:26.885731+0000","last_undegraded":"2023-11-24T06:10:26.885731+0000","last_fullsized":"2023-11-24T06:10:26.885731+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:43:50.924468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.887610+0000","last_change":"2023-11-24T06:10:24.690139+0000","last_active":"2023-11-24T06:10:26.887610+0000","last_peered":"2023-11-24T06:10:26.887610+0000","last_clean":"2023-11-24T06:10:26.887610+0000","last_became_active":"2023-11-24T06:10:24.689734+0000","last_became_peered":"2023-11-24T06:10:24.689734+0000","last_unstale":"2023-11-24T06:10:26.887610+0000","last_undegraded":"2023-11-24T06:10:26.887610+0000","last_fullsized":"2023-11-24T06:10:26.887610+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_clean_scrub_stamp":"2023-11-24T06:10:23.635255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:45:20.575759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.717184+0000","last_change":"2023-11-24T06:10:25.710787+0000","last_active":"2023-11-24T06:10:26.717184+0000","last_peered":"2023-11-24T06:10:26.717184+0000","last_clean":"2023-11-24T06:10:26.717184+0000","last_became_active":"2023-11-24T06:10:25.710522+0000","last_became_peered":"2023-11-24T06:10:25.710522+0000","last_unstale":"2023-11-24T06:10:26.717184+0000","last_undegraded":"2023-11-24T06:10:26.717184+0000","last_fullsized":"2023-11-24T06:10:26.717184+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:17:03.098818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.885579+0000","last_change":"2023-11-24T06:10:24.683405+0000","last_active":"2023-11-24T06:10:26.885579+0000","last_peered":"2023-11-24T06:10:26.885579+0000","last_clean":"2023-11-24T06:10:26.885579+0000","last_became_active":"2023-11-24T06:10:24.683144+0000","last_became_peered":"2023-11-24T06:10:24.683144+0000","last_unstale":"2023-11-24T06:10:26.885579+0000","last_undegraded":"2023-11-24T06:10:26.885579+0000","last_fullsized":"2023-11-24T06:10:26.885579+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_clean_scrub_stamp":"2023-11-24T06:10:23.635255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:18:53.146509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.888979+0000","last_change":"2023-11-24T06:10:25.734738+0000","last_active":"2023-11-24T06:10:26.888979+0000","last_peered":"2023-11-24T06:10:26.888979+0000","last_clean":"2023-11-24T06:10:26.888979+0000","last_became_active":"2023-11-24T06:10:25.733409+0000","last_became_peered":"2023-11-24T06:10:25.733409+0000","last_unstale":"2023-11-24T06:10:26.888979+0000","last_undegraded":"2023-11-24T06:10:26.888979+0000","last_fullsized":"2023-11-24T06:10:26.888979+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:59:37.010936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.888082+0000","last_change":"2023-11-24T06:10:24.701942+0000","last_active":"2023-11-24T06:10:26.888082+0000","last_peered":"2023-11-24T06:10:26.888082+0000","last_clean":"2023-11-24T06:10:26.888082+0000","last_became_active":"2023-11-24T06:10:24.701385+0000","last_became_peered":"2023-11-24T06:10:24.701385+0000","last_unstale":"2023-11-24T06:10:26.888082+0000","last_undegraded":"2023-11-24T06:10:26.888082+0000","last_fullsized":"2023-11-24T06:10:26.888082+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_clean_scrub_stamp":"2023-11-24T06:10:23.635255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:26:39.190781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.885358+0000","last_change":"2023-11-24T06:10:25.711825+0000","last_active":"2023-11-24T06:10:26.885358+0000","last_peered":"2023-11-24T06:10:26.885358+0000","last_clean":"2023-11-24T06:10:26.885358+0000","last_became_active":"2023-11-24T06:10:25.711161+0000","last_became_peered":"2023-11-24T06:10:25.711161+0000","last_unstale":"2023-11-24T06:10:26.885358+0000","last_undegraded":"2023-11-24T06:10:26.885358+0000","last_fullsized":"2023-11-24T06:10:26.885358+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:45:21.592844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:27.899957+0000","last_change":"2023-11-24T06:10:24.695283+0000","last_active":"2023-11-24T06:10:27.899957+0000","last_peered":"2023-11-24T06:10:27.899957+0000","last_clean":"2023-11-24T06:10:27.899957+0000","last_became_active":"2023-11-24T06:10:24.694723+0000","last_became_peered":"2023-11-24T06:10:24.694723+0000","last_unstale":"2023-11-24T06:10:27.899957+0000","last_undegraded":"2023-11-24T06:10:27.899957+0000","last_fullsized":"2023-11-24T06:10:27.899957+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_clean_scrub_stamp":"2023-11-24T06:10:23.635255+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:52:31.270989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.716963+0000","last_change":"2023-11-24T06:10:25.730440+0000","last_active":"2023-11-24T06:10:26.716963+0000","last_peered":"2023-11-24T06:10:26.716963+0000","last_clean":"2023-11-24T06:10:26.716963+0000","last_became_active":"2023-11-24T06:10:25.730097+0000","last_became_peered":"2023-11-24T06:10:25.730097+0000","last_unstale":"2023-11-24T06:10:26.716963+0000","last_undegraded":"2023-11-24T06:10:26.716963+0000","last_fullsized":"2023-11-24T06:10:26.716963+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:44:12.726705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.716893+0000","last_change":"2023-11-24T06:10:24.702923+0000","last_active":"2023-11-24T06:10:26.716893+0000","last_peered":"2023-11-24T06:10:26.716893+0000","last_clean":"2023-11-24T06:10:26.716893+0000","last_became_active":"2023-11-24T06:10:24.702428+0000","last_became_peered":"2023-11-24T06:10:24.702428+0000","last_unstale":"2023-11-24T06:10:26.716893+0000","last_undegraded":"2023-11-24T06:10:26.716893+0000","last_fullsized":"2023-11-24T06:10:26.716893+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_clean_scrub_stamp":"2023-11-24T06:10:23.635255+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:52:31.270989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.886761+0000","last_change":"2023-11-24T06:10:25.709739+0000","last_active":"2023-11-24T06:10:26.886761+0000","last_peered":"2023-11-24T06:10:26.886761+0000","last_clean":"2023-11-24T06:10:26.886761+0000","last_became_active":"2023-11-24T06:10:25.709392+0000","last_became_peered":"2023-11-24T06:10:25.709392+0000","last_unstale":"2023-11-24T06:10:26.886761+0000","last_undegraded":"2023-11-24T06:10:26.886761+0000","last_fullsized":"2023-11-24T06:10:26.886761+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:45:21.592844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.886005+0000","last_change":"2023-11-24T06:10:24.698896+0000","last_active":"2023-11-24T06:10:26.886005+0000","last_peered":"2023-11-24T06:10:26.886005+0000","last_clean":"2023-11-24T06:10:26.886005+0000","last_became_active":"2023-11-24T06:10:24.698632+0000","last_became_peered":"2023-11-24T06:10:24.698632+0000","last_unstale":"2023-11-24T06:10:26.886005+0000","last_undegraded":"2023-11-24T06:10:26.886005+0000","last_fullsized":"2023-11-24T06:10:26.886005+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_clean_scrub_stamp":"2023-11-24T06:10:23.635255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:22:19.918119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.715380+0000","last_change":"2023-11-24T06:10:25.716284+0000","last_active":"2023-11-24T06:10:26.715380+0000","last_peered":"2023-11-24T06:10:26.715380+0000","last_clean":"2023-11-24T06:10:26.715380+0000","last_became_active":"2023-11-24T06:10:25.715948+0000","last_became_peered":"2023-11-24T06:10:25.715948+0000","last_unstale":"2023-11-24T06:10:26.715380+0000","last_undegraded":"2023-11-24T06:10:26.715380+0000","last_fullsized":"2023-11-24T06:10:26.715380+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:20:40.598440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.885366+0000","last_change":"2023-11-24T06:10:24.692784+0000","last_active":"2023-11-24T06:10:26.885366+0000","last_peered":"2023-11-24T06:10:26.885366+0000","last_clean":"2023-11-24T06:10:26.885366+0000","last_became_active":"2023-11-24T06:10:24.692514+0000","last_became_peered":"2023-11-24T06:10:24.692514+0000","last_unstale":"2023-11-24T06:10:26.885366+0000","last_undegraded":"2023-11-24T06:10:26.885366+0000","last_fullsized":"2023-11-24T06:10:26.885366+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_clean_scrub_stamp":"2023-11-24T06:10:23.635255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:38:10.950543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.715214+0000","last_change":"2023-11-24T06:10:25.712156+0000","last_active":"2023-11-24T06:10:26.715214+0000","last_peered":"2023-11-24T06:10:26.715214+0000","last_clean":"2023-11-24T06:10:26.715214+0000","last_became_active":"2023-11-24T06:10:25.711924+0000","last_became_peered":"2023-11-24T06:10:25.711924+0000","last_unstale":"2023-11-24T06:10:26.715214+0000","last_undegraded":"2023-11-24T06:10:26.715214+0000","last_fullsized":"2023-11-24T06:10:26.715214+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:46:00.341282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.885937+0000","last_change":"2023-11-24T06:10:24.695919+0000","last_active":"2023-11-24T06:10:26.885937+0000","last_peered":"2023-11-24T06:10:26.885937+0000","last_clean":"2023-11-24T06:10:26.885937+0000","last_became_active":"2023-11-24T06:10:24.695650+0000","last_became_peered":"2023-11-24T06:10:24.695650+0000","last_unstale":"2023-11-24T06:10:26.885937+0000","last_undegraded":"2023-11-24T06:10:26.885937+0000","last_fullsized":"2023-11-24T06:10:26.885937+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_clean_scrub_stamp":"2023-11-24T06:10:23.635255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:09:05.283059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.716052+0000","last_change":"2023-11-24T06:10:25.719797+0000","last_active":"2023-11-24T06:10:26.716052+0000","last_peered":"2023-11-24T06:10:26.716052+0000","last_clean":"2023-11-24T06:10:26.716052+0000","last_became_active":"2023-11-24T06:10:25.719483+0000","last_became_peered":"2023-11-24T06:10:25.719483+0000","last_unstale":"2023-11-24T06:10:26.716052+0000","last_undegraded":"2023-11-24T06:10:26.716052+0000","last_fullsized":"2023-11-24T06:10:26.716052+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:43:50.924468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.887695+0000","last_change":"2023-11-24T06:10:24.699236+0000","last_active":"2023-11-24T06:10:26.887695+0000","last_peered":"2023-11-24T06:10:26.887695+0000","last_clean":"2023-11-24T06:10:26.887695+0000","last_became_active":"2023-11-24T06:10:24.698709+0000","last_became_peered":"2023-11-24T06:10:24.698709+0000","last_unstale":"2023-11-24T06:10:26.887695+0000","last_undegraded":"2023-11-24T06:10:26.887695+0000","last_fullsized":"2023-11-24T06:10:26.887695+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_clean_scrub_stamp":"2023-11-24T06:10:23.635255+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:37:00.276208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.716147+0000","last_change":"2023-11-24T06:10:25.714752+0000","last_active":"2023-11-24T06:10:26.716147+0000","last_peered":"2023-11-24T06:10:26.716147+0000","last_clean":"2023-11-24T06:10:26.716147+0000","last_became_active":"2023-11-24T06:10:25.714468+0000","last_became_peered":"2023-11-24T06:10:25.714468+0000","last_unstale":"2023-11-24T06:10:26.716147+0000","last_undegraded":"2023-11-24T06:10:26.716147+0000","last_fullsized":"2023-11-24T06:10:26.716147+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:45:21.592844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.888957+0000","last_change":"2023-11-24T06:10:25.719973+0000","last_active":"2023-11-24T06:10:26.888957+0000","last_peered":"2023-11-24T06:10:26.888957+0000","last_clean":"2023-11-24T06:10:26.888957+0000","last_became_active":"2023-11-24T06:10:25.719716+0000","last_became_peered":"2023-11-24T06:10:25.719716+0000","last_unstale":"2023-11-24T06:10:26.888957+0000","last_undegraded":"2023-11-24T06:10:26.888957+0000","last_fullsized":"2023-11-24T06:10:26.888957+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:40:47.125372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.886889+0000","last_change":"2023-11-24T06:10:24.701875+0000","last_active":"2023-11-24T06:10:26.886889+0000","last_peered":"2023-11-24T06:10:26.886889+0000","last_clean":"2023-11-24T06:10:26.886889+0000","last_became_active":"2023-11-24T06:10:24.701520+0000","last_became_peered":"2023-11-24T06:10:24.701520+0000","last_unstale":"2023-11-24T06:10:26.886889+0000","last_undegraded":"2023-11-24T06:10:26.886889+0000","last_fullsized":"2023-11-24T06:10:26.886889+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_clean_scrub_stamp":"2023-11-24T06:10:23.635255+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:18:53.146509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.888401+0000","last_change":"2023-11-24T06:10:25.729759+0000","last_active":"2023-11-24T06:10:26.888401+0000","last_peered":"2023-11-24T06:10:26.888401+0000","last_clean":"2023-11-24T06:10:26.888401+0000","last_became_active":"2023-11-24T06:10:25.721916+0000","last_became_peered":"2023-11-24T06:10:25.721916+0000","last_unstale":"2023-11-24T06:10:26.888401+0000","last_undegraded":"2023-11-24T06:10:26.888401+0000","last_fullsized":"2023-11-24T06:10:26.888401+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:03:41.511367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.887901+0000","last_change":"2023-11-24T06:10:24.699400+0000","last_active":"2023-11-24T06:10:26.887901+0000","last_peered":"2023-11-24T06:10:26.887901+0000","last_clean":"2023-11-24T06:10:26.887901+0000","last_became_active":"2023-11-24T06:10:24.699090+0000","last_became_peered":"2023-11-24T06:10:24.699090+0000","last_unstale":"2023-11-24T06:10:26.887901+0000","last_undegraded":"2023-11-24T06:10:26.887901+0000","last_fullsized":"2023-11-24T06:10:26.887901+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_clean_scrub_stamp":"2023-11-24T06:10:23.635255+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:14:19.359955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.887209+0000","last_change":"2023-11-24T06:10:25.732187+0000","last_active":"2023-11-24T06:10:26.887209+0000","last_peered":"2023-11-24T06:10:26.887209+0000","last_clean":"2023-11-24T06:10:26.887209+0000","last_became_active":"2023-11-24T06:10:25.731130+0000","last_became_peered":"2023-11-24T06:10:25.731130+0000","last_unstale":"2023-11-24T06:10:26.887209+0000","last_undegraded":"2023-11-24T06:10:26.887209+0000","last_fullsized":"2023-11-24T06:10:26.887209+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:37:01.293292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.716804+0000","last_change":"2023-11-24T06:10:24.694116+0000","last_active":"2023-11-24T06:10:26.716804+0000","last_peered":"2023-11-24T06:10:26.716804+0000","last_clean":"2023-11-24T06:10:26.716804+0000","last_became_active":"2023-11-24T06:10:24.693832+0000","last_became_peered":"2023-11-24T06:10:24.693832+0000","last_unstale":"2023-11-24T06:10:26.716804+0000","last_undegraded":"2023-11-24T06:10:26.716804+0000","last_fullsized":"2023-11-24T06:10:26.716804+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_clean_scrub_stamp":"2023-11-24T06:10:23.635255+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:38:10.950543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.716865+0000","last_change":"2023-11-24T06:10:25.707741+0000","last_active":"2023-11-24T06:10:26.716865+0000","last_peered":"2023-11-24T06:10:26.716865+0000","last_clean":"2023-11-24T06:10:26.716865+0000","last_became_active":"2023-11-24T06:10:25.707453+0000","last_became_peered":"2023-11-24T06:10:25.707453+0000","last_unstale":"2023-11-24T06:10:26.716865+0000","last_undegraded":"2023-11-24T06:10:26.716865+0000","last_fullsized":"2023-11-24T06:10:26.716865+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:26:40.207866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.887375+0000","last_change":"2023-11-24T06:10:24.699025+0000","last_active":"2023-11-24T06:10:26.887375+0000","last_peered":"2023-11-24T06:10:26.887375+0000","last_clean":"2023-11-24T06:10:26.887375+0000","last_became_active":"2023-11-24T06:10:24.698788+0000","last_became_peered":"2023-11-24T06:10:24.698788+0000","last_unstale":"2023-11-24T06:10:26.887375+0000","last_undegraded":"2023-11-24T06:10:26.887375+0000","last_fullsized":"2023-11-24T06:10:26.887375+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_clean_scrub_stamp":"2023-11-24T06:10:23.635255+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:18:53.146509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574853,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31964,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339876,"statfs":{"total":96636764160,"available":96604033024,"internally_reserved":0,"allocated":802816,"data_stored":661860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9630000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0259999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0720000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.593}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.232}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.413}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4790000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31400,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340440,"statfs":{"total":96636764160,"available":96604610560,"internally_reserved":0,"allocated":217088,"data_stored":72522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.579}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1719999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.294}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.784}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1440000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0659999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1099999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31388,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340452,"statfs":{"total":96636764160,"available":96604622848,"internally_reserved":0,"allocated":204800,"data_stored":70386,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6890000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.528}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4470000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7869999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.885}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.4300000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.2650000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574854,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31396,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340444,"statfs":{"total":96636764160,"available":96604614656,"internally_reserved":0,"allocated":217088,"data_stored":71036,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0660000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5580000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83399999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0309999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1819999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2350000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5}]}]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":217088,"data_stored":72522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.292}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.5779999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3879999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.3980000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0099999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.169}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.3090000000000002}]}]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":196,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":200704,"data_stored":70296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56200000000000006}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0489999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.27}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68300000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.306}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6180000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5820000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":212992,"data_stored":71070,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77400000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.462}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84299999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.429}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5529999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.333}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.52}]}]},{"osd":0,"up_from":13,"seq":55834574853,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27812,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344028,"statfs":{"total":96636764160,"available":96608284672,"internally_reserved":0,"allocated":811008,"data_stored":661916,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9259999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.5139999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.5840000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.036}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.2290000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.8860000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.9369999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-24T06:10:35.609 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-24T06:10:35.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.852+0000 7f3f98d01640 1 Processor -- start 2023-11-24T06:10:35.856 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.852+0000 7f3f98d01640 1 -- start start 2023-11-24T06:10:35.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.854+0000 7f3f98d01640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f3f940cf9a0 0x7f3f9415dca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:35.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.854+0000 7f3f98d01640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f940b1ee0 con 0x7f3f94143ae0 2023-11-24T06:10:35.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.854+0000 7f3f98d01640 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f940b2780 con 0x7f3f9415e1e0 2023-11-24T06:10:35.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.854+0000 7f3f98d01640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f9415e990 con 0x7f3f940cf9a0 2023-11-24T06:10:35.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.854+0000 7f3f8bfff640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f3f940cf9a0 0x7f3f9415dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:35.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.854+0000 7f3f9259c640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f3f94143ae0 0x7f3f940cf290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:55892/0 (socket says 172.21.15.92:55892) 2023-11-24T06:10:35.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.854+0000 7f3f9259c640 1 -- 172.21.15.92:0/4120596405 learned_addr learned my addr 172.21.15.92:0/4120596405 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-24T06:10:35.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.854+0000 7f3f8bfff640 1 -- 172.21.15.92:0/4120596405 >> v1:172.21.15.118:6789/0 conn(0x7f3f9415e1e0 legacy=0x7f3f940702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:35.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.854+0000 7f3f8bfff640 1 -- 172.21.15.92:0/4120596405 >> v1:172.21.15.92:6789/0 conn(0x7f3f94143ae0 legacy=0x7f3f940cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:35.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.854+0000 7f3f8bfff640 1 -- 172.21.15.92:0/4120596405 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f940b2130 con 0x7f3f940cf9a0 2023-11-24T06:10:35.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.855+0000 7f3f8bfff640 1 --2- 172.21.15.92:0/4120596405 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3f940cf9a0 0x7f3f9415dca0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f3f7c0097a0 tx=0x7f3f7c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c133476716732cc server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:35.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.856+0000 7f3f91d9b640 1 -- 172.21.15.92:0/4120596405 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f7c031830 con 0x7f3f940cf9a0 2023-11-24T06:10:35.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.856+0000 7f3f91d9b640 1 -- 172.21.15.92:0/4120596405 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f7c031990 con 0x7f3f940cf9a0 2023-11-24T06:10:35.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.856+0000 7f3f91d9b640 1 -- 172.21.15.92:0/4120596405 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f7c031d20 con 0x7f3f940cf9a0 2023-11-24T06:10:35.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.856+0000 7f3f98d01640 1 -- 172.21.15.92:0/4120596405 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3f940cf9a0 msgr2=0x7f3f9415dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:35.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.856+0000 7f3f98d01640 1 --2- 172.21.15.92:0/4120596405 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3f940cf9a0 0x7f3f9415dca0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f3f7c0097a0 tx=0x7f3f7c02b8d0 comp rx=0 tx=0).stop 2023-11-24T06:10:35.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.856+0000 7f3f98d01640 1 -- 172.21.15.92:0/4120596405 shutdown_connections 2023-11-24T06:10:35.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.856+0000 7f3f98d01640 1 --2- 172.21.15.92:0/4120596405 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3f940cf9a0 0x7f3f9415dca0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:35.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.856+0000 7f3f98d01640 1 -- 172.21.15.92:0/4120596405 >> 172.21.15.92:0/4120596405 conn(0x7f3f9406a800 msgr2=0x7f3f9406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:35.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.856+0000 7f3f98d01640 1 -- 172.21.15.92:0/4120596405 shutdown_connections 2023-11-24T06:10:35.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.856+0000 7f3f98d01640 1 -- 172.21.15.92:0/4120596405 wait complete. 2023-11-24T06:10:35.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.858+0000 7f3f98d01640 1 Processor -- start 2023-11-24T06:10:35.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.858+0000 7f3f98d01640 1 -- start start 2023-11-24T06:10:35.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.859+0000 7f3f98d01640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3f940cf9a0 0x7f3f94159900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:35.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.859+0000 7f3f98d01640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3f94143ae0 0x7f3f940a84c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:35.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.859+0000 7f3f98d01640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3f9415e1e0 0x7f3f940a8a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:35.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.859+0000 7f3f98d01640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f940a9cb0 con 0x7f3f940cf9a0 2023-11-24T06:10:35.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.859+0000 7f3f98d01640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f94159e40 con 0x7f3f9415e1e0 2023-11-24T06:10:35.865 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.859+0000 7f3f98d01640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f941367c0 con 0x7f3f94143ae0 2023-11-24T06:10:35.865 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.859+0000 7f3f9259c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3f940cf9a0 0x7f3f94159900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:35.865 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.859+0000 7f3f9259c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3f940cf9a0 0x7f3f94159900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47574/0 (socket says 172.21.15.92:47574) 2023-11-24T06:10:35.865 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.859+0000 7f3f9259c640 1 -- 172.21.15.92:0/2080706923 learned_addr learned my addr 172.21.15.92:0/2080706923 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:10:35.865 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.860+0000 7f3f9259c640 1 -- 172.21.15.92:0/2080706923 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3f94143ae0 msgr2=0x7f3f940a84c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:35.865 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.860+0000 7f3f9259c640 1 --2- 172.21.15.92:0/2080706923 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3f94143ae0 0x7f3f940a84c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:35.865 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.860+0000 7f3f9259c640 1 -- 172.21.15.92:0/2080706923 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3f9415e1e0 msgr2=0x7f3f940a8a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:35.865 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.860+0000 7f3f9259c640 1 --2- 172.21.15.92:0/2080706923 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3f9415e1e0 0x7f3f940a8a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:35.865 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.860+0000 7f3f9259c640 1 -- 172.21.15.92:0/2080706923 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f7c009400 con 0x7f3f940cf9a0 2023-11-24T06:10:35.866 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.860+0000 7f3f9259c640 1 --2- 172.21.15.92:0/2080706923 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3f940cf9a0 0x7f3f94159900 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f3f8400c4a0 tx=0x7f3f8400c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:35.866 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.860+0000 7f3f8affd640 1 -- 172.21.15.92:0/2080706923 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f84005ce0 con 0x7f3f940cf9a0 2023-11-24T06:10:35.866 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.860+0000 7f3f8affd640 1 -- 172.21.15.92:0/2080706923 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f84005e40 con 0x7f3f940cf9a0 2023-11-24T06:10:35.866 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.860+0000 7f3f8affd640 1 -- 172.21.15.92:0/2080706923 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f84007270 con 0x7f3f940cf9a0 2023-11-24T06:10:35.866 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.860+0000 7f3f98d01640 1 -- 172.21.15.92:0/2080706923 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f940a9030 con 0x7f3f940cf9a0 2023-11-24T06:10:35.866 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.860+0000 7f3f98d01640 1 -- 172.21.15.92:0/2080706923 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f940a95f0 con 0x7f3f940cf9a0 2023-11-24T06:10:35.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.862+0000 7f3f98d01640 1 -- 172.21.15.92:0/2080706923 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f940675d0 con 0x7f3f940cf9a0 2023-11-24T06:10:35.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.862+0000 7f3f8affd640 1 -- 172.21.15.92:0/2080706923 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f3f84007490 con 0x7f3f940cf9a0 2023-11-24T06:10:35.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.864+0000 7f3f8affd640 1 --2- 172.21.15.92:0/2080706923 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f3f7406ae40 0x7f3f7406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:35.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.864+0000 7f3f8affd640 1 -- 172.21.15.92:0/2080706923 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3f84088a40 con 0x7f3f940cf9a0 2023-11-24T06:10:35.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.864+0000 7f3f8bfff640 1 --2- 172.21.15.92:0/2080706923 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f3f7406ae40 0x7f3f7406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:35.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.864+0000 7f3f8bfff640 1 --2- 172.21.15.92:0/2080706923 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f3f7406ae40 0x7f3f7406d300 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f3f7c036000 tx=0x7f3f7c030000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:35.871 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:35.868+0000 7f3f8affd640 1 -- 172.21.15.92:0/2080706923 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3f8408ae60 con 0x7f3f940cf9a0 2023-11-24T06:10:36.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:36.038+0000 7f3f98d01640 1 -- 172.21.15.92:0/2080706923 --> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3f9415e670 con 0x7f3f7406ae40 2023-11-24T06:10:36.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:36.058+0000 7f3f8affd640 1 -- 172.21.15.92:0/2080706923 <== mgr.4101 v2:172.21.15.92:6800/784027401 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264389 (secure 0 0 0) 0x7f3f7c03b430 con 0x7f3f7406ae40 2023-11-24T06:10:36.064 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-24T06:10:36.065 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2023-11-24T06:10:36.067 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:36.062+0000 7f3f98d01640 1 -- 172.21.15.92:0/2080706923 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f3f7406ae40 msgr2=0x7f3f7406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:36.067 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:36.062+0000 7f3f98d01640 1 --2- 172.21.15.92:0/2080706923 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f3f7406ae40 0x7f3f7406d300 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f3f7c036000 tx=0x7f3f7c030000 comp rx=0 tx=0).stop 2023-11-24T06:10:36.067 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:36.062+0000 7f3f98d01640 1 -- 172.21.15.92:0/2080706923 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3f940cf9a0 msgr2=0x7f3f94159900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:36.067 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:36.062+0000 7f3f98d01640 1 --2- 172.21.15.92:0/2080706923 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3f940cf9a0 0x7f3f94159900 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f3f8400c4a0 tx=0x7f3f8400c960 comp rx=0 tx=0).stop 2023-11-24T06:10:36.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:36.063+0000 7f3f98d01640 1 -- 172.21.15.92:0/2080706923 shutdown_connections 2023-11-24T06:10:36.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:36.063+0000 7f3f98d01640 1 --2- 172.21.15.92:0/2080706923 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f3f7406ae40 0x7f3f7406d300 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:36.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:36.063+0000 7f3f98d01640 1 --2- 172.21.15.92:0/2080706923 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3f9415e1e0 0x7f3f940a8a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:36.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:36.063+0000 7f3f98d01640 1 --2- 172.21.15.92:0/2080706923 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3f94143ae0 0x7f3f940a84c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:36.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:36.063+0000 7f3f98d01640 1 --2- 172.21.15.92:0/2080706923 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3f940cf9a0 0x7f3f94159900 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:36.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:36.063+0000 7f3f98d01640 1 -- 172.21.15.92:0/2080706923 >> 172.21.15.92:0/2080706923 conn(0x7f3f9406a800 msgr2=0x7f3f94070b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:36.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:36.063+0000 7f3f98d01640 1 -- 172.21.15.92:0/2080706923 shutdown_connections 2023-11-24T06:10:36.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:36.063+0000 7f3f98d01640 1 -- 172.21.15.92:0/2080706923 wait complete. 2023-11-24T06:10:36.082 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2023-11-24T06:10:35.670197+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235656,"kb_used_data":2816,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754739064,"statfs":{"total":773094113280,"available":772852801536,"internally_reserved":0,"allocated":2883584,"data_stored":1751608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":10,"num_read_kb":9,"num_write":59,"num_write_kb":276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.988491"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.887289+0000","last_change":"2023-11-24T06:10:25.718900+0000","last_active":"2023-11-24T06:10:26.887289+0000","last_peered":"2023-11-24T06:10:26.887289+0000","last_clean":"2023-11-24T06:10:26.887289+0000","last_became_active":"2023-11-24T06:10:25.718569+0000","last_became_peered":"2023-11-24T06:10:25.718569+0000","last_unstale":"2023-11-24T06:10:26.887289+0000","last_undegraded":"2023-11-24T06:10:26.887289+0000","last_fullsized":"2023-11-24T06:10:26.887289+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:56:03.801033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.885991+0000","last_change":"2023-11-24T06:10:25.711437+0000","last_active":"2023-11-24T06:10:26.885991+0000","last_peered":"2023-11-24T06:10:26.885991+0000","last_clean":"2023-11-24T06:10:26.885991+0000","last_became_active":"2023-11-24T06:10:25.710555+0000","last_became_peered":"2023-11-24T06:10:25.710555+0000","last_unstale":"2023-11-24T06:10:26.885991+0000","last_undegraded":"2023-11-24T06:10:26.885991+0000","last_fullsized":"2023-11-24T06:10:26.885991+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:37:01.293292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.886876+0000","last_change":"2023-11-24T06:10:25.716423+0000","last_active":"2023-11-24T06:10:26.886876+0000","last_peered":"2023-11-24T06:10:26.886876+0000","last_clean":"2023-11-24T06:10:26.886876+0000","last_became_active":"2023-11-24T06:10:25.716112+0000","last_became_peered":"2023-11-24T06:10:25.716112+0000","last_unstale":"2023-11-24T06:10:26.886876+0000","last_undegraded":"2023-11-24T06:10:26.886876+0000","last_fullsized":"2023-11-24T06:10:26.886876+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:43:50.924468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.888341+0000","last_change":"2023-11-24T06:10:25.733338+0000","last_active":"2023-11-24T06:10:26.888341+0000","last_peered":"2023-11-24T06:10:26.888341+0000","last_clean":"2023-11-24T06:10:26.888341+0000","last_became_active":"2023-11-24T06:10:25.733087+0000","last_became_peered":"2023-11-24T06:10:25.733087+0000","last_unstale":"2023-11-24T06:10:26.888341+0000","last_undegraded":"2023-11-24T06:10:26.888341+0000","last_fullsized":"2023-11-24T06:10:26.888341+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:00:56.115984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.887727+0000","last_change":"2023-11-24T06:10:25.721479+0000","last_active":"2023-11-24T06:10:26.887727+0000","last_peered":"2023-11-24T06:10:26.887727+0000","last_clean":"2023-11-24T06:10:26.887727+0000","last_became_active":"2023-11-24T06:10:25.721135+0000","last_became_peered":"2023-11-24T06:10:25.721135+0000","last_unstale":"2023-11-24T06:10:26.887727+0000","last_undegraded":"2023-11-24T06:10:26.887727+0000","last_fullsized":"2023-11-24T06:10:26.887727+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:14:05.090678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.886536+0000","last_change":"2023-11-24T06:10:25.731299+0000","last_active":"2023-11-24T06:10:26.886536+0000","last_peered":"2023-11-24T06:10:26.886536+0000","last_clean":"2023-11-24T06:10:26.886536+0000","last_became_active":"2023-11-24T06:10:25.730627+0000","last_became_peered":"2023-11-24T06:10:25.730627+0000","last_unstale":"2023-11-24T06:10:26.886536+0000","last_undegraded":"2023-11-24T06:10:26.886536+0000","last_fullsized":"2023-11-24T06:10:26.886536+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:53:47.062623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.885972+0000","last_change":"2023-11-24T06:10:25.710797+0000","last_active":"2023-11-24T06:10:26.885972+0000","last_peered":"2023-11-24T06:10:26.885972+0000","last_clean":"2023-11-24T06:10:26.885972+0000","last_became_active":"2023-11-24T06:10:25.710509+0000","last_became_peered":"2023-11-24T06:10:25.710509+0000","last_unstale":"2023-11-24T06:10:26.885972+0000","last_undegraded":"2023-11-24T06:10:26.885972+0000","last_fullsized":"2023-11-24T06:10:26.885972+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:44:12.726705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.714692+0000","last_change":"2023-11-24T06:10:25.721521+0000","last_active":"2023-11-24T06:10:26.714692+0000","last_peered":"2023-11-24T06:10:26.714692+0000","last_clean":"2023-11-24T06:10:26.714692+0000","last_became_active":"2023-11-24T06:10:25.721206+0000","last_became_peered":"2023-11-24T06:10:25.721206+0000","last_unstale":"2023-11-24T06:10:26.714692+0000","last_undegraded":"2023-11-24T06:10:26.714692+0000","last_fullsized":"2023-11-24T06:10:26.714692+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:52:32.288074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.887339+0000","last_change":"2023-11-24T06:10:25.743664+0000","last_active":"2023-11-24T06:10:26.887339+0000","last_peered":"2023-11-24T06:10:26.887339+0000","last_clean":"2023-11-24T06:10:26.887339+0000","last_became_active":"2023-11-24T06:10:25.743346+0000","last_became_peered":"2023-11-24T06:10:25.743346+0000","last_unstale":"2023-11-24T06:10:26.887339+0000","last_undegraded":"2023-11-24T06:10:26.887339+0000","last_fullsized":"2023-11-24T06:10:26.887339+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:59:37.010936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.716729+0000","last_change":"2023-11-24T06:10:25.722633+0000","last_active":"2023-11-24T06:10:26.716729+0000","last_peered":"2023-11-24T06:10:26.716729+0000","last_clean":"2023-11-24T06:10:26.716729+0000","last_became_active":"2023-11-24T06:10:25.722320+0000","last_became_peered":"2023-11-24T06:10:25.722320+0000","last_unstale":"2023-11-24T06:10:26.716729+0000","last_undegraded":"2023-11-24T06:10:26.716729+0000","last_fullsized":"2023-11-24T06:10:26.716729+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:37:01.293292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.716342+0000","last_change":"2023-11-24T06:10:25.721320+0000","last_active":"2023-11-24T06:10:26.716342+0000","last_peered":"2023-11-24T06:10:26.716342+0000","last_clean":"2023-11-24T06:10:26.716342+0000","last_became_active":"2023-11-24T06:10:25.720925+0000","last_became_peered":"2023-11-24T06:10:25.720925+0000","last_unstale":"2023-11-24T06:10:26.716342+0000","last_undegraded":"2023-11-24T06:10:26.716342+0000","last_fullsized":"2023-11-24T06:10:26.716342+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:40:01.407324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.716059+0000","last_change":"2023-11-24T06:10:25.700499+0000","last_active":"2023-11-24T06:10:26.716059+0000","last_peered":"2023-11-24T06:10:26.716059+0000","last_clean":"2023-11-24T06:10:26.716059+0000","last_became_active":"2023-11-24T06:10:25.700070+0000","last_became_peered":"2023-11-24T06:10:25.700070+0000","last_unstale":"2023-11-24T06:10:26.716059+0000","last_undegraded":"2023-11-24T06:10:26.716059+0000","last_fullsized":"2023-11-24T06:10:26.716059+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:22:20.935204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.716045+0000","last_change":"2023-11-24T06:10:25.714833+0000","last_active":"2023-11-24T06:10:26.716045+0000","last_peered":"2023-11-24T06:10:26.716045+0000","last_clean":"2023-11-24T06:10:26.716045+0000","last_became_active":"2023-11-24T06:10:25.714265+0000","last_became_peered":"2023-11-24T06:10:25.714265+0000","last_unstale":"2023-11-24T06:10:26.716045+0000","last_undegraded":"2023-11-24T06:10:26.716045+0000","last_fullsized":"2023-11-24T06:10:26.716045+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:40:01.407324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.885149+0000","last_change":"2023-11-24T06:10:25.710049+0000","last_active":"2023-11-24T06:10:26.885149+0000","last_peered":"2023-11-24T06:10:26.885149+0000","last_clean":"2023-11-24T06:10:26.885149+0000","last_became_active":"2023-11-24T06:10:25.709681+0000","last_became_peered":"2023-11-24T06:10:25.709681+0000","last_unstale":"2023-11-24T06:10:26.885149+0000","last_undegraded":"2023-11-24T06:10:26.885149+0000","last_fullsized":"2023-11-24T06:10:26.885149+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:52:32.288074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.885255+0000","last_change":"2023-11-24T06:10:25.713948+0000","last_active":"2023-11-24T06:10:26.885255+0000","last_peered":"2023-11-24T06:10:26.885255+0000","last_clean":"2023-11-24T06:10:26.885255+0000","last_became_active":"2023-11-24T06:10:25.713633+0000","last_became_peered":"2023-11-24T06:10:25.713633+0000","last_unstale":"2023-11-24T06:10:26.885255+0000","last_undegraded":"2023-11-24T06:10:26.885255+0000","last_fullsized":"2023-11-24T06:10:26.885255+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:18:29.626191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.885915+0000","last_change":"2023-11-24T06:10:25.733647+0000","last_active":"2023-11-24T06:10:26.885915+0000","last_peered":"2023-11-24T06:10:26.885915+0000","last_clean":"2023-11-24T06:10:26.885915+0000","last_became_active":"2023-11-24T06:10:25.733342+0000","last_became_peered":"2023-11-24T06:10:25.733342+0000","last_unstale":"2023-11-24T06:10:26.885915+0000","last_undegraded":"2023-11-24T06:10:26.885915+0000","last_fullsized":"2023-11-24T06:10:26.885915+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:43:50.924468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.889359+0000","last_change":"2023-11-24T06:10:25.737712+0000","last_active":"2023-11-24T06:10:26.889359+0000","last_peered":"2023-11-24T06:10:26.889359+0000","last_clean":"2023-11-24T06:10:26.889359+0000","last_became_active":"2023-11-24T06:10:25.735939+0000","last_became_peered":"2023-11-24T06:10:25.735939+0000","last_unstale":"2023-11-24T06:10:26.889359+0000","last_undegraded":"2023-11-24T06:10:26.889359+0000","last_fullsized":"2023-11-24T06:10:26.889359+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:05:14.621899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.826109+0000","last_change":"2023-11-24T06:10:25.717816+0000","last_active":"2023-11-24T06:10:26.826109+0000","last_peered":"2023-11-24T06:10:26.826109+0000","last_clean":"2023-11-24T06:10:26.826109+0000","last_became_active":"2023-11-24T06:10:25.717132+0000","last_became_peered":"2023-11-24T06:10:25.717132+0000","last_unstale":"2023-11-24T06:10:26.826109+0000","last_undegraded":"2023-11-24T06:10:26.826109+0000","last_fullsized":"2023-11-24T06:10:26.826109+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:17:03.098818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.888164+0000","last_change":"2023-11-24T06:10:25.724997+0000","last_active":"2023-11-24T06:10:26.888164+0000","last_peered":"2023-11-24T06:10:26.888164+0000","last_clean":"2023-11-24T06:10:26.888164+0000","last_became_active":"2023-11-24T06:10:25.716385+0000","last_became_peered":"2023-11-24T06:10:25.716385+0000","last_unstale":"2023-11-24T06:10:26.888164+0000","last_undegraded":"2023-11-24T06:10:26.888164+0000","last_fullsized":"2023-11-24T06:10:26.888164+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:01:41.923534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.716353+0000","last_change":"2023-11-24T06:10:25.717325+0000","last_active":"2023-11-24T06:10:26.716353+0000","last_peered":"2023-11-24T06:10:26.716353+0000","last_clean":"2023-11-24T06:10:26.716353+0000","last_became_active":"2023-11-24T06:10:25.716998+0000","last_became_peered":"2023-11-24T06:10:25.716998+0000","last_unstale":"2023-11-24T06:10:26.716353+0000","last_undegraded":"2023-11-24T06:10:26.716353+0000","last_fullsized":"2023-11-24T06:10:26.716353+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:43:50.924468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.886459+0000","last_change":"2023-11-24T06:10:25.729153+0000","last_active":"2023-11-24T06:10:26.886459+0000","last_peered":"2023-11-24T06:10:26.886459+0000","last_clean":"2023-11-24T06:10:26.886459+0000","last_became_active":"2023-11-24T06:10:25.724982+0000","last_became_peered":"2023-11-24T06:10:25.724982+0000","last_unstale":"2023-11-24T06:10:26.886459+0000","last_undegraded":"2023-11-24T06:10:26.886459+0000","last_fullsized":"2023-11-24T06:10:26.886459+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:46:00.341282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.886867+0000","last_change":"2023-11-24T06:10:25.734081+0000","last_active":"2023-11-24T06:10:26.886867+0000","last_peered":"2023-11-24T06:10:26.886867+0000","last_clean":"2023-11-24T06:10:26.886867+0000","last_became_active":"2023-11-24T06:10:25.733760+0000","last_became_peered":"2023-11-24T06:10:25.733760+0000","last_unstale":"2023-11-24T06:10:26.886867+0000","last_undegraded":"2023-11-24T06:10:26.886867+0000","last_fullsized":"2023-11-24T06:10:26.886867+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:26:40.207866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.715619+0000","last_change":"2023-11-24T06:10:25.720936+0000","last_active":"2023-11-24T06:10:26.715619+0000","last_peered":"2023-11-24T06:10:26.715619+0000","last_clean":"2023-11-24T06:10:26.715619+0000","last_became_active":"2023-11-24T06:10:25.720682+0000","last_became_peered":"2023-11-24T06:10:25.720682+0000","last_unstale":"2023-11-24T06:10:26.715619+0000","last_undegraded":"2023-11-24T06:10:26.715619+0000","last_fullsized":"2023-11-24T06:10:26.715619+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:18:29.626191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.886110+0000","last_change":"2023-11-24T06:10:25.730235+0000","last_active":"2023-11-24T06:10:26.886110+0000","last_peered":"2023-11-24T06:10:26.886110+0000","last_clean":"2023-11-24T06:10:26.886110+0000","last_became_active":"2023-11-24T06:10:25.729972+0000","last_became_peered":"2023-11-24T06:10:25.729972+0000","last_unstale":"2023-11-24T06:10:26.886110+0000","last_undegraded":"2023-11-24T06:10:26.886110+0000","last_fullsized":"2023-11-24T06:10:26.886110+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:37:01.293292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.716777+0000","last_change":"2023-11-24T06:10:25.719429+0000","last_active":"2023-11-24T06:10:26.716777+0000","last_peered":"2023-11-24T06:10:26.716777+0000","last_clean":"2023-11-24T06:10:26.716777+0000","last_became_active":"2023-11-24T06:10:25.719099+0000","last_became_peered":"2023-11-24T06:10:25.719099+0000","last_unstale":"2023-11-24T06:10:26.716777+0000","last_undegraded":"2023-11-24T06:10:26.716777+0000","last_fullsized":"2023-11-24T06:10:26.716777+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:53:47.062623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.716868+0000","last_change":"2023-11-24T06:10:25.710010+0000","last_active":"2023-11-24T06:10:26.716868+0000","last_peered":"2023-11-24T06:10:26.716868+0000","last_clean":"2023-11-24T06:10:26.716868+0000","last_became_active":"2023-11-24T06:10:25.709727+0000","last_became_peered":"2023-11-24T06:10:25.709727+0000","last_unstale":"2023-11-24T06:10:26.716868+0000","last_undegraded":"2023-11-24T06:10:26.716868+0000","last_fullsized":"2023-11-24T06:10:26.716868+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:53:47.062623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.885531+0000","last_change":"2023-11-24T06:10:25.713669+0000","last_active":"2023-11-24T06:10:26.885531+0000","last_peered":"2023-11-24T06:10:26.885531+0000","last_clean":"2023-11-24T06:10:26.885531+0000","last_became_active":"2023-11-24T06:10:25.713339+0000","last_became_peered":"2023-11-24T06:10:25.713339+0000","last_unstale":"2023-11-24T06:10:26.885531+0000","last_undegraded":"2023-11-24T06:10:26.885531+0000","last_fullsized":"2023-11-24T06:10:26.885531+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:40:01.407324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.716342+0000","last_change":"2023-11-24T06:10:25.721615+0000","last_active":"2023-11-24T06:10:26.716342+0000","last_peered":"2023-11-24T06:10:26.716342+0000","last_clean":"2023-11-24T06:10:26.716342+0000","last_became_active":"2023-11-24T06:10:25.721241+0000","last_became_peered":"2023-11-24T06:10:25.721241+0000","last_unstale":"2023-11-24T06:10:26.716342+0000","last_undegraded":"2023-11-24T06:10:26.716342+0000","last_fullsized":"2023-11-24T06:10:26.716342+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:37:01.293292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.886687+0000","last_change":"2023-11-24T06:10:25.713056+0000","last_active":"2023-11-24T06:10:26.886687+0000","last_peered":"2023-11-24T06:10:26.886687+0000","last_clean":"2023-11-24T06:10:26.886687+0000","last_became_active":"2023-11-24T06:10:25.712467+0000","last_became_peered":"2023-11-24T06:10:25.712467+0000","last_unstale":"2023-11-24T06:10:26.886687+0000","last_undegraded":"2023-11-24T06:10:26.886687+0000","last_fullsized":"2023-11-24T06:10:26.886687+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:18:29.626191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.887172+0000","last_change":"2023-11-24T06:10:25.733233+0000","last_active":"2023-11-24T06:10:26.887172+0000","last_peered":"2023-11-24T06:10:26.887172+0000","last_clean":"2023-11-24T06:10:26.887172+0000","last_became_active":"2023-11-24T06:10:25.732933+0000","last_became_peered":"2023-11-24T06:10:25.732933+0000","last_unstale":"2023-11-24T06:10:26.887172+0000","last_undegraded":"2023-11-24T06:10:26.887172+0000","last_fullsized":"2023-11-24T06:10:26.887172+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:44:12.726705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.826224+0000","last_change":"2023-11-24T06:10:25.722324+0000","last_active":"2023-11-24T06:10:26.826224+0000","last_peered":"2023-11-24T06:10:26.826224+0000","last_clean":"2023-11-24T06:10:26.826224+0000","last_became_active":"2023-11-24T06:10:25.722036+0000","last_became_peered":"2023-11-24T06:10:25.722036+0000","last_unstale":"2023-11-24T06:10:26.826224+0000","last_undegraded":"2023-11-24T06:10:26.826224+0000","last_fullsized":"2023-11-24T06:10:26.826224+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:09:06.300144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.715780+0000","last_change":"2023-11-24T06:10:25.703106+0000","last_active":"2023-11-24T06:10:26.715780+0000","last_peered":"2023-11-24T06:10:26.715780+0000","last_clean":"2023-11-24T06:10:26.715780+0000","last_became_active":"2023-11-24T06:10:25.701998+0000","last_became_peered":"2023-11-24T06:10:25.701998+0000","last_unstale":"2023-11-24T06:10:26.715780+0000","last_undegraded":"2023-11-24T06:10:26.715780+0000","last_fullsized":"2023-11-24T06:10:26.715780+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:46:00.341282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.715676+0000","last_change":"2023-11-24T06:10:25.708408+0000","last_active":"2023-11-24T06:10:26.715676+0000","last_peered":"2023-11-24T06:10:26.715676+0000","last_clean":"2023-11-24T06:10:26.715676+0000","last_became_active":"2023-11-24T06:10:25.708102+0000","last_became_peered":"2023-11-24T06:10:25.708102+0000","last_unstale":"2023-11-24T06:10:26.715676+0000","last_undegraded":"2023-11-24T06:10:26.715676+0000","last_fullsized":"2023-11-24T06:10:26.715676+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:38:11.967627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.716406+0000","last_change":"2023-11-24T06:10:24.696915+0000","last_active":"2023-11-24T06:10:26.716406+0000","last_peered":"2023-11-24T06:10:26.716406+0000","last_clean":"2023-11-24T06:10:26.716406+0000","last_became_active":"2023-11-24T06:10:24.696608+0000","last_became_peered":"2023-11-24T06:10:24.696608+0000","last_unstale":"2023-11-24T06:10:26.716406+0000","last_undegraded":"2023-11-24T06:10:26.716406+0000","last_fullsized":"2023-11-24T06:10:26.716406+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_clean_scrub_stamp":"2023-11-24T06:10:23.635255+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:14:19.359955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.886766+0000","last_change":"2023-11-24T06:10:25.713232+0000","last_active":"2023-11-24T06:10:26.886766+0000","last_peered":"2023-11-24T06:10:26.886766+0000","last_clean":"2023-11-24T06:10:26.886766+0000","last_became_active":"2023-11-24T06:10:25.712813+0000","last_became_peered":"2023-11-24T06:10:25.712813+0000","last_unstale":"2023-11-24T06:10:26.886766+0000","last_undegraded":"2023-11-24T06:10:26.886766+0000","last_fullsized":"2023-11-24T06:10:26.886766+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:40:01.407324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.716067+0000","last_change":"2023-11-24T06:10:24.689472+0000","last_active":"2023-11-24T06:10:26.716067+0000","last_peered":"2023-11-24T06:10:26.716067+0000","last_clean":"2023-11-24T06:10:26.716067+0000","last_became_active":"2023-11-24T06:10:24.689143+0000","last_became_peered":"2023-11-24T06:10:24.689143+0000","last_unstale":"2023-11-24T06:10:26.716067+0000","last_undegraded":"2023-11-24T06:10:26.716067+0000","last_fullsized":"2023-11-24T06:10:26.716067+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_clean_scrub_stamp":"2023-11-24T06:10:23.635255+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:18:53.146509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.715380+0000","last_change":"2023-11-24T06:10:24.694706+0000","last_active":"2023-11-24T06:10:26.715380+0000","last_peered":"2023-11-24T06:10:26.715380+0000","last_clean":"2023-11-24T06:10:26.715380+0000","last_became_active":"2023-11-24T06:10:24.693990+0000","last_became_peered":"2023-11-24T06:10:24.693990+0000","last_unstale":"2023-11-24T06:10:26.715380+0000","last_undegraded":"2023-11-24T06:10:26.715380+0000","last_fullsized":"2023-11-24T06:10:26.715380+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_clean_scrub_stamp":"2023-11-24T06:10:23.635255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:17:02.081734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.714912+0000","last_change":"2023-11-24T06:10:25.715897+0000","last_active":"2023-11-24T06:10:26.714912+0000","last_peered":"2023-11-24T06:10:26.714912+0000","last_clean":"2023-11-24T06:10:26.714912+0000","last_became_active":"2023-11-24T06:10:25.715596+0000","last_became_peered":"2023-11-24T06:10:25.715596+0000","last_unstale":"2023-11-24T06:10:26.714912+0000","last_undegraded":"2023-11-24T06:10:26.714912+0000","last_fullsized":"2023-11-24T06:10:26.714912+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:54:02.837130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.886161+0000","last_change":"2023-11-24T06:10:24.692669+0000","last_active":"2023-11-24T06:10:26.886161+0000","last_peered":"2023-11-24T06:10:26.886161+0000","last_clean":"2023-11-24T06:10:26.886161+0000","last_became_active":"2023-11-24T06:10:24.692341+0000","last_became_peered":"2023-11-24T06:10:24.692341+0000","last_unstale":"2023-11-24T06:10:26.886161+0000","last_undegraded":"2023-11-24T06:10:26.886161+0000","last_fullsized":"2023-11-24T06:10:26.886161+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_clean_scrub_stamp":"2023-11-24T06:10:23.635255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:09:05.283059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.885105+0000","last_change":"2023-11-24T06:10:25.713815+0000","last_active":"2023-11-24T06:10:26.885105+0000","last_peered":"2023-11-24T06:10:26.885105+0000","last_clean":"2023-11-24T06:10:26.885105+0000","last_became_active":"2023-11-24T06:10:25.713207+0000","last_became_peered":"2023-11-24T06:10:25.713207+0000","last_unstale":"2023-11-24T06:10:26.885105+0000","last_undegraded":"2023-11-24T06:10:26.885105+0000","last_fullsized":"2023-11-24T06:10:26.885105+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:46:00.341282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.716681+0000","last_change":"2023-11-24T06:10:24.700771+0000","last_active":"2023-11-24T06:10:26.716681+0000","last_peered":"2023-11-24T06:10:26.716681+0000","last_clean":"2023-11-24T06:10:26.716681+0000","last_became_active":"2023-11-24T06:10:24.699736+0000","last_became_peered":"2023-11-24T06:10:24.699736+0000","last_unstale":"2023-11-24T06:10:26.716681+0000","last_undegraded":"2023-11-24T06:10:26.716681+0000","last_fullsized":"2023-11-24T06:10:26.716681+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_clean_scrub_stamp":"2023-11-24T06:10:23.635255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:10:14.268052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.888828+0000","last_change":"2023-11-24T06:10:25.735798+0000","last_active":"2023-11-24T06:10:26.888828+0000","last_peered":"2023-11-24T06:10:26.888828+0000","last_clean":"2023-11-24T06:10:26.888828+0000","last_became_active":"2023-11-24T06:10:25.735161+0000","last_became_peered":"2023-11-24T06:10:25.735161+0000","last_unstale":"2023-11-24T06:10:26.888828+0000","last_undegraded":"2023-11-24T06:10:26.888828+0000","last_fullsized":"2023-11-24T06:10:26.888828+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:56:03.801033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.888836+0000","last_change":"2023-11-24T06:10:24.694962+0000","last_active":"2023-11-24T06:10:26.888836+0000","last_peered":"2023-11-24T06:10:26.888836+0000","last_clean":"2023-11-24T06:10:26.888836+0000","last_became_active":"2023-11-24T06:10:24.694740+0000","last_became_peered":"2023-11-24T06:10:24.694740+0000","last_unstale":"2023-11-24T06:10:26.888836+0000","last_undegraded":"2023-11-24T06:10:26.888836+0000","last_fullsized":"2023-11-24T06:10:26.888836+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_clean_scrub_stamp":"2023-11-24T06:10:23.635255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:45:59.324198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.887375+0000","last_change":"2023-11-24T06:10:25.732152+0000","last_active":"2023-11-24T06:10:26.887375+0000","last_peered":"2023-11-24T06:10:26.887375+0000","last_clean":"2023-11-24T06:10:26.887375+0000","last_became_active":"2023-11-24T06:10:25.731460+0000","last_became_peered":"2023-11-24T06:10:25.731460+0000","last_unstale":"2023-11-24T06:10:26.887375+0000","last_undegraded":"2023-11-24T06:10:26.887375+0000","last_fullsized":"2023-11-24T06:10:26.887375+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:53:47.062623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.885628+0000","last_change":"2023-11-24T06:10:24.692129+0000","last_active":"2023-11-24T06:10:26.885628+0000","last_peered":"2023-11-24T06:10:26.885628+0000","last_clean":"2023-11-24T06:10:26.885628+0000","last_became_active":"2023-11-24T06:10:24.691871+0000","last_became_peered":"2023-11-24T06:10:24.691871+0000","last_unstale":"2023-11-24T06:10:26.885628+0000","last_undegraded":"2023-11-24T06:10:26.885628+0000","last_fullsized":"2023-11-24T06:10:26.885628+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_clean_scrub_stamp":"2023-11-24T06:10:23.635255+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:14:19.359955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.717432+0000","last_change":"2023-11-24T06:10:25.718217+0000","last_active":"2023-11-24T06:10:26.717432+0000","last_peered":"2023-11-24T06:10:26.717432+0000","last_clean":"2023-11-24T06:10:26.717432+0000","last_became_active":"2023-11-24T06:10:25.717956+0000","last_became_peered":"2023-11-24T06:10:25.717956+0000","last_unstale":"2023-11-24T06:10:26.717432+0000","last_undegraded":"2023-11-24T06:10:26.717432+0000","last_fullsized":"2023-11-24T06:10:26.717432+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:09:06.300144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.716773+0000","last_change":"2023-11-24T06:10:24.694293+0000","last_active":"2023-11-24T06:10:26.716773+0000","last_peered":"2023-11-24T06:10:26.716773+0000","last_clean":"2023-11-24T06:10:26.716773+0000","last_became_active":"2023-11-24T06:10:24.694005+0000","last_became_peered":"2023-11-24T06:10:24.694005+0000","last_unstale":"2023-11-24T06:10:26.716773+0000","last_undegraded":"2023-11-24T06:10:26.716773+0000","last_fullsized":"2023-11-24T06:10:26.716773+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_clean_scrub_stamp":"2023-11-24T06:10:23.635255+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:54:01.820045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.887235+0000","last_change":"2023-11-24T06:10:25.716082+0000","last_active":"2023-11-24T06:10:26.887235+0000","last_peered":"2023-11-24T06:10:26.887235+0000","last_clean":"2023-11-24T06:10:26.887235+0000","last_became_active":"2023-11-24T06:10:25.713813+0000","last_became_peered":"2023-11-24T06:10:25.713813+0000","last_unstale":"2023-11-24T06:10:26.887235+0000","last_undegraded":"2023-11-24T06:10:26.887235+0000","last_fullsized":"2023-11-24T06:10:26.887235+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:42:25.852028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.886925+0000","last_change":"2023-11-24T06:10:24.687076+0000","last_active":"2023-11-24T06:10:26.886925+0000","last_peered":"2023-11-24T06:10:26.886925+0000","last_clean":"2023-11-24T06:10:26.886925+0000","last_became_active":"2023-11-24T06:10:24.686789+0000","last_became_peered":"2023-11-24T06:10:24.686789+0000","last_unstale":"2023-11-24T06:10:26.886925+0000","last_undegraded":"2023-11-24T06:10:26.886925+0000","last_fullsized":"2023-11-24T06:10:26.886925+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_clean_scrub_stamp":"2023-11-24T06:10:23.635255+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:38:10.950543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.885206+0000","last_change":"2023-11-24T06:10:25.704389+0000","last_active":"2023-11-24T06:10:26.885206+0000","last_peered":"2023-11-24T06:10:26.885206+0000","last_clean":"2023-11-24T06:10:26.885206+0000","last_became_active":"2023-11-24T06:10:25.704101+0000","last_became_peered":"2023-11-24T06:10:25.704101+0000","last_unstale":"2023-11-24T06:10:26.885206+0000","last_undegraded":"2023-11-24T06:10:26.885206+0000","last_fullsized":"2023-11-24T06:10:26.885206+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:54:02.837130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.887386+0000","last_change":"2023-11-24T06:10:24.691369+0000","last_active":"2023-11-24T06:10:26.887386+0000","last_peered":"2023-11-24T06:10:26.887386+0000","last_clean":"2023-11-24T06:10:26.887386+0000","last_became_active":"2023-11-24T06:10:24.691053+0000","last_became_peered":"2023-11-24T06:10:24.691053+0000","last_unstale":"2023-11-24T06:10:26.887386+0000","last_undegraded":"2023-11-24T06:10:26.887386+0000","last_fullsized":"2023-11-24T06:10:26.887386+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_clean_scrub_stamp":"2023-11-24T06:10:23.635255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:18:28.609106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.887207+0000","last_change":"2023-11-24T06:10:25.730194+0000","last_active":"2023-11-24T06:10:26.887207+0000","last_peered":"2023-11-24T06:10:26.887207+0000","last_clean":"2023-11-24T06:10:26.887207+0000","last_became_active":"2023-11-24T06:10:25.729811+0000","last_became_peered":"2023-11-24T06:10:25.729811+0000","last_unstale":"2023-11-24T06:10:26.887207+0000","last_undegraded":"2023-11-24T06:10:26.887207+0000","last_fullsized":"2023-11-24T06:10:26.887207+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:03:41.511367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.716013+0000","last_change":"2023-11-24T06:10:24.691658+0000","last_active":"2023-11-24T06:10:26.716013+0000","last_peered":"2023-11-24T06:10:26.716013+0000","last_clean":"2023-11-24T06:10:26.716013+0000","last_became_active":"2023-11-24T06:10:24.691365+0000","last_became_peered":"2023-11-24T06:10:24.691365+0000","last_unstale":"2023-11-24T06:10:26.716013+0000","last_undegraded":"2023-11-24T06:10:26.716013+0000","last_fullsized":"2023-11-24T06:10:26.716013+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_clean_scrub_stamp":"2023-11-24T06:10:23.635255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:17:02.081734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.715937+0000","last_change":"2023-11-24T06:10:25.703380+0000","last_active":"2023-11-24T06:10:26.715937+0000","last_peered":"2023-11-24T06:10:26.715937+0000","last_clean":"2023-11-24T06:10:26.715937+0000","last_became_active":"2023-11-24T06:10:25.702696+0000","last_became_peered":"2023-11-24T06:10:25.702696+0000","last_unstale":"2023-11-24T06:10:26.715937+0000","last_undegraded":"2023-11-24T06:10:26.715937+0000","last_fullsized":"2023-11-24T06:10:26.715937+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:18:29.626191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.825875+0000","last_change":"2023-11-24T06:10:24.683519+0000","last_active":"2023-11-24T06:10:26.825875+0000","last_peered":"2023-11-24T06:10:26.825875+0000","last_clean":"2023-11-24T06:10:26.825875+0000","last_became_active":"2023-11-24T06:10:24.682405+0000","last_became_peered":"2023-11-24T06:10:24.682405+0000","last_unstale":"2023-11-24T06:10:26.825875+0000","last_undegraded":"2023-11-24T06:10:26.825875+0000","last_fullsized":"2023-11-24T06:10:26.825875+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_clean_scrub_stamp":"2023-11-24T06:10:23.635255+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:20:39.581356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.886486+0000","last_change":"2023-11-24T06:10:25.726307+0000","last_active":"2023-11-24T06:10:26.886486+0000","last_peered":"2023-11-24T06:10:26.886486+0000","last_clean":"2023-11-24T06:10:26.886486+0000","last_became_active":"2023-11-24T06:10:25.726038+0000","last_became_peered":"2023-11-24T06:10:25.726038+0000","last_unstale":"2023-11-24T06:10:26.886486+0000","last_undegraded":"2023-11-24T06:10:26.886486+0000","last_fullsized":"2023-11-24T06:10:26.886486+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:00:56.115984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.717432+0000","last_change":"2023-11-24T06:10:24.692338+0000","last_active":"2023-11-24T06:10:26.717432+0000","last_peered":"2023-11-24T06:10:26.717432+0000","last_clean":"2023-11-24T06:10:26.717432+0000","last_became_active":"2023-11-24T06:10:24.692135+0000","last_became_peered":"2023-11-24T06:10:24.692135+0000","last_unstale":"2023-11-24T06:10:26.717432+0000","last_undegraded":"2023-11-24T06:10:26.717432+0000","last_fullsized":"2023-11-24T06:10:26.717432+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_clean_scrub_stamp":"2023-11-24T06:10:23.635255+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:57:48.799543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.887337+0000","last_change":"2023-11-24T06:10:25.729383+0000","last_active":"2023-11-24T06:10:26.887337+0000","last_peered":"2023-11-24T06:10:26.887337+0000","last_clean":"2023-11-24T06:10:26.887337+0000","last_became_active":"2023-11-24T06:10:25.729090+0000","last_became_peered":"2023-11-24T06:10:25.729090+0000","last_unstale":"2023-11-24T06:10:26.887337+0000","last_undegraded":"2023-11-24T06:10:26.887337+0000","last_fullsized":"2023-11-24T06:10:26.887337+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:40:47.125372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.826232+0000","last_change":"2023-11-24T06:10:24.689238+0000","last_active":"2023-11-24T06:10:26.826232+0000","last_peered":"2023-11-24T06:10:26.826232+0000","last_clean":"2023-11-24T06:10:26.826232+0000","last_became_active":"2023-11-24T06:10:24.686650+0000","last_became_peered":"2023-11-24T06:10:24.686650+0000","last_unstale":"2023-11-24T06:10:26.826232+0000","last_undegraded":"2023-11-24T06:10:26.826232+0000","last_fullsized":"2023-11-24T06:10:26.826232+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_clean_scrub_stamp":"2023-11-24T06:10:23.635255+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:06:50.936889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.887656+0000","last_change":"2023-11-24T06:10:25.726737+0000","last_active":"2023-11-24T06:10:26.887656+0000","last_peered":"2023-11-24T06:10:26.887656+0000","last_clean":"2023-11-24T06:10:26.887656+0000","last_became_active":"2023-11-24T06:10:25.725070+0000","last_became_peered":"2023-11-24T06:10:25.725070+0000","last_unstale":"2023-11-24T06:10:26.887656+0000","last_undegraded":"2023-11-24T06:10:26.887656+0000","last_fullsized":"2023-11-24T06:10:26.887656+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:22:18.598271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.885715+0000","last_change":"2023-11-24T06:10:24.681713+0000","last_active":"2023-11-24T06:10:26.885715+0000","last_peered":"2023-11-24T06:10:26.885715+0000","last_clean":"2023-11-24T06:10:26.885715+0000","last_became_active":"2023-11-24T06:10:24.681336+0000","last_became_peered":"2023-11-24T06:10:24.681336+0000","last_unstale":"2023-11-24T06:10:26.885715+0000","last_undegraded":"2023-11-24T06:10:26.885715+0000","last_fullsized":"2023-11-24T06:10:26.885715+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_clean_scrub_stamp":"2023-11-24T06:10:23.635255+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:10:14.268052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.888736+0000","last_change":"2023-11-24T06:10:25.733821+0000","last_active":"2023-11-24T06:10:26.888736+0000","last_peered":"2023-11-24T06:10:26.888736+0000","last_clean":"2023-11-24T06:10:26.888736+0000","last_became_active":"2023-11-24T06:10:25.733510+0000","last_became_peered":"2023-11-24T06:10:25.733510+0000","last_unstale":"2023-11-24T06:10:26.888736+0000","last_undegraded":"2023-11-24T06:10:26.888736+0000","last_fullsized":"2023-11-24T06:10:26.888736+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:42:25.852028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.826174+0000","last_change":"2023-11-24T06:10:24.690450+0000","last_active":"2023-11-24T06:10:26.826174+0000","last_peered":"2023-11-24T06:10:26.826174+0000","last_clean":"2023-11-24T06:10:26.826174+0000","last_became_active":"2023-11-24T06:10:24.690024+0000","last_became_peered":"2023-11-24T06:10:24.690024+0000","last_unstale":"2023-11-24T06:10:26.826174+0000","last_undegraded":"2023-11-24T06:10:26.826174+0000","last_fullsized":"2023-11-24T06:10:26.826174+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_clean_scrub_stamp":"2023-11-24T06:10:23.635255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:51:55.108153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.887639+0000","last_change":"2023-11-24T06:10:25.734784+0000","last_active":"2023-11-24T06:10:26.887639+0000","last_peered":"2023-11-24T06:10:26.887639+0000","last_clean":"2023-11-24T06:10:26.887639+0000","last_became_active":"2023-11-24T06:10:25.734478+0000","last_became_peered":"2023-11-24T06:10:25.734478+0000","last_unstale":"2023-11-24T06:10:26.887639+0000","last_undegraded":"2023-11-24T06:10:26.887639+0000","last_fullsized":"2023-11-24T06:10:26.887639+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:05:14.621899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":165,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:10:25.700259+0000","last_change":"2023-11-24T06:10:20.082677+0000","last_active":"2023-11-24T06:10:25.700259+0000","last_peered":"2023-11-24T06:10:25.700259+0000","last_clean":"2023-11-24T06:10:25.700259+0000","last_became_active":"2023-11-24T06:10:20.080551+0000","last_became_peered":"2023-11-24T06:10:20.080551+0000","last_unstale":"2023-11-24T06:10:25.700259+0000","last_undegraded":"2023-11-24T06:10:25.700259+0000","last_fullsized":"2023-11-24T06:10:25.700259+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:18.602209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:18.602209+0000","last_clean_scrub_stamp":"2023-11-24T06:10:18.602209+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:47:24.216235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.887865+0000","last_change":"2023-11-24T06:10:24.684957+0000","last_active":"2023-11-24T06:10:26.887865+0000","last_peered":"2023-11-24T06:10:26.887865+0000","last_clean":"2023-11-24T06:10:26.887865+0000","last_became_active":"2023-11-24T06:10:24.684507+0000","last_became_peered":"2023-11-24T06:10:24.684507+0000","last_unstale":"2023-11-24T06:10:26.887865+0000","last_undegraded":"2023-11-24T06:10:26.887865+0000","last_fullsized":"2023-11-24T06:10:26.887865+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_clean_scrub_stamp":"2023-11-24T06:10:23.635255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:52:31.270989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.715709+0000","last_change":"2023-11-24T06:10:25.735264+0000","last_active":"2023-11-24T06:10:26.715709+0000","last_peered":"2023-11-24T06:10:26.715709+0000","last_clean":"2023-11-24T06:10:26.715709+0000","last_became_active":"2023-11-24T06:10:25.734163+0000","last_became_peered":"2023-11-24T06:10:25.734163+0000","last_unstale":"2023-11-24T06:10:26.715709+0000","last_undegraded":"2023-11-24T06:10:26.715709+0000","last_fullsized":"2023-11-24T06:10:26.715709+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:45:21.592844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.885683+0000","last_change":"2023-11-24T06:10:24.699469+0000","last_active":"2023-11-24T06:10:26.885683+0000","last_peered":"2023-11-24T06:10:26.885683+0000","last_clean":"2023-11-24T06:10:26.885683+0000","last_became_active":"2023-11-24T06:10:24.699153+0000","last_became_peered":"2023-11-24T06:10:24.699153+0000","last_unstale":"2023-11-24T06:10:26.885683+0000","last_undegraded":"2023-11-24T06:10:26.885683+0000","last_fullsized":"2023-11-24T06:10:26.885683+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_clean_scrub_stamp":"2023-11-24T06:10:23.635255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:09:05.283059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.884898+0000","last_change":"2023-11-24T06:10:25.716501+0000","last_active":"2023-11-24T06:10:26.884898+0000","last_peered":"2023-11-24T06:10:26.884898+0000","last_clean":"2023-11-24T06:10:26.884898+0000","last_became_active":"2023-11-24T06:10:25.712127+0000","last_became_peered":"2023-11-24T06:10:25.712127+0000","last_unstale":"2023-11-24T06:10:26.884898+0000","last_undegraded":"2023-11-24T06:10:26.884898+0000","last_fullsized":"2023-11-24T06:10:26.884898+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:10:15.285136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.826108+0000","last_change":"2023-11-24T06:10:24.686855+0000","last_active":"2023-11-24T06:10:26.826108+0000","last_peered":"2023-11-24T06:10:26.826108+0000","last_clean":"2023-11-24T06:10:26.826108+0000","last_became_active":"2023-11-24T06:10:24.685894+0000","last_became_peered":"2023-11-24T06:10:24.685894+0000","last_unstale":"2023-11-24T06:10:26.826108+0000","last_undegraded":"2023-11-24T06:10:26.826108+0000","last_fullsized":"2023-11-24T06:10:26.826108+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_clean_scrub_stamp":"2023-11-24T06:10:23.635255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:38:10.950543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.885731+0000","last_change":"2023-11-24T06:10:25.715833+0000","last_active":"2023-11-24T06:10:26.885731+0000","last_peered":"2023-11-24T06:10:26.885731+0000","last_clean":"2023-11-24T06:10:26.885731+0000","last_became_active":"2023-11-24T06:10:25.715500+0000","last_became_peered":"2023-11-24T06:10:25.715500+0000","last_unstale":"2023-11-24T06:10:26.885731+0000","last_undegraded":"2023-11-24T06:10:26.885731+0000","last_fullsized":"2023-11-24T06:10:26.885731+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:43:50.924468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.887610+0000","last_change":"2023-11-24T06:10:24.690139+0000","last_active":"2023-11-24T06:10:26.887610+0000","last_peered":"2023-11-24T06:10:26.887610+0000","last_clean":"2023-11-24T06:10:26.887610+0000","last_became_active":"2023-11-24T06:10:24.689734+0000","last_became_peered":"2023-11-24T06:10:24.689734+0000","last_unstale":"2023-11-24T06:10:26.887610+0000","last_undegraded":"2023-11-24T06:10:26.887610+0000","last_fullsized":"2023-11-24T06:10:26.887610+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_clean_scrub_stamp":"2023-11-24T06:10:23.635255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:45:20.575759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.717184+0000","last_change":"2023-11-24T06:10:25.710787+0000","last_active":"2023-11-24T06:10:26.717184+0000","last_peered":"2023-11-24T06:10:26.717184+0000","last_clean":"2023-11-24T06:10:26.717184+0000","last_became_active":"2023-11-24T06:10:25.710522+0000","last_became_peered":"2023-11-24T06:10:25.710522+0000","last_unstale":"2023-11-24T06:10:26.717184+0000","last_undegraded":"2023-11-24T06:10:26.717184+0000","last_fullsized":"2023-11-24T06:10:26.717184+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:17:03.098818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.885579+0000","last_change":"2023-11-24T06:10:24.683405+0000","last_active":"2023-11-24T06:10:26.885579+0000","last_peered":"2023-11-24T06:10:26.885579+0000","last_clean":"2023-11-24T06:10:26.885579+0000","last_became_active":"2023-11-24T06:10:24.683144+0000","last_became_peered":"2023-11-24T06:10:24.683144+0000","last_unstale":"2023-11-24T06:10:26.885579+0000","last_undegraded":"2023-11-24T06:10:26.885579+0000","last_fullsized":"2023-11-24T06:10:26.885579+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_clean_scrub_stamp":"2023-11-24T06:10:23.635255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:18:53.146509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.888979+0000","last_change":"2023-11-24T06:10:25.734738+0000","last_active":"2023-11-24T06:10:26.888979+0000","last_peered":"2023-11-24T06:10:26.888979+0000","last_clean":"2023-11-24T06:10:26.888979+0000","last_became_active":"2023-11-24T06:10:25.733409+0000","last_became_peered":"2023-11-24T06:10:25.733409+0000","last_unstale":"2023-11-24T06:10:26.888979+0000","last_undegraded":"2023-11-24T06:10:26.888979+0000","last_fullsized":"2023-11-24T06:10:26.888979+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:59:37.010936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.888082+0000","last_change":"2023-11-24T06:10:24.701942+0000","last_active":"2023-11-24T06:10:26.888082+0000","last_peered":"2023-11-24T06:10:26.888082+0000","last_clean":"2023-11-24T06:10:26.888082+0000","last_became_active":"2023-11-24T06:10:24.701385+0000","last_became_peered":"2023-11-24T06:10:24.701385+0000","last_unstale":"2023-11-24T06:10:26.888082+0000","last_undegraded":"2023-11-24T06:10:26.888082+0000","last_fullsized":"2023-11-24T06:10:26.888082+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_clean_scrub_stamp":"2023-11-24T06:10:23.635255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:26:39.190781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.885358+0000","last_change":"2023-11-24T06:10:25.711825+0000","last_active":"2023-11-24T06:10:26.885358+0000","last_peered":"2023-11-24T06:10:26.885358+0000","last_clean":"2023-11-24T06:10:26.885358+0000","last_became_active":"2023-11-24T06:10:25.711161+0000","last_became_peered":"2023-11-24T06:10:25.711161+0000","last_unstale":"2023-11-24T06:10:26.885358+0000","last_undegraded":"2023-11-24T06:10:26.885358+0000","last_fullsized":"2023-11-24T06:10:26.885358+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:45:21.592844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:27.899957+0000","last_change":"2023-11-24T06:10:24.695283+0000","last_active":"2023-11-24T06:10:27.899957+0000","last_peered":"2023-11-24T06:10:27.899957+0000","last_clean":"2023-11-24T06:10:27.899957+0000","last_became_active":"2023-11-24T06:10:24.694723+0000","last_became_peered":"2023-11-24T06:10:24.694723+0000","last_unstale":"2023-11-24T06:10:27.899957+0000","last_undegraded":"2023-11-24T06:10:27.899957+0000","last_fullsized":"2023-11-24T06:10:27.899957+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_clean_scrub_stamp":"2023-11-24T06:10:23.635255+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:52:31.270989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.716963+0000","last_change":"2023-11-24T06:10:25.730440+0000","last_active":"2023-11-24T06:10:26.716963+0000","last_peered":"2023-11-24T06:10:26.716963+0000","last_clean":"2023-11-24T06:10:26.716963+0000","last_became_active":"2023-11-24T06:10:25.730097+0000","last_became_peered":"2023-11-24T06:10:25.730097+0000","last_unstale":"2023-11-24T06:10:26.716963+0000","last_undegraded":"2023-11-24T06:10:26.716963+0000","last_fullsized":"2023-11-24T06:10:26.716963+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:44:12.726705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.716893+0000","last_change":"2023-11-24T06:10:24.702923+0000","last_active":"2023-11-24T06:10:26.716893+0000","last_peered":"2023-11-24T06:10:26.716893+0000","last_clean":"2023-11-24T06:10:26.716893+0000","last_became_active":"2023-11-24T06:10:24.702428+0000","last_became_peered":"2023-11-24T06:10:24.702428+0000","last_unstale":"2023-11-24T06:10:26.716893+0000","last_undegraded":"2023-11-24T06:10:26.716893+0000","last_fullsized":"2023-11-24T06:10:26.716893+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_clean_scrub_stamp":"2023-11-24T06:10:23.635255+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:52:31.270989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.886761+0000","last_change":"2023-11-24T06:10:25.709739+0000","last_active":"2023-11-24T06:10:26.886761+0000","last_peered":"2023-11-24T06:10:26.886761+0000","last_clean":"2023-11-24T06:10:26.886761+0000","last_became_active":"2023-11-24T06:10:25.709392+0000","last_became_peered":"2023-11-24T06:10:25.709392+0000","last_unstale":"2023-11-24T06:10:26.886761+0000","last_undegraded":"2023-11-24T06:10:26.886761+0000","last_fullsized":"2023-11-24T06:10:26.886761+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:45:21.592844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.886005+0000","last_change":"2023-11-24T06:10:24.698896+0000","last_active":"2023-11-24T06:10:26.886005+0000","last_peered":"2023-11-24T06:10:26.886005+0000","last_clean":"2023-11-24T06:10:26.886005+0000","last_became_active":"2023-11-24T06:10:24.698632+0000","last_became_peered":"2023-11-24T06:10:24.698632+0000","last_unstale":"2023-11-24T06:10:26.886005+0000","last_undegraded":"2023-11-24T06:10:26.886005+0000","last_fullsized":"2023-11-24T06:10:26.886005+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_clean_scrub_stamp":"2023-11-24T06:10:23.635255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:22:19.918119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.715380+0000","last_change":"2023-11-24T06:10:25.716284+0000","last_active":"2023-11-24T06:10:26.715380+0000","last_peered":"2023-11-24T06:10:26.715380+0000","last_clean":"2023-11-24T06:10:26.715380+0000","last_became_active":"2023-11-24T06:10:25.715948+0000","last_became_peered":"2023-11-24T06:10:25.715948+0000","last_unstale":"2023-11-24T06:10:26.715380+0000","last_undegraded":"2023-11-24T06:10:26.715380+0000","last_fullsized":"2023-11-24T06:10:26.715380+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:20:40.598440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.885366+0000","last_change":"2023-11-24T06:10:24.692784+0000","last_active":"2023-11-24T06:10:26.885366+0000","last_peered":"2023-11-24T06:10:26.885366+0000","last_clean":"2023-11-24T06:10:26.885366+0000","last_became_active":"2023-11-24T06:10:24.692514+0000","last_became_peered":"2023-11-24T06:10:24.692514+0000","last_unstale":"2023-11-24T06:10:26.885366+0000","last_undegraded":"2023-11-24T06:10:26.885366+0000","last_fullsized":"2023-11-24T06:10:26.885366+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_clean_scrub_stamp":"2023-11-24T06:10:23.635255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:38:10.950543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.715214+0000","last_change":"2023-11-24T06:10:25.712156+0000","last_active":"2023-11-24T06:10:26.715214+0000","last_peered":"2023-11-24T06:10:26.715214+0000","last_clean":"2023-11-24T06:10:26.715214+0000","last_became_active":"2023-11-24T06:10:25.711924+0000","last_became_peered":"2023-11-24T06:10:25.711924+0000","last_unstale":"2023-11-24T06:10:26.715214+0000","last_undegraded":"2023-11-24T06:10:26.715214+0000","last_fullsized":"2023-11-24T06:10:26.715214+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:46:00.341282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.885937+0000","last_change":"2023-11-24T06:10:24.695919+0000","last_active":"2023-11-24T06:10:26.885937+0000","last_peered":"2023-11-24T06:10:26.885937+0000","last_clean":"2023-11-24T06:10:26.885937+0000","last_became_active":"2023-11-24T06:10:24.695650+0000","last_became_peered":"2023-11-24T06:10:24.695650+0000","last_unstale":"2023-11-24T06:10:26.885937+0000","last_undegraded":"2023-11-24T06:10:26.885937+0000","last_fullsized":"2023-11-24T06:10:26.885937+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_clean_scrub_stamp":"2023-11-24T06:10:23.635255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:09:05.283059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.716052+0000","last_change":"2023-11-24T06:10:25.719797+0000","last_active":"2023-11-24T06:10:26.716052+0000","last_peered":"2023-11-24T06:10:26.716052+0000","last_clean":"2023-11-24T06:10:26.716052+0000","last_became_active":"2023-11-24T06:10:25.719483+0000","last_became_peered":"2023-11-24T06:10:25.719483+0000","last_unstale":"2023-11-24T06:10:26.716052+0000","last_undegraded":"2023-11-24T06:10:26.716052+0000","last_fullsized":"2023-11-24T06:10:26.716052+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:43:50.924468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.887695+0000","last_change":"2023-11-24T06:10:24.699236+0000","last_active":"2023-11-24T06:10:26.887695+0000","last_peered":"2023-11-24T06:10:26.887695+0000","last_clean":"2023-11-24T06:10:26.887695+0000","last_became_active":"2023-11-24T06:10:24.698709+0000","last_became_peered":"2023-11-24T06:10:24.698709+0000","last_unstale":"2023-11-24T06:10:26.887695+0000","last_undegraded":"2023-11-24T06:10:26.887695+0000","last_fullsized":"2023-11-24T06:10:26.887695+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_clean_scrub_stamp":"2023-11-24T06:10:23.635255+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:37:00.276208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.716147+0000","last_change":"2023-11-24T06:10:25.714752+0000","last_active":"2023-11-24T06:10:26.716147+0000","last_peered":"2023-11-24T06:10:26.716147+0000","last_clean":"2023-11-24T06:10:26.716147+0000","last_became_active":"2023-11-24T06:10:25.714468+0000","last_became_peered":"2023-11-24T06:10:25.714468+0000","last_unstale":"2023-11-24T06:10:26.716147+0000","last_undegraded":"2023-11-24T06:10:26.716147+0000","last_fullsized":"2023-11-24T06:10:26.716147+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:45:21.592844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.888957+0000","last_change":"2023-11-24T06:10:25.719973+0000","last_active":"2023-11-24T06:10:26.888957+0000","last_peered":"2023-11-24T06:10:26.888957+0000","last_clean":"2023-11-24T06:10:26.888957+0000","last_became_active":"2023-11-24T06:10:25.719716+0000","last_became_peered":"2023-11-24T06:10:25.719716+0000","last_unstale":"2023-11-24T06:10:26.888957+0000","last_undegraded":"2023-11-24T06:10:26.888957+0000","last_fullsized":"2023-11-24T06:10:26.888957+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:40:47.125372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.886889+0000","last_change":"2023-11-24T06:10:24.701875+0000","last_active":"2023-11-24T06:10:26.886889+0000","last_peered":"2023-11-24T06:10:26.886889+0000","last_clean":"2023-11-24T06:10:26.886889+0000","last_became_active":"2023-11-24T06:10:24.701520+0000","last_became_peered":"2023-11-24T06:10:24.701520+0000","last_unstale":"2023-11-24T06:10:26.886889+0000","last_undegraded":"2023-11-24T06:10:26.886889+0000","last_fullsized":"2023-11-24T06:10:26.886889+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_clean_scrub_stamp":"2023-11-24T06:10:23.635255+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:18:53.146509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.888401+0000","last_change":"2023-11-24T06:10:25.729759+0000","last_active":"2023-11-24T06:10:26.888401+0000","last_peered":"2023-11-24T06:10:26.888401+0000","last_clean":"2023-11-24T06:10:26.888401+0000","last_became_active":"2023-11-24T06:10:25.721916+0000","last_became_peered":"2023-11-24T06:10:25.721916+0000","last_unstale":"2023-11-24T06:10:26.888401+0000","last_undegraded":"2023-11-24T06:10:26.888401+0000","last_fullsized":"2023-11-24T06:10:26.888401+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:03:41.511367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.887901+0000","last_change":"2023-11-24T06:10:24.699400+0000","last_active":"2023-11-24T06:10:26.887901+0000","last_peered":"2023-11-24T06:10:26.887901+0000","last_clean":"2023-11-24T06:10:26.887901+0000","last_became_active":"2023-11-24T06:10:24.699090+0000","last_became_peered":"2023-11-24T06:10:24.699090+0000","last_unstale":"2023-11-24T06:10:26.887901+0000","last_undegraded":"2023-11-24T06:10:26.887901+0000","last_fullsized":"2023-11-24T06:10:26.887901+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_clean_scrub_stamp":"2023-11-24T06:10:23.635255+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:14:19.359955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.887209+0000","last_change":"2023-11-24T06:10:25.732187+0000","last_active":"2023-11-24T06:10:26.887209+0000","last_peered":"2023-11-24T06:10:26.887209+0000","last_clean":"2023-11-24T06:10:26.887209+0000","last_became_active":"2023-11-24T06:10:25.731130+0000","last_became_peered":"2023-11-24T06:10:25.731130+0000","last_unstale":"2023-11-24T06:10:26.887209+0000","last_undegraded":"2023-11-24T06:10:26.887209+0000","last_fullsized":"2023-11-24T06:10:26.887209+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:37:01.293292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.716804+0000","last_change":"2023-11-24T06:10:24.694116+0000","last_active":"2023-11-24T06:10:26.716804+0000","last_peered":"2023-11-24T06:10:26.716804+0000","last_clean":"2023-11-24T06:10:26.716804+0000","last_became_active":"2023-11-24T06:10:24.693832+0000","last_became_peered":"2023-11-24T06:10:24.693832+0000","last_unstale":"2023-11-24T06:10:26.716804+0000","last_undegraded":"2023-11-24T06:10:26.716804+0000","last_fullsized":"2023-11-24T06:10:26.716804+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_clean_scrub_stamp":"2023-11-24T06:10:23.635255+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:38:10.950543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.716865+0000","last_change":"2023-11-24T06:10:25.707741+0000","last_active":"2023-11-24T06:10:26.716865+0000","last_peered":"2023-11-24T06:10:26.716865+0000","last_clean":"2023-11-24T06:10:26.716865+0000","last_became_active":"2023-11-24T06:10:25.707453+0000","last_became_peered":"2023-11-24T06:10:25.707453+0000","last_unstale":"2023-11-24T06:10:26.716865+0000","last_undegraded":"2023-11-24T06:10:26.716865+0000","last_fullsized":"2023-11-24T06:10:26.716865+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:24.652339+0000","last_clean_scrub_stamp":"2023-11-24T06:10:24.652339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:26:40.207866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:10:26.887375+0000","last_change":"2023-11-24T06:10:24.699025+0000","last_active":"2023-11-24T06:10:26.887375+0000","last_peered":"2023-11-24T06:10:26.887375+0000","last_clean":"2023-11-24T06:10:26.887375+0000","last_became_active":"2023-11-24T06:10:24.698788+0000","last_became_peered":"2023-11-24T06:10:24.698788+0000","last_unstale":"2023-11-24T06:10:26.887375+0000","last_undegraded":"2023-11-24T06:10:26.887375+0000","last_fullsized":"2023-11-24T06:10:26.887375+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:23.635255+0000","last_clean_scrub_stamp":"2023-11-24T06:10:23.635255+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:18:53.146509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574853,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31964,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339876,"statfs":{"total":96636764160,"available":96604033024,"internally_reserved":0,"allocated":802816,"data_stored":661860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9630000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0259999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0720000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.593}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.232}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.413}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4790000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31400,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340440,"statfs":{"total":96636764160,"available":96604610560,"internally_reserved":0,"allocated":217088,"data_stored":72522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.579}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1719999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.294}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.784}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1440000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0659999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1099999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31388,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340452,"statfs":{"total":96636764160,"available":96604622848,"internally_reserved":0,"allocated":204800,"data_stored":70386,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6890000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.528}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4470000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7869999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.885}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.4300000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.2650000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574854,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31396,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340444,"statfs":{"total":96636764160,"available":96604614656,"internally_reserved":0,"allocated":217088,"data_stored":71036,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0660000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5580000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83399999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0309999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1819999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2350000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5}]}]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":217088,"data_stored":72522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.292}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.5779999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3879999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.3980000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0099999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.169}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.3090000000000002}]}]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":196,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":200704,"data_stored":70296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56200000000000006}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0489999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.27}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68300000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.306}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6180000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5820000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":212992,"data_stored":71070,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77400000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.462}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84299999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.429}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5529999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.333}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.52}]}]},{"osd":0,"up_from":13,"seq":55834574853,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27812,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344028,"statfs":{"total":96636764160,"available":96608284672,"internally_reserved":0,"allocated":811008,"data_stored":661916,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9259999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.5139999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.5840000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.036}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.2290000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.8860000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.9369999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-24T06:10:36.087 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-11-24T06:10:36.087 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-11-24T06:10:36.087 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2023-11-24T06:10:36.087 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-24T06:10:36.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:36.328+0000 7f14d4ba5640 1 Processor -- start 2023-11-24T06:10:36.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:36.329+0000 7f14d4ba5640 1 -- start start 2023-11-24T06:10:36.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:36.330+0000 7f14d4ba5640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f14d00cf9a0 0x7f14d015e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:36.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:36.330+0000 7f14d4ba5640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14d0070c40 con 0x7f14d00ae5a0 2023-11-24T06:10:36.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:36.330+0000 7f14d4ba5640 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14d006c470 con 0x7f14d00b3ae0 2023-11-24T06:10:36.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:36.330+0000 7f14d4ba5640 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14d0071450 con 0x7f14d00cf9a0 2023-11-24T06:10:36.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:36.330+0000 7f14ce59c640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f14d00ae5a0 0x7f14d006cbe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:55902/0 (socket says 172.21.15.92:55902) 2023-11-24T06:10:36.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:36.330+0000 7f14ce59c640 1 -- 172.21.15.92:0/1631886427 learned_addr learned my addr 172.21.15.92:0/1631886427 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-24T06:10:36.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:36.331+0000 7f14cd59a640 1 -- 172.21.15.92:0/1631886427 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 281136488 0 0) 0x7f14d0070c40 con 0x7f14d00ae5a0 2023-11-24T06:10:36.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:36.331+0000 7f14cd59a640 1 -- 172.21.15.92:0/1631886427 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f14b8003520 con 0x7f14d00ae5a0 2023-11-24T06:10:36.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:36.331+0000 7f14ced9d640 1 --2- 172.21.15.92:0/1631886427 >> v2:172.21.15.118:3300/0 conn(0x7f14d00cf9a0 0x7f14d015e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:36.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:36.331+0000 7f14cd59a640 1 -- 172.21.15.92:0/1631886427 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1209054782 0 0) 0x7f14b8003520 con 0x7f14d00ae5a0 2023-11-24T06:10:36.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:36.331+0000 7f14cd59a640 1 -- 172.21.15.92:0/1631886427 >> v2:172.21.15.118:3300/0 conn(0x7f14d00cf9a0 msgr2=0x7f14d015e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:36.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:36.331+0000 7f14cd59a640 1 --2- 172.21.15.92:0/1631886427 >> v2:172.21.15.118:3300/0 conn(0x7f14d00cf9a0 0x7f14d015e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:36.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:36.331+0000 7f14cd59a640 1 -- 172.21.15.92:0/1631886427 >> v1:172.21.15.118:6789/0 conn(0x7f14d00b3ae0 legacy=0x7f14d00cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:36.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:36.331+0000 7f14cd59a640 1 -- 172.21.15.92:0/1631886427 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14d006c750 con 0x7f14d00ae5a0 2023-11-24T06:10:36.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:36.331+0000 7f14cd59a640 1 -- 172.21.15.92:0/1631886427 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2607661350 0 0) 0x7f14c00019f0 con 0x7f14d00ae5a0 2023-11-24T06:10:36.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:36.331+0000 7f14ced9d640 1 --2- 172.21.15.92:0/1631886427 >> v2:172.21.15.118:3300/0 conn(0x7f14d00cf9a0 0x7f14d015e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:10:36.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:36.331+0000 7f14cd59a640 1 -- 172.21.15.92:0/1631886427 >> v1:172.21.15.92:6789/0 conn(0x7f14d00ae5a0 legacy=0x7f14d006cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:36.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:36.331+0000 7f14cd59a640 1 --2- 172.21.15.92:0/1631886427 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f14b8003970 0x7f14b8003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:36.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:36.331+0000 7f14cd59a640 1 --2- 172.21.15.92:0/1631886427 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f14b80047c0 0x7f14b8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:36.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:36.331+0000 7f14cd59a640 1 --2- 172.21.15.92:0/1631886427 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f14b8005180 0x7f14b800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:36.338 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:36.331+0000 7f14cd59a640 1 -- 172.21.15.92:0/1631886427 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14d0071450 con 0x7f14b80047c0 2023-11-24T06:10:36.338 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:36.331+0000 7f14cd59a640 1 -- 172.21.15.92:0/1631886427 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14b800c0b0 con 0x7f14b8005180 2023-11-24T06:10:36.338 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:36.331+0000 7f14cd59a640 1 -- 172.21.15.92:0/1631886427 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14b800c270 con 0x7f14b8003970 2023-11-24T06:10:36.338 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:36.332+0000 7f14cdd9b640 1 --2- 172.21.15.92:0/1631886427 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f14b80047c0 0x7f14b8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:36.338 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:36.332+0000 7f14ce59c640 1 --2- 172.21.15.92:0/1631886427 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f14b8003970 0x7f14b8003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:36.338 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:36.332+0000 7f14ced9d640 1 --2- 172.21.15.92:0/1631886427 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f14b8005180 0x7f14b800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:36.338 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:36.332+0000 7f14cdd9b640 1 -- 172.21.15.92:0/1631886427 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f14b8003970 msgr2=0x7f14b8003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:36.338 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:36.332+0000 7f14cdd9b640 1 --2- 172.21.15.92:0/1631886427 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f14b8003970 0x7f14b8003d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:36.338 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:36.332+0000 7f14cdd9b640 1 -- 172.21.15.92:0/1631886427 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f14b8005180 msgr2=0x7f14b800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:36.338 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:36.332+0000 7f14cdd9b640 1 --2- 172.21.15.92:0/1631886427 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f14b8005180 0x7f14b800b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:36.338 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:36.332+0000 7f14cdd9b640 1 -- 172.21.15.92:0/1631886427 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14b8003520 con 0x7f14b80047c0 2023-11-24T06:10:36.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:36.332+0000 7f14ced9d640 1 --2- 172.21.15.92:0/1631886427 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f14b8005180 0x7f14b800b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:10:36.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:36.332+0000 7f14ced9d640 1 -- 172.21.15.92:0/1631886427 reap_dead start 2023-11-24T06:10:36.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:36.333+0000 7f14cdd9b640 1 --2- 172.21.15.92:0/1631886427 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f14b80047c0 0x7f14b8004c40 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f14c4001750 tx=0x7f14b80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c804848aa1248d31 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:36.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:36.333+0000 7f14cd59a640 1 -- 172.21.15.92:0/1631886427 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14c40052d0 con 0x7f14b80047c0 2023-11-24T06:10:36.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:36.333+0000 7f14cd59a640 1 -- 172.21.15.92:0/1631886427 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14c402e050 con 0x7f14b80047c0 2023-11-24T06:10:36.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:36.333+0000 7f14cd59a640 1 -- 172.21.15.92:0/1631886427 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14c4003830 con 0x7f14b80047c0 2023-11-24T06:10:36.341 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:36.334+0000 7f14d4ba5640 1 -- 172.21.15.92:0/1631886427 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f14b80047c0 msgr2=0x7f14b8004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:36.342 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:36.334+0000 7f14d4ba5640 1 --2- 172.21.15.92:0/1631886427 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f14b80047c0 0x7f14b8004c40 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f14c4001750 tx=0x7f14b80057f0 comp rx=0 tx=0).stop 2023-11-24T06:10:36.342 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:36.334+0000 7f14d4ba5640 1 -- 172.21.15.92:0/1631886427 shutdown_connections 2023-11-24T06:10:36.342 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:36.334+0000 7f14d4ba5640 1 --2- 172.21.15.92:0/1631886427 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f14b80047c0 0x7f14b8004c40 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:36.342 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:36.334+0000 7f14d4ba5640 1 -- 172.21.15.92:0/1631886427 >> 172.21.15.92:0/1631886427 conn(0x7f14d006a800 msgr2=0x7f14d006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:36.342 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:36.334+0000 7f14d4ba5640 1 -- 172.21.15.92:0/1631886427 shutdown_connections 2023-11-24T06:10:36.342 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:36.334+0000 7f14d4ba5640 1 -- 172.21.15.92:0/1631886427 wait complete. 2023-11-24T06:10:36.342 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:36.336+0000 7f14d4ba5640 1 Processor -- start 2023-11-24T06:10:36.342 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:36.336+0000 7f14d4ba5640 1 -- start start 2023-11-24T06:10:36.342 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:36.336+0000 7f14d4ba5640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f14b80047c0 0x7f14d01f2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:36.342 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:36.336+0000 7f14d4ba5640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f14d0148910 0x7f14d00cc5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:36.342 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:36.337+0000 7f14d4ba5640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f14d00ccb20 0x7f14d00b2560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:36.343 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:36.337+0000 7f14d4ba5640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14d01f3430 con 0x7f14d00ccb20 2023-11-24T06:10:36.343 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:36.337+0000 7f14d4ba5640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14d00cd010 con 0x7f14b80047c0 2023-11-24T06:10:36.343 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:36.337+0000 7f14d4ba5640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14d01570d0 con 0x7f14d0148910 2023-11-24T06:10:36.343 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:36.337+0000 7f14ced9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f14d00ccb20 0x7f14d00b2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:36.343 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:36.337+0000 7f14ced9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f14d00ccb20 0x7f14d00b2560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47588/0 (socket says 172.21.15.92:47588) 2023-11-24T06:10:36.343 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:36.337+0000 7f14ced9d640 1 -- 172.21.15.92:0/1987291335 learned_addr learned my addr 172.21.15.92:0/1987291335 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:10:36.343 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:36.337+0000 7f14ced9d640 1 -- 172.21.15.92:0/1987291335 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f14d0148910 msgr2=0x7f14d00cc5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:36.343 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:36.337+0000 7f14cdd9b640 1 --2- 172.21.15.92:0/1987291335 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f14d0148910 0x7f14d00cc5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:36.343 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:36.337+0000 7f14ce59c640 1 --2- 172.21.15.92:0/1987291335 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f14b80047c0 0x7f14d01f2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:36.343 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:36.337+0000 7f14ced9d640 1 --2- 172.21.15.92:0/1987291335 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f14d0148910 0x7f14d00cc5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:36.343 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:36.337+0000 7f14ced9d640 1 -- 172.21.15.92:0/1987291335 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f14b80047c0 msgr2=0x7f14d01f2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:36.344 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:36.337+0000 7f14ced9d640 1 --2- 172.21.15.92:0/1987291335 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f14b80047c0 0x7f14d01f2ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:36.344 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:36.337+0000 7f14ced9d640 1 -- 172.21.15.92:0/1987291335 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14d01573b0 con 0x7f14d00ccb20 2023-11-24T06:10:36.344 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:36.337+0000 7f14ced9d640 1 --2- 172.21.15.92:0/1987291335 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f14d00ccb20 0x7f14d00b2560 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f14c8006fd0 tx=0x7f14c800f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:36.344 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:36.338+0000 7f14beffd640 1 -- 172.21.15.92:0/1987291335 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14c8014680 con 0x7f14d00ccb20 2023-11-24T06:10:36.344 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:36.338+0000 7f14beffd640 1 -- 172.21.15.92:0/1987291335 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14c800c030 con 0x7f14d00ccb20 2023-11-24T06:10:36.344 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:36.338+0000 7f14beffd640 1 -- 172.21.15.92:0/1987291335 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14c8014a40 con 0x7f14d00ccb20 2023-11-24T06:10:36.344 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:36.338+0000 7f14d4ba5640 1 -- 172.21.15.92:0/1987291335 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14d0157690 con 0x7f14d00ccb20 2023-11-24T06:10:36.344 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:36.338+0000 7f14d4ba5640 1 -- 172.21.15.92:0/1987291335 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14d0148d90 con 0x7f14d00ccb20 2023-11-24T06:10:36.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:36.339+0000 7f14d4ba5640 1 -- 172.21.15.92:0/1987291335 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14d0157bb0 con 0x7f14d00ccb20 2023-11-24T06:10:36.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:36.340+0000 7f14beffd640 1 -- 172.21.15.92:0/1987291335 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f14c8014ba0 con 0x7f14d00ccb20 2023-11-24T06:10:36.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:36.340+0000 7f14beffd640 1 --2- 172.21.15.92:0/1987291335 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f14b006ae70 0x7f14b006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:36.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:36.340+0000 7f14beffd640 1 -- 172.21.15.92:0/1987291335 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f14c80895f0 con 0x7f14d00ccb20 2023-11-24T06:10:36.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:36.341+0000 7f14ce59c640 1 --2- 172.21.15.92:0/1987291335 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f14b006ae70 0x7f14b006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:36.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:36.341+0000 7f14ce59c640 1 --2- 172.21.15.92:0/1987291335 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f14b006ae70 0x7f14b006d330 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f14c0001b20 tx=0x7f14c0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:36.348 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:36.345+0000 7f14beffd640 1 -- 172.21.15.92:0/1987291335 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f14d0157bb0 con 0x7f14d00ccb20 2023-11-24T06:10:36.557 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:36.554+0000 7f14d4ba5640 1 -- 172.21.15.92:0/1987291335 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f14d0157bb0 con 0x7f14d00ccb20 2023-11-24T06:10:36.558 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:36.555+0000 7f14beffd640 1 -- 172.21.15.92:0/1987291335 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f14d0157bb0 con 0x7f14d00ccb20 2023-11-24T06:10:36.558 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-24T06:10:36.559 INFO:teuthology.orchestra.run.smithi092.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-11-24T06:10:36.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:36.557+0000 7f14d4ba5640 1 -- 172.21.15.92:0/1987291335 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f14b006ae70 msgr2=0x7f14b006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:36.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:36.557+0000 7f14d4ba5640 1 --2- 172.21.15.92:0/1987291335 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f14b006ae70 0x7f14b006d330 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f14c0001b20 tx=0x7f14c0009040 comp rx=0 tx=0).stop 2023-11-24T06:10:36.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:36.557+0000 7f14d4ba5640 1 -- 172.21.15.92:0/1987291335 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f14d00ccb20 msgr2=0x7f14d00b2560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:36.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:36.557+0000 7f14d4ba5640 1 --2- 172.21.15.92:0/1987291335 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f14d00ccb20 0x7f14d00b2560 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f14c8006fd0 tx=0x7f14c800f690 comp rx=0 tx=0).stop 2023-11-24T06:10:36.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:36.557+0000 7f14d4ba5640 1 -- 172.21.15.92:0/1987291335 shutdown_connections 2023-11-24T06:10:36.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:36.557+0000 7f14d4ba5640 1 --2- 172.21.15.92:0/1987291335 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f14b006ae70 0x7f14b006d330 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:36.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:36.557+0000 7f14d4ba5640 1 --2- 172.21.15.92:0/1987291335 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f14d00ccb20 0x7f14d00b2560 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:36.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:36.557+0000 7f14d4ba5640 1 --2- 172.21.15.92:0/1987291335 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f14d0148910 0x7f14d00cc5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:36.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:36.557+0000 7f14d4ba5640 1 --2- 172.21.15.92:0/1987291335 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f14b80047c0 0x7f14d01f2ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:36.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:36.557+0000 7f14d4ba5640 1 -- 172.21.15.92:0/1987291335 >> 172.21.15.92:0/1987291335 conn(0x7f14d006a800 msgr2=0x7f14d01617c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:36.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:36.557+0000 7f14d4ba5640 1 -- 172.21.15.92:0/1987291335 shutdown_connections 2023-11-24T06:10:36.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:36.557+0000 7f14d4ba5640 1 -- 172.21.15.92:0/1987291335 wait complete. 2023-11-24T06:10:36.579 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2023-11-24T06:10:36.579 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:10:36.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:36.780+0000 7f92268e6640 1 Processor -- start 2023-11-24T06:10:36.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:36.780+0000 7f92268e6640 1 -- start start 2023-11-24T06:10:36.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:36.782+0000 7f92268e6640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f92200cc500 0x7f92200cc960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:36.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:36.782+0000 7f92268e6640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f922006d160 0x7f922006d590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:36.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:36.782+0000 7f92268e6640 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9220070e50 con 0x7f9220143c40 2023-11-24T06:10:36.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:36.782+0000 7f92268e6640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92200723c0 con 0x7f922006d160 2023-11-24T06:10:36.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:36.782+0000 7f92268e6640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9220144020 con 0x7f92200cc500 2023-11-24T06:10:36.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:36.782+0000 7f92260e5640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f922006d160 0x7f922006d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:36.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:36.782+0000 7f92260e5640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f922006d160 0x7f922006d590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47596/0 (socket says 172.21.15.92:47596) 2023-11-24T06:10:36.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:36.782+0000 7f92260e5640 1 -- 172.21.15.92:0/3791557494 learned_addr learned my addr 172.21.15.92:0/3791557494 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:10:36.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:36.782+0000 7f92250e3640 1 --2- 172.21.15.92:0/3791557494 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f92200cc500 0x7f92200cc960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:36.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:36.783+0000 7f92260e5640 1 -- 172.21.15.92:0/3791557494 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f92200cc500 msgr2=0x7f92200cc960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:36.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:36.783+0000 7f92260e5640 1 --2- 172.21.15.92:0/3791557494 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f92200cc500 0x7f92200cc960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:36.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:36.783+0000 7f92260e5640 1 -- 172.21.15.92:0/3791557494 >> v1:172.21.15.118:6789/0 conn(0x7f9220143c40 legacy=0x7f9220071290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:36.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:36.783+0000 7f92260e5640 1 -- 172.21.15.92:0/3791557494 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92200ccea0 con 0x7f922006d160 2023-11-24T06:10:36.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:36.784+0000 7f92260e5640 1 --2- 172.21.15.92:0/3791557494 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f922006d160 0x7f922006d590 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f9218009870 tx=0x7f921802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7d51d3fb19604da server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:36.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:36.785+0000 7f92248e2640 1 -- 172.21.15.92:0/3791557494 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9218031640 con 0x7f922006d160 2023-11-24T06:10:36.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:36.785+0000 7f92248e2640 1 -- 172.21.15.92:0/3791557494 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92180317a0 con 0x7f922006d160 2023-11-24T06:10:36.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:36.785+0000 7f92248e2640 1 -- 172.21.15.92:0/3791557494 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9218031b30 con 0x7f922006d160 2023-11-24T06:10:36.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:36.785+0000 7f92268e6640 1 -- 172.21.15.92:0/3791557494 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f922006d160 msgr2=0x7f922006d590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:36.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:36.785+0000 7f92268e6640 1 --2- 172.21.15.92:0/3791557494 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f922006d160 0x7f922006d590 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f9218009870 tx=0x7f921802b8f0 comp rx=0 tx=0).stop 2023-11-24T06:10:36.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:36.785+0000 7f92268e6640 1 -- 172.21.15.92:0/3791557494 shutdown_connections 2023-11-24T06:10:36.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:36.785+0000 7f92268e6640 1 --2- 172.21.15.92:0/3791557494 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f922006d160 0x7f922006d590 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:36.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:36.785+0000 7f92268e6640 1 --2- 172.21.15.92:0/3791557494 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f92200cc500 0x7f92200cc960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:36.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:36.785+0000 7f92268e6640 1 -- 172.21.15.92:0/3791557494 >> 172.21.15.92:0/3791557494 conn(0x7f922006a800 msgr2=0x7f922006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:36.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:36.786+0000 7f92268e6640 1 -- 172.21.15.92:0/3791557494 shutdown_connections 2023-11-24T06:10:36.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:36.786+0000 7f92268e6640 1 -- 172.21.15.92:0/3791557494 wait complete. 2023-11-24T06:10:36.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:36.787+0000 7f92268e6640 1 Processor -- start 2023-11-24T06:10:36.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:36.787+0000 7f92268e6640 1 -- start start 2023-11-24T06:10:36.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:36.788+0000 7f92268e6640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f922006d160 0x7f922015a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:36.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:36.788+0000 7f92268e6640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f92200cc500 0x7f92200a84c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:36.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:36.788+0000 7f92268e6640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9220143c40 0x7f92200a8a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:36.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:36.788+0000 7f92268e6640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92200a8f40 con 0x7f9220143c40 2023-11-24T06:10:36.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:36.788+0000 7f92268e6640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92200a9d50 con 0x7f922006d160 2023-11-24T06:10:36.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:36.788+0000 7f92268e6640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92200a9140 con 0x7f92200cc500 2023-11-24T06:10:36.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:36.789+0000 7f92260e5640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9220143c40 0x7f92200a8a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:36.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:36.789+0000 7f92260e5640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9220143c40 0x7f92200a8a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47602/0 (socket says 172.21.15.92:47602) 2023-11-24T06:10:36.794 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:36.789+0000 7f92260e5640 1 -- 172.21.15.92:0/2522392475 learned_addr learned my addr 172.21.15.92:0/2522392475 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:10:36.794 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:36.789+0000 7f92250e3640 1 --2- 172.21.15.92:0/2522392475 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f92200cc500 0x7f92200a84c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:36.794 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:36.789+0000 7f92258e4640 1 --2- 172.21.15.92:0/2522392475 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f922006d160 0x7f922015a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:36.794 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:36.789+0000 7f92260e5640 1 -- 172.21.15.92:0/2522392475 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f92200cc500 msgr2=0x7f92200a84c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:36.794 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:36.789+0000 7f92260e5640 1 --2- 172.21.15.92:0/2522392475 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f92200cc500 0x7f92200a84c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:36.794 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:36.789+0000 7f92260e5640 1 -- 172.21.15.92:0/2522392475 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f922006d160 msgr2=0x7f922015a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:36.794 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:36.789+0000 7f92260e5640 1 --2- 172.21.15.92:0/2522392475 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f922006d160 0x7f922015a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:36.794 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:36.789+0000 7f92260e5640 1 -- 172.21.15.92:0/2522392475 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92180094d0 con 0x7f9220143c40 2023-11-24T06:10:36.794 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:36.789+0000 7f92260e5640 1 --2- 172.21.15.92:0/2522392475 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9220143c40 0x7f92200a8a00 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f9218009870 tx=0x7f92180046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:36.794 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:36.790+0000 7f92127fc640 1 -- 172.21.15.92:0/2522392475 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f921803a930 con 0x7f9220143c40 2023-11-24T06:10:36.794 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:36.790+0000 7f92127fc640 1 -- 172.21.15.92:0/2522392475 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f921803aab0 con 0x7f9220143c40 2023-11-24T06:10:36.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:36.790+0000 7f92127fc640 1 -- 172.21.15.92:0/2522392475 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f921803ae60 con 0x7f9220143c40 2023-11-24T06:10:36.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:36.790+0000 7f92268e6640 1 -- 172.21.15.92:0/2522392475 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92200a9420 con 0x7f9220143c40 2023-11-24T06:10:36.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:36.790+0000 7f92268e6640 1 -- 172.21.15.92:0/2522392475 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92200a99e0 con 0x7f9220143c40 2023-11-24T06:10:36.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:36.791+0000 7f92268e6640 1 -- 172.21.15.92:0/2522392475 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92200a9ec0 con 0x7f9220143c40 2023-11-24T06:10:36.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:36.792+0000 7f92127fc640 1 -- 172.21.15.92:0/2522392475 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f921802f050 con 0x7f9220143c40 2023-11-24T06:10:36.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:36.793+0000 7f92127fc640 1 --2- 172.21.15.92:0/2522392475 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f920406ae70 0x7f920406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:36.797 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:36.793+0000 7f92127fc640 1 -- 172.21.15.92:0/2522392475 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f92180ab410 con 0x7f9220143c40 2023-11-24T06:10:36.797 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:36.793+0000 7f92258e4640 1 --2- 172.21.15.92:0/2522392475 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f920406ae70 0x7f920406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:36.797 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:36.793+0000 7f92258e4640 1 --2- 172.21.15.92:0/2522392475 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f920406ae70 0x7f920406d330 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f9220071390 tx=0x7f921c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:36.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:36.797+0000 7f92127fc640 1 -- 172.21.15.92:0/2522392475 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f92180764c0 con 0x7f9220143c40 2023-11-24T06:10:36.990 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:36.987+0000 7f92268e6640 1 -- 172.21.15.92:0/2522392475 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9220067450 con 0x7f9220143c40 2023-11-24T06:10:36.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:36.990+0000 7f92127fc640 1 -- 172.21.15.92:0/2522392475 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16457 (secure 0 0 0) 0x7f9218079ae0 con 0x7f9220143c40 2023-11-24T06:10:36.993 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-24T06:10:36.994 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":20,"fsid":"2d6dee5f-8aa3-470f-931a-50ade963b1cb","created":"2023-11-24T06:10:04.707204+0000","modified":"2023-11-24T06:10:26.698733+0000","last_up_change":"2023-11-24T06:10:17.598352+0000","last_in_change":"2023-11-24T06:10:09.040793+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:10:17.733098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:10:22.926445+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T06:10:24.151288+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ff37a9fb-135f-471f-b4b0-f7b5772006c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6803","nonce":20560714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6805","nonce":20560714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6809","nonce":20560714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6807","nonce":20560714}]},"public_addr":"172.21.15.92:6803/20560714","cluster_addr":"172.21.15.92:6805/20560714","heartbeat_back_addr":"172.21.15.92:6809/20560714","heartbeat_front_addr":"172.21.15.92:6807/20560714","state":["exists","up"]},{"osd":1,"uuid":"03cdd353-dcf1-4f38-94fe-a795137c06b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6819","nonce":2451239906}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6821","nonce":2451239906}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6825","nonce":2451239906}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6823","nonce":2451239906}]},"public_addr":"172.21.15.92:6819/2451239906","cluster_addr":"172.21.15.92:6821/2451239906","heartbeat_back_addr":"172.21.15.92:6825/2451239906","heartbeat_front_addr":"172.21.15.92:6823/2451239906","state":["exists","up"]},{"osd":2,"uuid":"a4a20d80-5d57-484b-becd-9aa8070dd3aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6811","nonce":1124979277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6813","nonce":1124979277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6817","nonce":1124979277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6815","nonce":1124979277}]},"public_addr":"172.21.15.92:6811/1124979277","cluster_addr":"172.21.15.92:6813/1124979277","heartbeat_back_addr":"172.21.15.92:6817/1124979277","heartbeat_front_addr":"172.21.15.92:6815/1124979277","state":["exists","up"]},{"osd":3,"uuid":"4648d7ce-9b29-4583-9425-4ba8fd2b1903","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6827","nonce":1734115782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6829","nonce":1734115782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6833","nonce":1734115782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6831","nonce":1734115782}]},"public_addr":"172.21.15.92:6827/1734115782","cluster_addr":"172.21.15.92:6829/1734115782","heartbeat_back_addr":"172.21.15.92:6833/1734115782","heartbeat_front_addr":"172.21.15.92:6831/1734115782","state":["exists","up"]},{"osd":4,"uuid":"3bc9d7eb-c6fa-4fcf-a3ec-ade604e05aac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6819","nonce":3425937229}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6822","nonce":3425937229}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6830","nonce":3425937229}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6826","nonce":3425937229}]},"public_addr":"172.21.15.118:6819/3425937229","cluster_addr":"172.21.15.118:6822/3425937229","heartbeat_back_addr":"172.21.15.118:6830/3425937229","heartbeat_front_addr":"172.21.15.118:6826/3425937229","state":["exists","up"]},{"osd":5,"uuid":"dce9ba6d-59ed-46d1-84fc-a84b13de10c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6801","nonce":827455984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6803","nonce":827455984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6807","nonce":827455984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6805","nonce":827455984}]},"public_addr":"172.21.15.118:6801/827455984","cluster_addr":"172.21.15.118:6803/827455984","heartbeat_back_addr":"172.21.15.118:6807/827455984","heartbeat_front_addr":"172.21.15.118:6805/827455984","state":["exists","up"]},{"osd":6,"uuid":"82aec504-ae58-4709-8d30-46f2b12081b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6809","nonce":2946986141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6811","nonce":2946986141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6815","nonce":2946986141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6813","nonce":2946986141}]},"public_addr":"172.21.15.118:6809/2946986141","cluster_addr":"172.21.15.118:6811/2946986141","heartbeat_back_addr":"172.21.15.118:6815/2946986141","heartbeat_front_addr":"172.21.15.118:6813/2946986141","state":["exists","up"]},{"osd":7,"uuid":"4ed4c624-c88f-4d35-bdb1-bf209b5dffb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6817","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6818","nonce":48122059}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6821","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6823","nonce":48122059}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6829","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6831","nonce":48122059}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6825","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6827","nonce":48122059}]},"public_addr":"172.21.15.118:6818/48122059","cluster_addr":"172.21.15.118:6823/48122059","heartbeat_back_addr":"172.21.15.118:6831/48122059","heartbeat_front_addr":"172.21.15.118:6827/48122059","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.536664+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.368324+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.271201+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.342971+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.621832+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.387110+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.632946+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.416032+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:10:36.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:36.992+0000 7f92268e6640 1 -- 172.21.15.92:0/2522392475 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f920406ae70 msgr2=0x7f920406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:36.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:36.992+0000 7f92268e6640 1 --2- 172.21.15.92:0/2522392475 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f920406ae70 0x7f920406d330 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f9220071390 tx=0x7f921c009040 comp rx=0 tx=0).stop 2023-11-24T06:10:36.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:36.992+0000 7f92268e6640 1 -- 172.21.15.92:0/2522392475 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9220143c40 msgr2=0x7f92200a8a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:36.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:36.992+0000 7f92268e6640 1 --2- 172.21.15.92:0/2522392475 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9220143c40 0x7f92200a8a00 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f9218009870 tx=0x7f92180046c0 comp rx=0 tx=0).stop 2023-11-24T06:10:36.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:36.992+0000 7f92268e6640 1 -- 172.21.15.92:0/2522392475 shutdown_connections 2023-11-24T06:10:36.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:36.992+0000 7f92268e6640 1 --2- 172.21.15.92:0/2522392475 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f920406ae70 0x7f920406d330 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:36.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:36.992+0000 7f92268e6640 1 --2- 172.21.15.92:0/2522392475 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9220143c40 0x7f92200a8a00 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:36.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:36.992+0000 7f92268e6640 1 --2- 172.21.15.92:0/2522392475 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f92200cc500 0x7f92200a84c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:36.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:36.992+0000 7f92268e6640 1 --2- 172.21.15.92:0/2522392475 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f922006d160 0x7f922015a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:36.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:36.993+0000 7f92268e6640 1 -- 172.21.15.92:0/2522392475 >> 172.21.15.92:0/2522392475 conn(0x7f922006a800 msgr2=0x7f92200af6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:36.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:36.993+0000 7f92268e6640 1 -- 172.21.15.92:0/2522392475 shutdown_connections 2023-11-24T06:10:36.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:36.993+0000 7f92268e6640 1 -- 172.21.15.92:0/2522392475 wait complete. 2023-11-24T06:10:37.016 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T06:10:17.733098+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T06:10:22.926445+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T06:10:24.151288+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T06:10:37.017 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T06:10:37.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:37.228+0000 7f0d73fff640 1 Processor -- start 2023-11-24T06:10:37.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:37.228+0000 7f0d73fff640 1 -- start start 2023-11-24T06:10:37.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:37.230+0000 7f0d73fff640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0d7406d6d0 0x7f0d740b3b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:37.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:37.230+0000 7f0d73fff640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f0d7415a690 0x7f0d740ccc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:37.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:37.230+0000 7f0d73fff640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d740b1e30 con 0x7f0d741505b0 2023-11-24T06:10:37.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:37.230+0000 7f0d73fff640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d740b20a0 con 0x7f0d7415a690 2023-11-24T06:10:37.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:37.230+0000 7f0d73fff640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d740b2380 con 0x7f0d7406d6d0 2023-11-24T06:10:37.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:37.230+0000 7f0d72ffd640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f0d741505b0 0x7f0d7406cfc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:55918/0 (socket says 172.21.15.92:55918) 2023-11-24T06:10:37.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:37.230+0000 7f0d72ffd640 1 -- 172.21.15.92:0/4080287780 learned_addr learned my addr 172.21.15.92:0/4080287780 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-24T06:10:37.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:37.230+0000 7f0d737fe640 1 --2- 172.21.15.92:0/4080287780 >> v2:172.21.15.92:3300/0 conn(0x7f0d7415a690 0x7f0d740ccc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:37.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:37.230+0000 7f0d6bfff640 1 --2- 172.21.15.92:0/4080287780 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0d7406d6d0 0x7f0d740b3b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:37.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:37.230+0000 7f0d727fc640 1 -- 172.21.15.92:0/4080287780 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1374762018 0 0) 0x7f0d740b1e30 con 0x7f0d741505b0 2023-11-24T06:10:37.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:37.231+0000 7f0d727fc640 1 -- 172.21.15.92:0/4080287780 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0d60004630 con 0x7f0d741505b0 2023-11-24T06:10:37.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:37.231+0000 7f0d737fe640 1 -- 172.21.15.92:0/4080287780 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0d7406d6d0 msgr2=0x7f0d740b3b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:37.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:37.231+0000 7f0d737fe640 1 --2- 172.21.15.92:0/4080287780 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0d7406d6d0 0x7f0d740b3b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:37.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:37.231+0000 7f0d737fe640 1 -- 172.21.15.92:0/4080287780 >> v1:172.21.15.92:6789/0 conn(0x7f0d741505b0 legacy=0x7f0d7406cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:37.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:37.231+0000 7f0d727fc640 1 -- 172.21.15.92:0/4080287780 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1466971765 0 0) 0x7f0d60004630 con 0x7f0d741505b0 2023-11-24T06:10:37.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:37.231+0000 7f0d737fe640 1 -- 172.21.15.92:0/4080287780 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d740b2660 con 0x7f0d7415a690 2023-11-24T06:10:37.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:37.232+0000 7f0d737fe640 1 --2- 172.21.15.92:0/4080287780 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0d7415a690 0x7f0d740ccc00 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f0d6c009d90 tx=0x7f0d6c02bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd7c55d8e5e2b1e0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:37.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:37.232+0000 7f0d727fc640 1 -- 172.21.15.92:0/4080287780 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d64001670 con 0x7f0d7415a690 2023-11-24T06:10:37.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:37.232+0000 7f0d727fc640 1 -- 172.21.15.92:0/4080287780 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d6c02c8a0 con 0x7f0d7415a690 2023-11-24T06:10:37.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:37.232+0000 7f0d727fc640 1 -- 172.21.15.92:0/4080287780 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d6c02cc30 con 0x7f0d7415a690 2023-11-24T06:10:37.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:37.233+0000 7f0d73fff640 1 -- 172.21.15.92:0/4080287780 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0d7415a690 msgr2=0x7f0d740ccc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:37.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:37.233+0000 7f0d73fff640 1 --2- 172.21.15.92:0/4080287780 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0d7415a690 0x7f0d740ccc00 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f0d6c009d90 tx=0x7f0d6c02bd70 comp rx=0 tx=0).stop 2023-11-24T06:10:37.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:37.233+0000 7f0d73fff640 1 -- 172.21.15.92:0/4080287780 shutdown_connections 2023-11-24T06:10:37.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:37.233+0000 7f0d73fff640 1 --2- 172.21.15.92:0/4080287780 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0d7415a690 0x7f0d740ccc00 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:37.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:37.233+0000 7f0d73fff640 1 --2- 172.21.15.92:0/4080287780 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0d7406d6d0 0x7f0d740b3b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:37.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:37.233+0000 7f0d73fff640 1 -- 172.21.15.92:0/4080287780 >> 172.21.15.92:0/4080287780 conn(0x7f0d7406a7d0 msgr2=0x7f0d7406abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:37.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:37.233+0000 7f0d73fff640 1 -- 172.21.15.92:0/4080287780 shutdown_connections 2023-11-24T06:10:37.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:37.233+0000 7f0d73fff640 1 -- 172.21.15.92:0/4080287780 wait complete. 2023-11-24T06:10:37.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:37.235+0000 7f0d73fff640 1 Processor -- start 2023-11-24T06:10:37.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:37.235+0000 7f0d73fff640 1 -- start start 2023-11-24T06:10:37.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:37.236+0000 7f0d73fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0d7406d6d0 0x7f0d740c5f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:37.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:37.236+0000 7f0d73fff640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0d741505b0 0x7f0d740c7310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:37.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:37.236+0000 7f0d73fff640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0d7415a690 0x7f0d740ca210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:37.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:37.236+0000 7f0d73fff640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d740c4e80 con 0x7f0d7406d6d0 2023-11-24T06:10:37.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:37.236+0000 7f0d73fff640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d740c5160 con 0x7f0d741505b0 2023-11-24T06:10:37.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:37.236+0000 7f0d73fff640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d740c5440 con 0x7f0d7415a690 2023-11-24T06:10:37.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:37.236+0000 7f0d72ffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0d7406d6d0 0x7f0d740c5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:37.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:37.236+0000 7f0d72ffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0d7406d6d0 0x7f0d740c5f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47616/0 (socket says 172.21.15.92:47616) 2023-11-24T06:10:37.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:37.236+0000 7f0d72ffd640 1 -- 172.21.15.92:0/3458553075 learned_addr learned my addr 172.21.15.92:0/3458553075 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:10:37.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:37.236+0000 7f0d72ffd640 1 -- 172.21.15.92:0/3458553075 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0d7415a690 msgr2=0x7f0d740ca210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:37.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:37.236+0000 7f0d6bfff640 1 --2- 172.21.15.92:0/3458553075 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0d741505b0 0x7f0d740c7310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:37.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:37.236+0000 7f0d72ffd640 1 --2- 172.21.15.92:0/3458553075 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0d7415a690 0x7f0d740ca210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:37.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:37.236+0000 7f0d72ffd640 1 -- 172.21.15.92:0/3458553075 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0d741505b0 msgr2=0x7f0d740c7310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:37.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:37.236+0000 7f0d72ffd640 1 --2- 172.21.15.92:0/3458553075 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0d741505b0 0x7f0d740c7310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:37.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:37.236+0000 7f0d72ffd640 1 -- 172.21.15.92:0/3458553075 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d6c009a70 con 0x7f0d7406d6d0 2023-11-24T06:10:37.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:37.237+0000 7f0d72ffd640 1 --2- 172.21.15.92:0/3458553075 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0d7406d6d0 0x7f0d740c5f20 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f0d6400c710 tx=0x7f0d6400cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:37.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:37.237+0000 7f0d6b7fe640 1 -- 172.21.15.92:0/3458553075 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d64005ad0 con 0x7f0d7406d6d0 2023-11-24T06:10:37.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:37.237+0000 7f0d6b7fe640 1 -- 172.21.15.92:0/3458553075 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d64005c30 con 0x7f0d7406d6d0 2023-11-24T06:10:37.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:37.237+0000 7f0d6b7fe640 1 -- 172.21.15.92:0/3458553075 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d64007070 con 0x7f0d7406d6d0 2023-11-24T06:10:37.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:37.237+0000 7f0d73fff640 1 -- 172.21.15.92:0/3458553075 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d740c5720 con 0x7f0d7406d6d0 2023-11-24T06:10:37.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:37.237+0000 7f0d73fff640 1 -- 172.21.15.92:0/3458553075 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d740c6460 con 0x7f0d7406d6d0 2023-11-24T06:10:37.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:37.238+0000 7f0d73fff640 1 -- 172.21.15.92:0/3458553075 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d74067450 con 0x7f0d7406d6d0 2023-11-24T06:10:37.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:37.239+0000 7f0d6b7fe640 1 -- 172.21.15.92:0/3458553075 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f0d640071f0 con 0x7f0d7406d6d0 2023-11-24T06:10:37.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:37.240+0000 7f0d6b7fe640 1 --2- 172.21.15.92:0/3458553075 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f0d5406ae40 0x7f0d5406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:37.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:37.240+0000 7f0d6b7fe640 1 -- 172.21.15.92:0/3458553075 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0d64089530 con 0x7f0d7406d6d0 2023-11-24T06:10:37.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:37.240+0000 7f0d6bfff640 1 --2- 172.21.15.92:0/3458553075 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f0d5406ae40 0x7f0d5406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:37.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:37.241+0000 7f0d6bfff640 1 --2- 172.21.15.92:0/3458553075 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f0d5406ae40 0x7f0d5406d300 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f0d74159e20 tx=0x7f0d5c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:37.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:37.244+0000 7f0d6b7fe640 1 -- 172.21.15.92:0/3458553075 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0d64054540 con 0x7f0d7406d6d0 2023-11-24T06:10:37.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:37.431+0000 7f0d73fff640 1 -- 172.21.15.92:0/3458553075 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0d740b3660 con 0x7f0d7406d6d0 2023-11-24T06:10:37.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:37.432+0000 7f0d6b7fe640 1 -- 172.21.15.92:0/3458553075 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0d64057b60 con 0x7f0d7406d6d0 2023-11-24T06:10:37.435 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 1 2023-11-24T06:10:37.438 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:37.435+0000 7f0d73fff640 1 -- 172.21.15.92:0/3458553075 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f0d5406ae40 msgr2=0x7f0d5406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:37.438 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:37.435+0000 7f0d73fff640 1 --2- 172.21.15.92:0/3458553075 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f0d5406ae40 0x7f0d5406d300 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f0d74159e20 tx=0x7f0d5c008040 comp rx=0 tx=0).stop 2023-11-24T06:10:37.439 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:37.435+0000 7f0d73fff640 1 -- 172.21.15.92:0/3458553075 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0d7406d6d0 msgr2=0x7f0d740c5f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:37.439 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:37.435+0000 7f0d73fff640 1 --2- 172.21.15.92:0/3458553075 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0d7406d6d0 0x7f0d740c5f20 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f0d6400c710 tx=0x7f0d6400cbd0 comp rx=0 tx=0).stop 2023-11-24T06:10:37.439 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:37.435+0000 7f0d73fff640 1 -- 172.21.15.92:0/3458553075 shutdown_connections 2023-11-24T06:10:37.439 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:37.435+0000 7f0d73fff640 1 --2- 172.21.15.92:0/3458553075 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f0d5406ae40 0x7f0d5406d300 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:37.439 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:37.435+0000 7f0d73fff640 1 --2- 172.21.15.92:0/3458553075 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0d7415a690 0x7f0d740ca210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:37.439 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:37.435+0000 7f0d73fff640 1 --2- 172.21.15.92:0/3458553075 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0d741505b0 0x7f0d740c7310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:37.439 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:37.435+0000 7f0d73fff640 1 --2- 172.21.15.92:0/3458553075 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0d7406d6d0 0x7f0d740c5f20 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:37.439 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:37.435+0000 7f0d73fff640 1 -- 172.21.15.92:0/3458553075 >> 172.21.15.92:0/3458553075 conn(0x7f0d7406a7d0 msgr2=0x7f0d7414f350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:37.439 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:37.435+0000 7f0d73fff640 1 -- 172.21.15.92:0/3458553075 shutdown_connections 2023-11-24T06:10:37.439 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:37.435+0000 7f0d73fff640 1 -- 172.21.15.92:0/3458553075 wait complete. 2023-11-24T06:10:37.457 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T06:10:37.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:37.671+0000 7fd15622d640 1 Processor -- start 2023-11-24T06:10:37.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:37.672+0000 7fd15622d640 1 -- start start 2023-11-24T06:10:37.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:37.673+0000 7fd15622d640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fd1500cf9a0 0x7fd150166030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:37.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:37.673+0000 7fd15622d640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd15015c5f0 con 0x7fd15013e360 2023-11-24T06:10:37.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:37.673+0000 7fd15622d640 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd15006c470 con 0x7fd15013df80 2023-11-24T06:10:37.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:37.673+0000 7fd15622d640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd15006c750 con 0x7fd1500cf9a0 2023-11-24T06:10:37.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:37.673+0000 7fd14ffff640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fd1500cf9a0 0x7fd150166030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:37.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:37.674+0000 7fd14effd640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fd15013e360 0x7fd1500cf290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:55924/0 (socket says 172.21.15.92:55924) 2023-11-24T06:10:37.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:37.674+0000 7fd14effd640 1 -- 172.21.15.92:0/620363807 learned_addr learned my addr 172.21.15.92:0/620363807 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-24T06:10:37.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:37.674+0000 7fd14ffff640 1 -- 172.21.15.92:0/620363807 >> v1:172.21.15.118:6789/0 conn(0x7fd15013df80 legacy=0x7fd15015bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:37.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:37.674+0000 7fd14ffff640 1 -- 172.21.15.92:0/620363807 >> v1:172.21.15.92:6789/0 conn(0x7fd15013e360 legacy=0x7fd1500cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:37.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:37.674+0000 7fd14ffff640 1 -- 172.21.15.92:0/620363807 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd15006ca30 con 0x7fd1500cf9a0 2023-11-24T06:10:37.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:37.675+0000 7fd14ffff640 1 --2- 172.21.15.92:0/620363807 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd1500cf9a0 0x7fd150166030 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fd148009d90 tx=0x7fd14802beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2927c64c3ff8e37b server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:37.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:37.675+0000 7fd14e7fc640 1 -- 172.21.15.92:0/620363807 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd14802cc60 con 0x7fd1500cf9a0 2023-11-24T06:10:37.679 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:37.675+0000 7fd14e7fc640 1 -- 172.21.15.92:0/620363807 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd14802cdc0 con 0x7fd1500cf9a0 2023-11-24T06:10:37.679 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:37.675+0000 7fd14e7fc640 1 -- 172.21.15.92:0/620363807 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd148034660 con 0x7fd1500cf9a0 2023-11-24T06:10:37.679 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:37.676+0000 7fd15622d640 1 -- 172.21.15.92:0/620363807 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd1500cf9a0 msgr2=0x7fd150166030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:37.679 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:37.676+0000 7fd15622d640 1 --2- 172.21.15.92:0/620363807 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd1500cf9a0 0x7fd150166030 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fd148009d90 tx=0x7fd14802beb0 comp rx=0 tx=0).stop 2023-11-24T06:10:37.679 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:37.676+0000 7fd15622d640 1 -- 172.21.15.92:0/620363807 shutdown_connections 2023-11-24T06:10:37.680 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:37.676+0000 7fd15622d640 1 --2- 172.21.15.92:0/620363807 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd1500cf9a0 0x7fd150166030 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:37.680 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:37.676+0000 7fd15622d640 1 -- 172.21.15.92:0/620363807 >> 172.21.15.92:0/620363807 conn(0x7fd15006a800 msgr2=0x7fd15006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:37.680 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:37.676+0000 7fd15622d640 1 -- 172.21.15.92:0/620363807 shutdown_connections 2023-11-24T06:10:37.680 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:37.676+0000 7fd15622d640 1 -- 172.21.15.92:0/620363807 wait complete. 2023-11-24T06:10:37.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:37.678+0000 7fd15622d640 1 Processor -- start 2023-11-24T06:10:37.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:37.678+0000 7fd15622d640 1 -- start start 2023-11-24T06:10:37.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:37.679+0000 7fd15622d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd1500cf9a0 0x7fd1501f60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:37.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:37.679+0000 7fd15622d640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd15013df80 0x7fd1501e86b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:37.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:37.679+0000 7fd15622d640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd15013e360 0x7fd1501e4ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:37.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:37.679+0000 7fd15622d640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd15013d9b0 con 0x7fd1500cf9a0 2023-11-24T06:10:37.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:37.679+0000 7fd15622d640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1501f5c60 con 0x7fd15013e360 2023-11-24T06:10:37.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:37.679+0000 7fd15622d640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1501f5050 con 0x7fd15013df80 2023-11-24T06:10:37.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:37.679+0000 7fd14f7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd1500cf9a0 0x7fd1501f60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:37.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:37.680+0000 7fd14f7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd1500cf9a0 0x7fd1501f60f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47640/0 (socket says 172.21.15.92:47640) 2023-11-24T06:10:37.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:37.680+0000 7fd14f7fe640 1 -- 172.21.15.92:0/1246295694 learned_addr learned my addr 172.21.15.92:0/1246295694 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:10:37.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:37.680+0000 7fd14ffff640 1 --2- 172.21.15.92:0/1246295694 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd15013e360 0x7fd1501e4ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:37.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:37.680+0000 7fd14f7fe640 1 -- 172.21.15.92:0/1246295694 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd15013df80 msgr2=0x7fd1501e86b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:37.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:37.680+0000 7fd14f7fe640 1 --2- 172.21.15.92:0/1246295694 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd15013df80 0x7fd1501e86b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:37.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:37.680+0000 7fd14f7fe640 1 -- 172.21.15.92:0/1246295694 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd15013e360 msgr2=0x7fd1501e4ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:37.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:37.680+0000 7fd14f7fe640 1 --2- 172.21.15.92:0/1246295694 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd15013e360 0x7fd1501e4ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:37.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:37.680+0000 7fd14f7fe640 1 -- 172.21.15.92:0/1246295694 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd148009a70 con 0x7fd1500cf9a0 2023-11-24T06:10:37.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:37.680+0000 7fd14ffff640 1 --2- 172.21.15.92:0/1246295694 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd15013e360 0x7fd1501e4ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:10:37.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:37.680+0000 7fd14f7fe640 1 --2- 172.21.15.92:0/1246295694 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd1500cf9a0 0x7fd1501f60f0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fd14000b3f0 tx=0x7fd14000b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:37.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:37.680+0000 7fd137fff640 1 -- 172.21.15.92:0/1246295694 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd140011830 con 0x7fd1500cf9a0 2023-11-24T06:10:37.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:37.680+0000 7fd137fff640 1 -- 172.21.15.92:0/1246295694 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd140011990 con 0x7fd1500cf9a0 2023-11-24T06:10:37.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:37.680+0000 7fd137fff640 1 -- 172.21.15.92:0/1246295694 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd140011d40 con 0x7fd1500cf9a0 2023-11-24T06:10:37.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:37.681+0000 7fd15622d640 1 -- 172.21.15.92:0/1246295694 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1501f5330 con 0x7fd1500cf9a0 2023-11-24T06:10:37.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:37.681+0000 7fd15622d640 1 -- 172.21.15.92:0/1246295694 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1501f58f0 con 0x7fd1500cf9a0 2023-11-24T06:10:37.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:37.682+0000 7fd15622d640 1 -- 172.21.15.92:0/1246295694 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1500b0640 con 0x7fd1500cf9a0 2023-11-24T06:10:37.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:37.683+0000 7fd137fff640 1 -- 172.21.15.92:0/1246295694 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd14001a400 con 0x7fd1500cf9a0 2023-11-24T06:10:37.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:37.684+0000 7fd137fff640 1 --2- 172.21.15.92:0/1246295694 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fd13c06ae40 0x7fd13c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:37.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:37.684+0000 7fd137fff640 1 -- 172.21.15.92:0/1246295694 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd1400894c0 con 0x7fd1500cf9a0 2023-11-24T06:10:37.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:37.685+0000 7fd14effd640 1 --2- 172.21.15.92:0/1246295694 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fd13c06ae40 0x7fd13c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:37.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:37.685+0000 7fd14effd640 1 --2- 172.21.15.92:0/1246295694 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fd13c06ae40 0x7fd13c06d300 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fd1500a8570 tx=0x7fd144007a40 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:37.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:37.689+0000 7fd137fff640 1 -- 172.21.15.92:0/1246295694 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd140054570 con 0x7fd1500cf9a0 2023-11-24T06:10:37.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:37.880+0000 7fd15622d640 1 -- 172.21.15.92:0/1246295694 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd1500ccb80 con 0x7fd1500cf9a0 2023-11-24T06:10:37.884 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:37.881+0000 7fd137fff640 1 -- 172.21.15.92:0/1246295694 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7fd140057b90 con 0x7fd1500cf9a0 2023-11-24T06:10:37.884 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 32 2023-11-24T06:10:37.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:37.883+0000 7fd15622d640 1 -- 172.21.15.92:0/1246295694 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fd13c06ae40 msgr2=0x7fd13c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:37.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:37.883+0000 7fd15622d640 1 --2- 172.21.15.92:0/1246295694 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fd13c06ae40 0x7fd13c06d300 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fd1500a8570 tx=0x7fd144007a40 comp rx=0 tx=0).stop 2023-11-24T06:10:37.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:37.883+0000 7fd15622d640 1 -- 172.21.15.92:0/1246295694 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd1500cf9a0 msgr2=0x7fd1501f60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:37.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:37.883+0000 7fd15622d640 1 --2- 172.21.15.92:0/1246295694 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd1500cf9a0 0x7fd1501f60f0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fd14000b3f0 tx=0x7fd14000b8b0 comp rx=0 tx=0).stop 2023-11-24T06:10:37.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:37.883+0000 7fd15622d640 1 -- 172.21.15.92:0/1246295694 shutdown_connections 2023-11-24T06:10:37.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:37.883+0000 7fd15622d640 1 --2- 172.21.15.92:0/1246295694 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fd13c06ae40 0x7fd13c06d300 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:37.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:37.883+0000 7fd15622d640 1 --2- 172.21.15.92:0/1246295694 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd15013e360 0x7fd1501e4ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:37.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:37.883+0000 7fd15622d640 1 --2- 172.21.15.92:0/1246295694 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd15013df80 0x7fd1501e86b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:37.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:37.883+0000 7fd15622d640 1 --2- 172.21.15.92:0/1246295694 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd1500cf9a0 0x7fd1501f60f0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:37.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:37.883+0000 7fd15622d640 1 -- 172.21.15.92:0/1246295694 >> 172.21.15.92:0/1246295694 conn(0x7fd15006a800 msgr2=0x7fd1500aff30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:37.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:37.883+0000 7fd15622d640 1 -- 172.21.15.92:0/1246295694 shutdown_connections 2023-11-24T06:10:37.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:37.883+0000 7fd15622d640 1 -- 172.21.15.92:0/1246295694 wait complete. 2023-11-24T06:10:37.907 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T06:10:38.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:38.107+0000 7f141f9be640 1 Processor -- start 2023-11-24T06:10:38.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:38.107+0000 7f141f9be640 1 -- start start 2023-11-24T06:10:38.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:38.109+0000 7f141f9be640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f1418143ae0 0x7f14180cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:38.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:38.109+0000 7f141f9be640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f14180cf7d0 0x7f141815dca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:38.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:38.109+0000 7f141f9be640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f141815e1e0 0x7f141815a670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:38.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:38.109+0000 7f141f9be640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14180cfc30 con 0x7f141815e1e0 2023-11-24T06:10:38.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:38.109+0000 7f141f9be640 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14180b1ee0 con 0x7f1418143ae0 2023-11-24T06:10:38.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:38.109+0000 7f141f9be640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14180b2780 con 0x7f14180cf7d0 2023-11-24T06:10:38.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:38.109+0000 7f141df5b640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f141815e1e0 0x7f141815a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:38.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:38.109+0000 7f141df5b640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f141815e1e0 0x7f141815a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47656/0 (socket says 172.21.15.92:47656) 2023-11-24T06:10:38.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:38.109+0000 7f141df5b640 1 -- 172.21.15.92:0/2053642702 learned_addr learned my addr 172.21.15.92:0/2053642702 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:10:38.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:38.109+0000 7f141d75a640 1 --2- 172.21.15.92:0/2053642702 >> v2:172.21.15.118:3300/0 conn(0x7f1418143ae0 0x7f14180cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:38.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:38.109+0000 7f141cf59640 1 --2- 172.21.15.92:0/2053642702 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f14180cf7d0 0x7f141815dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:38.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:38.110+0000 7f141df5b640 1 -- 172.21.15.92:0/2053642702 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f14180cf7d0 msgr2=0x7f141815dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:38.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:38.110+0000 7f141df5b640 1 --2- 172.21.15.92:0/2053642702 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f14180cf7d0 0x7f141815dca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:38.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:38.110+0000 7f141df5b640 1 -- 172.21.15.92:0/2053642702 >> v2:172.21.15.118:3300/0 conn(0x7f1418143ae0 msgr2=0x7f14180cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:38.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:38.110+0000 7f141df5b640 1 --2- 172.21.15.92:0/2053642702 >> v2:172.21.15.118:3300/0 conn(0x7f1418143ae0 0x7f14180cf290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:38.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:38.110+0000 7f141df5b640 1 -- 172.21.15.92:0/2053642702 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f141815e990 con 0x7f141815e1e0 2023-11-24T06:10:38.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:38.110+0000 7f141d75a640 1 --2- 172.21.15.92:0/2053642702 >> v2:172.21.15.118:3300/0 conn(0x7f1418143ae0 0x7f14180cf290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:10:38.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:38.111+0000 7f141df5b640 1 --2- 172.21.15.92:0/2053642702 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f141815e1e0 0x7f141815a670 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f14180cfea0 tx=0x7f141002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d13985ccea3394 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:38.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:38.111+0000 7f140ffff640 1 -- 172.21.15.92:0/2053642702 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f141002ca70 con 0x7f141815e1e0 2023-11-24T06:10:38.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:38.111+0000 7f140ffff640 1 -- 172.21.15.92:0/2053642702 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f141002cbd0 con 0x7f141815e1e0 2023-11-24T06:10:38.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:38.111+0000 7f140ffff640 1 -- 172.21.15.92:0/2053642702 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14100343f0 con 0x7f141815e1e0 2023-11-24T06:10:38.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:38.112+0000 7f141f9be640 1 -- 172.21.15.92:0/2053642702 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f141815e1e0 msgr2=0x7f141815a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:38.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:38.112+0000 7f141f9be640 1 --2- 172.21.15.92:0/2053642702 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f141815e1e0 0x7f141815a670 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f14180cfea0 tx=0x7f141002bed0 comp rx=0 tx=0).stop 2023-11-24T06:10:38.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:38.112+0000 7f141f9be640 1 -- 172.21.15.92:0/2053642702 shutdown_connections 2023-11-24T06:10:38.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:38.112+0000 7f141f9be640 1 --2- 172.21.15.92:0/2053642702 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f141815e1e0 0x7f141815a670 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:38.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:38.112+0000 7f141f9be640 1 --2- 172.21.15.92:0/2053642702 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f14180cf7d0 0x7f141815dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:38.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:38.112+0000 7f141f9be640 1 --2- 172.21.15.92:0/2053642702 >> v2:172.21.15.118:3300/0 conn(0x7f1418143ae0 0x7f14180cf290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:38.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:38.112+0000 7f141f9be640 1 -- 172.21.15.92:0/2053642702 >> 172.21.15.92:0/2053642702 conn(0x7f141806a800 msgr2=0x7f141806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:38.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:38.112+0000 7f141f9be640 1 -- 172.21.15.92:0/2053642702 shutdown_connections 2023-11-24T06:10:38.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:38.112+0000 7f141f9be640 1 -- 172.21.15.92:0/2053642702 wait complete. 2023-11-24T06:10:38.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:38.114+0000 7f141f9be640 1 Processor -- start 2023-11-24T06:10:38.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:38.114+0000 7f141f9be640 1 -- start start 2023-11-24T06:10:38.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:38.115+0000 7f141f9be640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f14180cf7d0 0x7f14181f22d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:38.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:38.115+0000 7f141f9be640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1418143ae0 0x7f141806c470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:38.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:38.115+0000 7f141f9be640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f141815e1e0 0x7f141806c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:38.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:38.115+0000 7f141f9be640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14181f2810 con 0x7f141815e1e0 2023-11-24T06:10:38.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:38.115+0000 7f141f9be640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14180cd950 con 0x7f1418143ae0 2023-11-24T06:10:38.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:38.115+0000 7f141f9be640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14180ad4b0 con 0x7f14180cf7d0 2023-11-24T06:10:38.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:38.115+0000 7f141df5b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f141815e1e0 0x7f141806c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:38.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:38.115+0000 7f141df5b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f141815e1e0 0x7f141806c9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47662/0 (socket says 172.21.15.92:47662) 2023-11-24T06:10:38.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:38.115+0000 7f141df5b640 1 -- 172.21.15.92:0/395370596 learned_addr learned my addr 172.21.15.92:0/395370596 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:10:38.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:38.115+0000 7f141cf59640 1 --2- 172.21.15.92:0/395370596 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1418143ae0 0x7f141806c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:38.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:38.116+0000 7f141d75a640 1 --2- 172.21.15.92:0/395370596 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f14180cf7d0 0x7f14181f22d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:38.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:38.116+0000 7f141df5b640 1 -- 172.21.15.92:0/395370596 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f14180cf7d0 msgr2=0x7f14181f22d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:38.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:38.116+0000 7f141df5b640 1 --2- 172.21.15.92:0/395370596 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f14180cf7d0 0x7f14181f22d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:38.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:38.116+0000 7f141df5b640 1 -- 172.21.15.92:0/395370596 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1418143ae0 msgr2=0x7f141806c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:38.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:38.116+0000 7f141df5b640 1 --2- 172.21.15.92:0/395370596 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1418143ae0 0x7f141806c470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:38.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:38.116+0000 7f141df5b640 1 -- 172.21.15.92:0/395370596 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1410009b30 con 0x7f141815e1e0 2023-11-24T06:10:38.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:38.116+0000 7f141df5b640 1 --2- 172.21.15.92:0/395370596 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f141815e1e0 0x7f141806c9b0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f1410036040 tx=0x7f1410004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:38.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:38.116+0000 7f140dffb640 1 -- 172.21.15.92:0/395370596 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1410040670 con 0x7f141815e1e0 2023-11-24T06:10:38.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:38.116+0000 7f140dffb640 1 -- 172.21.15.92:0/395370596 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14100407d0 con 0x7f141815e1e0 2023-11-24T06:10:38.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:38.116+0000 7f140dffb640 1 -- 172.21.15.92:0/395370596 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1410040b80 con 0x7f141815e1e0 2023-11-24T06:10:38.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:38.116+0000 7f141f9be640 1 -- 172.21.15.92:0/395370596 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14180ad790 con 0x7f141815e1e0 2023-11-24T06:10:38.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:38.117+0000 7f141f9be640 1 -- 172.21.15.92:0/395370596 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14180add50 con 0x7f141815e1e0 2023-11-24T06:10:38.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:38.118+0000 7f141f9be640 1 -- 172.21.15.92:0/395370596 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14181f2980 con 0x7f141815e1e0 2023-11-24T06:10:38.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:38.118+0000 7f140dffb640 1 -- 172.21.15.92:0/395370596 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f1410040ce0 con 0x7f141815e1e0 2023-11-24T06:10:38.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:38.119+0000 7f140dffb640 1 --2- 172.21.15.92:0/395370596 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f13fc06ae40 0x7f13fc06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:38.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:38.120+0000 7f140dffb640 1 -- 172.21.15.92:0/395370596 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f14100ac680 con 0x7f141815e1e0 2023-11-24T06:10:38.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:38.120+0000 7f141d75a640 1 --2- 172.21.15.92:0/395370596 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f13fc06ae40 0x7f13fc06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:38.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:38.120+0000 7f141d75a640 1 --2- 172.21.15.92:0/395370596 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f13fc06ae40 0x7f13fc06d300 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f1414006fd0 tx=0x7f1414008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:38.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:38.124+0000 7f140dffb640 1 -- 172.21.15.92:0/395370596 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1410077730 con 0x7f141815e1e0 2023-11-24T06:10:38.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:38.313+0000 7f141f9be640 1 -- 172.21.15.92:0/395370596 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f14180675d0 con 0x7f141815e1e0 2023-11-24T06:10:38.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:38.314+0000 7f140dffb640 1 -- 172.21.15.92:0/395370596 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f1410009510 con 0x7f141815e1e0 2023-11-24T06:10:38.318 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 64 2023-11-24T06:10:38.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:38.317+0000 7f141f9be640 1 -- 172.21.15.92:0/395370596 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f13fc06ae40 msgr2=0x7f13fc06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:38.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:38.317+0000 7f141f9be640 1 --2- 172.21.15.92:0/395370596 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f13fc06ae40 0x7f13fc06d300 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f1414006fd0 tx=0x7f1414008040 comp rx=0 tx=0).stop 2023-11-24T06:10:38.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:38.317+0000 7f141f9be640 1 -- 172.21.15.92:0/395370596 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f141815e1e0 msgr2=0x7f141806c9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:38.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:38.317+0000 7f141f9be640 1 --2- 172.21.15.92:0/395370596 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f141815e1e0 0x7f141806c9b0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f1410036040 tx=0x7f1410004a40 comp rx=0 tx=0).stop 2023-11-24T06:10:38.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:38.317+0000 7f141f9be640 1 -- 172.21.15.92:0/395370596 shutdown_connections 2023-11-24T06:10:38.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:38.317+0000 7f141f9be640 1 --2- 172.21.15.92:0/395370596 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f13fc06ae40 0x7f13fc06d300 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:38.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:38.317+0000 7f141f9be640 1 --2- 172.21.15.92:0/395370596 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f141815e1e0 0x7f141806c9b0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:38.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:38.317+0000 7f141f9be640 1 --2- 172.21.15.92:0/395370596 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1418143ae0 0x7f141806c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:38.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:38.317+0000 7f141f9be640 1 --2- 172.21.15.92:0/395370596 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f14180cf7d0 0x7f14181f22d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:38.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:38.317+0000 7f141f9be640 1 -- 172.21.15.92:0/395370596 >> 172.21.15.92:0/395370596 conn(0x7f141806a800 msgr2=0x7f141806abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:38.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:38.317+0000 7f141f9be640 1 -- 172.21.15.92:0/395370596 shutdown_connections 2023-11-24T06:10:38.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:38.317+0000 7f141f9be640 1 -- 172.21.15.92:0/395370596 wait complete. 2023-11-24T06:10:38.339 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T06:10:38.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:38.546+0000 7ff9624ea640 1 Processor -- start 2023-11-24T06:10:38.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:38.546+0000 7ff9624ea640 1 -- start start 2023-11-24T06:10:38.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:38.547+0000 7ff9624ea640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff95c13cfa0 0x7ff95c15afb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:38.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:38.548+0000 7ff9624ea640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ff95c0cd530 0x7ff95c13f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:38.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:38.548+0000 7ff9624ea640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff95c0cdec0 con 0x7ff95c0cd9b0 2023-11-24T06:10:38.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:38.548+0000 7ff9624ea640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff95c0ce1a0 con 0x7ff95c0cd530 2023-11-24T06:10:38.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:38.548+0000 7ff9624ea640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff95c0ce480 con 0x7ff95c13cfa0 2023-11-24T06:10:38.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:38.548+0000 7ff960ce7640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ff95c0cd530 0x7ff95c13f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:38.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:38.548+0000 7ff961ce9640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7ff95c0cd9b0 0x7ff95c15d9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:55930/0 (socket says 172.21.15.92:55930) 2023-11-24T06:10:38.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:38.548+0000 7ff961ce9640 1 -- 172.21.15.92:0/3706917505 learned_addr learned my addr 172.21.15.92:0/3706917505 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-24T06:10:38.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:38.548+0000 7ff9614e8640 1 --2- 172.21.15.92:0/3706917505 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff95c13cfa0 0x7ff95c15afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:38.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:38.548+0000 7ff960ce7640 1 -- 172.21.15.92:0/3706917505 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff95c13cfa0 msgr2=0x7ff95c15afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:38.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:38.548+0000 7ff960ce7640 1 --2- 172.21.15.92:0/3706917505 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff95c13cfa0 0x7ff95c15afb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:38.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:38.548+0000 7ff960ce7640 1 -- 172.21.15.92:0/3706917505 >> v1:172.21.15.92:6789/0 conn(0x7ff95c0cd9b0 legacy=0x7ff95c15d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:38.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:38.548+0000 7ff953fff640 1 -- 172.21.15.92:0/3706917505 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4024475697 0 0) 0x7ff95c0cdec0 con 0x7ff95c0cd9b0 2023-11-24T06:10:38.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:38.548+0000 7ff960ce7640 1 -- 172.21.15.92:0/3706917505 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff95c0ce760 con 0x7ff95c0cd530 2023-11-24T06:10:38.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:38.550+0000 7ff960ce7640 1 --2- 172.21.15.92:0/3706917505 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff95c0cd530 0x7ff95c13f5a0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7ff94c00ba40 tx=0x7ff94c02dff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a4778552b3ce5d3 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:38.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:38.550+0000 7ff953fff640 1 -- 172.21.15.92:0/3706917505 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff94c02eda0 con 0x7ff95c0cd530 2023-11-24T06:10:38.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:38.550+0000 7ff953fff640 1 -- 172.21.15.92:0/3706917505 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff94c036440 con 0x7ff95c0cd530 2023-11-24T06:10:38.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:38.550+0000 7ff953fff640 1 -- 172.21.15.92:0/3706917505 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff94c0367d0 con 0x7ff95c0cd530 2023-11-24T06:10:38.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:38.550+0000 7ff9624ea640 1 -- 172.21.15.92:0/3706917505 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff95c0cd530 msgr2=0x7ff95c13f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:38.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:38.550+0000 7ff9624ea640 1 --2- 172.21.15.92:0/3706917505 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff95c0cd530 0x7ff95c13f5a0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7ff94c00ba40 tx=0x7ff94c02dff0 comp rx=0 tx=0).stop 2023-11-24T06:10:38.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:38.550+0000 7ff9624ea640 1 -- 172.21.15.92:0/3706917505 shutdown_connections 2023-11-24T06:10:38.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:38.550+0000 7ff9624ea640 1 --2- 172.21.15.92:0/3706917505 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff95c0cd530 0x7ff95c13f5a0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:38.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:38.550+0000 7ff9624ea640 1 --2- 172.21.15.92:0/3706917505 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff95c13cfa0 0x7ff95c15afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:38.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:38.550+0000 7ff9624ea640 1 -- 172.21.15.92:0/3706917505 >> 172.21.15.92:0/3706917505 conn(0x7ff95c06a800 msgr2=0x7ff95c06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:38.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:38.551+0000 7ff9624ea640 1 -- 172.21.15.92:0/3706917505 shutdown_connections 2023-11-24T06:10:38.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:38.551+0000 7ff9624ea640 1 -- 172.21.15.92:0/3706917505 wait complete. 2023-11-24T06:10:38.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:38.552+0000 7ff9624ea640 1 Processor -- start 2023-11-24T06:10:38.556 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:38.553+0000 7ff9624ea640 1 -- start start 2023-11-24T06:10:38.557 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:38.553+0000 7ff9624ea640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff95c0cd530 0x7ff95c14c020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:38.557 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:38.553+0000 7ff9624ea640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff95c0cd9b0 0x7ff95c1572d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:38.557 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:38.554+0000 7ff9624ea640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff95c13cfa0 0x7ff95c14d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:38.557 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:38.554+0000 7ff9624ea640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff95c14ecc0 con 0x7ff95c0cd9b0 2023-11-24T06:10:38.557 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:38.554+0000 7ff9624ea640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff95c157810 con 0x7ff95c13cfa0 2023-11-24T06:10:38.558 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:38.554+0000 7ff9624ea640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff95c14c600 con 0x7ff95c0cd530 2023-11-24T06:10:38.558 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:38.554+0000 7ff960ce7640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff95c0cd9b0 0x7ff95c1572d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:38.558 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:38.554+0000 7ff960ce7640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff95c0cd9b0 0x7ff95c1572d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47672/0 (socket says 172.21.15.92:47672) 2023-11-24T06:10:38.558 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:38.554+0000 7ff960ce7640 1 -- 172.21.15.92:0/3984353698 learned_addr learned my addr 172.21.15.92:0/3984353698 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:10:38.558 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:38.554+0000 7ff960ce7640 1 -- 172.21.15.92:0/3984353698 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff95c0cd530 msgr2=0x7ff95c14c020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:38.558 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:38.554+0000 7ff960ce7640 1 --2- 172.21.15.92:0/3984353698 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff95c0cd530 0x7ff95c14c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:38.558 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:38.554+0000 7ff960ce7640 1 -- 172.21.15.92:0/3984353698 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff95c13cfa0 msgr2=0x7ff95c14d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:38.558 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:38.554+0000 7ff960ce7640 1 --2- 172.21.15.92:0/3984353698 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff95c13cfa0 0x7ff95c14d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:38.558 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:38.554+0000 7ff960ce7640 1 -- 172.21.15.92:0/3984353698 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff94c00b6a0 con 0x7ff95c0cd9b0 2023-11-24T06:10:38.559 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:38.554+0000 7ff960ce7640 1 --2- 172.21.15.92:0/3984353698 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff95c0cd9b0 0x7ff95c1572d0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7ff94c000c00 tx=0x7ff94c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:38.559 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:38.555+0000 7ff951ffb640 1 -- 172.21.15.92:0/3984353698 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff94c03e7d0 con 0x7ff95c0cd9b0 2023-11-24T06:10:38.559 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:38.555+0000 7ff951ffb640 1 -- 172.21.15.92:0/3984353698 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff94c03e950 con 0x7ff95c0cd9b0 2023-11-24T06:10:38.559 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:38.555+0000 7ff951ffb640 1 -- 172.21.15.92:0/3984353698 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff94c03ed00 con 0x7ff95c0cd9b0 2023-11-24T06:10:38.559 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:38.555+0000 7ff9624ea640 1 -- 172.21.15.92:0/3984353698 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff95c14c8e0 con 0x7ff95c0cd9b0 2023-11-24T06:10:38.560 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:38.555+0000 7ff9624ea640 1 -- 172.21.15.92:0/3984353698 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff95c14cea0 con 0x7ff95c0cd9b0 2023-11-24T06:10:38.560 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:38.556+0000 7ff9624ea640 1 -- 172.21.15.92:0/3984353698 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff95c067450 con 0x7ff95c0cd9b0 2023-11-24T06:10:38.560 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:38.557+0000 7ff951ffb640 1 -- 172.21.15.92:0/3984353698 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ff94c03d070 con 0x7ff95c0cd9b0 2023-11-24T06:10:38.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:38.558+0000 7ff951ffb640 1 --2- 172.21.15.92:0/3984353698 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7ff94006ae40 0x7ff94006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:38.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:38.558+0000 7ff951ffb640 1 -- 172.21.15.92:0/3984353698 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff94c0ab500 con 0x7ff95c0cd9b0 2023-11-24T06:10:38.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:38.558+0000 7ff9614e8640 1 --2- 172.21.15.92:0/3984353698 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7ff94006ae40 0x7ff94006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:38.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:38.558+0000 7ff9614e8640 1 --2- 172.21.15.92:0/3984353698 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7ff94006ae40 0x7ff94006d300 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7ff9580070b0 tx=0x7ff958007040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:38.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:38.562+0000 7ff951ffb640 1 -- 172.21.15.92:0/3984353698 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff94c0765a0 con 0x7ff95c0cd9b0 2023-11-24T06:10:38.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:38.784+0000 7ff9624ea640 1 -- 172.21.15.92:0/3984353698 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff95c0cbff0 con 0x7ff95c0cd9b0 2023-11-24T06:10:38.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:38.785+0000 7ff951ffb640 1 -- 172.21.15.92:0/3984353698 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4910 (secure 0 0 0) 0x7ff94c079bc0 con 0x7ff95c0cd9b0 2023-11-24T06:10:38.789 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-24T06:10:38.789 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4236,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6836/437604699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":437604699},{"type":"v1","addr":"172.21.15.92:6836","nonce":437604699}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6835/3368711442","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":3368711442},{"type":"v1","addr":"172.21.15.118:6835","nonce":3368711442}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4255,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/2767290380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":2767290380},{"type":"v1","addr":"172.21.15.92:6837","nonce":2767290380}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T06:10:25.115612+0000","modified":"2023-11-24T06:10:28.210438+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4264},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4264":{"gid":4264,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.118:6833/1146052262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":1146052262},{"type":"v1","addr":"172.21.15.118:6833","nonce":1146052262}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-24T06:10:38.789 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 8 2023-11-24T06:10:38.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:38.787+0000 7ff9624ea640 1 -- 172.21.15.92:0/3984353698 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7ff94006ae40 msgr2=0x7ff94006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:38.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:38.787+0000 7ff9624ea640 1 --2- 172.21.15.92:0/3984353698 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7ff94006ae40 0x7ff94006d300 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7ff9580070b0 tx=0x7ff958007040 comp rx=0 tx=0).stop 2023-11-24T06:10:38.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:38.787+0000 7ff9624ea640 1 -- 172.21.15.92:0/3984353698 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff95c0cd9b0 msgr2=0x7ff95c1572d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:38.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:38.788+0000 7ff9624ea640 1 --2- 172.21.15.92:0/3984353698 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff95c0cd9b0 0x7ff95c1572d0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7ff94c000c00 tx=0x7ff94c0046c0 comp rx=0 tx=0).stop 2023-11-24T06:10:38.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:38.788+0000 7ff9624ea640 1 -- 172.21.15.92:0/3984353698 shutdown_connections 2023-11-24T06:10:38.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:38.788+0000 7ff9624ea640 1 --2- 172.21.15.92:0/3984353698 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7ff94006ae40 0x7ff94006d300 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:38.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:38.788+0000 7ff9624ea640 1 --2- 172.21.15.92:0/3984353698 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff95c13cfa0 0x7ff95c14d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:38.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:38.788+0000 7ff9624ea640 1 --2- 172.21.15.92:0/3984353698 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff95c0cd9b0 0x7ff95c1572d0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:38.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:38.788+0000 7ff9624ea640 1 --2- 172.21.15.92:0/3984353698 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff95c0cd530 0x7ff95c14c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:38.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:38.788+0000 7ff9624ea640 1 -- 172.21.15.92:0/3984353698 >> 172.21.15.92:0/3984353698 conn(0x7ff95c06a800 msgr2=0x7ff95c159bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:38.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:38.788+0000 7ff9624ea640 1 -- 172.21.15.92:0/3984353698 shutdown_connections 2023-11-24T06:10:38.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:38.788+0000 7ff9624ea640 1 -- 172.21.15.92:0/3984353698 wait complete. 2023-11-24T06:10:38.809 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:10:39.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.013+0000 7f7ca95db640 1 Processor -- start 2023-11-24T06:10:39.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.014+0000 7f7ca95db640 1 -- start start 2023-11-24T06:10:39.023 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.015+0000 7f7ca95db640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f7ca4143c40 0x7f7ca4071290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:39.023 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.015+0000 7f7ca95db640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7ca40ccc10 0x7f7ca406d160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:39.023 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.015+0000 7f7ca95db640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ca4070e50 con 0x7f7ca40717d0 2023-11-24T06:10:39.023 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.015+0000 7f7ca95db640 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ca40723c0 con 0x7f7ca4143c40 2023-11-24T06:10:39.024 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.015+0000 7f7ca95db640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ca4144020 con 0x7f7ca40ccc10 2023-11-24T06:10:39.024 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.016+0000 7f7c9bfff640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f7ca40717d0 0x7f7ca40cc500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:52600/0 (socket says 172.21.15.92:52600) 2023-11-24T06:10:39.024 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.016+0000 7f7c9bfff640 1 -- 172.21.15.92:0/2680763843 learned_addr learned my addr 172.21.15.92:0/2680763843 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-24T06:10:39.024 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.016+0000 7f7ca3fff640 1 --2- 172.21.15.92:0/2680763843 >> v2:172.21.15.118:3300/0 conn(0x7f7ca4143c40 0x7f7ca4071290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:39.024 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.016+0000 7f7ca8dda640 1 --2- 172.21.15.92:0/2680763843 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7ca40ccc10 0x7f7ca406d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:39.024 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.016+0000 7f7ca37fe640 1 -- 172.21.15.92:0/2680763843 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2956243468 0 0) 0x7f7ca4070e50 con 0x7f7ca40717d0 2023-11-24T06:10:39.025 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.016+0000 7f7ca37fe640 1 -- 172.21.15.92:0/2680763843 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7c90003520 con 0x7f7ca40717d0 2023-11-24T06:10:39.025 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.016+0000 7f7ca37fe640 1 -- 172.21.15.92:0/2680763843 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1465450146 0 0) 0x7f7c90003520 con 0x7f7ca40717d0 2023-11-24T06:10:39.025 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.016+0000 7f7ca37fe640 1 -- 172.21.15.92:0/2680763843 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7ca40ccc10 msgr2=0x7f7ca406d160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:39.025 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.016+0000 7f7ca37fe640 1 --2- 172.21.15.92:0/2680763843 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7ca40ccc10 0x7f7ca406d160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:39.025 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.016+0000 7f7ca37fe640 1 -- 172.21.15.92:0/2680763843 >> v2:172.21.15.118:3300/0 conn(0x7f7ca4143c40 msgr2=0x7f7ca4071290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:39.025 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.016+0000 7f7ca37fe640 1 --2- 172.21.15.92:0/2680763843 >> v2:172.21.15.118:3300/0 conn(0x7f7ca4143c40 0x7f7ca4071290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:39.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.016+0000 7f7ca37fe640 1 -- 172.21.15.92:0/2680763843 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ca4071e00 con 0x7f7ca40717d0 2023-11-24T06:10:39.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.016+0000 7f7ca37fe640 1 -- 172.21.15.92:0/2680763843 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2607661350 0 0) 0x7f7c8c001670 con 0x7f7ca40717d0 2023-11-24T06:10:39.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.016+0000 7f7ca37fe640 1 -- 172.21.15.92:0/2680763843 >> v1:172.21.15.92:6789/0 conn(0x7f7ca40717d0 legacy=0x7f7ca40cc500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:39.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.016+0000 7f7ca37fe640 1 --2- 172.21.15.92:0/2680763843 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7c90003970 0x7f7c90003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:39.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.016+0000 7f7ca37fe640 1 --2- 172.21.15.92:0/2680763843 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7ca40ccc10 0x7f7c900047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:39.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.016+0000 7f7ca37fe640 1 --2- 172.21.15.92:0/2680763843 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7c90004d00 0x7f7c90005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:39.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.016+0000 7f7ca37fe640 1 -- 172.21.15.92:0/2680763843 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ca40723c0 con 0x7f7c90003970 2023-11-24T06:10:39.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.016+0000 7f7ca37fe640 1 -- 172.21.15.92:0/2680763843 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ca4144020 con 0x7f7c90004d00 2023-11-24T06:10:39.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.016+0000 7f7ca37fe640 1 -- 172.21.15.92:0/2680763843 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c9000b9b0 con 0x7f7ca40ccc10 2023-11-24T06:10:39.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.017+0000 7f7ca3fff640 1 --2- 172.21.15.92:0/2680763843 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7c90003970 0x7f7c90003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:39.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.017+0000 7f7ca3fff640 1 -- 172.21.15.92:0/2680763843 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7ca40ccc10 msgr2=0x7f7c900047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:39.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.017+0000 7f7ca3fff640 1 --2- 172.21.15.92:0/2680763843 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7ca40ccc10 0x7f7c900047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:39.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.017+0000 7f7ca3fff640 1 -- 172.21.15.92:0/2680763843 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7c90004d00 msgr2=0x7f7c90005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:39.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.017+0000 7f7ca3fff640 1 --2- 172.21.15.92:0/2680763843 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7c90004d00 0x7f7c90005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:39.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.017+0000 7f7ca3fff640 1 -- 172.21.15.92:0/2680763843 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c90003520 con 0x7f7c90003970 2023-11-24T06:10:39.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.018+0000 7f7ca3fff640 1 --2- 172.21.15.92:0/2680763843 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7c90003970 0x7f7c90003d90 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f7c94009ac0 tx=0x7f7c900057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea4619d79780813c server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:39.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.018+0000 7f7ca37fe640 1 -- 172.21.15.92:0/2680763843 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c9402c710 con 0x7f7c90003970 2023-11-24T06:10:39.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.018+0000 7f7ca37fe640 1 -- 172.21.15.92:0/2680763843 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c9402c870 con 0x7f7c90003970 2023-11-24T06:10:39.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.018+0000 7f7ca37fe640 1 -- 172.21.15.92:0/2680763843 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c9402cc00 con 0x7f7c90003970 2023-11-24T06:10:39.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.019+0000 7f7ca95db640 1 -- 172.21.15.92:0/2680763843 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7c90003970 msgr2=0x7f7c90003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:39.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.019+0000 7f7ca95db640 1 --2- 172.21.15.92:0/2680763843 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7c90003970 0x7f7c90003d90 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f7c94009ac0 tx=0x7f7c900057f0 comp rx=0 tx=0).stop 2023-11-24T06:10:39.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.019+0000 7f7ca8dda640 1 -- 172.21.15.92:0/2680763843 reap_dead start 2023-11-24T06:10:39.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.019+0000 7f7ca95db640 1 -- 172.21.15.92:0/2680763843 shutdown_connections 2023-11-24T06:10:39.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.019+0000 7f7ca95db640 1 -- 172.21.15.92:0/2680763843 >> 172.21.15.92:0/2680763843 conn(0x7f7ca406a800 msgr2=0x7f7ca406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:39.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.019+0000 7f7ca95db640 1 -- 172.21.15.92:0/2680763843 shutdown_connections 2023-11-24T06:10:39.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.019+0000 7f7ca95db640 1 -- 172.21.15.92:0/2680763843 wait complete. 2023-11-24T06:10:39.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.021+0000 7f7ca95db640 1 Processor -- start 2023-11-24T06:10:39.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.021+0000 7f7ca95db640 1 -- start start 2023-11-24T06:10:39.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.022+0000 7f7ca95db640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7c90003970 0x7f7ca415a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:39.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.022+0000 7f7ca95db640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7ca40a84c0 0x7f7ca40a8940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:39.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.022+0000 7f7ca95db640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7ca406d060 0x7f7ca406d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:39.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.022+0000 7f7ca95db640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ca406da20 con 0x7f7ca406d060 2023-11-24T06:10:39.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.022+0000 7f7ca95db640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ca415aec0 con 0x7f7ca40a84c0 2023-11-24T06:10:39.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.022+0000 7f7ca95db640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ca40a9030 con 0x7f7c90003970 2023-11-24T06:10:39.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.022+0000 7f7ca8dda640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7ca406d060 0x7f7ca406d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:39.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.022+0000 7f7ca8dda640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7ca406d060 0x7f7ca406d4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46946/0 (socket says 172.21.15.92:46946) 2023-11-24T06:10:39.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.022+0000 7f7ca8dda640 1 -- 172.21.15.92:0/1734307428 learned_addr learned my addr 172.21.15.92:0/1734307428 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:10:39.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.022+0000 7f7ca8dda640 1 -- 172.21.15.92:0/1734307428 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7c90003970 msgr2=0x7f7ca415a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:39.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.022+0000 7f7ca8dda640 1 --2- 172.21.15.92:0/1734307428 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7c90003970 0x7f7ca415a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:39.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.022+0000 7f7c9bfff640 1 --2- 172.21.15.92:0/1734307428 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7ca40a84c0 0x7f7ca40a8940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:39.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.022+0000 7f7ca8dda640 1 -- 172.21.15.92:0/1734307428 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7ca40a84c0 msgr2=0x7f7ca40a8940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:39.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.022+0000 7f7ca8dda640 1 --2- 172.21.15.92:0/1734307428 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7ca40a84c0 0x7f7ca40a8940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:39.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.022+0000 7f7ca8dda640 1 -- 172.21.15.92:0/1734307428 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ca40a9310 con 0x7f7ca406d060 2023-11-24T06:10:39.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.023+0000 7f7ca8dda640 1 --2- 172.21.15.92:0/1734307428 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7ca406d060 0x7f7ca406d4e0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f7ca4144ca0 tx=0x7f7c9c00b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:39.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.023+0000 7f7ca17fa640 1 -- 172.21.15.92:0/1734307428 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c9c010040 con 0x7f7ca406d060 2023-11-24T06:10:39.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.023+0000 7f7ca17fa640 1 -- 172.21.15.92:0/1734307428 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c9c00ba80 con 0x7f7ca406d060 2023-11-24T06:10:39.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.023+0000 7f7ca95db640 1 -- 172.21.15.92:0/1734307428 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ca40a95f0 con 0x7f7ca406d060 2023-11-24T06:10:39.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.023+0000 7f7ca17fa640 1 -- 172.21.15.92:0/1734307428 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c9c00be50 con 0x7f7ca406d060 2023-11-24T06:10:39.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.023+0000 7f7ca95db640 1 -- 172.21.15.92:0/1734307428 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ca413cda0 con 0x7f7ca406d060 2023-11-24T06:10:39.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.024+0000 7f7ca95db640 1 -- 172.21.15.92:0/1734307428 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ca4067450 con 0x7f7ca406d060 2023-11-24T06:10:39.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.025+0000 7f7ca17fa640 1 -- 172.21.15.92:0/1734307428 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f7c9c01e410 con 0x7f7ca406d060 2023-11-24T06:10:39.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.026+0000 7f7ca17fa640 1 --2- 172.21.15.92:0/1734307428 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f7c8406ae70 0x7f7c8406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:39.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.026+0000 7f7ca17fa640 1 -- 172.21.15.92:0/1734307428 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7c9c089920 con 0x7f7ca406d060 2023-11-24T06:10:39.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.026+0000 7f7ca3fff640 1 --2- 172.21.15.92:0/1734307428 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f7c8406ae70 0x7f7c8406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:39.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.027+0000 7f7ca3fff640 1 --2- 172.21.15.92:0/1734307428 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f7c8406ae70 0x7f7c8406d330 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f7c94036000 tx=0x7f7c9402e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:39.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.030+0000 7f7ca17fa640 1 -- 172.21.15.92:0/1734307428 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7c9c0583c0 con 0x7f7ca406d060 2023-11-24T06:10:39.220 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.216+0000 7f7ca95db640 1 -- 172.21.15.92:0/1734307428 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7ca40ceb30 con 0x7f7ca406d060 2023-11-24T06:10:39.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.218+0000 7f7ca17fa640 1 -- 172.21.15.92:0/1734307428 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16457 (secure 0 0 0) 0x7f7c9c057f50 con 0x7f7ca406d060 2023-11-24T06:10:39.222 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-24T06:10:39.222 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":20,"fsid":"2d6dee5f-8aa3-470f-931a-50ade963b1cb","created":"2023-11-24T06:10:04.707204+0000","modified":"2023-11-24T06:10:26.698733+0000","last_up_change":"2023-11-24T06:10:17.598352+0000","last_in_change":"2023-11-24T06:10:09.040793+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:10:17.733098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:10:22.926445+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T06:10:24.151288+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ff37a9fb-135f-471f-b4b0-f7b5772006c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6803","nonce":20560714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6805","nonce":20560714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6809","nonce":20560714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6807","nonce":20560714}]},"public_addr":"172.21.15.92:6803/20560714","cluster_addr":"172.21.15.92:6805/20560714","heartbeat_back_addr":"172.21.15.92:6809/20560714","heartbeat_front_addr":"172.21.15.92:6807/20560714","state":["exists","up"]},{"osd":1,"uuid":"03cdd353-dcf1-4f38-94fe-a795137c06b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6819","nonce":2451239906}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6821","nonce":2451239906}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6825","nonce":2451239906}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6823","nonce":2451239906}]},"public_addr":"172.21.15.92:6819/2451239906","cluster_addr":"172.21.15.92:6821/2451239906","heartbeat_back_addr":"172.21.15.92:6825/2451239906","heartbeat_front_addr":"172.21.15.92:6823/2451239906","state":["exists","up"]},{"osd":2,"uuid":"a4a20d80-5d57-484b-becd-9aa8070dd3aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6811","nonce":1124979277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6813","nonce":1124979277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6817","nonce":1124979277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6815","nonce":1124979277}]},"public_addr":"172.21.15.92:6811/1124979277","cluster_addr":"172.21.15.92:6813/1124979277","heartbeat_back_addr":"172.21.15.92:6817/1124979277","heartbeat_front_addr":"172.21.15.92:6815/1124979277","state":["exists","up"]},{"osd":3,"uuid":"4648d7ce-9b29-4583-9425-4ba8fd2b1903","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6827","nonce":1734115782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6829","nonce":1734115782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6833","nonce":1734115782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6831","nonce":1734115782}]},"public_addr":"172.21.15.92:6827/1734115782","cluster_addr":"172.21.15.92:6829/1734115782","heartbeat_back_addr":"172.21.15.92:6833/1734115782","heartbeat_front_addr":"172.21.15.92:6831/1734115782","state":["exists","up"]},{"osd":4,"uuid":"3bc9d7eb-c6fa-4fcf-a3ec-ade604e05aac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6819","nonce":3425937229}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6822","nonce":3425937229}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6830","nonce":3425937229}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6826","nonce":3425937229}]},"public_addr":"172.21.15.118:6819/3425937229","cluster_addr":"172.21.15.118:6822/3425937229","heartbeat_back_addr":"172.21.15.118:6830/3425937229","heartbeat_front_addr":"172.21.15.118:6826/3425937229","state":["exists","up"]},{"osd":5,"uuid":"dce9ba6d-59ed-46d1-84fc-a84b13de10c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6801","nonce":827455984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6803","nonce":827455984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6807","nonce":827455984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6805","nonce":827455984}]},"public_addr":"172.21.15.118:6801/827455984","cluster_addr":"172.21.15.118:6803/827455984","heartbeat_back_addr":"172.21.15.118:6807/827455984","heartbeat_front_addr":"172.21.15.118:6805/827455984","state":["exists","up"]},{"osd":6,"uuid":"82aec504-ae58-4709-8d30-46f2b12081b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6809","nonce":2946986141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6811","nonce":2946986141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6815","nonce":2946986141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6813","nonce":2946986141}]},"public_addr":"172.21.15.118:6809/2946986141","cluster_addr":"172.21.15.118:6811/2946986141","heartbeat_back_addr":"172.21.15.118:6815/2946986141","heartbeat_front_addr":"172.21.15.118:6813/2946986141","state":["exists","up"]},{"osd":7,"uuid":"4ed4c624-c88f-4d35-bdb1-bf209b5dffb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6817","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6818","nonce":48122059}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6821","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6823","nonce":48122059}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6829","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6831","nonce":48122059}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6825","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6827","nonce":48122059}]},"public_addr":"172.21.15.118:6818/48122059","cluster_addr":"172.21.15.118:6823/48122059","heartbeat_back_addr":"172.21.15.118:6831/48122059","heartbeat_front_addr":"172.21.15.118:6827/48122059","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.536664+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.368324+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.271201+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.342971+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.621832+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.387110+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.632946+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.416032+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:10:39.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.221+0000 7f7ca95db640 1 -- 172.21.15.92:0/1734307428 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f7c8406ae70 msgr2=0x7f7c8406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:39.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.221+0000 7f7ca95db640 1 --2- 172.21.15.92:0/1734307428 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f7c8406ae70 0x7f7c8406d330 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f7c94036000 tx=0x7f7c9402e000 comp rx=0 tx=0).stop 2023-11-24T06:10:39.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.221+0000 7f7ca95db640 1 -- 172.21.15.92:0/1734307428 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7ca406d060 msgr2=0x7f7ca406d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:39.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.221+0000 7f7ca95db640 1 --2- 172.21.15.92:0/1734307428 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7ca406d060 0x7f7ca406d4e0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f7ca4144ca0 tx=0x7f7c9c00b480 comp rx=0 tx=0).stop 2023-11-24T06:10:39.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.221+0000 7f7ca95db640 1 -- 172.21.15.92:0/1734307428 shutdown_connections 2023-11-24T06:10:39.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.221+0000 7f7ca95db640 1 --2- 172.21.15.92:0/1734307428 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f7c8406ae70 0x7f7c8406d330 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:39.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.221+0000 7f7ca95db640 1 --2- 172.21.15.92:0/1734307428 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7ca406d060 0x7f7ca406d4e0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:39.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.221+0000 7f7ca95db640 1 --2- 172.21.15.92:0/1734307428 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7ca40a84c0 0x7f7ca40a8940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:39.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.221+0000 7f7ca95db640 1 --2- 172.21.15.92:0/1734307428 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7c90003970 0x7f7ca415a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:39.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.221+0000 7f7ca95db640 1 -- 172.21.15.92:0/1734307428 >> 172.21.15.92:0/1734307428 conn(0x7f7ca406a800 msgr2=0x7f7ca40cc500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:39.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.221+0000 7f7ca95db640 1 -- 172.21.15.92:0/1734307428 shutdown_connections 2023-11-24T06:10:39.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.221+0000 7f7ca95db640 1 -- 172.21.15.92:0/1734307428 wait complete. 2023-11-24T06:10:39.244 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T06:10:17.733098+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T06:10:22.926445+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T06:10:24.151288+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T06:10:39.244 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T06:10:39.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.450+0000 7fd0f8090640 1 Processor -- start 2023-11-24T06:10:39.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.450+0000 7fd0f8090640 1 -- start start 2023-11-24T06:10:39.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.452+0000 7fd0f8090640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fd0f013e360 0x7fd0f00cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:39.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.452+0000 7fd0f8090640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0f015c5f0 con 0x7fd0f015f9c0 2023-11-24T06:10:39.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.452+0000 7fd0f8090640 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0f006c470 con 0x7fd0f013df80 2023-11-24T06:10:39.456 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.452+0000 7fd0f8090640 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0f006c750 con 0x7fd0f013e360 2023-11-24T06:10:39.456 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.452+0000 7fd0f662d640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fd0f015f9c0 0x7fd0f00cd840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:52604/0 (socket says 172.21.15.92:52604) 2023-11-24T06:10:39.456 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.452+0000 7fd0f662d640 1 -- 172.21.15.92:0/2087922598 learned_addr learned my addr 172.21.15.92:0/2087922598 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-24T06:10:39.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.453+0000 7fd0f562b640 1 --2- 172.21.15.92:0/2087922598 >> v2:172.21.15.118:3300/0 conn(0x7fd0f013e360 0x7fd0f00cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:39.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.453+0000 7fd0f4e2a640 1 -- 172.21.15.92:0/2087922598 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3161293614 0 0) 0x7fd0f015c5f0 con 0x7fd0f015f9c0 2023-11-24T06:10:39.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.453+0000 7fd0f4e2a640 1 -- 172.21.15.92:0/2087922598 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd0dc003520 con 0x7fd0f015f9c0 2023-11-24T06:10:39.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.453+0000 7fd0f4e2a640 1 -- 172.21.15.92:0/2087922598 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 447852357 0 0) 0x7fd0dc003520 con 0x7fd0f015f9c0 2023-11-24T06:10:39.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.453+0000 7fd0f4e2a640 1 -- 172.21.15.92:0/2087922598 >> v2:172.21.15.118:3300/0 conn(0x7fd0f013e360 msgr2=0x7fd0f00cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:39.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.453+0000 7fd0f4e2a640 1 --2- 172.21.15.92:0/2087922598 >> v2:172.21.15.118:3300/0 conn(0x7fd0f013e360 0x7fd0f00cf290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:39.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.453+0000 7fd0f4e2a640 1 -- 172.21.15.92:0/2087922598 >> v1:172.21.15.118:6789/0 conn(0x7fd0f013df80 legacy=0x7fd0f015bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:39.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.453+0000 7fd0f4e2a640 1 -- 172.21.15.92:0/2087922598 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0f006ca30 con 0x7fd0f015f9c0 2023-11-24T06:10:39.459 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.453+0000 7fd0f4e2a640 1 -- 172.21.15.92:0/2087922598 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2607661350 0 0) 0x7fd0e8001690 con 0x7fd0f015f9c0 2023-11-24T06:10:39.459 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.453+0000 7fd0f4e2a640 1 -- 172.21.15.92:0/2087922598 >> v1:172.21.15.92:6789/0 conn(0x7fd0f015f9c0 legacy=0x7fd0f00cd840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:39.459 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.453+0000 7fd0f4e2a640 1 --2- 172.21.15.92:0/2087922598 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd0dc0039c0 0x7fd0dc003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:39.459 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.453+0000 7fd0f4e2a640 1 --2- 172.21.15.92:0/2087922598 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd0dc0047c0 0x7fd0dc004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:39.459 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.453+0000 7fd0f4e2a640 1 --2- 172.21.15.92:0/2087922598 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd0dc005180 0x7fd0dc00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:39.459 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.453+0000 7fd0f4e2a640 1 -- 172.21.15.92:0/2087922598 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0f006c750 con 0x7fd0dc005180 2023-11-24T06:10:39.459 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.453+0000 7fd0f4e2a640 1 -- 172.21.15.92:0/2087922598 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0dc00c080 con 0x7fd0dc0047c0 2023-11-24T06:10:39.459 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.453+0000 7fd0f4e2a640 1 -- 172.21.15.92:0/2087922598 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0dc00c240 con 0x7fd0dc0039c0 2023-11-24T06:10:39.459 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.454+0000 7fd0f662d640 1 --2- 172.21.15.92:0/2087922598 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd0dc005180 0x7fd0dc00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:39.459 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.454+0000 7fd0f562b640 1 --2- 172.21.15.92:0/2087922598 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd0dc0047c0 0x7fd0dc004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:39.459 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.454+0000 7fd0f5e2c640 1 --2- 172.21.15.92:0/2087922598 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd0dc0039c0 0x7fd0dc003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:39.459 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.454+0000 7fd0f662d640 1 -- 172.21.15.92:0/2087922598 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd0dc0039c0 msgr2=0x7fd0dc003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:39.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.454+0000 7fd0f662d640 1 --2- 172.21.15.92:0/2087922598 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd0dc0039c0 0x7fd0dc003de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:39.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.454+0000 7fd0f662d640 1 -- 172.21.15.92:0/2087922598 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd0dc0047c0 msgr2=0x7fd0dc004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:39.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.454+0000 7fd0f662d640 1 --2- 172.21.15.92:0/2087922598 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd0dc0047c0 0x7fd0dc004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:39.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.454+0000 7fd0f662d640 1 -- 172.21.15.92:0/2087922598 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0dc003520 con 0x7fd0dc005180 2023-11-24T06:10:39.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.454+0000 7fd0f562b640 1 --2- 172.21.15.92:0/2087922598 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd0dc0047c0 0x7fd0dc004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:10:39.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.455+0000 7fd0f662d640 1 -- 172.21.15.92:0/2087922598 reap_dead start 2023-11-24T06:10:39.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.455+0000 7fd0f662d640 1 --2- 172.21.15.92:0/2087922598 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd0dc005180 0x7fd0dc00b980 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fd0f015c780 tx=0x7fd0dc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89deecdceda7a64 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:39.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.455+0000 7fd0f4e2a640 1 -- 172.21.15.92:0/2087922598 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0e8005ad0 con 0x7fd0dc005180 2023-11-24T06:10:39.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.455+0000 7fd0f4e2a640 1 -- 172.21.15.92:0/2087922598 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0e8005c50 con 0x7fd0dc005180 2023-11-24T06:10:39.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.455+0000 7fd0f4e2a640 1 -- 172.21.15.92:0/2087922598 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0e802c790 con 0x7fd0dc005180 2023-11-24T06:10:39.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.456+0000 7fd0f8090640 1 -- 172.21.15.92:0/2087922598 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd0dc005180 msgr2=0x7fd0dc00b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:39.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.456+0000 7fd0f8090640 1 --2- 172.21.15.92:0/2087922598 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd0dc005180 0x7fd0dc00b980 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fd0f015c780 tx=0x7fd0dc0057f0 comp rx=0 tx=0).stop 2023-11-24T06:10:39.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.456+0000 7fd0f8090640 1 -- 172.21.15.92:0/2087922598 shutdown_connections 2023-11-24T06:10:39.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.456+0000 7fd0f8090640 1 --2- 172.21.15.92:0/2087922598 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd0dc005180 0x7fd0dc00b980 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:39.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.456+0000 7fd0f8090640 1 -- 172.21.15.92:0/2087922598 >> 172.21.15.92:0/2087922598 conn(0x7fd0f006a800 msgr2=0x7fd0f006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:39.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.456+0000 7fd0f8090640 1 -- 172.21.15.92:0/2087922598 shutdown_connections 2023-11-24T06:10:39.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.456+0000 7fd0f8090640 1 -- 172.21.15.92:0/2087922598 wait complete. 2023-11-24T06:10:39.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.458+0000 7fd0f8090640 1 Processor -- start 2023-11-24T06:10:39.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.458+0000 7fd0f8090640 1 -- start start 2023-11-24T06:10:39.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.459+0000 7fd0f8090640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd0dc005180 0x7fd0f01581d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:39.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.459+0000 7fd0f8090640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd0f00ab870 0x7fd0f0159180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:39.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.459+0000 7fd0f8090640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd0f01f03c0 0x7fd0f015d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:39.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.459+0000 7fd0f8090640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0f00a9000 con 0x7fd0dc005180 2023-11-24T06:10:39.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.459+0000 7fd0f8090640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0f01596c0 con 0x7fd0f01f03c0 2023-11-24T06:10:39.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.459+0000 7fd0f8090640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0f00a8660 con 0x7fd0f00ab870 2023-11-24T06:10:39.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.459+0000 7fd0f5e2c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd0dc005180 0x7fd0f01581d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:39.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.459+0000 7fd0f5e2c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd0dc005180 0x7fd0f01581d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46954/0 (socket says 172.21.15.92:46954) 2023-11-24T06:10:39.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.459+0000 7fd0f5e2c640 1 -- 172.21.15.92:0/1817735192 learned_addr learned my addr 172.21.15.92:0/1817735192 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:10:39.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.460+0000 7fd0f5e2c640 1 -- 172.21.15.92:0/1817735192 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd0f00ab870 msgr2=0x7fd0f0159180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:39.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.460+0000 7fd0f5e2c640 1 --2- 172.21.15.92:0/1817735192 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd0f00ab870 0x7fd0f0159180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:39.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.460+0000 7fd0f662d640 1 --2- 172.21.15.92:0/1817735192 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd0f01f03c0 0x7fd0f015d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:39.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.460+0000 7fd0f5e2c640 1 -- 172.21.15.92:0/1817735192 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd0f01f03c0 msgr2=0x7fd0f015d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:39.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.460+0000 7fd0f5e2c640 1 --2- 172.21.15.92:0/1817735192 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd0f01f03c0 0x7fd0f015d140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:39.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.460+0000 7fd0f5e2c640 1 -- 172.21.15.92:0/1817735192 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0f00a8940 con 0x7fd0dc005180 2023-11-24T06:10:39.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.460+0000 7fd0f5e2c640 1 --2- 172.21.15.92:0/1817735192 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd0dc005180 0x7fd0f01581d0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fd0ec00d900 tx=0x7fd0ec00ddc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:39.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.460+0000 7fd0e67fc640 1 -- 172.21.15.92:0/1817735192 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0ec003da0 con 0x7fd0dc005180 2023-11-24T06:10:39.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.460+0000 7fd0e67fc640 1 -- 172.21.15.92:0/1817735192 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0ec00b770 con 0x7fd0dc005180 2023-11-24T06:10:39.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.460+0000 7fd0e67fc640 1 -- 172.21.15.92:0/1817735192 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0ec00bae0 con 0x7fd0dc005180 2023-11-24T06:10:39.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.460+0000 7fd0f8090640 1 -- 172.21.15.92:0/1817735192 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0f00a8c20 con 0x7fd0dc005180 2023-11-24T06:10:39.466 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.460+0000 7fd0f8090640 1 -- 172.21.15.92:0/1817735192 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0f015d680 con 0x7fd0dc005180 2023-11-24T06:10:39.466 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.462+0000 7fd0f8090640 1 -- 172.21.15.92:0/1817735192 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0f00675d0 con 0x7fd0dc005180 2023-11-24T06:10:39.466 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.462+0000 7fd0e67fc640 1 -- 172.21.15.92:0/1817735192 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd0ec00bc40 con 0x7fd0dc005180 2023-11-24T06:10:39.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.464+0000 7fd0e67fc640 1 --2- 172.21.15.92:0/1817735192 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fd0d406ae70 0x7fd0d406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:39.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.464+0000 7fd0e67fc640 1 -- 172.21.15.92:0/1817735192 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd0ec0897f0 con 0x7fd0dc005180 2023-11-24T06:10:39.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.464+0000 7fd0f562b640 1 --2- 172.21.15.92:0/1817735192 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fd0d406ae70 0x7fd0d406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:39.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.464+0000 7fd0f562b640 1 --2- 172.21.15.92:0/1817735192 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fd0d406ae70 0x7fd0d406d330 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fd0f013df50 tx=0x7fd0e0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:39.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.468+0000 7fd0e67fc640 1 -- 172.21.15.92:0/1817735192 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd0ec054930 con 0x7fd0dc005180 2023-11-24T06:10:39.646 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.643+0000 7fd0f8090640 1 -- 172.21.15.92:0/1817735192 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd0f01598f0 con 0x7fd0dc005180 2023-11-24T06:10:39.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.643+0000 7fd0e67fc640 1 -- 172.21.15.92:0/1817735192 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd0ec057f50 con 0x7fd0dc005180 2023-11-24T06:10:39.648 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 1 2023-11-24T06:10:39.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.646+0000 7fd0f8090640 1 -- 172.21.15.92:0/1817735192 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fd0d406ae70 msgr2=0x7fd0d406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:39.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.646+0000 7fd0f8090640 1 --2- 172.21.15.92:0/1817735192 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fd0d406ae70 0x7fd0d406d330 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fd0f013df50 tx=0x7fd0e0008040 comp rx=0 tx=0).stop 2023-11-24T06:10:39.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.646+0000 7fd0f8090640 1 -- 172.21.15.92:0/1817735192 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd0dc005180 msgr2=0x7fd0f01581d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:39.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.646+0000 7fd0f8090640 1 --2- 172.21.15.92:0/1817735192 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd0dc005180 0x7fd0f01581d0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fd0ec00d900 tx=0x7fd0ec00ddc0 comp rx=0 tx=0).stop 2023-11-24T06:10:39.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.646+0000 7fd0f8090640 1 -- 172.21.15.92:0/1817735192 shutdown_connections 2023-11-24T06:10:39.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.646+0000 7fd0f8090640 1 --2- 172.21.15.92:0/1817735192 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fd0d406ae70 0x7fd0d406d330 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:39.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.646+0000 7fd0f8090640 1 --2- 172.21.15.92:0/1817735192 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd0f01f03c0 0x7fd0f015d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:39.652 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.646+0000 7fd0f8090640 1 --2- 172.21.15.92:0/1817735192 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd0f00ab870 0x7fd0f0159180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:39.652 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.646+0000 7fd0f8090640 1 --2- 172.21.15.92:0/1817735192 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd0dc005180 0x7fd0f01581d0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:39.652 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.646+0000 7fd0f8090640 1 -- 172.21.15.92:0/1817735192 >> 172.21.15.92:0/1817735192 conn(0x7fd0f006a800 msgr2=0x7fd0f0154d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:39.652 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.646+0000 7fd0f8090640 1 -- 172.21.15.92:0/1817735192 shutdown_connections 2023-11-24T06:10:39.652 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.646+0000 7fd0f8090640 1 -- 172.21.15.92:0/1817735192 wait complete. 2023-11-24T06:10:39.668 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T06:10:39.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.873+0000 7f79a2c00640 1 Processor -- start 2023-11-24T06:10:39.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.873+0000 7f79a2c00640 1 -- start start 2023-11-24T06:10:39.878 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.875+0000 7f79a2c00640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f799c15e880 0x7f799c06ff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:39.879 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.875+0000 7f79a2c00640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f799c09a170 0x7f799c06e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:39.879 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.875+0000 7f79a2c00640 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f799c0cfa70 con 0x7f799c070490 2023-11-24T06:10:39.879 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.875+0000 7f79a2c00640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f799c0cff00 con 0x7f799c09a170 2023-11-24T06:10:39.879 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.875+0000 7f79a2c00640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f799c06fc70 con 0x7f799c15e880 2023-11-24T06:10:39.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.875+0000 7f79a119d640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f799c09a170 0x7f799c06e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:39.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.875+0000 7f79a119d640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f799c09a170 0x7f799c06e7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46970/0 (socket says 172.21.15.92:46970) 2023-11-24T06:10:39.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.875+0000 7f79a119d640 1 -- 172.21.15.92:0/2142645989 learned_addr learned my addr 172.21.15.92:0/2142645989 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:10:39.885 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.876+0000 7f79a099c640 1 --2- 172.21.15.92:0/2142645989 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f799c15e880 0x7f799c06ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:39.885 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.876+0000 7f79a119d640 1 -- 172.21.15.92:0/2142645989 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f799c15e880 msgr2=0x7f799c06ff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:39.885 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.876+0000 7f79a119d640 1 --2- 172.21.15.92:0/2142645989 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f799c15e880 0x7f799c06ff50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:39.885 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.876+0000 7f79a119d640 1 -- 172.21.15.92:0/2142645989 >> v1:172.21.15.118:6789/0 conn(0x7f799c070490 legacy=0x7f799c099a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:39.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.876+0000 7f79a119d640 1 -- 172.21.15.92:0/2142645989 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f799c0cf420 con 0x7f799c09a170 2023-11-24T06:10:39.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.877+0000 7f79a119d640 1 --2- 172.21.15.92:0/2142645989 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f799c09a170 0x7f799c06e7c0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f7994009870 tx=0x7f799402b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c3bba9fb9c430dd server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:39.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.877+0000 7f799b7fe640 1 -- 172.21.15.92:0/2142645989 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7994030840 con 0x7f799c09a170 2023-11-24T06:10:39.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.877+0000 7f799b7fe640 1 -- 172.21.15.92:0/2142645989 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79940309a0 con 0x7f799c09a170 2023-11-24T06:10:39.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.877+0000 7f799b7fe640 1 -- 172.21.15.92:0/2142645989 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7994030d30 con 0x7f799c09a170 2023-11-24T06:10:39.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.878+0000 7f79a2c00640 1 -- 172.21.15.92:0/2142645989 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f799c09a170 msgr2=0x7f799c06e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:39.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.878+0000 7f79a2c00640 1 --2- 172.21.15.92:0/2142645989 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f799c09a170 0x7f799c06e7c0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f7994009870 tx=0x7f799402b980 comp rx=0 tx=0).stop 2023-11-24T06:10:39.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.878+0000 7f79a2c00640 1 -- 172.21.15.92:0/2142645989 shutdown_connections 2023-11-24T06:10:39.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.878+0000 7f79a2c00640 1 --2- 172.21.15.92:0/2142645989 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f799c09a170 0x7f799c06e7c0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:39.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.878+0000 7f79a2c00640 1 --2- 172.21.15.92:0/2142645989 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f799c15e880 0x7f799c06ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:39.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.878+0000 7f79a2c00640 1 -- 172.21.15.92:0/2142645989 >> 172.21.15.92:0/2142645989 conn(0x7f799c0b2d00 msgr2=0x7f799c0b3110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:39.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.878+0000 7f79a2c00640 1 -- 172.21.15.92:0/2142645989 shutdown_connections 2023-11-24T06:10:39.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.878+0000 7f79a2c00640 1 -- 172.21.15.92:0/2142645989 wait complete. 2023-11-24T06:10:39.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.880+0000 7f79a2c00640 1 Processor -- start 2023-11-24T06:10:39.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.880+0000 7f79a2c00640 1 -- start start 2023-11-24T06:10:39.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.881+0000 7f79a2c00640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f799c070490 0x7f799c0b68c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:39.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.881+0000 7f79a2c00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f799c09a170 0x7f799c1f22f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:39.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.881+0000 7f79a2c00640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f799c15e880 0x7f799c0ca4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:39.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.881+0000 7f79a2c00640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f799c15d080 con 0x7f799c09a170 2023-11-24T06:10:39.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.881+0000 7f79a2c00640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f799c0b59d0 con 0x7f799c15e880 2023-11-24T06:10:39.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.881+0000 7f79a2c00640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f799c0b4e50 con 0x7f799c070490 2023-11-24T06:10:39.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.881+0000 7f799bfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f799c09a170 0x7f799c1f22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:39.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.881+0000 7f799bfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f799c09a170 0x7f799c1f22f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46974/0 (socket says 172.21.15.92:46974) 2023-11-24T06:10:39.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.881+0000 7f799bfff640 1 -- 172.21.15.92:0/3294585213 learned_addr learned my addr 172.21.15.92:0/3294585213 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:10:39.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.881+0000 7f79a099c640 1 --2- 172.21.15.92:0/3294585213 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f799c070490 0x7f799c0b68c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:39.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.881+0000 7f799bfff640 1 -- 172.21.15.92:0/3294585213 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f799c070490 msgr2=0x7f799c0b68c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:39.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.881+0000 7f799bfff640 1 --2- 172.21.15.92:0/3294585213 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f799c070490 0x7f799c0b68c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:39.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.881+0000 7f799bfff640 1 -- 172.21.15.92:0/3294585213 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f799c15e880 msgr2=0x7f799c0ca4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:39.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.881+0000 7f799bfff640 1 --2- 172.21.15.92:0/3294585213 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f799c15e880 0x7f799c0ca4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:39.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.881+0000 7f799bfff640 1 -- 172.21.15.92:0/3294585213 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79940094d0 con 0x7f799c09a170 2023-11-24T06:10:39.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.882+0000 7f799bfff640 1 --2- 172.21.15.92:0/3294585213 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f799c09a170 0x7f799c1f22f0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f798400c430 tx=0x7f798400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:39.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.882+0000 7f79997fa640 1 -- 172.21.15.92:0/3294585213 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7984011790 con 0x7f799c09a170 2023-11-24T06:10:39.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.882+0000 7f79997fa640 1 -- 172.21.15.92:0/3294585213 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79840118f0 con 0x7f799c09a170 2023-11-24T06:10:39.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.882+0000 7f79997fa640 1 -- 172.21.15.92:0/3294585213 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7984011ce0 con 0x7f799c09a170 2023-11-24T06:10:39.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.882+0000 7f79a2c00640 1 -- 172.21.15.92:0/3294585213 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f799c0b5130 con 0x7f799c09a170 2023-11-24T06:10:39.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.882+0000 7f79a2c00640 1 -- 172.21.15.92:0/3294585213 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f799c0b56f0 con 0x7f799c09a170 2023-11-24T06:10:39.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.883+0000 7f79a2c00640 1 -- 172.21.15.92:0/3294585213 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f799c0c8350 con 0x7f799c09a170 2023-11-24T06:10:39.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.884+0000 7f79997fa640 1 -- 172.21.15.92:0/3294585213 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f798401c460 con 0x7f799c09a170 2023-11-24T06:10:39.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.885+0000 7f79997fa640 1 --2- 172.21.15.92:0/3294585213 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f797c06ae70 0x7f797c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:39.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.885+0000 7f79997fa640 1 -- 172.21.15.92:0/3294585213 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7984089530 con 0x7f799c09a170 2023-11-24T06:10:39.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.885+0000 7f79a099c640 1 --2- 172.21.15.92:0/3294585213 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f797c06ae70 0x7f797c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:39.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.886+0000 7f79a099c640 1 --2- 172.21.15.92:0/3294585213 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f797c06ae70 0x7f797c06d330 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f799c070460 tx=0x7f798c00b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:39.892 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:39.889+0000 7f79997fa640 1 -- 172.21.15.92:0/3294585213 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f799c0c8350 con 0x7f799c09a170 2023-11-24T06:10:40.083 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:40.080+0000 7f79a2c00640 1 -- 172.21.15.92:0/3294585213 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f799c0c8350 con 0x7f799c09a170 2023-11-24T06:10:40.084 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:40.081+0000 7f79997fa640 1 -- 172.21.15.92:0/3294585213 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f799c0c8350 con 0x7f799c09a170 2023-11-24T06:10:40.084 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 32 2023-11-24T06:10:40.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:40.083+0000 7f79a2c00640 1 -- 172.21.15.92:0/3294585213 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f797c06ae70 msgr2=0x7f797c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:40.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:40.083+0000 7f79a2c00640 1 --2- 172.21.15.92:0/3294585213 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f797c06ae70 0x7f797c06d330 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f799c070460 tx=0x7f798c00b040 comp rx=0 tx=0).stop 2023-11-24T06:10:40.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:40.083+0000 7f79a2c00640 1 -- 172.21.15.92:0/3294585213 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f799c09a170 msgr2=0x7f799c1f22f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:40.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:40.083+0000 7f79a2c00640 1 --2- 172.21.15.92:0/3294585213 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f799c09a170 0x7f799c1f22f0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f798400c430 tx=0x7f798400c8f0 comp rx=0 tx=0).stop 2023-11-24T06:10:40.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:40.084+0000 7f79a2c00640 1 -- 172.21.15.92:0/3294585213 shutdown_connections 2023-11-24T06:10:40.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:40.084+0000 7f79a2c00640 1 --2- 172.21.15.92:0/3294585213 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f797c06ae70 0x7f797c06d330 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:40.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:40.084+0000 7f79a2c00640 1 --2- 172.21.15.92:0/3294585213 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f799c15e880 0x7f799c0ca4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:40.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:40.084+0000 7f79a2c00640 1 --2- 172.21.15.92:0/3294585213 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f799c09a170 0x7f799c1f22f0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:40.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:40.084+0000 7f79a2c00640 1 --2- 172.21.15.92:0/3294585213 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f799c070490 0x7f799c0b68c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:40.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:40.084+0000 7f79a2c00640 1 -- 172.21.15.92:0/3294585213 >> 172.21.15.92:0/3294585213 conn(0x7f799c0b2d00 msgr2=0x7f799c06c5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:40.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:40.084+0000 7f79a2c00640 1 -- 172.21.15.92:0/3294585213 shutdown_connections 2023-11-24T06:10:40.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:40.084+0000 7f79a2c00640 1 -- 172.21.15.92:0/3294585213 wait complete. 2023-11-24T06:10:40.106 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T06:10:40.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:40.307+0000 7fb607b3c640 1 Processor -- start 2023-11-24T06:10:40.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:40.307+0000 7fb607b3c640 1 -- start start 2023-11-24T06:10:40.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:40.309+0000 7fb607b3c640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb60015e880 0x7fb60006ff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:40.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:40.309+0000 7fb607b3c640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb600070490 0x7fb600099a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:40.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:40.309+0000 7fb607b3c640 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6000cfa70 con 0x7fb600099fa0 2023-11-24T06:10:40.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:40.309+0000 7fb607b3c640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb60009a410 con 0x7fb600070490 2023-11-24T06:10:40.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:40.309+0000 7fb607b3c640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6000cff00 con 0x7fb60015e880 2023-11-24T06:10:40.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:40.309+0000 7fb6050d7640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb600070490 0x7fb600099a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:40.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:40.309+0000 7fb6050d7640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb600070490 0x7fb600099a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46980/0 (socket says 172.21.15.92:46980) 2023-11-24T06:10:40.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:40.309+0000 7fb6050d7640 1 -- 172.21.15.92:0/3851256077 learned_addr learned my addr 172.21.15.92:0/3851256077 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:10:40.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:40.310+0000 7fb6058d8640 1 --2- 172.21.15.92:0/3851256077 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb60015e880 0x7fb60006ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:40.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:40.310+0000 7fb6050d7640 1 -- 172.21.15.92:0/3851256077 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb60015e880 msgr2=0x7fb60006ff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:40.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:40.310+0000 7fb6050d7640 1 --2- 172.21.15.92:0/3851256077 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb60015e880 0x7fb60006ff50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:40.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:40.310+0000 7fb6050d7640 1 -- 172.21.15.92:0/3851256077 >> v1:172.21.15.118:6789/0 conn(0x7fb600099fa0 legacy=0x7fb6000cbb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:40.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:40.310+0000 7fb6050d7640 1 -- 172.21.15.92:0/3851256077 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb60006fc70 con 0x7fb600070490 2023-11-24T06:10:40.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:40.310+0000 7fb6058d8640 1 --2- 172.21.15.92:0/3851256077 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb60015e880 0x7fb60006ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:10:40.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:40.311+0000 7fb6050d7640 1 --2- 172.21.15.92:0/3851256077 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb600070490 0x7fb600099a60 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fb5f0009870 tx=0x7fb5f002b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4eb1d017bce3849c server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:40.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:40.311+0000 7fb6048d6640 1 -- 172.21.15.92:0/3851256077 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5f0030840 con 0x7fb600070490 2023-11-24T06:10:40.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:40.311+0000 7fb6048d6640 1 -- 172.21.15.92:0/3851256077 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5f00309a0 con 0x7fb600070490 2023-11-24T06:10:40.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:40.311+0000 7fb6048d6640 1 -- 172.21.15.92:0/3851256077 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5f0030d30 con 0x7fb600070490 2023-11-24T06:10:40.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:40.312+0000 7fb607b3c640 1 -- 172.21.15.92:0/3851256077 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb600070490 msgr2=0x7fb600099a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:40.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:40.312+0000 7fb607b3c640 1 --2- 172.21.15.92:0/3851256077 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb600070490 0x7fb600099a60 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fb5f0009870 tx=0x7fb5f002b980 comp rx=0 tx=0).stop 2023-11-24T06:10:40.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:40.312+0000 7fb607b3c640 1 -- 172.21.15.92:0/3851256077 shutdown_connections 2023-11-24T06:10:40.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:40.312+0000 7fb607b3c640 1 --2- 172.21.15.92:0/3851256077 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb600070490 0x7fb600099a60 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:40.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:40.312+0000 7fb607b3c640 1 --2- 172.21.15.92:0/3851256077 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb60015e880 0x7fb60006ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:40.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:40.312+0000 7fb607b3c640 1 -- 172.21.15.92:0/3851256077 >> 172.21.15.92:0/3851256077 conn(0x7fb6000b2d00 msgr2=0x7fb6000b3110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:40.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:40.312+0000 7fb607b3c640 1 -- 172.21.15.92:0/3851256077 shutdown_connections 2023-11-24T06:10:40.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:40.312+0000 7fb607b3c640 1 -- 172.21.15.92:0/3851256077 wait complete. 2023-11-24T06:10:40.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:40.314+0000 7fb607b3c640 1 Processor -- start 2023-11-24T06:10:40.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:40.314+0000 7fb607b3c640 1 -- start start 2023-11-24T06:10:40.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:40.315+0000 7fb607b3c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb600070490 0x7fb6000b59d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:40.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:40.315+0000 7fb607b3c640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb600099fa0 0x7fb600157780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:40.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:40.315+0000 7fb607b3c640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb60015e880 0x7fb6000c9d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:40.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:40.315+0000 7fb607b3c640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb600157cc0 con 0x7fb600070490 2023-11-24T06:10:40.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:40.315+0000 7fb607b3c640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6000ca280 con 0x7fb600099fa0 2023-11-24T06:10:40.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:40.315+0000 7fb607b3c640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6000b6bc0 con 0x7fb60015e880 2023-11-24T06:10:40.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:40.315+0000 7fb6058d8640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb600070490 0x7fb6000b59d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:40.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:40.316+0000 7fb6058d8640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb600070490 0x7fb6000b59d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46984/0 (socket says 172.21.15.92:46984) 2023-11-24T06:10:40.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:40.316+0000 7fb6058d8640 1 -- 172.21.15.92:0/1825148142 learned_addr learned my addr 172.21.15.92:0/1825148142 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:10:40.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:40.316+0000 7fb6058d8640 1 -- 172.21.15.92:0/1825148142 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb60015e880 msgr2=0x7fb6000c9d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:40.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:40.316+0000 7fb6058d8640 1 --2- 172.21.15.92:0/1825148142 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb60015e880 0x7fb6000c9d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:40.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:40.316+0000 7fb6058d8640 1 -- 172.21.15.92:0/1825148142 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb600099fa0 msgr2=0x7fb600157780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:40.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:40.316+0000 7fb6050d7640 1 --2- 172.21.15.92:0/1825148142 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb600099fa0 0x7fb600157780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:40.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:40.316+0000 7fb6058d8640 1 --2- 172.21.15.92:0/1825148142 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb600099fa0 0x7fb600157780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:40.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:40.316+0000 7fb6058d8640 1 -- 172.21.15.92:0/1825148142 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5f00094d0 con 0x7fb600070490 2023-11-24T06:10:40.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:40.316+0000 7fb6050d7640 1 --2- 172.21.15.92:0/1825148142 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb600099fa0 0x7fb600157780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:10:40.323 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:40.316+0000 7fb6058d8640 1 --2- 172.21.15.92:0/1825148142 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb600070490 0x7fb6000b59d0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fb5fc00c7f0 tx=0x7fb5fc00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:40.323 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:40.316+0000 7fb5f67fc640 1 -- 172.21.15.92:0/1825148142 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5fc007820 con 0x7fb600070490 2023-11-24T06:10:40.323 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:40.316+0000 7fb5f67fc640 1 -- 172.21.15.92:0/1825148142 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5fc007980 con 0x7fb600070490 2023-11-24T06:10:40.323 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:40.316+0000 7fb607b3c640 1 -- 172.21.15.92:0/1825148142 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6000b5fb0 con 0x7fb600070490 2023-11-24T06:10:40.323 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:40.317+0000 7fb5f67fc640 1 -- 172.21.15.92:0/1825148142 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5fc007cf0 con 0x7fb600070490 2023-11-24T06:10:40.323 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:40.317+0000 7fb607b3c640 1 -- 172.21.15.92:0/1825148142 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6000b6570 con 0x7fb600070490 2023-11-24T06:10:40.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:40.318+0000 7fb607b3c640 1 -- 172.21.15.92:0/1825148142 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6000675d0 con 0x7fb600070490 2023-11-24T06:10:40.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:40.319+0000 7fb5f67fc640 1 -- 172.21.15.92:0/1825148142 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb5fc0113f0 con 0x7fb600070490 2023-11-24T06:10:40.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:40.320+0000 7fb5f67fc640 1 --2- 172.21.15.92:0/1825148142 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fb5e406ae40 0x7fb5e406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:40.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:40.320+0000 7fb5f67fc640 1 -- 172.21.15.92:0/1825148142 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb5fc0898c0 con 0x7fb600070490 2023-11-24T06:10:40.326 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:40.320+0000 7fb6050d7640 1 --2- 172.21.15.92:0/1825148142 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fb5e406ae40 0x7fb5e406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:40.326 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:40.321+0000 7fb6050d7640 1 --2- 172.21.15.92:0/1825148142 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fb5e406ae40 0x7fb5e406d300 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fb5f0036000 tx=0x7fb5f003b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:40.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:40.324+0000 7fb5f67fc640 1 -- 172.21.15.92:0/1825148142 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb5fc054a00 con 0x7fb600070490 2023-11-24T06:10:40.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:40.515+0000 7fb607b3c640 1 -- 172.21.15.92:0/1825148142 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb6000c5ce0 con 0x7fb600070490 2023-11-24T06:10:40.519 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:40.516+0000 7fb5f67fc640 1 -- 172.21.15.92:0/1825148142 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb5fc01f0a0 con 0x7fb600070490 2023-11-24T06:10:40.520 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 64 2023-11-24T06:10:40.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:40.518+0000 7fb607b3c640 1 -- 172.21.15.92:0/1825148142 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fb5e406ae40 msgr2=0x7fb5e406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:40.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:40.518+0000 7fb607b3c640 1 --2- 172.21.15.92:0/1825148142 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fb5e406ae40 0x7fb5e406d300 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fb5f0036000 tx=0x7fb5f003b040 comp rx=0 tx=0).stop 2023-11-24T06:10:40.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:40.518+0000 7fb607b3c640 1 -- 172.21.15.92:0/1825148142 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb600070490 msgr2=0x7fb6000b59d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:40.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:40.518+0000 7fb607b3c640 1 --2- 172.21.15.92:0/1825148142 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb600070490 0x7fb6000b59d0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fb5fc00c7f0 tx=0x7fb5fc00ccb0 comp rx=0 tx=0).stop 2023-11-24T06:10:40.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:40.519+0000 7fb607b3c640 1 -- 172.21.15.92:0/1825148142 shutdown_connections 2023-11-24T06:10:40.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:40.519+0000 7fb607b3c640 1 --2- 172.21.15.92:0/1825148142 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fb5e406ae40 0x7fb5e406d300 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:40.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:40.519+0000 7fb607b3c640 1 --2- 172.21.15.92:0/1825148142 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb60015e880 0x7fb6000c9d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:40.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:40.519+0000 7fb607b3c640 1 --2- 172.21.15.92:0/1825148142 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb600099fa0 0x7fb600157780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:40.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:40.519+0000 7fb607b3c640 1 --2- 172.21.15.92:0/1825148142 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb600070490 0x7fb6000b59d0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:40.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:40.519+0000 7fb607b3c640 1 -- 172.21.15.92:0/1825148142 >> 172.21.15.92:0/1825148142 conn(0x7fb6000b2d00 msgr2=0x7fb60015db40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:40.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:40.519+0000 7fb607b3c640 1 -- 172.21.15.92:0/1825148142 shutdown_connections 2023-11-24T06:10:40.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:40.519+0000 7fb607b3c640 1 -- 172.21.15.92:0/1825148142 wait complete. 2023-11-24T06:10:40.540 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T06:10:40.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:40.740+0000 7fa29b63d640 1 Processor -- start 2023-11-24T06:10:40.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:40.740+0000 7fa29b63d640 1 -- start start 2023-11-24T06:10:40.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:40.741+0000 7fa29b63d640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fa294143ae0 0x7fa2940cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:40.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:40.742+0000 7fa29b63d640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fa2940cf7d0 0x7fa29415dca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:40.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:40.742+0000 7fa29b63d640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa29415e1e0 0x7fa29415a670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:40.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:40.742+0000 7fa29b63d640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2940cfc30 con 0x7fa2940cf7d0 2023-11-24T06:10:40.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:40.742+0000 7fa29b63d640 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2940b1ee0 con 0x7fa294143ae0 2023-11-24T06:10:40.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:40.742+0000 7fa29b63d640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2940b2780 con 0x7fa29415e1e0 2023-11-24T06:10:40.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:40.742+0000 7fa298bd8640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fa2940cf7d0 0x7fa29415dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:40.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:40.742+0000 7fa298bd8640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fa2940cf7d0 0x7fa29415dca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46996/0 (socket says 172.21.15.92:46996) 2023-11-24T06:10:40.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:40.742+0000 7fa298bd8640 1 -- 172.21.15.92:0/3763573563 learned_addr learned my addr 172.21.15.92:0/3763573563 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:10:40.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:40.742+0000 7fa2993d9640 1 --2- 172.21.15.92:0/3763573563 >> v2:172.21.15.118:3300/0 conn(0x7fa294143ae0 0x7fa2940cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:40.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:40.742+0000 7fa299bda640 1 --2- 172.21.15.92:0/3763573563 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa29415e1e0 0x7fa29415a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:40.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:40.742+0000 7fa298bd8640 1 -- 172.21.15.92:0/3763573563 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa29415e1e0 msgr2=0x7fa29415a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:40.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:40.742+0000 7fa298bd8640 1 --2- 172.21.15.92:0/3763573563 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa29415e1e0 0x7fa29415a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:40.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:40.742+0000 7fa298bd8640 1 -- 172.21.15.92:0/3763573563 >> v2:172.21.15.118:3300/0 conn(0x7fa294143ae0 msgr2=0x7fa2940cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:40.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:40.742+0000 7fa298bd8640 1 --2- 172.21.15.92:0/3763573563 >> v2:172.21.15.118:3300/0 conn(0x7fa294143ae0 0x7fa2940cf290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:40.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:40.742+0000 7fa298bd8640 1 -- 172.21.15.92:0/3763573563 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa29415e990 con 0x7fa2940cf7d0 2023-11-24T06:10:40.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:40.744+0000 7fa298bd8640 1 --2- 172.21.15.92:0/3763573563 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa2940cf7d0 0x7fa29415dca0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fa2940cfea0 tx=0x7fa28402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc5e34e428500c8e server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:40.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:40.744+0000 7fa28bfff640 1 -- 172.21.15.92:0/3763573563 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa284031870 con 0x7fa2940cf7d0 2023-11-24T06:10:40.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:40.744+0000 7fa28bfff640 1 -- 172.21.15.92:0/3763573563 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2840319d0 con 0x7fa2940cf7d0 2023-11-24T06:10:40.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:40.744+0000 7fa28bfff640 1 -- 172.21.15.92:0/3763573563 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa284031d60 con 0x7fa2940cf7d0 2023-11-24T06:10:40.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:40.745+0000 7fa29b63d640 1 -- 172.21.15.92:0/3763573563 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa2940cf7d0 msgr2=0x7fa29415dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:40.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:40.745+0000 7fa29b63d640 1 --2- 172.21.15.92:0/3763573563 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa2940cf7d0 0x7fa29415dca0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fa2940cfea0 tx=0x7fa28402b910 comp rx=0 tx=0).stop 2023-11-24T06:10:40.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:40.745+0000 7fa29b63d640 1 -- 172.21.15.92:0/3763573563 shutdown_connections 2023-11-24T06:10:40.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:40.745+0000 7fa29b63d640 1 --2- 172.21.15.92:0/3763573563 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa29415e1e0 0x7fa29415a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:40.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:40.745+0000 7fa29b63d640 1 --2- 172.21.15.92:0/3763573563 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa2940cf7d0 0x7fa29415dca0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:40.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:40.745+0000 7fa29b63d640 1 --2- 172.21.15.92:0/3763573563 >> v2:172.21.15.118:3300/0 conn(0x7fa294143ae0 0x7fa2940cf290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:40.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:40.745+0000 7fa29b63d640 1 -- 172.21.15.92:0/3763573563 >> 172.21.15.92:0/3763573563 conn(0x7fa29406a800 msgr2=0x7fa29406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:40.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:40.745+0000 7fa29b63d640 1 -- 172.21.15.92:0/3763573563 shutdown_connections 2023-11-24T06:10:40.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:40.745+0000 7fa29b63d640 1 -- 172.21.15.92:0/3763573563 wait complete. 2023-11-24T06:10:40.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:40.747+0000 7fa29b63d640 1 Processor -- start 2023-11-24T06:10:40.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:40.747+0000 7fa29b63d640 1 -- start start 2023-11-24T06:10:40.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:40.748+0000 7fa29b63d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa2940cf7d0 0x7fa29412f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:40.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:40.748+0000 7fa29b63d640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa294143ae0 0x7fa294130e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:40.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:40.748+0000 7fa29b63d640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa29415e1e0 0x7fa294141690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:40.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:40.748+0000 7fa29b63d640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2941309b0 con 0x7fa2940cf7d0 2023-11-24T06:10:40.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:40.748+0000 7fa29b63d640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa29412fb00 con 0x7fa294143ae0 2023-11-24T06:10:40.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:40.748+0000 7fa29b63d640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa29412fda0 con 0x7fa29415e1e0 2023-11-24T06:10:40.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:40.749+0000 7fa2993d9640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa2940cf7d0 0x7fa29412f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:40.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:40.749+0000 7fa2993d9640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa2940cf7d0 0x7fa29412f5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47006/0 (socket says 172.21.15.92:47006) 2023-11-24T06:10:40.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:40.749+0000 7fa2993d9640 1 -- 172.21.15.92:0/3405133894 learned_addr learned my addr 172.21.15.92:0/3405133894 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:10:40.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:40.749+0000 7fa299bda640 1 --2- 172.21.15.92:0/3405133894 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa29415e1e0 0x7fa294141690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:40.756 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:40.749+0000 7fa298bd8640 1 --2- 172.21.15.92:0/3405133894 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa294143ae0 0x7fa294130e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:40.756 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:40.749+0000 7fa2993d9640 1 -- 172.21.15.92:0/3405133894 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa29415e1e0 msgr2=0x7fa294141690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:40.756 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:40.749+0000 7fa2993d9640 1 --2- 172.21.15.92:0/3405133894 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa29415e1e0 0x7fa294141690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:40.756 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:40.749+0000 7fa2993d9640 1 -- 172.21.15.92:0/3405133894 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa294143ae0 msgr2=0x7fa294130e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:40.756 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:40.749+0000 7fa2993d9640 1 --2- 172.21.15.92:0/3405133894 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa294143ae0 0x7fa294130e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:40.756 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:40.749+0000 7fa2993d9640 1 -- 172.21.15.92:0/3405133894 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2840094d0 con 0x7fa2940cf7d0 2023-11-24T06:10:40.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:40.749+0000 7fa298bd8640 1 --2- 172.21.15.92:0/3405133894 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa294143ae0 0x7fa294130e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:10:40.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:40.749+0000 7fa299bda640 1 --2- 172.21.15.92:0/3405133894 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa29415e1e0 0x7fa294141690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:10:40.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:40.749+0000 7fa2993d9640 1 --2- 172.21.15.92:0/3405133894 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa2940cf7d0 0x7fa29412f5c0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fa29000d800 tx=0x7fa29000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:40.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:40.750+0000 7fa289ffb640 1 -- 172.21.15.92:0/3405133894 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2900096a0 con 0x7fa2940cf7d0 2023-11-24T06:10:40.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:40.750+0000 7fa289ffb640 1 -- 172.21.15.92:0/3405133894 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa290009800 con 0x7fa2940cf7d0 2023-11-24T06:10:40.758 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:40.750+0000 7fa29b63d640 1 -- 172.21.15.92:0/3405133894 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa294130080 con 0x7fa2940cf7d0 2023-11-24T06:10:40.758 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:40.750+0000 7fa289ffb640 1 -- 172.21.15.92:0/3405133894 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa290009b90 con 0x7fa2940cf7d0 2023-11-24T06:10:40.758 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:40.750+0000 7fa29b63d640 1 -- 172.21.15.92:0/3405133894 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa294130640 con 0x7fa2940cf7d0 2023-11-24T06:10:40.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:40.751+0000 7fa29b63d640 1 -- 172.21.15.92:0/3405133894 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2940675d0 con 0x7fa2940cf7d0 2023-11-24T06:10:40.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:40.752+0000 7fa289ffb640 1 -- 172.21.15.92:0/3405133894 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa290009cf0 con 0x7fa2940cf7d0 2023-11-24T06:10:40.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:40.754+0000 7fa289ffb640 1 --2- 172.21.15.92:0/3405133894 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fa27806ae40 0x7fa27806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:40.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:40.754+0000 7fa289ffb640 1 -- 172.21.15.92:0/3405133894 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa290089b30 con 0x7fa2940cf7d0 2023-11-24T06:10:40.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:40.754+0000 7fa298bd8640 1 --2- 172.21.15.92:0/3405133894 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fa27806ae40 0x7fa27806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:40.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:40.755+0000 7fa298bd8640 1 --2- 172.21.15.92:0/3405133894 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fa27806ae40 0x7fa27806d300 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fa284036000 tx=0x7fa28403c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:40.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:40.759+0000 7fa289ffb640 1 -- 172.21.15.92:0/3405133894 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa290054b40 con 0x7fa2940cf7d0 2023-11-24T06:10:40.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:40.986+0000 7fa29b63d640 1 -- 172.21.15.92:0/3405133894 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa29415d2b0 con 0x7fa2940cf7d0 2023-11-24T06:10:40.991 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:40.987+0000 7fa289ffb640 1 -- 172.21.15.92:0/3405133894 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4910 (secure 0 0 0) 0x7fa29000b440 con 0x7fa2940cf7d0 2023-11-24T06:10:40.991 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-24T06:10:40.991 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4236,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6836/437604699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":437604699},{"type":"v1","addr":"172.21.15.92:6836","nonce":437604699}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6835/3368711442","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":3368711442},{"type":"v1","addr":"172.21.15.118:6835","nonce":3368711442}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4255,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/2767290380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":2767290380},{"type":"v1","addr":"172.21.15.92:6837","nonce":2767290380}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T06:10:25.115612+0000","modified":"2023-11-24T06:10:28.210438+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4264},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4264":{"gid":4264,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.118:6833/1146052262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":1146052262},{"type":"v1","addr":"172.21.15.118:6833","nonce":1146052262}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-24T06:10:40.992 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 8 2023-11-24T06:10:40.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:40.990+0000 7fa29b63d640 1 -- 172.21.15.92:0/3405133894 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fa27806ae40 msgr2=0x7fa27806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:40.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:40.990+0000 7fa29b63d640 1 --2- 172.21.15.92:0/3405133894 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fa27806ae40 0x7fa27806d300 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fa284036000 tx=0x7fa28403c040 comp rx=0 tx=0).stop 2023-11-24T06:10:40.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:40.990+0000 7fa29b63d640 1 -- 172.21.15.92:0/3405133894 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa2940cf7d0 msgr2=0x7fa29412f5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:40.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:40.990+0000 7fa29b63d640 1 --2- 172.21.15.92:0/3405133894 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa2940cf7d0 0x7fa29412f5c0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fa29000d800 tx=0x7fa29000dcc0 comp rx=0 tx=0).stop 2023-11-24T06:10:40.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:40.990+0000 7fa29b63d640 1 -- 172.21.15.92:0/3405133894 shutdown_connections 2023-11-24T06:10:40.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:40.990+0000 7fa29b63d640 1 --2- 172.21.15.92:0/3405133894 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fa27806ae40 0x7fa27806d300 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:40.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:40.990+0000 7fa29b63d640 1 --2- 172.21.15.92:0/3405133894 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa29415e1e0 0x7fa294141690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:40.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:40.990+0000 7fa29b63d640 1 --2- 172.21.15.92:0/3405133894 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa294143ae0 0x7fa294130e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:40.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:40.990+0000 7fa29b63d640 1 --2- 172.21.15.92:0/3405133894 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa2940cf7d0 0x7fa29412f5c0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:40.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:40.990+0000 7fa29b63d640 1 -- 172.21.15.92:0/3405133894 >> 172.21.15.92:0/3405133894 conn(0x7fa29406a800 msgr2=0x7fa294142e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:40.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:40.990+0000 7fa29b63d640 1 -- 172.21.15.92:0/3405133894 shutdown_connections 2023-11-24T06:10:40.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:40.990+0000 7fa29b63d640 1 -- 172.21.15.92:0/3405133894 wait complete. 2023-11-24T06:10:41.012 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:10:41.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:41.219+0000 7f3b05d9e640 1 Processor -- start 2023-11-24T06:10:41.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:41.219+0000 7f3b05d9e640 1 -- start start 2023-11-24T06:10:41.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:41.220+0000 7f3b05d9e640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f3b000cc990 0x7f3b0006d870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:41.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:41.220+0000 7f3b05d9e640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b00070e50 con 0x7f3b00143c40 2023-11-24T06:10:41.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:41.220+0000 7f3b05d9e640 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b00144020 con 0x7f3b000cc500 2023-11-24T06:10:41.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:41.220+0000 7f3b05d9e640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b000723c0 con 0x7f3b000cc990 2023-11-24T06:10:41.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:41.220+0000 7f3b0559d640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f3b000cc990 0x7f3b0006d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:41.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:41.221+0000 7f3b04d9c640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f3b00143c40 0x7f3b00071290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:52614/0 (socket says 172.21.15.92:52614) 2023-11-24T06:10:41.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:41.221+0000 7f3b04d9c640 1 -- 172.21.15.92:0/1467859035 learned_addr learned my addr 172.21.15.92:0/1467859035 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-24T06:10:41.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:41.221+0000 7f3b0559d640 1 -- 172.21.15.92:0/1467859035 >> v1:172.21.15.118:6789/0 conn(0x7f3b000cc500 legacy=0x7f3b0006d160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:41.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:41.221+0000 7f3b0559d640 1 -- 172.21.15.92:0/1467859035 >> v1:172.21.15.92:6789/0 conn(0x7f3b00143c40 legacy=0x7f3b00071290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:41.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:41.221+0000 7f3b0559d640 1 -- 172.21.15.92:0/1467859035 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b00071e00 con 0x7f3b000cc990 2023-11-24T06:10:41.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:41.222+0000 7f3b0559d640 1 --2- 172.21.15.92:0/1467859035 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3b000cc990 0x7f3b0006d870 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f3af8009d90 tx=0x7f3af802be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc3e6f7add1bff6e server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:41.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:41.222+0000 7f3aff7fe640 1 -- 172.21.15.92:0/1467859035 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3af802cbd0 con 0x7f3b000cc990 2023-11-24T06:10:41.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:41.222+0000 7f3aff7fe640 1 -- 172.21.15.92:0/1467859035 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3af802cd30 con 0x7f3b000cc990 2023-11-24T06:10:41.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:41.222+0000 7f3aff7fe640 1 -- 172.21.15.92:0/1467859035 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3af802cea0 con 0x7f3b000cc990 2023-11-24T06:10:41.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:41.223+0000 7f3b05d9e640 1 -- 172.21.15.92:0/1467859035 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3b000cc990 msgr2=0x7f3b0006d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:41.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:41.223+0000 7f3b05d9e640 1 --2- 172.21.15.92:0/1467859035 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3b000cc990 0x7f3b0006d870 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f3af8009d90 tx=0x7f3af802be20 comp rx=0 tx=0).stop 2023-11-24T06:10:41.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:41.223+0000 7f3b05d9e640 1 -- 172.21.15.92:0/1467859035 shutdown_connections 2023-11-24T06:10:41.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:41.223+0000 7f3b05d9e640 1 --2- 172.21.15.92:0/1467859035 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3b000cc990 0x7f3b0006d870 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:41.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:41.223+0000 7f3b05d9e640 1 -- 172.21.15.92:0/1467859035 >> 172.21.15.92:0/1467859035 conn(0x7f3b0006a800 msgr2=0x7f3b0006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:41.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:41.223+0000 7f3b05d9e640 1 -- 172.21.15.92:0/1467859035 shutdown_connections 2023-11-24T06:10:41.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:41.223+0000 7f3b05d9e640 1 -- 172.21.15.92:0/1467859035 wait complete. 2023-11-24T06:10:41.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:41.225+0000 7f3b05d9e640 1 Processor -- start 2023-11-24T06:10:41.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:41.225+0000 7f3b05d9e640 1 -- start start 2023-11-24T06:10:41.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:41.226+0000 7f3b05d9e640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3b000cc500 0x7f3b0015a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:41.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:41.226+0000 7f3b05d9e640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3b000cc990 0x7f3b0013b220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:41.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:41.226+0000 7f3b05d9e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3b00143c40 0x7f3b0013b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:41.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:41.226+0000 7f3b05d9e640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b001f7090 con 0x7f3b00143c40 2023-11-24T06:10:41.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:41.226+0000 7f3b05d9e640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b0013f140 con 0x7f3b000cc500 2023-11-24T06:10:41.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:41.226+0000 7f3b05d9e640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b001f6760 con 0x7f3b000cc990 2023-11-24T06:10:41.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:41.226+0000 7f3b0559d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3b00143c40 0x7f3b0013b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:41.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:41.226+0000 7f3b0559d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3b00143c40 0x7f3b0013b760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47036/0 (socket says 172.21.15.92:47036) 2023-11-24T06:10:41.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:41.226+0000 7f3b0559d640 1 -- 172.21.15.92:0/2791812374 learned_addr learned my addr 172.21.15.92:0/2791812374 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:10:41.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:41.227+0000 7f3afffff640 1 --2- 172.21.15.92:0/2791812374 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3b000cc990 0x7f3b0013b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:41.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:41.227+0000 7f3b04d9c640 1 --2- 172.21.15.92:0/2791812374 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3b000cc500 0x7f3b0015a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:41.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:41.227+0000 7f3b0559d640 1 -- 172.21.15.92:0/2791812374 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3b000cc990 msgr2=0x7f3b0013b220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:41.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:41.227+0000 7f3b0559d640 1 --2- 172.21.15.92:0/2791812374 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3b000cc990 0x7f3b0013b220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:41.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:41.227+0000 7f3b0559d640 1 -- 172.21.15.92:0/2791812374 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3b000cc500 msgr2=0x7f3b0015a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:41.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:41.227+0000 7f3b0559d640 1 --2- 172.21.15.92:0/2791812374 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3b000cc500 0x7f3b0015a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:41.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:41.227+0000 7f3b0559d640 1 -- 172.21.15.92:0/2791812374 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3af8009a70 con 0x7f3b00143c40 2023-11-24T06:10:41.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:41.227+0000 7f3b0559d640 1 --2- 172.21.15.92:0/2791812374 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3b00143c40 0x7f3b0013b760 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f3af8009d90 tx=0x7f3af8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:41.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:41.227+0000 7f3afd7fa640 1 -- 172.21.15.92:0/2791812374 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3af802cea0 con 0x7f3b00143c40 2023-11-24T06:10:41.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:41.227+0000 7f3afd7fa640 1 -- 172.21.15.92:0/2791812374 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3af8040450 con 0x7f3b00143c40 2023-11-24T06:10:41.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:41.227+0000 7f3b05d9e640 1 -- 172.21.15.92:0/2791812374 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b001f6a40 con 0x7f3b00143c40 2023-11-24T06:10:41.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:41.227+0000 7f3afd7fa640 1 -- 172.21.15.92:0/2791812374 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3af80407e0 con 0x7f3b00143c40 2023-11-24T06:10:41.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:41.228+0000 7f3b05d9e640 1 -- 172.21.15.92:0/2791812374 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b001f7520 con 0x7f3b00143c40 2023-11-24T06:10:41.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:41.229+0000 7f3b05d9e640 1 -- 172.21.15.92:0/2791812374 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b001f7200 con 0x7f3b00143c40 2023-11-24T06:10:41.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:41.230+0000 7f3afd7fa640 1 -- 172.21.15.92:0/2791812374 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f3af8040940 con 0x7f3b00143c40 2023-11-24T06:10:41.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:41.231+0000 7f3afd7fa640 1 --2- 172.21.15.92:0/2791812374 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f3ae406ae70 0x7f3ae406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:41.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:41.231+0000 7f3afd7fa640 1 -- 172.21.15.92:0/2791812374 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3af80aba40 con 0x7f3b00143c40 2023-11-24T06:10:41.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:41.231+0000 7f3b04d9c640 1 --2- 172.21.15.92:0/2791812374 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f3ae406ae70 0x7f3ae406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:41.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:41.232+0000 7f3b04d9c640 1 --2- 172.21.15.92:0/2791812374 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f3ae406ae70 0x7f3ae406d330 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f3af4005880 tx=0x7f3af4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:41.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:41.235+0000 7f3afd7fa640 1 -- 172.21.15.92:0/2791812374 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3af80ade60 con 0x7f3b00143c40 2023-11-24T06:10:41.427 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:41.424+0000 7f3b05d9e640 1 -- 172.21.15.92:0/2791812374 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3b00067450 con 0x7f3b00143c40 2023-11-24T06:10:41.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:41.425+0000 7f3afd7fa640 1 -- 172.21.15.92:0/2791812374 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16457 (secure 0 0 0) 0x7f3af8076bb0 con 0x7f3b00143c40 2023-11-24T06:10:41.429 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-24T06:10:41.429 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":20,"fsid":"2d6dee5f-8aa3-470f-931a-50ade963b1cb","created":"2023-11-24T06:10:04.707204+0000","modified":"2023-11-24T06:10:26.698733+0000","last_up_change":"2023-11-24T06:10:17.598352+0000","last_in_change":"2023-11-24T06:10:09.040793+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:10:17.733098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:10:22.926445+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T06:10:24.151288+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ff37a9fb-135f-471f-b4b0-f7b5772006c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6803","nonce":20560714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6805","nonce":20560714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6809","nonce":20560714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6807","nonce":20560714}]},"public_addr":"172.21.15.92:6803/20560714","cluster_addr":"172.21.15.92:6805/20560714","heartbeat_back_addr":"172.21.15.92:6809/20560714","heartbeat_front_addr":"172.21.15.92:6807/20560714","state":["exists","up"]},{"osd":1,"uuid":"03cdd353-dcf1-4f38-94fe-a795137c06b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6819","nonce":2451239906}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6821","nonce":2451239906}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6825","nonce":2451239906}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6823","nonce":2451239906}]},"public_addr":"172.21.15.92:6819/2451239906","cluster_addr":"172.21.15.92:6821/2451239906","heartbeat_back_addr":"172.21.15.92:6825/2451239906","heartbeat_front_addr":"172.21.15.92:6823/2451239906","state":["exists","up"]},{"osd":2,"uuid":"a4a20d80-5d57-484b-becd-9aa8070dd3aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6811","nonce":1124979277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6813","nonce":1124979277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6817","nonce":1124979277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6815","nonce":1124979277}]},"public_addr":"172.21.15.92:6811/1124979277","cluster_addr":"172.21.15.92:6813/1124979277","heartbeat_back_addr":"172.21.15.92:6817/1124979277","heartbeat_front_addr":"172.21.15.92:6815/1124979277","state":["exists","up"]},{"osd":3,"uuid":"4648d7ce-9b29-4583-9425-4ba8fd2b1903","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6827","nonce":1734115782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6829","nonce":1734115782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6833","nonce":1734115782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6831","nonce":1734115782}]},"public_addr":"172.21.15.92:6827/1734115782","cluster_addr":"172.21.15.92:6829/1734115782","heartbeat_back_addr":"172.21.15.92:6833/1734115782","heartbeat_front_addr":"172.21.15.92:6831/1734115782","state":["exists","up"]},{"osd":4,"uuid":"3bc9d7eb-c6fa-4fcf-a3ec-ade604e05aac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6819","nonce":3425937229}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6822","nonce":3425937229}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6830","nonce":3425937229}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6826","nonce":3425937229}]},"public_addr":"172.21.15.118:6819/3425937229","cluster_addr":"172.21.15.118:6822/3425937229","heartbeat_back_addr":"172.21.15.118:6830/3425937229","heartbeat_front_addr":"172.21.15.118:6826/3425937229","state":["exists","up"]},{"osd":5,"uuid":"dce9ba6d-59ed-46d1-84fc-a84b13de10c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6801","nonce":827455984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6803","nonce":827455984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6807","nonce":827455984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6805","nonce":827455984}]},"public_addr":"172.21.15.118:6801/827455984","cluster_addr":"172.21.15.118:6803/827455984","heartbeat_back_addr":"172.21.15.118:6807/827455984","heartbeat_front_addr":"172.21.15.118:6805/827455984","state":["exists","up"]},{"osd":6,"uuid":"82aec504-ae58-4709-8d30-46f2b12081b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6809","nonce":2946986141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6811","nonce":2946986141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6815","nonce":2946986141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6813","nonce":2946986141}]},"public_addr":"172.21.15.118:6809/2946986141","cluster_addr":"172.21.15.118:6811/2946986141","heartbeat_back_addr":"172.21.15.118:6815/2946986141","heartbeat_front_addr":"172.21.15.118:6813/2946986141","state":["exists","up"]},{"osd":7,"uuid":"4ed4c624-c88f-4d35-bdb1-bf209b5dffb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6817","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6818","nonce":48122059}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6821","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6823","nonce":48122059}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6829","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6831","nonce":48122059}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6825","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6827","nonce":48122059}]},"public_addr":"172.21.15.118:6818/48122059","cluster_addr":"172.21.15.118:6823/48122059","heartbeat_back_addr":"172.21.15.118:6831/48122059","heartbeat_front_addr":"172.21.15.118:6827/48122059","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.536664+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.368324+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.271201+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.342971+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.621832+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.387110+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.632946+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.416032+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:10:41.431 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:41.428+0000 7f3b05d9e640 1 -- 172.21.15.92:0/2791812374 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f3ae406ae70 msgr2=0x7f3ae406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:41.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:41.428+0000 7f3b05d9e640 1 --2- 172.21.15.92:0/2791812374 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f3ae406ae70 0x7f3ae406d330 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f3af4005880 tx=0x7f3af4009040 comp rx=0 tx=0).stop 2023-11-24T06:10:41.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:41.428+0000 7f3b05d9e640 1 -- 172.21.15.92:0/2791812374 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3b00143c40 msgr2=0x7f3b0013b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:41.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:41.428+0000 7f3b05d9e640 1 --2- 172.21.15.92:0/2791812374 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3b00143c40 0x7f3b0013b760 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f3af8009d90 tx=0x7f3af8004a40 comp rx=0 tx=0).stop 2023-11-24T06:10:41.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:41.428+0000 7f3b05d9e640 1 -- 172.21.15.92:0/2791812374 shutdown_connections 2023-11-24T06:10:41.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:41.428+0000 7f3b05d9e640 1 --2- 172.21.15.92:0/2791812374 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f3ae406ae70 0x7f3ae406d330 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:41.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:41.428+0000 7f3b05d9e640 1 --2- 172.21.15.92:0/2791812374 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3b00143c40 0x7f3b0013b760 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:41.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:41.428+0000 7f3b05d9e640 1 --2- 172.21.15.92:0/2791812374 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3b000cc990 0x7f3b0013b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:41.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:41.428+0000 7f3b05d9e640 1 --2- 172.21.15.92:0/2791812374 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3b000cc500 0x7f3b0015a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:41.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:41.428+0000 7f3b05d9e640 1 -- 172.21.15.92:0/2791812374 >> 172.21.15.92:0/2791812374 conn(0x7f3b0006a800 msgr2=0x7f3b00140a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:41.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:41.428+0000 7f3b05d9e640 1 -- 172.21.15.92:0/2791812374 shutdown_connections 2023-11-24T06:10:41.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:41.428+0000 7f3b05d9e640 1 -- 172.21.15.92:0/2791812374 wait complete. 2023-11-24T06:10:41.451 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T06:10:41.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:41.657+0000 7f339a17c640 1 Processor -- start 2023-11-24T06:10:41.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:41.657+0000 7f339a17c640 1 -- start start 2023-11-24T06:10:41.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:41.659+0000 7f339a17c640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f339413df80 0x7f339415bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:41.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:41.659+0000 7f339a17c640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f339413e360 0x7f33940cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:41.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:41.659+0000 7f339a17c640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f339415f9c0 0x7f33940cf7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:41.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:41.659+0000 7f339a17c640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f339406c470 con 0x7f339415f9c0 2023-11-24T06:10:41.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:41.659+0000 7f339a17c640 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f339406c750 con 0x7f339413e360 2023-11-24T06:10:41.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:41.659+0000 7f339a17c640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f339406ca30 con 0x7f339413df80 2023-11-24T06:10:41.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:41.659+0000 7f3393fff640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f339415f9c0 0x7f33940cf7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:41.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:41.659+0000 7f3393fff640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f339415f9c0 0x7f33940cf7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47050/0 (socket says 172.21.15.92:47050) 2023-11-24T06:10:41.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:41.659+0000 7f3393fff640 1 -- 172.21.15.92:0/1484737925 learned_addr learned my addr 172.21.15.92:0/1484737925 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:10:41.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:41.659+0000 7f33937fe640 1 --2- 172.21.15.92:0/1484737925 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f339413df80 0x7f339415bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:41.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:41.660+0000 7f338bfff640 1 --2- 172.21.15.92:0/1484737925 >> v2:172.21.15.118:3300/0 conn(0x7f339413e360 0x7f33940cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:41.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:41.660+0000 7f3393fff640 1 -- 172.21.15.92:0/1484737925 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f339413df80 msgr2=0x7f339415bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:41.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:41.660+0000 7f3393fff640 1 --2- 172.21.15.92:0/1484737925 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f339413df80 0x7f339415bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:41.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:41.660+0000 7f3393fff640 1 -- 172.21.15.92:0/1484737925 >> v2:172.21.15.118:3300/0 conn(0x7f339413e360 msgr2=0x7f33940cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:41.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:41.660+0000 7f3393fff640 1 --2- 172.21.15.92:0/1484737925 >> v2:172.21.15.118:3300/0 conn(0x7f339413e360 0x7f33940cf290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:41.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:41.660+0000 7f3393fff640 1 -- 172.21.15.92:0/1484737925 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f339406cd10 con 0x7f339415f9c0 2023-11-24T06:10:41.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:41.661+0000 7f3393fff640 1 --2- 172.21.15.92:0/1484737925 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f339415f9c0 0x7f33940cf7d0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f33940cfed0 tx=0x7f338c02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=776ec7db364cdab4 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:41.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:41.661+0000 7f3392ffd640 1 -- 172.21.15.92:0/1484737925 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f338c02ca70 con 0x7f339415f9c0 2023-11-24T06:10:41.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:41.661+0000 7f3392ffd640 1 -- 172.21.15.92:0/1484737925 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f338c02cbd0 con 0x7f339415f9c0 2023-11-24T06:10:41.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:41.661+0000 7f3392ffd640 1 -- 172.21.15.92:0/1484737925 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f338c0343f0 con 0x7f339415f9c0 2023-11-24T06:10:41.666 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:41.662+0000 7f339a17c640 1 -- 172.21.15.92:0/1484737925 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f339415f9c0 msgr2=0x7f33940cf7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:41.666 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:41.662+0000 7f339a17c640 1 --2- 172.21.15.92:0/1484737925 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f339415f9c0 0x7f33940cf7d0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f33940cfed0 tx=0x7f338c02bed0 comp rx=0 tx=0).stop 2023-11-24T06:10:41.666 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:41.662+0000 7f339a17c640 1 -- 172.21.15.92:0/1484737925 shutdown_connections 2023-11-24T06:10:41.666 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:41.662+0000 7f339a17c640 1 --2- 172.21.15.92:0/1484737925 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f339415f9c0 0x7f33940cf7d0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:41.666 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:41.662+0000 7f339a17c640 1 --2- 172.21.15.92:0/1484737925 >> v2:172.21.15.118:3300/0 conn(0x7f339413e360 0x7f33940cf290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:41.666 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:41.662+0000 7f339a17c640 1 --2- 172.21.15.92:0/1484737925 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f339413df80 0x7f339415bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:41.666 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:41.662+0000 7f339a17c640 1 -- 172.21.15.92:0/1484737925 >> 172.21.15.92:0/1484737925 conn(0x7f339406a800 msgr2=0x7f339406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:41.666 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:41.662+0000 7f339a17c640 1 -- 172.21.15.92:0/1484737925 shutdown_connections 2023-11-24T06:10:41.666 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:41.662+0000 7f339a17c640 1 -- 172.21.15.92:0/1484737925 wait complete. 2023-11-24T06:10:41.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:41.664+0000 7f339a17c640 1 Processor -- start 2023-11-24T06:10:41.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:41.664+0000 7f339a17c640 1 -- start start 2023-11-24T06:10:41.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:41.665+0000 7f339a17c640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f339413df80 0x7f339406d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:41.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:41.665+0000 7f339a17c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f339413e360 0x7f33940ce0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:41.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:41.665+0000 7f339a17c640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f339415f9c0 0x7f33940ad9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:41.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:41.665+0000 7f339a17c640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3394146230 con 0x7f339413e360 2023-11-24T06:10:41.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:41.665+0000 7f339a17c640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33941487d0 con 0x7f339415f9c0 2023-11-24T06:10:41.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:41.665+0000 7f339a17c640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3394148ab0 con 0x7f339413df80 2023-11-24T06:10:41.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:41.665+0000 7f338bfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f339413e360 0x7f33940ce0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:41.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:41.665+0000 7f338bfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f339413e360 0x7f33940ce0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47058/0 (socket says 172.21.15.92:47058) 2023-11-24T06:10:41.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:41.665+0000 7f338bfff640 1 -- 172.21.15.92:0/1992852183 learned_addr learned my addr 172.21.15.92:0/1992852183 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:10:41.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:41.665+0000 7f33937fe640 1 --2- 172.21.15.92:0/1992852183 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f339413df80 0x7f339406d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:41.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:41.666+0000 7f3393fff640 1 --2- 172.21.15.92:0/1992852183 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f339415f9c0 0x7f33940ad9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:41.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:41.666+0000 7f338bfff640 1 -- 172.21.15.92:0/1992852183 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f339413df80 msgr2=0x7f339406d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:41.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:41.666+0000 7f338bfff640 1 --2- 172.21.15.92:0/1992852183 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f339413df80 0x7f339406d410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:41.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:41.666+0000 7f338bfff640 1 -- 172.21.15.92:0/1992852183 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f339415f9c0 msgr2=0x7f33940ad9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:41.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:41.666+0000 7f338bfff640 1 --2- 172.21.15.92:0/1992852183 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f339415f9c0 0x7f33940ad9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:41.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:41.666+0000 7f338bfff640 1 -- 172.21.15.92:0/1992852183 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f338c009b30 con 0x7f339413e360 2023-11-24T06:10:41.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:41.666+0000 7f338bfff640 1 --2- 172.21.15.92:0/1992852183 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f339413e360 0x7f33940ce0e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f337c00d800 tx=0x7f337c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:41.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:41.666+0000 7f3390ff9640 1 -- 172.21.15.92:0/1992852183 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f337c00b6c0 con 0x7f339413e360 2023-11-24T06:10:41.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:41.666+0000 7f3390ff9640 1 -- 172.21.15.92:0/1992852183 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f337c00b820 con 0x7f339413e360 2023-11-24T06:10:41.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:41.666+0000 7f3390ff9640 1 -- 172.21.15.92:0/1992852183 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f337c00bbb0 con 0x7f339413e360 2023-11-24T06:10:41.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:41.666+0000 7f339a17c640 1 -- 172.21.15.92:0/1992852183 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3394148d90 con 0x7f339413e360 2023-11-24T06:10:41.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:41.666+0000 7f339a17c640 1 -- 172.21.15.92:0/1992852183 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3394154570 con 0x7f339413e360 2023-11-24T06:10:41.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:41.668+0000 7f339a17c640 1 -- 172.21.15.92:0/1992852183 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33940675d0 con 0x7f339413e360 2023-11-24T06:10:41.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:41.668+0000 7f3390ff9640 1 -- 172.21.15.92:0/1992852183 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f337c00bd10 con 0x7f339413e360 2023-11-24T06:10:41.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:41.669+0000 7f3390ff9640 1 --2- 172.21.15.92:0/1992852183 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f337406ae40 0x7f337406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:41.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:41.669+0000 7f3390ff9640 1 -- 172.21.15.92:0/1992852183 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f337c089b50 con 0x7f339413e360 2023-11-24T06:10:41.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:41.669+0000 7f33937fe640 1 --2- 172.21.15.92:0/1992852183 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f337406ae40 0x7f337406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:41.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:41.670+0000 7f33937fe640 1 --2- 172.21.15.92:0/1992852183 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f337406ae40 0x7f337406d300 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f3384006fd0 tx=0x7f3384008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:41.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:41.674+0000 7f3390ff9640 1 -- 172.21.15.92:0/1992852183 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f337c054c00 con 0x7f339413e360 2023-11-24T06:10:41.910 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:41.907+0000 7f339a17c640 1 -- 172.21.15.92:0/1992852183 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f339406df00 con 0x7f339413e360 2023-11-24T06:10:41.911 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:41.908+0000 7f3390ff9640 1 -- 172.21.15.92:0/1992852183 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4910 (secure 0 0 0) 0x7f337c058220 con 0x7f339413e360 2023-11-24T06:10:41.911 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-24T06:10:41.911 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4236,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6836/437604699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":437604699},{"type":"v1","addr":"172.21.15.92:6836","nonce":437604699}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6835/3368711442","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":3368711442},{"type":"v1","addr":"172.21.15.118:6835","nonce":3368711442}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4255,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/2767290380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":2767290380},{"type":"v1","addr":"172.21.15.92:6837","nonce":2767290380}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T06:10:25.115612+0000","modified":"2023-11-24T06:10:28.210438+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4264},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4264":{"gid":4264,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.118:6833/1146052262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":1146052262},{"type":"v1","addr":"172.21.15.118:6833","nonce":1146052262}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-24T06:10:41.912 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 8 2023-11-24T06:10:41.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:41.910+0000 7f339a17c640 1 -- 172.21.15.92:0/1992852183 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f337406ae40 msgr2=0x7f337406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:41.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:41.910+0000 7f339a17c640 1 --2- 172.21.15.92:0/1992852183 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f337406ae40 0x7f337406d300 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f3384006fd0 tx=0x7f3384008040 comp rx=0 tx=0).stop 2023-11-24T06:10:41.914 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:41.910+0000 7f339a17c640 1 -- 172.21.15.92:0/1992852183 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f339413e360 msgr2=0x7f33940ce0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:41.914 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:41.910+0000 7f339a17c640 1 --2- 172.21.15.92:0/1992852183 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f339413e360 0x7f33940ce0e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f337c00d800 tx=0x7f337c00dcc0 comp rx=0 tx=0).stop 2023-11-24T06:10:41.914 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:41.910+0000 7f339a17c640 1 -- 172.21.15.92:0/1992852183 shutdown_connections 2023-11-24T06:10:41.914 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:41.910+0000 7f339a17c640 1 --2- 172.21.15.92:0/1992852183 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f337406ae40 0x7f337406d300 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:41.914 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:41.910+0000 7f339a17c640 1 --2- 172.21.15.92:0/1992852183 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f339415f9c0 0x7f33940ad9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:41.914 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:41.910+0000 7f339a17c640 1 --2- 172.21.15.92:0/1992852183 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f339413e360 0x7f33940ce0e0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:41.914 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:41.910+0000 7f339a17c640 1 --2- 172.21.15.92:0/1992852183 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f339413df80 0x7f339406d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:41.915 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:41.910+0000 7f339a17c640 1 -- 172.21.15.92:0/1992852183 >> 172.21.15.92:0/1992852183 conn(0x7f339406a800 msgr2=0x7f339413ebd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:41.915 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:41.910+0000 7f339a17c640 1 -- 172.21.15.92:0/1992852183 shutdown_connections 2023-11-24T06:10:41.915 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:41.910+0000 7f339a17c640 1 -- 172.21.15.92:0/1992852183 wait complete. 2023-11-24T06:10:42.335 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 8, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-24T06:10:25.115612+0000', 'modified': '2023-11-24T06:10:28.210438+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4264}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4264': {'gid': 4264, 'name': 'd', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.118:6833/1146052262', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.118:6832', 'nonce': 1146052262}, {'type': 'v1', 'addr': '172.21.15.118:6833', 'nonce': 1146052262}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-24T06:10:42.336 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-24T06:10:42.336 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2023-11-24T06:10:42.557 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.554+0000 7f4e48c48640 1 Processor -- start 2023-11-24T06:10:42.558 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.554+0000 7f4e48c48640 1 -- start start 2023-11-24T06:10:42.560 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.556+0000 7f4e48c48640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f4e4413df80 0x7f4e4415bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:42.560 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.556+0000 7f4e48c48640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f4e4413e360 0x7f4e440cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:42.560 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.556+0000 7f4e48c48640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4e4415f9c0 0x7f4e440cf7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:42.560 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.556+0000 7f4e48c48640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e4406c470 con 0x7f4e4413e360 2023-11-24T06:10:42.560 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.556+0000 7f4e48c48640 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e4406c750 con 0x7f4e4413df80 2023-11-24T06:10:42.560 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.556+0000 7f4e48c48640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e4406ca30 con 0x7f4e4415f9c0 2023-11-24T06:10:42.560 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.556+0000 7f4e39d9b640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f4e4413e360 0x7f4e440cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:42.560 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.556+0000 7f4e39d9b640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f4e4413e360 0x7f4e440cf290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47068/0 (socket says 172.21.15.92:47068) 2023-11-24T06:10:42.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.556+0000 7f4e39d9b640 1 -- 172.21.15.92:0/2968886509 learned_addr learned my addr 172.21.15.92:0/2968886509 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:10:42.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.557+0000 7f4e4259c640 1 --2- 172.21.15.92:0/2968886509 >> v2:172.21.15.118:3300/0 conn(0x7f4e4413df80 0x7f4e4415bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:42.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.557+0000 7f4e39d9b640 1 -- 172.21.15.92:0/2968886509 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4e4415f9c0 msgr2=0x7f4e440cf7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:42.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.557+0000 7f4e39d9b640 1 --2- 172.21.15.92:0/2968886509 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4e4415f9c0 0x7f4e440cf7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:42.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.557+0000 7f4e39d9b640 1 -- 172.21.15.92:0/2968886509 >> v2:172.21.15.118:3300/0 conn(0x7f4e4413df80 msgr2=0x7f4e4415bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:42.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.557+0000 7f4e39d9b640 1 --2- 172.21.15.92:0/2968886509 >> v2:172.21.15.118:3300/0 conn(0x7f4e4413df80 0x7f4e4415bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:42.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.557+0000 7f4e39d9b640 1 -- 172.21.15.92:0/2968886509 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e4406cd10 con 0x7f4e4413e360 2023-11-24T06:10:42.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.558+0000 7f4e39d9b640 1 --2- 172.21.15.92:0/2968886509 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4e4413e360 0x7f4e440cf290 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f4e44072770 tx=0x7f4e2c02b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6aedb8fe0e632959 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:42.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.558+0000 7f4e41d9b640 1 -- 172.21.15.92:0/2968886509 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e2c031860 con 0x7f4e4413e360 2023-11-24T06:10:42.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.558+0000 7f4e41d9b640 1 -- 172.21.15.92:0/2968886509 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e2c0319c0 con 0x7f4e4413e360 2023-11-24T06:10:42.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.558+0000 7f4e41d9b640 1 -- 172.21.15.92:0/2968886509 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e2c031d50 con 0x7f4e4413e360 2023-11-24T06:10:42.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.559+0000 7f4e48c48640 1 -- 172.21.15.92:0/2968886509 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4e4413e360 msgr2=0x7f4e440cf290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:42.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.559+0000 7f4e48c48640 1 --2- 172.21.15.92:0/2968886509 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4e4413e360 0x7f4e440cf290 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f4e44072770 tx=0x7f4e2c02b900 comp rx=0 tx=0).stop 2023-11-24T06:10:42.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.559+0000 7f4e48c48640 1 -- 172.21.15.92:0/2968886509 shutdown_connections 2023-11-24T06:10:42.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.559+0000 7f4e48c48640 1 --2- 172.21.15.92:0/2968886509 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4e4415f9c0 0x7f4e440cf7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:42.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.559+0000 7f4e48c48640 1 --2- 172.21.15.92:0/2968886509 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4e4413e360 0x7f4e440cf290 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:42.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.559+0000 7f4e48c48640 1 --2- 172.21.15.92:0/2968886509 >> v2:172.21.15.118:3300/0 conn(0x7f4e4413df80 0x7f4e4415bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:42.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.559+0000 7f4e48c48640 1 -- 172.21.15.92:0/2968886509 >> 172.21.15.92:0/2968886509 conn(0x7f4e4406a800 msgr2=0x7f4e4406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:42.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.559+0000 7f4e48c48640 1 -- 172.21.15.92:0/2968886509 shutdown_connections 2023-11-24T06:10:42.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.559+0000 7f4e48c48640 1 -- 172.21.15.92:0/2968886509 wait complete. 2023-11-24T06:10:42.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.561+0000 7f4e48c48640 1 Processor -- start 2023-11-24T06:10:42.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.561+0000 7f4e48c48640 1 -- start start 2023-11-24T06:10:42.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.562+0000 7f4e48c48640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4e4413df80 0x7f4e440ade60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:42.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.562+0000 7f4e48c48640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4e4413e360 0x7f4e441331d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:42.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.562+0000 7f4e48c48640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4e4415f9c0 0x7f4e441f4d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:42.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.562+0000 7f4e48c48640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e441f5260 con 0x7f4e4413e360 2023-11-24T06:10:42.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.562+0000 7f4e48c48640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e44155e10 con 0x7f4e4413df80 2023-11-24T06:10:42.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.562+0000 7f4e48c48640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e441560f0 con 0x7f4e4415f9c0 2023-11-24T06:10:42.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.562+0000 7f4e39d9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4e4413e360 0x7f4e441331d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:42.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.562+0000 7f4e39d9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4e4413e360 0x7f4e441331d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47082/0 (socket says 172.21.15.92:47082) 2023-11-24T06:10:42.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.562+0000 7f4e39d9b640 1 -- 172.21.15.92:0/3992929268 learned_addr learned my addr 172.21.15.92:0/3992929268 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:10:42.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.562+0000 7f4e39d9b640 1 -- 172.21.15.92:0/3992929268 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4e4415f9c0 msgr2=0x7f4e441f4d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:42.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.562+0000 7f4e4259c640 1 --2- 172.21.15.92:0/3992929268 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4e4413df80 0x7f4e440ade60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:42.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.562+0000 7f4e42d9d640 1 --2- 172.21.15.92:0/3992929268 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4e4415f9c0 0x7f4e441f4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:42.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.562+0000 7f4e39d9b640 1 --2- 172.21.15.92:0/3992929268 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4e4415f9c0 0x7f4e441f4d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:42.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.562+0000 7f4e39d9b640 1 -- 172.21.15.92:0/3992929268 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4e4413df80 msgr2=0x7f4e440ade60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:42.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.562+0000 7f4e39d9b640 1 --2- 172.21.15.92:0/3992929268 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4e4413df80 0x7f4e440ade60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:42.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.562+0000 7f4e39d9b640 1 -- 172.21.15.92:0/3992929268 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e2c0094d0 con 0x7f4e4413e360 2023-11-24T06:10:42.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.562+0000 7f4e42d9d640 1 --2- 172.21.15.92:0/3992929268 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4e4415f9c0 0x7f4e441f4d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:10:42.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.562+0000 7f4e4259c640 1 --2- 172.21.15.92:0/3992929268 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4e4413df80 0x7f4e440ade60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:10:42.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.563+0000 7f4e39d9b640 1 --2- 172.21.15.92:0/3992929268 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4e4413e360 0x7f4e441331d0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f4e2c000c00 tx=0x7f4e2c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:42.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.563+0000 7f4e3b7fe640 1 -- 172.21.15.92:0/3992929268 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e2c03ac60 con 0x7f4e4413e360 2023-11-24T06:10:42.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.563+0000 7f4e3b7fe640 1 -- 172.21.15.92:0/3992929268 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e2c03adc0 con 0x7f4e4413e360 2023-11-24T06:10:42.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.563+0000 7f4e48c48640 1 -- 172.21.15.92:0/3992929268 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e441563d0 con 0x7f4e4413e360 2023-11-24T06:10:42.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.563+0000 7f4e3b7fe640 1 -- 172.21.15.92:0/3992929268 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e2c044610 con 0x7f4e4413e360 2023-11-24T06:10:42.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.563+0000 7f4e48c48640 1 -- 172.21.15.92:0/3992929268 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e44156990 con 0x7f4e4413e360 2023-11-24T06:10:42.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.565+0000 7f4e3b7fe640 1 -- 172.21.15.92:0/3992929268 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f4e2c04d450 con 0x7f4e4413e360 2023-11-24T06:10:42.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.566+0000 7f4e3b7fe640 1 --2- 172.21.15.92:0/3992929268 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f4e2406ae40 0x7f4e2406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:42.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.566+0000 7f4e3b7fe640 1 -- 172.21.15.92:0/3992929268 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4e2c0abc00 con 0x7f4e4413e360 2023-11-24T06:10:42.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.567+0000 7f4e4259c640 1 --2- 172.21.15.92:0/3992929268 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f4e2406ae40 0x7f4e2406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:42.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.567+0000 7f4e4259c640 1 --2- 172.21.15.92:0/3992929268 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f4e2406ae40 0x7f4e2406d300 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f4e34006fd0 tx=0x7f4e34008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:42.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.567+0000 7f4e4b7da740 1 Processor -- start 2023-11-24T06:10:42.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.568+0000 7f4e4b7da740 1 -- start start 2023-11-24T06:10:42.572 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.568+0000 7f4e4b7da740 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x561d801c2ea0 0x561d801c3280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:42.572 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.568+0000 7f4e4b7da740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x561d801bdba0 0x561d801c20a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:42.572 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.568+0000 7f4e4b7da740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561d801c25e0 0x561d80206560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:42.572 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.568+0000 7f4e4b7da740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d80121880 con 0x561d801c25e0 2023-11-24T06:10:42.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.568+0000 7f4e4b7da740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d80130ae0 con 0x561d801bdba0 2023-11-24T06:10:42.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.568+0000 7f4e4b7da740 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x561d8012e1d0 con 0x561d801c2ea0 2023-11-24T06:10:42.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.569+0000 7f4e42d9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561d801c25e0 0x561d80206560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:42.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.569+0000 7f4e42d9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561d801c25e0 0x561d80206560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47088/0 (socket says 172.21.15.92:47088) 2023-11-24T06:10:42.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.569+0000 7f4e42d9d640 1 -- 172.21.15.92:0/2543173744 learned_addr learned my addr 172.21.15.92:0/2543173744 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:10:42.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.569+0000 7f4e4259c640 1 --2- 172.21.15.92:0/2543173744 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x561d801c2ea0 0x561d801c3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:42.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.569+0000 7f4e39d9b640 1 --2- 172.21.15.92:0/2543173744 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x561d801bdba0 0x561d801c20a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:42.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.569+0000 7f4e42d9d640 1 -- 172.21.15.92:0/2543173744 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x561d801c2ea0 msgr2=0x561d801c3280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:42.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.569+0000 7f4e42d9d640 1 --2- 172.21.15.92:0/2543173744 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x561d801c2ea0 0x561d801c3280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:42.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.569+0000 7f4e42d9d640 1 -- 172.21.15.92:0/2543173744 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x561d801bdba0 msgr2=0x561d801c20a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:42.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.569+0000 7f4e42d9d640 1 --2- 172.21.15.92:0/2543173744 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x561d801bdba0 0x561d801c20a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:42.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.569+0000 7f4e42d9d640 1 -- 172.21.15.92:0/2543173744 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561d800f1860 con 0x561d801c25e0 2023-11-24T06:10:42.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.569+0000 7f4e42d9d640 1 --2- 172.21.15.92:0/2543173744 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561d801c25e0 0x561d80206560 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f4e3c007f60 tx=0x7f4e3c007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d51d0973765ed26 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:42.575 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.570+0000 7f4e23fff640 1 -- 172.21.15.92:0/2543173744 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e3c00f850 con 0x561d801c25e0 2023-11-24T06:10:42.575 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.570+0000 7f4e23fff640 1 -- 172.21.15.92:0/2543173744 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e3c00f9b0 con 0x561d801c25e0 2023-11-24T06:10:42.575 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.570+0000 7f4e23fff640 1 -- 172.21.15.92:0/2543173744 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e3c00fd40 con 0x561d801c25e0 2023-11-24T06:10:42.575 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.570+0000 7f4e4b7da740 1 -- 172.21.15.92:0/2543173744 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561d801c25e0 msgr2=0x561d80206560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:42.575 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.570+0000 7f4e4b7da740 1 --2- 172.21.15.92:0/2543173744 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561d801c25e0 0x561d80206560 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f4e3c007f60 tx=0x7f4e3c007f90 comp rx=0 tx=0).stop 2023-11-24T06:10:42.575 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.570+0000 7f4e4b7da740 1 -- 172.21.15.92:0/2543173744 shutdown_connections 2023-11-24T06:10:42.575 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.570+0000 7f4e4b7da740 1 --2- 172.21.15.92:0/2543173744 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561d801c25e0 0x561d80206560 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:42.575 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.570+0000 7f4e4b7da740 1 --2- 172.21.15.92:0/2543173744 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x561d801bdba0 0x561d801c20a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:42.575 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.570+0000 7f4e4b7da740 1 --2- 172.21.15.92:0/2543173744 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x561d801c2ea0 0x561d801c3280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:42.575 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.570+0000 7f4e4b7da740 1 -- 172.21.15.92:0/2543173744 >> 172.21.15.92:0/2543173744 conn(0x561d80184c70 msgr2=0x561d80208990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:42.576 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.571+0000 7f4e4b7da740 1 -- 172.21.15.92:0/2543173744 shutdown_connections 2023-11-24T06:10:42.576 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.571+0000 7f4e4b7da740 1 -- 172.21.15.92:0/2543173744 wait complete. 2023-11-24T06:10:42.576 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.571+0000 7f4e4b7da740 1 -- start start 2023-11-24T06:10:42.576 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.572+0000 7f4e4b7da740 1 Processor -- start 2023-11-24T06:10:42.578 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.573+0000 7f4e4b7da740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561d801bdba0 0x561d80200e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:42.578 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.573+0000 7f4e4b7da740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x561d801c25e0 0x561d802013c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:42.578 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.573+0000 7f4e4b7da740 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x561d801c2ea0 0x561d801ffa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:42.578 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.573+0000 7f4e4b7da740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d80129ac0 con 0x561d801bdba0 2023-11-24T06:10:42.578 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.573+0000 7f4e4b7da740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d800c62f0 con 0x561d801c25e0 2023-11-24T06:10:42.578 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.573+0000 7f4e4b7da740 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x561d800e3c70 con 0x561d801c2ea0 2023-11-24T06:10:42.578 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.573+0000 7f4e4b7da740 10 client.0 _ll_register_callbacks cb 0x561d800e83b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-24T06:10:42.578 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.573+0000 7f4e4259c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561d801bdba0 0x561d80200e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:42.578 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.573+0000 7f4e4259c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561d801bdba0 0x561d80200e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47090/0 (socket says 172.21.15.92:47090) 2023-11-24T06:10:42.578 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.573+0000 7f4e4259c640 1 -- 172.21.15.92:0/1314483214 learned_addr learned my addr 172.21.15.92:0/1314483214 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:10:42.578 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.573+0000 7f4e4259c640 1 -- 172.21.15.92:0/1314483214 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x561d801c2ea0 msgr2=0x561d801ffa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:42.579 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.573+0000 7f4e4259c640 1 --2- 172.21.15.92:0/1314483214 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x561d801c2ea0 0x561d801ffa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:42.579 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.573+0000 7f4e4259c640 1 -- 172.21.15.92:0/1314483214 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x561d801c25e0 msgr2=0x561d802013c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:42.579 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.573+0000 7f4e4259c640 1 --2- 172.21.15.92:0/1314483214 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x561d801c25e0 0x561d802013c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:42.579 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.573+0000 7f4e4259c640 1 -- 172.21.15.92:0/1314483214 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e3c007b30 con 0x561d801bdba0 2023-11-24T06:10:42.579 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.573+0000 7f4e4259c640 1 --2- 172.21.15.92:0/1314483214 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561d801bdba0 0x561d80200e80 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f4e3400ea20 tx=0x7f4e340046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50dca3e2c7a5e229 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:42.579 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.573+0000 7f4e217fa640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-11-24T06:10:42.579 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.574+0000 7f4e217fa640 1 -- 172.21.15.92:0/1314483214 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e3400eee0 con 0x561d801bdba0 2023-11-24T06:10:42.579 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.574+0000 7f4e217fa640 1 -- 172.21.15.92:0/1314483214 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4e34013430 con 0x561d801bdba0 2023-11-24T06:10:42.579 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.574+0000 7f4e4b7da740 1 -- 172.21.15.92:0/1314483214 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561d80122aa0 con 0x561d801bdba0 2023-11-24T06:10:42.579 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.574+0000 7f4e217fa640 1 -- 172.21.15.92:0/1314483214 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4e340135f0 con 0x561d801bdba0 2023-11-24T06:10:42.579 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.574+0000 7f4e4b7da740 10 client.4635 fetch_fsmap learned FSMap version 8 2023-11-24T06:10:42.579 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.574+0000 7f4e4b7da740 1 -- 172.21.15.92:0/1314483214 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) v3 -- 0x561d80128140 con 0x561d801bdba0 2023-11-24T06:10:42.579 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.574+0000 7f4e217fa640 1 -- 172.21.15.92:0/1314483214 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x561d80122aa0 con 0x561d801bdba0 2023-11-24T06:10:42.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.574+0000 7f4e217fa640 1 -- 172.21.15.92:0/1314483214 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 8) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f4e34013750 con 0x561d801bdba0 2023-11-24T06:10:42.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.574+0000 7f4e4b7da740 10 client.4635 fetch_fsmap finished waiting for FSMap version 8 2023-11-24T06:10:42.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.574+0000 7f4e4b7da740 10 client.4635 resolve_mds: resolved name 'd' to daemon mds.d 2023-11-24T06:10:42.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.574+0000 7f4e4b7da740 20 client.4635 populate_metadata read hostname 'smithi092' 2023-11-24T06:10:42.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.574+0000 7f4e4b7da740 1 --2- 172.21.15.92:0/1314483214 >> [v2:172.21.15.118:6832/1146052262,v1:172.21.15.118:6833/1146052262] conn(0x561d802005b0 0x561d80225cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:42.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.574+0000 7f4e4b7da740 4 client.4635 mds_command: new command op to 4264 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-24T06:10:42.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.574+0000 7f4e4b7da740 1 -- 172.21.15.92:0/1314483214 --> [v2:172.21.15.118:6832/1146052262,v1:172.21.15.118:6833/1146052262] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561d801227c0 con 0x561d802005b0 2023-11-24T06:10:42.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.575+0000 7f4e39d9b640 1 --2- 172.21.15.92:0/1314483214 >> [v2:172.21.15.118:6832/1146052262,v1:172.21.15.118:6833/1146052262] conn(0x561d802005b0 0x561d80225cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:42.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.576+0000 7f4e39d9b640 1 --2- 172.21.15.92:0/1314483214 >> [v2:172.21.15.118:6832/1146052262,v1:172.21.15.118:6833/1146052262] conn(0x561d802005b0 0x561d80225cb0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=34afafc3a1ba925a server_cookie=497118d81a69085e in_seq=0 out_seq=0 2023-11-24T06:10:42.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.576+0000 7f4e217fa640 10 client.4635 ms_handle_connect on v2:172.21.15.118:6832/1146052262 2023-11-24T06:10:42.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.578+0000 7f4e217fa640 1 -- 172.21.15.92:0/1314483214 <== mds.0 v2:172.21.15.118:6832/1146052262 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x561d801227c0 con 0x561d802005b0 2023-11-24T06:10:42.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.578+0000 7f4e217fa640 10 client.4635 handle_command_reply: tid=0 2023-11-24T06:10:42.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.578+0000 7f4e4b7da740 1 client.4635 shutdown 2023-11-24T06:10:42.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.579+0000 7f4e4b7da740 1 -- 172.21.15.92:0/1314483214 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561d801bdba0 msgr2=0x561d80200e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:42.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.579+0000 7f4e4b7da740 1 --2- 172.21.15.92:0/1314483214 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561d801bdba0 0x561d80200e80 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f4e3400ea20 tx=0x7f4e340046c0 comp rx=0 tx=0).stop 2023-11-24T06:10:42.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.579+0000 7f4e4b7da740 1 -- 172.21.15.92:0/1314483214 shutdown_connections 2023-11-24T06:10:42.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.579+0000 7f4e4b7da740 1 --2- 172.21.15.92:0/1314483214 >> [v2:172.21.15.118:6832/1146052262,v1:172.21.15.118:6833/1146052262] conn(0x561d802005b0 0x561d80225cb0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:42.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.579+0000 7f4e4b7da740 1 --2- 172.21.15.92:0/1314483214 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x561d801c2ea0 0x561d801ffa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:42.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.579+0000 7f4e4b7da740 1 --2- 172.21.15.92:0/1314483214 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x561d801c25e0 0x561d802013c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:42.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.579+0000 7f4e217fa640 0 client.4635 ms_handle_reset on v2:172.21.15.118:6832/1146052262 2023-11-24T06:10:42.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.579+0000 7f4e4b7da740 1 --2- 172.21.15.92:0/1314483214 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561d801bdba0 0x561d80200e80 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:42.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.579+0000 7f4e4b7da740 1 -- 172.21.15.92:0/1314483214 >> 172.21.15.92:0/1314483214 conn(0x561d80184c70 msgr2=0x561d801c3280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:42.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.579+0000 7f4e4b7da740 1 -- 172.21.15.92:0/1314483214 shutdown_connections 2023-11-24T06:10:42.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.579+0000 7f4e4b7da740 1 -- 172.21.15.92:0/1314483214 wait complete. 2023-11-24T06:10:42.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.579+0000 7f4e4b7da740 20 client.4635 trim_cache size 0 max 16384 2023-11-24T06:10:42.599 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.596+0000 7f4e4b7da740 1 Processor -- start 2023-11-24T06:10:42.599 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.596+0000 7f4e4b7da740 1 -- start start 2023-11-24T06:10:42.600 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.596+0000 7f4e4b7da740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x561d801bdba0 0x561d801c1790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:42.600 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.597+0000 7f4e4b7da740 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x561d801c25e0 0x561d801c1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:42.600 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.597+0000 7f4e4b7da740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561d801c2ea0 0x561d8020ec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:42.600 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.597+0000 7f4e4b7da740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d7fe8cf20 con 0x561d801c2ea0 2023-11-24T06:10:42.600 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.597+0000 7f4e4b7da740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d801ffe90 con 0x561d801bdba0 2023-11-24T06:10:42.600 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.597+0000 7f4e4b7da740 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x561d80229fb0 con 0x561d801c25e0 2023-11-24T06:10:42.600 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.597+0000 7f4e42d9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561d801c2ea0 0x561d8020ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:42.601 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.597+0000 7f4e42d9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561d801c2ea0 0x561d8020ec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47100/0 (socket says 172.21.15.92:47100) 2023-11-24T06:10:42.601 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.597+0000 7f4e42d9d640 1 -- 172.21.15.92:0/2337982456 learned_addr learned my addr 172.21.15.92:0/2337982456 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:10:42.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.597+0000 7f4e42d9d640 1 -- 172.21.15.92:0/2337982456 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x561d801c25e0 msgr2=0x561d801c1cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:42.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.597+0000 7f4e42d9d640 1 --2- 172.21.15.92:0/2337982456 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x561d801c25e0 0x561d801c1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:42.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.597+0000 7f4e4259c640 1 --2- 172.21.15.92:0/2337982456 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x561d801bdba0 0x561d801c1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:42.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.597+0000 7f4e42d9d640 1 -- 172.21.15.92:0/2337982456 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x561d801bdba0 msgr2=0x561d801c1790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:42.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.597+0000 7f4e42d9d640 1 --2- 172.21.15.92:0/2337982456 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x561d801bdba0 0x561d801c1790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:42.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.597+0000 7f4e42d9d640 1 -- 172.21.15.92:0/2337982456 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561d800c5b80 con 0x561d801c2ea0 2023-11-24T06:10:42.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.597+0000 7f4e4259c640 1 --2- 172.21.15.92:0/2337982456 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x561d801bdba0 0x561d801c1790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:10:42.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.598+0000 7f4e42d9d640 1 --2- 172.21.15.92:0/2337982456 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561d801c2ea0 0x561d8020ec60 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f4e3c013000 tx=0x7f4e3c00f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c1f00b63e545939 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:42.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.598+0000 7f4e38d99640 1 -- 172.21.15.92:0/2337982456 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e3c007510 con 0x561d801c2ea0 2023-11-24T06:10:42.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.598+0000 7f4e38d99640 1 -- 172.21.15.92:0/2337982456 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e3c01f040 con 0x561d801c2ea0 2023-11-24T06:10:42.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.598+0000 7f4e38d99640 1 -- 172.21.15.92:0/2337982456 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e3c00fb20 con 0x561d801c2ea0 2023-11-24T06:10:42.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.598+0000 7f4e4b7da740 1 -- 172.21.15.92:0/2337982456 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561d801c2ea0 msgr2=0x561d8020ec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:42.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.598+0000 7f4e4b7da740 1 --2- 172.21.15.92:0/2337982456 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561d801c2ea0 0x561d8020ec60 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f4e3c013000 tx=0x7f4e3c00f000 comp rx=0 tx=0).stop 2023-11-24T06:10:42.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.599+0000 7f4e4b7da740 1 -- 172.21.15.92:0/2337982456 shutdown_connections 2023-11-24T06:10:42.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.599+0000 7f4e4b7da740 1 --2- 172.21.15.92:0/2337982456 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561d801c2ea0 0x561d8020ec60 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:42.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.599+0000 7f4e4b7da740 1 --2- 172.21.15.92:0/2337982456 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x561d801c25e0 0x561d801c1cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:42.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.599+0000 7f4e4b7da740 1 --2- 172.21.15.92:0/2337982456 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x561d801bdba0 0x561d801c1790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:42.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.599+0000 7f4e4b7da740 1 -- 172.21.15.92:0/2337982456 >> 172.21.15.92:0/2337982456 conn(0x561d80184c70 msgr2=0x561d801e54e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:42.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.599+0000 7f4e4b7da740 1 -- 172.21.15.92:0/2337982456 shutdown_connections 2023-11-24T06:10:42.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.599+0000 7f4e4b7da740 1 -- 172.21.15.92:0/2337982456 wait complete. 2023-11-24T06:10:42.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.599+0000 7f4e4b7da740 1 -- start start 2023-11-24T06:10:42.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.599+0000 7f4e4b7da740 1 Processor -- start 2023-11-24T06:10:42.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.600+0000 7f4e4b7da740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561d801bdba0 0x561d80274230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:42.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.600+0000 7f4e4b7da740 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x561d801c25e0 0x561d8020f410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:42.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.600+0000 7f4e4b7da740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x561d801c2ea0 0x561d801c1980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:42.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.600+0000 7f4e4b7da740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d80200df0 con 0x561d801bdba0 2023-11-24T06:10:42.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.600+0000 7f4e4b7da740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d801e5e50 con 0x561d801c2ea0 2023-11-24T06:10:42.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.600+0000 7f4e4b7da740 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x561d8020c3e0 con 0x561d801c25e0 2023-11-24T06:10:42.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.601+0000 7f4e4b7da740 10 client.0 _ll_register_callbacks cb 0x7f4e34009d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-24T06:10:42.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.601+0000 7f4e4259c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561d801bdba0 0x561d80274230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:42.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.601+0000 7f4e4259c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561d801bdba0 0x561d80274230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47114/0 (socket says 172.21.15.92:47114) 2023-11-24T06:10:42.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.601+0000 7f4e4259c640 1 -- 172.21.15.92:0/2998234798 learned_addr learned my addr 172.21.15.92:0/2998234798 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:10:42.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.601+0000 7f4e4259c640 1 -- 172.21.15.92:0/2998234798 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x561d801c25e0 msgr2=0x561d8020f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:42.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.601+0000 7f4e4259c640 1 --2- 172.21.15.92:0/2998234798 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x561d801c25e0 0x561d8020f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:42.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.601+0000 7f4e4259c640 1 -- 172.21.15.92:0/2998234798 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x561d801c2ea0 msgr2=0x561d801c1980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:42.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.601+0000 7f4e4259c640 1 --2- 172.21.15.92:0/2998234798 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x561d801c2ea0 0x561d801c1980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:42.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.601+0000 7f4e4259c640 1 -- 172.21.15.92:0/2998234798 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561d800f1860 con 0x561d801bdba0 2023-11-24T06:10:42.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.601+0000 7f4e4259c640 1 --2- 172.21.15.92:0/2998234798 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561d801bdba0 0x561d80274230 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f4e34013f40 tx=0x7f4e34013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd6b9959d0973819 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:42.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.601+0000 7f4e217fa640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-11-24T06:10:42.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.601+0000 7f4e217fa640 1 -- 172.21.15.92:0/2998234798 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e34013630 con 0x561d801bdba0 2023-11-24T06:10:42.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.602+0000 7f4e217fa640 1 -- 172.21.15.92:0/2998234798 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4e3400e5c0 con 0x561d801bdba0 2023-11-24T06:10:42.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.602+0000 7f4e4b7da740 1 -- 172.21.15.92:0/2998234798 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4e3c00f9b0 con 0x561d801bdba0 2023-11-24T06:10:42.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.602+0000 7f4e217fa640 1 -- 172.21.15.92:0/2998234798 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4e34013790 con 0x561d801bdba0 2023-11-24T06:10:42.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.602+0000 7f4e4b7da740 10 client.4641 fetch_fsmap learned FSMap version 8 2023-11-24T06:10:42.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.602+0000 7f4e4b7da740 1 -- 172.21.15.92:0/2998234798 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) v3 -- 0x561d800eb170 con 0x561d801bdba0 2023-11-24T06:10:42.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.602+0000 7f4e217fa640 1 -- 172.21.15.92:0/2998234798 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e3c00f9b0 con 0x561d801bdba0 2023-11-24T06:10:42.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.602+0000 7f4e217fa640 1 -- 172.21.15.92:0/2998234798 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 8) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f4e34005990 con 0x561d801bdba0 2023-11-24T06:10:42.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.602+0000 7f4e4b7da740 10 client.4641 fetch_fsmap finished waiting for FSMap version 8 2023-11-24T06:10:42.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.602+0000 7f4e4b7da740 10 client.4641 resolve_mds: resolved name 'd' to daemon mds.d 2023-11-24T06:10:42.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.602+0000 7f4e4b7da740 20 client.4641 populate_metadata read hostname 'smithi092' 2023-11-24T06:10:42.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.602+0000 7f4e4b7da740 1 --2- 172.21.15.92:0/2998234798 >> [v2:172.21.15.118:6832/1146052262,v1:172.21.15.118:6833/1146052262] conn(0x561d801e8660 0x561d801e9700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:42.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.602+0000 7f4e4b7da740 4 client.4641 mds_command: new command op to 4264 tid=0 multi_id=0 [{"prefix": "status"}] 2023-11-24T06:10:42.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.602+0000 7f4e4b7da740 1 -- 172.21.15.92:0/2998234798 --> [v2:172.21.15.118:6832/1146052262,v1:172.21.15.118:6833/1146052262] -- command(tid 0: {"prefix": "status"}) v1 -- 0x561d7fe2a6c0 con 0x561d801e8660 2023-11-24T06:10:42.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.603+0000 7f4e39d9b640 1 --2- 172.21.15.92:0/2998234798 >> [v2:172.21.15.118:6832/1146052262,v1:172.21.15.118:6833/1146052262] conn(0x561d801e8660 0x561d801e9700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:42.610 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.603+0000 7f4e39d9b640 1 --2- 172.21.15.92:0/2998234798 >> [v2:172.21.15.118:6832/1146052262,v1:172.21.15.118:6833/1146052262] conn(0x561d801e8660 0x561d801e9700 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b04ae1b14725d3d5 server_cookie=f8e4be447f7fdebd in_seq=0 out_seq=0 2023-11-24T06:10:42.610 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.603+0000 7f4e217fa640 10 client.4641 ms_handle_connect on v2:172.21.15.118:6832/1146052262 2023-11-24T06:10:42.610 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.604+0000 7f4e217fa640 1 -- 172.21.15.92:0/2998234798 <== mds.0 v2:172.21.15.118:6832/1146052262 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+324 (crc 0 0 0) 0x561d7fe2a6c0 con 0x561d801e8660 2023-11-24T06:10:42.610 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.604+0000 7f4e217fa640 10 client.4641 handle_command_reply: tid=0 2023-11-24T06:10:42.610 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.604+0000 7f4e4b7da740 1 client.4641 shutdown 2023-11-24T06:10:42.610 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.604+0000 7f4e4b7da740 1 -- 172.21.15.92:0/2998234798 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561d801bdba0 msgr2=0x561d80274230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:42.611 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.604+0000 7f4e4b7da740 1 --2- 172.21.15.92:0/2998234798 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561d801bdba0 0x561d80274230 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f4e34013f40 tx=0x7f4e34013000 comp rx=0 tx=0).stop 2023-11-24T06:10:42.611 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.604+0000 7f4e4b7da740 1 -- 172.21.15.92:0/2998234798 shutdown_connections 2023-11-24T06:10:42.611 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.604+0000 7f4e4b7da740 1 --2- 172.21.15.92:0/2998234798 >> [v2:172.21.15.118:6832/1146052262,v1:172.21.15.118:6833/1146052262] conn(0x561d801e8660 0x561d801e9700 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:42.611 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.604+0000 7f4e4b7da740 1 --2- 172.21.15.92:0/2998234798 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x561d801c2ea0 0x561d801c1980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:42.611 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.604+0000 7f4e4b7da740 1 --2- 172.21.15.92:0/2998234798 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x561d801c25e0 0x561d8020f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:42.611 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.604+0000 7f4e217fa640 0 client.4641 ms_handle_reset on v2:172.21.15.118:6832/1146052262 2023-11-24T06:10:42.611 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.604+0000 7f4e4b7da740 1 --2- 172.21.15.92:0/2998234798 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561d801bdba0 0x561d80274230 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:42.611 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.604+0000 7f4e4b7da740 1 -- 172.21.15.92:0/2998234798 >> 172.21.15.92:0/2998234798 conn(0x561d80184c70 msgr2=0x561d80201840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:42.611 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.604+0000 7f4e4b7da740 1 -- 172.21.15.92:0/2998234798 shutdown_connections 2023-11-24T06:10:42.611 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.604+0000 7f4e4b7da740 1 -- 172.21.15.92:0/2998234798 wait complete. 2023-11-24T06:10:42.611 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.605+0000 7f4e4b7da740 20 client.4641 trim_cache size 0 max 16384 2023-11-24T06:10:42.612 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-24T06:10:42.612 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_fsid": "2d6dee5f-8aa3-470f-931a-50ade963b1cb", 2023-11-24T06:10:42.612 INFO:teuthology.orchestra.run.smithi092.stdout: "whoami": 0, 2023-11-24T06:10:42.612 INFO:teuthology.orchestra.run.smithi092.stdout: "id": 4264, 2023-11-24T06:10:42.612 INFO:teuthology.orchestra.run.smithi092.stdout: "want_state": "up:active", 2023-11-24T06:10:42.612 INFO:teuthology.orchestra.run.smithi092.stdout: "state": "up:active", 2023-11-24T06:10:42.612 INFO:teuthology.orchestra.run.smithi092.stdout: "fs_name": "cephfs", 2023-11-24T06:10:42.612 INFO:teuthology.orchestra.run.smithi092.stdout: "rank_uptime": 16.894424493999999, 2023-11-24T06:10:42.612 INFO:teuthology.orchestra.run.smithi092.stdout: "mdsmap_epoch": 8, 2023-11-24T06:10:42.613 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_epoch": 20, 2023-11-24T06:10:42.613 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_epoch_barrier": 20, 2023-11-24T06:10:42.613 INFO:teuthology.orchestra.run.smithi092.stdout: "uptime": 24.662131914 2023-11-24T06:10:42.613 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-24T06:10:42.613 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.605+0000 7f4e48c48640 1 -- 172.21.15.92:0/3992929268 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f4e2406ae40 msgr2=0x7f4e2406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:42.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.605+0000 7f4e48c48640 1 --2- 172.21.15.92:0/3992929268 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f4e2406ae40 0x7f4e2406d300 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f4e34006fd0 tx=0x7f4e34008040 comp rx=0 tx=0).stop 2023-11-24T06:10:42.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.605+0000 7f4e48c48640 1 -- 172.21.15.92:0/3992929268 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4e4413e360 msgr2=0x7f4e441331d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:42.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.605+0000 7f4e48c48640 1 --2- 172.21.15.92:0/3992929268 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4e4413e360 0x7f4e441331d0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f4e2c000c00 tx=0x7f4e2c0046c0 comp rx=0 tx=0).stop 2023-11-24T06:10:42.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.605+0000 7f4e48c48640 1 -- 172.21.15.92:0/3992929268 shutdown_connections 2023-11-24T06:10:42.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.605+0000 7f4e48c48640 1 --2- 172.21.15.92:0/3992929268 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f4e2406ae40 0x7f4e2406d300 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:42.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.605+0000 7f4e48c48640 1 --2- 172.21.15.92:0/3992929268 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4e4415f9c0 0x7f4e441f4d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:42.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.605+0000 7f4e48c48640 1 --2- 172.21.15.92:0/3992929268 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4e4413e360 0x7f4e441331d0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:42.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.605+0000 7f4e48c48640 1 --2- 172.21.15.92:0/3992929268 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4e4413df80 0x7f4e440ade60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:42.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.605+0000 7f4e48c48640 1 -- 172.21.15.92:0/3992929268 >> 172.21.15.92:0/3992929268 conn(0x7f4e4406a800 msgr2=0x7f4e4413f1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:42.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.605+0000 7f4e48c48640 1 -- 172.21.15.92:0/3992929268 shutdown_connections 2023-11-24T06:10:42.615 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:42.605+0000 7f4e48c48640 1 -- 172.21.15.92:0/3992929268 wait complete. 2023-11-24T06:10:42.624 INFO:teuthology.run_tasks:Running task ceph-fuse... 2023-11-24T06:10:42.648 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2023-11-24T06:10:42.648 INFO:tasks.ceph_fuse:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2023-11-24T06:10:42.648 INFO:tasks.ceph_fuse:client.0 config is {} 2023-11-24T06:10:42.648 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-11-24T06:10:42.648 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-24T06:10:42.648 INFO:tasks.ceph_fuse:client.2 config is {} 2023-11-24T06:10:42.648 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-11-24T06:10:42.649 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2023-11-24T06:10:42.649 INFO:tasks.ceph_fuse:client.1 config is {} 2023-11-24T06:10:42.649 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-11-24T06:10:42.649 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-11-24T06:10:42.649 INFO:tasks.ceph_fuse:client.3 config is {} 2023-11-24T06:10:42.649 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-11-24T06:10:42.649 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2023-11-24T06:10:42.649 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:10:42.649 DEBUG:teuthology.orchestra.run.smithi092:> ip netns list 2023-11-24T06:10:42.703 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:10:42.704 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip link delete ceph-brx 2023-11-24T06:10:42.798 INFO:teuthology.orchestra.run.smithi092.stderr:Cannot find device "ceph-brx" 2023-11-24T06:10:42.802 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-24T06:10:42.803 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:10:42.803 DEBUG:teuthology.orchestra.run.smithi118:> ip netns list 2023-11-24T06:10:42.866 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:10:42.866 DEBUG:teuthology.orchestra.run.smithi118:> sudo ip link delete ceph-brx 2023-11-24T06:10:42.970 INFO:teuthology.orchestra.run.smithi118.stderr:Cannot find device "ceph-brx" 2023-11-24T06:10:42.974 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-24T06:10:42.975 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2023-11-24T06:10:42.975 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-24T06:10:43.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:43.192+0000 7f50c224b640 1 Processor -- start 2023-11-24T06:10:43.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:43.192+0000 7f50c224b640 1 -- start start 2023-11-24T06:10:43.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:43.193+0000 7f50c224b640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f50bc162430 0x7f50bc0b3110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:43.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:43.193+0000 7f50c224b640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f50bc162810 0x7f50bc071240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:43.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:43.194+0000 7f50c224b640 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50bc0702e0 con 0x7f50bc0b3650 2023-11-24T06:10:43.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:43.194+0000 7f50c224b640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50bc0705c0 con 0x7f50bc162810 2023-11-24T06:10:43.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:43.194+0000 7f50c224b640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50bc0708a0 con 0x7f50bc162430 2023-11-24T06:10:43.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:43.194+0000 7f50c0a48640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f50bc162810 0x7f50bc071240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:43.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:43.194+0000 7f50c0a48640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f50bc162810 0x7f50bc071240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47116/0 (socket says 172.21.15.92:47116) 2023-11-24T06:10:43.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:43.194+0000 7f50c0a48640 1 -- 172.21.15.92:0/1434284791 learned_addr learned my addr 172.21.15.92:0/1434284791 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:10:43.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:43.194+0000 7f50c1249640 1 --2- 172.21.15.92:0/1434284791 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f50bc162430 0x7f50bc0b3110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:43.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:43.194+0000 7f50c0a48640 1 -- 172.21.15.92:0/1434284791 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f50bc162430 msgr2=0x7f50bc0b3110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:43.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:43.194+0000 7f50c0a48640 1 --2- 172.21.15.92:0/1434284791 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f50bc162430 0x7f50bc0b3110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:43.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:43.194+0000 7f50c0a48640 1 -- 172.21.15.92:0/1434284791 >> v1:172.21.15.118:6789/0 conn(0x7f50bc0b3650 legacy=0x7f50bc1650c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:43.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:43.194+0000 7f50c0a48640 1 -- 172.21.15.92:0/1434284791 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50bc070b80 con 0x7f50bc162810 2023-11-24T06:10:43.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:43.195+0000 7f50c0a48640 1 --2- 172.21.15.92:0/1434284791 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f50bc162810 0x7f50bc071240 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f50ac009870 tx=0x7f50ac02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9e55bc49a4521d2 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:43.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:43.196+0000 7f50b3fff640 1 -- 172.21.15.92:0/1434284791 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50ac031640 con 0x7f50bc162810 2023-11-24T06:10:43.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:43.196+0000 7f50b3fff640 1 -- 172.21.15.92:0/1434284791 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50ac0317a0 con 0x7f50bc162810 2023-11-24T06:10:43.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:43.196+0000 7f50b3fff640 1 -- 172.21.15.92:0/1434284791 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50ac031b30 con 0x7f50bc162810 2023-11-24T06:10:43.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:43.196+0000 7f50c224b640 1 -- 172.21.15.92:0/1434284791 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f50bc162810 msgr2=0x7f50bc071240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:43.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:43.196+0000 7f50c224b640 1 --2- 172.21.15.92:0/1434284791 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f50bc162810 0x7f50bc071240 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f50ac009870 tx=0x7f50ac02b8f0 comp rx=0 tx=0).stop 2023-11-24T06:10:43.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:43.197+0000 7f50c224b640 1 -- 172.21.15.92:0/1434284791 shutdown_connections 2023-11-24T06:10:43.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:43.197+0000 7f50c224b640 1 --2- 172.21.15.92:0/1434284791 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f50bc162810 0x7f50bc071240 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:43.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:43.197+0000 7f50c224b640 1 --2- 172.21.15.92:0/1434284791 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f50bc162430 0x7f50bc0b3110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:43.201 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:43.197+0000 7f50c224b640 1 -- 172.21.15.92:0/1434284791 >> 172.21.15.92:0/1434284791 conn(0x7f50bc06a800 msgr2=0x7f50bc06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:43.201 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:43.197+0000 7f50c224b640 1 -- 172.21.15.92:0/1434284791 shutdown_connections 2023-11-24T06:10:43.201 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:43.197+0000 7f50c224b640 1 -- 172.21.15.92:0/1434284791 wait complete. 2023-11-24T06:10:43.201 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:43.198+0000 7f50c224b640 1 Processor -- start 2023-11-24T06:10:43.201 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:43.198+0000 7f50c224b640 1 -- start start 2023-11-24T06:10:43.203 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:43.199+0000 7f50c224b640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f50bc0b3650 0x7f50bc162c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:43.203 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:43.199+0000 7f50c224b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f50bc162430 0x7f50bc148ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:43.203 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:43.199+0000 7f50c224b640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f50bc159930 0x7f50bc15a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:43.203 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:43.199+0000 7f50c224b640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50bc163430 con 0x7f50bc162430 2023-11-24T06:10:43.203 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:43.199+0000 7f50c224b640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50bc163710 con 0x7f50bc0b3650 2023-11-24T06:10:43.203 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:43.199+0000 7f50c224b640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50bc1639f0 con 0x7f50bc159930 2023-11-24T06:10:43.203 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:43.200+0000 7f50c0a48640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f50bc162430 0x7f50bc148ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:43.203 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:43.200+0000 7f50c0a48640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f50bc162430 0x7f50bc148ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47122/0 (socket says 172.21.15.92:47122) 2023-11-24T06:10:43.203 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:43.200+0000 7f50c0a48640 1 -- 172.21.15.92:0/2864214129 learned_addr learned my addr 172.21.15.92:0/2864214129 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:10:43.205 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:43.200+0000 7f50c1249640 1 --2- 172.21.15.92:0/2864214129 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f50bc0b3650 0x7f50bc162c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:43.205 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:43.200+0000 7f50c1a4a640 1 --2- 172.21.15.92:0/2864214129 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f50bc159930 0x7f50bc15a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:43.205 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:43.200+0000 7f50c0a48640 1 -- 172.21.15.92:0/2864214129 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f50bc159930 msgr2=0x7f50bc15a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:43.205 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:43.200+0000 7f50c0a48640 1 --2- 172.21.15.92:0/2864214129 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f50bc159930 0x7f50bc15a3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:43.205 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:43.200+0000 7f50c0a48640 1 -- 172.21.15.92:0/2864214129 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f50bc0b3650 msgr2=0x7f50bc162c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:43.205 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:43.200+0000 7f50c0a48640 1 --2- 172.21.15.92:0/2864214129 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f50bc0b3650 0x7f50bc162c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:43.205 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:43.200+0000 7f50c0a48640 1 -- 172.21.15.92:0/2864214129 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50ac0094d0 con 0x7f50bc162430 2023-11-24T06:10:43.205 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:43.200+0000 7f50c1249640 1 --2- 172.21.15.92:0/2864214129 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f50bc0b3650 0x7f50bc162c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:10:43.205 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:43.200+0000 7f50c0a48640 1 --2- 172.21.15.92:0/2864214129 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f50bc162430 0x7f50bc148ff0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f50ac0090b0 tx=0x7f50ac0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:43.206 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:43.201+0000 7f50b1ffb640 1 -- 172.21.15.92:0/2864214129 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50ac03ac60 con 0x7f50bc162430 2023-11-24T06:10:43.206 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:43.201+0000 7f50b1ffb640 1 -- 172.21.15.92:0/2864214129 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50ac03ade0 con 0x7f50bc162430 2023-11-24T06:10:43.206 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:43.201+0000 7f50b1ffb640 1 -- 172.21.15.92:0/2864214129 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50ac044630 con 0x7f50bc162430 2023-11-24T06:10:43.206 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:43.201+0000 7f50c224b640 1 -- 172.21.15.92:0/2864214129 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50bc163cd0 con 0x7f50bc162430 2023-11-24T06:10:43.206 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:43.201+0000 7f50c224b640 1 -- 172.21.15.92:0/2864214129 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50bc163140 con 0x7f50bc162430 2023-11-24T06:10:43.206 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:43.202+0000 7f50c224b640 1 -- 172.21.15.92:0/2864214129 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50bc1641f0 con 0x7f50bc162430 2023-11-24T06:10:43.207 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:43.203+0000 7f50b1ffb640 1 -- 172.21.15.92:0/2864214129 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f50ac02f050 con 0x7f50bc162430 2023-11-24T06:10:43.207 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:43.204+0000 7f50b1ffb640 1 --2- 172.21.15.92:0/2864214129 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f50a006ae40 0x7f50a006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:43.207 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:43.204+0000 7f50b1ffb640 1 -- 172.21.15.92:0/2864214129 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f50ac0ab490 con 0x7f50bc162430 2023-11-24T06:10:43.207 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:43.204+0000 7f50c1249640 1 --2- 172.21.15.92:0/2864214129 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f50a006ae40 0x7f50a006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:43.207 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:43.204+0000 7f50c1249640 1 --2- 172.21.15.92:0/2864214129 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f50a006ae40 0x7f50a006d300 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f50b8006fd0 tx=0x7f50b8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:43.211 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:43.208+0000 7f50b1ffb640 1 -- 172.21.15.92:0/2864214129 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f50bc1641f0 con 0x7f50bc162430 2023-11-24T06:10:43.445 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:43.441+0000 7f50c224b640 1 -- 172.21.15.92:0/2864214129 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f50bc1641f0 con 0x7f50bc162430 2023-11-24T06:10:43.445 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:43.442+0000 7f50b1ffb640 1 -- 172.21.15.92:0/2864214129 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7f50bc1641f0 con 0x7f50bc162430 2023-11-24T06:10:43.445 INFO:teuthology.orchestra.run.smithi092.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-24T06:10:43.448 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:43.445+0000 7f50c224b640 1 -- 172.21.15.92:0/2864214129 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f50a006ae40 msgr2=0x7f50a006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:43.449 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:43.445+0000 7f50c224b640 1 --2- 172.21.15.92:0/2864214129 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f50a006ae40 0x7f50a006d300 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f50b8006fd0 tx=0x7f50b8008040 comp rx=0 tx=0).stop 2023-11-24T06:10:43.449 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:43.445+0000 7f50c224b640 1 -- 172.21.15.92:0/2864214129 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f50bc162430 msgr2=0x7f50bc148ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:43.449 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:43.445+0000 7f50c224b640 1 --2- 172.21.15.92:0/2864214129 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f50bc162430 0x7f50bc148ff0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f50ac0090b0 tx=0x7f50ac0046c0 comp rx=0 tx=0).stop 2023-11-24T06:10:43.449 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:43.445+0000 7f50c224b640 1 -- 172.21.15.92:0/2864214129 shutdown_connections 2023-11-24T06:10:43.449 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:43.445+0000 7f50c224b640 1 --2- 172.21.15.92:0/2864214129 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f50a006ae40 0x7f50a006d300 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:43.449 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:43.445+0000 7f50c224b640 1 --2- 172.21.15.92:0/2864214129 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f50bc159930 0x7f50bc15a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:43.449 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:43.445+0000 7f50c224b640 1 --2- 172.21.15.92:0/2864214129 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f50bc162430 0x7f50bc148ff0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:43.449 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:43.445+0000 7f50c224b640 1 --2- 172.21.15.92:0/2864214129 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f50bc0b3650 0x7f50bc162c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:43.449 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:43.445+0000 7f50c224b640 1 -- 172.21.15.92:0/2864214129 >> 172.21.15.92:0/2864214129 conn(0x7f50bc06a800 msgr2=0x7f50bc071990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:43.449 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:43.445+0000 7f50c224b640 1 -- 172.21.15.92:0/2864214129 shutdown_connections 2023-11-24T06:10:43.449 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:43.445+0000 7f50c224b640 1 -- 172.21.15.92:0/2864214129 wait complete. 2023-11-24T06:10:43.468 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-24T06:10:43.468 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-24T06:10:43.468 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi092.front.sepia.ceph.com 2023-11-24T06:10:43.468 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-24T06:10:43.468 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-24T06:10:43.468 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-24T06:10:43.468 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-24T06:10:43.468 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-24T06:10:43.469 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-24T06:10:43.469 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:10:43.469 DEBUG:teuthology.orchestra.run.smithi092:> ip addr 2023-11-24T06:10:43.519 INFO:teuthology.orchestra.run.smithi092.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-24T06:10:43.519 INFO:teuthology.orchestra.run.smithi092.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-24T06:10:43.519 INFO:teuthology.orchestra.run.smithi092.stdout: inet 127.0.0.1/8 scope host lo 2023-11-24T06:10:43.519 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:10:43.519 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 ::1/128 scope host 2023-11-24T06:10:43.519 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:10:43.520 INFO:teuthology.orchestra.run.smithi092.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T06:10:43.520 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:d8:a1:b2 brd ff:ff:ff:ff:ff:ff 2023-11-24T06:10:43.520 INFO:teuthology.orchestra.run.smithi092.stdout: altname enp7s0f0 2023-11-24T06:10:43.520 INFO:teuthology.orchestra.run.smithi092.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T06:10:43.520 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:d8:a1:b3 brd ff:ff:ff:ff:ff:ff 2023-11-24T06:10:43.520 INFO:teuthology.orchestra.run.smithi092.stdout: altname enp7s0f1 2023-11-24T06:10:43.520 INFO:teuthology.orchestra.run.smithi092.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T06:10:43.520 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:8f:cd:44 brd ff:ff:ff:ff:ff:ff 2023-11-24T06:10:43.520 INFO:teuthology.orchestra.run.smithi092.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-24T06:10:43.520 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:8f:cd:45 brd ff:ff:ff:ff:ff:ff 2023-11-24T06:10:43.520 INFO:teuthology.orchestra.run.smithi092.stdout: inet 172.21.15.92/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-24T06:10:43.521 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft 42198sec preferred_lft 42198sec 2023-11-24T06:10:43.521 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 fe80::ec4:7aff:fe8f:cd45/64 scope link 2023-11-24T06:10:43.521 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:10:43.522 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-11-24T06:10:43.522 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:10:43.522 DEBUG:teuthology.orchestra.run.smithi092:> (cd / && exec bash -c ' 2023-11-24T06:10:43.522 DEBUG:teuthology.orchestra.run.smithi092:> set -e 2023-11-24T06:10:43.522 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip link add name ceph-brx type bridge 2023-11-24T06:10:43.522 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip addr flush dev ceph-brx 2023-11-24T06:10:43.522 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip link set ceph-brx up 2023-11-24T06:10:43.523 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-11-24T06:10:43.523 DEBUG:teuthology.orchestra.run.smithi092:> ') 2023-11-24T06:10:43.978 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:10:43.979 DEBUG:teuthology.orchestra.run.smithi092:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-24T06:10:44.058 INFO:teuthology.orchestra.run.smithi092.stdout:1 2023-11-24T06:10:44.061 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:10:44.061 DEBUG:teuthology.orchestra.run.smithi092:> route 2023-11-24T06:10:44.119 INFO:teuthology.orchestra.run.smithi092.stdout:Kernel IP routing table 2023-11-24T06:10:44.119 INFO:teuthology.orchestra.run.smithi092.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-24T06:10:44.119 INFO:teuthology.orchestra.run.smithi092.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-24T06:10:44.119 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-24T06:10:44.119 INFO:teuthology.orchestra.run.smithi092.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-24T06:10:44.120 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:10:44.121 DEBUG:teuthology.orchestra.run.smithi092:> (cd / && exec bash -c ' 2023-11-24T06:10:44.121 DEBUG:teuthology.orchestra.run.smithi092:> set -e 2023-11-24T06:10:44.121 DEBUG:teuthology.orchestra.run.smithi092:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-24T06:10:44.121 DEBUG:teuthology.orchestra.run.smithi092:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-24T06:10:44.121 DEBUG:teuthology.orchestra.run.smithi092:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-24T06:10:44.121 DEBUG:teuthology.orchestra.run.smithi092:> ') 2023-11-24T06:10:44.897 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:10:44.897 DEBUG:teuthology.orchestra.run.smithi092:> ip netns list 2023-11-24T06:10:44.946 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:10:44.946 DEBUG:teuthology.orchestra.run.smithi092:> ip netns list-id 2023-11-24T06:10:45.037 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:10:45.037 DEBUG:teuthology.orchestra.run.smithi092:> (cd / && exec bash -c ' 2023-11-24T06:10:45.037 DEBUG:teuthology.orchestra.run.smithi092:> set -e 2023-11-24T06:10:45.037 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-24T06:10:45.037 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2023-11-24T06:10:45.037 DEBUG:teuthology.orchestra.run.smithi092:> ') 2023-11-24T06:10:45.165 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2023-11-24T06:10:45.166 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:10:45.166 DEBUG:teuthology.orchestra.run.smithi092:> (cd / && exec bash -c ' 2023-11-24T06:10:45.166 DEBUG:teuthology.orchestra.run.smithi092:> set -e 2023-11-24T06:10:45.166 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2023-11-24T06:10:45.166 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-11-24T06:10:45.166 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2023-11-24T06:10:45.166 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2023-11-24T06:10:45.167 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2023-11-24T06:10:45.167 DEBUG:teuthology.orchestra.run.smithi092:> ') 2023-11-24T06:10:45.424 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:10:45.425 DEBUG:teuthology.orchestra.run.smithi092:> (cd / && exec bash -c ' 2023-11-24T06:10:45.425 DEBUG:teuthology.orchestra.run.smithi092:> set -e 2023-11-24T06:10:45.425 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip link set brx.0 up 2023-11-24T06:10:45.425 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip link set dev brx.0 master ceph-brx 2023-11-24T06:10:45.425 DEBUG:teuthology.orchestra.run.smithi092:> ') 2023-11-24T06:10:45.574 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-11-24T06:10:45.574 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T06:10:45.574 DEBUG:teuthology.orchestra.run.smithi092:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-11-24T06:10:45.618 INFO:teuthology.orchestra.run.smithi092.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-11-24T06:10:45.618 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T06:10:45.618 DEBUG:teuthology.orchestra.run.smithi092:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-11-24T06:10:45.704 DEBUG:teuthology.orchestra.run.smithi092:> sudo modprobe fuse 2023-11-24T06:10:45.832 DEBUG:teuthology.orchestra.run.smithi092:> cat /proc/self/mounts | awk '{print $2}' 2023-11-24T06:10:45.890 INFO:teuthology.orchestra.run.smithi092.stdout:/proc 2023-11-24T06:10:45.890 INFO:teuthology.orchestra.run.smithi092.stdout:/sys 2023-11-24T06:10:45.890 INFO:teuthology.orchestra.run.smithi092.stdout:/dev 2023-11-24T06:10:45.890 INFO:teuthology.orchestra.run.smithi092.stdout:/sys/kernel/security 2023-11-24T06:10:45.890 INFO:teuthology.orchestra.run.smithi092.stdout:/dev/shm 2023-11-24T06:10:45.890 INFO:teuthology.orchestra.run.smithi092.stdout:/dev/pts 2023-11-24T06:10:45.891 INFO:teuthology.orchestra.run.smithi092.stdout:/run 2023-11-24T06:10:45.891 INFO:teuthology.orchestra.run.smithi092.stdout:/sys/fs/cgroup 2023-11-24T06:10:45.891 INFO:teuthology.orchestra.run.smithi092.stdout:/sys/fs/pstore 2023-11-24T06:10:45.891 INFO:teuthology.orchestra.run.smithi092.stdout:/ 2023-11-24T06:10:45.891 INFO:teuthology.orchestra.run.smithi092.stdout:/sys/fs/selinux 2023-11-24T06:10:45.891 INFO:teuthology.orchestra.run.smithi092.stdout:/proc/sys/fs/binfmt_misc 2023-11-24T06:10:45.891 INFO:teuthology.orchestra.run.smithi092.stdout:/dev/hugepages 2023-11-24T06:10:45.891 INFO:teuthology.orchestra.run.smithi092.stdout:/dev/mqueue 2023-11-24T06:10:45.891 INFO:teuthology.orchestra.run.smithi092.stdout:/sys/kernel/debug 2023-11-24T06:10:45.891 INFO:teuthology.orchestra.run.smithi092.stdout:/sys/kernel/tracing 2023-11-24T06:10:45.891 INFO:teuthology.orchestra.run.smithi092.stdout:/sys/fs/fuse/connections 2023-11-24T06:10:45.892 INFO:teuthology.orchestra.run.smithi092.stdout:/sys/kernel/config 2023-11-24T06:10:45.892 INFO:teuthology.orchestra.run.smithi092.stdout:/run/credentials/systemd-sysctl.service 2023-11-24T06:10:45.892 INFO:teuthology.orchestra.run.smithi092.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-24T06:10:45.892 INFO:teuthology.orchestra.run.smithi092.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-24T06:10:45.892 INFO:teuthology.orchestra.run.smithi092.stdout:/var/lib/nfs/rpc_pipefs 2023-11-24T06:10:45.892 INFO:teuthology.orchestra.run.smithi092.stdout:/run/user/1000 2023-11-24T06:10:45.892 INFO:teuthology.orchestra.run.smithi092.stdout:/var/lib/ceph 2023-11-24T06:10:45.892 INFO:teuthology.orchestra.run.smithi092.stdout:/var/lib/ceph/osd/ceph-0 2023-11-24T06:10:45.892 INFO:teuthology.orchestra.run.smithi092.stdout:/var/lib/ceph/osd/ceph-1 2023-11-24T06:10:45.892 INFO:teuthology.orchestra.run.smithi092.stdout:/var/lib/ceph/osd/ceph-2 2023-11-24T06:10:45.892 INFO:teuthology.orchestra.run.smithi092.stdout:/var/lib/ceph/osd/ceph-3 2023-11-24T06:10:45.892 INFO:teuthology.orchestra.run.smithi092.stdout:/run/netns 2023-11-24T06:10:45.893 INFO:teuthology.orchestra.run.smithi092.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-24T06:10:45.893 INFO:teuthology.orchestra.run.smithi092.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-24T06:10:45.894 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:10:45.894 DEBUG:teuthology.orchestra.run.smithi092:> ls /sys/fs/fuse/connections 2023-11-24T06:10:45.938 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-11-24T06:10:45.938 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-11-24T06:10:45.987 DEBUG:teuthology.orchestra.run.smithi092:> sudo modprobe fuse 2023-11-24T06:10:46.065 DEBUG:teuthology.orchestra.run.smithi092:> cat /proc/self/mounts | awk '{print $2}' 2023-11-24T06:10:46.107 INFO:teuthology.orchestra.run.smithi092.stdout:/proc 2023-11-24T06:10:46.107 INFO:teuthology.orchestra.run.smithi092.stdout:/sys 2023-11-24T06:10:46.107 INFO:teuthology.orchestra.run.smithi092.stdout:/dev 2023-11-24T06:10:46.108 INFO:teuthology.orchestra.run.smithi092.stdout:/sys/kernel/security 2023-11-24T06:10:46.108 INFO:teuthology.orchestra.run.smithi092.stdout:/dev/shm 2023-11-24T06:10:46.108 INFO:teuthology.orchestra.run.smithi092.stdout:/dev/pts 2023-11-24T06:10:46.108 INFO:teuthology.orchestra.run.smithi092.stdout:/run 2023-11-24T06:10:46.108 INFO:teuthology.orchestra.run.smithi092.stdout:/sys/fs/cgroup 2023-11-24T06:10:46.108 INFO:teuthology.orchestra.run.smithi092.stdout:/sys/fs/pstore 2023-11-24T06:10:46.108 INFO:teuthology.orchestra.run.smithi092.stdout:/ 2023-11-24T06:10:46.108 INFO:teuthology.orchestra.run.smithi092.stdout:/sys/fs/selinux 2023-11-24T06:10:46.108 INFO:teuthology.orchestra.run.smithi092.stdout:/proc/sys/fs/binfmt_misc 2023-11-24T06:10:46.108 INFO:teuthology.orchestra.run.smithi092.stdout:/dev/hugepages 2023-11-24T06:10:46.108 INFO:teuthology.orchestra.run.smithi092.stdout:/dev/mqueue 2023-11-24T06:10:46.109 INFO:teuthology.orchestra.run.smithi092.stdout:/sys/kernel/debug 2023-11-24T06:10:46.109 INFO:teuthology.orchestra.run.smithi092.stdout:/sys/kernel/tracing 2023-11-24T06:10:46.109 INFO:teuthology.orchestra.run.smithi092.stdout:/sys/fs/fuse/connections 2023-11-24T06:10:46.109 INFO:teuthology.orchestra.run.smithi092.stdout:/sys/kernel/config 2023-11-24T06:10:46.109 INFO:teuthology.orchestra.run.smithi092.stdout:/run/credentials/systemd-sysctl.service 2023-11-24T06:10:46.109 INFO:teuthology.orchestra.run.smithi092.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-24T06:10:46.109 INFO:teuthology.orchestra.run.smithi092.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-24T06:10:46.109 INFO:teuthology.orchestra.run.smithi092.stdout:/var/lib/nfs/rpc_pipefs 2023-11-24T06:10:46.109 INFO:teuthology.orchestra.run.smithi092.stdout:/run/user/1000 2023-11-24T06:10:46.109 INFO:teuthology.orchestra.run.smithi092.stdout:/var/lib/ceph 2023-11-24T06:10:46.109 INFO:teuthology.orchestra.run.smithi092.stdout:/var/lib/ceph/osd/ceph-0 2023-11-24T06:10:46.110 INFO:teuthology.orchestra.run.smithi092.stdout:/var/lib/ceph/osd/ceph-1 2023-11-24T06:10:46.110 INFO:teuthology.orchestra.run.smithi092.stdout:/var/lib/ceph/osd/ceph-2 2023-11-24T06:10:46.110 INFO:teuthology.orchestra.run.smithi092.stdout:/var/lib/ceph/osd/ceph-3 2023-11-24T06:10:46.110 INFO:teuthology.orchestra.run.smithi092.stdout:/run/netns 2023-11-24T06:10:46.110 INFO:teuthology.orchestra.run.smithi092.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-24T06:10:46.110 INFO:teuthology.orchestra.run.smithi092.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-24T06:10:46.111 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:10:46.111 DEBUG:teuthology.orchestra.run.smithi092:> ls /sys/fs/fuse/connections 2023-11-24T06:10:46.184 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi092.stderr:2023-11-24T06:10:46.182+0000 7fcb11241480 -1 init, newargv = 0x5639f82fa630 newargc=15 2023-11-24T06:10:46.184 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi092.stderr:ceph-fuse[97264]: starting ceph client 2023-11-24T06:10:46.196 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi092.stderr:ceph-fuse[97264]: starting fuse 2023-11-24T06:10:46.307 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T06:10:46.315 DEBUG:teuthology.orchestra.run.smithi118:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T06:10:47.297 DEBUG:teuthology.orchestra.run.smithi092:> sudo modprobe fuse 2023-11-24T06:10:47.402 DEBUG:teuthology.orchestra.run.smithi092:> cat /proc/self/mounts | awk '{print $2}' 2023-11-24T06:10:47.456 INFO:teuthology.orchestra.run.smithi092.stdout:/proc 2023-11-24T06:10:47.456 INFO:teuthology.orchestra.run.smithi092.stdout:/sys 2023-11-24T06:10:47.456 INFO:teuthology.orchestra.run.smithi092.stdout:/dev 2023-11-24T06:10:47.456 INFO:teuthology.orchestra.run.smithi092.stdout:/sys/kernel/security 2023-11-24T06:10:47.457 INFO:teuthology.orchestra.run.smithi092.stdout:/dev/shm 2023-11-24T06:10:47.457 INFO:teuthology.orchestra.run.smithi092.stdout:/dev/pts 2023-11-24T06:10:47.457 INFO:teuthology.orchestra.run.smithi092.stdout:/run 2023-11-24T06:10:47.457 INFO:teuthology.orchestra.run.smithi092.stdout:/sys/fs/cgroup 2023-11-24T06:10:47.458 INFO:teuthology.orchestra.run.smithi092.stdout:/sys/fs/pstore 2023-11-24T06:10:47.458 INFO:teuthology.orchestra.run.smithi092.stdout:/ 2023-11-24T06:10:47.458 INFO:teuthology.orchestra.run.smithi092.stdout:/sys/fs/selinux 2023-11-24T06:10:47.458 INFO:teuthology.orchestra.run.smithi092.stdout:/proc/sys/fs/binfmt_misc 2023-11-24T06:10:47.458 INFO:teuthology.orchestra.run.smithi092.stdout:/dev/hugepages 2023-11-24T06:10:47.458 INFO:teuthology.orchestra.run.smithi092.stdout:/dev/mqueue 2023-11-24T06:10:47.458 INFO:teuthology.orchestra.run.smithi092.stdout:/sys/kernel/debug 2023-11-24T06:10:47.458 INFO:teuthology.orchestra.run.smithi092.stdout:/sys/kernel/tracing 2023-11-24T06:10:47.459 INFO:teuthology.orchestra.run.smithi092.stdout:/sys/fs/fuse/connections 2023-11-24T06:10:47.459 INFO:teuthology.orchestra.run.smithi092.stdout:/sys/kernel/config 2023-11-24T06:10:47.459 INFO:teuthology.orchestra.run.smithi092.stdout:/run/credentials/systemd-sysctl.service 2023-11-24T06:10:47.459 INFO:teuthology.orchestra.run.smithi092.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-24T06:10:47.459 INFO:teuthology.orchestra.run.smithi092.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-24T06:10:47.459 INFO:teuthology.orchestra.run.smithi092.stdout:/var/lib/nfs/rpc_pipefs 2023-11-24T06:10:47.459 INFO:teuthology.orchestra.run.smithi092.stdout:/run/user/1000 2023-11-24T06:10:47.459 INFO:teuthology.orchestra.run.smithi092.stdout:/var/lib/ceph 2023-11-24T06:10:47.459 INFO:teuthology.orchestra.run.smithi092.stdout:/var/lib/ceph/osd/ceph-0 2023-11-24T06:10:47.459 INFO:teuthology.orchestra.run.smithi092.stdout:/var/lib/ceph/osd/ceph-1 2023-11-24T06:10:47.459 INFO:teuthology.orchestra.run.smithi092.stdout:/var/lib/ceph/osd/ceph-2 2023-11-24T06:10:47.459 INFO:teuthology.orchestra.run.smithi092.stdout:/var/lib/ceph/osd/ceph-3 2023-11-24T06:10:47.460 INFO:teuthology.orchestra.run.smithi092.stdout:/run/netns 2023-11-24T06:10:47.460 INFO:teuthology.orchestra.run.smithi092.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-24T06:10:47.460 INFO:teuthology.orchestra.run.smithi092.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-24T06:10:47.460 INFO:teuthology.orchestra.run.smithi092.stdout:/home/ubuntu/cephtest/mnt.0 2023-11-24T06:10:47.461 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:10:47.461 DEBUG:teuthology.orchestra.run.smithi092:> ls /sys/fs/fuse/connections 2023-11-24T06:10:47.510 INFO:teuthology.orchestra.run.smithi092.stdout:40 2023-11-24T06:10:47.510 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [40] 2023-11-24T06:10:47.511 DEBUG:teuthology.orchestra.run.smithi092:> sudo stdin-killer -- python3 -c ' 2023-11-24T06:10:47.511 DEBUG:teuthology.orchestra.run.smithi092:> import glob 2023-11-24T06:10:47.511 DEBUG:teuthology.orchestra.run.smithi092:> import re 2023-11-24T06:10:47.511 DEBUG:teuthology.orchestra.run.smithi092:> import os 2023-11-24T06:10:47.511 DEBUG:teuthology.orchestra.run.smithi092:> import subprocess 2023-11-24T06:10:47.511 DEBUG:teuthology.orchestra.run.smithi092:> 2023-11-24T06:10:47.511 DEBUG:teuthology.orchestra.run.smithi092:> def _find_admin_socket(client_name): 2023-11-24T06:10:47.511 DEBUG:teuthology.orchestra.run.smithi092:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-11-24T06:10:47.511 DEBUG:teuthology.orchestra.run.smithi092:> files = glob.glob(asok_path) 2023-11-24T06:10:47.511 DEBUG:teuthology.orchestra.run.smithi092:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-11-24T06:10:47.511 DEBUG:teuthology.orchestra.run.smithi092:> 2023-11-24T06:10:47.512 DEBUG:teuthology.orchestra.run.smithi092:> # Given a non-glob path, it better be there 2023-11-24T06:10:47.512 DEBUG:teuthology.orchestra.run.smithi092:> if "*" not in asok_path: 2023-11-24T06:10:47.512 DEBUG:teuthology.orchestra.run.smithi092:> assert(len(files) == 1) 2023-11-24T06:10:47.512 DEBUG:teuthology.orchestra.run.smithi092:> return files[0] 2023-11-24T06:10:47.512 DEBUG:teuthology.orchestra.run.smithi092:> 2023-11-24T06:10:47.512 DEBUG:teuthology.orchestra.run.smithi092:> for f in files: 2023-11-24T06:10:47.512 DEBUG:teuthology.orchestra.run.smithi092:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-11-24T06:10:47.512 DEBUG:teuthology.orchestra.run.smithi092:> if os.path.exists("/proc/{0}".format(pid)): 2023-11-24T06:10:47.512 DEBUG:teuthology.orchestra.run.smithi092:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-11-24T06:10:47.512 DEBUG:teuthology.orchestra.run.smithi092:> contents = proc_f.read() 2023-11-24T06:10:47.512 DEBUG:teuthology.orchestra.run.smithi092:> if mountpoint in contents: 2023-11-24T06:10:47.512 DEBUG:teuthology.orchestra.run.smithi092:> return f 2023-11-24T06:10:47.512 DEBUG:teuthology.orchestra.run.smithi092:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-11-24T06:10:47.513 DEBUG:teuthology.orchestra.run.smithi092:> 2023-11-24T06:10:47.513 DEBUG:teuthology.orchestra.run.smithi092:> print(_find_admin_socket("client.0")) 2023-11-24T06:10:47.513 DEBUG:teuthology.orchestra.run.smithi092:> ' 2023-11-24T06:10:47.681 INFO:teuthology.orchestra.run.smithi092.stdout:/var/run/ceph/ceph-client.0.97264.asok 2023-11-24T06:10:47.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-24T06:10:47.698 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.97264.asok 2023-11-24T06:10:47.698 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:10:47.699 DEBUG:teuthology.orchestra.run.smithi092:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.97264.asok status 2023-11-24T06:10:47.938 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-24T06:10:47.938 INFO:teuthology.orchestra.run.smithi092.stdout: "metadata": { 2023-11-24T06:10:47.938 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph_sha1": "a13ea5ff00cb4df7a9cc941166ddeb358881144c", 2023-11-24T06:10:47.938 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph_version": "ceph version 18.0.0-7484-ga13ea5ff (a13ea5ff00cb4df7a9cc941166ddeb358881144c) reef (dev)", 2023-11-24T06:10:47.938 INFO:teuthology.orchestra.run.smithi092.stdout: "entity_id": "0", 2023-11-24T06:10:47.939 INFO:teuthology.orchestra.run.smithi092.stdout: "hostname": "smithi092", 2023-11-24T06:10:47.939 INFO:teuthology.orchestra.run.smithi092.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-11-24T06:10:47.939 INFO:teuthology.orchestra.run.smithi092.stdout: "pid": "97264", 2023-11-24T06:10:47.939 INFO:teuthology.orchestra.run.smithi092.stdout: "root": "/" 2023-11-24T06:10:47.939 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-24T06:10:47.939 INFO:teuthology.orchestra.run.smithi092.stdout: "dentry_count": 0, 2023-11-24T06:10:47.939 INFO:teuthology.orchestra.run.smithi092.stdout: "dentry_pinned_count": 0, 2023-11-24T06:10:47.939 INFO:teuthology.orchestra.run.smithi092.stdout: "id": 4653, 2023-11-24T06:10:47.939 INFO:teuthology.orchestra.run.smithi092.stdout: "inst": { 2023-11-24T06:10:47.939 INFO:teuthology.orchestra.run.smithi092.stdout: "name": { 2023-11-24T06:10:47.939 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "client", 2023-11-24T06:10:47.940 INFO:teuthology.orchestra.run.smithi092.stdout: "num": 4653 2023-11-24T06:10:47.940 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-24T06:10:47.940 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": { 2023-11-24T06:10:47.940 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2023-11-24T06:10:47.940 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "192.168.0.1:0", 2023-11-24T06:10:47.940 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 3972356301 2023-11-24T06:10:47.940 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-24T06:10:47.940 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-24T06:10:47.940 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": { 2023-11-24T06:10:47.940 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2023-11-24T06:10:47.940 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "192.168.0.1:0", 2023-11-24T06:10:47.940 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 3972356301 2023-11-24T06:10:47.941 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-24T06:10:47.941 INFO:teuthology.orchestra.run.smithi092.stdout: "inst_str": "client.4653 192.168.0.1:0/3972356301", 2023-11-24T06:10:47.941 INFO:teuthology.orchestra.run.smithi092.stdout: "addr_str": "192.168.0.1:0/3972356301", 2023-11-24T06:10:47.941 INFO:teuthology.orchestra.run.smithi092.stdout: "inode_count": 1, 2023-11-24T06:10:47.941 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_epoch": 8, 2023-11-24T06:10:47.941 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_epoch": 20, 2023-11-24T06:10:47.941 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_epoch_barrier": 0, 2023-11-24T06:10:47.941 INFO:teuthology.orchestra.run.smithi092.stdout: "blocklisted": false, 2023-11-24T06:10:47.941 INFO:teuthology.orchestra.run.smithi092.stdout: "fs_name": "cephfs" 2023-11-24T06:10:47.941 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-24T06:10:47.953 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-24T06:10:48.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:48.160+0000 7fb05b0ad640 1 Processor -- start 2023-11-24T06:10:48.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:48.161+0000 7fb05b0ad640 1 -- start start 2023-11-24T06:10:48.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:48.162+0000 7fb05b0ad640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fb05414b2c0 0x7fb0540cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:48.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:48.162+0000 7fb05b0ad640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb0540cf7d0 0x7fb054070960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:48.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:48.162+0000 7fb05b0ad640 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0540cfbf0 con 0x7fb054070ea0 2023-11-24T06:10:48.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:48.162+0000 7fb05b0ad640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb05414b6a0 con 0x7fb0540cf7d0 2023-11-24T06:10:48.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:48.162+0000 7fb05b0ad640 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb054071690 con 0x7fb05414b2c0 2023-11-24T06:10:48.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:48.162+0000 7fb053fff640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb0540cf7d0 0x7fb054070960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:48.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:48.163+0000 7fb053fff640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb0540cf7d0 0x7fb054070960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47130/0 (socket says 172.21.15.92:47130) 2023-11-24T06:10:48.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:48.163+0000 7fb053fff640 1 -- 172.21.15.92:0/1573539493 learned_addr learned my addr 172.21.15.92:0/1573539493 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:10:48.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:48.163+0000 7fb053fff640 1 -- 172.21.15.92:0/1573539493 >> v2:172.21.15.118:3300/0 conn(0x7fb05414b2c0 msgr2=0x7fb0540cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:48.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:48.163+0000 7fb053fff640 1 --2- 172.21.15.92:0/1573539493 >> v2:172.21.15.118:3300/0 conn(0x7fb05414b2c0 0x7fb0540cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:48.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:48.163+0000 7fb053fff640 1 -- 172.21.15.92:0/1573539493 >> v1:172.21.15.118:6789/0 conn(0x7fb054070ea0 legacy=0x7fb0540b1ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:48.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:48.163+0000 7fb053fff640 1 -- 172.21.15.92:0/1573539493 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb054071970 con 0x7fb0540cf7d0 2023-11-24T06:10:48.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:48.164+0000 7fb053fff640 1 --2- 172.21.15.92:0/1573539493 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb0540cf7d0 0x7fb054070960 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fb05406e650 tx=0x7fb04002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1194ed0a4a593839 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:48.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:48.164+0000 7fb0537fe640 1 -- 172.21.15.92:0/1573539493 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb040031870 con 0x7fb0540cf7d0 2023-11-24T06:10:48.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:48.164+0000 7fb0537fe640 1 -- 172.21.15.92:0/1573539493 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0400319d0 con 0x7fb0540cf7d0 2023-11-24T06:10:48.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:48.164+0000 7fb0537fe640 1 -- 172.21.15.92:0/1573539493 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb040031d60 con 0x7fb0540cf7d0 2023-11-24T06:10:48.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:48.165+0000 7fb05b0ad640 1 -- 172.21.15.92:0/1573539493 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb0540cf7d0 msgr2=0x7fb054070960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:48.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:48.165+0000 7fb05b0ad640 1 --2- 172.21.15.92:0/1573539493 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb0540cf7d0 0x7fb054070960 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fb05406e650 tx=0x7fb04002b910 comp rx=0 tx=0).stop 2023-11-24T06:10:48.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:48.165+0000 7fb05b0ad640 1 -- 172.21.15.92:0/1573539493 shutdown_connections 2023-11-24T06:10:48.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:48.165+0000 7fb05b0ad640 1 --2- 172.21.15.92:0/1573539493 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb0540cf7d0 0x7fb054070960 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:48.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:48.165+0000 7fb05b0ad640 1 --2- 172.21.15.92:0/1573539493 >> v2:172.21.15.118:3300/0 conn(0x7fb05414b2c0 0x7fb0540cf290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:48.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:48.165+0000 7fb05b0ad640 1 -- 172.21.15.92:0/1573539493 >> 172.21.15.92:0/1573539493 conn(0x7fb05406a800 msgr2=0x7fb05406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:48.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:48.165+0000 7fb05b0ad640 1 -- 172.21.15.92:0/1573539493 shutdown_connections 2023-11-24T06:10:48.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:48.165+0000 7fb05b0ad640 1 -- 172.21.15.92:0/1573539493 wait complete. 2023-11-24T06:10:48.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:48.167+0000 7fb05b0ad640 1 Processor -- start 2023-11-24T06:10:48.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:48.167+0000 7fb05b0ad640 1 -- start start 2023-11-24T06:10:48.170 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:48.168+0000 7fb05b0ad640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb054070ea0 0x7fb054146340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:48.170 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:48.168+0000 7fb05b0ad640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb0540cf7d0 0x7fb054161e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:48.170 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:48.168+0000 7fb05b0ad640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb05414b2c0 0x7fb0541623b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:48.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:48.168+0000 7fb05b0ad640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb05415ff30 con 0x7fb05414b2c0 2023-11-24T06:10:48.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:48.168+0000 7fb05b0ad640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb054163040 con 0x7fb054070ea0 2023-11-24T06:10:48.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:48.168+0000 7fb05b0ad640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0541644d0 con 0x7fb0540cf7d0 2023-11-24T06:10:48.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:48.168+0000 7fb05964a640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb05414b2c0 0x7fb0541623b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:48.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:48.168+0000 7fb05964a640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb05414b2c0 0x7fb0541623b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47140/0 (socket says 172.21.15.92:47140) 2023-11-24T06:10:48.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:48.168+0000 7fb05964a640 1 -- 172.21.15.92:0/2182701785 learned_addr learned my addr 172.21.15.92:0/2182701785 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:10:48.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:48.168+0000 7fb058e49640 1 --2- 172.21.15.92:0/2182701785 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb054070ea0 0x7fb054146340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:48.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:48.169+0000 7fb053fff640 1 --2- 172.21.15.92:0/2182701785 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb0540cf7d0 0x7fb054161e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:48.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:48.169+0000 7fb05964a640 1 -- 172.21.15.92:0/2182701785 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb0540cf7d0 msgr2=0x7fb054161e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:48.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:48.169+0000 7fb05964a640 1 --2- 172.21.15.92:0/2182701785 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb0540cf7d0 0x7fb054161e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:48.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:48.169+0000 7fb05964a640 1 -- 172.21.15.92:0/2182701785 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb054070ea0 msgr2=0x7fb054146340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:48.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:48.169+0000 7fb05964a640 1 --2- 172.21.15.92:0/2182701785 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb054070ea0 0x7fb054146340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:48.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:48.169+0000 7fb05964a640 1 -- 172.21.15.92:0/2182701785 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0400094d0 con 0x7fb05414b2c0 2023-11-24T06:10:48.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:48.169+0000 7fb05964a640 1 --2- 172.21.15.92:0/2182701785 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb05414b2c0 0x7fb0541623b0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fb04c00c780 tx=0x7fb04c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:48.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:48.169+0000 7fb0517fa640 1 -- 172.21.15.92:0/2182701785 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb04c003d80 con 0x7fb05414b2c0 2023-11-24T06:10:48.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:48.169+0000 7fb0517fa640 1 -- 172.21.15.92:0/2182701785 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb04c00ce90 con 0x7fb05414b2c0 2023-11-24T06:10:48.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:48.169+0000 7fb05b0ad640 1 -- 172.21.15.92:0/2182701785 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb054164d30 con 0x7fb05414b2c0 2023-11-24T06:10:48.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:48.169+0000 7fb0517fa640 1 -- 172.21.15.92:0/2182701785 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb04c0052b0 con 0x7fb05414b2c0 2023-11-24T06:10:48.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:48.169+0000 7fb05b0ad640 1 -- 172.21.15.92:0/2182701785 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb054164770 con 0x7fb05414b2c0 2023-11-24T06:10:48.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:48.170+0000 7fb05b0ad640 1 -- 172.21.15.92:0/2182701785 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb05415e530 con 0x7fb05414b2c0 2023-11-24T06:10:48.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:48.171+0000 7fb0517fa640 1 -- 172.21.15.92:0/2182701785 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb04c005410 con 0x7fb05414b2c0 2023-11-24T06:10:48.175 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:48.172+0000 7fb0517fa640 1 --2- 172.21.15.92:0/2182701785 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fb03806ae40 0x7fb03806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:48.175 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:48.173+0000 7fb0517fa640 1 -- 172.21.15.92:0/2182701785 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb04c088ef0 con 0x7fb05414b2c0 2023-11-24T06:10:48.175 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:48.173+0000 7fb058e49640 1 --2- 172.21.15.92:0/2182701785 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fb03806ae40 0x7fb03806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:48.175 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:48.173+0000 7fb058e49640 1 --2- 172.21.15.92:0/2182701785 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fb03806ae40 0x7fb03806d300 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fb04800b810 tx=0x7fb048008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:48.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:48.178+0000 7fb0517fa640 1 -- 172.21.15.92:0/2182701785 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb05415e530 con 0x7fb05414b2c0 2023-11-24T06:10:48.406 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:48.404+0000 7fb05b0ad640 1 -- 172.21.15.92:0/2182701785 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fb05415e530 con 0x7fb05414b2c0 2023-11-24T06:10:48.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:48.405+0000 7fb0517fa640 1 -- 172.21.15.92:0/2182701785 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7fb05415e530 con 0x7fb05414b2c0 2023-11-24T06:10:48.407 INFO:teuthology.orchestra.run.smithi092.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-24T06:10:48.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:48.407+0000 7fb05b0ad640 1 -- 172.21.15.92:0/2182701785 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fb03806ae40 msgr2=0x7fb03806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:48.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:48.407+0000 7fb05b0ad640 1 --2- 172.21.15.92:0/2182701785 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fb03806ae40 0x7fb03806d300 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fb04800b810 tx=0x7fb048008040 comp rx=0 tx=0).stop 2023-11-24T06:10:48.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:48.407+0000 7fb05b0ad640 1 -- 172.21.15.92:0/2182701785 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb05414b2c0 msgr2=0x7fb0541623b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:48.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:48.407+0000 7fb05b0ad640 1 --2- 172.21.15.92:0/2182701785 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb05414b2c0 0x7fb0541623b0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fb04c00c780 tx=0x7fb04c00cc40 comp rx=0 tx=0).stop 2023-11-24T06:10:48.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:48.408+0000 7fb05b0ad640 1 -- 172.21.15.92:0/2182701785 shutdown_connections 2023-11-24T06:10:48.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:48.408+0000 7fb05b0ad640 1 --2- 172.21.15.92:0/2182701785 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fb03806ae40 0x7fb03806d300 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:48.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:48.408+0000 7fb05b0ad640 1 --2- 172.21.15.92:0/2182701785 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb05414b2c0 0x7fb0541623b0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:48.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:48.408+0000 7fb05b0ad640 1 --2- 172.21.15.92:0/2182701785 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb0540cf7d0 0x7fb054161e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:48.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:48.408+0000 7fb05b0ad640 1 --2- 172.21.15.92:0/2182701785 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb054070ea0 0x7fb054146340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:48.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:48.408+0000 7fb05b0ad640 1 -- 172.21.15.92:0/2182701785 >> 172.21.15.92:0/2182701785 conn(0x7fb05406a800 msgr2=0x7fb05406c470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:48.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:48.408+0000 7fb05b0ad640 1 -- 172.21.15.92:0/2182701785 shutdown_connections 2023-11-24T06:10:48.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:48.408+0000 7fb05b0ad640 1 -- 172.21.15.92:0/2182701785 wait complete. 2023-11-24T06:10:48.429 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-24T06:10:48.429 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-24T06:10:48.429 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi092.front.sepia.ceph.com 2023-11-24T06:10:48.429 INFO:tasks.cephfs.mount:self.client.name = client.2 2023-11-24T06:10:48.429 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2023-11-24T06:10:48.429 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-24T06:10:48.429 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-24T06:10:48.429 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-24T06:10:48.430 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2023-11-24T06:10:48.430 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:10:48.430 DEBUG:teuthology.orchestra.run.smithi092:> ip addr 2023-11-24T06:10:48.480 INFO:teuthology.orchestra.run.smithi092.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-24T06:10:48.480 INFO:teuthology.orchestra.run.smithi092.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-24T06:10:48.481 INFO:teuthology.orchestra.run.smithi092.stdout: inet 127.0.0.1/8 scope host lo 2023-11-24T06:10:48.481 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:10:48.481 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 ::1/128 scope host 2023-11-24T06:10:48.481 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:10:48.481 INFO:teuthology.orchestra.run.smithi092.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T06:10:48.481 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:d8:a1:b2 brd ff:ff:ff:ff:ff:ff 2023-11-24T06:10:48.481 INFO:teuthology.orchestra.run.smithi092.stdout: altname enp7s0f0 2023-11-24T06:10:48.481 INFO:teuthology.orchestra.run.smithi092.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T06:10:48.481 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:d8:a1:b3 brd ff:ff:ff:ff:ff:ff 2023-11-24T06:10:48.482 INFO:teuthology.orchestra.run.smithi092.stdout: altname enp7s0f1 2023-11-24T06:10:48.482 INFO:teuthology.orchestra.run.smithi092.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T06:10:48.482 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:8f:cd:44 brd ff:ff:ff:ff:ff:ff 2023-11-24T06:10:48.482 INFO:teuthology.orchestra.run.smithi092.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-24T06:10:48.482 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:8f:cd:45 brd ff:ff:ff:ff:ff:ff 2023-11-24T06:10:48.482 INFO:teuthology.orchestra.run.smithi092.stdout: inet 172.21.15.92/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-24T06:10:48.482 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft 42193sec preferred_lft 42193sec 2023-11-24T06:10:48.482 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 fe80::ec4:7aff:fe8f:cd45/64 scope link 2023-11-24T06:10:48.482 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:10:48.483 INFO:teuthology.orchestra.run.smithi092.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-24T06:10:48.483 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether fa:d3:03:06:91:4f brd ff:ff:ff:ff:ff:ff 2023-11-24T06:10:48.483 INFO:teuthology.orchestra.run.smithi092.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-24T06:10:48.483 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:10:48.483 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 fe80::9055:2fff:fec5:ff24/64 scope link 2023-11-24T06:10:48.483 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:10:48.483 INFO:teuthology.orchestra.run.smithi092.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-24T06:10:48.483 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether fa:d3:03:06:91:4f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-24T06:10:48.483 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 fe80::f8d3:3ff:fe06:914f/64 scope link 2023-11-24T06:10:48.484 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:10:48.484 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:10:48.484 DEBUG:teuthology.orchestra.run.smithi092:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-24T06:10:48.567 INFO:teuthology.orchestra.run.smithi092.stdout:1 2023-11-24T06:10:48.571 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:10:48.571 DEBUG:teuthology.orchestra.run.smithi092:> route 2023-11-24T06:10:48.625 INFO:teuthology.orchestra.run.smithi092.stdout:Kernel IP routing table 2023-11-24T06:10:48.625 INFO:teuthology.orchestra.run.smithi092.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-24T06:10:48.625 INFO:teuthology.orchestra.run.smithi092.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-24T06:10:48.625 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-24T06:10:48.625 INFO:teuthology.orchestra.run.smithi092.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-24T06:10:48.625 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:10:48.626 DEBUG:teuthology.orchestra.run.smithi092:> (cd / && exec bash -c ' 2023-11-24T06:10:48.626 DEBUG:teuthology.orchestra.run.smithi092:> set -e 2023-11-24T06:10:48.626 DEBUG:teuthology.orchestra.run.smithi092:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-24T06:10:48.626 DEBUG:teuthology.orchestra.run.smithi092:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-24T06:10:48.626 DEBUG:teuthology.orchestra.run.smithi092:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-24T06:10:48.626 DEBUG:teuthology.orchestra.run.smithi092:> ') 2023-11-24T06:10:48.809 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:10:48.809 DEBUG:teuthology.orchestra.run.smithi092:> ip netns list 2023-11-24T06:10:48.863 INFO:teuthology.orchestra.run.smithi092.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-24T06:10:48.864 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:10:48.864 DEBUG:teuthology.orchestra.run.smithi092:> ip netns list-id 2023-11-24T06:10:48.915 INFO:teuthology.orchestra.run.smithi092.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2023-11-24T06:10:48.915 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:10:48.915 DEBUG:teuthology.orchestra.run.smithi092:> (cd / && exec bash -c ' 2023-11-24T06:10:48.916 DEBUG:teuthology.orchestra.run.smithi092:> set -e 2023-11-24T06:10:48.916 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-24T06:10:48.916 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2023-11-24T06:10:48.916 DEBUG:teuthology.orchestra.run.smithi092:> ') 2023-11-24T06:10:49.058 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:10:49.058 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-11-24T06:10:49.150 INFO:teuthology.orchestra.run.smithi092.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-24T06:10:49.150 INFO:teuthology.orchestra.run.smithi092.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-24T06:10:49.150 INFO:teuthology.orchestra.run.smithi092.stdout: inet 127.0.0.1/8 scope host lo 2023-11-24T06:10:49.150 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:10:49.150 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 ::1/128 scope host 2023-11-24T06:10:49.150 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:10:49.151 INFO:teuthology.orchestra.run.smithi092.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-24T06:10:49.151 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 8a:3d:3e:9e:0e:5b brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-11-24T06:10:49.151 INFO:teuthology.orchestra.run.smithi092.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-11-24T06:10:49.151 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:10:49.151 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 fe80::883d:3eff:fe9e:e5b/64 scope link 2023-11-24T06:10:49.151 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:10:49.153 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:10:49.154 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-11-24T06:10:49.255 INFO:teuthology.orchestra.run.smithi092.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-24T06:10:49.255 INFO:teuthology.orchestra.run.smithi092.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-24T06:10:49.255 INFO:teuthology.orchestra.run.smithi092.stdout: inet 127.0.0.1/8 scope host lo 2023-11-24T06:10:49.255 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:10:49.255 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 ::1/128 scope host 2023-11-24T06:10:49.256 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:10:49.256 INFO:teuthology.orchestra.run.smithi092.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-24T06:10:49.256 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 8a:3d:3e:9e:0e:5b brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-11-24T06:10:49.256 INFO:teuthology.orchestra.run.smithi092.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-11-24T06:10:49.256 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:10:49.256 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 fe80::883d:3eff:fe9e:e5b/64 scope link 2023-11-24T06:10:49.256 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:10:49.260 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2023-11-24T06:10:49.260 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:10:49.260 DEBUG:teuthology.orchestra.run.smithi092:> (cd / && exec bash -c ' 2023-11-24T06:10:49.261 DEBUG:teuthology.orchestra.run.smithi092:> set -e 2023-11-24T06:10:49.261 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2023-11-24T06:10:49.261 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-11-24T06:10:49.261 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2023-11-24T06:10:49.261 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2023-11-24T06:10:49.261 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2023-11-24T06:10:49.261 DEBUG:teuthology.orchestra.run.smithi092:> ') 2023-11-24T06:10:49.524 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:10:49.524 DEBUG:teuthology.orchestra.run.smithi092:> (cd / && exec bash -c ' 2023-11-24T06:10:49.524 DEBUG:teuthology.orchestra.run.smithi092:> set -e 2023-11-24T06:10:49.524 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip link set brx.1 up 2023-11-24T06:10:49.524 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip link set dev brx.1 master ceph-brx 2023-11-24T06:10:49.524 DEBUG:teuthology.orchestra.run.smithi092:> ') 2023-11-24T06:10:49.687 INFO:tasks.cephfs.fuse_mount:Client client.2 config is {} 2023-11-24T06:10:49.687 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T06:10:49.687 DEBUG:teuthology.orchestra.run.smithi092:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2023-11-24T06:10:49.740 INFO:teuthology.orchestra.run.smithi092.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2023-11-24T06:10:49.740 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T06:10:49.741 DEBUG:teuthology.orchestra.run.smithi092:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2023-11-24T06:10:49.788 DEBUG:teuthology.orchestra.run.smithi092:> sudo modprobe fuse 2023-11-24T06:10:49.934 DEBUG:teuthology.orchestra.run.smithi092:> cat /proc/self/mounts | awk '{print $2}' 2023-11-24T06:10:49.988 INFO:teuthology.orchestra.run.smithi092.stdout:/proc 2023-11-24T06:10:49.988 INFO:teuthology.orchestra.run.smithi092.stdout:/sys 2023-11-24T06:10:49.988 INFO:teuthology.orchestra.run.smithi092.stdout:/dev 2023-11-24T06:10:49.988 INFO:teuthology.orchestra.run.smithi092.stdout:/sys/kernel/security 2023-11-24T06:10:49.989 INFO:teuthology.orchestra.run.smithi092.stdout:/dev/shm 2023-11-24T06:10:49.989 INFO:teuthology.orchestra.run.smithi092.stdout:/dev/pts 2023-11-24T06:10:49.989 INFO:teuthology.orchestra.run.smithi092.stdout:/run 2023-11-24T06:10:49.989 INFO:teuthology.orchestra.run.smithi092.stdout:/sys/fs/cgroup 2023-11-24T06:10:49.989 INFO:teuthology.orchestra.run.smithi092.stdout:/sys/fs/pstore 2023-11-24T06:10:49.989 INFO:teuthology.orchestra.run.smithi092.stdout:/ 2023-11-24T06:10:49.990 INFO:teuthology.orchestra.run.smithi092.stdout:/sys/fs/selinux 2023-11-24T06:10:49.990 INFO:teuthology.orchestra.run.smithi092.stdout:/proc/sys/fs/binfmt_misc 2023-11-24T06:10:49.990 INFO:teuthology.orchestra.run.smithi092.stdout:/dev/hugepages 2023-11-24T06:10:49.990 INFO:teuthology.orchestra.run.smithi092.stdout:/dev/mqueue 2023-11-24T06:10:49.990 INFO:teuthology.orchestra.run.smithi092.stdout:/sys/kernel/debug 2023-11-24T06:10:49.990 INFO:teuthology.orchestra.run.smithi092.stdout:/sys/kernel/tracing 2023-11-24T06:10:49.990 INFO:teuthology.orchestra.run.smithi092.stdout:/sys/fs/fuse/connections 2023-11-24T06:10:49.991 INFO:teuthology.orchestra.run.smithi092.stdout:/sys/kernel/config 2023-11-24T06:10:49.991 INFO:teuthology.orchestra.run.smithi092.stdout:/run/credentials/systemd-sysctl.service 2023-11-24T06:10:49.991 INFO:teuthology.orchestra.run.smithi092.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-24T06:10:49.991 INFO:teuthology.orchestra.run.smithi092.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-24T06:10:49.991 INFO:teuthology.orchestra.run.smithi092.stdout:/var/lib/nfs/rpc_pipefs 2023-11-24T06:10:49.991 INFO:teuthology.orchestra.run.smithi092.stdout:/run/user/1000 2023-11-24T06:10:49.991 INFO:teuthology.orchestra.run.smithi092.stdout:/var/lib/ceph 2023-11-24T06:10:49.992 INFO:teuthology.orchestra.run.smithi092.stdout:/var/lib/ceph/osd/ceph-0 2023-11-24T06:10:49.992 INFO:teuthology.orchestra.run.smithi092.stdout:/var/lib/ceph/osd/ceph-1 2023-11-24T06:10:49.992 INFO:teuthology.orchestra.run.smithi092.stdout:/var/lib/ceph/osd/ceph-2 2023-11-24T06:10:49.992 INFO:teuthology.orchestra.run.smithi092.stdout:/var/lib/ceph/osd/ceph-3 2023-11-24T06:10:49.992 INFO:teuthology.orchestra.run.smithi092.stdout:/run/netns 2023-11-24T06:10:49.992 INFO:teuthology.orchestra.run.smithi092.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-24T06:10:49.992 INFO:teuthology.orchestra.run.smithi092.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-24T06:10:49.992 INFO:teuthology.orchestra.run.smithi092.stdout:/home/ubuntu/cephtest/mnt.0 2023-11-24T06:10:49.992 INFO:teuthology.orchestra.run.smithi092.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-24T06:10:49.993 INFO:teuthology.orchestra.run.smithi092.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-24T06:10:49.994 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:10:49.994 DEBUG:teuthology.orchestra.run.smithi092:> ls /sys/fs/fuse/connections 2023-11-24T06:10:50.046 INFO:teuthology.orchestra.run.smithi092.stdout:40 2023-11-24T06:10:50.047 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [40] 2023-11-24T06:10:50.047 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.2 --id 2) 2023-11-24T06:10:50.052 DEBUG:teuthology.orchestra.run.smithi092:> sudo modprobe fuse 2023-11-24T06:10:50.129 DEBUG:teuthology.orchestra.run.smithi092:> cat /proc/self/mounts | awk '{print $2}' 2023-11-24T06:10:50.171 INFO:teuthology.orchestra.run.smithi092.stdout:/proc 2023-11-24T06:10:50.171 INFO:teuthology.orchestra.run.smithi092.stdout:/sys 2023-11-24T06:10:50.171 INFO:teuthology.orchestra.run.smithi092.stdout:/dev 2023-11-24T06:10:50.172 INFO:teuthology.orchestra.run.smithi092.stdout:/sys/kernel/security 2023-11-24T06:10:50.172 INFO:teuthology.orchestra.run.smithi092.stdout:/dev/shm 2023-11-24T06:10:50.172 INFO:teuthology.orchestra.run.smithi092.stdout:/dev/pts 2023-11-24T06:10:50.172 INFO:teuthology.orchestra.run.smithi092.stdout:/run 2023-11-24T06:10:50.172 INFO:teuthology.orchestra.run.smithi092.stdout:/sys/fs/cgroup 2023-11-24T06:10:50.172 INFO:teuthology.orchestra.run.smithi092.stdout:/sys/fs/pstore 2023-11-24T06:10:50.172 INFO:teuthology.orchestra.run.smithi092.stdout:/ 2023-11-24T06:10:50.172 INFO:teuthology.orchestra.run.smithi092.stdout:/sys/fs/selinux 2023-11-24T06:10:50.172 INFO:teuthology.orchestra.run.smithi092.stdout:/proc/sys/fs/binfmt_misc 2023-11-24T06:10:50.172 INFO:teuthology.orchestra.run.smithi092.stdout:/dev/hugepages 2023-11-24T06:10:50.173 INFO:teuthology.orchestra.run.smithi092.stdout:/dev/mqueue 2023-11-24T06:10:50.173 INFO:teuthology.orchestra.run.smithi092.stdout:/sys/kernel/debug 2023-11-24T06:10:50.173 INFO:teuthology.orchestra.run.smithi092.stdout:/sys/kernel/tracing 2023-11-24T06:10:50.173 INFO:teuthology.orchestra.run.smithi092.stdout:/sys/fs/fuse/connections 2023-11-24T06:10:50.173 INFO:teuthology.orchestra.run.smithi092.stdout:/sys/kernel/config 2023-11-24T06:10:50.173 INFO:teuthology.orchestra.run.smithi092.stdout:/run/credentials/systemd-sysctl.service 2023-11-24T06:10:50.173 INFO:teuthology.orchestra.run.smithi092.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-24T06:10:50.173 INFO:teuthology.orchestra.run.smithi092.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-24T06:10:50.173 INFO:teuthology.orchestra.run.smithi092.stdout:/var/lib/nfs/rpc_pipefs 2023-11-24T06:10:50.173 INFO:teuthology.orchestra.run.smithi092.stdout:/run/user/1000 2023-11-24T06:10:50.173 INFO:teuthology.orchestra.run.smithi092.stdout:/var/lib/ceph 2023-11-24T06:10:50.173 INFO:teuthology.orchestra.run.smithi092.stdout:/var/lib/ceph/osd/ceph-0 2023-11-24T06:10:50.173 INFO:teuthology.orchestra.run.smithi092.stdout:/var/lib/ceph/osd/ceph-1 2023-11-24T06:10:50.174 INFO:teuthology.orchestra.run.smithi092.stdout:/var/lib/ceph/osd/ceph-2 2023-11-24T06:10:50.174 INFO:teuthology.orchestra.run.smithi092.stdout:/var/lib/ceph/osd/ceph-3 2023-11-24T06:10:50.174 INFO:teuthology.orchestra.run.smithi092.stdout:/run/netns 2023-11-24T06:10:50.174 INFO:teuthology.orchestra.run.smithi092.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-24T06:10:50.174 INFO:teuthology.orchestra.run.smithi092.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-24T06:10:50.174 INFO:teuthology.orchestra.run.smithi092.stdout:/home/ubuntu/cephtest/mnt.0 2023-11-24T06:10:50.174 INFO:teuthology.orchestra.run.smithi092.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-24T06:10:50.174 INFO:teuthology.orchestra.run.smithi092.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-24T06:10:50.175 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:10:50.176 DEBUG:teuthology.orchestra.run.smithi092:> ls /sys/fs/fuse/connections 2023-11-24T06:10:50.240 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi092.stderr:2023-11-24T06:10:50.238+0000 7f729604c480 -1 init, newargv = 0x560e79a87910 newargc=15 2023-11-24T06:10:50.240 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi092.stderr:ceph-fuse[97844]: starting ceph client 2023-11-24T06:10:50.250 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi092.stderr:ceph-fuse[97844]: starting fuse 2023-11-24T06:10:50.265 INFO:teuthology.orchestra.run.smithi092.stdout:40 2023-11-24T06:10:50.265 INFO:teuthology.orchestra.run.smithi092.stdout:42 2023-11-24T06:10:50.266 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [40, 42] 2023-11-24T06:10:50.266 DEBUG:teuthology.orchestra.run.smithi092:> sudo stdin-killer -- python3 -c ' 2023-11-24T06:10:50.266 DEBUG:teuthology.orchestra.run.smithi092:> import glob 2023-11-24T06:10:50.266 DEBUG:teuthology.orchestra.run.smithi092:> import re 2023-11-24T06:10:50.267 DEBUG:teuthology.orchestra.run.smithi092:> import os 2023-11-24T06:10:50.267 DEBUG:teuthology.orchestra.run.smithi092:> import subprocess 2023-11-24T06:10:50.267 DEBUG:teuthology.orchestra.run.smithi092:> 2023-11-24T06:10:50.267 DEBUG:teuthology.orchestra.run.smithi092:> def _find_admin_socket(client_name): 2023-11-24T06:10:50.267 DEBUG:teuthology.orchestra.run.smithi092:> asok_path = "/var/run/ceph/ceph-client.2.*.asok" 2023-11-24T06:10:50.267 DEBUG:teuthology.orchestra.run.smithi092:> files = glob.glob(asok_path) 2023-11-24T06:10:50.268 DEBUG:teuthology.orchestra.run.smithi092:> mountpoint = "/home/ubuntu/cephtest/mnt.2" 2023-11-24T06:10:50.268 DEBUG:teuthology.orchestra.run.smithi092:> 2023-11-24T06:10:50.268 DEBUG:teuthology.orchestra.run.smithi092:> # Given a non-glob path, it better be there 2023-11-24T06:10:50.268 DEBUG:teuthology.orchestra.run.smithi092:> if "*" not in asok_path: 2023-11-24T06:10:50.268 DEBUG:teuthology.orchestra.run.smithi092:> assert(len(files) == 1) 2023-11-24T06:10:50.269 DEBUG:teuthology.orchestra.run.smithi092:> return files[0] 2023-11-24T06:10:50.269 DEBUG:teuthology.orchestra.run.smithi092:> 2023-11-24T06:10:50.269 DEBUG:teuthology.orchestra.run.smithi092:> for f in files: 2023-11-24T06:10:50.269 DEBUG:teuthology.orchestra.run.smithi092:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-11-24T06:10:50.269 DEBUG:teuthology.orchestra.run.smithi092:> if os.path.exists("/proc/{0}".format(pid)): 2023-11-24T06:10:50.269 DEBUG:teuthology.orchestra.run.smithi092:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-11-24T06:10:50.270 DEBUG:teuthology.orchestra.run.smithi092:> contents = proc_f.read() 2023-11-24T06:10:50.270 DEBUG:teuthology.orchestra.run.smithi092:> if mountpoint in contents: 2023-11-24T06:10:50.270 DEBUG:teuthology.orchestra.run.smithi092:> return f 2023-11-24T06:10:50.270 DEBUG:teuthology.orchestra.run.smithi092:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-11-24T06:10:50.270 DEBUG:teuthology.orchestra.run.smithi092:> 2023-11-24T06:10:50.271 DEBUG:teuthology.orchestra.run.smithi092:> print(_find_admin_socket("client.2")) 2023-11-24T06:10:50.271 DEBUG:teuthology.orchestra.run.smithi092:> ' 2023-11-24T06:10:50.473 INFO:teuthology.orchestra.run.smithi092.stdout:/var/run/ceph/ceph-client.2.97844.asok 2023-11-24T06:10:50.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-24T06:10:50.491 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.2.97844.asok 2023-11-24T06:10:50.491 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:10:50.492 DEBUG:teuthology.orchestra.run.smithi092:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.2.97844.asok status 2023-11-24T06:10:50.731 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-24T06:10:50.731 INFO:teuthology.orchestra.run.smithi092.stdout: "metadata": { 2023-11-24T06:10:50.731 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph_sha1": "a13ea5ff00cb4df7a9cc941166ddeb358881144c", 2023-11-24T06:10:50.731 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph_version": "ceph version 18.0.0-7484-ga13ea5ff (a13ea5ff00cb4df7a9cc941166ddeb358881144c) reef (dev)", 2023-11-24T06:10:50.731 INFO:teuthology.orchestra.run.smithi092.stdout: "entity_id": "2", 2023-11-24T06:10:50.731 INFO:teuthology.orchestra.run.smithi092.stdout: "hostname": "smithi092", 2023-11-24T06:10:50.731 INFO:teuthology.orchestra.run.smithi092.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.2", 2023-11-24T06:10:50.731 INFO:teuthology.orchestra.run.smithi092.stdout: "pid": "97844", 2023-11-24T06:10:50.731 INFO:teuthology.orchestra.run.smithi092.stdout: "root": "/" 2023-11-24T06:10:50.731 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-24T06:10:50.732 INFO:teuthology.orchestra.run.smithi092.stdout: "dentry_count": 0, 2023-11-24T06:10:50.732 INFO:teuthology.orchestra.run.smithi092.stdout: "dentry_pinned_count": 0, 2023-11-24T06:10:50.732 INFO:teuthology.orchestra.run.smithi092.stdout: "id": 4665, 2023-11-24T06:10:50.732 INFO:teuthology.orchestra.run.smithi092.stdout: "inst": { 2023-11-24T06:10:50.732 INFO:teuthology.orchestra.run.smithi092.stdout: "name": { 2023-11-24T06:10:50.732 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "client", 2023-11-24T06:10:50.732 INFO:teuthology.orchestra.run.smithi092.stdout: "num": 4665 2023-11-24T06:10:50.732 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-24T06:10:50.732 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": { 2023-11-24T06:10:50.733 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2023-11-24T06:10:50.733 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "192.168.0.2:0", 2023-11-24T06:10:50.733 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 355273277 2023-11-24T06:10:50.733 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-24T06:10:50.733 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-24T06:10:50.733 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": { 2023-11-24T06:10:50.733 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2023-11-24T06:10:50.733 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "192.168.0.2:0", 2023-11-24T06:10:50.733 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 355273277 2023-11-24T06:10:50.733 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-24T06:10:50.733 INFO:teuthology.orchestra.run.smithi092.stdout: "inst_str": "client.4665 192.168.0.2:0/355273277", 2023-11-24T06:10:50.733 INFO:teuthology.orchestra.run.smithi092.stdout: "addr_str": "192.168.0.2:0/355273277", 2023-11-24T06:10:50.734 INFO:teuthology.orchestra.run.smithi092.stdout: "inode_count": 1, 2023-11-24T06:10:50.734 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_epoch": 8, 2023-11-24T06:10:50.734 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_epoch": 20, 2023-11-24T06:10:50.734 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_epoch_barrier": 0, 2023-11-24T06:10:50.734 INFO:teuthology.orchestra.run.smithi092.stdout: "blocklisted": false, 2023-11-24T06:10:50.734 INFO:teuthology.orchestra.run.smithi092.stdout: "fs_name": "cephfs" 2023-11-24T06:10:50.734 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-24T06:10:50.745 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-24T06:10:50.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:50.953+0000 7fbe08251640 1 Processor -- start 2023-11-24T06:10:50.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:50.953+0000 7fbe08251640 1 -- start start 2023-11-24T06:10:50.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:50.955+0000 7fbe08251640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fbe0013e360 0x7fbe000cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:50.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:50.955+0000 7fbe08251640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fbe0015f9c0 0x7fbe000cf7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:50.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:50.955+0000 7fbe08251640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe0015c5f0 con 0x7fbe0013df80 2023-11-24T06:10:50.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:50.955+0000 7fbe08251640 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe0006c470 con 0x7fbe0013e360 2023-11-24T06:10:50.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:50.955+0000 7fbe08251640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe0006c750 con 0x7fbe0015f9c0 2023-11-24T06:10:50.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:50.955+0000 7fbe05fed640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fbe0013df80 0x7fbe0015bee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:39900/0 (socket says 172.21.15.92:39900) 2023-11-24T06:10:50.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:50.955+0000 7fbe05fed640 1 -- 172.21.15.92:0/1920351417 learned_addr learned my addr 172.21.15.92:0/1920351417 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-24T06:10:50.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:50.956+0000 7fbe067ee640 1 --2- 172.21.15.92:0/1920351417 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fbe0015f9c0 0x7fbe000cf7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:50.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:50.956+0000 7fbe057ec640 1 --2- 172.21.15.92:0/1920351417 >> v2:172.21.15.118:3300/0 conn(0x7fbe0013e360 0x7fbe000cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:50.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:50.956+0000 7fbe04feb640 1 -- 172.21.15.92:0/1920351417 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1404916530 0 0) 0x7fbe0015c5f0 con 0x7fbe0013df80 2023-11-24T06:10:50.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:50.956+0000 7fbe04feb640 1 -- 172.21.15.92:0/1920351417 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbdec003520 con 0x7fbe0013df80 2023-11-24T06:10:50.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:50.956+0000 7fbe04feb640 1 -- 172.21.15.92:0/1920351417 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 496642881 0 0) 0x7fbdec003520 con 0x7fbe0013df80 2023-11-24T06:10:50.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:50.956+0000 7fbe04feb640 1 -- 172.21.15.92:0/1920351417 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fbe0015f9c0 msgr2=0x7fbe000cf7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:50.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:50.956+0000 7fbe04feb640 1 --2- 172.21.15.92:0/1920351417 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fbe0015f9c0 0x7fbe000cf7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:50.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:50.956+0000 7fbe04feb640 1 -- 172.21.15.92:0/1920351417 >> v2:172.21.15.118:3300/0 conn(0x7fbe0013e360 msgr2=0x7fbe000cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:50.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:50.956+0000 7fbe04feb640 1 --2- 172.21.15.92:0/1920351417 >> v2:172.21.15.118:3300/0 conn(0x7fbe0013e360 0x7fbe000cf290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:50.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:50.956+0000 7fbe04feb640 1 -- 172.21.15.92:0/1920351417 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe0006ca30 con 0x7fbe0013df80 2023-11-24T06:10:50.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:50.956+0000 7fbe04feb640 1 -- 172.21.15.92:0/1920351417 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2607661350 0 0) 0x7fbdfc001690 con 0x7fbe0013df80 2023-11-24T06:10:50.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:50.956+0000 7fbe04feb640 1 -- 172.21.15.92:0/1920351417 >> v1:172.21.15.92:6789/0 conn(0x7fbe0013df80 legacy=0x7fbe0015bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:50.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:50.956+0000 7fbe04feb640 1 --2- 172.21.15.92:0/1920351417 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbdec003970 0x7fbdec003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:50.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:50.956+0000 7fbe04feb640 1 --2- 172.21.15.92:0/1920351417 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fbe0015f9c0 0x7fbdec0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:50.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:50.956+0000 7fbe04feb640 1 --2- 172.21.15.92:0/1920351417 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbdec004d00 0x7fbdec005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:50.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:50.956+0000 7fbe04feb640 1 -- 172.21.15.92:0/1920351417 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe0006c470 con 0x7fbdec004d00 2023-11-24T06:10:50.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:50.956+0000 7fbe04feb640 1 -- 172.21.15.92:0/1920351417 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe0006c750 con 0x7fbdec003970 2023-11-24T06:10:50.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:50.956+0000 7fbe04feb640 1 -- 172.21.15.92:0/1920351417 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdec00b980 con 0x7fbe0015f9c0 2023-11-24T06:10:50.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:50.957+0000 7fbe067ee640 1 --2- 172.21.15.92:0/1920351417 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbdec004d00 0x7fbdec005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:50.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:50.957+0000 7fbe05fed640 1 --2- 172.21.15.92:0/1920351417 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbdec003970 0x7fbdec003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:50.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:50.957+0000 7fbe057ec640 1 --2- 172.21.15.92:0/1920351417 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fbe0015f9c0 0x7fbdec0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:50.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:50.957+0000 7fbe067ee640 1 -- 172.21.15.92:0/1920351417 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fbe0015f9c0 msgr2=0x7fbdec0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:50.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:50.957+0000 7fbe067ee640 1 --2- 172.21.15.92:0/1920351417 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fbe0015f9c0 0x7fbdec0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:50.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:50.957+0000 7fbe067ee640 1 -- 172.21.15.92:0/1920351417 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbdec003970 msgr2=0x7fbdec003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:50.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:50.957+0000 7fbe067ee640 1 --2- 172.21.15.92:0/1920351417 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbdec003970 0x7fbdec003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:50.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:50.957+0000 7fbe067ee640 1 -- 172.21.15.92:0/1920351417 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdec003520 con 0x7fbdec004d00 2023-11-24T06:10:50.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:50.957+0000 7fbe05fed640 1 --2- 172.21.15.92:0/1920351417 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbdec003970 0x7fbdec003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:10:50.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:50.958+0000 7fbe067ee640 1 --2- 172.21.15.92:0/1920351417 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbdec004d00 0x7fbdec005180 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fbdf8009f90 tx=0x7fbdec0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa5424c5eb27c625 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:50.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:50.958+0000 7fbe04feb640 1 -- 172.21.15.92:0/1920351417 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdf802cbd0 con 0x7fbdec004d00 2023-11-24T06:10:50.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:50.959+0000 7fbe04feb640 1 -- 172.21.15.92:0/1920351417 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdf800f040 con 0x7fbdec004d00 2023-11-24T06:10:50.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:50.959+0000 7fbe04feb640 1 -- 172.21.15.92:0/1920351417 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdf8034440 con 0x7fbdec004d00 2023-11-24T06:10:50.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:50.959+0000 7fbe08251640 1 -- 172.21.15.92:0/1920351417 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbdec004d00 msgr2=0x7fbdec005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:50.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:50.959+0000 7fbe08251640 1 --2- 172.21.15.92:0/1920351417 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbdec004d00 0x7fbdec005180 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fbdf8009f90 tx=0x7fbdec0057f0 comp rx=0 tx=0).stop 2023-11-24T06:10:50.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:50.959+0000 7fbe067ee640 1 -- 172.21.15.92:0/1920351417 reap_dead start 2023-11-24T06:10:50.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:50.959+0000 7fbe08251640 1 -- 172.21.15.92:0/1920351417 shutdown_connections 2023-11-24T06:10:50.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:50.959+0000 7fbe08251640 1 -- 172.21.15.92:0/1920351417 >> 172.21.15.92:0/1920351417 conn(0x7fbe0006a800 msgr2=0x7fbe0006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:50.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:50.959+0000 7fbe08251640 1 -- 172.21.15.92:0/1920351417 shutdown_connections 2023-11-24T06:10:50.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:50.959+0000 7fbe08251640 1 -- 172.21.15.92:0/1920351417 wait complete. 2023-11-24T06:10:50.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:50.961+0000 7fbe08251640 1 Processor -- start 2023-11-24T06:10:50.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:50.961+0000 7fbe08251640 1 -- start start 2023-11-24T06:10:50.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:50.962+0000 7fbe08251640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbdec004d00 0x7fbe00153da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:50.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:50.962+0000 7fbe08251640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbe0014e000 0x7fbe0013d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:50.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:50.962+0000 7fbe08251640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fbe001532c0 0x7fbe0015a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:50.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:50.962+0000 7fbe08251640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe00152e30 con 0x7fbe0014e000 2023-11-24T06:10:50.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:50.962+0000 7fbe08251640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe0014e7e0 con 0x7fbdec004d00 2023-11-24T06:10:50.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:50.962+0000 7fbe08251640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe0014eac0 con 0x7fbe001532c0 2023-11-24T06:10:50.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:50.962+0000 7fbe057ec640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbe0014e000 0x7fbe0013d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:50.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:50.962+0000 7fbe057ec640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbe0014e000 0x7fbe0013d7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:36616/0 (socket says 172.21.15.92:36616) 2023-11-24T06:10:50.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:50.962+0000 7fbe057ec640 1 -- 172.21.15.92:0/1270234653 learned_addr learned my addr 172.21.15.92:0/1270234653 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:10:50.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:50.963+0000 7fbe067ee640 1 --2- 172.21.15.92:0/1270234653 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fbe001532c0 0x7fbe0015a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:50.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:50.963+0000 7fbe05fed640 1 --2- 172.21.15.92:0/1270234653 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbdec004d00 0x7fbe00153da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:50.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:50.963+0000 7fbe057ec640 1 -- 172.21.15.92:0/1270234653 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fbe001532c0 msgr2=0x7fbe0015a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:50.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:50.963+0000 7fbe057ec640 1 --2- 172.21.15.92:0/1270234653 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fbe001532c0 0x7fbe0015a180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:50.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:50.963+0000 7fbe057ec640 1 -- 172.21.15.92:0/1270234653 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbdec004d00 msgr2=0x7fbe00153da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:50.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:50.963+0000 7fbe057ec640 1 --2- 172.21.15.92:0/1270234653 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbdec004d00 0x7fbe00153da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:50.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:50.963+0000 7fbe057ec640 1 -- 172.21.15.92:0/1270234653 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe001527e0 con 0x7fbe0014e000 2023-11-24T06:10:50.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:50.963+0000 7fbe057ec640 1 --2- 172.21.15.92:0/1270234653 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbe0014e000 0x7fbe0013d7d0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fbdf00115f0 tx=0x7fbdf0011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:50.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:50.963+0000 7fbdf67fc640 1 -- 172.21.15.92:0/1270234653 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdf000f8f0 con 0x7fbe0014e000 2023-11-24T06:10:50.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:50.963+0000 7fbdf67fc640 1 -- 172.21.15.92:0/1270234653 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdf000fa50 con 0x7fbe0014e000 2023-11-24T06:10:50.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:50.963+0000 7fbe08251640 1 -- 172.21.15.92:0/1270234653 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe00152ac0 con 0x7fbe0014e000 2023-11-24T06:10:50.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:50.963+0000 7fbdf67fc640 1 -- 172.21.15.92:0/1270234653 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdf000fde0 con 0x7fbe0014e000 2023-11-24T06:10:50.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:50.964+0000 7fbe08251640 1 -- 172.21.15.92:0/1270234653 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe0014e460 con 0x7fbe0014e000 2023-11-24T06:10:50.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:50.965+0000 7fbe08251640 1 -- 172.21.15.92:0/1270234653 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe00152fa0 con 0x7fbe0014e000 2023-11-24T06:10:50.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:50.966+0000 7fbdf67fc640 1 -- 172.21.15.92:0/1270234653 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fbdf0015460 con 0x7fbe0014e000 2023-11-24T06:10:50.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:50.967+0000 7fbdf67fc640 1 --2- 172.21.15.92:0/1270234653 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fbde406ae70 0x7fbde406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:50.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:50.967+0000 7fbdf67fc640 1 -- 172.21.15.92:0/1270234653 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbdf0089fa0 con 0x7fbe0014e000 2023-11-24T06:10:50.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:50.967+0000 7fbe05fed640 1 --2- 172.21.15.92:0/1270234653 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fbde406ae70 0x7fbde406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:50.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:50.968+0000 7fbe05fed640 1 --2- 172.21.15.92:0/1270234653 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fbde406ae70 0x7fbde406d330 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fbe0015c780 tx=0x7fbdfc000f50 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:50.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:50.971+0000 7fbdf67fc640 1 -- 172.21.15.92:0/1270234653 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fbdf0054f20 con 0x7fbe0014e000 2023-11-24T06:10:51.201 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:51.199+0000 7fbe08251640 1 -- 172.21.15.92:0/1270234653 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fbe000675d0 con 0x7fbe0014e000 2023-11-24T06:10:51.202 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:51.199+0000 7fbdf67fc640 1 -- 172.21.15.92:0/1270234653 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7fbdf0058540 con 0x7fbe0014e000 2023-11-24T06:10:51.202 INFO:teuthology.orchestra.run.smithi092.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-24T06:10:51.204 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:51.202+0000 7fbe08251640 1 -- 172.21.15.92:0/1270234653 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fbde406ae70 msgr2=0x7fbde406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:51.204 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:51.202+0000 7fbe08251640 1 --2- 172.21.15.92:0/1270234653 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fbde406ae70 0x7fbde406d330 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fbe0015c780 tx=0x7fbdfc000f50 comp rx=0 tx=0).stop 2023-11-24T06:10:51.204 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:51.202+0000 7fbe08251640 1 -- 172.21.15.92:0/1270234653 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbe0014e000 msgr2=0x7fbe0013d7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:51.205 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:51.202+0000 7fbe08251640 1 --2- 172.21.15.92:0/1270234653 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbe0014e000 0x7fbe0013d7d0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fbdf00115f0 tx=0x7fbdf0011ab0 comp rx=0 tx=0).stop 2023-11-24T06:10:51.205 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:51.202+0000 7fbe08251640 1 -- 172.21.15.92:0/1270234653 shutdown_connections 2023-11-24T06:10:51.205 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:51.202+0000 7fbe08251640 1 --2- 172.21.15.92:0/1270234653 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fbde406ae70 0x7fbde406d330 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:51.205 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:51.202+0000 7fbe08251640 1 --2- 172.21.15.92:0/1270234653 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fbe001532c0 0x7fbe0015a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:51.205 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:51.202+0000 7fbe08251640 1 --2- 172.21.15.92:0/1270234653 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbe0014e000 0x7fbe0013d7d0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:51.205 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:51.202+0000 7fbe08251640 1 --2- 172.21.15.92:0/1270234653 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbdec004d00 0x7fbe00153da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:51.206 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:51.202+0000 7fbe08251640 1 -- 172.21.15.92:0/1270234653 >> 172.21.15.92:0/1270234653 conn(0x7fbe0006a800 msgr2=0x7fbe000ce090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:51.206 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:51.202+0000 7fbe08251640 1 -- 172.21.15.92:0/1270234653 shutdown_connections 2023-11-24T06:10:51.206 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:51.202+0000 7fbe08251640 1 -- 172.21.15.92:0/1270234653 wait complete. 2023-11-24T06:10:51.223 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-24T06:10:51.223 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-24T06:10:51.223 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi118.front.sepia.ceph.com 2023-11-24T06:10:51.223 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-11-24T06:10:51.223 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-11-24T06:10:51.223 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-24T06:10:51.223 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-24T06:10:51.224 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-24T06:10:51.224 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2023-11-24T06:10:51.224 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:10:51.224 DEBUG:teuthology.orchestra.run.smithi118:> ip addr 2023-11-24T06:10:51.285 INFO:teuthology.orchestra.run.smithi118.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-24T06:10:51.285 INFO:teuthology.orchestra.run.smithi118.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-24T06:10:51.285 INFO:teuthology.orchestra.run.smithi118.stdout: inet 127.0.0.1/8 scope host lo 2023-11-24T06:10:51.285 INFO:teuthology.orchestra.run.smithi118.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:10:51.285 INFO:teuthology.orchestra.run.smithi118.stdout: inet6 ::1/128 scope host 2023-11-24T06:10:51.285 INFO:teuthology.orchestra.run.smithi118.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:10:51.286 INFO:teuthology.orchestra.run.smithi118.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T06:10:51.286 INFO:teuthology.orchestra.run.smithi118.stdout: link/ether 0c:c4:7a:d8:a4:cc brd ff:ff:ff:ff:ff:ff 2023-11-24T06:10:51.286 INFO:teuthology.orchestra.run.smithi118.stdout: altname enp7s0f0 2023-11-24T06:10:51.286 INFO:teuthology.orchestra.run.smithi118.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T06:10:51.286 INFO:teuthology.orchestra.run.smithi118.stdout: link/ether 0c:c4:7a:d8:a4:cd brd ff:ff:ff:ff:ff:ff 2023-11-24T06:10:51.286 INFO:teuthology.orchestra.run.smithi118.stdout: altname enp7s0f1 2023-11-24T06:10:51.286 INFO:teuthology.orchestra.run.smithi118.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T06:10:51.286 INFO:teuthology.orchestra.run.smithi118.stdout: link/ether 0c:c4:7a:8f:d7:24 brd ff:ff:ff:ff:ff:ff 2023-11-24T06:10:51.287 INFO:teuthology.orchestra.run.smithi118.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-24T06:10:51.287 INFO:teuthology.orchestra.run.smithi118.stdout: link/ether 0c:c4:7a:8f:d7:25 brd ff:ff:ff:ff:ff:ff 2023-11-24T06:10:51.287 INFO:teuthology.orchestra.run.smithi118.stdout: inet 172.21.15.118/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-24T06:10:51.287 INFO:teuthology.orchestra.run.smithi118.stdout: valid_lft 42196sec preferred_lft 42196sec 2023-11-24T06:10:51.287 INFO:teuthology.orchestra.run.smithi118.stdout: inet6 fe80::ec4:7aff:fe8f:d725/64 scope link 2023-11-24T06:10:51.287 INFO:teuthology.orchestra.run.smithi118.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:10:51.288 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-11-24T06:10:51.288 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:10:51.288 DEBUG:teuthology.orchestra.run.smithi118:> (cd / && exec bash -c ' 2023-11-24T06:10:51.288 DEBUG:teuthology.orchestra.run.smithi118:> set -e 2023-11-24T06:10:51.289 DEBUG:teuthology.orchestra.run.smithi118:> sudo ip link add name ceph-brx type bridge 2023-11-24T06:10:51.289 DEBUG:teuthology.orchestra.run.smithi118:> sudo ip addr flush dev ceph-brx 2023-11-24T06:10:51.289 DEBUG:teuthology.orchestra.run.smithi118:> sudo ip link set ceph-brx up 2023-11-24T06:10:51.289 DEBUG:teuthology.orchestra.run.smithi118:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-11-24T06:10:51.289 DEBUG:teuthology.orchestra.run.smithi118:> ') 2023-11-24T06:10:51.772 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:10:51.772 DEBUG:teuthology.orchestra.run.smithi118:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-24T06:10:51.850 INFO:teuthology.orchestra.run.smithi118.stdout:1 2023-11-24T06:10:51.853 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:10:51.853 DEBUG:teuthology.orchestra.run.smithi118:> route 2023-11-24T06:10:51.908 INFO:teuthology.orchestra.run.smithi118.stdout:Kernel IP routing table 2023-11-24T06:10:51.908 INFO:teuthology.orchestra.run.smithi118.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-24T06:10:51.908 INFO:teuthology.orchestra.run.smithi118.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-24T06:10:51.908 INFO:teuthology.orchestra.run.smithi118.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-24T06:10:51.908 INFO:teuthology.orchestra.run.smithi118.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-24T06:10:51.909 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:10:51.909 DEBUG:teuthology.orchestra.run.smithi118:> (cd / && exec bash -c ' 2023-11-24T06:10:51.909 DEBUG:teuthology.orchestra.run.smithi118:> set -e 2023-11-24T06:10:51.909 DEBUG:teuthology.orchestra.run.smithi118:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-24T06:10:51.909 DEBUG:teuthology.orchestra.run.smithi118:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-24T06:10:51.910 DEBUG:teuthology.orchestra.run.smithi118:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-24T06:10:51.910 DEBUG:teuthology.orchestra.run.smithi118:> ') 2023-11-24T06:10:52.696 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:10:52.696 DEBUG:teuthology.orchestra.run.smithi118:> ip netns list 2023-11-24T06:10:52.755 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:10:52.755 DEBUG:teuthology.orchestra.run.smithi118:> ip netns list-id 2023-11-24T06:10:52.808 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:10:52.808 DEBUG:teuthology.orchestra.run.smithi118:> (cd / && exec bash -c ' 2023-11-24T06:10:52.808 DEBUG:teuthology.orchestra.run.smithi118:> set -e 2023-11-24T06:10:52.808 DEBUG:teuthology.orchestra.run.smithi118:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-24T06:10:52.808 DEBUG:teuthology.orchestra.run.smithi118:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2023-11-24T06:10:52.808 DEBUG:teuthology.orchestra.run.smithi118:> ') 2023-11-24T06:10:52.946 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2023-11-24T06:10:52.946 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:10:52.946 DEBUG:teuthology.orchestra.run.smithi118:> (cd / && exec bash -c ' 2023-11-24T06:10:52.946 DEBUG:teuthology.orchestra.run.smithi118:> set -e 2023-11-24T06:10:52.946 DEBUG:teuthology.orchestra.run.smithi118:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2023-11-24T06:10:52.947 DEBUG:teuthology.orchestra.run.smithi118:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-11-24T06:10:52.947 DEBUG:teuthology.orchestra.run.smithi118:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2023-11-24T06:10:52.947 DEBUG:teuthology.orchestra.run.smithi118:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2023-11-24T06:10:52.947 DEBUG:teuthology.orchestra.run.smithi118:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2023-11-24T06:10:52.947 DEBUG:teuthology.orchestra.run.smithi118:> ') 2023-11-24T06:10:53.253 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:10:53.254 DEBUG:teuthology.orchestra.run.smithi118:> (cd / && exec bash -c ' 2023-11-24T06:10:53.254 DEBUG:teuthology.orchestra.run.smithi118:> set -e 2023-11-24T06:10:53.254 DEBUG:teuthology.orchestra.run.smithi118:> sudo ip link set brx.0 up 2023-11-24T06:10:53.254 DEBUG:teuthology.orchestra.run.smithi118:> sudo ip link set dev brx.0 master ceph-brx 2023-11-24T06:10:53.254 DEBUG:teuthology.orchestra.run.smithi118:> ') 2023-11-24T06:10:53.418 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2023-11-24T06:10:53.419 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T06:10:53.419 DEBUG:teuthology.orchestra.run.smithi118:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-11-24T06:10:53.466 INFO:teuthology.orchestra.run.smithi118.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-11-24T06:10:53.467 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T06:10:53.467 DEBUG:teuthology.orchestra.run.smithi118:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-11-24T06:10:53.563 DEBUG:teuthology.orchestra.run.smithi118:> sudo modprobe fuse 2023-11-24T06:10:53.650 DEBUG:teuthology.orchestra.run.smithi118:> cat /proc/self/mounts | awk '{print $2}' 2023-11-24T06:10:53.697 INFO:teuthology.orchestra.run.smithi118.stdout:/proc 2023-11-24T06:10:53.697 INFO:teuthology.orchestra.run.smithi118.stdout:/sys 2023-11-24T06:10:53.697 INFO:teuthology.orchestra.run.smithi118.stdout:/dev 2023-11-24T06:10:53.697 INFO:teuthology.orchestra.run.smithi118.stdout:/sys/kernel/security 2023-11-24T06:10:53.697 INFO:teuthology.orchestra.run.smithi118.stdout:/dev/shm 2023-11-24T06:10:53.697 INFO:teuthology.orchestra.run.smithi118.stdout:/dev/pts 2023-11-24T06:10:53.698 INFO:teuthology.orchestra.run.smithi118.stdout:/run 2023-11-24T06:10:53.698 INFO:teuthology.orchestra.run.smithi118.stdout:/sys/fs/cgroup 2023-11-24T06:10:53.698 INFO:teuthology.orchestra.run.smithi118.stdout:/sys/fs/pstore 2023-11-24T06:10:53.698 INFO:teuthology.orchestra.run.smithi118.stdout:/ 2023-11-24T06:10:53.698 INFO:teuthology.orchestra.run.smithi118.stdout:/sys/fs/selinux 2023-11-24T06:10:53.698 INFO:teuthology.orchestra.run.smithi118.stdout:/proc/sys/fs/binfmt_misc 2023-11-24T06:10:53.698 INFO:teuthology.orchestra.run.smithi118.stdout:/dev/mqueue 2023-11-24T06:10:53.698 INFO:teuthology.orchestra.run.smithi118.stdout:/dev/hugepages 2023-11-24T06:10:53.698 INFO:teuthology.orchestra.run.smithi118.stdout:/sys/kernel/debug 2023-11-24T06:10:53.698 INFO:teuthology.orchestra.run.smithi118.stdout:/sys/kernel/tracing 2023-11-24T06:10:53.698 INFO:teuthology.orchestra.run.smithi118.stdout:/sys/fs/fuse/connections 2023-11-24T06:10:53.699 INFO:teuthology.orchestra.run.smithi118.stdout:/sys/kernel/config 2023-11-24T06:10:53.699 INFO:teuthology.orchestra.run.smithi118.stdout:/run/credentials/systemd-sysctl.service 2023-11-24T06:10:53.699 INFO:teuthology.orchestra.run.smithi118.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-24T06:10:53.699 INFO:teuthology.orchestra.run.smithi118.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-24T06:10:53.699 INFO:teuthology.orchestra.run.smithi118.stdout:/var/lib/nfs/rpc_pipefs 2023-11-24T06:10:53.699 INFO:teuthology.orchestra.run.smithi118.stdout:/run/user/1000 2023-11-24T06:10:53.699 INFO:teuthology.orchestra.run.smithi118.stdout:/var/lib/ceph 2023-11-24T06:10:53.699 INFO:teuthology.orchestra.run.smithi118.stdout:/var/lib/ceph/osd/ceph-4 2023-11-24T06:10:53.699 INFO:teuthology.orchestra.run.smithi118.stdout:/var/lib/ceph/osd/ceph-5 2023-11-24T06:10:53.699 INFO:teuthology.orchestra.run.smithi118.stdout:/var/lib/ceph/osd/ceph-6 2023-11-24T06:10:53.699 INFO:teuthology.orchestra.run.smithi118.stdout:/var/lib/ceph/osd/ceph-7 2023-11-24T06:10:53.700 INFO:teuthology.orchestra.run.smithi118.stdout:/run/netns 2023-11-24T06:10:53.700 INFO:teuthology.orchestra.run.smithi118.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-24T06:10:53.700 INFO:teuthology.orchestra.run.smithi118.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-24T06:10:53.701 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:10:53.701 DEBUG:teuthology.orchestra.run.smithi118:> ls /sys/fs/fuse/connections 2023-11-24T06:10:53.746 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-11-24T06:10:53.747 DEBUG:teuthology.orchestra.run.smithi118:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2023-11-24T06:10:53.796 DEBUG:teuthology.orchestra.run.smithi118:> sudo modprobe fuse 2023-11-24T06:10:53.878 DEBUG:teuthology.orchestra.run.smithi118:> cat /proc/self/mounts | awk '{print $2}' 2023-11-24T06:10:53.921 INFO:teuthology.orchestra.run.smithi118.stdout:/proc 2023-11-24T06:10:53.921 INFO:teuthology.orchestra.run.smithi118.stdout:/sys 2023-11-24T06:10:53.921 INFO:teuthology.orchestra.run.smithi118.stdout:/dev 2023-11-24T06:10:53.921 INFO:teuthology.orchestra.run.smithi118.stdout:/sys/kernel/security 2023-11-24T06:10:53.921 INFO:teuthology.orchestra.run.smithi118.stdout:/dev/shm 2023-11-24T06:10:53.921 INFO:teuthology.orchestra.run.smithi118.stdout:/dev/pts 2023-11-24T06:10:53.921 INFO:teuthology.orchestra.run.smithi118.stdout:/run 2023-11-24T06:10:53.922 INFO:teuthology.orchestra.run.smithi118.stdout:/sys/fs/cgroup 2023-11-24T06:10:53.922 INFO:teuthology.orchestra.run.smithi118.stdout:/sys/fs/pstore 2023-11-24T06:10:53.922 INFO:teuthology.orchestra.run.smithi118.stdout:/ 2023-11-24T06:10:53.922 INFO:teuthology.orchestra.run.smithi118.stdout:/sys/fs/selinux 2023-11-24T06:10:53.922 INFO:teuthology.orchestra.run.smithi118.stdout:/proc/sys/fs/binfmt_misc 2023-11-24T06:10:53.922 INFO:teuthology.orchestra.run.smithi118.stdout:/dev/mqueue 2023-11-24T06:10:53.922 INFO:teuthology.orchestra.run.smithi118.stdout:/dev/hugepages 2023-11-24T06:10:53.922 INFO:teuthology.orchestra.run.smithi118.stdout:/sys/kernel/debug 2023-11-24T06:10:53.922 INFO:teuthology.orchestra.run.smithi118.stdout:/sys/kernel/tracing 2023-11-24T06:10:53.923 INFO:teuthology.orchestra.run.smithi118.stdout:/sys/fs/fuse/connections 2023-11-24T06:10:53.923 INFO:teuthology.orchestra.run.smithi118.stdout:/sys/kernel/config 2023-11-24T06:10:53.923 INFO:teuthology.orchestra.run.smithi118.stdout:/run/credentials/systemd-sysctl.service 2023-11-24T06:10:53.923 INFO:teuthology.orchestra.run.smithi118.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-24T06:10:53.923 INFO:teuthology.orchestra.run.smithi118.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-24T06:10:53.923 INFO:teuthology.orchestra.run.smithi118.stdout:/var/lib/nfs/rpc_pipefs 2023-11-24T06:10:53.923 INFO:teuthology.orchestra.run.smithi118.stdout:/run/user/1000 2023-11-24T06:10:53.923 INFO:teuthology.orchestra.run.smithi118.stdout:/var/lib/ceph 2023-11-24T06:10:53.923 INFO:teuthology.orchestra.run.smithi118.stdout:/var/lib/ceph/osd/ceph-4 2023-11-24T06:10:53.923 INFO:teuthology.orchestra.run.smithi118.stdout:/var/lib/ceph/osd/ceph-5 2023-11-24T06:10:53.924 INFO:teuthology.orchestra.run.smithi118.stdout:/var/lib/ceph/osd/ceph-6 2023-11-24T06:10:53.924 INFO:teuthology.orchestra.run.smithi118.stdout:/var/lib/ceph/osd/ceph-7 2023-11-24T06:10:53.924 INFO:teuthology.orchestra.run.smithi118.stdout:/run/netns 2023-11-24T06:10:53.924 INFO:teuthology.orchestra.run.smithi118.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-24T06:10:53.924 INFO:teuthology.orchestra.run.smithi118.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-24T06:10:53.925 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:10:53.925 DEBUG:teuthology.orchestra.run.smithi118:> ls /sys/fs/fuse/connections 2023-11-24T06:10:53.998 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi118.stderr:2023-11-24T06:10:53.996+0000 7fd28c153480 -1 init, newargv = 0x55d4a59f8590 newargc=15 2023-11-24T06:10:53.998 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi118.stderr:ceph-fuse[93142]: starting ceph client 2023-11-24T06:10:54.009 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi118.stderr:ceph-fuse[93142]: starting fuse 2023-11-24T06:10:54.015 INFO:teuthology.orchestra.run.smithi118.stdout:41 2023-11-24T06:10:54.016 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2023-11-24T06:10:54.016 DEBUG:teuthology.orchestra.run.smithi118:> sudo stdin-killer -- python3 -c ' 2023-11-24T06:10:54.016 DEBUG:teuthology.orchestra.run.smithi118:> import glob 2023-11-24T06:10:54.016 DEBUG:teuthology.orchestra.run.smithi118:> import re 2023-11-24T06:10:54.017 DEBUG:teuthology.orchestra.run.smithi118:> import os 2023-11-24T06:10:54.017 DEBUG:teuthology.orchestra.run.smithi118:> import subprocess 2023-11-24T06:10:54.017 DEBUG:teuthology.orchestra.run.smithi118:> 2023-11-24T06:10:54.017 DEBUG:teuthology.orchestra.run.smithi118:> def _find_admin_socket(client_name): 2023-11-24T06:10:54.017 DEBUG:teuthology.orchestra.run.smithi118:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2023-11-24T06:10:54.017 DEBUG:teuthology.orchestra.run.smithi118:> files = glob.glob(asok_path) 2023-11-24T06:10:54.017 DEBUG:teuthology.orchestra.run.smithi118:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2023-11-24T06:10:54.017 DEBUG:teuthology.orchestra.run.smithi118:> 2023-11-24T06:10:54.017 DEBUG:teuthology.orchestra.run.smithi118:> # Given a non-glob path, it better be there 2023-11-24T06:10:54.017 DEBUG:teuthology.orchestra.run.smithi118:> if "*" not in asok_path: 2023-11-24T06:10:54.017 DEBUG:teuthology.orchestra.run.smithi118:> assert(len(files) == 1) 2023-11-24T06:10:54.017 DEBUG:teuthology.orchestra.run.smithi118:> return files[0] 2023-11-24T06:10:54.017 DEBUG:teuthology.orchestra.run.smithi118:> 2023-11-24T06:10:54.018 DEBUG:teuthology.orchestra.run.smithi118:> for f in files: 2023-11-24T06:10:54.018 DEBUG:teuthology.orchestra.run.smithi118:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-11-24T06:10:54.018 DEBUG:teuthology.orchestra.run.smithi118:> if os.path.exists("/proc/{0}".format(pid)): 2023-11-24T06:10:54.018 DEBUG:teuthology.orchestra.run.smithi118:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-11-24T06:10:54.018 DEBUG:teuthology.orchestra.run.smithi118:> contents = proc_f.read() 2023-11-24T06:10:54.018 DEBUG:teuthology.orchestra.run.smithi118:> if mountpoint in contents: 2023-11-24T06:10:54.018 DEBUG:teuthology.orchestra.run.smithi118:> return f 2023-11-24T06:10:54.018 DEBUG:teuthology.orchestra.run.smithi118:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-11-24T06:10:54.018 DEBUG:teuthology.orchestra.run.smithi118:> 2023-11-24T06:10:54.018 DEBUG:teuthology.orchestra.run.smithi118:> print(_find_admin_socket("client.1")) 2023-11-24T06:10:54.018 DEBUG:teuthology.orchestra.run.smithi118:> ' 2023-11-24T06:10:54.217 INFO:teuthology.orchestra.run.smithi118.stdout:/var/run/ceph/ceph-client.1.93142.asok 2023-11-24T06:10:54.221 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-24T06:10:54.234 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.93142.asok 2023-11-24T06:10:54.234 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:10:54.234 DEBUG:teuthology.orchestra.run.smithi118:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.93142.asok status 2023-11-24T06:10:54.476 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2023-11-24T06:10:54.477 INFO:teuthology.orchestra.run.smithi118.stdout: "metadata": { 2023-11-24T06:10:54.477 INFO:teuthology.orchestra.run.smithi118.stdout: "ceph_sha1": "a13ea5ff00cb4df7a9cc941166ddeb358881144c", 2023-11-24T06:10:54.477 INFO:teuthology.orchestra.run.smithi118.stdout: "ceph_version": "ceph version 18.0.0-7484-ga13ea5ff (a13ea5ff00cb4df7a9cc941166ddeb358881144c) reef (dev)", 2023-11-24T06:10:54.477 INFO:teuthology.orchestra.run.smithi118.stdout: "entity_id": "1", 2023-11-24T06:10:54.477 INFO:teuthology.orchestra.run.smithi118.stdout: "hostname": "smithi118", 2023-11-24T06:10:54.477 INFO:teuthology.orchestra.run.smithi118.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2023-11-24T06:10:54.477 INFO:teuthology.orchestra.run.smithi118.stdout: "pid": "93142", 2023-11-24T06:10:54.477 INFO:teuthology.orchestra.run.smithi118.stdout: "root": "/" 2023-11-24T06:10:54.477 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-11-24T06:10:54.477 INFO:teuthology.orchestra.run.smithi118.stdout: "dentry_count": 0, 2023-11-24T06:10:54.478 INFO:teuthology.orchestra.run.smithi118.stdout: "dentry_pinned_count": 0, 2023-11-24T06:10:54.478 INFO:teuthology.orchestra.run.smithi118.stdout: "id": 4466, 2023-11-24T06:10:54.478 INFO:teuthology.orchestra.run.smithi118.stdout: "inst": { 2023-11-24T06:10:54.478 INFO:teuthology.orchestra.run.smithi118.stdout: "name": { 2023-11-24T06:10:54.478 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "client", 2023-11-24T06:10:54.478 INFO:teuthology.orchestra.run.smithi118.stdout: "num": 4466 2023-11-24T06:10:54.478 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-11-24T06:10:54.478 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": { 2023-11-24T06:10:54.478 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v1", 2023-11-24T06:10:54.478 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "192.168.0.1:0", 2023-11-24T06:10:54.478 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 2710050971 2023-11-24T06:10:54.478 INFO:teuthology.orchestra.run.smithi118.stdout: } 2023-11-24T06:10:54.479 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-11-24T06:10:54.479 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": { 2023-11-24T06:10:54.479 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v1", 2023-11-24T06:10:54.479 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "192.168.0.1:0", 2023-11-24T06:10:54.479 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 2710050971 2023-11-24T06:10:54.479 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-11-24T06:10:54.479 INFO:teuthology.orchestra.run.smithi118.stdout: "inst_str": "client.4466 192.168.0.1:0/2710050971", 2023-11-24T06:10:54.479 INFO:teuthology.orchestra.run.smithi118.stdout: "addr_str": "192.168.0.1:0/2710050971", 2023-11-24T06:10:54.479 INFO:teuthology.orchestra.run.smithi118.stdout: "inode_count": 1, 2023-11-24T06:10:54.479 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_epoch": 8, 2023-11-24T06:10:54.479 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_epoch": 20, 2023-11-24T06:10:54.479 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_epoch_barrier": 0, 2023-11-24T06:10:54.479 INFO:teuthology.orchestra.run.smithi118.stdout: "blocklisted": false, 2023-11-24T06:10:54.480 INFO:teuthology.orchestra.run.smithi118.stdout: "fs_name": "cephfs" 2023-11-24T06:10:54.480 INFO:teuthology.orchestra.run.smithi118.stdout:} 2023-11-24T06:10:54.492 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-24T06:10:54.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:54.706+0000 7fdb0497c640 1 Processor -- start 2023-11-24T06:10:54.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:54.706+0000 7fdb0497c640 1 -- start start 2023-11-24T06:10:54.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:54.708+0000 7fdb0497c640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fdb000b3ae0 0x7fdb0015e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:54.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:54.708+0000 7fdb0497c640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fdb000cf290 0x7fdb000cf720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:54.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:54.708+0000 7fdb0497c640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb00070c40 con 0x7fdb000ae5a0 2023-11-24T06:10:54.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:54.708+0000 7fdb0497c640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb00071450 con 0x7fdb000cf290 2023-11-24T06:10:54.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:54.708+0000 7fdb0497c640 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb000cfc60 con 0x7fdb000b3ae0 2023-11-24T06:10:54.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:54.708+0000 7fdafe59c640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fdb000ae5a0 0x7fdb0006cbe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:39902/0 (socket says 172.21.15.92:39902) 2023-11-24T06:10:54.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:54.708+0000 7fdafed9d640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fdb000cf290 0x7fdb000cf720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:54.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:54.708+0000 7fdafe59c640 1 -- 172.21.15.92:0/4023311547 learned_addr learned my addr 172.21.15.92:0/4023311547 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-24T06:10:54.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:54.708+0000 7fdafdd9b640 1 --2- 172.21.15.92:0/4023311547 >> v2:172.21.15.118:3300/0 conn(0x7fdb000b3ae0 0x7fdb0015e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:54.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:54.708+0000 7fdafd59a640 1 -- 172.21.15.92:0/4023311547 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 83534089 0 0) 0x7fdb00070c40 con 0x7fdb000ae5a0 2023-11-24T06:10:54.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:54.709+0000 7fdafd59a640 1 -- 172.21.15.92:0/4023311547 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdae8003520 con 0x7fdb000ae5a0 2023-11-24T06:10:54.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:54.709+0000 7fdafed9d640 1 -- 172.21.15.92:0/4023311547 >> v2:172.21.15.118:3300/0 conn(0x7fdb000b3ae0 msgr2=0x7fdb0015e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:54.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:54.709+0000 7fdafed9d640 1 --2- 172.21.15.92:0/4023311547 >> v2:172.21.15.118:3300/0 conn(0x7fdb000b3ae0 0x7fdb0015e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:54.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:54.709+0000 7fdafed9d640 1 -- 172.21.15.92:0/4023311547 >> v1:172.21.15.92:6789/0 conn(0x7fdb000ae5a0 legacy=0x7fdb0006cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:54.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:54.709+0000 7fdafed9d640 1 -- 172.21.15.92:0/4023311547 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb0006c470 con 0x7fdb000cf290 2023-11-24T06:10:54.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:54.710+0000 7fdafed9d640 1 --2- 172.21.15.92:0/4023311547 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdb000cf290 0x7fdb000cf720 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fdb000cfea0 tx=0x7fdaf802bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec3e225b958e8569 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:54.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:54.710+0000 7fdafd59a640 1 -- 172.21.15.92:0/4023311547 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaf802cd00 con 0x7fdb000cf290 2023-11-24T06:10:54.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:54.710+0000 7fdafd59a640 1 -- 172.21.15.92:0/4023311547 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdaf802ce60 con 0x7fdb000cf290 2023-11-24T06:10:54.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:54.710+0000 7fdafd59a640 1 -- 172.21.15.92:0/4023311547 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaf8034660 con 0x7fdb000cf290 2023-11-24T06:10:54.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:54.710+0000 7fdb0497c640 1 -- 172.21.15.92:0/4023311547 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdb000cf290 msgr2=0x7fdb000cf720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:54.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:54.710+0000 7fdb0497c640 1 --2- 172.21.15.92:0/4023311547 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdb000cf290 0x7fdb000cf720 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fdb000cfea0 tx=0x7fdaf802bf50 comp rx=0 tx=0).stop 2023-11-24T06:10:54.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:54.711+0000 7fdb0497c640 1 -- 172.21.15.92:0/4023311547 shutdown_connections 2023-11-24T06:10:54.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:54.711+0000 7fdb0497c640 1 --2- 172.21.15.92:0/4023311547 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdb000cf290 0x7fdb000cf720 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:54.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:54.711+0000 7fdb0497c640 1 --2- 172.21.15.92:0/4023311547 >> v2:172.21.15.118:3300/0 conn(0x7fdb000b3ae0 0x7fdb0015e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:54.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:54.711+0000 7fdb0497c640 1 -- 172.21.15.92:0/4023311547 >> 172.21.15.92:0/4023311547 conn(0x7fdb0006a800 msgr2=0x7fdb0006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:54.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:54.711+0000 7fdb0497c640 1 -- 172.21.15.92:0/4023311547 shutdown_connections 2023-11-24T06:10:54.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:54.711+0000 7fdb0497c640 1 -- 172.21.15.92:0/4023311547 wait complete. 2023-11-24T06:10:54.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:54.712+0000 7fdb0497c640 1 Processor -- start 2023-11-24T06:10:54.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:54.713+0000 7fdb0497c640 1 -- start start 2023-11-24T06:10:54.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:54.713+0000 7fdb0497c640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fdb000ae5a0 0x7fdb0014de10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:54.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:54.713+0000 7fdb0497c640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fdb000b3ae0 0x7fdb00157f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:54.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:54.713+0000 7fdb0497c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdb000cf290 0x7fdb00158450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:54.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:54.713+0000 7fdb0497c640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb0013c3b0 con 0x7fdb000cf290 2023-11-24T06:10:54.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:54.713+0000 7fdb0497c640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb0015eb50 con 0x7fdb000ae5a0 2023-11-24T06:10:54.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:54.713+0000 7fdb0497c640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb0014e350 con 0x7fdb000b3ae0 2023-11-24T06:10:54.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:54.714+0000 7fdafed9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdb000cf290 0x7fdb00158450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:54.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:54.714+0000 7fdafed9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdb000cf290 0x7fdb00158450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:36642/0 (socket says 172.21.15.92:36642) 2023-11-24T06:10:54.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:54.714+0000 7fdafed9d640 1 -- 172.21.15.92:0/3591123982 learned_addr learned my addr 172.21.15.92:0/3591123982 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:10:54.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:54.714+0000 7fdafe59c640 1 --2- 172.21.15.92:0/3591123982 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fdb000ae5a0 0x7fdb0014de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:54.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:54.714+0000 7fdafdd9b640 1 --2- 172.21.15.92:0/3591123982 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fdb000b3ae0 0x7fdb00157f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:54.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:54.714+0000 7fdafed9d640 1 -- 172.21.15.92:0/3591123982 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fdb000b3ae0 msgr2=0x7fdb00157f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:54.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:54.714+0000 7fdafed9d640 1 --2- 172.21.15.92:0/3591123982 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fdb000b3ae0 0x7fdb00157f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:54.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:54.714+0000 7fdafed9d640 1 -- 172.21.15.92:0/3591123982 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fdb000ae5a0 msgr2=0x7fdb0014de10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:54.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:54.714+0000 7fdafed9d640 1 --2- 172.21.15.92:0/3591123982 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fdb000ae5a0 0x7fdb0014de10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:54.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:54.714+0000 7fdafed9d640 1 -- 172.21.15.92:0/3591123982 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdaf8009a70 con 0x7fdb000cf290 2023-11-24T06:10:54.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:54.715+0000 7fdafed9d640 1 --2- 172.21.15.92:0/3591123982 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdb000cf290 0x7fdb00158450 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fdaf802c560 tx=0x7fdaf8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:54.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:54.715+0000 7fdaeeffd640 1 -- 172.21.15.92:0/3591123982 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaf803d600 con 0x7fdb000cf290 2023-11-24T06:10:54.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:54.715+0000 7fdaeeffd640 1 -- 172.21.15.92:0/3591123982 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdaf803d780 con 0x7fdb000cf290 2023-11-24T06:10:54.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:54.715+0000 7fdb0497c640 1 -- 172.21.15.92:0/3591123982 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb0015c460 con 0x7fdb000cf290 2023-11-24T06:10:54.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:54.715+0000 7fdaeeffd640 1 -- 172.21.15.92:0/3591123982 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaf803db30 con 0x7fdb000cf290 2023-11-24T06:10:54.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:54.715+0000 7fdb0497c640 1 -- 172.21.15.92:0/3591123982 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb0013c8e0 con 0x7fdb000cf290 2023-11-24T06:10:54.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:54.716+0000 7fdb0497c640 1 -- 172.21.15.92:0/3591123982 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb000675d0 con 0x7fdb000cf290 2023-11-24T06:10:54.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:54.717+0000 7fdaeeffd640 1 -- 172.21.15.92:0/3591123982 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fdaf803b040 con 0x7fdb000cf290 2023-11-24T06:10:54.720 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:54.718+0000 7fdaeeffd640 1 --2- 172.21.15.92:0/3591123982 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fdae006ae70 0x7fdae006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:54.720 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:54.718+0000 7fdaeeffd640 1 -- 172.21.15.92:0/3591123982 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdaf80abba0 con 0x7fdb000cf290 2023-11-24T06:10:54.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:54.718+0000 7fdafe59c640 1 --2- 172.21.15.92:0/3591123982 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fdae006ae70 0x7fdae006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:54.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:54.719+0000 7fdafe59c640 1 --2- 172.21.15.92:0/3591123982 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fdae006ae70 0x7fdae006d330 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fdaf00011e0 tx=0x7fdaf0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:54.725 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:54.723+0000 7fdaeeffd640 1 -- 172.21.15.92:0/3591123982 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fdaf8076bb0 con 0x7fdb000cf290 2023-11-24T06:10:54.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:54.957+0000 7fdb0497c640 1 -- 172.21.15.92:0/3591123982 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fdb000708d0 con 0x7fdb000cf290 2023-11-24T06:10:54.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:54.958+0000 7fdaeeffd640 1 -- 172.21.15.92:0/3591123982 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7fdaf8039020 con 0x7fdb000cf290 2023-11-24T06:10:54.960 INFO:teuthology.orchestra.run.smithi092.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-24T06:10:54.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:54.960+0000 7fdb0497c640 1 -- 172.21.15.92:0/3591123982 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fdae006ae70 msgr2=0x7fdae006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:54.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:54.960+0000 7fdb0497c640 1 --2- 172.21.15.92:0/3591123982 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fdae006ae70 0x7fdae006d330 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fdaf00011e0 tx=0x7fdaf0009040 comp rx=0 tx=0).stop 2023-11-24T06:10:54.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:54.960+0000 7fdb0497c640 1 -- 172.21.15.92:0/3591123982 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdb000cf290 msgr2=0x7fdb00158450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:54.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:54.960+0000 7fdb0497c640 1 --2- 172.21.15.92:0/3591123982 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdb000cf290 0x7fdb00158450 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fdaf802c560 tx=0x7fdaf8004a40 comp rx=0 tx=0).stop 2023-11-24T06:10:54.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:54.960+0000 7fdb0497c640 1 -- 172.21.15.92:0/3591123982 shutdown_connections 2023-11-24T06:10:54.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:54.960+0000 7fdb0497c640 1 --2- 172.21.15.92:0/3591123982 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fdae006ae70 0x7fdae006d330 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:54.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:54.960+0000 7fdb0497c640 1 --2- 172.21.15.92:0/3591123982 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdb000cf290 0x7fdb00158450 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:54.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:54.960+0000 7fdb0497c640 1 --2- 172.21.15.92:0/3591123982 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fdb000b3ae0 0x7fdb00157f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:54.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:54.960+0000 7fdb0497c640 1 --2- 172.21.15.92:0/3591123982 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fdb000ae5a0 0x7fdb0014de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:54.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:54.960+0000 7fdb0497c640 1 -- 172.21.15.92:0/3591123982 >> 172.21.15.92:0/3591123982 conn(0x7fdb0006a800 msgr2=0x7fdb0015a990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:54.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:54.961+0000 7fdb0497c640 1 -- 172.21.15.92:0/3591123982 shutdown_connections 2023-11-24T06:10:54.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:54.961+0000 7fdb0497c640 1 -- 172.21.15.92:0/3591123982 wait complete. 2023-11-24T06:10:54.981 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-24T06:10:54.982 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-24T06:10:54.982 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi118.front.sepia.ceph.com 2023-11-24T06:10:54.982 INFO:tasks.cephfs.mount:self.client.name = client.3 2023-11-24T06:10:54.982 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2023-11-24T06:10:54.982 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-24T06:10:54.982 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-24T06:10:54.982 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-24T06:10:54.982 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2023-11-24T06:10:54.982 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:10:54.983 DEBUG:teuthology.orchestra.run.smithi118:> ip addr 2023-11-24T06:10:55.042 INFO:teuthology.orchestra.run.smithi118.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-24T06:10:55.042 INFO:teuthology.orchestra.run.smithi118.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-24T06:10:55.042 INFO:teuthology.orchestra.run.smithi118.stdout: inet 127.0.0.1/8 scope host lo 2023-11-24T06:10:55.042 INFO:teuthology.orchestra.run.smithi118.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:10:55.042 INFO:teuthology.orchestra.run.smithi118.stdout: inet6 ::1/128 scope host 2023-11-24T06:10:55.042 INFO:teuthology.orchestra.run.smithi118.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:10:55.043 INFO:teuthology.orchestra.run.smithi118.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T06:10:55.043 INFO:teuthology.orchestra.run.smithi118.stdout: link/ether 0c:c4:7a:d8:a4:cc brd ff:ff:ff:ff:ff:ff 2023-11-24T06:10:55.043 INFO:teuthology.orchestra.run.smithi118.stdout: altname enp7s0f0 2023-11-24T06:10:55.043 INFO:teuthology.orchestra.run.smithi118.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T06:10:55.043 INFO:teuthology.orchestra.run.smithi118.stdout: link/ether 0c:c4:7a:d8:a4:cd brd ff:ff:ff:ff:ff:ff 2023-11-24T06:10:55.043 INFO:teuthology.orchestra.run.smithi118.stdout: altname enp7s0f1 2023-11-24T06:10:55.043 INFO:teuthology.orchestra.run.smithi118.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T06:10:55.043 INFO:teuthology.orchestra.run.smithi118.stdout: link/ether 0c:c4:7a:8f:d7:24 brd ff:ff:ff:ff:ff:ff 2023-11-24T06:10:55.043 INFO:teuthology.orchestra.run.smithi118.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-24T06:10:55.043 INFO:teuthology.orchestra.run.smithi118.stdout: link/ether 0c:c4:7a:8f:d7:25 brd ff:ff:ff:ff:ff:ff 2023-11-24T06:10:55.043 INFO:teuthology.orchestra.run.smithi118.stdout: inet 172.21.15.118/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-24T06:10:55.043 INFO:teuthology.orchestra.run.smithi118.stdout: valid_lft 42192sec preferred_lft 42192sec 2023-11-24T06:10:55.044 INFO:teuthology.orchestra.run.smithi118.stdout: inet6 fe80::ec4:7aff:fe8f:d725/64 scope link 2023-11-24T06:10:55.044 INFO:teuthology.orchestra.run.smithi118.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:10:55.044 INFO:teuthology.orchestra.run.smithi118.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-24T06:10:55.044 INFO:teuthology.orchestra.run.smithi118.stdout: link/ether 16:ef:1e:16:88:fb brd ff:ff:ff:ff:ff:ff 2023-11-24T06:10:55.044 INFO:teuthology.orchestra.run.smithi118.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-24T06:10:55.044 INFO:teuthology.orchestra.run.smithi118.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:10:55.044 INFO:teuthology.orchestra.run.smithi118.stdout: inet6 fe80::50a3:24ff:fea4:c41e/64 scope link 2023-11-24T06:10:55.044 INFO:teuthology.orchestra.run.smithi118.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:10:55.044 INFO:teuthology.orchestra.run.smithi118.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-24T06:10:55.044 INFO:teuthology.orchestra.run.smithi118.stdout: link/ether 16:ef:1e:16:88:fb brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-24T06:10:55.044 INFO:teuthology.orchestra.run.smithi118.stdout: inet6 fe80::14ef:1eff:fe16:88fb/64 scope link 2023-11-24T06:10:55.045 INFO:teuthology.orchestra.run.smithi118.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:10:55.045 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:10:55.045 DEBUG:teuthology.orchestra.run.smithi118:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-24T06:10:55.124 INFO:teuthology.orchestra.run.smithi118.stdout:1 2023-11-24T06:10:55.128 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:10:55.128 DEBUG:teuthology.orchestra.run.smithi118:> route 2023-11-24T06:10:55.180 INFO:teuthology.orchestra.run.smithi118.stdout:Kernel IP routing table 2023-11-24T06:10:55.181 INFO:teuthology.orchestra.run.smithi118.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-24T06:10:55.181 INFO:teuthology.orchestra.run.smithi118.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-24T06:10:55.181 INFO:teuthology.orchestra.run.smithi118.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-24T06:10:55.181 INFO:teuthology.orchestra.run.smithi118.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-24T06:10:55.181 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:10:55.181 DEBUG:teuthology.orchestra.run.smithi118:> (cd / && exec bash -c ' 2023-11-24T06:10:55.181 DEBUG:teuthology.orchestra.run.smithi118:> set -e 2023-11-24T06:10:55.181 DEBUG:teuthology.orchestra.run.smithi118:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-24T06:10:55.181 DEBUG:teuthology.orchestra.run.smithi118:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-24T06:10:55.182 DEBUG:teuthology.orchestra.run.smithi118:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-24T06:10:55.182 DEBUG:teuthology.orchestra.run.smithi118:> ') 2023-11-24T06:10:55.363 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:10:55.364 DEBUG:teuthology.orchestra.run.smithi118:> ip netns list 2023-11-24T06:10:55.418 INFO:teuthology.orchestra.run.smithi118.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-11-24T06:10:55.418 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:10:55.418 DEBUG:teuthology.orchestra.run.smithi118:> ip netns list-id 2023-11-24T06:10:55.468 INFO:teuthology.orchestra.run.smithi118.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2023-11-24T06:10:55.469 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:10:55.469 DEBUG:teuthology.orchestra.run.smithi118:> (cd / && exec bash -c ' 2023-11-24T06:10:55.469 DEBUG:teuthology.orchestra.run.smithi118:> set -e 2023-11-24T06:10:55.469 DEBUG:teuthology.orchestra.run.smithi118:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-24T06:10:55.469 DEBUG:teuthology.orchestra.run.smithi118:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2023-11-24T06:10:55.469 DEBUG:teuthology.orchestra.run.smithi118:> ') 2023-11-24T06:10:55.609 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:10:55.609 DEBUG:teuthology.orchestra.run.smithi118:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-11-24T06:10:55.696 INFO:teuthology.orchestra.run.smithi118.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-24T06:10:55.696 INFO:teuthology.orchestra.run.smithi118.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-24T06:10:55.696 INFO:teuthology.orchestra.run.smithi118.stdout: inet 127.0.0.1/8 scope host lo 2023-11-24T06:10:55.696 INFO:teuthology.orchestra.run.smithi118.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:10:55.697 INFO:teuthology.orchestra.run.smithi118.stdout: inet6 ::1/128 scope host 2023-11-24T06:10:55.697 INFO:teuthology.orchestra.run.smithi118.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:10:55.697 INFO:teuthology.orchestra.run.smithi118.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-24T06:10:55.697 INFO:teuthology.orchestra.run.smithi118.stdout: link/ether 4e:04:67:16:ef:a3 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-11-24T06:10:55.697 INFO:teuthology.orchestra.run.smithi118.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-11-24T06:10:55.697 INFO:teuthology.orchestra.run.smithi118.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:10:55.697 INFO:teuthology.orchestra.run.smithi118.stdout: inet6 fe80::4c04:67ff:fe16:efa3/64 scope link 2023-11-24T06:10:55.697 INFO:teuthology.orchestra.run.smithi118.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:10:55.700 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:10:55.700 DEBUG:teuthology.orchestra.run.smithi118:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-11-24T06:10:55.790 INFO:teuthology.orchestra.run.smithi118.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-24T06:10:55.791 INFO:teuthology.orchestra.run.smithi118.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-24T06:10:55.791 INFO:teuthology.orchestra.run.smithi118.stdout: inet 127.0.0.1/8 scope host lo 2023-11-24T06:10:55.791 INFO:teuthology.orchestra.run.smithi118.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:10:55.791 INFO:teuthology.orchestra.run.smithi118.stdout: inet6 ::1/128 scope host 2023-11-24T06:10:55.791 INFO:teuthology.orchestra.run.smithi118.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:10:55.791 INFO:teuthology.orchestra.run.smithi118.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-24T06:10:55.791 INFO:teuthology.orchestra.run.smithi118.stdout: link/ether 4e:04:67:16:ef:a3 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-11-24T06:10:55.791 INFO:teuthology.orchestra.run.smithi118.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-11-24T06:10:55.791 INFO:teuthology.orchestra.run.smithi118.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:10:55.791 INFO:teuthology.orchestra.run.smithi118.stdout: inet6 fe80::4c04:67ff:fe16:efa3/64 scope link 2023-11-24T06:10:55.792 INFO:teuthology.orchestra.run.smithi118.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:10:55.794 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2023-11-24T06:10:55.795 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:10:55.795 DEBUG:teuthology.orchestra.run.smithi118:> (cd / && exec bash -c ' 2023-11-24T06:10:55.795 DEBUG:teuthology.orchestra.run.smithi118:> set -e 2023-11-24T06:10:55.795 DEBUG:teuthology.orchestra.run.smithi118:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2023-11-24T06:10:55.795 DEBUG:teuthology.orchestra.run.smithi118:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-11-24T06:10:55.795 DEBUG:teuthology.orchestra.run.smithi118:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2023-11-24T06:10:55.795 DEBUG:teuthology.orchestra.run.smithi118:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2023-11-24T06:10:55.795 DEBUG:teuthology.orchestra.run.smithi118:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2023-11-24T06:10:55.795 DEBUG:teuthology.orchestra.run.smithi118:> ') 2023-11-24T06:10:56.060 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:10:56.060 DEBUG:teuthology.orchestra.run.smithi118:> (cd / && exec bash -c ' 2023-11-24T06:10:56.060 DEBUG:teuthology.orchestra.run.smithi118:> set -e 2023-11-24T06:10:56.060 DEBUG:teuthology.orchestra.run.smithi118:> sudo ip link set brx.1 up 2023-11-24T06:10:56.060 DEBUG:teuthology.orchestra.run.smithi118:> sudo ip link set dev brx.1 master ceph-brx 2023-11-24T06:10:56.061 DEBUG:teuthology.orchestra.run.smithi118:> ') 2023-11-24T06:10:56.226 INFO:tasks.cephfs.fuse_mount:Client client.3 config is {} 2023-11-24T06:10:56.226 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T06:10:56.226 DEBUG:teuthology.orchestra.run.smithi118:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2023-11-24T06:10:56.276 INFO:teuthology.orchestra.run.smithi118.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2023-11-24T06:10:56.276 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T06:10:56.276 DEBUG:teuthology.orchestra.run.smithi118:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2023-11-24T06:10:56.324 DEBUG:teuthology.orchestra.run.smithi118:> sudo modprobe fuse 2023-11-24T06:10:56.416 DEBUG:teuthology.orchestra.run.smithi118:> cat /proc/self/mounts | awk '{print $2}' 2023-11-24T06:10:56.473 INFO:teuthology.orchestra.run.smithi118.stdout:/proc 2023-11-24T06:10:56.473 INFO:teuthology.orchestra.run.smithi118.stdout:/sys 2023-11-24T06:10:56.473 INFO:teuthology.orchestra.run.smithi118.stdout:/dev 2023-11-24T06:10:56.473 INFO:teuthology.orchestra.run.smithi118.stdout:/sys/kernel/security 2023-11-24T06:10:56.473 INFO:teuthology.orchestra.run.smithi118.stdout:/dev/shm 2023-11-24T06:10:56.473 INFO:teuthology.orchestra.run.smithi118.stdout:/dev/pts 2023-11-24T06:10:56.473 INFO:teuthology.orchestra.run.smithi118.stdout:/run 2023-11-24T06:10:56.473 INFO:teuthology.orchestra.run.smithi118.stdout:/sys/fs/cgroup 2023-11-24T06:10:56.473 INFO:teuthology.orchestra.run.smithi118.stdout:/sys/fs/pstore 2023-11-24T06:10:56.473 INFO:teuthology.orchestra.run.smithi118.stdout:/ 2023-11-24T06:10:56.474 INFO:teuthology.orchestra.run.smithi118.stdout:/sys/fs/selinux 2023-11-24T06:10:56.474 INFO:teuthology.orchestra.run.smithi118.stdout:/proc/sys/fs/binfmt_misc 2023-11-24T06:10:56.474 INFO:teuthology.orchestra.run.smithi118.stdout:/dev/mqueue 2023-11-24T06:10:56.474 INFO:teuthology.orchestra.run.smithi118.stdout:/dev/hugepages 2023-11-24T06:10:56.474 INFO:teuthology.orchestra.run.smithi118.stdout:/sys/kernel/debug 2023-11-24T06:10:56.474 INFO:teuthology.orchestra.run.smithi118.stdout:/sys/kernel/tracing 2023-11-24T06:10:56.474 INFO:teuthology.orchestra.run.smithi118.stdout:/sys/fs/fuse/connections 2023-11-24T06:10:56.474 INFO:teuthology.orchestra.run.smithi118.stdout:/sys/kernel/config 2023-11-24T06:10:56.474 INFO:teuthology.orchestra.run.smithi118.stdout:/run/credentials/systemd-sysctl.service 2023-11-24T06:10:56.474 INFO:teuthology.orchestra.run.smithi118.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-24T06:10:56.474 INFO:teuthology.orchestra.run.smithi118.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-24T06:10:56.474 INFO:teuthology.orchestra.run.smithi118.stdout:/var/lib/nfs/rpc_pipefs 2023-11-24T06:10:56.475 INFO:teuthology.orchestra.run.smithi118.stdout:/run/user/1000 2023-11-24T06:10:56.475 INFO:teuthology.orchestra.run.smithi118.stdout:/var/lib/ceph 2023-11-24T06:10:56.475 INFO:teuthology.orchestra.run.smithi118.stdout:/var/lib/ceph/osd/ceph-4 2023-11-24T06:10:56.475 INFO:teuthology.orchestra.run.smithi118.stdout:/var/lib/ceph/osd/ceph-5 2023-11-24T06:10:56.475 INFO:teuthology.orchestra.run.smithi118.stdout:/var/lib/ceph/osd/ceph-6 2023-11-24T06:10:56.475 INFO:teuthology.orchestra.run.smithi118.stdout:/var/lib/ceph/osd/ceph-7 2023-11-24T06:10:56.475 INFO:teuthology.orchestra.run.smithi118.stdout:/run/netns 2023-11-24T06:10:56.475 INFO:teuthology.orchestra.run.smithi118.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-24T06:10:56.475 INFO:teuthology.orchestra.run.smithi118.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-24T06:10:56.475 INFO:teuthology.orchestra.run.smithi118.stdout:/home/ubuntu/cephtest/mnt.1 2023-11-24T06:10:56.475 INFO:teuthology.orchestra.run.smithi118.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-24T06:10:56.475 INFO:teuthology.orchestra.run.smithi118.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-24T06:10:56.476 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:10:56.476 DEBUG:teuthology.orchestra.run.smithi118:> ls /sys/fs/fuse/connections 2023-11-24T06:10:56.528 INFO:teuthology.orchestra.run.smithi118.stdout:41 2023-11-24T06:10:56.528 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [41] 2023-11-24T06:10:56.529 DEBUG:teuthology.orchestra.run.smithi118:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.3 --id 3) 2023-11-24T06:10:56.534 DEBUG:teuthology.orchestra.run.smithi118:> sudo modprobe fuse 2023-11-24T06:10:56.611 DEBUG:teuthology.orchestra.run.smithi118:> cat /proc/self/mounts | awk '{print $2}' 2023-11-24T06:10:56.655 INFO:teuthology.orchestra.run.smithi118.stdout:/proc 2023-11-24T06:10:56.656 INFO:teuthology.orchestra.run.smithi118.stdout:/sys 2023-11-24T06:10:56.656 INFO:teuthology.orchestra.run.smithi118.stdout:/dev 2023-11-24T06:10:56.656 INFO:teuthology.orchestra.run.smithi118.stdout:/sys/kernel/security 2023-11-24T06:10:56.656 INFO:teuthology.orchestra.run.smithi118.stdout:/dev/shm 2023-11-24T06:10:56.656 INFO:teuthology.orchestra.run.smithi118.stdout:/dev/pts 2023-11-24T06:10:56.656 INFO:teuthology.orchestra.run.smithi118.stdout:/run 2023-11-24T06:10:56.656 INFO:teuthology.orchestra.run.smithi118.stdout:/sys/fs/cgroup 2023-11-24T06:10:56.656 INFO:teuthology.orchestra.run.smithi118.stdout:/sys/fs/pstore 2023-11-24T06:10:56.656 INFO:teuthology.orchestra.run.smithi118.stdout:/ 2023-11-24T06:10:56.656 INFO:teuthology.orchestra.run.smithi118.stdout:/sys/fs/selinux 2023-11-24T06:10:56.656 INFO:teuthology.orchestra.run.smithi118.stdout:/proc/sys/fs/binfmt_misc 2023-11-24T06:10:56.657 INFO:teuthology.orchestra.run.smithi118.stdout:/dev/mqueue 2023-11-24T06:10:56.657 INFO:teuthology.orchestra.run.smithi118.stdout:/dev/hugepages 2023-11-24T06:10:56.657 INFO:teuthology.orchestra.run.smithi118.stdout:/sys/kernel/debug 2023-11-24T06:10:56.657 INFO:teuthology.orchestra.run.smithi118.stdout:/sys/kernel/tracing 2023-11-24T06:10:56.657 INFO:teuthology.orchestra.run.smithi118.stdout:/sys/fs/fuse/connections 2023-11-24T06:10:56.657 INFO:teuthology.orchestra.run.smithi118.stdout:/sys/kernel/config 2023-11-24T06:10:56.657 INFO:teuthology.orchestra.run.smithi118.stdout:/run/credentials/systemd-sysctl.service 2023-11-24T06:10:56.657 INFO:teuthology.orchestra.run.smithi118.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-24T06:10:56.657 INFO:teuthology.orchestra.run.smithi118.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-24T06:10:56.657 INFO:teuthology.orchestra.run.smithi118.stdout:/var/lib/nfs/rpc_pipefs 2023-11-24T06:10:56.657 INFO:teuthology.orchestra.run.smithi118.stdout:/run/user/1000 2023-11-24T06:10:56.658 INFO:teuthology.orchestra.run.smithi118.stdout:/var/lib/ceph 2023-11-24T06:10:56.658 INFO:teuthology.orchestra.run.smithi118.stdout:/var/lib/ceph/osd/ceph-4 2023-11-24T06:10:56.658 INFO:teuthology.orchestra.run.smithi118.stdout:/var/lib/ceph/osd/ceph-5 2023-11-24T06:10:56.658 INFO:teuthology.orchestra.run.smithi118.stdout:/var/lib/ceph/osd/ceph-6 2023-11-24T06:10:56.658 INFO:teuthology.orchestra.run.smithi118.stdout:/var/lib/ceph/osd/ceph-7 2023-11-24T06:10:56.658 INFO:teuthology.orchestra.run.smithi118.stdout:/run/netns 2023-11-24T06:10:56.658 INFO:teuthology.orchestra.run.smithi118.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-24T06:10:56.658 INFO:teuthology.orchestra.run.smithi118.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-24T06:10:56.658 INFO:teuthology.orchestra.run.smithi118.stdout:/home/ubuntu/cephtest/mnt.1 2023-11-24T06:10:56.658 INFO:teuthology.orchestra.run.smithi118.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-24T06:10:56.658 INFO:teuthology.orchestra.run.smithi118.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-24T06:10:56.659 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:10:56.660 DEBUG:teuthology.orchestra.run.smithi118:> ls /sys/fs/fuse/connections 2023-11-24T06:10:56.723 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi118.stderr:2023-11-24T06:10:56.720+0000 7f9058e1f480 -1 init, newargv = 0x55687a88f8b0 newargc=15 2023-11-24T06:10:56.723 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi118.stderr:ceph-fuse[93594]: starting ceph client 2023-11-24T06:10:56.732 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi118.stderr:ceph-fuse[93594]: starting fuse 2023-11-24T06:10:56.748 INFO:teuthology.orchestra.run.smithi118.stdout:41 2023-11-24T06:10:56.749 INFO:teuthology.orchestra.run.smithi118.stdout:42 2023-11-24T06:10:56.749 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41, 42] 2023-11-24T06:10:56.749 DEBUG:teuthology.orchestra.run.smithi118:> sudo stdin-killer -- python3 -c ' 2023-11-24T06:10:56.749 DEBUG:teuthology.orchestra.run.smithi118:> import glob 2023-11-24T06:10:56.749 DEBUG:teuthology.orchestra.run.smithi118:> import re 2023-11-24T06:10:56.749 DEBUG:teuthology.orchestra.run.smithi118:> import os 2023-11-24T06:10:56.749 DEBUG:teuthology.orchestra.run.smithi118:> import subprocess 2023-11-24T06:10:56.749 DEBUG:teuthology.orchestra.run.smithi118:> 2023-11-24T06:10:56.749 DEBUG:teuthology.orchestra.run.smithi118:> def _find_admin_socket(client_name): 2023-11-24T06:10:56.749 DEBUG:teuthology.orchestra.run.smithi118:> asok_path = "/var/run/ceph/ceph-client.3.*.asok" 2023-11-24T06:10:56.750 DEBUG:teuthology.orchestra.run.smithi118:> files = glob.glob(asok_path) 2023-11-24T06:10:56.750 DEBUG:teuthology.orchestra.run.smithi118:> mountpoint = "/home/ubuntu/cephtest/mnt.3" 2023-11-24T06:10:56.750 DEBUG:teuthology.orchestra.run.smithi118:> 2023-11-24T06:10:56.750 DEBUG:teuthology.orchestra.run.smithi118:> # Given a non-glob path, it better be there 2023-11-24T06:10:56.750 DEBUG:teuthology.orchestra.run.smithi118:> if "*" not in asok_path: 2023-11-24T06:10:56.750 DEBUG:teuthology.orchestra.run.smithi118:> assert(len(files) == 1) 2023-11-24T06:10:56.750 DEBUG:teuthology.orchestra.run.smithi118:> return files[0] 2023-11-24T06:10:56.750 DEBUG:teuthology.orchestra.run.smithi118:> 2023-11-24T06:10:56.750 DEBUG:teuthology.orchestra.run.smithi118:> for f in files: 2023-11-24T06:10:56.750 DEBUG:teuthology.orchestra.run.smithi118:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-11-24T06:10:56.750 DEBUG:teuthology.orchestra.run.smithi118:> if os.path.exists("/proc/{0}".format(pid)): 2023-11-24T06:10:56.750 DEBUG:teuthology.orchestra.run.smithi118:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-11-24T06:10:56.751 DEBUG:teuthology.orchestra.run.smithi118:> contents = proc_f.read() 2023-11-24T06:10:56.751 DEBUG:teuthology.orchestra.run.smithi118:> if mountpoint in contents: 2023-11-24T06:10:56.751 DEBUG:teuthology.orchestra.run.smithi118:> return f 2023-11-24T06:10:56.751 DEBUG:teuthology.orchestra.run.smithi118:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-11-24T06:10:56.751 DEBUG:teuthology.orchestra.run.smithi118:> 2023-11-24T06:10:56.751 DEBUG:teuthology.orchestra.run.smithi118:> print(_find_admin_socket("client.3")) 2023-11-24T06:10:56.751 DEBUG:teuthology.orchestra.run.smithi118:> ' 2023-11-24T06:10:56.953 INFO:teuthology.orchestra.run.smithi118.stdout:/var/run/ceph/ceph-client.3.93594.asok 2023-11-24T06:10:56.957 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:10:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-24T06:10:56.970 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.3.93594.asok 2023-11-24T06:10:56.970 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:10:56.970 DEBUG:teuthology.orchestra.run.smithi118:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.3.93594.asok status 2023-11-24T06:10:57.210 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2023-11-24T06:10:57.210 INFO:teuthology.orchestra.run.smithi118.stdout: "metadata": { 2023-11-24T06:10:57.210 INFO:teuthology.orchestra.run.smithi118.stdout: "ceph_sha1": "a13ea5ff00cb4df7a9cc941166ddeb358881144c", 2023-11-24T06:10:57.210 INFO:teuthology.orchestra.run.smithi118.stdout: "ceph_version": "ceph version 18.0.0-7484-ga13ea5ff (a13ea5ff00cb4df7a9cc941166ddeb358881144c) reef (dev)", 2023-11-24T06:10:57.210 INFO:teuthology.orchestra.run.smithi118.stdout: "entity_id": "3", 2023-11-24T06:10:57.210 INFO:teuthology.orchestra.run.smithi118.stdout: "hostname": "smithi118", 2023-11-24T06:10:57.210 INFO:teuthology.orchestra.run.smithi118.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.3", 2023-11-24T06:10:57.210 INFO:teuthology.orchestra.run.smithi118.stdout: "pid": "93594", 2023-11-24T06:10:57.210 INFO:teuthology.orchestra.run.smithi118.stdout: "root": "/" 2023-11-24T06:10:57.211 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-11-24T06:10:57.211 INFO:teuthology.orchestra.run.smithi118.stdout: "dentry_count": 0, 2023-11-24T06:10:57.211 INFO:teuthology.orchestra.run.smithi118.stdout: "dentry_pinned_count": 0, 2023-11-24T06:10:57.211 INFO:teuthology.orchestra.run.smithi118.stdout: "id": 4469, 2023-11-24T06:10:57.211 INFO:teuthology.orchestra.run.smithi118.stdout: "inst": { 2023-11-24T06:10:57.211 INFO:teuthology.orchestra.run.smithi118.stdout: "name": { 2023-11-24T06:10:57.211 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "client", 2023-11-24T06:10:57.211 INFO:teuthology.orchestra.run.smithi118.stdout: "num": 4469 2023-11-24T06:10:57.211 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-11-24T06:10:57.211 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": { 2023-11-24T06:10:57.211 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v1", 2023-11-24T06:10:57.211 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "192.168.0.2:0", 2023-11-24T06:10:57.212 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 2345603605 2023-11-24T06:10:57.212 INFO:teuthology.orchestra.run.smithi118.stdout: } 2023-11-24T06:10:57.212 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-11-24T06:10:57.212 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": { 2023-11-24T06:10:57.212 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v1", 2023-11-24T06:10:57.212 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "192.168.0.2:0", 2023-11-24T06:10:57.212 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 2345603605 2023-11-24T06:10:57.212 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-11-24T06:10:57.212 INFO:teuthology.orchestra.run.smithi118.stdout: "inst_str": "client.4469 192.168.0.2:0/2345603605", 2023-11-24T06:10:57.212 INFO:teuthology.orchestra.run.smithi118.stdout: "addr_str": "192.168.0.2:0/2345603605", 2023-11-24T06:10:57.212 INFO:teuthology.orchestra.run.smithi118.stdout: "inode_count": 1, 2023-11-24T06:10:57.212 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_epoch": 8, 2023-11-24T06:10:57.213 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_epoch": 20, 2023-11-24T06:10:57.213 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_epoch_barrier": 0, 2023-11-24T06:10:57.213 INFO:teuthology.orchestra.run.smithi118.stdout: "blocklisted": false, 2023-11-24T06:10:57.213 INFO:teuthology.orchestra.run.smithi118.stdout: "fs_name": "cephfs" 2023-11-24T06:10:57.213 INFO:teuthology.orchestra.run.smithi118.stdout:} 2023-11-24T06:10:57.224 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:10:57.224 DEBUG:teuthology.orchestra.run.smithi092:> stat --file-system '--printf=%T 2023-11-24T06:10:57.224 DEBUG:teuthology.orchestra.run.smithi092:> ' -- /home/ubuntu/cephtest/mnt.0 2023-11-24T06:10:57.286 INFO:teuthology.orchestra.run.smithi092.stdout:fuseblk 2023-11-24T06:10:57.286 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-11-24T06:10:57.286 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:10:57.286 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-24T06:10:57.386 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:10:57.386 DEBUG:teuthology.orchestra.run.smithi092:> stat --file-system '--printf=%T 2023-11-24T06:10:57.387 DEBUG:teuthology.orchestra.run.smithi092:> ' -- /home/ubuntu/cephtest/mnt.2 2023-11-24T06:10:57.441 INFO:teuthology.orchestra.run.smithi092.stdout:fuseblk 2023-11-24T06:10:57.442 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.2 2023-11-24T06:10:57.442 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:10:57.442 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2023-11-24T06:10:57.540 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:10:57.540 DEBUG:teuthology.orchestra.run.smithi118:> stat --file-system '--printf=%T 2023-11-24T06:10:57.540 DEBUG:teuthology.orchestra.run.smithi118:> ' -- /home/ubuntu/cephtest/mnt.1 2023-11-24T06:10:57.602 INFO:teuthology.orchestra.run.smithi118.stdout:fuseblk 2023-11-24T06:10:57.602 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2023-11-24T06:10:57.603 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:10:57.603 DEBUG:teuthology.orchestra.run.smithi118:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-11-24T06:10:57.697 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:10:57.697 DEBUG:teuthology.orchestra.run.smithi118:> stat --file-system '--printf=%T 2023-11-24T06:10:57.697 DEBUG:teuthology.orchestra.run.smithi118:> ' -- /home/ubuntu/cephtest/mnt.3 2023-11-24T06:10:57.754 INFO:teuthology.orchestra.run.smithi118.stdout:fuseblk 2023-11-24T06:10:57.754 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.3 2023-11-24T06:10:57.754 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:10:57.754 DEBUG:teuthology.orchestra.run.smithi118:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2023-11-24T06:10:57.842 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2023-11-24T06:10:57.857 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:10:58.082 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.079+0000 7f7a88733640 1 Processor -- start 2023-11-24T06:10:58.083 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.079+0000 7f7a88733640 1 -- start start 2023-11-24T06:10:58.084 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.081+0000 7f7a88733640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f7a800ae5a0 0x7f7a8015e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:58.084 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.081+0000 7f7a88733640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7a800cf290 0x7f7a800cf720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:58.084 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.081+0000 7f7a88733640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a80070c40 con 0x7f7a800b3ae0 2023-11-24T06:10:58.084 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.081+0000 7f7a88733640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a80071450 con 0x7f7a800ae5a0 2023-11-24T06:10:58.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.081+0000 7f7a88733640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a800cfc60 con 0x7f7a800cf290 2023-11-24T06:10:58.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.081+0000 7f7a864cf640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f7a800ae5a0 0x7f7a8015e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:58.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.081+0000 7f7a85cce640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f7a800b3ae0 0x7f7a8006cbe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:39904/0 (socket says 172.21.15.92:39904) 2023-11-24T06:10:58.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.081+0000 7f7a864cf640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f7a800ae5a0 0x7f7a8015e430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:36654/0 (socket says 172.21.15.92:36654) 2023-11-24T06:10:58.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.081+0000 7f7a85cce640 1 -- 172.21.15.92:0/2293028984 learned_addr learned my addr 172.21.15.92:0/2293028984 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-24T06:10:58.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.081+0000 7f7a86cd0640 1 --2- 172.21.15.92:0/2293028984 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7a800cf290 0x7f7a800cf720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:58.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.082+0000 7f7a864cf640 1 -- 172.21.15.92:0/2293028984 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7a800cf290 msgr2=0x7f7a800cf720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:58.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.082+0000 7f7a864cf640 1 --2- 172.21.15.92:0/2293028984 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7a800cf290 0x7f7a800cf720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:58.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.082+0000 7f7a864cf640 1 -- 172.21.15.92:0/2293028984 >> v1:172.21.15.92:6789/0 conn(0x7f7a800b3ae0 legacy=0x7f7a8006cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:58.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.082+0000 7f7a864cf640 1 -- 172.21.15.92:0/2293028984 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a8006c470 con 0x7f7a800ae5a0 2023-11-24T06:10:58.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.082+0000 7f7a854cd640 1 -- 172.21.15.92:0/2293028984 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1768053785 0 0) 0x7f7a80070c40 con 0x7f7a800b3ae0 2023-11-24T06:10:58.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.083+0000 7f7a864cf640 1 --2- 172.21.15.92:0/2293028984 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7a800ae5a0 0x7f7a8015e430 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f7a7c0097a0 tx=0x7f7a7c02b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=563affd39df88caf server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:58.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.083+0000 7f7a854cd640 1 -- 172.21.15.92:0/2293028984 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a7c030610 con 0x7f7a800ae5a0 2023-11-24T06:10:58.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.083+0000 7f7a854cd640 1 -- 172.21.15.92:0/2293028984 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a7c030770 con 0x7f7a800ae5a0 2023-11-24T06:10:58.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.083+0000 7f7a854cd640 1 -- 172.21.15.92:0/2293028984 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a7c030b00 con 0x7f7a800ae5a0 2023-11-24T06:10:58.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.084+0000 7f7a88733640 1 -- 172.21.15.92:0/2293028984 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7a800ae5a0 msgr2=0x7f7a8015e430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:58.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.084+0000 7f7a88733640 1 --2- 172.21.15.92:0/2293028984 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7a800ae5a0 0x7f7a8015e430 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f7a7c0097a0 tx=0x7f7a7c02b960 comp rx=0 tx=0).stop 2023-11-24T06:10:58.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.084+0000 7f7a88733640 1 -- 172.21.15.92:0/2293028984 shutdown_connections 2023-11-24T06:10:58.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.084+0000 7f7a88733640 1 --2- 172.21.15.92:0/2293028984 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7a800cf290 0x7f7a800cf720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:58.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.084+0000 7f7a88733640 1 --2- 172.21.15.92:0/2293028984 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7a800ae5a0 0x7f7a8015e430 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:58.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.084+0000 7f7a88733640 1 -- 172.21.15.92:0/2293028984 >> 172.21.15.92:0/2293028984 conn(0x7f7a8006a800 msgr2=0x7f7a8006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:58.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.084+0000 7f7a88733640 1 -- 172.21.15.92:0/2293028984 shutdown_connections 2023-11-24T06:10:58.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.084+0000 7f7a88733640 1 -- 172.21.15.92:0/2293028984 wait complete. 2023-11-24T06:10:58.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.085+0000 7f7a88733640 1 Processor -- start 2023-11-24T06:10:58.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.086+0000 7f7a88733640 1 -- start start 2023-11-24T06:10:58.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.086+0000 7f7a88733640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7a800ae5a0 0x7f7a80156c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:58.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.086+0000 7f7a88733640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7a800b3ae0 0x7f7a801450b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:58.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.086+0000 7f7a88733640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7a800cf290 0x7f7a800cc8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:58.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.086+0000 7f7a88733640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a801455f0 con 0x7f7a800ae5a0 2023-11-24T06:10:58.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.086+0000 7f7a88733640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a8015f9b0 con 0x7f7a800cf290 2023-11-24T06:10:58.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.086+0000 7f7a88733640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a80148880 con 0x7f7a800b3ae0 2023-11-24T06:10:58.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.087+0000 7f7a864cf640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7a800ae5a0 0x7f7a80156c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:58.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.087+0000 7f7a864cf640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7a800ae5a0 0x7f7a80156c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:36664/0 (socket says 172.21.15.92:36664) 2023-11-24T06:10:58.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.087+0000 7f7a864cf640 1 -- 172.21.15.92:0/3549730040 learned_addr learned my addr 172.21.15.92:0/3549730040 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:10:58.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.087+0000 7f7a864cf640 1 -- 172.21.15.92:0/3549730040 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7a800b3ae0 msgr2=0x7f7a801450b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:58.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.087+0000 7f7a864cf640 1 --2- 172.21.15.92:0/3549730040 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7a800b3ae0 0x7f7a801450b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:58.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.087+0000 7f7a864cf640 1 -- 172.21.15.92:0/3549730040 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7a800cf290 msgr2=0x7f7a800cc8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:58.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.087+0000 7f7a864cf640 1 --2- 172.21.15.92:0/3549730040 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7a800cf290 0x7f7a800cc8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:58.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.087+0000 7f7a864cf640 1 -- 172.21.15.92:0/3549730040 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a7c009400 con 0x7f7a800ae5a0 2023-11-24T06:10:58.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.088+0000 7f7a864cf640 1 --2- 172.21.15.92:0/3549730040 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7a800ae5a0 0x7f7a80156c60 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f7a7c02bf70 tx=0x7f7a7c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:58.093 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.088+0000 7f7a76ffd640 1 -- 172.21.15.92:0/3549730040 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a7c03e040 con 0x7f7a800ae5a0 2023-11-24T06:10:58.093 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.088+0000 7f7a76ffd640 1 -- 172.21.15.92:0/3549730040 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a7c039cb0 con 0x7f7a800ae5a0 2023-11-24T06:10:58.094 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.088+0000 7f7a76ffd640 1 -- 172.21.15.92:0/3549730040 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a7c044630 con 0x7f7a800ae5a0 2023-11-24T06:10:58.094 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.088+0000 7f7a88733640 1 -- 172.21.15.92:0/3549730040 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a801485a0 con 0x7f7a800ae5a0 2023-11-24T06:10:58.094 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.088+0000 7f7a88733640 1 -- 172.21.15.92:0/3549730040 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a8015f640 con 0x7f7a800ae5a0 2023-11-24T06:10:58.094 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.089+0000 7f7a88733640 1 -- 172.21.15.92:0/3549730040 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a800675d0 con 0x7f7a800ae5a0 2023-11-24T06:10:58.094 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.090+0000 7f7a76ffd640 1 -- 172.21.15.92:0/3549730040 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f7a7c043390 con 0x7f7a800ae5a0 2023-11-24T06:10:58.095 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.091+0000 7f7a76ffd640 1 --2- 172.21.15.92:0/3549730040 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f7a6406ae40 0x7f7a6406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:58.095 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.091+0000 7f7a76ffd640 1 -- 172.21.15.92:0/3549730040 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7a7c0ab8a0 con 0x7f7a800ae5a0 2023-11-24T06:10:58.095 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.091+0000 7f7a85cce640 1 --2- 172.21.15.92:0/3549730040 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f7a6406ae40 0x7f7a6406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:58.095 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.092+0000 7f7a85cce640 1 --2- 172.21.15.92:0/3549730040 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f7a6406ae40 0x7f7a6406d300 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f7a70007fd0 tx=0x7f7a70007920 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:58.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.095+0000 7f7a76ffd640 1 -- 172.21.15.92:0/3549730040 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7a7c0768b0 con 0x7f7a800ae5a0 2023-11-24T06:10:58.290 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.287+0000 7f7a88733640 1 -- 172.21.15.92:0/3549730040 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7a800b34d0 con 0x7f7a800ae5a0 2023-11-24T06:10:58.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.290+0000 7f7a76ffd640 1 -- 172.21.15.92:0/3549730040 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16457 (secure 0 0 0) 0x7f7a7c006e10 con 0x7f7a800ae5a0 2023-11-24T06:10:58.294 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-24T06:10:58.294 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":20,"fsid":"2d6dee5f-8aa3-470f-931a-50ade963b1cb","created":"2023-11-24T06:10:04.707204+0000","modified":"2023-11-24T06:10:26.698733+0000","last_up_change":"2023-11-24T06:10:17.598352+0000","last_in_change":"2023-11-24T06:10:09.040793+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:10:17.733098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:10:22.926445+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T06:10:24.151288+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ff37a9fb-135f-471f-b4b0-f7b5772006c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6803","nonce":20560714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6805","nonce":20560714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6809","nonce":20560714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6807","nonce":20560714}]},"public_addr":"172.21.15.92:6803/20560714","cluster_addr":"172.21.15.92:6805/20560714","heartbeat_back_addr":"172.21.15.92:6809/20560714","heartbeat_front_addr":"172.21.15.92:6807/20560714","state":["exists","up"]},{"osd":1,"uuid":"03cdd353-dcf1-4f38-94fe-a795137c06b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6819","nonce":2451239906}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6821","nonce":2451239906}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6825","nonce":2451239906}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6823","nonce":2451239906}]},"public_addr":"172.21.15.92:6819/2451239906","cluster_addr":"172.21.15.92:6821/2451239906","heartbeat_back_addr":"172.21.15.92:6825/2451239906","heartbeat_front_addr":"172.21.15.92:6823/2451239906","state":["exists","up"]},{"osd":2,"uuid":"a4a20d80-5d57-484b-becd-9aa8070dd3aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6811","nonce":1124979277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6813","nonce":1124979277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6817","nonce":1124979277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6815","nonce":1124979277}]},"public_addr":"172.21.15.92:6811/1124979277","cluster_addr":"172.21.15.92:6813/1124979277","heartbeat_back_addr":"172.21.15.92:6817/1124979277","heartbeat_front_addr":"172.21.15.92:6815/1124979277","state":["exists","up"]},{"osd":3,"uuid":"4648d7ce-9b29-4583-9425-4ba8fd2b1903","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6827","nonce":1734115782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6829","nonce":1734115782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6833","nonce":1734115782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6831","nonce":1734115782}]},"public_addr":"172.21.15.92:6827/1734115782","cluster_addr":"172.21.15.92:6829/1734115782","heartbeat_back_addr":"172.21.15.92:6833/1734115782","heartbeat_front_addr":"172.21.15.92:6831/1734115782","state":["exists","up"]},{"osd":4,"uuid":"3bc9d7eb-c6fa-4fcf-a3ec-ade604e05aac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6819","nonce":3425937229}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6822","nonce":3425937229}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6830","nonce":3425937229}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6826","nonce":3425937229}]},"public_addr":"172.21.15.118:6819/3425937229","cluster_addr":"172.21.15.118:6822/3425937229","heartbeat_back_addr":"172.21.15.118:6830/3425937229","heartbeat_front_addr":"172.21.15.118:6826/3425937229","state":["exists","up"]},{"osd":5,"uuid":"dce9ba6d-59ed-46d1-84fc-a84b13de10c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6801","nonce":827455984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6803","nonce":827455984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6807","nonce":827455984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6805","nonce":827455984}]},"public_addr":"172.21.15.118:6801/827455984","cluster_addr":"172.21.15.118:6803/827455984","heartbeat_back_addr":"172.21.15.118:6807/827455984","heartbeat_front_addr":"172.21.15.118:6805/827455984","state":["exists","up"]},{"osd":6,"uuid":"82aec504-ae58-4709-8d30-46f2b12081b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6809","nonce":2946986141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6811","nonce":2946986141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6815","nonce":2946986141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6813","nonce":2946986141}]},"public_addr":"172.21.15.118:6809/2946986141","cluster_addr":"172.21.15.118:6811/2946986141","heartbeat_back_addr":"172.21.15.118:6815/2946986141","heartbeat_front_addr":"172.21.15.118:6813/2946986141","state":["exists","up"]},{"osd":7,"uuid":"4ed4c624-c88f-4d35-bdb1-bf209b5dffb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6817","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6818","nonce":48122059}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6821","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6823","nonce":48122059}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6829","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6831","nonce":48122059}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6825","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6827","nonce":48122059}]},"public_addr":"172.21.15.118:6818/48122059","cluster_addr":"172.21.15.118:6823/48122059","heartbeat_back_addr":"172.21.15.118:6831/48122059","heartbeat_front_addr":"172.21.15.118:6827/48122059","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.536664+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.368324+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.271201+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.342971+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.621832+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.387110+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.632946+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.416032+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:10:58.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.293+0000 7f7a88733640 1 -- 172.21.15.92:0/3549730040 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f7a6406ae40 msgr2=0x7f7a6406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:58.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.293+0000 7f7a88733640 1 --2- 172.21.15.92:0/3549730040 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f7a6406ae40 0x7f7a6406d300 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f7a70007fd0 tx=0x7f7a70007920 comp rx=0 tx=0).stop 2023-11-24T06:10:58.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.293+0000 7f7a88733640 1 -- 172.21.15.92:0/3549730040 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7a800ae5a0 msgr2=0x7f7a80156c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:58.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.293+0000 7f7a88733640 1 --2- 172.21.15.92:0/3549730040 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7a800ae5a0 0x7f7a80156c60 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f7a7c02bf70 tx=0x7f7a7c004730 comp rx=0 tx=0).stop 2023-11-24T06:10:58.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.293+0000 7f7a88733640 1 -- 172.21.15.92:0/3549730040 shutdown_connections 2023-11-24T06:10:58.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.293+0000 7f7a88733640 1 --2- 172.21.15.92:0/3549730040 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f7a6406ae40 0x7f7a6406d300 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:58.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.293+0000 7f7a88733640 1 --2- 172.21.15.92:0/3549730040 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7a800cf290 0x7f7a800cc8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:58.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.293+0000 7f7a88733640 1 --2- 172.21.15.92:0/3549730040 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7a800b3ae0 0x7f7a801450b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:58.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.293+0000 7f7a88733640 1 --2- 172.21.15.92:0/3549730040 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7a800ae5a0 0x7f7a80156c60 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:58.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.293+0000 7f7a88733640 1 -- 172.21.15.92:0/3549730040 >> 172.21.15.92:0/3549730040 conn(0x7f7a8006a800 msgr2=0x7f7a8015d820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:58.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.293+0000 7f7a88733640 1 -- 172.21.15.92:0/3549730040 shutdown_connections 2023-11-24T06:10:58.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.293+0000 7f7a88733640 1 -- 172.21.15.92:0/3549730040 wait complete. 2023-11-24T06:10:58.314 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T06:10:17.733098+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T06:10:22.926445+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T06:10:24.151288+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T06:10:58.315 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T06:10:58.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.521+0000 7efd194da640 1 Processor -- start 2023-11-24T06:10:58.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.522+0000 7efd194da640 1 -- start start 2023-11-24T06:10:58.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.523+0000 7efd194da640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7efd140adf20 0x7efd14070a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:58.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.523+0000 7efd194da640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7efd1415bf10 0x7efd140cc440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:58.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.523+0000 7efd194da640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd1415f4e0 con 0x7efd140cc980 2023-11-24T06:10:58.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.523+0000 7efd194da640 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd1415f970 con 0x7efd1415bf10 2023-11-24T06:10:58.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.523+0000 7efd194da640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd140cce10 con 0x7efd140adf20 2023-11-24T06:10:58.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.524+0000 7efd18cd9640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7efd140cc980 0x7efd14143d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:39908/0 (socket says 172.21.15.92:39908) 2023-11-24T06:10:58.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.524+0000 7efd18cd9640 1 -- 172.21.15.92:0/1480364487 learned_addr learned my addr 172.21.15.92:0/1480364487 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-24T06:10:58.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.524+0000 7efd13fff640 1 --2- 172.21.15.92:0/1480364487 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7efd140adf20 0x7efd14070a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:58.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.524+0000 7efd137fe640 1 --2- 172.21.15.92:0/1480364487 >> v2:172.21.15.118:3300/0 conn(0x7efd1415bf10 0x7efd140cc440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:58.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.524+0000 7efd12ffd640 1 -- 172.21.15.92:0/1480364487 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1378913272 0 0) 0x7efd1415f4e0 con 0x7efd140cc980 2023-11-24T06:10:58.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.524+0000 7efd12ffd640 1 -- 172.21.15.92:0/1480364487 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efcfc003520 con 0x7efd140cc980 2023-11-24T06:10:58.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.524+0000 7efd12ffd640 1 -- 172.21.15.92:0/1480364487 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3394260632 0 0) 0x7efcfc003520 con 0x7efd140cc980 2023-11-24T06:10:58.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.524+0000 7efd12ffd640 1 -- 172.21.15.92:0/1480364487 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7efd140adf20 msgr2=0x7efd14070a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:58.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.524+0000 7efd12ffd640 1 --2- 172.21.15.92:0/1480364487 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7efd140adf20 0x7efd14070a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:58.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.525+0000 7efd12ffd640 1 -- 172.21.15.92:0/1480364487 >> v2:172.21.15.118:3300/0 conn(0x7efd1415bf10 msgr2=0x7efd140cc440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:58.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.525+0000 7efd12ffd640 1 --2- 172.21.15.92:0/1480364487 >> v2:172.21.15.118:3300/0 conn(0x7efd1415bf10 0x7efd140cc440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:58.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.525+0000 7efd12ffd640 1 -- 172.21.15.92:0/1480364487 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd14071130 con 0x7efd140cc980 2023-11-24T06:10:58.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.525+0000 7efd12ffd640 1 -- 172.21.15.92:0/1480364487 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2607661350 0 0) 0x7efd0c0019f0 con 0x7efd140cc980 2023-11-24T06:10:58.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.525+0000 7efd12ffd640 1 -- 172.21.15.92:0/1480364487 >> v1:172.21.15.92:6789/0 conn(0x7efd140cc980 legacy=0x7efd14143d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:58.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.525+0000 7efd12ffd640 1 --2- 172.21.15.92:0/1480364487 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7efcfc003970 0x7efcfc003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:58.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.525+0000 7efd12ffd640 1 --2- 172.21.15.92:0/1480364487 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7efd140adf20 0x7efcfc0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:58.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.525+0000 7efd12ffd640 1 --2- 172.21.15.92:0/1480364487 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efcfc004d00 0x7efcfc005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:58.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.525+0000 7efd12ffd640 1 -- 172.21.15.92:0/1480364487 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd1415f970 con 0x7efcfc004d00 2023-11-24T06:10:58.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.525+0000 7efd12ffd640 1 -- 172.21.15.92:0/1480364487 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd140cce10 con 0x7efcfc003970 2023-11-24T06:10:58.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.525+0000 7efd12ffd640 1 -- 172.21.15.92:0/1480364487 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcfc00ba60 con 0x7efd140adf20 2023-11-24T06:10:58.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.525+0000 7efd137fe640 1 --2- 172.21.15.92:0/1480364487 >> v2:172.21.15.118:3300/0 conn(0x7efd1415bf10 0x7efd140cc440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:10:58.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.525+0000 7efd18cd9640 1 --2- 172.21.15.92:0/1480364487 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efcfc004d00 0x7efcfc005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:58.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.525+0000 7efd18cd9640 1 -- 172.21.15.92:0/1480364487 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7efd140adf20 msgr2=0x7efcfc0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:58.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.525+0000 7efd18cd9640 1 --2- 172.21.15.92:0/1480364487 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7efd140adf20 0x7efcfc0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:58.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.525+0000 7efd18cd9640 1 -- 172.21.15.92:0/1480364487 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7efcfc003970 msgr2=0x7efcfc003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:58.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.525+0000 7efd18cd9640 1 --2- 172.21.15.92:0/1480364487 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7efcfc003970 0x7efcfc003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:58.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.525+0000 7efd18cd9640 1 -- 172.21.15.92:0/1480364487 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcfc003520 con 0x7efcfc004d00 2023-11-24T06:10:58.532 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.526+0000 7efd18cd9640 1 --2- 172.21.15.92:0/1480364487 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efcfc004d00 0x7efcfc005180 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7efd0c00a770 tx=0x7efcfc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1da956e9ba6d3db8 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:58.532 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.527+0000 7efd12ffd640 1 -- 172.21.15.92:0/1480364487 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd0c034630 con 0x7efcfc004d00 2023-11-24T06:10:58.532 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.527+0000 7efd12ffd640 1 -- 172.21.15.92:0/1480364487 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd0c02f070 con 0x7efcfc004d00 2023-11-24T06:10:58.532 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.527+0000 7efd12ffd640 1 -- 172.21.15.92:0/1480364487 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd0c0349f0 con 0x7efcfc004d00 2023-11-24T06:10:58.532 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.527+0000 7efd194da640 1 -- 172.21.15.92:0/1480364487 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efcfc004d00 msgr2=0x7efcfc005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:58.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.527+0000 7efd194da640 1 --2- 172.21.15.92:0/1480364487 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efcfc004d00 0x7efcfc005180 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7efd0c00a770 tx=0x7efcfc0057f0 comp rx=0 tx=0).stop 2023-11-24T06:10:58.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.527+0000 7efd18cd9640 1 -- 172.21.15.92:0/1480364487 reap_dead start 2023-11-24T06:10:58.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.527+0000 7efd194da640 1 -- 172.21.15.92:0/1480364487 shutdown_connections 2023-11-24T06:10:58.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.527+0000 7efd194da640 1 -- 172.21.15.92:0/1480364487 >> 172.21.15.92:0/1480364487 conn(0x7efd1406a7d0 msgr2=0x7efd1406abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:58.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.527+0000 7efd194da640 1 -- 172.21.15.92:0/1480364487 shutdown_connections 2023-11-24T06:10:58.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.527+0000 7efd194da640 1 -- 172.21.15.92:0/1480364487 wait complete. 2023-11-24T06:10:58.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.529+0000 7efd194da640 1 Processor -- start 2023-11-24T06:10:58.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.529+0000 7efd194da640 1 -- start start 2023-11-24T06:10:58.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.530+0000 7efd194da640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7efd141f0140 0x7efd140ab630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:58.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.530+0000 7efd194da640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7efd1415d5f0 0x7efd140abb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:58.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.530+0000 7efd194da640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efd14143bb0 0x7efd141f3d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:58.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.530+0000 7efd194da640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd1415de40 con 0x7efd14143bb0 2023-11-24T06:10:58.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.530+0000 7efd194da640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd140a96c0 con 0x7efd141f0140 2023-11-24T06:10:58.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.530+0000 7efd194da640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd1415db60 con 0x7efd1415d5f0 2023-11-24T06:10:58.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.531+0000 7efd18cd9640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efd14143bb0 0x7efd141f3d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:58.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.531+0000 7efd18cd9640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efd14143bb0 0x7efd141f3d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:36694/0 (socket says 172.21.15.92:36694) 2023-11-24T06:10:58.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.531+0000 7efd18cd9640 1 -- 172.21.15.92:0/3408583901 learned_addr learned my addr 172.21.15.92:0/3408583901 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:10:58.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.531+0000 7efd13fff640 1 --2- 172.21.15.92:0/3408583901 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7efd141f0140 0x7efd140ab630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:58.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.531+0000 7efd137fe640 1 --2- 172.21.15.92:0/3408583901 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7efd1415d5f0 0x7efd140abb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:58.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.531+0000 7efd18cd9640 1 -- 172.21.15.92:0/3408583901 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7efd1415d5f0 msgr2=0x7efd140abb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:58.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.531+0000 7efd18cd9640 1 --2- 172.21.15.92:0/3408583901 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7efd1415d5f0 0x7efd140abb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:58.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.531+0000 7efd18cd9640 1 -- 172.21.15.92:0/3408583901 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7efd141f0140 msgr2=0x7efd140ab630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:58.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.531+0000 7efd18cd9640 1 --2- 172.21.15.92:0/3408583901 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7efd141f0140 0x7efd140ab630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:58.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.531+0000 7efd18cd9640 1 -- 172.21.15.92:0/3408583901 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd140a9070 con 0x7efd14143bb0 2023-11-24T06:10:58.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.531+0000 7efd18cd9640 1 --2- 172.21.15.92:0/3408583901 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efd14143bb0 0x7efd141f3d70 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7efd0c00a890 tx=0x7efd1415f450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:58.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.532+0000 7efd10ff9640 1 -- 172.21.15.92:0/3408583901 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd0c034630 con 0x7efd14143bb0 2023-11-24T06:10:58.537 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.532+0000 7efd10ff9640 1 -- 172.21.15.92:0/3408583901 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd0c034790 con 0x7efd14143bb0 2023-11-24T06:10:58.537 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.532+0000 7efd10ff9640 1 -- 172.21.15.92:0/3408583901 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd0c034b20 con 0x7efd14143bb0 2023-11-24T06:10:58.537 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.532+0000 7efd194da640 1 -- 172.21.15.92:0/3408583901 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd140a9350 con 0x7efd14143bb0 2023-11-24T06:10:58.537 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.532+0000 7efd194da640 1 -- 172.21.15.92:0/3408583901 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd1413fe10 con 0x7efd14143bb0 2023-11-24T06:10:58.537 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.533+0000 7efd194da640 1 -- 172.21.15.92:0/3408583901 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd140a9510 con 0x7efd14143bb0 2023-11-24T06:10:58.537 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.534+0000 7efd10ff9640 1 -- 172.21.15.92:0/3408583901 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7efd0c031030 con 0x7efd14143bb0 2023-11-24T06:10:58.538 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.535+0000 7efd10ff9640 1 --2- 172.21.15.92:0/3408583901 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7efd0006ae70 0x7efd0006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:58.538 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.535+0000 7efd10ff9640 1 -- 172.21.15.92:0/3408583901 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7efd0c0ab9b0 con 0x7efd14143bb0 2023-11-24T06:10:58.538 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.535+0000 7efd13fff640 1 --2- 172.21.15.92:0/3408583901 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7efd0006ae70 0x7efd0006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:58.539 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.536+0000 7efd13fff640 1 --2- 172.21.15.92:0/3408583901 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7efd0006ae70 0x7efd0006d330 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7efd04004770 tx=0x7efd04008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:58.543 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.540+0000 7efd10ff9640 1 -- 172.21.15.92:0/3408583901 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7efd0c076af0 con 0x7efd14143bb0 2023-11-24T06:10:58.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.727+0000 7efd194da640 1 -- 172.21.15.92:0/3408583901 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7efd140a9510 con 0x7efd14143bb0 2023-11-24T06:10:58.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.728+0000 7efd10ff9640 1 -- 172.21.15.92:0/3408583901 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7efd0c07a110 con 0x7efd14143bb0 2023-11-24T06:10:58.731 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 1 2023-11-24T06:10:58.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.731+0000 7efd194da640 1 -- 172.21.15.92:0/3408583901 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7efd0006ae70 msgr2=0x7efd0006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:58.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.731+0000 7efd194da640 1 --2- 172.21.15.92:0/3408583901 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7efd0006ae70 0x7efd0006d330 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7efd04004770 tx=0x7efd04008040 comp rx=0 tx=0).stop 2023-11-24T06:10:58.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.731+0000 7efd194da640 1 -- 172.21.15.92:0/3408583901 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efd14143bb0 msgr2=0x7efd141f3d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:58.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.731+0000 7efd194da640 1 --2- 172.21.15.92:0/3408583901 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efd14143bb0 0x7efd141f3d70 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7efd0c00a890 tx=0x7efd1415f450 comp rx=0 tx=0).stop 2023-11-24T06:10:58.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.731+0000 7efd194da640 1 -- 172.21.15.92:0/3408583901 shutdown_connections 2023-11-24T06:10:58.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.731+0000 7efd194da640 1 --2- 172.21.15.92:0/3408583901 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7efd0006ae70 0x7efd0006d330 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:58.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.731+0000 7efd194da640 1 --2- 172.21.15.92:0/3408583901 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efd14143bb0 0x7efd141f3d70 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:58.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.731+0000 7efd194da640 1 --2- 172.21.15.92:0/3408583901 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7efd1415d5f0 0x7efd140abb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:58.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.731+0000 7efd194da640 1 --2- 172.21.15.92:0/3408583901 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7efd141f0140 0x7efd140ab630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:58.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.731+0000 7efd194da640 1 -- 172.21.15.92:0/3408583901 >> 172.21.15.92:0/3408583901 conn(0x7efd1406a7d0 msgr2=0x7efd1415b500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:58.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.731+0000 7efd194da640 1 -- 172.21.15.92:0/3408583901 shutdown_connections 2023-11-24T06:10:58.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.731+0000 7efd194da640 1 -- 172.21.15.92:0/3408583901 wait complete. 2023-11-24T06:10:58.754 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T06:10:58.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.971+0000 7fc737516640 1 Processor -- start 2023-11-24T06:10:58.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.971+0000 7fc737516640 1 -- start start 2023-11-24T06:10:58.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.973+0000 7fc737516640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc7300cf9a0 0x7fc730166030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:58.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.973+0000 7fc737516640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc73015c5f0 con 0x7fc73013df80 2023-11-24T06:10:58.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.973+0000 7fc737516640 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc73006c470 con 0x7fc73013e360 2023-11-24T06:10:58.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.973+0000 7fc737516640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc73006c750 con 0x7fc7300cf9a0 2023-11-24T06:10:58.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.973+0000 7fc7352b2640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fc73013df80 0x7fc73015bee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:35476/0 (socket says 172.21.15.92:35476) 2023-11-24T06:10:58.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.973+0000 7fc7352b2640 1 -- 172.21.15.92:0/2892058387 learned_addr learned my addr 172.21.15.92:0/2892058387 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-24T06:10:58.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.973+0000 7fc735ab3640 1 --2- 172.21.15.92:0/2892058387 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc7300cf9a0 0x7fc730166030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:58.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.973+0000 7fc727fff640 1 -- 172.21.15.92:0/2892058387 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1843626552 0 0) 0x7fc73015c5f0 con 0x7fc73013df80 2023-11-24T06:10:58.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.974+0000 7fc727fff640 1 -- 172.21.15.92:0/2892058387 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc71c003520 con 0x7fc73013df80 2023-11-24T06:10:58.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.974+0000 7fc727fff640 1 -- 172.21.15.92:0/2892058387 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 939692302 0 0) 0x7fc71c003520 con 0x7fc73013df80 2023-11-24T06:10:58.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.974+0000 7fc727fff640 1 -- 172.21.15.92:0/2892058387 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc7300cf9a0 msgr2=0x7fc730166030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:58.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.974+0000 7fc727fff640 1 --2- 172.21.15.92:0/2892058387 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc7300cf9a0 0x7fc730166030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:58.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.974+0000 7fc727fff640 1 -- 172.21.15.92:0/2892058387 >> v1:172.21.15.118:6789/0 conn(0x7fc73013e360 legacy=0x7fc7300cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:58.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.974+0000 7fc727fff640 1 -- 172.21.15.92:0/2892058387 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc73006ca30 con 0x7fc73013df80 2023-11-24T06:10:58.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.974+0000 7fc727fff640 1 -- 172.21.15.92:0/2892058387 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2607661350 0 0) 0x7fc72c001670 con 0x7fc73013df80 2023-11-24T06:10:58.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.974+0000 7fc727fff640 1 -- 172.21.15.92:0/2892058387 >> v1:172.21.15.92:6789/0 conn(0x7fc73013df80 legacy=0x7fc73015bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:58.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.974+0000 7fc727fff640 1 --2- 172.21.15.92:0/2892058387 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc7300cf9a0 0x7fc71c0039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:58.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.974+0000 7fc727fff640 1 --2- 172.21.15.92:0/2892058387 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc71c003f70 0x7fc71c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:58.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.974+0000 7fc727fff640 1 --2- 172.21.15.92:0/2892058387 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc71c004d00 0x7fc71c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:58.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.974+0000 7fc727fff640 1 -- 172.21.15.92:0/2892058387 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc73006c750 con 0x7fc71c004d00 2023-11-24T06:10:58.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.974+0000 7fc727fff640 1 -- 172.21.15.92:0/2892058387 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc71c00bac0 con 0x7fc71c003f70 2023-11-24T06:10:58.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.974+0000 7fc727fff640 1 -- 172.21.15.92:0/2892058387 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc71c00bc80 con 0x7fc7300cf9a0 2023-11-24T06:10:58.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.974+0000 7fc735ab3640 1 --2- 172.21.15.92:0/2892058387 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc71c004d00 0x7fc71c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:58.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.975+0000 7fc7352b2640 1 --2- 172.21.15.92:0/2892058387 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc7300cf9a0 0x7fc71c0039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:58.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.975+0000 7fc734ab1640 1 --2- 172.21.15.92:0/2892058387 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc71c003f70 0x7fc71c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:58.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.975+0000 7fc735ab3640 1 -- 172.21.15.92:0/2892058387 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc7300cf9a0 msgr2=0x7fc71c0039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:58.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.975+0000 7fc735ab3640 1 --2- 172.21.15.92:0/2892058387 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc7300cf9a0 0x7fc71c0039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:58.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.975+0000 7fc735ab3640 1 -- 172.21.15.92:0/2892058387 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc71c003f70 msgr2=0x7fc71c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:58.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.975+0000 7fc735ab3640 1 --2- 172.21.15.92:0/2892058387 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc71c003f70 0x7fc71c0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:58.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.975+0000 7fc735ab3640 1 -- 172.21.15.92:0/2892058387 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc71c003520 con 0x7fc71c004d00 2023-11-24T06:10:58.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.975+0000 7fc734ab1640 1 --2- 172.21.15.92:0/2892058387 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc71c003f70 0x7fc71c0047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:10:58.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.976+0000 7fc735ab3640 1 --2- 172.21.15.92:0/2892058387 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc71c004d00 0x7fc71c005180 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fc728009f90 tx=0x7fc71c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bfcc8d5b1d850b1 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:58.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.976+0000 7fc727fff640 1 -- 172.21.15.92:0/2892058387 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc72802cbd0 con 0x7fc71c004d00 2023-11-24T06:10:58.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.976+0000 7fc727fff640 1 -- 172.21.15.92:0/2892058387 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc72800f040 con 0x7fc71c004d00 2023-11-24T06:10:58.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.976+0000 7fc727fff640 1 -- 172.21.15.92:0/2892058387 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc728034440 con 0x7fc71c004d00 2023-11-24T06:10:58.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.976+0000 7fc737516640 1 -- 172.21.15.92:0/2892058387 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc71c004d00 msgr2=0x7fc71c005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:58.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.976+0000 7fc737516640 1 --2- 172.21.15.92:0/2892058387 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc71c004d00 0x7fc71c005180 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fc728009f90 tx=0x7fc71c0057f0 comp rx=0 tx=0).stop 2023-11-24T06:10:58.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.977+0000 7fc735ab3640 1 -- 172.21.15.92:0/2892058387 reap_dead start 2023-11-24T06:10:58.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.977+0000 7fc737516640 1 -- 172.21.15.92:0/2892058387 shutdown_connections 2023-11-24T06:10:58.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.977+0000 7fc737516640 1 -- 172.21.15.92:0/2892058387 >> 172.21.15.92:0/2892058387 conn(0x7fc73006a800 msgr2=0x7fc73006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:58.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.977+0000 7fc737516640 1 -- 172.21.15.92:0/2892058387 shutdown_connections 2023-11-24T06:10:58.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.977+0000 7fc737516640 1 -- 172.21.15.92:0/2892058387 wait complete. 2023-11-24T06:10:58.990 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.978+0000 7fc737516640 1 Processor -- start 2023-11-24T06:10:58.990 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.979+0000 7fc737516640 1 -- start start 2023-11-24T06:10:58.991 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.980+0000 7fc737516640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc71c004d00 0x7fc73015e070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:58.991 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.980+0000 7fc737516640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc73012b040 0x7fc73012b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:58.991 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.980+0000 7fc737516640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc730153890 0x7fc730153d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:58.991 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.980+0000 7fc737516640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc73012bd70 con 0x7fc71c004d00 2023-11-24T06:10:58.991 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.980+0000 7fc737516640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc73012c050 con 0x7fc730153890 2023-11-24T06:10:58.991 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.980+0000 7fc737516640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc730127e20 con 0x7fc73012b040 2023-11-24T06:10:58.991 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.980+0000 7fc7352b2640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc71c004d00 0x7fc73015e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:58.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.980+0000 7fc7352b2640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc71c004d00 0x7fc73015e070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53030/0 (socket says 172.21.15.92:53030) 2023-11-24T06:10:58.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.980+0000 7fc7352b2640 1 -- 172.21.15.92:0/2681604129 learned_addr learned my addr 172.21.15.92:0/2681604129 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:10:58.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.980+0000 7fc734ab1640 1 --2- 172.21.15.92:0/2681604129 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc73012b040 0x7fc73012b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:58.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.980+0000 7fc7352b2640 1 -- 172.21.15.92:0/2681604129 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc73012b040 msgr2=0x7fc73012b4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:58.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.980+0000 7fc7352b2640 1 --2- 172.21.15.92:0/2681604129 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc73012b040 0x7fc73012b4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:58.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.980+0000 7fc7352b2640 1 -- 172.21.15.92:0/2681604129 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc730153890 msgr2=0x7fc730153d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:58.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.980+0000 7fc7352b2640 1 --2- 172.21.15.92:0/2681604129 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc730153890 0x7fc730153d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:58.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.980+0000 7fc7352b2640 1 -- 172.21.15.92:0/2681604129 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc730128100 con 0x7fc71c004d00 2023-11-24T06:10:58.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.980+0000 7fc734ab1640 1 --2- 172.21.15.92:0/2681604129 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc73012b040 0x7fc73012b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:10:58.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.980+0000 7fc7352b2640 1 --2- 172.21.15.92:0/2681604129 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc71c004d00 0x7fc73015e070 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fc72c00e7e0 tx=0x7fc72c00eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:58.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.981+0000 7fc725ffb640 1 -- 172.21.15.92:0/2681604129 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc72c00c850 con 0x7fc71c004d00 2023-11-24T06:10:58.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.981+0000 7fc725ffb640 1 -- 172.21.15.92:0/2681604129 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc72c00c9b0 con 0x7fc71c004d00 2023-11-24T06:10:58.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.981+0000 7fc725ffb640 1 -- 172.21.15.92:0/2681604129 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc72c00cd60 con 0x7fc71c004d00 2023-11-24T06:10:58.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.981+0000 7fc737516640 1 -- 172.21.15.92:0/2681604129 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7301283e0 con 0x7fc71c004d00 2023-11-24T06:10:58.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.981+0000 7fc737516640 1 -- 172.21.15.92:0/2681604129 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc73012b9e0 con 0x7fc71c004d00 2023-11-24T06:10:58.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.982+0000 7fc737516640 1 -- 172.21.15.92:0/2681604129 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7301285d0 con 0x7fc71c004d00 2023-11-24T06:10:58.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.983+0000 7fc725ffb640 1 -- 172.21.15.92:0/2681604129 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc72c01d460 con 0x7fc71c004d00 2023-11-24T06:10:58.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.984+0000 7fc725ffb640 1 --2- 172.21.15.92:0/2681604129 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fc71406ae70 0x7fc71406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:58.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.984+0000 7fc725ffb640 1 -- 172.21.15.92:0/2681604129 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc72c089700 con 0x7fc71c004d00 2023-11-24T06:10:58.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.984+0000 7fc734ab1640 1 --2- 172.21.15.92:0/2681604129 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fc71406ae70 0x7fc71406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:58.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.985+0000 7fc734ab1640 1 --2- 172.21.15.92:0/2681604129 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fc71406ae70 0x7fc71406d330 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fc7300680c0 tx=0x7fc720009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:58.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:58.988+0000 7fc725ffb640 1 -- 172.21.15.92:0/2681604129 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc72c054680 con 0x7fc71c004d00 2023-11-24T06:10:59.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:59.180+0000 7fc737516640 1 -- 172.21.15.92:0/2681604129 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc73012beb0 con 0x7fc71c004d00 2023-11-24T06:10:59.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:59.181+0000 7fc725ffb640 1 -- 172.21.15.92:0/2681604129 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7fc73012beb0 con 0x7fc71c004d00 2023-11-24T06:10:59.185 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 32 2023-11-24T06:10:59.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:59.184+0000 7fc737516640 1 -- 172.21.15.92:0/2681604129 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fc71406ae70 msgr2=0x7fc71406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:59.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:59.184+0000 7fc737516640 1 --2- 172.21.15.92:0/2681604129 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fc71406ae70 0x7fc71406d330 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fc7300680c0 tx=0x7fc720009040 comp rx=0 tx=0).stop 2023-11-24T06:10:59.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:59.184+0000 7fc737516640 1 -- 172.21.15.92:0/2681604129 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc71c004d00 msgr2=0x7fc73015e070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:59.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:59.184+0000 7fc737516640 1 --2- 172.21.15.92:0/2681604129 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc71c004d00 0x7fc73015e070 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fc72c00e7e0 tx=0x7fc72c00eca0 comp rx=0 tx=0).stop 2023-11-24T06:10:59.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:59.184+0000 7fc737516640 1 -- 172.21.15.92:0/2681604129 shutdown_connections 2023-11-24T06:10:59.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:59.184+0000 7fc737516640 1 --2- 172.21.15.92:0/2681604129 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fc71406ae70 0x7fc71406d330 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:59.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:59.184+0000 7fc737516640 1 --2- 172.21.15.92:0/2681604129 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc730153890 0x7fc730153d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:59.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:59.184+0000 7fc737516640 1 --2- 172.21.15.92:0/2681604129 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc73012b040 0x7fc73012b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:59.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:59.184+0000 7fc737516640 1 --2- 172.21.15.92:0/2681604129 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc71c004d00 0x7fc73015e070 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:59.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:59.184+0000 7fc737516640 1 -- 172.21.15.92:0/2681604129 >> 172.21.15.92:0/2681604129 conn(0x7fc73006a800 msgr2=0x7fc73015aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:59.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:59.184+0000 7fc737516640 1 -- 172.21.15.92:0/2681604129 shutdown_connections 2023-11-24T06:10:59.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:59.184+0000 7fc737516640 1 -- 172.21.15.92:0/2681604129 wait complete. 2023-11-24T06:10:59.206 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T06:10:59.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:59.407+0000 7f699f389640 1 Processor -- start 2023-11-24T06:10:59.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:59.408+0000 7f699f389640 1 -- start start 2023-11-24T06:10:59.412 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:59.409+0000 7f699f389640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f699815e3b0 0x7f6998164de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:59.412 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:59.409+0000 7f699f389640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69980b1ee0 con 0x7f69980cf9a0 2023-11-24T06:10:59.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:59.409+0000 7f699f389640 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69980b2780 con 0x7f6998143ae0 2023-11-24T06:10:59.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:59.409+0000 7f699f389640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f699815e990 con 0x7f699815e3b0 2023-11-24T06:10:59.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:59.409+0000 7f699c924640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f69980cf9a0 0x7f699815dca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:35492/0 (socket says 172.21.15.92:35492) 2023-11-24T06:10:59.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:59.409+0000 7f699c924640 1 -- 172.21.15.92:0/1054978429 learned_addr learned my addr 172.21.15.92:0/1054978429 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-24T06:10:59.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:59.409+0000 7f699d926640 1 --2- 172.21.15.92:0/1054978429 >> v2:172.21.15.92:3300/0 conn(0x7f699815e3b0 0x7f6998164de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:59.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:59.410+0000 7f699d926640 1 -- 172.21.15.92:0/1054978429 >> v1:172.21.15.118:6789/0 conn(0x7f6998143ae0 legacy=0x7f69980cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:59.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:59.410+0000 7f698ffff640 1 -- 172.21.15.92:0/1054978429 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2198919961 0 0) 0x7f69980b1ee0 con 0x7f69980cf9a0 2023-11-24T06:10:59.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:59.410+0000 7f699d926640 1 -- 172.21.15.92:0/1054978429 >> v1:172.21.15.92:6789/0 conn(0x7f69980cf9a0 legacy=0x7f699815dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:59.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:59.410+0000 7f699d926640 1 -- 172.21.15.92:0/1054978429 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69980b2130 con 0x7f699815e3b0 2023-11-24T06:10:59.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:59.411+0000 7f699d926640 1 --2- 172.21.15.92:0/1054978429 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f699815e3b0 0x7f6998164de0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f6990009d90 tx=0x7f699002bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42100caa4f1ff180 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:59.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:59.411+0000 7f698ffff640 1 -- 172.21.15.92:0/1054978429 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f699002cd00 con 0x7f699815e3b0 2023-11-24T06:10:59.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:59.411+0000 7f698ffff640 1 -- 172.21.15.92:0/1054978429 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f699002ce60 con 0x7f699815e3b0 2023-11-24T06:10:59.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:59.411+0000 7f698ffff640 1 -- 172.21.15.92:0/1054978429 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6990034660 con 0x7f699815e3b0 2023-11-24T06:10:59.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:59.412+0000 7f699f389640 1 -- 172.21.15.92:0/1054978429 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f699815e3b0 msgr2=0x7f6998164de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:59.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:59.412+0000 7f699f389640 1 --2- 172.21.15.92:0/1054978429 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f699815e3b0 0x7f6998164de0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f6990009d90 tx=0x7f699002bf50 comp rx=0 tx=0).stop 2023-11-24T06:10:59.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:59.412+0000 7f699f389640 1 -- 172.21.15.92:0/1054978429 shutdown_connections 2023-11-24T06:10:59.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:59.412+0000 7f699f389640 1 --2- 172.21.15.92:0/1054978429 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f699815e3b0 0x7f6998164de0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:59.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:59.412+0000 7f699f389640 1 -- 172.21.15.92:0/1054978429 >> 172.21.15.92:0/1054978429 conn(0x7f699806a800 msgr2=0x7f699806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:59.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:59.412+0000 7f699f389640 1 -- 172.21.15.92:0/1054978429 shutdown_connections 2023-11-24T06:10:59.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:59.412+0000 7f699f389640 1 -- 172.21.15.92:0/1054978429 wait complete. 2023-11-24T06:10:59.417 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:59.414+0000 7f699f389640 1 Processor -- start 2023-11-24T06:10:59.417 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:59.414+0000 7f699f389640 1 -- start start 2023-11-24T06:10:59.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:59.415+0000 7f699f389640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f69980cf9a0 0x7f69980cdd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:59.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:59.415+0000 7f699f389640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6998143ae0 0x7f699813f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:59.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:59.415+0000 7f699f389640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f699815e3b0 0x7f699813f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:59.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:59.415+0000 7f699f389640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69981644e0 con 0x7f6998143ae0 2023-11-24T06:10:59.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:59.415+0000 7f699f389640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6998163b80 con 0x7f69980cf9a0 2023-11-24T06:10:59.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:59.415+0000 7f699f389640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69980ce400 con 0x7f699815e3b0 2023-11-24T06:10:59.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:59.415+0000 7f699c924640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6998143ae0 0x7f699813f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:59.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:59.415+0000 7f699c924640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6998143ae0 0x7f699813f2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53044/0 (socket says 172.21.15.92:53044) 2023-11-24T06:10:59.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:59.415+0000 7f699c924640 1 -- 172.21.15.92:0/2046352386 learned_addr learned my addr 172.21.15.92:0/2046352386 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:10:59.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:59.416+0000 7f699d926640 1 --2- 172.21.15.92:0/2046352386 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f699815e3b0 0x7f699813f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:59.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:59.416+0000 7f699d125640 1 --2- 172.21.15.92:0/2046352386 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f69980cf9a0 0x7f69980cdd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:59.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:59.416+0000 7f699c924640 1 -- 172.21.15.92:0/2046352386 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f699815e3b0 msgr2=0x7f699813f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:59.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:59.416+0000 7f699c924640 1 --2- 172.21.15.92:0/2046352386 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f699815e3b0 0x7f699813f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:59.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:59.416+0000 7f699c924640 1 -- 172.21.15.92:0/2046352386 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f69980cf9a0 msgr2=0x7f69980cdd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:59.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:59.416+0000 7f699c924640 1 --2- 172.21.15.92:0/2046352386 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f69980cf9a0 0x7f69980cdd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:59.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:59.416+0000 7f699c924640 1 -- 172.21.15.92:0/2046352386 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6990009a70 con 0x7f6998143ae0 2023-11-24T06:10:59.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:59.416+0000 7f699c924640 1 --2- 172.21.15.92:0/2046352386 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6998143ae0 0x7f699813f2d0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f698800c4a0 tx=0x7f698800c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:59.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:59.416+0000 7f698dffb640 1 -- 172.21.15.92:0/2046352386 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69880074e0 con 0x7f6998143ae0 2023-11-24T06:10:59.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:59.416+0000 7f698dffb640 1 -- 172.21.15.92:0/2046352386 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6988007640 con 0x7f6998143ae0 2023-11-24T06:10:59.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:59.416+0000 7f699f389640 1 -- 172.21.15.92:0/2046352386 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6998163e20 con 0x7f6998143ae0 2023-11-24T06:10:59.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:59.416+0000 7f698dffb640 1 -- 172.21.15.92:0/2046352386 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6988004a00 con 0x7f6998143ae0 2023-11-24T06:10:59.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:59.417+0000 7f699f389640 1 -- 172.21.15.92:0/2046352386 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69980b0860 con 0x7f6998143ae0 2023-11-24T06:10:59.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:59.418+0000 7f699f389640 1 -- 172.21.15.92:0/2046352386 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6998164620 con 0x7f6998143ae0 2023-11-24T06:10:59.423 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:59.420+0000 7f698dffb640 1 -- 172.21.15.92:0/2046352386 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6988004b60 con 0x7f6998143ae0 2023-11-24T06:10:59.424 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:59.421+0000 7f698dffb640 1 --2- 172.21.15.92:0/2046352386 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f697c06ae40 0x7f697c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:59.424 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:59.421+0000 7f698dffb640 1 -- 172.21.15.92:0/2046352386 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6988088ba0 con 0x7f6998143ae0 2023-11-24T06:10:59.424 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:59.421+0000 7f699d125640 1 --2- 172.21.15.92:0/2046352386 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f697c06ae40 0x7f697c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:59.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:59.422+0000 7f699d125640 1 --2- 172.21.15.92:0/2046352386 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f697c06ae40 0x7f697c06d300 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f6994000f60 tx=0x7f6994009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:59.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:59.426+0000 7f698dffb640 1 -- 172.21.15.92:0/2046352386 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6998164620 con 0x7f6998143ae0 2023-11-24T06:10:59.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:59.619+0000 7f699f389640 1 -- 172.21.15.92:0/2046352386 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f6998164620 con 0x7f6998143ae0 2023-11-24T06:10:59.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:59.620+0000 7f698dffb640 1 -- 172.21.15.92:0/2046352386 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f6998164620 con 0x7f6998143ae0 2023-11-24T06:10:59.623 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 64 2023-11-24T06:10:59.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:59.622+0000 7f699f389640 1 -- 172.21.15.92:0/2046352386 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f697c06ae40 msgr2=0x7f697c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:59.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:59.622+0000 7f699f389640 1 --2- 172.21.15.92:0/2046352386 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f697c06ae40 0x7f697c06d300 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f6994000f60 tx=0x7f6994009040 comp rx=0 tx=0).stop 2023-11-24T06:10:59.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:59.622+0000 7f699f389640 1 -- 172.21.15.92:0/2046352386 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6998143ae0 msgr2=0x7f699813f2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:59.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:59.622+0000 7f699f389640 1 --2- 172.21.15.92:0/2046352386 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6998143ae0 0x7f699813f2d0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f698800c4a0 tx=0x7f698800c960 comp rx=0 tx=0).stop 2023-11-24T06:10:59.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:59.623+0000 7f699f389640 1 -- 172.21.15.92:0/2046352386 shutdown_connections 2023-11-24T06:10:59.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:59.623+0000 7f699f389640 1 --2- 172.21.15.92:0/2046352386 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f697c06ae40 0x7f697c06d300 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:59.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:59.623+0000 7f699f389640 1 --2- 172.21.15.92:0/2046352386 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f699815e3b0 0x7f699813f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:59.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:59.623+0000 7f699f389640 1 --2- 172.21.15.92:0/2046352386 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6998143ae0 0x7f699813f2d0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:59.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:59.623+0000 7f699f389640 1 --2- 172.21.15.92:0/2046352386 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f69980cf9a0 0x7f69980cdd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:59.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:59.623+0000 7f699f389640 1 -- 172.21.15.92:0/2046352386 >> 172.21.15.92:0/2046352386 conn(0x7f699806a800 msgr2=0x7f69980cf260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:59.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:59.623+0000 7f699f389640 1 -- 172.21.15.92:0/2046352386 shutdown_connections 2023-11-24T06:10:59.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:59.623+0000 7f699f389640 1 -- 172.21.15.92:0/2046352386 wait complete. 2023-11-24T06:10:59.643 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:10:59.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:59.828+0000 7f26a5fe5640 1 Processor -- start 2023-11-24T06:10:59.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:59.828+0000 7f26a5fe5640 1 -- start start 2023-11-24T06:10:59.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:59.830+0000 7f26a5fe5640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f26a00cf9a0 0x7f26a0166030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:59.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:59.830+0000 7f26a5fe5640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26a015c5f0 con 0x7f26a013e360 2023-11-24T06:10:59.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:59.830+0000 7f26a5fe5640 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26a006c470 con 0x7f26a013df80 2023-11-24T06:10:59.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:59.830+0000 7f26a5fe5640 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26a006c750 con 0x7f26a00cf9a0 2023-11-24T06:10:59.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:59.830+0000 7f2697fff640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f26a013e360 0x7f26a00cf290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:35502/0 (socket says 172.21.15.92:35502) 2023-11-24T06:10:59.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:59.830+0000 7f2697fff640 1 -- 172.21.15.92:0/234645290 learned_addr learned my addr 172.21.15.92:0/234645290 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-24T06:10:59.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:59.831+0000 7f269effd640 1 -- 172.21.15.92:0/234645290 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 579729492 0 0) 0x7f26a015c5f0 con 0x7f26a013e360 2023-11-24T06:10:59.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:59.831+0000 7f269effd640 1 -- 172.21.15.92:0/234645290 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f268c003520 con 0x7f26a013e360 2023-11-24T06:10:59.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:59.831+0000 7f269ffff640 1 --2- 172.21.15.92:0/234645290 >> v2:172.21.15.118:3300/0 conn(0x7f26a00cf9a0 0x7f26a0166030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:59.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:59.831+0000 7f269effd640 1 -- 172.21.15.92:0/234645290 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3298799286 0 0) 0x7f268c003520 con 0x7f26a013e360 2023-11-24T06:10:59.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:59.831+0000 7f269effd640 1 -- 172.21.15.92:0/234645290 >> v2:172.21.15.118:3300/0 conn(0x7f26a00cf9a0 msgr2=0x7f26a0166030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:59.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:59.831+0000 7f269effd640 1 --2- 172.21.15.92:0/234645290 >> v2:172.21.15.118:3300/0 conn(0x7f26a00cf9a0 0x7f26a0166030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:59.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:59.831+0000 7f269effd640 1 -- 172.21.15.92:0/234645290 >> v1:172.21.15.118:6789/0 conn(0x7f26a013df80 legacy=0x7f26a015bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:59.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:59.831+0000 7f269effd640 1 -- 172.21.15.92:0/234645290 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26a006ca30 con 0x7f26a013e360 2023-11-24T06:10:59.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:59.831+0000 7f269effd640 1 -- 172.21.15.92:0/234645290 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2607661350 0 0) 0x7f2688001670 con 0x7f26a013e360 2023-11-24T06:10:59.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:59.831+0000 7f269effd640 1 -- 172.21.15.92:0/234645290 >> v1:172.21.15.92:6789/0 conn(0x7f26a013e360 legacy=0x7f26a00cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:59.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:59.831+0000 7f269effd640 1 --2- 172.21.15.92:0/234645290 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f268c0039c0 0x7f268c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:59.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:59.831+0000 7f269effd640 1 --2- 172.21.15.92:0/234645290 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f268c0047c0 0x7f268c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:59.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:59.831+0000 7f269effd640 1 --2- 172.21.15.92:0/234645290 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f268c0051d0 0x7f268c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:59.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:59.831+0000 7f269effd640 1 -- 172.21.15.92:0/234645290 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26a006c750 con 0x7f268c0051d0 2023-11-24T06:10:59.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:59.831+0000 7f269effd640 1 -- 172.21.15.92:0/234645290 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f268c00c120 con 0x7f268c0047c0 2023-11-24T06:10:59.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:59.831+0000 7f269effd640 1 -- 172.21.15.92:0/234645290 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f268c00c2e0 con 0x7f268c0039c0 2023-11-24T06:10:59.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:59.832+0000 7f269ffff640 1 --2- 172.21.15.92:0/234645290 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f268c0051d0 0x7f268c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:59.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:59.832+0000 7f269f7fe640 1 --2- 172.21.15.92:0/234645290 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f268c0039c0 0x7f268c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:59.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:59.832+0000 7f269ffff640 1 -- 172.21.15.92:0/234645290 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f268c0039c0 msgr2=0x7f268c003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:59.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:59.832+0000 7f2697fff640 1 --2- 172.21.15.92:0/234645290 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f268c0047c0 0x7f268c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:59.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:59.832+0000 7f269ffff640 1 --2- 172.21.15.92:0/234645290 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f268c0039c0 0x7f268c003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:59.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:59.832+0000 7f269ffff640 1 -- 172.21.15.92:0/234645290 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f268c0047c0 msgr2=0x7f268c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:59.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:59.832+0000 7f269ffff640 1 --2- 172.21.15.92:0/234645290 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f268c0047c0 0x7f268c004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:59.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:59.832+0000 7f269ffff640 1 -- 172.21.15.92:0/234645290 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f268c003520 con 0x7f268c0051d0 2023-11-24T06:10:59.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:59.832+0000 7f2697fff640 1 --2- 172.21.15.92:0/234645290 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f268c0047c0 0x7f268c004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:10:59.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:59.833+0000 7f269ffff640 1 -- 172.21.15.92:0/234645290 reap_dead start 2023-11-24T06:10:59.839 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:59.833+0000 7f269ffff640 1 --2- 172.21.15.92:0/234645290 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f268c0051d0 0x7f268c00b980 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f2698007330 tx=0x7f268c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=338638e993eca825 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:59.839 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:59.833+0000 7f269effd640 1 -- 172.21.15.92:0/234645290 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f269802ccd0 con 0x7f268c0051d0 2023-11-24T06:10:59.839 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:59.833+0000 7f269effd640 1 -- 172.21.15.92:0/234645290 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f269802ce30 con 0x7f268c0051d0 2023-11-24T06:10:59.839 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:59.833+0000 7f269effd640 1 -- 172.21.15.92:0/234645290 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2698003bd0 con 0x7f268c0051d0 2023-11-24T06:10:59.839 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:59.834+0000 7f26a5fe5640 1 -- 172.21.15.92:0/234645290 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f268c0051d0 msgr2=0x7f268c00b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:59.839 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:59.834+0000 7f26a5fe5640 1 --2- 172.21.15.92:0/234645290 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f268c0051d0 0x7f268c00b980 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f2698007330 tx=0x7f268c0057f0 comp rx=0 tx=0).stop 2023-11-24T06:10:59.839 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:59.834+0000 7f26a5fe5640 1 -- 172.21.15.92:0/234645290 shutdown_connections 2023-11-24T06:10:59.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:59.834+0000 7f26a5fe5640 1 --2- 172.21.15.92:0/234645290 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f268c0051d0 0x7f268c00b980 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:59.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:59.834+0000 7f26a5fe5640 1 -- 172.21.15.92:0/234645290 >> 172.21.15.92:0/234645290 conn(0x7f26a006a800 msgr2=0x7f26a006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:59.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:59.834+0000 7f26a5fe5640 1 -- 172.21.15.92:0/234645290 shutdown_connections 2023-11-24T06:10:59.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:59.834+0000 7f26a5fe5640 1 -- 172.21.15.92:0/234645290 wait complete. 2023-11-24T06:10:59.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:59.836+0000 7f26a5fe5640 1 Processor -- start 2023-11-24T06:10:59.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:59.836+0000 7f26a5fe5640 1 -- start start 2023-11-24T06:10:59.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:59.837+0000 7f26a5fe5640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f268c0051d0 0x7f26a0132980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:59.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:59.837+0000 7f26a5fe5640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f26a00adce0 0x7f26a0153890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:59.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:59.837+0000 7f26a5fe5640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f26a0153dd0 0x7f26a00abcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:59.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:59.837+0000 7f26a5fe5640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26a012f490 con 0x7f26a0153dd0 2023-11-24T06:10:59.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:59.837+0000 7f26a5fe5640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26a012f730 con 0x7f26a00adce0 2023-11-24T06:10:59.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:59.837+0000 7f26a5fe5640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26a012fa10 con 0x7f268c0051d0 2023-11-24T06:10:59.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:59.837+0000 7f269ffff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f26a0153dd0 0x7f26a00abcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:59.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:59.837+0000 7f269ffff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f26a0153dd0 0x7f26a00abcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53062/0 (socket says 172.21.15.92:53062) 2023-11-24T06:10:59.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:59.837+0000 7f269ffff640 1 -- 172.21.15.92:0/1564385732 learned_addr learned my addr 172.21.15.92:0/1564385732 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:10:59.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:59.837+0000 7f269f7fe640 1 --2- 172.21.15.92:0/1564385732 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f268c0051d0 0x7f26a0132980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:59.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:59.837+0000 7f2697fff640 1 --2- 172.21.15.92:0/1564385732 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f26a00adce0 0x7f26a0153890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:59.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:59.838+0000 7f269ffff640 1 -- 172.21.15.92:0/1564385732 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f268c0051d0 msgr2=0x7f26a0132980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:59.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:59.838+0000 7f269ffff640 1 --2- 172.21.15.92:0/1564385732 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f268c0051d0 0x7f26a0132980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:59.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:59.838+0000 7f269ffff640 1 -- 172.21.15.92:0/1564385732 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f26a00adce0 msgr2=0x7f26a0153890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:59.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:59.838+0000 7f269ffff640 1 --2- 172.21.15.92:0/1564385732 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f26a00adce0 0x7f26a0153890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:59.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:59.838+0000 7f269ffff640 1 -- 172.21.15.92:0/1564385732 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26a012fcf0 con 0x7f26a0153dd0 2023-11-24T06:10:59.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:59.838+0000 7f2697fff640 1 --2- 172.21.15.92:0/1564385732 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f26a00adce0 0x7f26a0153890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:10:59.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:59.838+0000 7f269ffff640 1 --2- 172.21.15.92:0/1564385732 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f26a0153dd0 0x7f26a00abcf0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f2698007450 tx=0x7f2698004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:59.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:59.838+0000 7f269cff9640 1 -- 172.21.15.92:0/1564385732 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2698004550 con 0x7f26a0153dd0 2023-11-24T06:10:59.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:59.838+0000 7f269cff9640 1 -- 172.21.15.92:0/1564385732 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26980046d0 con 0x7f26a0153dd0 2023-11-24T06:10:59.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:59.838+0000 7f269cff9640 1 -- 172.21.15.92:0/1564385732 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2698034680 con 0x7f26a0153dd0 2023-11-24T06:10:59.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:59.838+0000 7f26a5fe5640 1 -- 172.21.15.92:0/1564385732 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26a012ffd0 con 0x7f26a0153dd0 2023-11-24T06:10:59.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:59.838+0000 7f26a5fe5640 1 -- 172.21.15.92:0/1564385732 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26a0132ec0 con 0x7f26a0153dd0 2023-11-24T06:10:59.844 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:59.839+0000 7f26a5fe5640 1 -- 172.21.15.92:0/1564385732 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26a01304f0 con 0x7f26a0153dd0 2023-11-24T06:10:59.844 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:59.841+0000 7f269cff9640 1 -- 172.21.15.92:0/1564385732 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f269803b070 con 0x7f26a0153dd0 2023-11-24T06:10:59.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:59.842+0000 7f269cff9640 1 --2- 172.21.15.92:0/1564385732 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f268006ae40 0x7f268006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:59.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:59.842+0000 7f269cff9640 1 -- 172.21.15.92:0/1564385732 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f26980aad30 con 0x7f26a0153dd0 2023-11-24T06:10:59.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:59.842+0000 7f269f7fe640 1 --2- 172.21.15.92:0/1564385732 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f268006ae40 0x7f268006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:59.846 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:59.843+0000 7f269f7fe640 1 --2- 172.21.15.92:0/1564385732 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f268006ae40 0x7f268006d300 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f2690005490 tx=0x7f2690009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:59.850 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:10:59.847+0000 7f269cff9640 1 -- 172.21.15.92:0/1564385732 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f26a01304f0 con 0x7f26a0153dd0 2023-11-24T06:11:00.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:00.038+0000 7f26a5fe5640 1 -- 172.21.15.92:0/1564385732 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f26a01304f0 con 0x7f26a0153dd0 2023-11-24T06:11:00.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:00.040+0000 7f269cff9640 1 -- 172.21.15.92:0/1564385732 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16457 (secure 0 0 0) 0x7f26980a8060 con 0x7f26a0153dd0 2023-11-24T06:11:00.044 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-24T06:11:00.045 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":20,"fsid":"2d6dee5f-8aa3-470f-931a-50ade963b1cb","created":"2023-11-24T06:10:04.707204+0000","modified":"2023-11-24T06:10:26.698733+0000","last_up_change":"2023-11-24T06:10:17.598352+0000","last_in_change":"2023-11-24T06:10:09.040793+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:10:17.733098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:10:22.926445+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T06:10:24.151288+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ff37a9fb-135f-471f-b4b0-f7b5772006c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6803","nonce":20560714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6805","nonce":20560714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6809","nonce":20560714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6807","nonce":20560714}]},"public_addr":"172.21.15.92:6803/20560714","cluster_addr":"172.21.15.92:6805/20560714","heartbeat_back_addr":"172.21.15.92:6809/20560714","heartbeat_front_addr":"172.21.15.92:6807/20560714","state":["exists","up"]},{"osd":1,"uuid":"03cdd353-dcf1-4f38-94fe-a795137c06b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6819","nonce":2451239906}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6821","nonce":2451239906}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6825","nonce":2451239906}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6823","nonce":2451239906}]},"public_addr":"172.21.15.92:6819/2451239906","cluster_addr":"172.21.15.92:6821/2451239906","heartbeat_back_addr":"172.21.15.92:6825/2451239906","heartbeat_front_addr":"172.21.15.92:6823/2451239906","state":["exists","up"]},{"osd":2,"uuid":"a4a20d80-5d57-484b-becd-9aa8070dd3aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6811","nonce":1124979277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6813","nonce":1124979277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6817","nonce":1124979277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6815","nonce":1124979277}]},"public_addr":"172.21.15.92:6811/1124979277","cluster_addr":"172.21.15.92:6813/1124979277","heartbeat_back_addr":"172.21.15.92:6817/1124979277","heartbeat_front_addr":"172.21.15.92:6815/1124979277","state":["exists","up"]},{"osd":3,"uuid":"4648d7ce-9b29-4583-9425-4ba8fd2b1903","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6827","nonce":1734115782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6829","nonce":1734115782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6833","nonce":1734115782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6831","nonce":1734115782}]},"public_addr":"172.21.15.92:6827/1734115782","cluster_addr":"172.21.15.92:6829/1734115782","heartbeat_back_addr":"172.21.15.92:6833/1734115782","heartbeat_front_addr":"172.21.15.92:6831/1734115782","state":["exists","up"]},{"osd":4,"uuid":"3bc9d7eb-c6fa-4fcf-a3ec-ade604e05aac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6819","nonce":3425937229}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6822","nonce":3425937229}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6830","nonce":3425937229}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6826","nonce":3425937229}]},"public_addr":"172.21.15.118:6819/3425937229","cluster_addr":"172.21.15.118:6822/3425937229","heartbeat_back_addr":"172.21.15.118:6830/3425937229","heartbeat_front_addr":"172.21.15.118:6826/3425937229","state":["exists","up"]},{"osd":5,"uuid":"dce9ba6d-59ed-46d1-84fc-a84b13de10c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6801","nonce":827455984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6803","nonce":827455984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6807","nonce":827455984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6805","nonce":827455984}]},"public_addr":"172.21.15.118:6801/827455984","cluster_addr":"172.21.15.118:6803/827455984","heartbeat_back_addr":"172.21.15.118:6807/827455984","heartbeat_front_addr":"172.21.15.118:6805/827455984","state":["exists","up"]},{"osd":6,"uuid":"82aec504-ae58-4709-8d30-46f2b12081b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6809","nonce":2946986141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6811","nonce":2946986141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6815","nonce":2946986141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6813","nonce":2946986141}]},"public_addr":"172.21.15.118:6809/2946986141","cluster_addr":"172.21.15.118:6811/2946986141","heartbeat_back_addr":"172.21.15.118:6815/2946986141","heartbeat_front_addr":"172.21.15.118:6813/2946986141","state":["exists","up"]},{"osd":7,"uuid":"4ed4c624-c88f-4d35-bdb1-bf209b5dffb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6817","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6818","nonce":48122059}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6821","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6823","nonce":48122059}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6829","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6831","nonce":48122059}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6825","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6827","nonce":48122059}]},"public_addr":"172.21.15.118:6818/48122059","cluster_addr":"172.21.15.118:6823/48122059","heartbeat_back_addr":"172.21.15.118:6831/48122059","heartbeat_front_addr":"172.21.15.118:6827/48122059","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.536664+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.368324+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.271201+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.342971+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.621832+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.387110+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.632946+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.416032+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:11:00.047 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:00.043+0000 7f26a5fe5640 1 -- 172.21.15.92:0/1564385732 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f268006ae40 msgr2=0x7f268006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:00.047 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:00.043+0000 7f26a5fe5640 1 --2- 172.21.15.92:0/1564385732 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f268006ae40 0x7f268006d300 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f2690005490 tx=0x7f2690009040 comp rx=0 tx=0).stop 2023-11-24T06:11:00.047 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:00.043+0000 7f26a5fe5640 1 -- 172.21.15.92:0/1564385732 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f26a0153dd0 msgr2=0x7f26a00abcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:00.048 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:00.043+0000 7f26a5fe5640 1 --2- 172.21.15.92:0/1564385732 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f26a0153dd0 0x7f26a00abcf0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f2698007450 tx=0x7f2698004ab0 comp rx=0 tx=0).stop 2023-11-24T06:11:00.048 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:00.043+0000 7f26a5fe5640 1 -- 172.21.15.92:0/1564385732 shutdown_connections 2023-11-24T06:11:00.048 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:00.043+0000 7f26a5fe5640 1 --2- 172.21.15.92:0/1564385732 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f268006ae40 0x7f268006d300 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:00.048 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:00.043+0000 7f26a5fe5640 1 --2- 172.21.15.92:0/1564385732 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f26a0153dd0 0x7f26a00abcf0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:00.048 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:00.043+0000 7f26a5fe5640 1 --2- 172.21.15.92:0/1564385732 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f26a00adce0 0x7f26a0153890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:00.048 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:00.043+0000 7f26a5fe5640 1 --2- 172.21.15.92:0/1564385732 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f268c0051d0 0x7f26a0132980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:00.049 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:00.043+0000 7f26a5fe5640 1 -- 172.21.15.92:0/1564385732 >> 172.21.15.92:0/1564385732 conn(0x7f26a006a800 msgr2=0x7f26a00aea60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:00.049 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:00.043+0000 7f26a5fe5640 1 -- 172.21.15.92:0/1564385732 shutdown_connections 2023-11-24T06:11:00.049 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:00.043+0000 7f26a5fe5640 1 -- 172.21.15.92:0/1564385732 wait complete. 2023-11-24T06:11:00.067 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T06:10:17.733098+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T06:10:22.926445+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T06:10:24.151288+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T06:11:00.067 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T06:11:00.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:00.262+0000 7f1605831640 1 Processor -- start 2023-11-24T06:11:00.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:00.262+0000 7f1605831640 1 -- start start 2023-11-24T06:11:00.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:00.263+0000 7f1605831640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f1600143ae0 0x7f16000cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:00.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:00.263+0000 7f1605831640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f160015e3b0 0x7f16000702c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:00.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:00.263+0000 7f1605831640 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16000cfc30 con 0x7f16000cf7d0 2023-11-24T06:11:00.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:00.263+0000 7f1605831640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16000b1ee0 con 0x7f1600143ae0 2023-11-24T06:11:00.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:00.263+0000 7f1605831640 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16000b2780 con 0x7f160015e3b0 2023-11-24T06:11:00.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:00.264+0000 7f15feffd640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f1600143ae0 0x7f16000cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:00.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:00.264+0000 7f15feffd640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f1600143ae0 0x7f16000cf290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53070/0 (socket says 172.21.15.92:53070) 2023-11-24T06:11:00.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:00.264+0000 7f15feffd640 1 -- 172.21.15.92:0/3911130080 learned_addr learned my addr 172.21.15.92:0/3911130080 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:11:00.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:00.264+0000 7f15feffd640 1 -- 172.21.15.92:0/3911130080 >> v2:172.21.15.118:3300/0 conn(0x7f160015e3b0 msgr2=0x7f16000702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:00.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:00.264+0000 7f15feffd640 1 --2- 172.21.15.92:0/3911130080 >> v2:172.21.15.118:3300/0 conn(0x7f160015e3b0 0x7f16000702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:00.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:00.264+0000 7f15feffd640 1 -- 172.21.15.92:0/3911130080 >> v1:172.21.15.118:6789/0 conn(0x7f16000cf7d0 legacy=0x7f160015dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:00.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:00.264+0000 7f15feffd640 1 -- 172.21.15.92:0/3911130080 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f160015e990 con 0x7f1600143ae0 2023-11-24T06:11:00.269 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:00.265+0000 7f15feffd640 1 --2- 172.21.15.92:0/3911130080 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1600143ae0 0x7f16000cf290 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f160006e4f0 tx=0x7f15f002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2805015c218822 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:00.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:00.266+0000 7f15fe7fc640 1 -- 172.21.15.92:0/3911130080 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15f002ca80 con 0x7f1600143ae0 2023-11-24T06:11:00.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:00.266+0000 7f15fe7fc640 1 -- 172.21.15.92:0/3911130080 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15f002cbe0 con 0x7f1600143ae0 2023-11-24T06:11:00.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:00.266+0000 7f15fe7fc640 1 -- 172.21.15.92:0/3911130080 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15f00343f0 con 0x7f1600143ae0 2023-11-24T06:11:00.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:00.266+0000 7f1605831640 1 -- 172.21.15.92:0/3911130080 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1600143ae0 msgr2=0x7f16000cf290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:00.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:00.266+0000 7f1605831640 1 --2- 172.21.15.92:0/3911130080 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1600143ae0 0x7f16000cf290 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f160006e4f0 tx=0x7f15f002bee0 comp rx=0 tx=0).stop 2023-11-24T06:11:00.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:00.266+0000 7f1605831640 1 -- 172.21.15.92:0/3911130080 shutdown_connections 2023-11-24T06:11:00.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:00.266+0000 7f1605831640 1 --2- 172.21.15.92:0/3911130080 >> v2:172.21.15.118:3300/0 conn(0x7f160015e3b0 0x7f16000702c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:00.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:00.266+0000 7f1605831640 1 --2- 172.21.15.92:0/3911130080 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1600143ae0 0x7f16000cf290 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:00.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:00.266+0000 7f1605831640 1 -- 172.21.15.92:0/3911130080 >> 172.21.15.92:0/3911130080 conn(0x7f160006a800 msgr2=0x7f160006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:00.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:00.266+0000 7f1605831640 1 -- 172.21.15.92:0/3911130080 shutdown_connections 2023-11-24T06:11:00.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:00.266+0000 7f1605831640 1 -- 172.21.15.92:0/3911130080 wait complete. 2023-11-24T06:11:00.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:00.268+0000 7f1605831640 1 Processor -- start 2023-11-24T06:11:00.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:00.268+0000 7f1605831640 1 -- start start 2023-11-24T06:11:00.272 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:00.269+0000 7f1605831640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f16000cf7d0 0x7f1600155da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:00.272 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:00.269+0000 7f1605831640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1600143ae0 0x7f16000ce180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:00.272 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:00.269+0000 7f1605831640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f160015e3b0 0x7f16001397a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:00.272 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:00.269+0000 7f1605831640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1600159cd0 con 0x7f160015e3b0 2023-11-24T06:11:00.273 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:00.269+0000 7f1605831640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f160015cba0 con 0x7f1600143ae0 2023-11-24T06:11:00.273 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:00.269+0000 7f1605831640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f160013c340 con 0x7f16000cf7d0 2023-11-24T06:11:00.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:00.269+0000 7f15ff7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f160015e3b0 0x7f16001397a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:00.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:00.270+0000 7f15ff7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f160015e3b0 0x7f16001397a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53076/0 (socket says 172.21.15.92:53076) 2023-11-24T06:11:00.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:00.270+0000 7f15ff7fe640 1 -- 172.21.15.92:0/4103854671 learned_addr learned my addr 172.21.15.92:0/4103854671 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:11:00.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:00.270+0000 7f15feffd640 1 --2- 172.21.15.92:0/4103854671 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f16000cf7d0 0x7f1600155da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:00.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:00.270+0000 7f15f67fc640 1 --2- 172.21.15.92:0/4103854671 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1600143ae0 0x7f16000ce180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:00.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:00.270+0000 7f15ff7fe640 1 -- 172.21.15.92:0/4103854671 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f16000cf7d0 msgr2=0x7f1600155da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:00.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:00.270+0000 7f15ff7fe640 1 --2- 172.21.15.92:0/4103854671 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f16000cf7d0 0x7f1600155da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:00.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:00.270+0000 7f15ff7fe640 1 -- 172.21.15.92:0/4103854671 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1600143ae0 msgr2=0x7f16000ce180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:00.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:00.270+0000 7f15ff7fe640 1 --2- 172.21.15.92:0/4103854671 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1600143ae0 0x7f16000ce180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:00.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:00.270+0000 7f15ff7fe640 1 -- 172.21.15.92:0/4103854671 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15f0009b30 con 0x7f160015e3b0 2023-11-24T06:11:00.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:00.270+0000 7f15f67fc640 1 --2- 172.21.15.92:0/4103854671 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1600143ae0 0x7f16000ce180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:11:00.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:00.270+0000 7f15ff7fe640 1 --2- 172.21.15.92:0/4103854671 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f160015e3b0 0x7f16001397a0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f15f800b440 tx=0x7f15f800b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:00.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:00.271+0000 7f15fcff9640 1 -- 172.21.15.92:0/4103854671 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15f8011830 con 0x7f160015e3b0 2023-11-24T06:11:00.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:00.271+0000 7f15fcff9640 1 -- 172.21.15.92:0/4103854671 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15f8011990 con 0x7f160015e3b0 2023-11-24T06:11:00.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:00.271+0000 7f1605831640 1 -- 172.21.15.92:0/4103854671 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f160013c8c0 con 0x7f160015e3b0 2023-11-24T06:11:00.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:00.271+0000 7f15fcff9640 1 -- 172.21.15.92:0/4103854671 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15f8011d40 con 0x7f160015e3b0 2023-11-24T06:11:00.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:00.271+0000 7f1605831640 1 -- 172.21.15.92:0/4103854671 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f160013c5e0 con 0x7f160015e3b0 2023-11-24T06:11:00.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:00.272+0000 7f1605831640 1 -- 172.21.15.92:0/4103854671 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16000cdea0 con 0x7f160015e3b0 2023-11-24T06:11:00.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:00.273+0000 7f15fcff9640 1 -- 172.21.15.92:0/4103854671 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f15f801a400 con 0x7f160015e3b0 2023-11-24T06:11:00.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:00.275+0000 7f15fcff9640 1 --2- 172.21.15.92:0/4103854671 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f15e006ae40 0x7f15e006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:00.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:00.275+0000 7f15fcff9640 1 -- 172.21.15.92:0/4103854671 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f15f8089a20 con 0x7f160015e3b0 2023-11-24T06:11:00.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:00.275+0000 7f15feffd640 1 --2- 172.21.15.92:0/4103854671 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f15e006ae40 0x7f15e006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:00.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:00.276+0000 7f15feffd640 1 --2- 172.21.15.92:0/4103854671 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f15e006ae40 0x7f15e006d300 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f15f0035000 tx=0x7f15f0030000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:00.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:00.278+0000 7f15fcff9640 1 -- 172.21.15.92:0/4103854671 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f15f8054c40 con 0x7f160015e3b0 2023-11-24T06:11:00.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:00.466+0000 7f1605831640 1 -- 172.21.15.92:0/4103854671 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f16000cdea0 con 0x7f160015e3b0 2023-11-24T06:11:00.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:00.470+0000 7f15fcff9640 1 -- 172.21.15.92:0/4103854671 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f16000cdea0 con 0x7f160015e3b0 2023-11-24T06:11:00.473 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 1 2023-11-24T06:11:00.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:00.472+0000 7f1605831640 1 -- 172.21.15.92:0/4103854671 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f15e006ae40 msgr2=0x7f15e006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:00.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:00.472+0000 7f1605831640 1 --2- 172.21.15.92:0/4103854671 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f15e006ae40 0x7f15e006d300 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f15f0035000 tx=0x7f15f0030000 comp rx=0 tx=0).stop 2023-11-24T06:11:00.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:00.472+0000 7f1605831640 1 -- 172.21.15.92:0/4103854671 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f160015e3b0 msgr2=0x7f16001397a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:00.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:00.472+0000 7f1605831640 1 --2- 172.21.15.92:0/4103854671 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f160015e3b0 0x7f16001397a0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f15f800b440 tx=0x7f15f800b900 comp rx=0 tx=0).stop 2023-11-24T06:11:00.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:00.472+0000 7f1605831640 1 -- 172.21.15.92:0/4103854671 shutdown_connections 2023-11-24T06:11:00.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:00.472+0000 7f1605831640 1 --2- 172.21.15.92:0/4103854671 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f15e006ae40 0x7f15e006d300 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:00.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:00.472+0000 7f1605831640 1 --2- 172.21.15.92:0/4103854671 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f160015e3b0 0x7f16001397a0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:00.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:00.472+0000 7f1605831640 1 --2- 172.21.15.92:0/4103854671 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1600143ae0 0x7f16000ce180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:00.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:00.472+0000 7f1605831640 1 --2- 172.21.15.92:0/4103854671 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f16000cf7d0 0x7f1600155da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:00.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:00.472+0000 7f1605831640 1 -- 172.21.15.92:0/4103854671 >> 172.21.15.92:0/4103854671 conn(0x7f160006a800 msgr2=0x7f160015da10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:00.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:00.472+0000 7f1605831640 1 -- 172.21.15.92:0/4103854671 shutdown_connections 2023-11-24T06:11:00.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:00.472+0000 7f1605831640 1 -- 172.21.15.92:0/4103854671 wait complete. 2023-11-24T06:11:00.494 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T06:11:00.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:00.682+0000 7f2fd4fd5640 1 Processor -- start 2023-11-24T06:11:00.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:00.683+0000 7f2fd4fd5640 1 -- start start 2023-11-24T06:11:00.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:00.684+0000 7f2fd4fd5640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f2fd015d610 0x7f2fd00ce820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:00.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:00.684+0000 7f2fd4fd5640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2fd00ced60 0x7f2fd00d17e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:00.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:00.684+0000 7f2fd4fd5640 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2fd00b3400 con 0x7f2fd00d1d20 2023-11-24T06:11:00.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:00.684+0000 7f2fd4fd5640 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fd00d1390 con 0x7f2fd015d610 2023-11-24T06:11:00.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:00.684+0000 7f2fd4fd5640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fd00d21b0 con 0x7f2fd00ced60 2023-11-24T06:11:00.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:00.685+0000 7f2fcdd9b640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2fd00ced60 0x7f2fd00d17e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:00.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:00.685+0000 7f2fcdd9b640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2fd00ced60 0x7f2fd00d17e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3301/0 says I am v2:172.21.15.92:36436/0 (socket says 172.21.15.92:36436) 2023-11-24T06:11:00.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:00.685+0000 7f2fcdd9b640 1 -- 172.21.15.92:0/3620146768 learned_addr learned my addr 172.21.15.92:0/3620146768 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:11:00.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:00.685+0000 7f2fce59c640 1 --2- 172.21.15.92:0/3620146768 >> v2:172.21.15.118:3300/0 conn(0x7f2fd015d610 0x7f2fd00ce820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:00.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:00.685+0000 7f2fcdd9b640 1 -- 172.21.15.92:0/3620146768 >> v2:172.21.15.118:3300/0 conn(0x7f2fd015d610 msgr2=0x7f2fd00ce820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:00.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:00.685+0000 7f2fcdd9b640 1 --2- 172.21.15.92:0/3620146768 >> v2:172.21.15.118:3300/0 conn(0x7f2fd015d610 0x7f2fd00ce820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:00.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:00.686+0000 7f2fcdd9b640 1 -- 172.21.15.92:0/3620146768 >> v1:172.21.15.118:6789/0 conn(0x7f2fd00d1d20 legacy=0x7f2fd00cf830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:00.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:00.686+0000 7f2fcdd9b640 1 -- 172.21.15.92:0/3620146768 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fd00b27f0 con 0x7f2fd00ced60 2023-11-24T06:11:00.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:00.685+0000 7f2fcd59a640 1 -- 172.21.15.92:0/3620146768 <== mon.1 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3429352387 0 0) 0x7f2fd00b3400 con 0x7f2fd00d1d20 2023-11-24T06:11:00.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:00.687+0000 7f2fcdd9b640 1 --2- 172.21.15.92:0/3620146768 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2fd00ced60 0x7f2fd00d17e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f2fb8009e50 tx=0x7f2fb802bf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5fce202c2c0d7b3c server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:00.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:00.687+0000 7f2fcd59a640 1 -- 172.21.15.92:0/3620146768 <== mon.2 v2:172.21.15.118:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fb802cb00 con 0x7f2fd00ced60 2023-11-24T06:11:00.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:00.687+0000 7f2fcd59a640 1 -- 172.21.15.92:0/3620146768 <== mon.2 v2:172.21.15.118:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fb802cc60 con 0x7f2fd00ced60 2023-11-24T06:11:00.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:00.687+0000 7f2fcd59a640 1 -- 172.21.15.92:0/3620146768 <== mon.2 v2:172.21.15.118:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fb80343f0 con 0x7f2fd00ced60 2023-11-24T06:11:00.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:00.688+0000 7f2fd4fd5640 1 -- 172.21.15.92:0/3620146768 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2fd00ced60 msgr2=0x7f2fd00d17e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:00.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:00.688+0000 7f2fd4fd5640 1 --2- 172.21.15.92:0/3620146768 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2fd00ced60 0x7f2fd00d17e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f2fb8009e50 tx=0x7f2fb802bf60 comp rx=0 tx=0).stop 2023-11-24T06:11:00.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:00.688+0000 7f2fd4fd5640 1 -- 172.21.15.92:0/3620146768 shutdown_connections 2023-11-24T06:11:00.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:00.688+0000 7f2fd4fd5640 1 --2- 172.21.15.92:0/3620146768 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2fd00ced60 0x7f2fd00d17e0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:00.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:00.688+0000 7f2fd4fd5640 1 --2- 172.21.15.92:0/3620146768 >> v2:172.21.15.118:3300/0 conn(0x7f2fd015d610 0x7f2fd00ce820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:00.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:00.688+0000 7f2fd4fd5640 1 -- 172.21.15.92:0/3620146768 >> 172.21.15.92:0/3620146768 conn(0x7f2fd009c540 msgr2=0x7f2fd00d3200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:00.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:00.688+0000 7f2fd4fd5640 1 -- 172.21.15.92:0/3620146768 shutdown_connections 2023-11-24T06:11:00.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:00.688+0000 7f2fd4fd5640 1 -- 172.21.15.92:0/3620146768 wait complete. 2023-11-24T06:11:00.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:00.690+0000 7f2fd4fd5640 1 Processor -- start 2023-11-24T06:11:00.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:00.690+0000 7f2fd4fd5640 1 -- start start 2023-11-24T06:11:00.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:00.691+0000 7f2fd4fd5640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2fd00ced60 0x7f2fd00b6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:00.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:00.691+0000 7f2fd4fd5640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2fd00d1d20 0x7f2fd01f2650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:00.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:00.691+0000 7f2fd4fd5640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2fd015d610 0x7f2fd00b8b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:00.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:00.691+0000 7f2fd4fd5640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fd014d420 con 0x7f2fd015d610 2023-11-24T06:11:00.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:00.691+0000 7f2fd4fd5640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fd00b90a0 con 0x7f2fd00d1d20 2023-11-24T06:11:00.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:00.691+0000 7f2fd4fd5640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fd015ed40 con 0x7f2fd00ced60 2023-11-24T06:11:00.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:00.692+0000 7f2fced9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2fd015d610 0x7f2fd00b8b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:00.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:00.692+0000 7f2fced9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2fd015d610 0x7f2fd00b8b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53090/0 (socket says 172.21.15.92:53090) 2023-11-24T06:11:00.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:00.692+0000 7f2fced9d640 1 -- 172.21.15.92:0/1634519038 learned_addr learned my addr 172.21.15.92:0/1634519038 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:11:00.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:00.692+0000 7f2fce59c640 1 --2- 172.21.15.92:0/1634519038 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2fd00ced60 0x7f2fd00b6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:00.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:00.692+0000 7f2fcdd9b640 1 --2- 172.21.15.92:0/1634519038 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2fd00d1d20 0x7f2fd01f2650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:00.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:00.692+0000 7f2fced9d640 1 -- 172.21.15.92:0/1634519038 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2fd00ced60 msgr2=0x7f2fd00b6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:00.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:00.692+0000 7f2fced9d640 1 --2- 172.21.15.92:0/1634519038 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2fd00ced60 0x7f2fd00b6170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:00.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:00.692+0000 7f2fced9d640 1 -- 172.21.15.92:0/1634519038 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2fd00d1d20 msgr2=0x7f2fd01f2650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:00.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:00.692+0000 7f2fced9d640 1 --2- 172.21.15.92:0/1634519038 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2fd00d1d20 0x7f2fd01f2650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:00.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:00.692+0000 7f2fced9d640 1 -- 172.21.15.92:0/1634519038 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fb8009b30 con 0x7f2fd015d610 2023-11-24T06:11:00.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:00.692+0000 7f2fcdd9b640 1 --2- 172.21.15.92:0/1634519038 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2fd00d1d20 0x7f2fd01f2650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:11:00.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:00.692+0000 7f2fced9d640 1 --2- 172.21.15.92:0/1634519038 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2fd015d610 0x7f2fd00b8b60 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f2fc800c430 tx=0x7f2fc800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:00.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:00.693+0000 7f2fc6ffd640 1 -- 172.21.15.92:0/1634519038 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fc80035f0 con 0x7f2fd015d610 2023-11-24T06:11:00.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:00.693+0000 7f2fc6ffd640 1 -- 172.21.15.92:0/1634519038 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fc8003770 con 0x7f2fd015d610 2023-11-24T06:11:00.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:00.693+0000 7f2fc6ffd640 1 -- 172.21.15.92:0/1634519038 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fc8003b20 con 0x7f2fd015d610 2023-11-24T06:11:00.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:00.693+0000 7f2fd4fd5640 1 -- 172.21.15.92:0/1634519038 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fd015f020 con 0x7f2fd015d610 2023-11-24T06:11:00.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:00.693+0000 7f2fd4fd5640 1 -- 172.21.15.92:0/1634519038 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fd015f5e0 con 0x7f2fd015d610 2023-11-24T06:11:00.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:00.694+0000 7f2fd4fd5640 1 -- 172.21.15.92:0/1634519038 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fd015f7d0 con 0x7f2fd015d610 2023-11-24T06:11:00.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:00.695+0000 7f2fc6ffd640 1 -- 172.21.15.92:0/1634519038 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2fc8003c80 con 0x7f2fd015d610 2023-11-24T06:11:00.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:00.696+0000 7f2fc6ffd640 1 --2- 172.21.15.92:0/1634519038 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f2fb406ae40 0x7f2fb406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:00.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:00.696+0000 7f2fc6ffd640 1 -- 172.21.15.92:0/1634519038 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2fc8088d70 con 0x7f2fd015d610 2023-11-24T06:11:00.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:00.696+0000 7f2fce59c640 1 --2- 172.21.15.92:0/1634519038 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f2fb406ae40 0x7f2fb406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:00.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:00.697+0000 7f2fce59c640 1 --2- 172.21.15.92:0/1634519038 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f2fb406ae40 0x7f2fb406d300 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f2fc0009ad0 tx=0x7f2fc0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:00.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:00.700+0000 7f2fc6ffd640 1 -- 172.21.15.92:0/1634519038 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2fc8053cf0 con 0x7f2fd015d610 2023-11-24T06:11:00.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:00.879+0000 7f2fd4fd5640 1 -- 172.21.15.92:0/1634519038 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f2fd00675d0 con 0x7f2fd015d610 2023-11-24T06:11:00.884 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:00.880+0000 7f2fc6ffd640 1 -- 172.21.15.92:0/1634519038 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f2fc8017020 con 0x7f2fd015d610 2023-11-24T06:11:00.884 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 32 2023-11-24T06:11:00.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:00.883+0000 7f2fd4fd5640 1 -- 172.21.15.92:0/1634519038 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f2fb406ae40 msgr2=0x7f2fb406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:00.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:00.883+0000 7f2fd4fd5640 1 --2- 172.21.15.92:0/1634519038 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f2fb406ae40 0x7f2fb406d300 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f2fc0009ad0 tx=0x7f2fc0008040 comp rx=0 tx=0).stop 2023-11-24T06:11:00.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:00.883+0000 7f2fd4fd5640 1 -- 172.21.15.92:0/1634519038 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2fd015d610 msgr2=0x7f2fd00b8b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:00.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:00.883+0000 7f2fd4fd5640 1 --2- 172.21.15.92:0/1634519038 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2fd015d610 0x7f2fd00b8b60 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f2fc800c430 tx=0x7f2fc800c8f0 comp rx=0 tx=0).stop 2023-11-24T06:11:00.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:00.883+0000 7f2fd4fd5640 1 -- 172.21.15.92:0/1634519038 shutdown_connections 2023-11-24T06:11:00.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:00.883+0000 7f2fd4fd5640 1 --2- 172.21.15.92:0/1634519038 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f2fb406ae40 0x7f2fb406d300 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:00.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:00.883+0000 7f2fd4fd5640 1 --2- 172.21.15.92:0/1634519038 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2fd015d610 0x7f2fd00b8b60 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:00.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:00.883+0000 7f2fd4fd5640 1 --2- 172.21.15.92:0/1634519038 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2fd00d1d20 0x7f2fd01f2650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:00.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:00.883+0000 7f2fd4fd5640 1 --2- 172.21.15.92:0/1634519038 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2fd00ced60 0x7f2fd00b6170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:00.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:00.883+0000 7f2fd4fd5640 1 -- 172.21.15.92:0/1634519038 >> 172.21.15.92:0/1634519038 conn(0x7f2fd009c540 msgr2=0x7f2fd015c850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:00.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:00.883+0000 7f2fd4fd5640 1 -- 172.21.15.92:0/1634519038 shutdown_connections 2023-11-24T06:11:00.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:00.883+0000 7f2fd4fd5640 1 -- 172.21.15.92:0/1634519038 wait complete. 2023-11-24T06:11:00.903 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T06:11:01.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.101+0000 7f92ab690640 1 Processor -- start 2023-11-24T06:11:01.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.102+0000 7f92ab690640 1 -- start start 2023-11-24T06:11:01.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.103+0000 7f92ab690640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f92a409a170 0x7f92a40b3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:01.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.103+0000 7f92ab690640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f92a406c6b0 0x7f92a40cf640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:01.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.103+0000 7f92ab690640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92a406cea0 con 0x7f92a40cd240 2023-11-24T06:11:01.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.103+0000 7f92ab690640 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92a406cbc0 con 0x7f92a406c6b0 2023-11-24T06:11:01.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.103+0000 7f92ab690640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92a406f350 con 0x7f92a409a170 2023-11-24T06:11:01.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.103+0000 7f92a942c640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f92a40cd240 0x7f92a4099a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:35508/0 (socket says 172.21.15.92:35508) 2023-11-24T06:11:01.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.104+0000 7f92a942c640 1 -- 172.21.15.92:0/3527613821 learned_addr learned my addr 172.21.15.92:0/3527613821 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-24T06:11:01.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.104+0000 7f92a8c2b640 1 --2- 172.21.15.92:0/3527613821 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f92a409a170 0x7f92a40b3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:01.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.104+0000 7f92a3fff640 1 -- 172.21.15.92:0/3527613821 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1035083893 0 0) 0x7f92a406cea0 con 0x7f92a40cd240 2023-11-24T06:11:01.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.104+0000 7f92a3fff640 1 -- 172.21.15.92:0/3527613821 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f928c003520 con 0x7f92a40cd240 2023-11-24T06:11:01.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.104+0000 7f92a9c2d640 1 --2- 172.21.15.92:0/3527613821 >> v2:172.21.15.118:3300/0 conn(0x7f92a406c6b0 0x7f92a40cf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:01.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.104+0000 7f92a3fff640 1 -- 172.21.15.92:0/3527613821 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1025863570 0 0) 0x7f928c003520 con 0x7f92a40cd240 2023-11-24T06:11:01.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.104+0000 7f92a3fff640 1 -- 172.21.15.92:0/3527613821 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f92a409a170 msgr2=0x7f92a40b3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:01.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.104+0000 7f92a3fff640 1 --2- 172.21.15.92:0/3527613821 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f92a409a170 0x7f92a40b3820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:01.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.104+0000 7f92a3fff640 1 -- 172.21.15.92:0/3527613821 >> v2:172.21.15.118:3300/0 conn(0x7f92a406c6b0 msgr2=0x7f92a40cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:01.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.104+0000 7f92a3fff640 1 --2- 172.21.15.92:0/3527613821 >> v2:172.21.15.118:3300/0 conn(0x7f92a406c6b0 0x7f92a40cf640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:01.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.104+0000 7f92a3fff640 1 -- 172.21.15.92:0/3527613821 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92a406f630 con 0x7f92a40cd240 2023-11-24T06:11:01.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.104+0000 7f92a3fff640 1 -- 172.21.15.92:0/3527613821 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2607661350 0 0) 0x7f92940019f0 con 0x7f92a40cd240 2023-11-24T06:11:01.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.104+0000 7f92a3fff640 1 -- 172.21.15.92:0/3527613821 >> v1:172.21.15.92:6789/0 conn(0x7f92a40cd240 legacy=0x7f92a4099a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:01.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.105+0000 7f92a3fff640 1 --2- 172.21.15.92:0/3527613821 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f928c003970 0x7f928c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:01.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.105+0000 7f92a3fff640 1 --2- 172.21.15.92:0/3527613821 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f92a409a170 0x7f928c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:01.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.105+0000 7f92a3fff640 1 --2- 172.21.15.92:0/3527613821 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f928c004d00 0x7f928c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:01.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.105+0000 7f92a3fff640 1 -- 172.21.15.92:0/3527613821 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92a406cbc0 con 0x7f928c003970 2023-11-24T06:11:01.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.105+0000 7f92a3fff640 1 -- 172.21.15.92:0/3527613821 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92a406f350 con 0x7f928c004d00 2023-11-24T06:11:01.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.105+0000 7f92a3fff640 1 -- 172.21.15.92:0/3527613821 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f928c00b980 con 0x7f92a409a170 2023-11-24T06:11:01.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.105+0000 7f92a942c640 1 --2- 172.21.15.92:0/3527613821 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f928c003970 0x7f928c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:01.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.105+0000 7f92a8c2b640 1 --2- 172.21.15.92:0/3527613821 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f92a409a170 0x7f928c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:01.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.105+0000 7f92a942c640 1 -- 172.21.15.92:0/3527613821 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f92a409a170 msgr2=0x7f928c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:01.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.105+0000 7f92a942c640 1 --2- 172.21.15.92:0/3527613821 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f92a409a170 0x7f928c0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:01.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.105+0000 7f92a942c640 1 -- 172.21.15.92:0/3527613821 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f928c004d00 msgr2=0x7f928c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:01.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.105+0000 7f92a942c640 1 --2- 172.21.15.92:0/3527613821 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f928c004d00 0x7f928c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:01.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.105+0000 7f92a942c640 1 -- 172.21.15.92:0/3527613821 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f928c003520 con 0x7f928c003970 2023-11-24T06:11:01.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.105+0000 7f92a8c2b640 1 --2- 172.21.15.92:0/3527613821 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f92a409a170 0x7f928c0047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:11:01.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.107+0000 7f92a942c640 1 --2- 172.21.15.92:0/3527613821 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f928c003970 0x7f928c003d90 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f929400a790 tx=0x7f928c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21c8d9cc0b82feec server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:01.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.107+0000 7f92a3fff640 1 -- 172.21.15.92:0/3527613821 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9294034630 con 0x7f928c003970 2023-11-24T06:11:01.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.107+0000 7f92a3fff640 1 -- 172.21.15.92:0/3527613821 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f929402f070 con 0x7f928c003970 2023-11-24T06:11:01.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.107+0000 7f92a3fff640 1 -- 172.21.15.92:0/3527613821 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9294034a10 con 0x7f928c003970 2023-11-24T06:11:01.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.107+0000 7f92ab690640 1 -- 172.21.15.92:0/3527613821 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f928c003970 msgr2=0x7f928c003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:01.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.108+0000 7f92ab690640 1 --2- 172.21.15.92:0/3527613821 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f928c003970 0x7f928c003d90 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f929400a790 tx=0x7f928c0057f0 comp rx=0 tx=0).stop 2023-11-24T06:11:01.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.108+0000 7f92a9c2d640 1 -- 172.21.15.92:0/3527613821 reap_dead start 2023-11-24T06:11:01.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.108+0000 7f92ab690640 1 -- 172.21.15.92:0/3527613821 shutdown_connections 2023-11-24T06:11:01.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.108+0000 7f92ab690640 1 -- 172.21.15.92:0/3527613821 >> 172.21.15.92:0/3527613821 conn(0x7f92a40b2d00 msgr2=0x7f92a40b3110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:01.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.108+0000 7f92ab690640 1 -- 172.21.15.92:0/3527613821 shutdown_connections 2023-11-24T06:11:01.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.108+0000 7f92ab690640 1 -- 172.21.15.92:0/3527613821 wait complete. 2023-11-24T06:11:01.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.110+0000 7f92ab690640 1 Processor -- start 2023-11-24T06:11:01.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.110+0000 7f92ab690640 1 -- start start 2023-11-24T06:11:01.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.111+0000 7f92ab690640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f928c003970 0x7f92a40c7770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:01.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.111+0000 7f92ab690640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f92a40b4060 0x7f92a415b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:01.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.111+0000 7f92ab690640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f92a415b6d0 0x7f92a40b5650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:01.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.111+0000 7f92ab690640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92a40b51c0 con 0x7f92a40b4060 2023-11-24T06:11:01.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.111+0000 7f92ab690640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92a41f2a70 con 0x7f928c003970 2023-11-24T06:11:01.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.111+0000 7f92ab690640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92a40b45b0 con 0x7f92a415b6d0 2023-11-24T06:11:01.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.111+0000 7f92a8c2b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f92a40b4060 0x7f92a415b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:01.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.111+0000 7f92a8c2b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f92a40b4060 0x7f92a415b0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53106/0 (socket says 172.21.15.92:53106) 2023-11-24T06:11:01.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.111+0000 7f92a8c2b640 1 -- 172.21.15.92:0/2759831370 learned_addr learned my addr 172.21.15.92:0/2759831370 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:11:01.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.111+0000 7f92a9c2d640 1 --2- 172.21.15.92:0/2759831370 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f92a415b6d0 0x7f92a40b5650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:01.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.111+0000 7f92a8c2b640 1 -- 172.21.15.92:0/2759831370 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f92a415b6d0 msgr2=0x7f92a40b5650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:01.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.111+0000 7f92a8c2b640 1 --2- 172.21.15.92:0/2759831370 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f92a415b6d0 0x7f92a40b5650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:01.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.111+0000 7f92a8c2b640 1 -- 172.21.15.92:0/2759831370 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f928c003970 msgr2=0x7f92a40c7770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:01.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.111+0000 7f92a8c2b640 1 --2- 172.21.15.92:0/2759831370 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f928c003970 0x7f92a40c7770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:01.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.111+0000 7f92a8c2b640 1 -- 172.21.15.92:0/2759831370 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92a40b4890 con 0x7f92a40b4060 2023-11-24T06:11:01.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.112+0000 7f92a8c2b640 1 --2- 172.21.15.92:0/2759831370 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f92a40b4060 0x7f92a415b0d0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f92980108b0 tx=0x7f9298010d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:01.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.112+0000 7f92a1ffb640 1 -- 172.21.15.92:0/2759831370 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f929800eb60 con 0x7f92a40b4060 2023-11-24T06:11:01.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.112+0000 7f92a1ffb640 1 -- 172.21.15.92:0/2759831370 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f929800ecc0 con 0x7f92a40b4060 2023-11-24T06:11:01.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.112+0000 7f92a1ffb640 1 -- 172.21.15.92:0/2759831370 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f929801c400 con 0x7f92a40b4060 2023-11-24T06:11:01.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.112+0000 7f92ab690640 1 -- 172.21.15.92:0/2759831370 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92a40b4b70 con 0x7f92a40b4060 2023-11-24T06:11:01.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.112+0000 7f92ab690640 1 -- 172.21.15.92:0/2759831370 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92a415de80 con 0x7f92a40b4060 2023-11-24T06:11:01.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.113+0000 7f92ab690640 1 -- 172.21.15.92:0/2759831370 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92a40675d0 con 0x7f92a40b4060 2023-11-24T06:11:01.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.114+0000 7f92a1ffb640 1 -- 172.21.15.92:0/2759831370 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f929801c560 con 0x7f92a40b4060 2023-11-24T06:11:01.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.115+0000 7f92a1ffb640 1 --2- 172.21.15.92:0/2759831370 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f929006ae40 0x7f929006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:01.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.115+0000 7f92a1ffb640 1 -- 172.21.15.92:0/2759831370 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9298089cb0 con 0x7f92a40b4060 2023-11-24T06:11:01.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.115+0000 7f92a942c640 1 --2- 172.21.15.92:0/2759831370 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f929006ae40 0x7f929006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:01.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.116+0000 7f92a942c640 1 --2- 172.21.15.92:0/2759831370 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f929006ae40 0x7f929006d300 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f9294034000 tx=0x7f9294007000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:01.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.119+0000 7f92a1ffb640 1 -- 172.21.15.92:0/2759831370 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9298054d70 con 0x7f92a40b4060 2023-11-24T06:11:01.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.308+0000 7f92ab690640 1 -- 172.21.15.92:0/2759831370 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f92a4099c00 con 0x7f92a40b4060 2023-11-24T06:11:01.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.309+0000 7f92a1ffb640 1 -- 172.21.15.92:0/2759831370 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f9298058390 con 0x7f92a40b4060 2023-11-24T06:11:01.312 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 64 2023-11-24T06:11:01.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.311+0000 7f92ab690640 1 -- 172.21.15.92:0/2759831370 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f929006ae40 msgr2=0x7f929006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:01.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.311+0000 7f92ab690640 1 --2- 172.21.15.92:0/2759831370 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f929006ae40 0x7f929006d300 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f9294034000 tx=0x7f9294007000 comp rx=0 tx=0).stop 2023-11-24T06:11:01.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.312+0000 7f92ab690640 1 -- 172.21.15.92:0/2759831370 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f92a40b4060 msgr2=0x7f92a415b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:01.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.312+0000 7f92ab690640 1 --2- 172.21.15.92:0/2759831370 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f92a40b4060 0x7f92a415b0d0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f92980108b0 tx=0x7f9298010d70 comp rx=0 tx=0).stop 2023-11-24T06:11:01.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.312+0000 7f92ab690640 1 -- 172.21.15.92:0/2759831370 shutdown_connections 2023-11-24T06:11:01.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.312+0000 7f92ab690640 1 --2- 172.21.15.92:0/2759831370 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f929006ae40 0x7f929006d300 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:01.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.312+0000 7f92ab690640 1 --2- 172.21.15.92:0/2759831370 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f92a415b6d0 0x7f92a40b5650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:01.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.312+0000 7f92ab690640 1 --2- 172.21.15.92:0/2759831370 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f92a40b4060 0x7f92a415b0d0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:01.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.312+0000 7f92ab690640 1 --2- 172.21.15.92:0/2759831370 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f928c003970 0x7f92a40c7770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:01.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.312+0000 7f92ab690640 1 -- 172.21.15.92:0/2759831370 >> 172.21.15.92:0/2759831370 conn(0x7f92a40b2d00 msgr2=0x7f92a40cdb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:01.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.312+0000 7f92ab690640 1 -- 172.21.15.92:0/2759831370 shutdown_connections 2023-11-24T06:11:01.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.312+0000 7f92ab690640 1 -- 172.21.15.92:0/2759831370 wait complete. 2023-11-24T06:11:01.333 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:11:01.538 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.535+0000 7f74c67e5640 1 Processor -- start 2023-11-24T06:11:01.539 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.535+0000 7f74c67e5640 1 -- start start 2023-11-24T06:11:01.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.537+0000 7f74c67e5640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f74c013e360 0x7f74c00cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:01.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.537+0000 7f74c67e5640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f74c015f9c0 0x7f74c00cf7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:01.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.537+0000 7f74c67e5640 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74c015c5f0 con 0x7f74c013df80 2023-11-24T06:11:01.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.537+0000 7f74c67e5640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74c006c470 con 0x7f74c013e360 2023-11-24T06:11:01.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.537+0000 7f74c67e5640 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74c006c750 con 0x7f74c015f9c0 2023-11-24T06:11:01.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.537+0000 7f74b7fff640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f74c013e360 0x7f74c00cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:01.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.537+0000 7f74b7fff640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f74c013e360 0x7f74c00cf290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53122/0 (socket says 172.21.15.92:53122) 2023-11-24T06:11:01.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.537+0000 7f74b7fff640 1 -- 172.21.15.92:0/261909246 learned_addr learned my addr 172.21.15.92:0/261909246 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:11:01.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.537+0000 7f74b7fff640 1 -- 172.21.15.92:0/261909246 >> v2:172.21.15.118:3300/0 conn(0x7f74c015f9c0 msgr2=0x7f74c00cf7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:01.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.537+0000 7f74b7fff640 1 --2- 172.21.15.92:0/261909246 >> v2:172.21.15.118:3300/0 conn(0x7f74c015f9c0 0x7f74c00cf7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:01.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.538+0000 7f74b7fff640 1 -- 172.21.15.92:0/261909246 >> v1:172.21.15.118:6789/0 conn(0x7f74c013df80 legacy=0x7f74c015bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:01.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.538+0000 7f74b7fff640 1 -- 172.21.15.92:0/261909246 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74c006ca30 con 0x7f74c013e360 2023-11-24T06:11:01.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.539+0000 7f74b7fff640 1 --2- 172.21.15.92:0/261909246 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f74c013e360 0x7f74c00cf290 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f74c006ded0 tx=0x7f74a802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71e46912cdccd590 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:01.542 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.539+0000 7f74bf7fe640 1 -- 172.21.15.92:0/261909246 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74a8031870 con 0x7f74c013e360 2023-11-24T06:11:01.542 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.539+0000 7f74bf7fe640 1 -- 172.21.15.92:0/261909246 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74a80319d0 con 0x7f74c013e360 2023-11-24T06:11:01.542 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.539+0000 7f74bf7fe640 1 -- 172.21.15.92:0/261909246 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74a8031d60 con 0x7f74c013e360 2023-11-24T06:11:01.543 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.539+0000 7f74c67e5640 1 -- 172.21.15.92:0/261909246 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f74c013e360 msgr2=0x7f74c00cf290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:01.543 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.539+0000 7f74c67e5640 1 --2- 172.21.15.92:0/261909246 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f74c013e360 0x7f74c00cf290 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f74c006ded0 tx=0x7f74a802b910 comp rx=0 tx=0).stop 2023-11-24T06:11:01.543 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.540+0000 7f74c67e5640 1 -- 172.21.15.92:0/261909246 shutdown_connections 2023-11-24T06:11:01.543 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.540+0000 7f74c67e5640 1 --2- 172.21.15.92:0/261909246 >> v2:172.21.15.118:3300/0 conn(0x7f74c015f9c0 0x7f74c00cf7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:01.543 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.540+0000 7f74c67e5640 1 --2- 172.21.15.92:0/261909246 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f74c013e360 0x7f74c00cf290 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:01.543 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.540+0000 7f74c67e5640 1 -- 172.21.15.92:0/261909246 >> 172.21.15.92:0/261909246 conn(0x7f74c006a800 msgr2=0x7f74c006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:01.543 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.540+0000 7f74c67e5640 1 -- 172.21.15.92:0/261909246 shutdown_connections 2023-11-24T06:11:01.543 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.540+0000 7f74c67e5640 1 -- 172.21.15.92:0/261909246 wait complete. 2023-11-24T06:11:01.544 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.541+0000 7f74c67e5640 1 Processor -- start 2023-11-24T06:11:01.544 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.541+0000 7f74c67e5640 1 -- start start 2023-11-24T06:11:01.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.542+0000 7f74c67e5640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f74c013df80 0x7f74c0159ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:01.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.542+0000 7f74c67e5640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f74c013e360 0x7f74c0144ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:01.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.542+0000 7f74c67e5640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f74c015f9c0 0x7f74c0143120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:01.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.542+0000 7f74c67e5640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74c0143a30 con 0x7f74c015f9c0 2023-11-24T06:11:01.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.542+0000 7f74c67e5640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74c0143d10 con 0x7f74c013e360 2023-11-24T06:11:01.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.542+0000 7f74c67e5640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74c0143ff0 con 0x7f74c013df80 2023-11-24T06:11:01.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.543+0000 7f74c4d82640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f74c015f9c0 0x7f74c0143120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:01.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.543+0000 7f74c4d82640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f74c015f9c0 0x7f74c0143120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53128/0 (socket says 172.21.15.92:53128) 2023-11-24T06:11:01.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.543+0000 7f74c4d82640 1 -- 172.21.15.92:0/926986255 learned_addr learned my addr 172.21.15.92:0/926986255 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:11:01.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.543+0000 7f74c4d82640 1 -- 172.21.15.92:0/926986255 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f74c013df80 msgr2=0x7f74c0159ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:01.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.543+0000 7f74c4d82640 1 --2- 172.21.15.92:0/926986255 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f74c013df80 0x7f74c0159ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:01.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.543+0000 7f74c4d82640 1 -- 172.21.15.92:0/926986255 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f74c013e360 msgr2=0x7f74c0144ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:01.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.543+0000 7f74c4d82640 1 --2- 172.21.15.92:0/926986255 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f74c013e360 0x7f74c0144ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:01.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.543+0000 7f74c4d82640 1 -- 172.21.15.92:0/926986255 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74a80094d0 con 0x7f74c015f9c0 2023-11-24T06:11:01.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.543+0000 7f74c4d82640 1 --2- 172.21.15.92:0/926986255 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f74c015f9c0 0x7f74c0143120 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f74b800b9e0 tx=0x7f74b800bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:01.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.544+0000 7f74bd7fa640 1 -- 172.21.15.92:0/926986255 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74b800cb20 con 0x7f74c015f9c0 2023-11-24T06:11:01.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.544+0000 7f74bd7fa640 1 -- 172.21.15.92:0/926986255 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74b800cc80 con 0x7f74c015f9c0 2023-11-24T06:11:01.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.544+0000 7f74c67e5640 1 -- 172.21.15.92:0/926986255 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74c01442d0 con 0x7f74c015f9c0 2023-11-24T06:11:01.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.544+0000 7f74bd7fa640 1 -- 172.21.15.92:0/926986255 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74b8014680 con 0x7f74c015f9c0 2023-11-24T06:11:01.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.544+0000 7f74c67e5640 1 -- 172.21.15.92:0/926986255 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74c0143660 con 0x7f74c015f9c0 2023-11-24T06:11:01.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.545+0000 7f74c67e5640 1 -- 172.21.15.92:0/926986255 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74c01447f0 con 0x7f74c015f9c0 2023-11-24T06:11:01.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.546+0000 7f74bd7fa640 1 -- 172.21.15.92:0/926986255 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f74b801d3b0 con 0x7f74c015f9c0 2023-11-24T06:11:01.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.546+0000 7f74bd7fa640 1 --2- 172.21.15.92:0/926986255 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f74a006ae40 0x7f74a006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:01.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.547+0000 7f74bd7fa640 1 -- 172.21.15.92:0/926986255 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f74b8089570 con 0x7f74c015f9c0 2023-11-24T06:11:01.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.547+0000 7f74bffff640 1 --2- 172.21.15.92:0/926986255 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f74a006ae40 0x7f74a006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:01.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.547+0000 7f74bffff640 1 --2- 172.21.15.92:0/926986255 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f74a006ae40 0x7f74a006d300 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f74b0009650 tx=0x7f74b0006ba0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:01.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.551+0000 7f74bd7fa640 1 -- 172.21.15.92:0/926986255 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f74c01447f0 con 0x7f74c015f9c0 2023-11-24T06:11:01.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.739+0000 7f74c67e5640 1 -- 172.21.15.92:0/926986255 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f74c01447f0 con 0x7f74c015f9c0 2023-11-24T06:11:01.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.742+0000 7f74bd7fa640 1 -- 172.21.15.92:0/926986255 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16457 (secure 0 0 0) 0x7f74c01447f0 con 0x7f74c015f9c0 2023-11-24T06:11:01.745 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-24T06:11:01.745 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":20,"fsid":"2d6dee5f-8aa3-470f-931a-50ade963b1cb","created":"2023-11-24T06:10:04.707204+0000","modified":"2023-11-24T06:10:26.698733+0000","last_up_change":"2023-11-24T06:10:17.598352+0000","last_in_change":"2023-11-24T06:10:09.040793+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:10:17.733098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:10:22.926445+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T06:10:24.151288+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ff37a9fb-135f-471f-b4b0-f7b5772006c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6803","nonce":20560714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6805","nonce":20560714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6809","nonce":20560714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6807","nonce":20560714}]},"public_addr":"172.21.15.92:6803/20560714","cluster_addr":"172.21.15.92:6805/20560714","heartbeat_back_addr":"172.21.15.92:6809/20560714","heartbeat_front_addr":"172.21.15.92:6807/20560714","state":["exists","up"]},{"osd":1,"uuid":"03cdd353-dcf1-4f38-94fe-a795137c06b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6819","nonce":2451239906}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6821","nonce":2451239906}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6825","nonce":2451239906}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6823","nonce":2451239906}]},"public_addr":"172.21.15.92:6819/2451239906","cluster_addr":"172.21.15.92:6821/2451239906","heartbeat_back_addr":"172.21.15.92:6825/2451239906","heartbeat_front_addr":"172.21.15.92:6823/2451239906","state":["exists","up"]},{"osd":2,"uuid":"a4a20d80-5d57-484b-becd-9aa8070dd3aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6811","nonce":1124979277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6813","nonce":1124979277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6817","nonce":1124979277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6815","nonce":1124979277}]},"public_addr":"172.21.15.92:6811/1124979277","cluster_addr":"172.21.15.92:6813/1124979277","heartbeat_back_addr":"172.21.15.92:6817/1124979277","heartbeat_front_addr":"172.21.15.92:6815/1124979277","state":["exists","up"]},{"osd":3,"uuid":"4648d7ce-9b29-4583-9425-4ba8fd2b1903","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6827","nonce":1734115782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6829","nonce":1734115782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6833","nonce":1734115782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6831","nonce":1734115782}]},"public_addr":"172.21.15.92:6827/1734115782","cluster_addr":"172.21.15.92:6829/1734115782","heartbeat_back_addr":"172.21.15.92:6833/1734115782","heartbeat_front_addr":"172.21.15.92:6831/1734115782","state":["exists","up"]},{"osd":4,"uuid":"3bc9d7eb-c6fa-4fcf-a3ec-ade604e05aac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6819","nonce":3425937229}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6822","nonce":3425937229}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6830","nonce":3425937229}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6826","nonce":3425937229}]},"public_addr":"172.21.15.118:6819/3425937229","cluster_addr":"172.21.15.118:6822/3425937229","heartbeat_back_addr":"172.21.15.118:6830/3425937229","heartbeat_front_addr":"172.21.15.118:6826/3425937229","state":["exists","up"]},{"osd":5,"uuid":"dce9ba6d-59ed-46d1-84fc-a84b13de10c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6801","nonce":827455984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6803","nonce":827455984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6807","nonce":827455984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6805","nonce":827455984}]},"public_addr":"172.21.15.118:6801/827455984","cluster_addr":"172.21.15.118:6803/827455984","heartbeat_back_addr":"172.21.15.118:6807/827455984","heartbeat_front_addr":"172.21.15.118:6805/827455984","state":["exists","up"]},{"osd":6,"uuid":"82aec504-ae58-4709-8d30-46f2b12081b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6809","nonce":2946986141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6811","nonce":2946986141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6815","nonce":2946986141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6813","nonce":2946986141}]},"public_addr":"172.21.15.118:6809/2946986141","cluster_addr":"172.21.15.118:6811/2946986141","heartbeat_back_addr":"172.21.15.118:6815/2946986141","heartbeat_front_addr":"172.21.15.118:6813/2946986141","state":["exists","up"]},{"osd":7,"uuid":"4ed4c624-c88f-4d35-bdb1-bf209b5dffb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6817","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6818","nonce":48122059}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6821","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6823","nonce":48122059}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6829","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6831","nonce":48122059}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6825","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6827","nonce":48122059}]},"public_addr":"172.21.15.118:6818/48122059","cluster_addr":"172.21.15.118:6823/48122059","heartbeat_back_addr":"172.21.15.118:6831/48122059","heartbeat_front_addr":"172.21.15.118:6827/48122059","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.536664+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.368324+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.271201+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.342971+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.621832+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.387110+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.632946+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.416032+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:11:01.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.744+0000 7f74c67e5640 1 -- 172.21.15.92:0/926986255 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f74a006ae40 msgr2=0x7f74a006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:01.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.744+0000 7f74c67e5640 1 --2- 172.21.15.92:0/926986255 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f74a006ae40 0x7f74a006d300 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f74b0009650 tx=0x7f74b0006ba0 comp rx=0 tx=0).stop 2023-11-24T06:11:01.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.744+0000 7f74c67e5640 1 -- 172.21.15.92:0/926986255 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f74c015f9c0 msgr2=0x7f74c0143120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:01.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.744+0000 7f74c67e5640 1 --2- 172.21.15.92:0/926986255 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f74c015f9c0 0x7f74c0143120 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f74b800b9e0 tx=0x7f74b800bea0 comp rx=0 tx=0).stop 2023-11-24T06:11:01.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.744+0000 7f74c67e5640 1 -- 172.21.15.92:0/926986255 shutdown_connections 2023-11-24T06:11:01.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.744+0000 7f74c67e5640 1 --2- 172.21.15.92:0/926986255 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f74a006ae40 0x7f74a006d300 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:01.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.744+0000 7f74c67e5640 1 --2- 172.21.15.92:0/926986255 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f74c015f9c0 0x7f74c0143120 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:01.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.744+0000 7f74c67e5640 1 --2- 172.21.15.92:0/926986255 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f74c013e360 0x7f74c0144ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:01.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.744+0000 7f74c67e5640 1 --2- 172.21.15.92:0/926986255 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f74c013df80 0x7f74c0159ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:01.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.744+0000 7f74c67e5640 1 -- 172.21.15.92:0/926986255 >> 172.21.15.92:0/926986255 conn(0x7f74c006a800 msgr2=0x7f74c00cf5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:01.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.745+0000 7f74c67e5640 1 -- 172.21.15.92:0/926986255 shutdown_connections 2023-11-24T06:11:01.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.745+0000 7f74c67e5640 1 -- 172.21.15.92:0/926986255 wait complete. 2023-11-24T06:11:01.765 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T06:10:17.733098+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T06:10:22.926445+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T06:10:24.151288+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T06:11:01.766 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T06:11:01.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.965+0000 7f862cebb640 1 Processor -- start 2023-11-24T06:11:01.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.965+0000 7f862cebb640 1 -- start start 2023-11-24T06:11:01.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.966+0000 7f862cebb640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f86280d17e0 0x7f86280d1c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:01.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.967+0000 7f862cebb640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f86280b3890 0x7f86280b3d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:01.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.967+0000 7f862cebb640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86280d2180 con 0x7f862815d610 2023-11-24T06:11:01.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.967+0000 7f862cebb640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86280d1390 con 0x7f86280d17e0 2023-11-24T06:11:01.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.967+0000 7f862cebb640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86280b3400 con 0x7f86280b3890 2023-11-24T06:11:01.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.967+0000 7f862659c640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f862815d610 0x7f86280ce820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:35516/0 (socket says 172.21.15.92:35516) 2023-11-24T06:11:01.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.967+0000 7f862659c640 1 -- 172.21.15.92:0/220546297 learned_addr learned my addr 172.21.15.92:0/220546297 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-24T06:11:01.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.967+0000 7f8625d9b640 1 --2- 172.21.15.92:0/220546297 >> v2:172.21.15.92:3300/0 conn(0x7f86280d17e0 0x7f86280d1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:01.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.967+0000 7f862559a640 1 -- 172.21.15.92:0/220546297 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 893585680 0 0) 0x7f86280d2180 con 0x7f862815d610 2023-11-24T06:11:01.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.967+0000 7f8626d9d640 1 --2- 172.21.15.92:0/220546297 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f86280b3890 0x7f86280b3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:01.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.967+0000 7f862559a640 1 -- 172.21.15.92:0/220546297 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8610003520 con 0x7f862815d610 2023-11-24T06:11:01.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.967+0000 7f8625d9b640 1 -- 172.21.15.92:0/220546297 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f86280b3890 msgr2=0x7f86280b3d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:01.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.967+0000 7f8625d9b640 1 --2- 172.21.15.92:0/220546297 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f86280b3890 0x7f86280b3d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:01.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.967+0000 7f8625d9b640 1 -- 172.21.15.92:0/220546297 >> v1:172.21.15.92:6789/0 conn(0x7f862815d610 legacy=0x7f86280ce820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:01.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.967+0000 7f8625d9b640 1 -- 172.21.15.92:0/220546297 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86280b27f0 con 0x7f86280d17e0 2023-11-24T06:11:01.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.968+0000 7f8625d9b640 1 --2- 172.21.15.92:0/220546297 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f86280d17e0 0x7f86280d1c40 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f861c0097a0 tx=0x7f861c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f87abf2923514f83 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:01.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.969+0000 7f862559a640 1 -- 172.21.15.92:0/220546297 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f861c031830 con 0x7f86280d17e0 2023-11-24T06:11:01.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.969+0000 7f862559a640 1 -- 172.21.15.92:0/220546297 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f861c031990 con 0x7f86280d17e0 2023-11-24T06:11:01.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.969+0000 7f862559a640 1 -- 172.21.15.92:0/220546297 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f861c031d00 con 0x7f86280d17e0 2023-11-24T06:11:01.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.969+0000 7f862cebb640 1 -- 172.21.15.92:0/220546297 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f86280d17e0 msgr2=0x7f86280d1c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:01.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.969+0000 7f862cebb640 1 --2- 172.21.15.92:0/220546297 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f86280d17e0 0x7f86280d1c40 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f861c0097a0 tx=0x7f861c02b8d0 comp rx=0 tx=0).stop 2023-11-24T06:11:01.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.969+0000 7f862cebb640 1 -- 172.21.15.92:0/220546297 shutdown_connections 2023-11-24T06:11:01.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.969+0000 7f862cebb640 1 --2- 172.21.15.92:0/220546297 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f86280b3890 0x7f86280b3d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:01.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.969+0000 7f862cebb640 1 --2- 172.21.15.92:0/220546297 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f86280d17e0 0x7f86280d1c40 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:01.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.969+0000 7f862cebb640 1 -- 172.21.15.92:0/220546297 >> 172.21.15.92:0/220546297 conn(0x7f862809c540 msgr2=0x7f86280d3200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:01.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.969+0000 7f862cebb640 1 -- 172.21.15.92:0/220546297 shutdown_connections 2023-11-24T06:11:01.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.969+0000 7f862cebb640 1 -- 172.21.15.92:0/220546297 wait complete. 2023-11-24T06:11:01.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.971+0000 7f862cebb640 1 Processor -- start 2023-11-24T06:11:01.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.971+0000 7f862cebb640 1 -- start start 2023-11-24T06:11:01.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.972+0000 7f862cebb640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f86280b3890 0x7f86281f6d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:01.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.972+0000 7f862cebb640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f86280d17e0 0x7f86280b14c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:01.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.972+0000 7f862cebb640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f862815d610 0x7f862815b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:01.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.972+0000 7f862cebb640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86281f7270 con 0x7f86280b3890 2023-11-24T06:11:01.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.972+0000 7f862cebb640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86280cb210 con 0x7f862815d610 2023-11-24T06:11:01.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.972+0000 7f862cebb640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86280ca690 con 0x7f86280d17e0 2023-11-24T06:11:01.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.972+0000 7f862659c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f86280b3890 0x7f86281f6d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:01.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.972+0000 7f862659c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f86280b3890 0x7f86281f6d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53148/0 (socket says 172.21.15.92:53148) 2023-11-24T06:11:01.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.972+0000 7f862659c640 1 -- 172.21.15.92:0/483467258 learned_addr learned my addr 172.21.15.92:0/483467258 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:11:01.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.973+0000 7f862659c640 1 -- 172.21.15.92:0/483467258 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f86280d17e0 msgr2=0x7f86280b14c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:01.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.973+0000 7f862659c640 1 --2- 172.21.15.92:0/483467258 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f86280d17e0 0x7f86280b14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:01.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.973+0000 7f862659c640 1 -- 172.21.15.92:0/483467258 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f862815d610 msgr2=0x7f862815b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:01.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.973+0000 7f862659c640 1 --2- 172.21.15.92:0/483467258 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f862815d610 0x7f862815b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:01.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.973+0000 7f862659c640 1 -- 172.21.15.92:0/483467258 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f861c009400 con 0x7f86280b3890 2023-11-24T06:11:01.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.973+0000 7f862659c640 1 --2- 172.21.15.92:0/483467258 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f86280b3890 0x7f86281f6d30 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f861800c390 tx=0x7f861800c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:01.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.973+0000 7f8616ffd640 1 -- 172.21.15.92:0/483467258 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8618005ce0 con 0x7f86280b3890 2023-11-24T06:11:01.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.973+0000 7f8616ffd640 1 -- 172.21.15.92:0/483467258 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8618005e40 con 0x7f86280b3890 2023-11-24T06:11:01.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.973+0000 7f8616ffd640 1 -- 172.21.15.92:0/483467258 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86180072f0 con 0x7f86280b3890 2023-11-24T06:11:01.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.973+0000 7f862cebb640 1 -- 172.21.15.92:0/483467258 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86280ca970 con 0x7f86280b3890 2023-11-24T06:11:01.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.973+0000 7f862cebb640 1 -- 172.21.15.92:0/483467258 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86280caf30 con 0x7f86280b3890 2023-11-24T06:11:01.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.974+0000 7f862cebb640 1 -- 172.21.15.92:0/483467258 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86280b60e0 con 0x7f86280b3890 2023-11-24T06:11:01.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.975+0000 7f8616ffd640 1 -- 172.21.15.92:0/483467258 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f8618007450 con 0x7f86280b3890 2023-11-24T06:11:01.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.977+0000 7f8616ffd640 1 --2- 172.21.15.92:0/483467258 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f860806ae40 0x7f860806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:01.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.977+0000 7f8616ffd640 1 -- 172.21.15.92:0/483467258 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8618088be0 con 0x7f86280b3890 2023-11-24T06:11:01.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.977+0000 7f8625d9b640 1 --2- 172.21.15.92:0/483467258 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f860806ae40 0x7f860806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:01.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.978+0000 7f8625d9b640 1 --2- 172.21.15.92:0/483467258 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f860806ae40 0x7f860806d300 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f861c036000 tx=0x7f861c030000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:01.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:01.981+0000 7f8616ffd640 1 -- 172.21.15.92:0/483467258 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f86280b60e0 con 0x7f86280b3890 2023-11-24T06:11:02.158 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:02.155+0000 7f862cebb640 1 -- 172.21.15.92:0/483467258 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f86280b60e0 con 0x7f86280b3890 2023-11-24T06:11:02.160 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:02.156+0000 7f8616ffd640 1 -- 172.21.15.92:0/483467258 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8618017020 con 0x7f86280b3890 2023-11-24T06:11:02.160 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 1 2023-11-24T06:11:02.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:02.158+0000 7f862cebb640 1 -- 172.21.15.92:0/483467258 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f860806ae40 msgr2=0x7f860806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:02.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:02.159+0000 7f862cebb640 1 --2- 172.21.15.92:0/483467258 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f860806ae40 0x7f860806d300 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f861c036000 tx=0x7f861c030000 comp rx=0 tx=0).stop 2023-11-24T06:11:02.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:02.159+0000 7f862cebb640 1 -- 172.21.15.92:0/483467258 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f86280b3890 msgr2=0x7f86281f6d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:02.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:02.159+0000 7f862cebb640 1 --2- 172.21.15.92:0/483467258 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f86280b3890 0x7f86281f6d30 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f861800c390 tx=0x7f861800c850 comp rx=0 tx=0).stop 2023-11-24T06:11:02.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:02.159+0000 7f862cebb640 1 -- 172.21.15.92:0/483467258 shutdown_connections 2023-11-24T06:11:02.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:02.159+0000 7f862cebb640 1 --2- 172.21.15.92:0/483467258 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f860806ae40 0x7f860806d300 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:02.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:02.159+0000 7f862cebb640 1 --2- 172.21.15.92:0/483467258 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f862815d610 0x7f862815b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:02.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:02.159+0000 7f862cebb640 1 --2- 172.21.15.92:0/483467258 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f86280d17e0 0x7f86280b14c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:02.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:02.159+0000 7f862cebb640 1 --2- 172.21.15.92:0/483467258 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f86280b3890 0x7f86281f6d30 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:02.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:02.159+0000 7f862cebb640 1 -- 172.21.15.92:0/483467258 >> 172.21.15.92:0/483467258 conn(0x7f862809c540 msgr2=0x7f862815c900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:02.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:02.159+0000 7f862cebb640 1 -- 172.21.15.92:0/483467258 shutdown_connections 2023-11-24T06:11:02.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:02.159+0000 7f862cebb640 1 -- 172.21.15.92:0/483467258 wait complete. 2023-11-24T06:11:02.181 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T06:11:02.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:02.393+0000 7f57cfb01640 1 Processor -- start 2023-11-24T06:11:02.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:02.393+0000 7f57cfb01640 1 -- start start 2023-11-24T06:11:02.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:02.395+0000 7f57cfb01640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f57c813e360 0x7f57c80cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:02.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:02.395+0000 7f57cfb01640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57c815c5f0 con 0x7f57c813df80 2023-11-24T06:11:02.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:02.395+0000 7f57cfb01640 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57c806c470 con 0x7f57c815f9c0 2023-11-24T06:11:02.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:02.395+0000 7f57cfb01640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57c806c750 con 0x7f57c813e360 2023-11-24T06:11:02.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:02.395+0000 7f57cd89d640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f57c813df80 0x7f57c815bee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:35530/0 (socket says 172.21.15.92:35530) 2023-11-24T06:11:02.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:02.395+0000 7f57cd89d640 1 -- 172.21.15.92:0/2184853477 learned_addr learned my addr 172.21.15.92:0/2184853477 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-24T06:11:02.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:02.395+0000 7f57cd09c640 1 --2- 172.21.15.92:0/2184853477 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f57c813e360 0x7f57c80cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:02.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:02.395+0000 7f57cc89b640 1 -- 172.21.15.92:0/2184853477 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1910921187 0 0) 0x7f57c815c5f0 con 0x7f57c813df80 2023-11-24T06:11:02.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:02.395+0000 7f57cc89b640 1 -- 172.21.15.92:0/2184853477 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f57b4003520 con 0x7f57c813df80 2023-11-24T06:11:02.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:02.396+0000 7f57cc89b640 1 -- 172.21.15.92:0/2184853477 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3707239241 0 0) 0x7f57b4003520 con 0x7f57c813df80 2023-11-24T06:11:02.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:02.396+0000 7f57cc89b640 1 -- 172.21.15.92:0/2184853477 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f57c813e360 msgr2=0x7f57c80cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:02.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:02.396+0000 7f57cc89b640 1 --2- 172.21.15.92:0/2184853477 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f57c813e360 0x7f57c80cf290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:02.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:02.396+0000 7f57cc89b640 1 -- 172.21.15.92:0/2184853477 >> v1:172.21.15.118:6789/0 conn(0x7f57c815f9c0 legacy=0x7f57c80cd840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:02.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:02.396+0000 7f57cc89b640 1 -- 172.21.15.92:0/2184853477 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57c806ca30 con 0x7f57c813df80 2023-11-24T06:11:02.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:02.396+0000 7f57cc89b640 1 -- 172.21.15.92:0/2184853477 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2607661350 0 0) 0x7f57c4001670 con 0x7f57c813df80 2023-11-24T06:11:02.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:02.396+0000 7f57cc89b640 1 -- 172.21.15.92:0/2184853477 >> v1:172.21.15.92:6789/0 conn(0x7f57c813df80 legacy=0x7f57c815bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:02.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:02.396+0000 7f57cc89b640 1 --2- 172.21.15.92:0/2184853477 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f57c813e360 0x7f57b40039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:02.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:02.396+0000 7f57cc89b640 1 --2- 172.21.15.92:0/2184853477 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f57b4004020 0x7f57b40047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:02.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:02.396+0000 7f57cc89b640 1 --2- 172.21.15.92:0/2184853477 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f57b4004d00 0x7f57b4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:02.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:02.396+0000 7f57cc89b640 1 -- 172.21.15.92:0/2184853477 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57c806c750 con 0x7f57b4004d00 2023-11-24T06:11:02.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:02.396+0000 7f57cc89b640 1 -- 172.21.15.92:0/2184853477 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57b400bbc0 con 0x7f57b4004020 2023-11-24T06:11:02.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:02.396+0000 7f57cc89b640 1 -- 172.21.15.92:0/2184853477 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57b400bd80 con 0x7f57c813e360 2023-11-24T06:11:02.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:02.396+0000 7f57ce09e640 1 --2- 172.21.15.92:0/2184853477 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f57b4004d00 0x7f57b4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:02.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:02.396+0000 7f57cd89d640 1 --2- 172.21.15.92:0/2184853477 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f57c813e360 0x7f57b40039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:02.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:02.397+0000 7f57cd09c640 1 --2- 172.21.15.92:0/2184853477 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f57b4004020 0x7f57b40047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:02.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:02.397+0000 7f57ce09e640 1 -- 172.21.15.92:0/2184853477 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f57c813e360 msgr2=0x7f57b40039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:02.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:02.397+0000 7f57ce09e640 1 --2- 172.21.15.92:0/2184853477 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f57c813e360 0x7f57b40039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:02.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:02.397+0000 7f57ce09e640 1 -- 172.21.15.92:0/2184853477 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f57b4004020 msgr2=0x7f57b40047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:02.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:02.397+0000 7f57ce09e640 1 --2- 172.21.15.92:0/2184853477 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f57b4004020 0x7f57b40047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:02.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:02.397+0000 7f57ce09e640 1 -- 172.21.15.92:0/2184853477 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57b4003520 con 0x7f57b4004d00 2023-11-24T06:11:02.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:02.398+0000 7f57ce09e640 1 --2- 172.21.15.92:0/2184853477 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f57b4004d00 0x7f57b4005180 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f57c0001e50 tx=0x7f57b40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=364fd4730ccafd40 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:02.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:02.398+0000 7f57cc89b640 1 -- 172.21.15.92:0/2184853477 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57c0003b70 con 0x7f57b4004d00 2023-11-24T06:11:02.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:02.398+0000 7f57cc89b640 1 -- 172.21.15.92:0/2184853477 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57c0003cd0 con 0x7f57b4004d00 2023-11-24T06:11:02.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:02.398+0000 7f57cc89b640 1 -- 172.21.15.92:0/2184853477 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57c0005310 con 0x7f57b4004d00 2023-11-24T06:11:02.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:02.398+0000 7f57cfb01640 1 -- 172.21.15.92:0/2184853477 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f57b4004d00 msgr2=0x7f57b4005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:02.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:02.398+0000 7f57cfb01640 1 --2- 172.21.15.92:0/2184853477 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f57b4004d00 0x7f57b4005180 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f57c0001e50 tx=0x7f57b40057f0 comp rx=0 tx=0).stop 2023-11-24T06:11:02.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:02.398+0000 7f57ce09e640 1 -- 172.21.15.92:0/2184853477 reap_dead start 2023-11-24T06:11:02.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:02.399+0000 7f57cfb01640 1 -- 172.21.15.92:0/2184853477 shutdown_connections 2023-11-24T06:11:02.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:02.399+0000 7f57cfb01640 1 -- 172.21.15.92:0/2184853477 >> 172.21.15.92:0/2184853477 conn(0x7f57c806a800 msgr2=0x7f57c806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:02.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:02.399+0000 7f57cfb01640 1 -- 172.21.15.92:0/2184853477 shutdown_connections 2023-11-24T06:11:02.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:02.399+0000 7f57cfb01640 1 -- 172.21.15.92:0/2184853477 wait complete. 2023-11-24T06:11:02.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:02.400+0000 7f57cfb01640 1 Processor -- start 2023-11-24T06:11:02.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:02.401+0000 7f57cfb01640 1 -- start start 2023-11-24T06:11:02.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:02.402+0000 7f57cfb01640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f57b4004d00 0x7f57c80b2f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:02.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:02.402+0000 7f57cfb01640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f57c813cf50 0x7f57c813d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:02.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:02.402+0000 7f57cfb01640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f57c80ac270 0x7f57c80ac700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:02.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:02.402+0000 7f57cfb01640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57c80aa1b0 con 0x7f57b4004d00 2023-11-24T06:11:02.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:02.402+0000 7f57cfb01640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57c80ab460 con 0x7f57c80ac270 2023-11-24T06:11:02.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:02.402+0000 7f57cfb01640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57c813d910 con 0x7f57c813cf50 2023-11-24T06:11:02.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:02.402+0000 7f57cd89d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f57b4004d00 0x7f57c80b2f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:02.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:02.402+0000 7f57cd89d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f57b4004d00 0x7f57c80b2f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53154/0 (socket says 172.21.15.92:53154) 2023-11-24T06:11:02.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:02.402+0000 7f57cd89d640 1 -- 172.21.15.92:0/4136725615 learned_addr learned my addr 172.21.15.92:0/4136725615 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:11:02.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:02.402+0000 7f57cd89d640 1 -- 172.21.15.92:0/4136725615 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f57c813cf50 msgr2=0x7f57c813d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:02.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:02.402+0000 7f57cd89d640 1 --2- 172.21.15.92:0/4136725615 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f57c813cf50 0x7f57c813d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:02.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:02.402+0000 7f57cd89d640 1 -- 172.21.15.92:0/4136725615 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f57c80ac270 msgr2=0x7f57c80ac700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:02.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:02.402+0000 7f57cd89d640 1 --2- 172.21.15.92:0/4136725615 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f57c80ac270 0x7f57c80ac700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:02.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:02.402+0000 7f57cd89d640 1 -- 172.21.15.92:0/4136725615 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57c80aa3c0 con 0x7f57b4004d00 2023-11-24T06:11:02.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:02.402+0000 7f57cd89d640 1 --2- 172.21.15.92:0/4136725615 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f57b4004d00 0x7f57c80b2f60 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f57c400e7e0 tx=0x7f57c400eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:02.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:02.403+0000 7f57be7fc640 1 -- 172.21.15.92:0/4136725615 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57c400c850 con 0x7f57b4004d00 2023-11-24T06:11:02.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:02.403+0000 7f57be7fc640 1 -- 172.21.15.92:0/4136725615 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57c400c9b0 con 0x7f57b4004d00 2023-11-24T06:11:02.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:02.403+0000 7f57be7fc640 1 -- 172.21.15.92:0/4136725615 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57c400cd60 con 0x7f57b4004d00 2023-11-24T06:11:02.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:02.403+0000 7f57cfb01640 1 -- 172.21.15.92:0/4136725615 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57c80aa6a0 con 0x7f57b4004d00 2023-11-24T06:11:02.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:02.403+0000 7f57cfb01640 1 -- 172.21.15.92:0/4136725615 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57c80aac60 con 0x7f57b4004d00 2023-11-24T06:11:02.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:02.404+0000 7f57cfb01640 1 -- 172.21.15.92:0/4136725615 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57c80675d0 con 0x7f57b4004d00 2023-11-24T06:11:02.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:02.405+0000 7f57be7fc640 1 -- 172.21.15.92:0/4136725615 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f57c401d460 con 0x7f57b4004d00 2023-11-24T06:11:02.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:02.406+0000 7f57be7fc640 1 --2- 172.21.15.92:0/4136725615 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f57ac06ae40 0x7f57ac06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:02.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:02.406+0000 7f57be7fc640 1 -- 172.21.15.92:0/4136725615 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f57c40896b0 con 0x7f57b4004d00 2023-11-24T06:11:02.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:02.406+0000 7f57cd09c640 1 --2- 172.21.15.92:0/4136725615 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f57ac06ae40 0x7f57ac06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:02.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:02.407+0000 7f57cd09c640 1 --2- 172.21.15.92:0/4136725615 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f57ac06ae40 0x7f57ac06d300 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f57b8006fd0 tx=0x7f57b800b2e0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:02.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:02.410+0000 7f57be7fc640 1 -- 172.21.15.92:0/4136725615 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f57c40546c0 con 0x7f57b4004d00 2023-11-24T06:11:02.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:02.600+0000 7f57cfb01640 1 -- 172.21.15.92:0/4136725615 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f57c806df00 con 0x7f57b4004d00 2023-11-24T06:11:02.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:02.601+0000 7f57be7fc640 1 -- 172.21.15.92:0/4136725615 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f57c40015a0 con 0x7f57b4004d00 2023-11-24T06:11:02.604 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 32 2023-11-24T06:11:02.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:02.604+0000 7f57cfb01640 1 -- 172.21.15.92:0/4136725615 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f57ac06ae40 msgr2=0x7f57ac06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:02.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:02.604+0000 7f57cfb01640 1 --2- 172.21.15.92:0/4136725615 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f57ac06ae40 0x7f57ac06d300 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f57b8006fd0 tx=0x7f57b800b2e0 comp rx=0 tx=0).stop 2023-11-24T06:11:02.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:02.604+0000 7f57cfb01640 1 -- 172.21.15.92:0/4136725615 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f57b4004d00 msgr2=0x7f57c80b2f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:02.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:02.604+0000 7f57cfb01640 1 --2- 172.21.15.92:0/4136725615 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f57b4004d00 0x7f57c80b2f60 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f57c400e7e0 tx=0x7f57c400eca0 comp rx=0 tx=0).stop 2023-11-24T06:11:02.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:02.604+0000 7f57cfb01640 1 -- 172.21.15.92:0/4136725615 shutdown_connections 2023-11-24T06:11:02.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:02.604+0000 7f57cfb01640 1 --2- 172.21.15.92:0/4136725615 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f57ac06ae40 0x7f57ac06d300 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:02.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:02.604+0000 7f57cfb01640 1 --2- 172.21.15.92:0/4136725615 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f57c80ac270 0x7f57c80ac700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:02.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:02.604+0000 7f57cfb01640 1 --2- 172.21.15.92:0/4136725615 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f57c813cf50 0x7f57c813d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:02.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:02.604+0000 7f57cfb01640 1 --2- 172.21.15.92:0/4136725615 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f57b4004d00 0x7f57c80b2f60 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:02.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:02.604+0000 7f57cfb01640 1 -- 172.21.15.92:0/4136725615 >> 172.21.15.92:0/4136725615 conn(0x7f57c806a800 msgr2=0x7f57c80af040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:02.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:02.604+0000 7f57cfb01640 1 -- 172.21.15.92:0/4136725615 shutdown_connections 2023-11-24T06:11:02.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:02.604+0000 7f57cfb01640 1 -- 172.21.15.92:0/4136725615 wait complete. 2023-11-24T06:11:02.626 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T06:11:02.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:02.825+0000 7fe76f9ce640 1 Processor -- start 2023-11-24T06:11:02.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:02.825+0000 7fe76f9ce640 1 -- start start 2023-11-24T06:11:02.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:02.826+0000 7fe76f9ce640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe768143ae0 0x7fe7680cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:02.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:02.826+0000 7fe76f9ce640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fe7680cf7d0 0x7fe76815dca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:02.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:02.826+0000 7fe76f9ce640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7680cfc30 con 0x7fe76815e1e0 2023-11-24T06:11:02.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:02.826+0000 7fe76f9ce640 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7680b1ee0 con 0x7fe7680cf7d0 2023-11-24T06:11:02.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:02.826+0000 7fe76f9ce640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7680b2780 con 0x7fe768143ae0 2023-11-24T06:11:02.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:02.827+0000 7fe76df6b640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fe76815e1e0 0x7fe76815a670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:35544/0 (socket says 172.21.15.92:35544) 2023-11-24T06:11:02.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:02.827+0000 7fe76df6b640 1 -- 172.21.15.92:0/1373200166 learned_addr learned my addr 172.21.15.92:0/1373200166 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-24T06:11:02.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:02.827+0000 7fe76d76a640 1 --2- 172.21.15.92:0/1373200166 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe768143ae0 0x7fe7680cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:02.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:02.827+0000 7fe76cf69640 1 --2- 172.21.15.92:0/1373200166 >> v2:172.21.15.118:3300/0 conn(0x7fe7680cf7d0 0x7fe76815dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:02.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:02.827+0000 7fe767fff640 1 -- 172.21.15.92:0/1373200166 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3533021074 0 0) 0x7fe7680cfc30 con 0x7fe76815e1e0 2023-11-24T06:11:02.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:02.827+0000 7fe767fff640 1 -- 172.21.15.92:0/1373200166 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe758003520 con 0x7fe76815e1e0 2023-11-24T06:11:02.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:02.828+0000 7fe767fff640 1 -- 172.21.15.92:0/1373200166 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3604909442 0 0) 0x7fe758003520 con 0x7fe76815e1e0 2023-11-24T06:11:02.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:02.828+0000 7fe767fff640 1 -- 172.21.15.92:0/1373200166 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe768143ae0 msgr2=0x7fe7680cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:02.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:02.828+0000 7fe767fff640 1 --2- 172.21.15.92:0/1373200166 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe768143ae0 0x7fe7680cf290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:02.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:02.828+0000 7fe767fff640 1 -- 172.21.15.92:0/1373200166 >> v2:172.21.15.118:3300/0 conn(0x7fe7680cf7d0 msgr2=0x7fe76815dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:02.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:02.828+0000 7fe767fff640 1 --2- 172.21.15.92:0/1373200166 >> v2:172.21.15.118:3300/0 conn(0x7fe7680cf7d0 0x7fe76815dca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:02.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:02.828+0000 7fe767fff640 1 -- 172.21.15.92:0/1373200166 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe76815e990 con 0x7fe76815e1e0 2023-11-24T06:11:02.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:02.828+0000 7fe767fff640 1 -- 172.21.15.92:0/1373200166 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2607661350 0 0) 0x7fe760001670 con 0x7fe76815e1e0 2023-11-24T06:11:02.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:02.828+0000 7fe767fff640 1 -- 172.21.15.92:0/1373200166 >> v1:172.21.15.92:6789/0 conn(0x7fe76815e1e0 legacy=0x7fe76815a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:02.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:02.828+0000 7fe767fff640 1 --2- 172.21.15.92:0/1373200166 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe758003970 0x7fe758003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:02.839 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:02.828+0000 7fe767fff640 1 --2- 172.21.15.92:0/1373200166 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe768143ae0 0x7fe7580047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:02.839 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:02.828+0000 7fe767fff640 1 --2- 172.21.15.92:0/1373200166 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe758004d00 0x7fe758005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:02.839 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:02.828+0000 7fe767fff640 1 -- 172.21.15.92:0/1373200166 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7680b1ee0 con 0x7fe758004d00 2023-11-24T06:11:02.839 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:02.828+0000 7fe767fff640 1 -- 172.21.15.92:0/1373200166 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7680b2780 con 0x7fe758003970 2023-11-24T06:11:02.839 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:02.828+0000 7fe767fff640 1 -- 172.21.15.92:0/1373200166 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe75800db90 con 0x7fe768143ae0 2023-11-24T06:11:02.839 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:02.828+0000 7fe76df6b640 1 --2- 172.21.15.92:0/1373200166 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe758004d00 0x7fe758005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:02.839 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:02.828+0000 7fe76cf69640 1 --2- 172.21.15.92:0/1373200166 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe768143ae0 0x7fe7580047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:02.839 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:02.829+0000 7fe76d76a640 1 --2- 172.21.15.92:0/1373200166 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe758003970 0x7fe758003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:02.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:02.829+0000 7fe76df6b640 1 -- 172.21.15.92:0/1373200166 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe768143ae0 msgr2=0x7fe7580047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:02.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:02.829+0000 7fe76df6b640 1 --2- 172.21.15.92:0/1373200166 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe768143ae0 0x7fe7580047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:02.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:02.829+0000 7fe76df6b640 1 -- 172.21.15.92:0/1373200166 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe758003970 msgr2=0x7fe758003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:02.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:02.829+0000 7fe76df6b640 1 --2- 172.21.15.92:0/1373200166 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe758003970 0x7fe758003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:02.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:02.829+0000 7fe76df6b640 1 -- 172.21.15.92:0/1373200166 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe758003520 con 0x7fe758004d00 2023-11-24T06:11:02.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:02.829+0000 7fe76d76a640 1 --2- 172.21.15.92:0/1373200166 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe758003970 0x7fe758003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:11:02.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:02.830+0000 7fe76df6b640 1 --2- 172.21.15.92:0/1373200166 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe758004d00 0x7fe758005180 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fe76000c0e0 tx=0x7fe7580057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91164ea6dab491a4 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:02.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:02.830+0000 7fe767fff640 1 -- 172.21.15.92:0/1373200166 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe760005ad0 con 0x7fe758004d00 2023-11-24T06:11:02.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:02.830+0000 7fe767fff640 1 -- 172.21.15.92:0/1373200166 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe760005c30 con 0x7fe758004d00 2023-11-24T06:11:02.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:02.830+0000 7fe767fff640 1 -- 172.21.15.92:0/1373200166 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe76002c810 con 0x7fe758004d00 2023-11-24T06:11:02.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:02.831+0000 7fe76f9ce640 1 -- 172.21.15.92:0/1373200166 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe758004d00 msgr2=0x7fe758005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:02.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:02.831+0000 7fe76f9ce640 1 --2- 172.21.15.92:0/1373200166 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe758004d00 0x7fe758005180 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fe76000c0e0 tx=0x7fe7580057f0 comp rx=0 tx=0).stop 2023-11-24T06:11:02.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:02.831+0000 7fe76df6b640 1 -- 172.21.15.92:0/1373200166 reap_dead start 2023-11-24T06:11:02.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:02.831+0000 7fe76f9ce640 1 -- 172.21.15.92:0/1373200166 shutdown_connections 2023-11-24T06:11:02.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:02.831+0000 7fe76f9ce640 1 -- 172.21.15.92:0/1373200166 >> 172.21.15.92:0/1373200166 conn(0x7fe76806a800 msgr2=0x7fe76806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:02.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:02.831+0000 7fe76f9ce640 1 -- 172.21.15.92:0/1373200166 shutdown_connections 2023-11-24T06:11:02.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:02.831+0000 7fe76f9ce640 1 -- 172.21.15.92:0/1373200166 wait complete. 2023-11-24T06:11:02.844 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:02.833+0000 7fe76f9ce640 1 Processor -- start 2023-11-24T06:11:02.844 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:02.833+0000 7fe76f9ce640 1 -- start start 2023-11-24T06:11:02.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:02.834+0000 7fe76f9ce640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe758004d00 0x7fe76813d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:02.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:02.834+0000 7fe76f9ce640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe76815bab0 0x7fe76815cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:02.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:02.834+0000 7fe76f9ce640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe768159c30 0x7fe768159390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:02.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:02.834+0000 7fe76f9ce640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe76815cb50 con 0x7fe76815bab0 2023-11-24T06:11:02.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:02.834+0000 7fe76f9ce640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe76815bf40 con 0x7fe758004d00 2023-11-24T06:11:02.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:02.834+0000 7fe76f9ce640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe76815c220 con 0x7fe768159c30 2023-11-24T06:11:02.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:02.834+0000 7fe76cf69640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe76815bab0 0x7fe76815cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:02.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:02.834+0000 7fe76cf69640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe76815bab0 0x7fe76815cfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53174/0 (socket says 172.21.15.92:53174) 2023-11-24T06:11:02.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:02.834+0000 7fe76cf69640 1 -- 172.21.15.92:0/2818894176 learned_addr learned my addr 172.21.15.92:0/2818894176 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:11:02.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:02.834+0000 7fe76d76a640 1 --2- 172.21.15.92:0/2818894176 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe758004d00 0x7fe76813d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:02.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:02.834+0000 7fe76df6b640 1 --2- 172.21.15.92:0/2818894176 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe768159c30 0x7fe768159390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:02.846 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:02.834+0000 7fe76cf69640 1 -- 172.21.15.92:0/2818894176 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe768159c30 msgr2=0x7fe768159390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:02.846 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:02.834+0000 7fe76cf69640 1 --2- 172.21.15.92:0/2818894176 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe768159c30 0x7fe768159390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:02.846 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:02.834+0000 7fe76cf69640 1 -- 172.21.15.92:0/2818894176 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe758004d00 msgr2=0x7fe76813d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:02.846 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:02.834+0000 7fe76cf69640 1 --2- 172.21.15.92:0/2818894176 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe758004d00 0x7fe76813d440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:02.846 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:02.834+0000 7fe76cf69640 1 -- 172.21.15.92:0/2818894176 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe76815c500 con 0x7fe76815bab0 2023-11-24T06:11:02.846 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:02.835+0000 7fe76d76a640 1 --2- 172.21.15.92:0/2818894176 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe758004d00 0x7fe76813d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:11:02.846 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:02.835+0000 7fe76cf69640 1 --2- 172.21.15.92:0/2818894176 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe76815bab0 0x7fe76815cfe0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fe7540116d0 tx=0x7fe754011b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:02.846 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:02.835+0000 7fe765ffb640 1 -- 172.21.15.92:0/2818894176 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe75400f9c0 con 0x7fe76815bab0 2023-11-24T06:11:02.847 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:02.835+0000 7fe765ffb640 1 -- 172.21.15.92:0/2818894176 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe75400fb20 con 0x7fe76815bab0 2023-11-24T06:11:02.847 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:02.835+0000 7fe765ffb640 1 -- 172.21.15.92:0/2818894176 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe75400fe90 con 0x7fe76815bab0 2023-11-24T06:11:02.847 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:02.835+0000 7fe76f9ce640 1 -- 172.21.15.92:0/2818894176 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe76815c7e0 con 0x7fe76815bab0 2023-11-24T06:11:02.847 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:02.835+0000 7fe76f9ce640 1 -- 172.21.15.92:0/2818894176 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7681598d0 con 0x7fe76815bab0 2023-11-24T06:11:02.847 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:02.836+0000 7fe76f9ce640 1 -- 172.21.15.92:0/2818894176 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7680675d0 con 0x7fe76815bab0 2023-11-24T06:11:02.847 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:02.838+0000 7fe765ffb640 1 -- 172.21.15.92:0/2818894176 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe754015460 con 0x7fe76815bab0 2023-11-24T06:11:02.847 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:02.839+0000 7fe765ffb640 1 --2- 172.21.15.92:0/2818894176 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fe74c06ae40 0x7fe74c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:02.847 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:02.839+0000 7fe765ffb640 1 -- 172.21.15.92:0/2818894176 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe754089ca0 con 0x7fe76815bab0 2023-11-24T06:11:02.847 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:02.839+0000 7fe76d76a640 1 --2- 172.21.15.92:0/2818894176 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fe74c06ae40 0x7fe74c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:02.847 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:02.840+0000 7fe76d76a640 1 --2- 172.21.15.92:0/2818894176 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fe74c06ae40 0x7fe74c06d300 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fe75c007520 tx=0x7fe75c00d040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:02.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:02.843+0000 7fe765ffb640 1 -- 172.21.15.92:0/2818894176 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe754054c20 con 0x7fe76815bab0 2023-11-24T06:11:03.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:03.037+0000 7fe76f9ce640 1 -- 172.21.15.92:0/2818894176 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe76806cf60 con 0x7fe76815bab0 2023-11-24T06:11:03.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:03.038+0000 7fe765ffb640 1 -- 172.21.15.92:0/2818894176 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe754022020 con 0x7fe76815bab0 2023-11-24T06:11:03.042 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 64 2023-11-24T06:11:03.046 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:03.041+0000 7fe76f9ce640 1 -- 172.21.15.92:0/2818894176 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fe74c06ae40 msgr2=0x7fe74c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:03.046 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:03.041+0000 7fe76f9ce640 1 --2- 172.21.15.92:0/2818894176 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fe74c06ae40 0x7fe74c06d300 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fe75c007520 tx=0x7fe75c00d040 comp rx=0 tx=0).stop 2023-11-24T06:11:03.046 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:03.041+0000 7fe76f9ce640 1 -- 172.21.15.92:0/2818894176 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe76815bab0 msgr2=0x7fe76815cfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:03.046 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:03.041+0000 7fe76f9ce640 1 --2- 172.21.15.92:0/2818894176 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe76815bab0 0x7fe76815cfe0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fe7540116d0 tx=0x7fe754011b90 comp rx=0 tx=0).stop 2023-11-24T06:11:03.046 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:03.041+0000 7fe76f9ce640 1 -- 172.21.15.92:0/2818894176 shutdown_connections 2023-11-24T06:11:03.046 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:03.041+0000 7fe76f9ce640 1 --2- 172.21.15.92:0/2818894176 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fe74c06ae40 0x7fe74c06d300 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:03.046 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:03.041+0000 7fe76f9ce640 1 --2- 172.21.15.92:0/2818894176 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe768159c30 0x7fe768159390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:03.046 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:03.041+0000 7fe76f9ce640 1 --2- 172.21.15.92:0/2818894176 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe76815bab0 0x7fe76815cfe0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:03.046 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:03.041+0000 7fe76f9ce640 1 --2- 172.21.15.92:0/2818894176 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe758004d00 0x7fe76813d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:03.046 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:03.041+0000 7fe76f9ce640 1 -- 172.21.15.92:0/2818894176 >> 172.21.15.92:0/2818894176 conn(0x7fe76806a800 msgr2=0x7fe7680ce660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:03.046 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:03.041+0000 7fe76f9ce640 1 -- 172.21.15.92:0/2818894176 shutdown_connections 2023-11-24T06:11:03.047 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:03.041+0000 7fe76f9ce640 1 -- 172.21.15.92:0/2818894176 wait complete. 2023-11-24T06:11:03.062 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:11:03.272 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:03.269+0000 7f4ddaa85640 1 Processor -- start 2023-11-24T06:11:03.272 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:03.269+0000 7f4ddaa85640 1 -- start start 2023-11-24T06:11:03.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:03.271+0000 7f4ddaa85640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4dd4162430 0x7f4dd40b3110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:03.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:03.271+0000 7f4ddaa85640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f4dd40b3650 0x7f4dd406c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:03.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:03.271+0000 7f4ddaa85640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4dd40702e0 con 0x7f4dd4162810 2023-11-24T06:11:03.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:03.271+0000 7f4ddaa85640 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4dd40705c0 con 0x7f4dd40b3650 2023-11-24T06:11:03.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:03.271+0000 7f4ddaa85640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4dd40708a0 con 0x7f4dd4162430 2023-11-24T06:11:03.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:03.271+0000 7f4dd9282640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f4dd4162810 0x7f4dd4071240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:35556/0 (socket says 172.21.15.92:35556) 2023-11-24T06:11:03.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:03.271+0000 7f4dd9282640 1 -- 172.21.15.92:0/195742337 learned_addr learned my addr 172.21.15.92:0/195742337 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-24T06:11:03.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:03.271+0000 7f4dda284640 1 --2- 172.21.15.92:0/195742337 >> v2:172.21.15.118:3300/0 conn(0x7f4dd40b3650 0x7f4dd406c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:03.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:03.271+0000 7f4dd9a83640 1 --2- 172.21.15.92:0/195742337 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4dd4162430 0x7f4dd40b3110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:03.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:03.271+0000 7f4dd8a81640 1 -- 172.21.15.92:0/195742337 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2107451527 0 0) 0x7f4dd40702e0 con 0x7f4dd4162810 2023-11-24T06:11:03.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:03.272+0000 7f4dd8a81640 1 -- 172.21.15.92:0/195742337 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4dc0003520 con 0x7f4dd4162810 2023-11-24T06:11:03.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:03.272+0000 7f4dd8a81640 1 -- 172.21.15.92:0/195742337 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3645515671 0 0) 0x7f4dc0003520 con 0x7f4dd4162810 2023-11-24T06:11:03.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:03.272+0000 7f4dd8a81640 1 -- 172.21.15.92:0/195742337 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4dd4162430 msgr2=0x7f4dd40b3110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:03.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:03.272+0000 7f4dd8a81640 1 --2- 172.21.15.92:0/195742337 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4dd4162430 0x7f4dd40b3110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:03.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:03.272+0000 7f4dd8a81640 1 -- 172.21.15.92:0/195742337 >> v2:172.21.15.118:3300/0 conn(0x7f4dd40b3650 msgr2=0x7f4dd406c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:03.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:03.272+0000 7f4dd8a81640 1 --2- 172.21.15.92:0/195742337 >> v2:172.21.15.118:3300/0 conn(0x7f4dd40b3650 0x7f4dd406c8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:03.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:03.272+0000 7f4dd8a81640 1 -- 172.21.15.92:0/195742337 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4dd4070b80 con 0x7f4dd4162810 2023-11-24T06:11:03.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:03.272+0000 7f4dd8a81640 1 -- 172.21.15.92:0/195742337 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2607661350 0 0) 0x7f4dc4001670 con 0x7f4dd4162810 2023-11-24T06:11:03.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:03.272+0000 7f4dd9a83640 1 --2- 172.21.15.92:0/195742337 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4dd4162430 0x7f4dd40b3110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:11:03.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:03.272+0000 7f4dd8a81640 1 -- 172.21.15.92:0/195742337 >> v1:172.21.15.92:6789/0 conn(0x7f4dd4162810 legacy=0x7f4dd4071240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:03.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:03.272+0000 7f4dd8a81640 1 --2- 172.21.15.92:0/195742337 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4dc0003970 0x7f4dc0003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:03.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:03.272+0000 7f4dd8a81640 1 --2- 172.21.15.92:0/195742337 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4dc00047c0 0x7f4dc0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:03.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:03.272+0000 7f4dd8a81640 1 --2- 172.21.15.92:0/195742337 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4dd4162430 0x7f4dc0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:03.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:03.272+0000 7f4dd8a81640 1 -- 172.21.15.92:0/195742337 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dd40705c0 con 0x7f4dc0003970 2023-11-24T06:11:03.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:03.272+0000 7f4dd8a81640 1 -- 172.21.15.92:0/195742337 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dd40708a0 con 0x7f4dc00047c0 2023-11-24T06:11:03.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:03.272+0000 7f4dd8a81640 1 -- 172.21.15.92:0/195742337 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4dc000b980 con 0x7f4dd4162430 2023-11-24T06:11:03.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:03.272+0000 7f4dd9a83640 1 --2- 172.21.15.92:0/195742337 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4dc0003970 0x7f4dc0003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:03.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:03.273+0000 7f4dd9282640 1 --2- 172.21.15.92:0/195742337 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4dc00047c0 0x7f4dc0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:03.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:03.273+0000 7f4dd9a83640 1 -- 172.21.15.92:0/195742337 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4dd4162430 msgr2=0x7f4dc0005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:03.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:03.273+0000 7f4dd9a83640 1 --2- 172.21.15.92:0/195742337 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4dd4162430 0x7f4dc0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:03.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:03.273+0000 7f4dd9a83640 1 -- 172.21.15.92:0/195742337 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4dc00047c0 msgr2=0x7f4dc0004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:03.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:03.273+0000 7f4dd9a83640 1 --2- 172.21.15.92:0/195742337 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4dc00047c0 0x7f4dc0004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:03.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:03.273+0000 7f4dd9a83640 1 -- 172.21.15.92:0/195742337 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4dc0003520 con 0x7f4dc0003970 2023-11-24T06:11:03.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:03.273+0000 7f4dd9282640 1 --2- 172.21.15.92:0/195742337 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4dc00047c0 0x7f4dc0004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:11:03.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:03.274+0000 7f4dd9a83640 1 --2- 172.21.15.92:0/195742337 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4dc0003970 0x7f4dc0003d90 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f4dd00098d0 tx=0x7f4dc00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db7a8b2fa6a93d7f server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:03.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:03.274+0000 7f4dd8a81640 1 -- 172.21.15.92:0/195742337 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dd0004230 con 0x7f4dc0003970 2023-11-24T06:11:03.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:03.274+0000 7f4dd8a81640 1 -- 172.21.15.92:0/195742337 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4dd0004390 con 0x7f4dc0003970 2023-11-24T06:11:03.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:03.274+0000 7f4dd8a81640 1 -- 172.21.15.92:0/195742337 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dd0003830 con 0x7f4dc0003970 2023-11-24T06:11:03.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:03.275+0000 7f4ddaa85640 1 -- 172.21.15.92:0/195742337 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4dc0003970 msgr2=0x7f4dc0003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:03.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:03.275+0000 7f4ddaa85640 1 --2- 172.21.15.92:0/195742337 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4dc0003970 0x7f4dc0003d90 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f4dd00098d0 tx=0x7f4dc00057f0 comp rx=0 tx=0).stop 2023-11-24T06:11:03.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:03.275+0000 7f4dda284640 1 -- 172.21.15.92:0/195742337 reap_dead start 2023-11-24T06:11:03.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:03.275+0000 7f4ddaa85640 1 -- 172.21.15.92:0/195742337 shutdown_connections 2023-11-24T06:11:03.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:03.275+0000 7f4ddaa85640 1 -- 172.21.15.92:0/195742337 >> 172.21.15.92:0/195742337 conn(0x7f4dd406a800 msgr2=0x7f4dd406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:03.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:03.275+0000 7f4ddaa85640 1 -- 172.21.15.92:0/195742337 shutdown_connections 2023-11-24T06:11:03.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:03.275+0000 7f4ddaa85640 1 -- 172.21.15.92:0/195742337 wait complete. 2023-11-24T06:11:03.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:03.277+0000 7f4ddaa85640 1 Processor -- start 2023-11-24T06:11:03.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:03.277+0000 7f4ddaa85640 1 -- start start 2023-11-24T06:11:03.283 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:03.278+0000 7f4ddaa85640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4dc0003970 0x7f4dd4143520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:03.283 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:03.278+0000 7f4ddaa85640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4dd4145900 0x7f4dd4143da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:03.283 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:03.278+0000 7f4ddaa85640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4dd41442e0 0x7f4dd4145e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:03.283 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:03.278+0000 7f4ddaa85640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dd4144860 con 0x7f4dd41442e0 2023-11-24T06:11:03.283 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:03.278+0000 7f4ddaa85640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dd4144b40 con 0x7f4dc0003970 2023-11-24T06:11:03.283 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:03.278+0000 7f4ddaa85640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4dd4144e20 con 0x7f4dd4145900 2023-11-24T06:11:03.283 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:03.278+0000 7f4dda284640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4dd41442e0 0x7f4dd4145e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:03.283 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:03.278+0000 7f4dda284640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4dd41442e0 0x7f4dd4145e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53198/0 (socket says 172.21.15.92:53198) 2023-11-24T06:11:03.283 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:03.278+0000 7f4dda284640 1 -- 172.21.15.92:0/3947219180 learned_addr learned my addr 172.21.15.92:0/3947219180 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:11:03.283 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:03.278+0000 7f4dd9282640 1 --2- 172.21.15.92:0/3947219180 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4dd4145900 0x7f4dd4143da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:03.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:03.278+0000 7f4dd9a83640 1 --2- 172.21.15.92:0/3947219180 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4dc0003970 0x7f4dd4143520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:03.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:03.278+0000 7f4dda284640 1 -- 172.21.15.92:0/3947219180 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4dd4145900 msgr2=0x7f4dd4143da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:03.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:03.278+0000 7f4dda284640 1 --2- 172.21.15.92:0/3947219180 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4dd4145900 0x7f4dd4143da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:03.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:03.278+0000 7f4dda284640 1 -- 172.21.15.92:0/3947219180 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4dc0003970 msgr2=0x7f4dd4143520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:03.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:03.278+0000 7f4dda284640 1 --2- 172.21.15.92:0/3947219180 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4dc0003970 0x7f4dd4143520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:03.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:03.278+0000 7f4dda284640 1 -- 172.21.15.92:0/3947219180 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4dd4145100 con 0x7f4dd41442e0 2023-11-24T06:11:03.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:03.279+0000 7f4dda284640 1 --2- 172.21.15.92:0/3947219180 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4dd41442e0 0x7f4dd4145e30 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f4dd4072810 tx=0x7f4dcc00b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:03.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:03.279+0000 7f4dca7fc640 1 -- 172.21.15.92:0/3947219180 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dcc010040 con 0x7f4dd41442e0 2023-11-24T06:11:03.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:03.279+0000 7f4dca7fc640 1 -- 172.21.15.92:0/3947219180 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4dcc00bae0 con 0x7f4dd41442e0 2023-11-24T06:11:03.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:03.279+0000 7f4dca7fc640 1 -- 172.21.15.92:0/3947219180 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dcc00be70 con 0x7f4dd41442e0 2023-11-24T06:11:03.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:03.279+0000 7f4ddaa85640 1 -- 172.21.15.92:0/3947219180 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4dd41453e0 con 0x7f4dd41442e0 2023-11-24T06:11:03.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:03.279+0000 7f4ddaa85640 1 -- 172.21.15.92:0/3947219180 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4dd4154070 con 0x7f4dd41442e0 2023-11-24T06:11:03.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:03.280+0000 7f4ddaa85640 1 -- 172.21.15.92:0/3947219180 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4dd41455d0 con 0x7f4dd41442e0 2023-11-24T06:11:03.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:03.281+0000 7f4dca7fc640 1 -- 172.21.15.92:0/3947219180 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f4dcc01e410 con 0x7f4dd41442e0 2023-11-24T06:11:03.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:03.282+0000 7f4dca7fc640 1 --2- 172.21.15.92:0/3947219180 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f4db806ae70 0x7f4db806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:03.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:03.282+0000 7f4dca7fc640 1 -- 172.21.15.92:0/3947219180 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4dcc089830 con 0x7f4dd41442e0 2023-11-24T06:11:03.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:03.282+0000 7f4dd9a83640 1 --2- 172.21.15.92:0/3947219180 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f4db806ae70 0x7f4db806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:03.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:03.283+0000 7f4dd9a83640 1 --2- 172.21.15.92:0/3947219180 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f4db806ae70 0x7f4db806d330 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f4dd0035000 tx=0x7f4dd003a040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:03.289 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:03.286+0000 7f4dca7fc640 1 -- 172.21.15.92:0/3947219180 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4dcc0548e0 con 0x7f4dd41442e0 2023-11-24T06:11:03.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:03.479+0000 7f4ddaa85640 1 -- 172.21.15.92:0/3947219180 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4dd4067450 con 0x7f4dd41442e0 2023-11-24T06:11:03.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:03.482+0000 7f4dca7fc640 1 -- 172.21.15.92:0/3947219180 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16457 (secure 0 0 0) 0x7f4dcc057f00 con 0x7f4dd41442e0 2023-11-24T06:11:03.485 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-24T06:11:03.485 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":20,"fsid":"2d6dee5f-8aa3-470f-931a-50ade963b1cb","created":"2023-11-24T06:10:04.707204+0000","modified":"2023-11-24T06:10:26.698733+0000","last_up_change":"2023-11-24T06:10:17.598352+0000","last_in_change":"2023-11-24T06:10:09.040793+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:10:17.733098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:10:22.926445+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T06:10:24.151288+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ff37a9fb-135f-471f-b4b0-f7b5772006c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6803","nonce":20560714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6805","nonce":20560714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6809","nonce":20560714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6807","nonce":20560714}]},"public_addr":"172.21.15.92:6803/20560714","cluster_addr":"172.21.15.92:6805/20560714","heartbeat_back_addr":"172.21.15.92:6809/20560714","heartbeat_front_addr":"172.21.15.92:6807/20560714","state":["exists","up"]},{"osd":1,"uuid":"03cdd353-dcf1-4f38-94fe-a795137c06b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6819","nonce":2451239906}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6821","nonce":2451239906}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6825","nonce":2451239906}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6823","nonce":2451239906}]},"public_addr":"172.21.15.92:6819/2451239906","cluster_addr":"172.21.15.92:6821/2451239906","heartbeat_back_addr":"172.21.15.92:6825/2451239906","heartbeat_front_addr":"172.21.15.92:6823/2451239906","state":["exists","up"]},{"osd":2,"uuid":"a4a20d80-5d57-484b-becd-9aa8070dd3aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6811","nonce":1124979277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6813","nonce":1124979277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6817","nonce":1124979277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6815","nonce":1124979277}]},"public_addr":"172.21.15.92:6811/1124979277","cluster_addr":"172.21.15.92:6813/1124979277","heartbeat_back_addr":"172.21.15.92:6817/1124979277","heartbeat_front_addr":"172.21.15.92:6815/1124979277","state":["exists","up"]},{"osd":3,"uuid":"4648d7ce-9b29-4583-9425-4ba8fd2b1903","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6827","nonce":1734115782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6829","nonce":1734115782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6833","nonce":1734115782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6831","nonce":1734115782}]},"public_addr":"172.21.15.92:6827/1734115782","cluster_addr":"172.21.15.92:6829/1734115782","heartbeat_back_addr":"172.21.15.92:6833/1734115782","heartbeat_front_addr":"172.21.15.92:6831/1734115782","state":["exists","up"]},{"osd":4,"uuid":"3bc9d7eb-c6fa-4fcf-a3ec-ade604e05aac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6819","nonce":3425937229}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6822","nonce":3425937229}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6830","nonce":3425937229}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6826","nonce":3425937229}]},"public_addr":"172.21.15.118:6819/3425937229","cluster_addr":"172.21.15.118:6822/3425937229","heartbeat_back_addr":"172.21.15.118:6830/3425937229","heartbeat_front_addr":"172.21.15.118:6826/3425937229","state":["exists","up"]},{"osd":5,"uuid":"dce9ba6d-59ed-46d1-84fc-a84b13de10c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6801","nonce":827455984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6803","nonce":827455984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6807","nonce":827455984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6805","nonce":827455984}]},"public_addr":"172.21.15.118:6801/827455984","cluster_addr":"172.21.15.118:6803/827455984","heartbeat_back_addr":"172.21.15.118:6807/827455984","heartbeat_front_addr":"172.21.15.118:6805/827455984","state":["exists","up"]},{"osd":6,"uuid":"82aec504-ae58-4709-8d30-46f2b12081b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6809","nonce":2946986141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6811","nonce":2946986141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6815","nonce":2946986141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6813","nonce":2946986141}]},"public_addr":"172.21.15.118:6809/2946986141","cluster_addr":"172.21.15.118:6811/2946986141","heartbeat_back_addr":"172.21.15.118:6815/2946986141","heartbeat_front_addr":"172.21.15.118:6813/2946986141","state":["exists","up"]},{"osd":7,"uuid":"4ed4c624-c88f-4d35-bdb1-bf209b5dffb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6817","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6818","nonce":48122059}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6821","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6823","nonce":48122059}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6829","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6831","nonce":48122059}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6825","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6827","nonce":48122059}]},"public_addr":"172.21.15.118:6818/48122059","cluster_addr":"172.21.15.118:6823/48122059","heartbeat_back_addr":"172.21.15.118:6831/48122059","heartbeat_front_addr":"172.21.15.118:6827/48122059","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.536664+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.368324+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.271201+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.342971+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.621832+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.387110+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.632946+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.416032+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:11:03.488 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:03.484+0000 7f4ddaa85640 1 -- 172.21.15.92:0/3947219180 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f4db806ae70 msgr2=0x7f4db806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:03.488 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:03.484+0000 7f4ddaa85640 1 --2- 172.21.15.92:0/3947219180 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f4db806ae70 0x7f4db806d330 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f4dd0035000 tx=0x7f4dd003a040 comp rx=0 tx=0).stop 2023-11-24T06:11:03.488 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:03.484+0000 7f4ddaa85640 1 -- 172.21.15.92:0/3947219180 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4dd41442e0 msgr2=0x7f4dd4145e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:03.488 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:03.484+0000 7f4ddaa85640 1 --2- 172.21.15.92:0/3947219180 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4dd41442e0 0x7f4dd4145e30 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f4dd4072810 tx=0x7f4dcc00b480 comp rx=0 tx=0).stop 2023-11-24T06:11:03.488 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:03.484+0000 7f4ddaa85640 1 -- 172.21.15.92:0/3947219180 shutdown_connections 2023-11-24T06:11:03.488 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:03.484+0000 7f4ddaa85640 1 --2- 172.21.15.92:0/3947219180 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f4db806ae70 0x7f4db806d330 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:03.488 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:03.484+0000 7f4ddaa85640 1 --2- 172.21.15.92:0/3947219180 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4dd41442e0 0x7f4dd4145e30 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:03.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:03.484+0000 7f4ddaa85640 1 --2- 172.21.15.92:0/3947219180 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4dd4145900 0x7f4dd4143da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:03.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:03.484+0000 7f4ddaa85640 1 --2- 172.21.15.92:0/3947219180 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4dc0003970 0x7f4dd4143520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:03.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:03.484+0000 7f4ddaa85640 1 -- 172.21.15.92:0/3947219180 >> 172.21.15.92:0/3947219180 conn(0x7f4dd406a800 msgr2=0x7f4dd40711d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:03.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:03.485+0000 7f4ddaa85640 1 -- 172.21.15.92:0/3947219180 shutdown_connections 2023-11-24T06:11:03.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:03.485+0000 7f4ddaa85640 1 -- 172.21.15.92:0/3947219180 wait complete. 2023-11-24T06:11:03.508 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T06:10:17.733098+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T06:10:22.926445+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T06:10:24.151288+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T06:11:03.509 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T06:11:03.726 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:03.723+0000 7f00a826b640 1 Processor -- start 2023-11-24T06:11:03.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:03.723+0000 7f00a826b640 1 -- start start 2023-11-24T06:11:03.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:03.724+0000 7f00a826b640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f00a01540b0 0x7f00a0159f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:03.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:03.724+0000 7f00a826b640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f00a006b690 0x7f00a015dd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:03.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:03.724+0000 7f00a826b640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00a0154490 con 0x7f00a015e270 2023-11-24T06:11:03.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:03.724+0000 7f00a826b640 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00a015d550 con 0x7f00a01540b0 2023-11-24T06:11:03.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:03.724+0000 7f00a826b640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00a015a7b0 con 0x7f00a006b690 2023-11-24T06:11:03.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:03.725+0000 7f00a6808640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f00a015e270 0x7f00a0070af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:35564/0 (socket says 172.21.15.92:35564) 2023-11-24T06:11:03.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:03.725+0000 7f00a6808640 1 -- 172.21.15.92:0/108157504 learned_addr learned my addr 172.21.15.92:0/108157504 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-24T06:11:03.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:03.725+0000 7f00a6007640 1 --2- 172.21.15.92:0/108157504 >> v2:172.21.15.118:3300/0 conn(0x7f00a01540b0 0x7f00a0159f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:03.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:03.725+0000 7f00a5806640 1 --2- 172.21.15.92:0/108157504 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f00a006b690 0x7f00a015dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:03.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:03.725+0000 7f00a5005640 1 -- 172.21.15.92:0/108157504 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2250205035 0 0) 0x7f00a0154490 con 0x7f00a015e270 2023-11-24T06:11:03.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:03.725+0000 7f00a5005640 1 -- 172.21.15.92:0/108157504 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0088005730 con 0x7f00a015e270 2023-11-24T06:11:03.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:03.726+0000 7f00a5005640 1 -- 172.21.15.92:0/108157504 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4194776057 0 0) 0x7f0088005730 con 0x7f00a015e270 2023-11-24T06:11:03.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:03.726+0000 7f00a5005640 1 -- 172.21.15.92:0/108157504 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f00a006b690 msgr2=0x7f00a015dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:03.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:03.726+0000 7f00a5005640 1 --2- 172.21.15.92:0/108157504 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f00a006b690 0x7f00a015dd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:03.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:03.726+0000 7f00a5005640 1 -- 172.21.15.92:0/108157504 >> v2:172.21.15.118:3300/0 conn(0x7f00a01540b0 msgr2=0x7f00a0159f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:03.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:03.726+0000 7f00a5005640 1 --2- 172.21.15.92:0/108157504 >> v2:172.21.15.118:3300/0 conn(0x7f00a01540b0 0x7f00a0159f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:03.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:03.726+0000 7f00a5005640 1 -- 172.21.15.92:0/108157504 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00a015aa90 con 0x7f00a015e270 2023-11-24T06:11:03.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:03.726+0000 7f00a5005640 1 -- 172.21.15.92:0/108157504 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2607661350 0 0) 0x7f0098001670 con 0x7f00a015e270 2023-11-24T06:11:03.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:03.726+0000 7f00a5005640 1 -- 172.21.15.92:0/108157504 >> v1:172.21.15.92:6789/0 conn(0x7f00a015e270 legacy=0x7f00a0070af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:03.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:03.726+0000 7f00a5005640 1 --2- 172.21.15.92:0/108157504 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f00a006b690 0x7f0088005b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:03.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:03.726+0000 7f00a5005640 1 --2- 172.21.15.92:0/108157504 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0088006150 0x7f00880069d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:03.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:03.726+0000 7f00a5005640 1 --2- 172.21.15.92:0/108157504 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0088006f10 0x7f0088007390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:03.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:03.726+0000 7f00a5005640 1 -- 172.21.15.92:0/108157504 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00a015d550 con 0x7f0088006150 2023-11-24T06:11:03.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:03.726+0000 7f00a5005640 1 -- 172.21.15.92:0/108157504 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00a015a7b0 con 0x7f0088006f10 2023-11-24T06:11:03.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:03.726+0000 7f00a5005640 1 -- 172.21.15.92:0/108157504 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f008800dc40 con 0x7f00a006b690 2023-11-24T06:11:03.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:03.726+0000 7f00a5806640 1 --2- 172.21.15.92:0/108157504 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0088006150 0x7f00880069d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:03.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:03.726+0000 7f00a6007640 1 --2- 172.21.15.92:0/108157504 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f00a006b690 0x7f0088005b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:03.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:03.726+0000 7f00a5806640 1 -- 172.21.15.92:0/108157504 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f00a006b690 msgr2=0x7f0088005b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:03.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:03.726+0000 7f00a5806640 1 --2- 172.21.15.92:0/108157504 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f00a006b690 0x7f0088005b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:03.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:03.727+0000 7f00a5806640 1 -- 172.21.15.92:0/108157504 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0088006f10 msgr2=0x7f0088007390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:03.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:03.727+0000 7f00a5806640 1 --2- 172.21.15.92:0/108157504 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0088006f10 0x7f0088007390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:03.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:03.727+0000 7f00a5806640 1 -- 172.21.15.92:0/108157504 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0088005730 con 0x7f0088006150 2023-11-24T06:11:03.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:03.728+0000 7f00a5806640 1 --2- 172.21.15.92:0/108157504 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0088006150 0x7f00880069d0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f0094009fc0 tx=0x7f0088007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca0ba58bdd17de35 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:03.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:03.728+0000 7f00a5005640 1 -- 172.21.15.92:0/108157504 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f009402cc20 con 0x7f0088006150 2023-11-24T06:11:03.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:03.728+0000 7f00a5005640 1 -- 172.21.15.92:0/108157504 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f009400f040 con 0x7f0088006150 2023-11-24T06:11:03.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:03.728+0000 7f00a5005640 1 -- 172.21.15.92:0/108157504 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0094034440 con 0x7f0088006150 2023-11-24T06:11:03.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:03.728+0000 7f00a826b640 1 -- 172.21.15.92:0/108157504 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0088006150 msgr2=0x7f00880069d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:03.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:03.728+0000 7f00a826b640 1 --2- 172.21.15.92:0/108157504 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0088006150 0x7f00880069d0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f0094009fc0 tx=0x7f0088007a00 comp rx=0 tx=0).stop 2023-11-24T06:11:03.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:03.728+0000 7f00a6808640 1 -- 172.21.15.92:0/108157504 reap_dead start 2023-11-24T06:11:03.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:03.728+0000 7f00a826b640 1 -- 172.21.15.92:0/108157504 shutdown_connections 2023-11-24T06:11:03.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:03.728+0000 7f00a826b640 1 -- 172.21.15.92:0/108157504 >> 172.21.15.92:0/108157504 conn(0x7f00a00cf700 msgr2=0x7f00a00cfae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:03.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:03.728+0000 7f00a826b640 1 -- 172.21.15.92:0/108157504 shutdown_connections 2023-11-24T06:11:03.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:03.729+0000 7f00a826b640 1 -- 172.21.15.92:0/108157504 wait complete. 2023-11-24T06:11:03.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:03.730+0000 7f00a826b640 1 Processor -- start 2023-11-24T06:11:03.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:03.730+0000 7f00a826b640 1 -- start start 2023-11-24T06:11:03.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:03.731+0000 7f00a826b640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0088006150 0x7f00a0158e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:03.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:03.731+0000 7f00a826b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f00a00ce500 0x7f00a01551f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:03.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:03.731+0000 7f00a826b640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f00a0155730 0x7f00a0149d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:03.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:03.731+0000 7f00a826b640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00a01568e0 con 0x7f00a00ce500 2023-11-24T06:11:03.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:03.731+0000 7f00a826b640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00a0155c60 con 0x7f00a0155730 2023-11-24T06:11:03.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:03.731+0000 7f00a826b640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00a0155f40 con 0x7f0088006150 2023-11-24T06:11:03.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:03.732+0000 7f00a5806640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f00a00ce500 0x7f00a01551f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:03.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:03.732+0000 7f00a5806640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f00a00ce500 0x7f00a01551f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53214/0 (socket says 172.21.15.92:53214) 2023-11-24T06:11:03.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:03.732+0000 7f00a5806640 1 -- 172.21.15.92:0/4036633570 learned_addr learned my addr 172.21.15.92:0/4036633570 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:11:03.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:03.732+0000 7f00a6007640 1 --2- 172.21.15.92:0/4036633570 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0088006150 0x7f00a0158e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:03.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:03.732+0000 7f00a5806640 1 -- 172.21.15.92:0/4036633570 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0088006150 msgr2=0x7f00a0158e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:03.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:03.732+0000 7f00a6808640 1 --2- 172.21.15.92:0/4036633570 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f00a0155730 0x7f00a0149d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:03.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:03.732+0000 7f00a5806640 1 --2- 172.21.15.92:0/4036633570 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0088006150 0x7f00a0158e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:03.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:03.732+0000 7f00a5806640 1 -- 172.21.15.92:0/4036633570 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f00a0155730 msgr2=0x7f00a0149d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:03.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:03.732+0000 7f00a5806640 1 --2- 172.21.15.92:0/4036633570 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f00a0155730 0x7f00a0149d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:03.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:03.732+0000 7f00a5806640 1 -- 172.21.15.92:0/4036633570 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00a0156220 con 0x7f00a00ce500 2023-11-24T06:11:03.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:03.732+0000 7f00a5806640 1 --2- 172.21.15.92:0/4036633570 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f00a00ce500 0x7f00a01551f0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f009400e230 tx=0x7f0094009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:03.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:03.733+0000 7f009effd640 1 -- 172.21.15.92:0/4036633570 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0094034bd0 con 0x7f00a00ce500 2023-11-24T06:11:03.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:03.733+0000 7f009effd640 1 -- 172.21.15.92:0/4036633570 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0094034d50 con 0x7f00a00ce500 2023-11-24T06:11:03.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:03.733+0000 7f009effd640 1 -- 172.21.15.92:0/4036633570 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0094003bd0 con 0x7f00a00ce500 2023-11-24T06:11:03.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:03.733+0000 7f00a826b640 1 -- 172.21.15.92:0/4036633570 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00a0156500 con 0x7f00a00ce500 2023-11-24T06:11:03.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:03.733+0000 7f00a826b640 1 -- 172.21.15.92:0/4036633570 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00a0159530 con 0x7f00a00ce500 2023-11-24T06:11:03.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:03.734+0000 7f00a826b640 1 -- 172.21.15.92:0/4036633570 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00a0156a20 con 0x7f00a00ce500 2023-11-24T06:11:03.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:03.735+0000 7f009effd640 1 -- 172.21.15.92:0/4036633570 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f0094040450 con 0x7f00a00ce500 2023-11-24T06:11:03.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:03.736+0000 7f009effd640 1 --2- 172.21.15.92:0/4036633570 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f008c06ae70 0x7f008c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:03.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:03.736+0000 7f009effd640 1 -- 172.21.15.92:0/4036633570 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f00940abc00 con 0x7f00a00ce500 2023-11-24T06:11:03.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:03.736+0000 7f00a6007640 1 --2- 172.21.15.92:0/4036633570 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f008c06ae70 0x7f008c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:03.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:03.737+0000 7f00a6007640 1 --2- 172.21.15.92:0/4036633570 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f008c06ae70 0x7f008c06d330 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f0090009a90 tx=0x7f0090008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:03.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:03.740+0000 7f009effd640 1 -- 172.21.15.92:0/4036633570 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f00a0156a20 con 0x7f00a00ce500 2023-11-24T06:11:03.934 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:03.931+0000 7f00a826b640 1 -- 172.21.15.92:0/4036633570 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f00a0159530 con 0x7f00a00ce500 2023-11-24T06:11:03.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:03.932+0000 7f009effd640 1 -- 172.21.15.92:0/4036633570 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f00a0159530 con 0x7f00a00ce500 2023-11-24T06:11:03.935 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 1 2023-11-24T06:11:03.939 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:03.935+0000 7f00a826b640 1 -- 172.21.15.92:0/4036633570 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f008c06ae70 msgr2=0x7f008c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:03.939 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:03.935+0000 7f00a826b640 1 --2- 172.21.15.92:0/4036633570 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f008c06ae70 0x7f008c06d330 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f0090009a90 tx=0x7f0090008040 comp rx=0 tx=0).stop 2023-11-24T06:11:03.939 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:03.935+0000 7f00a826b640 1 -- 172.21.15.92:0/4036633570 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f00a00ce500 msgr2=0x7f00a01551f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:03.939 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:03.935+0000 7f00a826b640 1 --2- 172.21.15.92:0/4036633570 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f00a00ce500 0x7f00a01551f0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f009400e230 tx=0x7f0094009750 comp rx=0 tx=0).stop 2023-11-24T06:11:03.939 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:03.935+0000 7f00a826b640 1 -- 172.21.15.92:0/4036633570 shutdown_connections 2023-11-24T06:11:03.939 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:03.935+0000 7f00a826b640 1 --2- 172.21.15.92:0/4036633570 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f008c06ae70 0x7f008c06d330 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:03.939 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:03.935+0000 7f00a826b640 1 --2- 172.21.15.92:0/4036633570 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f00a0155730 0x7f00a0149d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:03.939 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:03.935+0000 7f00a826b640 1 --2- 172.21.15.92:0/4036633570 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f00a00ce500 0x7f00a01551f0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:03.939 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:03.935+0000 7f00a826b640 1 --2- 172.21.15.92:0/4036633570 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0088006150 0x7f00a0158e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:03.939 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:03.935+0000 7f00a826b640 1 -- 172.21.15.92:0/4036633570 >> 172.21.15.92:0/4036633570 conn(0x7f00a00cf700 msgr2=0x7f00a00cfae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:03.939 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:03.935+0000 7f00a826b640 1 -- 172.21.15.92:0/4036633570 shutdown_connections 2023-11-24T06:11:03.939 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:03.935+0000 7f00a826b640 1 -- 172.21.15.92:0/4036633570 wait complete. 2023-11-24T06:11:03.957 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T06:11:04.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:04.171+0000 7f58b1615640 1 Processor -- start 2023-11-24T06:11:04.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:04.171+0000 7f58b1615640 1 -- start start 2023-11-24T06:11:04.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:04.172+0000 7f58b1615640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f58ac0cf9a0 0x7f58ac15dca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:04.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:04.172+0000 7f58b1615640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58ac0b1ee0 con 0x7f58ac15e1e0 2023-11-24T06:11:04.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:04.172+0000 7f58b1615640 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58ac0b2780 con 0x7f58ac143ae0 2023-11-24T06:11:04.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:04.172+0000 7f58b1615640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58ac15e990 con 0x7f58ac0cf9a0 2023-11-24T06:11:04.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:04.173+0000 7f58ab7fe640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f58ac15e1e0 0x7f58ac0702c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:35574/0 (socket says 172.21.15.92:35574) 2023-11-24T06:11:04.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:04.173+0000 7f58ab7fe640 1 -- 172.21.15.92:0/602537031 learned_addr learned my addr 172.21.15.92:0/602537031 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-24T06:11:04.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:04.173+0000 7f58a27fc640 1 --2- 172.21.15.92:0/602537031 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f58ac0cf9a0 0x7f58ac15dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:04.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:04.173+0000 7f58aa7fc640 1 -- 172.21.15.92:0/602537031 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 900028492 0 0) 0x7f58ac0b1ee0 con 0x7f58ac15e1e0 2023-11-24T06:11:04.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:04.173+0000 7f58aa7fc640 1 -- 172.21.15.92:0/602537031 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5898003520 con 0x7f58ac15e1e0 2023-11-24T06:11:04.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:04.173+0000 7f58aa7fc640 1 -- 172.21.15.92:0/602537031 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3450450946 0 0) 0x7f5898003520 con 0x7f58ac15e1e0 2023-11-24T06:11:04.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:04.173+0000 7f58aa7fc640 1 -- 172.21.15.92:0/602537031 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f58ac0cf9a0 msgr2=0x7f58ac15dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:04.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:04.174+0000 7f58aa7fc640 1 --2- 172.21.15.92:0/602537031 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f58ac0cf9a0 0x7f58ac15dca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:04.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:04.174+0000 7f58aa7fc640 1 -- 172.21.15.92:0/602537031 >> v1:172.21.15.118:6789/0 conn(0x7f58ac143ae0 legacy=0x7f58ac0cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:04.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:04.174+0000 7f58aa7fc640 1 -- 172.21.15.92:0/602537031 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58ac0b2130 con 0x7f58ac15e1e0 2023-11-24T06:11:04.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:04.174+0000 7f58aa7fc640 1 -- 172.21.15.92:0/602537031 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2607661350 0 0) 0x7f58a40019f0 con 0x7f58ac15e1e0 2023-11-24T06:11:04.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:04.174+0000 7f58aa7fc640 1 -- 172.21.15.92:0/602537031 >> v1:172.21.15.92:6789/0 conn(0x7f58ac15e1e0 legacy=0x7f58ac0702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:04.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:04.174+0000 7f58aa7fc640 1 --2- 172.21.15.92:0/602537031 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f58ac0cf9a0 0x7f58980039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:04.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:04.174+0000 7f58aa7fc640 1 --2- 172.21.15.92:0/602537031 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5898003f70 0x7f58980047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:04.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:04.174+0000 7f58aa7fc640 1 --2- 172.21.15.92:0/602537031 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5898004d00 0x7f5898005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:04.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:04.174+0000 7f58aa7fc640 1 -- 172.21.15.92:0/602537031 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58ac15e990 con 0x7f5898004d00 2023-11-24T06:11:04.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:04.174+0000 7f58aa7fc640 1 -- 172.21.15.92:0/602537031 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f589800bac0 con 0x7f5898003f70 2023-11-24T06:11:04.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:04.174+0000 7f58aa7fc640 1 -- 172.21.15.92:0/602537031 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f589800bc80 con 0x7f58ac0cf9a0 2023-11-24T06:11:04.180 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:04.174+0000 7f58ab7fe640 1 --2- 172.21.15.92:0/602537031 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5898004d00 0x7f5898005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:04.180 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:04.174+0000 7f58a27fc640 1 --2- 172.21.15.92:0/602537031 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5898003f70 0x7f58980047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:04.180 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:04.174+0000 7f58aaffd640 1 --2- 172.21.15.92:0/602537031 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f58ac0cf9a0 0x7f58980039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:04.180 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:04.175+0000 7f58ab7fe640 1 -- 172.21.15.92:0/602537031 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f58ac0cf9a0 msgr2=0x7f58980039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:04.180 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:04.175+0000 7f58ab7fe640 1 --2- 172.21.15.92:0/602537031 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f58ac0cf9a0 0x7f58980039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:04.180 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:04.175+0000 7f58ab7fe640 1 -- 172.21.15.92:0/602537031 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5898003f70 msgr2=0x7f58980047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:04.180 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:04.175+0000 7f58ab7fe640 1 --2- 172.21.15.92:0/602537031 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5898003f70 0x7f58980047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:04.180 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:04.175+0000 7f58ab7fe640 1 -- 172.21.15.92:0/602537031 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5898003520 con 0x7f5898004d00 2023-11-24T06:11:04.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:04.176+0000 7f58ab7fe640 1 --2- 172.21.15.92:0/602537031 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5898004d00 0x7f5898005180 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f58a400a750 tx=0x7f58980057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea476a6e44211f2 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:04.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:04.176+0000 7f58aa7fc640 1 -- 172.21.15.92:0/602537031 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58a4034630 con 0x7f5898004d00 2023-11-24T06:11:04.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:04.176+0000 7f58aa7fc640 1 -- 172.21.15.92:0/602537031 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58a402f070 con 0x7f5898004d00 2023-11-24T06:11:04.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:04.176+0000 7f58aa7fc640 1 -- 172.21.15.92:0/602537031 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58a40349d0 con 0x7f5898004d00 2023-11-24T06:11:04.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:04.176+0000 7f58b1615640 1 -- 172.21.15.92:0/602537031 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5898004d00 msgr2=0x7f5898005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:04.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:04.176+0000 7f58b1615640 1 --2- 172.21.15.92:0/602537031 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5898004d00 0x7f5898005180 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f58a400a750 tx=0x7f58980057f0 comp rx=0 tx=0).stop 2023-11-24T06:11:04.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:04.176+0000 7f58ab7fe640 1 -- 172.21.15.92:0/602537031 reap_dead start 2023-11-24T06:11:04.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:04.177+0000 7f58b1615640 1 -- 172.21.15.92:0/602537031 shutdown_connections 2023-11-24T06:11:04.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:04.177+0000 7f58b1615640 1 -- 172.21.15.92:0/602537031 >> 172.21.15.92:0/602537031 conn(0x7f58ac06a800 msgr2=0x7f58ac06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:04.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:04.177+0000 7f58b1615640 1 -- 172.21.15.92:0/602537031 shutdown_connections 2023-11-24T06:11:04.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:04.177+0000 7f58b1615640 1 -- 172.21.15.92:0/602537031 wait complete. 2023-11-24T06:11:04.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:04.178+0000 7f58b1615640 1 Processor -- start 2023-11-24T06:11:04.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:04.179+0000 7f58b1615640 1 -- start start 2023-11-24T06:11:04.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:04.179+0000 7f58b1615640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f58ac14c800 0x7f58ac14d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:04.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:04.179+0000 7f58b1615640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f58ac14cbe0 0x7f58ac13b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:04.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:04.179+0000 7f58b1615640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f58ac0a84c0 0x7f58ac14b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:04.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:04.179+0000 7f58b1615640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58ac13bcf0 con 0x7f58ac0a84c0 2023-11-24T06:11:04.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:04.179+0000 7f58b1615640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58ac14a6a0 con 0x7f58ac14cbe0 2023-11-24T06:11:04.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:04.180+0000 7f58b1615640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58ac14a980 con 0x7f58ac14c800 2023-11-24T06:11:04.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:04.180+0000 7f58ab7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f58ac0a84c0 0x7f58ac14b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:04.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:04.180+0000 7f58ab7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f58ac0a84c0 0x7f58ac14b740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53236/0 (socket says 172.21.15.92:53236) 2023-11-24T06:11:04.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:04.180+0000 7f58ab7fe640 1 -- 172.21.15.92:0/1905119722 learned_addr learned my addr 172.21.15.92:0/1905119722 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:11:04.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:04.180+0000 7f58aaffd640 1 --2- 172.21.15.92:0/1905119722 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f58ac14c800 0x7f58ac14d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:04.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:04.180+0000 7f58a27fc640 1 --2- 172.21.15.92:0/1905119722 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f58ac14cbe0 0x7f58ac13b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:04.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:04.180+0000 7f58ab7fe640 1 -- 172.21.15.92:0/1905119722 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f58ac14c800 msgr2=0x7f58ac14d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:04.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:04.180+0000 7f58ab7fe640 1 --2- 172.21.15.92:0/1905119722 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f58ac14c800 0x7f58ac14d150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:04.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:04.180+0000 7f58ab7fe640 1 -- 172.21.15.92:0/1905119722 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f58ac14cbe0 msgr2=0x7f58ac13b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:04.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:04.180+0000 7f58ab7fe640 1 --2- 172.21.15.92:0/1905119722 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f58ac14cbe0 0x7f58ac13b7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:04.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:04.180+0000 7f58ab7fe640 1 -- 172.21.15.92:0/1905119722 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58ac14ac60 con 0x7f58ac0a84c0 2023-11-24T06:11:04.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:04.180+0000 7f58a27fc640 1 --2- 172.21.15.92:0/1905119722 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f58ac14cbe0 0x7f58ac13b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:11:04.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:04.181+0000 7f58ab7fe640 1 --2- 172.21.15.92:0/1905119722 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f58ac0a84c0 0x7f58ac14b740 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f58a400a750 tx=0x7f58ac0b26f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:04.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:04.181+0000 7f58a3fff640 1 -- 172.21.15.92:0/1905119722 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58a4034840 con 0x7f58ac0a84c0 2023-11-24T06:11:04.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:04.181+0000 7f58a3fff640 1 -- 172.21.15.92:0/1905119722 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58a40349a0 con 0x7f58ac0a84c0 2023-11-24T06:11:04.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:04.181+0000 7f58a3fff640 1 -- 172.21.15.92:0/1905119722 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58a4034d30 con 0x7f58ac0a84c0 2023-11-24T06:11:04.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:04.181+0000 7f58b1615640 1 -- 172.21.15.92:0/1905119722 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58ac14af40 con 0x7f58ac0a84c0 2023-11-24T06:11:04.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:04.181+0000 7f58b1615640 1 -- 172.21.15.92:0/1905119722 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58ac0a8940 con 0x7f58ac0a84c0 2023-11-24T06:11:04.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:04.182+0000 7f58b1615640 1 -- 172.21.15.92:0/1905119722 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58ac0675d0 con 0x7f58ac0a84c0 2023-11-24T06:11:04.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:04.183+0000 7f58a3fff640 1 -- 172.21.15.92:0/1905119722 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f58a4031030 con 0x7f58ac0a84c0 2023-11-24T06:11:04.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:04.184+0000 7f58a3fff640 1 --2- 172.21.15.92:0/1905119722 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f588c06ae40 0x7f588c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:04.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:04.184+0000 7f58a3fff640 1 -- 172.21.15.92:0/1905119722 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f58a40abaf0 con 0x7f58ac0a84c0 2023-11-24T06:11:04.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:04.184+0000 7f58aaffd640 1 --2- 172.21.15.92:0/1905119722 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f588c06ae40 0x7f588c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:04.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:04.185+0000 7f58aaffd640 1 --2- 172.21.15.92:0/1905119722 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f588c06ae40 0x7f588c06d300 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f589c007fd0 tx=0x7f589c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:04.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:04.188+0000 7f58a3fff640 1 -- 172.21.15.92:0/1905119722 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f58a4076c30 con 0x7f58ac0a84c0 2023-11-24T06:11:04.382 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:04.378+0000 7f58b1615640 1 -- 172.21.15.92:0/1905119722 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f58ac0675d0 con 0x7f58ac0a84c0 2023-11-24T06:11:04.382 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:04.379+0000 7f58a3fff640 1 -- 172.21.15.92:0/1905119722 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f58a407a250 con 0x7f58ac0a84c0 2023-11-24T06:11:04.383 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 32 2023-11-24T06:11:04.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:04.381+0000 7f58b1615640 1 -- 172.21.15.92:0/1905119722 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f588c06ae40 msgr2=0x7f588c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:04.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:04.381+0000 7f58b1615640 1 --2- 172.21.15.92:0/1905119722 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f588c06ae40 0x7f588c06d300 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f589c007fd0 tx=0x7f589c009040 comp rx=0 tx=0).stop 2023-11-24T06:11:04.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:04.381+0000 7f58b1615640 1 -- 172.21.15.92:0/1905119722 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f58ac0a84c0 msgr2=0x7f58ac14b740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:04.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:04.381+0000 7f58b1615640 1 --2- 172.21.15.92:0/1905119722 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f58ac0a84c0 0x7f58ac14b740 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f58a400a750 tx=0x7f58ac0b26f0 comp rx=0 tx=0).stop 2023-11-24T06:11:04.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:04.382+0000 7f58b1615640 1 -- 172.21.15.92:0/1905119722 shutdown_connections 2023-11-24T06:11:04.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:04.382+0000 7f58b1615640 1 --2- 172.21.15.92:0/1905119722 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f588c06ae40 0x7f588c06d300 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:04.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:04.382+0000 7f58b1615640 1 --2- 172.21.15.92:0/1905119722 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f58ac0a84c0 0x7f58ac14b740 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:04.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:04.382+0000 7f58b1615640 1 --2- 172.21.15.92:0/1905119722 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f58ac14cbe0 0x7f58ac13b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:04.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:04.382+0000 7f58b1615640 1 --2- 172.21.15.92:0/1905119722 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f58ac14c800 0x7f58ac14d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:04.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:04.382+0000 7f58b1615640 1 -- 172.21.15.92:0/1905119722 >> 172.21.15.92:0/1905119722 conn(0x7f58ac06a800 msgr2=0x7f58ac0cf290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:04.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:04.382+0000 7f58b1615640 1 -- 172.21.15.92:0/1905119722 shutdown_connections 2023-11-24T06:11:04.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:04.382+0000 7f58b1615640 1 -- 172.21.15.92:0/1905119722 wait complete. 2023-11-24T06:11:04.405 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T06:11:04.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:04.624+0000 7fa827a7c640 1 Processor -- start 2023-11-24T06:11:04.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:04.624+0000 7fa827a7c640 1 -- start start 2023-11-24T06:11:04.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:04.626+0000 7fa827a7c640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa82013df80 0x7fa82015bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:04.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:04.626+0000 7fa827a7c640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fa82013e360 0x7fa8200cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:04.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:04.626+0000 7fa827a7c640 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa82006c470 con 0x7fa82015f9c0 2023-11-24T06:11:04.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:04.626+0000 7fa827a7c640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa82006c750 con 0x7fa82013e360 2023-11-24T06:11:04.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:04.626+0000 7fa827a7c640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa82006ca30 con 0x7fa82013df80 2023-11-24T06:11:04.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:04.626+0000 7fa825017640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fa82013e360 0x7fa8200cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:04.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:04.626+0000 7fa825017640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fa82013e360 0x7fa8200cf290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53252/0 (socket says 172.21.15.92:53252) 2023-11-24T06:11:04.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:04.626+0000 7fa825017640 1 -- 172.21.15.92:0/579276052 learned_addr learned my addr 172.21.15.92:0/579276052 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:11:04.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:04.626+0000 7fa825017640 1 -- 172.21.15.92:0/579276052 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa82013df80 msgr2=0x7fa82015bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:04.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:04.626+0000 7fa825818640 1 --2- 172.21.15.92:0/579276052 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa82013df80 0x7fa82015bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:04.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:04.627+0000 7fa825017640 1 --2- 172.21.15.92:0/579276052 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa82013df80 0x7fa82015bee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:04.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:04.627+0000 7fa825017640 1 -- 172.21.15.92:0/579276052 >> v1:172.21.15.118:6789/0 conn(0x7fa82015f9c0 legacy=0x7fa8201408d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:04.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:04.627+0000 7fa825017640 1 -- 172.21.15.92:0/579276052 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa82006cd10 con 0x7fa82013e360 2023-11-24T06:11:04.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:04.628+0000 7fa825017640 1 --2- 172.21.15.92:0/579276052 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa82013e360 0x7fa8200cf290 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fa810009870 tx=0x7fa81002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=136a98c8a8ff1906 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:04.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:04.628+0000 7fa824816640 1 -- 172.21.15.92:0/579276052 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa810031640 con 0x7fa82013e360 2023-11-24T06:11:04.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:04.628+0000 7fa824816640 1 -- 172.21.15.92:0/579276052 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8100317a0 con 0x7fa82013e360 2023-11-24T06:11:04.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:04.628+0000 7fa824816640 1 -- 172.21.15.92:0/579276052 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa810031b30 con 0x7fa82013e360 2023-11-24T06:11:04.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:04.628+0000 7fa827a7c640 1 -- 172.21.15.92:0/579276052 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa82013e360 msgr2=0x7fa8200cf290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:04.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:04.628+0000 7fa827a7c640 1 --2- 172.21.15.92:0/579276052 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa82013e360 0x7fa8200cf290 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fa810009870 tx=0x7fa81002b8f0 comp rx=0 tx=0).stop 2023-11-24T06:11:04.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:04.629+0000 7fa827a7c640 1 -- 172.21.15.92:0/579276052 shutdown_connections 2023-11-24T06:11:04.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:04.629+0000 7fa827a7c640 1 --2- 172.21.15.92:0/579276052 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa82013e360 0x7fa8200cf290 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:04.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:04.629+0000 7fa827a7c640 1 --2- 172.21.15.92:0/579276052 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa82013df80 0x7fa82015bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:04.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:04.629+0000 7fa827a7c640 1 -- 172.21.15.92:0/579276052 >> 172.21.15.92:0/579276052 conn(0x7fa82006a800 msgr2=0x7fa82006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:04.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:04.629+0000 7fa827a7c640 1 -- 172.21.15.92:0/579276052 shutdown_connections 2023-11-24T06:11:04.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:04.629+0000 7fa827a7c640 1 -- 172.21.15.92:0/579276052 wait complete. 2023-11-24T06:11:04.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:04.630+0000 7fa827a7c640 1 Processor -- start 2023-11-24T06:11:04.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:04.630+0000 7fa827a7c640 1 -- start start 2023-11-24T06:11:04.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:04.631+0000 7fa827a7c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa82013df80 0x7fa82014e030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:04.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:04.631+0000 7fa827a7c640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa82013e360 0x7fa82014d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:04.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:04.631+0000 7fa827a7c640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa82015f9c0 0x7fa820154180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:04.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:04.631+0000 7fa827a7c640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa82014d580 con 0x7fa82013df80 2023-11-24T06:11:04.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:04.631+0000 7fa827a7c640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa82014c900 con 0x7fa82015f9c0 2023-11-24T06:11:04.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:04.631+0000 7fa827a7c640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa82014cbe0 con 0x7fa82013e360 2023-11-24T06:11:04.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:04.632+0000 7fa825818640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa82013df80 0x7fa82014e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:04.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:04.632+0000 7fa825818640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa82013df80 0x7fa82014e030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53254/0 (socket says 172.21.15.92:53254) 2023-11-24T06:11:04.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:04.632+0000 7fa825818640 1 -- 172.21.15.92:0/2337444970 learned_addr learned my addr 172.21.15.92:0/2337444970 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:11:04.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:04.632+0000 7fa825818640 1 -- 172.21.15.92:0/2337444970 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa82013e360 msgr2=0x7fa82014d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:04.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:04.632+0000 7fa825818640 1 --2- 172.21.15.92:0/2337444970 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa82013e360 0x7fa82014d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:04.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:04.632+0000 7fa825818640 1 -- 172.21.15.92:0/2337444970 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa82015f9c0 msgr2=0x7fa820154180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:04.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:04.632+0000 7fa825818640 1 --2- 172.21.15.92:0/2337444970 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa82015f9c0 0x7fa820154180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:04.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:04.632+0000 7fa825818640 1 -- 172.21.15.92:0/2337444970 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8100094d0 con 0x7fa82013df80 2023-11-24T06:11:04.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:04.632+0000 7fa825818640 1 --2- 172.21.15.92:0/2337444970 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa82013df80 0x7fa82014e030 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fa81c00c7f0 tx=0x7fa81c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:04.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:04.633+0000 7fa8167fc640 1 -- 172.21.15.92:0/2337444970 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa81c004220 con 0x7fa82013df80 2023-11-24T06:11:04.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:04.633+0000 7fa8167fc640 1 -- 172.21.15.92:0/2337444970 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa81c004380 con 0x7fa82013df80 2023-11-24T06:11:04.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:04.633+0000 7fa8167fc640 1 -- 172.21.15.92:0/2337444970 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa81c007840 con 0x7fa82013df80 2023-11-24T06:11:04.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:04.633+0000 7fa827a7c640 1 -- 172.21.15.92:0/2337444970 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa82014cec0 con 0x7fa82013df80 2023-11-24T06:11:04.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:04.633+0000 7fa827a7c640 1 -- 172.21.15.92:0/2337444970 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa820151200 con 0x7fa82013df80 2023-11-24T06:11:04.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:04.634+0000 7fa827a7c640 1 -- 172.21.15.92:0/2337444970 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8200675d0 con 0x7fa82013df80 2023-11-24T06:11:04.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:04.635+0000 7fa8167fc640 1 -- 172.21.15.92:0/2337444970 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa81c007bb0 con 0x7fa82013df80 2023-11-24T06:11:04.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:04.636+0000 7fa8167fc640 1 --2- 172.21.15.92:0/2337444970 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fa80406ae40 0x7fa80406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:04.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:04.636+0000 7fa8167fc640 1 -- 172.21.15.92:0/2337444970 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa81c089520 con 0x7fa82013df80 2023-11-24T06:11:04.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:04.636+0000 7fa825017640 1 --2- 172.21.15.92:0/2337444970 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fa80406ae40 0x7fa80406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:04.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:04.637+0000 7fa825017640 1 --2- 172.21.15.92:0/2337444970 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fa80406ae40 0x7fa80406d300 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fa810036000 tx=0x7fa810030000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:04.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:04.640+0000 7fa8167fc640 1 -- 172.21.15.92:0/2337444970 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa81c0545c0 con 0x7fa82013df80 2023-11-24T06:11:04.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:04.836+0000 7fa827a7c640 1 -- 172.21.15.92:0/2337444970 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa82015bf70 con 0x7fa82013df80 2023-11-24T06:11:04.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:04.837+0000 7fa8167fc640 1 -- 172.21.15.92:0/2337444970 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7fa81c01e020 con 0x7fa82013df80 2023-11-24T06:11:04.841 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 64 2023-11-24T06:11:04.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:04.839+0000 7fa827a7c640 1 -- 172.21.15.92:0/2337444970 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fa80406ae40 msgr2=0x7fa80406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:04.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:04.839+0000 7fa827a7c640 1 --2- 172.21.15.92:0/2337444970 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fa80406ae40 0x7fa80406d300 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fa810036000 tx=0x7fa810030000 comp rx=0 tx=0).stop 2023-11-24T06:11:04.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:04.839+0000 7fa827a7c640 1 -- 172.21.15.92:0/2337444970 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa82013df80 msgr2=0x7fa82014e030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:04.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:04.839+0000 7fa827a7c640 1 --2- 172.21.15.92:0/2337444970 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa82013df80 0x7fa82014e030 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fa81c00c7f0 tx=0x7fa81c00ccb0 comp rx=0 tx=0).stop 2023-11-24T06:11:04.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:04.840+0000 7fa827a7c640 1 -- 172.21.15.92:0/2337444970 shutdown_connections 2023-11-24T06:11:04.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:04.840+0000 7fa827a7c640 1 --2- 172.21.15.92:0/2337444970 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fa80406ae40 0x7fa80406d300 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:04.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:04.840+0000 7fa827a7c640 1 --2- 172.21.15.92:0/2337444970 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa82015f9c0 0x7fa820154180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:04.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:04.840+0000 7fa827a7c640 1 --2- 172.21.15.92:0/2337444970 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa82013e360 0x7fa82014d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:04.844 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:04.840+0000 7fa827a7c640 1 --2- 172.21.15.92:0/2337444970 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa82013df80 0x7fa82014e030 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:04.844 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:04.840+0000 7fa827a7c640 1 -- 172.21.15.92:0/2337444970 >> 172.21.15.92:0/2337444970 conn(0x7fa82006a800 msgr2=0x7fa82015d170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:04.844 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:04.840+0000 7fa827a7c640 1 -- 172.21.15.92:0/2337444970 shutdown_connections 2023-11-24T06:11:04.844 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:04.840+0000 7fa827a7c640 1 -- 172.21.15.92:0/2337444970 wait complete. 2023-11-24T06:11:04.926 INFO:tasks.cephfs_test_runner:Starting test: test_volume_create (tasks.cephfs.test_volumes.TestVolumes) That the volume can be created and then cleans up 2023-11-24T06:11:04.926 INFO:tasks.cephfs_test_runner:test_volume_create (tasks.cephfs.test_volumes.TestVolumes) 2023-11-24T06:11:04.926 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:11:05.145 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:05.141+0000 7fd5f50bb640 1 Processor -- start 2023-11-24T06:11:05.145 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:05.142+0000 7fd5f50bb640 1 -- start start 2023-11-24T06:11:05.146 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:05.143+0000 7fd5f50bb640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd5f013e360 0x7fd5f00cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:05.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:05.143+0000 7fd5f50bb640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fd5f015f9c0 0x7fd5f00cf7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:05.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:05.143+0000 7fd5f50bb640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5f015c5f0 con 0x7fd5f013df80 2023-11-24T06:11:05.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:05.143+0000 7fd5f50bb640 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5f006c470 con 0x7fd5f015f9c0 2023-11-24T06:11:05.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:05.143+0000 7fd5f50bb640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5f006c750 con 0x7fd5f013e360 2023-11-24T06:11:05.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:05.143+0000 7fd5eed9d640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fd5f013df80 0x7fd5f015bee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:35590/0 (socket says 172.21.15.92:35590) 2023-11-24T06:11:05.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:05.143+0000 7fd5eed9d640 1 -- 172.21.15.92:0/3781425806 learned_addr learned my addr 172.21.15.92:0/3781425806 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-24T06:11:05.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:05.144+0000 7fd5ee59c640 1 -- 172.21.15.92:0/3781425806 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 441819719 0 0) 0x7fd5f015c5f0 con 0x7fd5f013df80 2023-11-24T06:11:05.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:05.144+0000 7fd5ee59c640 1 -- 172.21.15.92:0/3781425806 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd5dc003520 con 0x7fd5f013df80 2023-11-24T06:11:05.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:05.144+0000 7fd5e7fff640 1 --2- 172.21.15.92:0/3781425806 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd5f013e360 0x7fd5f00cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:05.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:05.144+0000 7fd5ef59e640 1 --2- 172.21.15.92:0/3781425806 >> v2:172.21.15.118:3300/0 conn(0x7fd5f015f9c0 0x7fd5f00cf7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:05.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:05.144+0000 7fd5ee59c640 1 -- 172.21.15.92:0/3781425806 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 669524009 0 0) 0x7fd5dc003520 con 0x7fd5f013df80 2023-11-24T06:11:05.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:05.144+0000 7fd5ee59c640 1 -- 172.21.15.92:0/3781425806 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd5f013e360 msgr2=0x7fd5f00cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:05.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:05.144+0000 7fd5ee59c640 1 --2- 172.21.15.92:0/3781425806 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd5f013e360 0x7fd5f00cf290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:05.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:05.144+0000 7fd5ee59c640 1 -- 172.21.15.92:0/3781425806 >> v2:172.21.15.118:3300/0 conn(0x7fd5f015f9c0 msgr2=0x7fd5f00cf7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:05.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:05.144+0000 7fd5ee59c640 1 --2- 172.21.15.92:0/3781425806 >> v2:172.21.15.118:3300/0 conn(0x7fd5f015f9c0 0x7fd5f00cf7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:05.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:05.144+0000 7fd5ee59c640 1 -- 172.21.15.92:0/3781425806 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5f006ca30 con 0x7fd5f013df80 2023-11-24T06:11:05.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:05.144+0000 7fd5ee59c640 1 -- 172.21.15.92:0/3781425806 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2607661350 0 0) 0x7fd5e0001670 con 0x7fd5f013df80 2023-11-24T06:11:05.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:05.144+0000 7fd5ef59e640 1 --2- 172.21.15.92:0/3781425806 >> v2:172.21.15.118:3300/0 conn(0x7fd5f015f9c0 0x7fd5f00cf7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:11:05.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:05.144+0000 7fd5ee59c640 1 -- 172.21.15.92:0/3781425806 >> v1:172.21.15.92:6789/0 conn(0x7fd5f013df80 legacy=0x7fd5f015bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:05.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:05.144+0000 7fd5ee59c640 1 --2- 172.21.15.92:0/3781425806 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd5dc003970 0x7fd5dc003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:05.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:05.145+0000 7fd5ee59c640 1 --2- 172.21.15.92:0/3781425806 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd5f013e360 0x7fd5dc0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:05.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:05.145+0000 7fd5ee59c640 1 --2- 172.21.15.92:0/3781425806 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd5dc004d00 0x7fd5dc005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:05.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:05.145+0000 7fd5ee59c640 1 -- 172.21.15.92:0/3781425806 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5f006c470 con 0x7fd5dc003970 2023-11-24T06:11:05.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:05.145+0000 7fd5ee59c640 1 -- 172.21.15.92:0/3781425806 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5f006c750 con 0x7fd5dc004d00 2023-11-24T06:11:05.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:05.145+0000 7fd5ee59c640 1 -- 172.21.15.92:0/3781425806 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5dc00ba30 con 0x7fd5f013e360 2023-11-24T06:11:05.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:05.145+0000 7fd5eed9d640 1 --2- 172.21.15.92:0/3781425806 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd5dc003970 0x7fd5dc003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:05.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:05.145+0000 7fd5e7fff640 1 --2- 172.21.15.92:0/3781425806 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd5f013e360 0x7fd5dc0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:05.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:05.145+0000 7fd5eed9d640 1 -- 172.21.15.92:0/3781425806 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd5f013e360 msgr2=0x7fd5dc0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:05.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:05.145+0000 7fd5eed9d640 1 --2- 172.21.15.92:0/3781425806 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd5f013e360 0x7fd5dc0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:05.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:05.145+0000 7fd5eed9d640 1 -- 172.21.15.92:0/3781425806 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd5dc004d00 msgr2=0x7fd5dc005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:05.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:05.145+0000 7fd5eed9d640 1 --2- 172.21.15.92:0/3781425806 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd5dc004d00 0x7fd5dc005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:05.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:05.145+0000 7fd5eed9d640 1 -- 172.21.15.92:0/3781425806 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5dc003520 con 0x7fd5dc003970 2023-11-24T06:11:05.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:05.146+0000 7fd5eed9d640 1 --2- 172.21.15.92:0/3781425806 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd5dc003970 0x7fd5dc003d90 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fd5e000c160 tx=0x7fd5dc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eceafc183af0ec17 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:05.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:05.147+0000 7fd5ee59c640 1 -- 172.21.15.92:0/3781425806 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5e0005ad0 con 0x7fd5dc003970 2023-11-24T06:11:05.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:05.147+0000 7fd5ee59c640 1 -- 172.21.15.92:0/3781425806 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5e0005c30 con 0x7fd5dc003970 2023-11-24T06:11:05.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:05.147+0000 7fd5ee59c640 1 -- 172.21.15.92:0/3781425806 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5e002c810 con 0x7fd5dc003970 2023-11-24T06:11:05.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:05.147+0000 7fd5f50bb640 1 -- 172.21.15.92:0/3781425806 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd5dc003970 msgr2=0x7fd5dc003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:05.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:05.147+0000 7fd5f50bb640 1 --2- 172.21.15.92:0/3781425806 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd5dc003970 0x7fd5dc003d90 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fd5e000c160 tx=0x7fd5dc0057f0 comp rx=0 tx=0).stop 2023-11-24T06:11:05.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:05.147+0000 7fd5ef59e640 1 -- 172.21.15.92:0/3781425806 reap_dead start 2023-11-24T06:11:05.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:05.147+0000 7fd5f50bb640 1 -- 172.21.15.92:0/3781425806 shutdown_connections 2023-11-24T06:11:05.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:05.147+0000 7fd5f50bb640 1 -- 172.21.15.92:0/3781425806 >> 172.21.15.92:0/3781425806 conn(0x7fd5f006a800 msgr2=0x7fd5f006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:05.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:05.147+0000 7fd5f50bb640 1 -- 172.21.15.92:0/3781425806 shutdown_connections 2023-11-24T06:11:05.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:05.147+0000 7fd5f50bb640 1 -- 172.21.15.92:0/3781425806 wait complete. 2023-11-24T06:11:05.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:05.149+0000 7fd5f50bb640 1 Processor -- start 2023-11-24T06:11:05.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:05.149+0000 7fd5f50bb640 1 -- start start 2023-11-24T06:11:05.155 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:05.151+0000 7fd5f50bb640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd5dc003970 0x7fd5f014a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:05.155 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:05.151+0000 7fd5f50bb640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd5f014d980 0x7fd5f01336c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:05.155 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:05.151+0000 7fd5f50bb640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd5f0150980 0x7fd5f0150e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:05.155 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:05.151+0000 7fd5f50bb640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5f014aeb0 con 0x7fd5dc003970 2023-11-24T06:11:05.155 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:05.151+0000 7fd5f50bb640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5f013dba0 con 0x7fd5f0150980 2023-11-24T06:11:05.155 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:05.151+0000 7fd5f50bb640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5f014f920 con 0x7fd5f014d980 2023-11-24T06:11:05.155 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:05.151+0000 7fd5eed9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd5dc003970 0x7fd5f014a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:05.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:05.151+0000 7fd5eed9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd5dc003970 0x7fd5f014a970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53262/0 (socket says 172.21.15.92:53262) 2023-11-24T06:11:05.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:05.151+0000 7fd5eed9d640 1 -- 172.21.15.92:0/885482008 learned_addr learned my addr 172.21.15.92:0/885482008 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:11:05.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:05.151+0000 7fd5ef59e640 1 --2- 172.21.15.92:0/885482008 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd5f0150980 0x7fd5f0150e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:05.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:05.151+0000 7fd5e7fff640 1 --2- 172.21.15.92:0/885482008 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd5f014d980 0x7fd5f01336c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:05.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:05.151+0000 7fd5eed9d640 1 -- 172.21.15.92:0/885482008 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd5f014d980 msgr2=0x7fd5f01336c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:05.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:05.151+0000 7fd5eed9d640 1 --2- 172.21.15.92:0/885482008 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd5f014d980 0x7fd5f01336c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:05.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:05.151+0000 7fd5eed9d640 1 -- 172.21.15.92:0/885482008 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd5f0150980 msgr2=0x7fd5f0150e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:05.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:05.151+0000 7fd5eed9d640 1 --2- 172.21.15.92:0/885482008 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd5f0150980 0x7fd5f0150e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:05.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:05.151+0000 7fd5eed9d640 1 -- 172.21.15.92:0/885482008 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5f0133c00 con 0x7fd5dc003970 2023-11-24T06:11:05.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:05.151+0000 7fd5e7fff640 1 --2- 172.21.15.92:0/885482008 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd5f014d980 0x7fd5f01336c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:11:05.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:05.151+0000 7fd5ef59e640 1 --2- 172.21.15.92:0/885482008 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd5f0150980 0x7fd5f0150e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:11:05.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:05.152+0000 7fd5eed9d640 1 --2- 172.21.15.92:0/885482008 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd5dc003970 0x7fd5f014a970 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fd5e000c280 tx=0x7fd5e0000fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:05.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:05.152+0000 7fd5e77fe640 1 -- 172.21.15.92:0/885482008 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5e003c040 con 0x7fd5dc003970 2023-11-24T06:11:05.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:05.152+0000 7fd5e77fe640 1 -- 172.21.15.92:0/885482008 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5e0007270 con 0x7fd5dc003970 2023-11-24T06:11:05.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:05.152+0000 7fd5e77fe640 1 -- 172.21.15.92:0/885482008 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5e0007620 con 0x7fd5dc003970 2023-11-24T06:11:05.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:05.152+0000 7fd5f50bb640 1 -- 172.21.15.92:0/885482008 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5f014fbc0 con 0x7fd5dc003970 2023-11-24T06:11:05.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:05.152+0000 7fd5f50bb640 1 -- 172.21.15.92:0/885482008 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5f0150180 con 0x7fd5dc003970 2023-11-24T06:11:05.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:05.154+0000 7fd5f50bb640 1 -- 172.21.15.92:0/885482008 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5f00675d0 con 0x7fd5dc003970 2023-11-24T06:11:05.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:05.154+0000 7fd5e77fe640 1 -- 172.21.15.92:0/885482008 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd5e0032070 con 0x7fd5dc003970 2023-11-24T06:11:05.158 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:05.155+0000 7fd5e77fe640 1 --2- 172.21.15.92:0/885482008 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fd5d006ae70 0x7fd5d006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:05.159 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:05.155+0000 7fd5e77fe640 1 -- 172.21.15.92:0/885482008 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd5e00aafe0 con 0x7fd5dc003970 2023-11-24T06:11:05.159 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:05.156+0000 7fd5e7fff640 1 --2- 172.21.15.92:0/885482008 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fd5d006ae70 0x7fd5d006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:05.159 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:05.156+0000 7fd5e7fff640 1 --2- 172.21.15.92:0/885482008 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fd5d006ae70 0x7fd5d006d330 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fd5f015d5c0 tx=0x7fd5d8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:05.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:05.160+0000 7fd5e77fe640 1 -- 172.21.15.92:0/885482008 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd5e0076120 con 0x7fd5dc003970 2023-11-24T06:11:05.355 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:05.351+0000 7fd5f50bb640 1 -- 172.21.15.92:0/885482008 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd5f00cfa90 con 0x7fd5dc003970 2023-11-24T06:11:05.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:05.354+0000 7fd5e77fe640 1 -- 172.21.15.92:0/885482008 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16457 (secure 0 0 0) 0x7fd5e002de60 con 0x7fd5dc003970 2023-11-24T06:11:05.358 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-24T06:11:05.358 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":20,"fsid":"2d6dee5f-8aa3-470f-931a-50ade963b1cb","created":"2023-11-24T06:10:04.707204+0000","modified":"2023-11-24T06:10:26.698733+0000","last_up_change":"2023-11-24T06:10:17.598352+0000","last_in_change":"2023-11-24T06:10:09.040793+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:10:17.733098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:10:22.926445+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T06:10:24.151288+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ff37a9fb-135f-471f-b4b0-f7b5772006c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6803","nonce":20560714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6805","nonce":20560714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6809","nonce":20560714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6807","nonce":20560714}]},"public_addr":"172.21.15.92:6803/20560714","cluster_addr":"172.21.15.92:6805/20560714","heartbeat_back_addr":"172.21.15.92:6809/20560714","heartbeat_front_addr":"172.21.15.92:6807/20560714","state":["exists","up"]},{"osd":1,"uuid":"03cdd353-dcf1-4f38-94fe-a795137c06b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6819","nonce":2451239906}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6821","nonce":2451239906}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6825","nonce":2451239906}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6823","nonce":2451239906}]},"public_addr":"172.21.15.92:6819/2451239906","cluster_addr":"172.21.15.92:6821/2451239906","heartbeat_back_addr":"172.21.15.92:6825/2451239906","heartbeat_front_addr":"172.21.15.92:6823/2451239906","state":["exists","up"]},{"osd":2,"uuid":"a4a20d80-5d57-484b-becd-9aa8070dd3aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6811","nonce":1124979277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6813","nonce":1124979277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6817","nonce":1124979277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6815","nonce":1124979277}]},"public_addr":"172.21.15.92:6811/1124979277","cluster_addr":"172.21.15.92:6813/1124979277","heartbeat_back_addr":"172.21.15.92:6817/1124979277","heartbeat_front_addr":"172.21.15.92:6815/1124979277","state":["exists","up"]},{"osd":3,"uuid":"4648d7ce-9b29-4583-9425-4ba8fd2b1903","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6827","nonce":1734115782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6829","nonce":1734115782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6833","nonce":1734115782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6831","nonce":1734115782}]},"public_addr":"172.21.15.92:6827/1734115782","cluster_addr":"172.21.15.92:6829/1734115782","heartbeat_back_addr":"172.21.15.92:6833/1734115782","heartbeat_front_addr":"172.21.15.92:6831/1734115782","state":["exists","up"]},{"osd":4,"uuid":"3bc9d7eb-c6fa-4fcf-a3ec-ade604e05aac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6819","nonce":3425937229}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6822","nonce":3425937229}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6830","nonce":3425937229}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6826","nonce":3425937229}]},"public_addr":"172.21.15.118:6819/3425937229","cluster_addr":"172.21.15.118:6822/3425937229","heartbeat_back_addr":"172.21.15.118:6830/3425937229","heartbeat_front_addr":"172.21.15.118:6826/3425937229","state":["exists","up"]},{"osd":5,"uuid":"dce9ba6d-59ed-46d1-84fc-a84b13de10c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6801","nonce":827455984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6803","nonce":827455984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6807","nonce":827455984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6805","nonce":827455984}]},"public_addr":"172.21.15.118:6801/827455984","cluster_addr":"172.21.15.118:6803/827455984","heartbeat_back_addr":"172.21.15.118:6807/827455984","heartbeat_front_addr":"172.21.15.118:6805/827455984","state":["exists","up"]},{"osd":6,"uuid":"82aec504-ae58-4709-8d30-46f2b12081b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6809","nonce":2946986141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6811","nonce":2946986141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6815","nonce":2946986141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6813","nonce":2946986141}]},"public_addr":"172.21.15.118:6809/2946986141","cluster_addr":"172.21.15.118:6811/2946986141","heartbeat_back_addr":"172.21.15.118:6815/2946986141","heartbeat_front_addr":"172.21.15.118:6813/2946986141","state":["exists","up"]},{"osd":7,"uuid":"4ed4c624-c88f-4d35-bdb1-bf209b5dffb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6817","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6818","nonce":48122059}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6821","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6823","nonce":48122059}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6829","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6831","nonce":48122059}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6825","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6827","nonce":48122059}]},"public_addr":"172.21.15.118:6818/48122059","cluster_addr":"172.21.15.118:6823/48122059","heartbeat_back_addr":"172.21.15.118:6831/48122059","heartbeat_front_addr":"172.21.15.118:6827/48122059","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.536664+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.368324+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.271201+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.342971+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.621832+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.387110+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.632946+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.416032+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:11:05.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:05.357+0000 7fd5f50bb640 1 -- 172.21.15.92:0/885482008 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fd5d006ae70 msgr2=0x7fd5d006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:05.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:05.357+0000 7fd5f50bb640 1 --2- 172.21.15.92:0/885482008 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fd5d006ae70 0x7fd5d006d330 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fd5f015d5c0 tx=0x7fd5d8008040 comp rx=0 tx=0).stop 2023-11-24T06:11:05.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:05.357+0000 7fd5f50bb640 1 -- 172.21.15.92:0/885482008 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd5dc003970 msgr2=0x7fd5f014a970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:05.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:05.357+0000 7fd5f50bb640 1 --2- 172.21.15.92:0/885482008 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd5dc003970 0x7fd5f014a970 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fd5e000c280 tx=0x7fd5e0000fe0 comp rx=0 tx=0).stop 2023-11-24T06:11:05.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:05.357+0000 7fd5f50bb640 1 -- 172.21.15.92:0/885482008 shutdown_connections 2023-11-24T06:11:05.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:05.357+0000 7fd5f50bb640 1 --2- 172.21.15.92:0/885482008 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fd5d006ae70 0x7fd5d006d330 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:05.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:05.357+0000 7fd5f50bb640 1 --2- 172.21.15.92:0/885482008 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd5f0150980 0x7fd5f0150e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:05.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:05.357+0000 7fd5f50bb640 1 --2- 172.21.15.92:0/885482008 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd5f014d980 0x7fd5f01336c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:05.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:05.357+0000 7fd5f50bb640 1 --2- 172.21.15.92:0/885482008 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd5dc003970 0x7fd5f014a970 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:05.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:05.357+0000 7fd5f50bb640 1 -- 172.21.15.92:0/885482008 >> 172.21.15.92:0/885482008 conn(0x7fd5f006a800 msgr2=0x7fd5f015cba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:05.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:05.357+0000 7fd5f50bb640 1 -- 172.21.15.92:0/885482008 shutdown_connections 2023-11-24T06:11:05.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:05.357+0000 7fd5f50bb640 1 -- 172.21.15.92:0/885482008 wait complete. 2023-11-24T06:11:05.380 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T06:10:17.733098+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T06:10:22.926445+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T06:10:24.151288+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T06:11:05.381 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T06:11:05.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:05.583+0000 7f3c5bfff640 1 Processor -- start 2023-11-24T06:11:05.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:05.583+0000 7f3c5bfff640 1 -- start start 2023-11-24T06:11:05.589 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:05.585+0000 7f3c5bfff640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f3c5c143810 0x7f3c5c071e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:05.589 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:05.585+0000 7f3c5bfff640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3c5c15d770 0x7f3c5c15dc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:05.589 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:05.585+0000 7f3c5bfff640 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c5c0ae620 con 0x7f3c5c0ae170 2023-11-24T06:11:05.589 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:05.585+0000 7f3c5bfff640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c5c143bf0 con 0x7f3c5c143810 2023-11-24T06:11:05.589 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:05.585+0000 7f3c5bfff640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c5c15e140 con 0x7f3c5c15d770 2023-11-24T06:11:05.589 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:05.586+0000 7f3c5affd640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f3c5c143810 0x7f3c5c071e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:05.589 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:05.586+0000 7f3c5affd640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f3c5c143810 0x7f3c5c071e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53274/0 (socket says 172.21.15.92:53274) 2023-11-24T06:11:05.589 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:05.586+0000 7f3c5affd640 1 -- 172.21.15.92:0/298317757 learned_addr learned my addr 172.21.15.92:0/298317757 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:11:05.590 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:05.586+0000 7f3c5b7fe640 1 --2- 172.21.15.92:0/298317757 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3c5c15d770 0x7f3c5c15dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:05.590 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:05.586+0000 7f3c5affd640 1 -- 172.21.15.92:0/298317757 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3c5c15d770 msgr2=0x7f3c5c15dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:05.590 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:05.586+0000 7f3c5affd640 1 --2- 172.21.15.92:0/298317757 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3c5c15d770 0x7f3c5c15dc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:05.590 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:05.586+0000 7f3c5affd640 1 -- 172.21.15.92:0/298317757 >> v1:172.21.15.118:6789/0 conn(0x7f3c5c0ae170 legacy=0x7f3c5c15e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:05.590 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:05.586+0000 7f3c5affd640 1 -- 172.21.15.92:0/298317757 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c5c06c730 con 0x7f3c5c143810 2023-11-24T06:11:05.590 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:05.587+0000 7f3c5affd640 1 --2- 172.21.15.92:0/298317757 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3c5c143810 0x7f3c5c071e40 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f3c4c009870 tx=0x7f3c4c02b7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0595895d22b7181 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:05.591 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:05.587+0000 7f3c5a7fc640 1 -- 172.21.15.92:0/298317757 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c4c031740 con 0x7f3c5c143810 2023-11-24T06:11:05.591 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:05.588+0000 7f3c5a7fc640 1 -- 172.21.15.92:0/298317757 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c4c0318a0 con 0x7f3c5c143810 2023-11-24T06:11:05.592 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:05.588+0000 7f3c5a7fc640 1 -- 172.21.15.92:0/298317757 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c4c031c30 con 0x7f3c5c143810 2023-11-24T06:11:05.592 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:05.588+0000 7f3c5bfff640 1 -- 172.21.15.92:0/298317757 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3c5c143810 msgr2=0x7f3c5c071e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:05.592 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:05.588+0000 7f3c5bfff640 1 --2- 172.21.15.92:0/298317757 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3c5c143810 0x7f3c5c071e40 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f3c4c009870 tx=0x7f3c4c02b7e0 comp rx=0 tx=0).stop 2023-11-24T06:11:05.592 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:05.588+0000 7f3c5bfff640 1 -- 172.21.15.92:0/298317757 shutdown_connections 2023-11-24T06:11:05.592 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:05.588+0000 7f3c5bfff640 1 --2- 172.21.15.92:0/298317757 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3c5c15d770 0x7f3c5c15dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:05.592 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:05.588+0000 7f3c5bfff640 1 --2- 172.21.15.92:0/298317757 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3c5c143810 0x7f3c5c071e40 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:05.592 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:05.588+0000 7f3c5bfff640 1 -- 172.21.15.92:0/298317757 >> 172.21.15.92:0/298317757 conn(0x7f3c5c06a7d0 msgr2=0x7f3c5c06abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:05.592 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:05.588+0000 7f3c5bfff640 1 -- 172.21.15.92:0/298317757 shutdown_connections 2023-11-24T06:11:05.592 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:05.588+0000 7f3c5bfff640 1 -- 172.21.15.92:0/298317757 wait complete. 2023-11-24T06:11:05.593 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:05.590+0000 7f3c5bfff640 1 Processor -- start 2023-11-24T06:11:05.593 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:05.590+0000 7f3c5bfff640 1 -- start start 2023-11-24T06:11:05.594 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:05.591+0000 7f3c5bfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3c5c0ae170 0x7f3c5c13dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:05.595 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:05.591+0000 7f3c5bfff640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3c5c143810 0x7f3c5c0b2bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:05.595 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:05.591+0000 7f3c5bfff640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3c5c15d770 0x7f3c5c0b30f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:05.595 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:05.591+0000 7f3c5bfff640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c5c13cfe0 con 0x7f3c5c0ae170 2023-11-24T06:11:05.595 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:05.591+0000 7f3c5bfff640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c5c1f6ee0 con 0x7f3c5c143810 2023-11-24T06:11:05.595 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:05.591+0000 7f3c5bfff640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c5c1f71c0 con 0x7f3c5c15d770 2023-11-24T06:11:05.595 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:05.591+0000 7f3c5affd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3c5c0ae170 0x7f3c5c13dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:05.595 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:05.591+0000 7f3c5affd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3c5c0ae170 0x7f3c5c13dfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53278/0 (socket says 172.21.15.92:53278) 2023-11-24T06:11:05.595 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:05.591+0000 7f3c5affd640 1 -- 172.21.15.92:0/2397575343 learned_addr learned my addr 172.21.15.92:0/2397575343 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:11:05.596 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:05.592+0000 7f3c5affd640 1 -- 172.21.15.92:0/2397575343 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3c5c15d770 msgr2=0x7f3c5c0b30f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:05.596 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:05.592+0000 7f3c5affd640 1 --2- 172.21.15.92:0/2397575343 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3c5c15d770 0x7f3c5c0b30f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:05.597 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:05.592+0000 7f3c5affd640 1 -- 172.21.15.92:0/2397575343 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3c5c143810 msgr2=0x7f3c5c0b2bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:05.597 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:05.592+0000 7f3c527fc640 1 --2- 172.21.15.92:0/2397575343 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3c5c143810 0x7f3c5c0b2bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:05.597 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:05.592+0000 7f3c5affd640 1 --2- 172.21.15.92:0/2397575343 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3c5c143810 0x7f3c5c0b2bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:05.597 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:05.592+0000 7f3c5affd640 1 -- 172.21.15.92:0/2397575343 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c4c0094d0 con 0x7f3c5c0ae170 2023-11-24T06:11:05.597 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:05.592+0000 7f3c527fc640 1 --2- 172.21.15.92:0/2397575343 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3c5c143810 0x7f3c5c0b2bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:11:05.597 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:05.592+0000 7f3c5affd640 1 --2- 172.21.15.92:0/2397575343 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3c5c0ae170 0x7f3c5c13dfc0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f3c4c009870 tx=0x7f3c4c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:05.597 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:05.593+0000 7f3c53fff640 1 -- 172.21.15.92:0/2397575343 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c4c033d60 con 0x7f3c5c0ae170 2023-11-24T06:11:05.597 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:05.593+0000 7f3c53fff640 1 -- 172.21.15.92:0/2397575343 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c4c03c400 con 0x7f3c5c0ae170 2023-11-24T06:11:05.597 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:05.593+0000 7f3c5bfff640 1 -- 172.21.15.92:0/2397575343 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c5c1f74a0 con 0x7f3c5c0ae170 2023-11-24T06:11:05.597 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:05.593+0000 7f3c53fff640 1 -- 172.21.15.92:0/2397575343 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c4c03c7b0 con 0x7f3c5c0ae170 2023-11-24T06:11:05.597 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:05.593+0000 7f3c5bfff640 1 -- 172.21.15.92:0/2397575343 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c5c1f7ca0 con 0x7f3c5c0ae170 2023-11-24T06:11:05.598 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:05.594+0000 7f3c5bfff640 1 -- 172.21.15.92:0/2397575343 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c5c1f79c0 con 0x7f3c5c0ae170 2023-11-24T06:11:05.598 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:05.595+0000 7f3c53fff640 1 -- 172.21.15.92:0/2397575343 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f3c4c03a040 con 0x7f3c5c0ae170 2023-11-24T06:11:05.599 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:05.596+0000 7f3c53fff640 1 --2- 172.21.15.92:0/2397575343 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f3c3c06ae40 0x7f3c3c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:05.599 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:05.596+0000 7f3c53fff640 1 -- 172.21.15.92:0/2397575343 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3c4c0ab370 con 0x7f3c5c0ae170 2023-11-24T06:11:05.599 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:05.596+0000 7f3c527fc640 1 --2- 172.21.15.92:0/2397575343 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f3c3c06ae40 0x7f3c3c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:05.600 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:05.597+0000 7f3c527fc640 1 --2- 172.21.15.92:0/2397575343 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f3c3c06ae40 0x7f3c3c06d300 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f3c5c0b0890 tx=0x7f3c44009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:05.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:05.600+0000 7f3c53fff640 1 -- 172.21.15.92:0/2397575343 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3c5c1f79c0 con 0x7f3c5c0ae170 2023-11-24T06:11:05.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:05.798+0000 7f3c5bfff640 1 -- 172.21.15.92:0/2397575343 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3c5c1f79c0 con 0x7f3c5c0ae170 2023-11-24T06:11:05.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:05.799+0000 7f3c53fff640 1 -- 172.21.15.92:0/2397575343 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3c5c1f79c0 con 0x7f3c5c0ae170 2023-11-24T06:11:05.802 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 1 2023-11-24T06:11:05.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:05.801+0000 7f3c5bfff640 1 -- 172.21.15.92:0/2397575343 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f3c3c06ae40 msgr2=0x7f3c3c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:05.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:05.801+0000 7f3c5bfff640 1 --2- 172.21.15.92:0/2397575343 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f3c3c06ae40 0x7f3c3c06d300 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f3c5c0b0890 tx=0x7f3c44009040 comp rx=0 tx=0).stop 2023-11-24T06:11:05.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:05.801+0000 7f3c5bfff640 1 -- 172.21.15.92:0/2397575343 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3c5c0ae170 msgr2=0x7f3c5c13dfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:05.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:05.801+0000 7f3c5bfff640 1 --2- 172.21.15.92:0/2397575343 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3c5c0ae170 0x7f3c5c13dfc0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f3c4c009870 tx=0x7f3c4c0046c0 comp rx=0 tx=0).stop 2023-11-24T06:11:05.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:05.801+0000 7f3c5bfff640 1 -- 172.21.15.92:0/2397575343 shutdown_connections 2023-11-24T06:11:05.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:05.801+0000 7f3c5bfff640 1 --2- 172.21.15.92:0/2397575343 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f3c3c06ae40 0x7f3c3c06d300 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:05.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:05.801+0000 7f3c5bfff640 1 --2- 172.21.15.92:0/2397575343 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3c5c15d770 0x7f3c5c0b30f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:05.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:05.801+0000 7f3c5bfff640 1 --2- 172.21.15.92:0/2397575343 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3c5c143810 0x7f3c5c0b2bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:05.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:05.801+0000 7f3c5bfff640 1 --2- 172.21.15.92:0/2397575343 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3c5c0ae170 0x7f3c5c13dfc0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:05.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:05.801+0000 7f3c5bfff640 1 -- 172.21.15.92:0/2397575343 >> 172.21.15.92:0/2397575343 conn(0x7f3c5c06a7d0 msgr2=0x7f3c5c15fa50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:05.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:05.801+0000 7f3c5bfff640 1 -- 172.21.15.92:0/2397575343 shutdown_connections 2023-11-24T06:11:05.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:05.802+0000 7f3c5bfff640 1 -- 172.21.15.92:0/2397575343 wait complete. 2023-11-24T06:11:05.821 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T06:11:06.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.026+0000 7f29209db640 1 Processor -- start 2023-11-24T06:11:06.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.026+0000 7f29209db640 1 -- start start 2023-11-24T06:11:06.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.028+0000 7f29209db640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f291c15d4b0 0x7f291c0d1550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:06.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.028+0000 7f29209db640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f291c0cf9a0 0x7f291c0cfde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:06.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.028+0000 7f29209db640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f291c0cddf0 0x7f291c0ce280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:06.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.028+0000 7f29209db640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f291c0d0320 con 0x7f291c0cf9a0 2023-11-24T06:11:06.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.028+0000 7f29209db640 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f291c0b2730 con 0x7f291c15d4b0 2023-11-24T06:11:06.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.028+0000 7f29209db640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f291c0d1ec0 con 0x7f291c0cddf0 2023-11-24T06:11:06.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.028+0000 7f291a59c640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f291c0cf9a0 0x7f291c0cfde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:06.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.028+0000 7f291a59c640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f291c0cf9a0 0x7f291c0cfde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53286/0 (socket says 172.21.15.92:53286) 2023-11-24T06:11:06.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.028+0000 7f291a59c640 1 -- 172.21.15.92:0/3045126902 learned_addr learned my addr 172.21.15.92:0/3045126902 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:11:06.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.028+0000 7f291b59e640 1 --2- 172.21.15.92:0/3045126902 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f291c0cddf0 0x7f291c0ce280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:06.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.028+0000 7f291a59c640 1 -- 172.21.15.92:0/3045126902 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f291c0cddf0 msgr2=0x7f291c0ce280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:06.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.028+0000 7f291ad9d640 1 --2- 172.21.15.92:0/3045126902 >> v2:172.21.15.118:3300/0 conn(0x7f291c15d4b0 0x7f291c0d1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:06.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.029+0000 7f291a59c640 1 --2- 172.21.15.92:0/3045126902 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f291c0cddf0 0x7f291c0ce280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:06.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.029+0000 7f291a59c640 1 -- 172.21.15.92:0/3045126902 >> v2:172.21.15.118:3300/0 conn(0x7f291c15d4b0 msgr2=0x7f291c0d1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:06.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.029+0000 7f291a59c640 1 --2- 172.21.15.92:0/3045126902 >> v2:172.21.15.118:3300/0 conn(0x7f291c15d4b0 0x7f291c0d1550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:06.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.029+0000 7f291a59c640 1 -- 172.21.15.92:0/3045126902 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f291c0b29d0 con 0x7f291c0cf9a0 2023-11-24T06:11:06.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.029+0000 7f291ad9d640 1 --2- 172.21.15.92:0/3045126902 >> v2:172.21.15.118:3300/0 conn(0x7f291c15d4b0 0x7f291c0d1550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:11:06.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.030+0000 7f291a59c640 1 --2- 172.21.15.92:0/3045126902 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f291c0cf9a0 0x7f291c0cfde0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f291c08b5b0 tx=0x7f290402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef7e593c623388c9 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:06.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.030+0000 7f2919d9b640 1 -- 172.21.15.92:0/3045126902 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2904031870 con 0x7f291c0cf9a0 2023-11-24T06:11:06.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.030+0000 7f2919d9b640 1 -- 172.21.15.92:0/3045126902 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29040319d0 con 0x7f291c0cf9a0 2023-11-24T06:11:06.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.030+0000 7f2919d9b640 1 -- 172.21.15.92:0/3045126902 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2904031d60 con 0x7f291c0cf9a0 2023-11-24T06:11:06.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.031+0000 7f29209db640 1 -- 172.21.15.92:0/3045126902 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f291c0cf9a0 msgr2=0x7f291c0cfde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:06.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.031+0000 7f29209db640 1 --2- 172.21.15.92:0/3045126902 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f291c0cf9a0 0x7f291c0cfde0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f291c08b5b0 tx=0x7f290402b910 comp rx=0 tx=0).stop 2023-11-24T06:11:06.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.031+0000 7f29209db640 1 -- 172.21.15.92:0/3045126902 shutdown_connections 2023-11-24T06:11:06.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.031+0000 7f29209db640 1 --2- 172.21.15.92:0/3045126902 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f291c0cddf0 0x7f291c0ce280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:06.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.031+0000 7f29209db640 1 --2- 172.21.15.92:0/3045126902 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f291c0cf9a0 0x7f291c0cfde0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:06.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.031+0000 7f29209db640 1 --2- 172.21.15.92:0/3045126902 >> v2:172.21.15.118:3300/0 conn(0x7f291c15d4b0 0x7f291c0d1550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:06.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.031+0000 7f29209db640 1 -- 172.21.15.92:0/3045126902 >> 172.21.15.92:0/3045126902 conn(0x7f291c0d3000 msgr2=0x7f291c0d3410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:06.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.031+0000 7f29209db640 1 -- 172.21.15.92:0/3045126902 shutdown_connections 2023-11-24T06:11:06.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.031+0000 7f29209db640 1 -- 172.21.15.92:0/3045126902 wait complete. 2023-11-24T06:11:06.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.032+0000 7f29209db640 1 Processor -- start 2023-11-24T06:11:06.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.033+0000 7f29209db640 1 -- start start 2023-11-24T06:11:06.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.033+0000 7f29209db640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f291c0cddf0 0x7f291c148ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:06.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.033+0000 7f29209db640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f291c0cf9a0 0x7f291c14a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:06.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.033+0000 7f29209db640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f291c15d4b0 0x7f291c1f5ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:06.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.033+0000 7f29209db640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f291c149530 con 0x7f291c0cf9a0 2023-11-24T06:11:06.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.033+0000 7f29209db640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f291c14a330 con 0x7f291c0cddf0 2023-11-24T06:11:06.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.033+0000 7f29209db640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f291c149720 con 0x7f291c15d4b0 2023-11-24T06:11:06.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.034+0000 7f291a59c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f291c0cf9a0 0x7f291c14a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:06.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.034+0000 7f291a59c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f291c0cf9a0 0x7f291c14a7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53300/0 (socket says 172.21.15.92:53300) 2023-11-24T06:11:06.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.034+0000 7f291a59c640 1 -- 172.21.15.92:0/1285945062 learned_addr learned my addr 172.21.15.92:0/1285945062 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:11:06.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.034+0000 7f291a59c640 1 -- 172.21.15.92:0/1285945062 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f291c15d4b0 msgr2=0x7f291c1f5ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:06.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.034+0000 7f291a59c640 1 --2- 172.21.15.92:0/1285945062 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f291c15d4b0 0x7f291c1f5ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:06.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.034+0000 7f291a59c640 1 -- 172.21.15.92:0/1285945062 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f291c0cddf0 msgr2=0x7f291c148ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:06.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.034+0000 7f291a59c640 1 --2- 172.21.15.92:0/1285945062 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f291c0cddf0 0x7f291c148ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:06.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.034+0000 7f291a59c640 1 -- 172.21.15.92:0/1285945062 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29040094d0 con 0x7f291c0cf9a0 2023-11-24T06:11:06.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.034+0000 7f291a59c640 1 --2- 172.21.15.92:0/1285945062 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f291c0cf9a0 0x7f291c14a7c0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f2904000c00 tx=0x7f29040046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:06.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.035+0000 7f29137fe640 1 -- 172.21.15.92:0/1285945062 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2904031820 con 0x7f291c0cf9a0 2023-11-24T06:11:06.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.035+0000 7f29137fe640 1 -- 172.21.15.92:0/1285945062 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29040443f0 con 0x7f291c0cf9a0 2023-11-24T06:11:06.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.035+0000 7f29137fe640 1 -- 172.21.15.92:0/1285945062 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29040447a0 con 0x7f291c0cf9a0 2023-11-24T06:11:06.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.035+0000 7f29209db640 1 -- 172.21.15.92:0/1285945062 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f291c15da10 con 0x7f291c0cf9a0 2023-11-24T06:11:06.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.035+0000 7f29209db640 1 -- 172.21.15.92:0/1285945062 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f291c149ce0 con 0x7f291c0cf9a0 2023-11-24T06:11:06.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.036+0000 7f29209db640 1 -- 172.21.15.92:0/1285945062 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f291c067450 con 0x7f291c0cf9a0 2023-11-24T06:11:06.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.037+0000 7f29137fe640 1 -- 172.21.15.92:0/1285945062 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f290404d450 con 0x7f291c0cf9a0 2023-11-24T06:11:06.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.037+0000 7f29137fe640 1 --2- 172.21.15.92:0/1285945062 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f28fc06ae40 0x7f28fc06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:06.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.038+0000 7f29137fe640 1 -- 172.21.15.92:0/1285945062 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f290402d040 con 0x7f291c0cf9a0 2023-11-24T06:11:06.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.038+0000 7f291ad9d640 1 --2- 172.21.15.92:0/1285945062 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f28fc06ae40 0x7f28fc06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:06.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.038+0000 7f291ad9d640 1 --2- 172.21.15.92:0/1285945062 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f28fc06ae40 0x7f28fc06d300 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f290c006fd0 tx=0x7f290c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:06.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.042+0000 7f29137fe640 1 -- 172.21.15.92:0/1285945062 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2904076e50 con 0x7f291c0cf9a0 2023-11-24T06:11:06.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.226+0000 7f29209db640 1 -- 172.21.15.92:0/1285945062 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f291c14a470 con 0x7f291c0cf9a0 2023-11-24T06:11:06.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.227+0000 7f29137fe640 1 -- 172.21.15.92:0/1285945062 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f29040350a0 con 0x7f291c0cf9a0 2023-11-24T06:11:06.230 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 32 2023-11-24T06:11:06.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.230+0000 7f29209db640 1 -- 172.21.15.92:0/1285945062 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f28fc06ae40 msgr2=0x7f28fc06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:06.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.230+0000 7f29209db640 1 --2- 172.21.15.92:0/1285945062 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f28fc06ae40 0x7f28fc06d300 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f290c006fd0 tx=0x7f290c008040 comp rx=0 tx=0).stop 2023-11-24T06:11:06.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.230+0000 7f29209db640 1 -- 172.21.15.92:0/1285945062 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f291c0cf9a0 msgr2=0x7f291c14a7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:06.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.230+0000 7f29209db640 1 --2- 172.21.15.92:0/1285945062 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f291c0cf9a0 0x7f291c14a7c0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f2904000c00 tx=0x7f29040046c0 comp rx=0 tx=0).stop 2023-11-24T06:11:06.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.230+0000 7f29209db640 1 -- 172.21.15.92:0/1285945062 shutdown_connections 2023-11-24T06:11:06.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.230+0000 7f29209db640 1 --2- 172.21.15.92:0/1285945062 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f28fc06ae40 0x7f28fc06d300 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:06.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.230+0000 7f29209db640 1 --2- 172.21.15.92:0/1285945062 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f291c15d4b0 0x7f291c1f5ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:06.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.230+0000 7f29209db640 1 --2- 172.21.15.92:0/1285945062 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f291c0cf9a0 0x7f291c14a7c0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:06.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.230+0000 7f29209db640 1 --2- 172.21.15.92:0/1285945062 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f291c0cddf0 0x7f291c148ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:06.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.230+0000 7f29209db640 1 -- 172.21.15.92:0/1285945062 >> 172.21.15.92:0/1285945062 conn(0x7f291c0d3000 msgr2=0x7f291c06b1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:06.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.230+0000 7f29209db640 1 -- 172.21.15.92:0/1285945062 shutdown_connections 2023-11-24T06:11:06.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.230+0000 7f29209db640 1 -- 172.21.15.92:0/1285945062 wait complete. 2023-11-24T06:11:06.251 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T06:11:06.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.457+0000 7f1331c23640 1 Processor -- start 2023-11-24T06:11:06.666 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.458+0000 7f1331c23640 1 -- start start 2023-11-24T06:11:06.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.459+0000 7f1331c23640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f132c13e360 0x7f132c0cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:06.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.459+0000 7f1331c23640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f132c15c5f0 con 0x7f132c13df80 2023-11-24T06:11:06.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.459+0000 7f1331c23640 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f132c06c470 con 0x7f132c15f9c0 2023-11-24T06:11:06.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.459+0000 7f1331c23640 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f132c06c750 con 0x7f132c13e360 2023-11-24T06:11:06.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.459+0000 7f132b7fe640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f132c13df80 0x7f132c15bee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:35606/0 (socket says 172.21.15.92:35606) 2023-11-24T06:11:06.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.459+0000 7f132b7fe640 1 -- 172.21.15.92:0/786886800 learned_addr learned my addr 172.21.15.92:0/786886800 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-24T06:11:06.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.459+0000 7f132affd640 1 -- 172.21.15.92:0/786886800 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2237139884 0 0) 0x7f132c15c5f0 con 0x7f132c13df80 2023-11-24T06:11:06.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.459+0000 7f132affd640 1 -- 172.21.15.92:0/786886800 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1318003520 con 0x7f132c13df80 2023-11-24T06:11:06.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.460+0000 7f1322ffd640 1 --2- 172.21.15.92:0/786886800 >> v2:172.21.15.118:3300/0 conn(0x7f132c13e360 0x7f132c0cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:06.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.460+0000 7f132affd640 1 -- 172.21.15.92:0/786886800 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2263767818 0 0) 0x7f1318003520 con 0x7f132c13df80 2023-11-24T06:11:06.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.460+0000 7f132affd640 1 -- 172.21.15.92:0/786886800 >> v2:172.21.15.118:3300/0 conn(0x7f132c13e360 msgr2=0x7f132c0cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:06.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.460+0000 7f132affd640 1 --2- 172.21.15.92:0/786886800 >> v2:172.21.15.118:3300/0 conn(0x7f132c13e360 0x7f132c0cf290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:06.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.460+0000 7f132affd640 1 -- 172.21.15.92:0/786886800 >> v1:172.21.15.118:6789/0 conn(0x7f132c15f9c0 legacy=0x7f132c0cd840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:06.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.460+0000 7f132affd640 1 -- 172.21.15.92:0/786886800 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f132c06ca30 con 0x7f132c13df80 2023-11-24T06:11:06.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.460+0000 7f132affd640 1 -- 172.21.15.92:0/786886800 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2607661350 0 0) 0x7f131c001690 con 0x7f132c13df80 2023-11-24T06:11:06.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.460+0000 7f132affd640 1 -- 172.21.15.92:0/786886800 >> v1:172.21.15.92:6789/0 conn(0x7f132c13df80 legacy=0x7f132c15bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:06.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.460+0000 7f132affd640 1 --2- 172.21.15.92:0/786886800 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1318003970 0x7f1318003d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:06.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.460+0000 7f132affd640 1 --2- 172.21.15.92:0/786886800 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f13180047c0 0x7f1318004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:06.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.460+0000 7f132affd640 1 --2- 172.21.15.92:0/786886800 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1318005180 0x7f131800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:06.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.460+0000 7f132affd640 1 -- 172.21.15.92:0/786886800 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f132c06c750 con 0x7f1318005180 2023-11-24T06:11:06.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.460+0000 7f132affd640 1 -- 172.21.15.92:0/786886800 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f131800c0b0 con 0x7f13180047c0 2023-11-24T06:11:06.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.460+0000 7f132affd640 1 -- 172.21.15.92:0/786886800 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f131800c270 con 0x7f1318003970 2023-11-24T06:11:06.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.460+0000 7f132bfff640 1 --2- 172.21.15.92:0/786886800 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1318005180 0x7f131800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:06.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.461+0000 7f1322ffd640 1 --2- 172.21.15.92:0/786886800 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f13180047c0 0x7f1318004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:06.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.461+0000 7f132bfff640 1 -- 172.21.15.92:0/786886800 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1318003970 msgr2=0x7f1318003d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:06.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.461+0000 7f132b7fe640 1 --2- 172.21.15.92:0/786886800 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1318003970 0x7f1318003d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:06.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.461+0000 7f132bfff640 1 --2- 172.21.15.92:0/786886800 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1318003970 0x7f1318003d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:06.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.461+0000 7f132bfff640 1 -- 172.21.15.92:0/786886800 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f13180047c0 msgr2=0x7f1318004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:06.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.461+0000 7f132bfff640 1 --2- 172.21.15.92:0/786886800 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f13180047c0 0x7f1318004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:06.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.461+0000 7f132bfff640 1 -- 172.21.15.92:0/786886800 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1318003520 con 0x7f1318005180 2023-11-24T06:11:06.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.462+0000 7f132bfff640 1 -- 172.21.15.92:0/786886800 reap_dead start 2023-11-24T06:11:06.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.462+0000 7f132bfff640 1 --2- 172.21.15.92:0/786886800 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1318005180 0x7f131800b980 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f132c15b6f0 tx=0x7f13180057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50e9ff57a14eef63 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:06.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.462+0000 7f132affd640 1 -- 172.21.15.92:0/786886800 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1324032600 con 0x7f1318005180 2023-11-24T06:11:06.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.462+0000 7f132affd640 1 -- 172.21.15.92:0/786886800 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f132402e050 con 0x7f1318005180 2023-11-24T06:11:06.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.462+0000 7f132affd640 1 -- 172.21.15.92:0/786886800 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1324032980 con 0x7f1318005180 2023-11-24T06:11:06.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.462+0000 7f1331c23640 1 -- 172.21.15.92:0/786886800 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1318005180 msgr2=0x7f131800b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:06.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.462+0000 7f1331c23640 1 --2- 172.21.15.92:0/786886800 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1318005180 0x7f131800b980 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f132c15b6f0 tx=0x7f13180057f0 comp rx=0 tx=0).stop 2023-11-24T06:11:06.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.463+0000 7f1331c23640 1 -- 172.21.15.92:0/786886800 shutdown_connections 2023-11-24T06:11:06.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.463+0000 7f1331c23640 1 --2- 172.21.15.92:0/786886800 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1318005180 0x7f131800b980 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:06.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.463+0000 7f1331c23640 1 -- 172.21.15.92:0/786886800 >> 172.21.15.92:0/786886800 conn(0x7f132c06a800 msgr2=0x7f132c06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:06.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.463+0000 7f1331c23640 1 -- 172.21.15.92:0/786886800 shutdown_connections 2023-11-24T06:11:06.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.463+0000 7f1331c23640 1 -- 172.21.15.92:0/786886800 wait complete. 2023-11-24T06:11:06.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.464+0000 7f1331c23640 1 Processor -- start 2023-11-24T06:11:06.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.465+0000 7f1331c23640 1 -- start start 2023-11-24T06:11:06.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.465+0000 7f1331c23640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1318005180 0x7f132c15a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:06.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.465+0000 7f1331c23640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f132c070470 0x7f132c1f35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:06.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.465+0000 7f1331c23640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f132c1f3b20 0x7f132c1f63f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:06.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.465+0000 7f1331c23640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f132c1f3150 con 0x7f132c1f3b20 2023-11-24T06:11:06.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.465+0000 7f1331c23640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f132c1f2e00 con 0x7f1318005180 2023-11-24T06:11:06.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.465+0000 7f1331c23640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f132c14b4f0 con 0x7f132c070470 2023-11-24T06:11:06.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.466+0000 7f132bfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f132c1f3b20 0x7f132c1f63f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:06.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.466+0000 7f132bfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f132c1f3b20 0x7f132c1f63f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53316/0 (socket says 172.21.15.92:53316) 2023-11-24T06:11:06.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.466+0000 7f132bfff640 1 -- 172.21.15.92:0/3197528967 learned_addr learned my addr 172.21.15.92:0/3197528967 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:11:06.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.466+0000 7f132b7fe640 1 --2- 172.21.15.92:0/3197528967 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1318005180 0x7f132c15a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:06.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.466+0000 7f1322ffd640 1 --2- 172.21.15.92:0/3197528967 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f132c070470 0x7f132c1f35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:06.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.466+0000 7f132bfff640 1 -- 172.21.15.92:0/3197528967 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f132c070470 msgr2=0x7f132c1f35e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:06.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.466+0000 7f132bfff640 1 --2- 172.21.15.92:0/3197528967 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f132c070470 0x7f132c1f35e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:06.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.466+0000 7f132bfff640 1 -- 172.21.15.92:0/3197528967 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1318005180 msgr2=0x7f132c15a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:06.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.466+0000 7f132bfff640 1 --2- 172.21.15.92:0/3197528967 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1318005180 0x7f132c15a3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:06.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.466+0000 7f132bfff640 1 -- 172.21.15.92:0/3197528967 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f132c14b7d0 con 0x7f132c1f3b20 2023-11-24T06:11:06.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.466+0000 7f132bfff640 1 --2- 172.21.15.92:0/3197528967 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f132c1f3b20 0x7f132c1f63f0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f1324000c00 tx=0x7f1324003760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:06.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.467+0000 7f1328ff9640 1 -- 172.21.15.92:0/3197528967 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1324032ea0 con 0x7f132c1f3b20 2023-11-24T06:11:06.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.467+0000 7f1328ff9640 1 -- 172.21.15.92:0/3197528967 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13240069d0 con 0x7f132c1f3b20 2023-11-24T06:11:06.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.467+0000 7f1328ff9640 1 -- 172.21.15.92:0/3197528967 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1324006d40 con 0x7f132c1f3b20 2023-11-24T06:11:06.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.467+0000 7f1331c23640 1 -- 172.21.15.92:0/3197528967 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f132c14bab0 con 0x7f132c1f3b20 2023-11-24T06:11:06.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.467+0000 7f1331c23640 1 -- 172.21.15.92:0/3197528967 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f132c15a8f0 con 0x7f132c1f3b20 2023-11-24T06:11:06.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.468+0000 7f1331c23640 1 -- 172.21.15.92:0/3197528967 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f132c1f3300 con 0x7f132c1f3b20 2023-11-24T06:11:06.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.469+0000 7f1328ff9640 1 -- 172.21.15.92:0/3197528967 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f1324007030 con 0x7f132c1f3b20 2023-11-24T06:11:06.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.469+0000 7f1328ff9640 1 --2- 172.21.15.92:0/3197528967 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f130c06ae40 0x7f130c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:06.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.470+0000 7f1328ff9640 1 -- 172.21.15.92:0/3197528967 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f13240aa6b0 con 0x7f132c1f3b20 2023-11-24T06:11:06.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.470+0000 7f132b7fe640 1 --2- 172.21.15.92:0/3197528967 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f130c06ae40 0x7f130c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:06.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.470+0000 7f132b7fe640 1 --2- 172.21.15.92:0/3197528967 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f130c06ae40 0x7f130c06d300 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f131c001900 tx=0x7f131c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:06.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.474+0000 7f1328ff9640 1 -- 172.21.15.92:0/3197528967 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f132c1f3300 con 0x7f132c1f3b20 2023-11-24T06:11:06.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.647+0000 7f1331c23640 1 -- 172.21.15.92:0/3197528967 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f132c0675d0 con 0x7f132c1f3b20 2023-11-24T06:11:06.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.648+0000 7f1328ff9640 1 -- 172.21.15.92:0/3197528967 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f13240758a0 con 0x7f132c1f3b20 2023-11-24T06:11:06.678 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 64 2023-11-24T06:11:06.680 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.651+0000 7f1331c23640 1 -- 172.21.15.92:0/3197528967 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f130c06ae40 msgr2=0x7f130c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:06.680 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.651+0000 7f1331c23640 1 --2- 172.21.15.92:0/3197528967 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f130c06ae40 0x7f130c06d300 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f131c001900 tx=0x7f131c009040 comp rx=0 tx=0).stop 2023-11-24T06:11:06.680 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.651+0000 7f1331c23640 1 -- 172.21.15.92:0/3197528967 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f132c1f3b20 msgr2=0x7f132c1f63f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:06.680 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.651+0000 7f1331c23640 1 --2- 172.21.15.92:0/3197528967 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f132c1f3b20 0x7f132c1f63f0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f1324000c00 tx=0x7f1324003760 comp rx=0 tx=0).stop 2023-11-24T06:11:06.680 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.651+0000 7f1331c23640 1 -- 172.21.15.92:0/3197528967 shutdown_connections 2023-11-24T06:11:06.680 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.651+0000 7f1331c23640 1 --2- 172.21.15.92:0/3197528967 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f130c06ae40 0x7f130c06d300 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:06.680 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.651+0000 7f1331c23640 1 --2- 172.21.15.92:0/3197528967 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f132c1f3b20 0x7f132c1f63f0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:06.680 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.651+0000 7f1331c23640 1 --2- 172.21.15.92:0/3197528967 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f132c070470 0x7f132c1f35e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:06.680 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.651+0000 7f1331c23640 1 --2- 172.21.15.92:0/3197528967 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1318005180 0x7f132c15a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:06.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.651+0000 7f1331c23640 1 -- 172.21.15.92:0/3197528967 >> 172.21.15.92:0/3197528967 conn(0x7f132c06a800 msgr2=0x7f132c15c850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:06.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.651+0000 7f1331c23640 1 -- 172.21.15.92:0/3197528967 shutdown_connections 2023-11-24T06:11:06.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.651+0000 7f1331c23640 1 -- 172.21.15.92:0/3197528967 wait complete. 2023-11-24T06:11:06.681 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_create' 2023-11-24T06:11:06.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.884+0000 7fea9eb16640 1 Processor -- start 2023-11-24T06:11:06.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.884+0000 7fea9eb16640 1 -- start start 2023-11-24T06:11:06.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.886+0000 7fea9eb16640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fea9813cfa0 0x7fea9815afb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:06.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.886+0000 7fea9eb16640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea980cdec0 con 0x7fea9813fcb0 2023-11-24T06:11:06.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.886+0000 7fea9eb16640 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea980ce1a0 con 0x7fea980cd530 2023-11-24T06:11:06.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.886+0000 7fea9eb16640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea980ce480 con 0x7fea9813cfa0 2023-11-24T06:11:06.891 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.886+0000 7fea9e315640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fea9813fcb0 0x7fea980b2040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:35616/0 (socket says 172.21.15.92:35616) 2023-11-24T06:11:06.891 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.886+0000 7fea9e315640 1 -- 172.21.15.92:0/3576055437 learned_addr learned my addr 172.21.15.92:0/3576055437 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-24T06:11:06.891 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.887+0000 7fea9db14640 1 --2- 172.21.15.92:0/3576055437 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fea9813cfa0 0x7fea9815afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:06.891 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.887+0000 7fea9cb12640 1 -- 172.21.15.92:0/3576055437 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2323494320 0 0) 0x7fea980cdec0 con 0x7fea9813fcb0 2023-11-24T06:11:06.891 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.887+0000 7fea9cb12640 1 -- 172.21.15.92:0/3576055437 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fea84003520 con 0x7fea9813fcb0 2023-11-24T06:11:06.891 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.887+0000 7fea9cb12640 1 -- 172.21.15.92:0/3576055437 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3922681039 0 0) 0x7fea84003520 con 0x7fea9813fcb0 2023-11-24T06:11:06.891 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.887+0000 7fea9cb12640 1 -- 172.21.15.92:0/3576055437 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fea9813cfa0 msgr2=0x7fea9815afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:06.891 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.887+0000 7fea9cb12640 1 --2- 172.21.15.92:0/3576055437 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fea9813cfa0 0x7fea9815afb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:06.892 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.887+0000 7fea9cb12640 1 -- 172.21.15.92:0/3576055437 >> v1:172.21.15.118:6789/0 conn(0x7fea980cd530 legacy=0x7fea9813f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:06.892 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.887+0000 7fea9cb12640 1 -- 172.21.15.92:0/3576055437 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea980ce760 con 0x7fea9813fcb0 2023-11-24T06:11:06.892 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.887+0000 7fea9cb12640 1 -- 172.21.15.92:0/3576055437 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2607661350 0 0) 0x7fea900019f0 con 0x7fea9813fcb0 2023-11-24T06:11:06.892 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.887+0000 7fea9cb12640 1 -- 172.21.15.92:0/3576055437 >> v1:172.21.15.92:6789/0 conn(0x7fea9813fcb0 legacy=0x7fea980b2040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:06.892 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.887+0000 7fea9cb12640 1 --2- 172.21.15.92:0/3576055437 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fea840039a0 0x7fea84003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:06.892 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.887+0000 7fea9cb12640 1 --2- 172.21.15.92:0/3576055437 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fea840047c0 0x7fea84004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:06.892 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.887+0000 7fea9cb12640 1 --2- 172.21.15.92:0/3576055437 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fea9813cfa0 0x7fea84005230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:06.892 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.887+0000 7fea9cb12640 1 -- 172.21.15.92:0/3576055437 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea980ce480 con 0x7fea840047c0 2023-11-24T06:11:06.892 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.887+0000 7fea9cb12640 1 -- 172.21.15.92:0/3576055437 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea8400bb10 con 0x7fea840039a0 2023-11-24T06:11:06.892 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.887+0000 7fea9cb12640 1 -- 172.21.15.92:0/3576055437 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea8400bcd0 con 0x7fea9813cfa0 2023-11-24T06:11:06.892 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.888+0000 7fea9d313640 1 --2- 172.21.15.92:0/3576055437 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fea840047c0 0x7fea84004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:06.892 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.888+0000 7fea9db14640 1 --2- 172.21.15.92:0/3576055437 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fea840039a0 0x7fea84003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:06.892 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.888+0000 7fea9d313640 1 -- 172.21.15.92:0/3576055437 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fea9813cfa0 msgr2=0x7fea84005230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:06.892 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.888+0000 7fea9d313640 1 --2- 172.21.15.92:0/3576055437 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fea9813cfa0 0x7fea84005230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:06.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.888+0000 7fea9d313640 1 -- 172.21.15.92:0/3576055437 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fea840039a0 msgr2=0x7fea84003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:06.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.888+0000 7fea9d313640 1 --2- 172.21.15.92:0/3576055437 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fea840039a0 0x7fea84003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:06.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.888+0000 7fea9d313640 1 -- 172.21.15.92:0/3576055437 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea84003520 con 0x7fea840047c0 2023-11-24T06:11:06.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.889+0000 7fea9d313640 1 --2- 172.21.15.92:0/3576055437 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fea840047c0 0x7fea84004c40 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fea88001750 tx=0x7fea840057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80c8cd4bfb1bcd5 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:06.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.889+0000 7fea9cb12640 1 -- 172.21.15.92:0/3576055437 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea880052d0 con 0x7fea840047c0 2023-11-24T06:11:06.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.889+0000 7fea9cb12640 1 -- 172.21.15.92:0/3576055437 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea8802e050 con 0x7fea840047c0 2023-11-24T06:11:06.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.889+0000 7fea9cb12640 1 -- 172.21.15.92:0/3576055437 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea880035f0 con 0x7fea840047c0 2023-11-24T06:11:06.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.890+0000 7fea9eb16640 1 -- 172.21.15.92:0/3576055437 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fea840047c0 msgr2=0x7fea84004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:06.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.890+0000 7fea9eb16640 1 --2- 172.21.15.92:0/3576055437 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fea840047c0 0x7fea84004c40 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fea88001750 tx=0x7fea840057f0 comp rx=0 tx=0).stop 2023-11-24T06:11:06.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.890+0000 7fea9e315640 1 -- 172.21.15.92:0/3576055437 reap_dead start 2023-11-24T06:11:06.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.890+0000 7fea9eb16640 1 -- 172.21.15.92:0/3576055437 shutdown_connections 2023-11-24T06:11:06.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.890+0000 7fea9eb16640 1 -- 172.21.15.92:0/3576055437 >> 172.21.15.92:0/3576055437 conn(0x7fea9806a800 msgr2=0x7fea9806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:06.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.890+0000 7fea9eb16640 1 -- 172.21.15.92:0/3576055437 shutdown_connections 2023-11-24T06:11:06.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.890+0000 7fea9eb16640 1 -- 172.21.15.92:0/3576055437 wait complete. 2023-11-24T06:11:06.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.892+0000 7fea9eb16640 1 Processor -- start 2023-11-24T06:11:06.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.892+0000 7fea9eb16640 1 -- start start 2023-11-24T06:11:06.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.892+0000 7fea9eb16640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fea840047c0 0x7fea98070ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:06.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.893+0000 7fea9eb16640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fea981493a0 0x7fea98146660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:06.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.893+0000 7fea9eb16640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fea9813a4c0 0x7fea981591a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:06.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.893+0000 7fea9eb16640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea980a9250 con 0x7fea981493a0 2023-11-24T06:11:06.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.893+0000 7fea9eb16640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea980a85d0 con 0x7fea840047c0 2023-11-24T06:11:06.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.893+0000 7fea9eb16640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea980a88b0 con 0x7fea9813a4c0 2023-11-24T06:11:06.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.893+0000 7fea9d313640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fea981493a0 0x7fea98146660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:06.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.893+0000 7fea9d313640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fea981493a0 0x7fea98146660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53342/0 (socket says 172.21.15.92:53342) 2023-11-24T06:11:06.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.893+0000 7fea9d313640 1 -- 172.21.15.92:0/2172714425 learned_addr learned my addr 172.21.15.92:0/2172714425 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:11:06.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.893+0000 7fea9db14640 1 --2- 172.21.15.92:0/2172714425 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fea840047c0 0x7fea98070ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:06.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.893+0000 7fea9d313640 1 -- 172.21.15.92:0/2172714425 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fea9813a4c0 msgr2=0x7fea981591a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:06.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.893+0000 7fea9d313640 1 --2- 172.21.15.92:0/2172714425 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fea9813a4c0 0x7fea981591a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:06.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.893+0000 7fea9d313640 1 -- 172.21.15.92:0/2172714425 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fea840047c0 msgr2=0x7fea98070ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:06.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.893+0000 7fea9d313640 1 --2- 172.21.15.92:0/2172714425 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fea840047c0 0x7fea98070ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:06.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.893+0000 7fea9d313640 1 -- 172.21.15.92:0/2172714425 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea980a8b90 con 0x7fea981493a0 2023-11-24T06:11:06.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.893+0000 7fea9d313640 1 --2- 172.21.15.92:0/2172714425 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fea981493a0 0x7fea98146660 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fea8802bc30 tx=0x7fea88001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:06.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.894+0000 7fea8e7fc640 1 -- 172.21.15.92:0/2172714425 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea880076c0 con 0x7fea981493a0 2023-11-24T06:11:06.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.894+0000 7fea8e7fc640 1 -- 172.21.15.92:0/2172714425 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea88007a30 con 0x7fea981493a0 2023-11-24T06:11:06.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.894+0000 7fea9eb16640 1 -- 172.21.15.92:0/2172714425 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea980a8e70 con 0x7fea981493a0 2023-11-24T06:11:06.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.894+0000 7fea8e7fc640 1 -- 172.21.15.92:0/2172714425 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea88006be0 con 0x7fea981493a0 2023-11-24T06:11:06.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.894+0000 7fea9eb16640 1 -- 172.21.15.92:0/2172714425 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea98146ba0 con 0x7fea981493a0 2023-11-24T06:11:06.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.895+0000 7fea9eb16640 1 -- 172.21.15.92:0/2172714425 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea98067450 con 0x7fea981493a0 2023-11-24T06:11:06.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.896+0000 7fea8e7fc640 1 -- 172.21.15.92:0/2172714425 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fea88006d40 con 0x7fea981493a0 2023-11-24T06:11:06.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.897+0000 7fea8e7fc640 1 --2- 172.21.15.92:0/2172714425 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fea7c06ae40 0x7fea7c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:06.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.897+0000 7fea8e7fc640 1 -- 172.21.15.92:0/2172714425 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fea880ab450 con 0x7fea981493a0 2023-11-24T06:11:06.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.898+0000 7fea9db14640 1 --2- 172.21.15.92:0/2172714425 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fea7c06ae40 0x7fea7c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:06.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.898+0000 7fea9db14640 1 --2- 172.21.15.92:0/2172714425 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fea7c06ae40 0x7fea7c06d300 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fea94009970 tx=0x7fea94008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:06.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:06.902+0000 7fea8e7fc640 1 -- 172.21.15.92:0/2172714425 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fea88076460 con 0x7fea981493a0 2023-11-24T06:11:07.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:07.160+0000 7fea9eb16640 1 -- 172.21.15.92:0/2172714425 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_create"]} v 0) v1 -- 0x7fea980a9000 con 0x7fea981493a0 2023-11-24T06:11:07.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:07.801+0000 7fea8e7fc640 1 -- 172.21.15.92:0/2172714425 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_create"]}]=0 v57) v1 ==== 138+0+0 (secure 0 0 0) 0x7fea88079a80 con 0x7fea981493a0 2023-11-24T06:11:07.808 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:07.805+0000 7fea9eb16640 1 -- 172.21.15.92:0/2172714425 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fea7c06ae40 msgr2=0x7fea7c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:07.808 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:07.805+0000 7fea9eb16640 1 --2- 172.21.15.92:0/2172714425 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fea7c06ae40 0x7fea7c06d300 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fea94009970 tx=0x7fea94008040 comp rx=0 tx=0).stop 2023-11-24T06:11:07.808 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:07.805+0000 7fea9eb16640 1 -- 172.21.15.92:0/2172714425 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fea981493a0 msgr2=0x7fea98146660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:07.808 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:07.805+0000 7fea9eb16640 1 --2- 172.21.15.92:0/2172714425 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fea981493a0 0x7fea98146660 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fea8802bc30 tx=0x7fea88001120 comp rx=0 tx=0).stop 2023-11-24T06:11:07.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:07.805+0000 7fea9eb16640 1 -- 172.21.15.92:0/2172714425 shutdown_connections 2023-11-24T06:11:07.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:07.805+0000 7fea9eb16640 1 --2- 172.21.15.92:0/2172714425 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fea7c06ae40 0x7fea7c06d300 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:07.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:07.805+0000 7fea9eb16640 1 --2- 172.21.15.92:0/2172714425 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fea9813a4c0 0x7fea981591a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:07.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:07.805+0000 7fea9eb16640 1 --2- 172.21.15.92:0/2172714425 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fea981493a0 0x7fea98146660 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:07.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:07.805+0000 7fea9eb16640 1 --2- 172.21.15.92:0/2172714425 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fea840047c0 0x7fea98070ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:07.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:07.805+0000 7fea9eb16640 1 -- 172.21.15.92:0/2172714425 >> 172.21.15.92:0/2172714425 conn(0x7fea9806a800 msgr2=0x7fea9813b1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:07.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:07.805+0000 7fea9eb16640 1 -- 172.21.15.92:0/2172714425 shutdown_connections 2023-11-24T06:11:07.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:07.805+0000 7fea9eb16640 1 -- 172.21.15.92:0/2172714425 wait complete. 2023-11-24T06:11:07.827 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-24T06:11:08.046 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:08.042+0000 7f7df4df1640 1 Processor -- start 2023-11-24T06:11:08.046 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:08.043+0000 7f7df4df1640 1 -- start start 2023-11-24T06:11:08.047 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:08.044+0000 7f7df4df1640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7df00d0280 0x7f7df00ce1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:08.047 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:08.044+0000 7f7df4df1640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7df00b6750 con 0x7f7df015d590 2023-11-24T06:11:08.048 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:08.044+0000 7f7df4df1640 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7df00cf1f0 con 0x7f7df00ce6f0 2023-11-24T06:11:08.048 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:08.044+0000 7f7df4df1640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7df00cef10 con 0x7f7df00d0280 2023-11-24T06:11:08.048 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:08.044+0000 7f7dee59c640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f7df015d590 0x7f7df00cfb70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:35622/0 (socket says 172.21.15.92:35622) 2023-11-24T06:11:08.048 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:08.044+0000 7f7dee59c640 1 -- 172.21.15.92:0/424162429 learned_addr learned my addr 172.21.15.92:0/424162429 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-24T06:11:08.049 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:08.045+0000 7f7ded59a640 1 -- 172.21.15.92:0/424162429 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2072429806 0 0) 0x7f7df00b6750 con 0x7f7df015d590 2023-11-24T06:11:08.050 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:08.045+0000 7f7ded59a640 1 -- 172.21.15.92:0/424162429 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7ddc003520 con 0x7f7df015d590 2023-11-24T06:11:08.050 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:08.045+0000 7f7dedd9b640 1 --2- 172.21.15.92:0/424162429 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7df00d0280 0x7f7df00ce1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:08.050 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:08.045+0000 7f7ded59a640 1 -- 172.21.15.92:0/424162429 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3749624549 0 0) 0x7f7ddc003520 con 0x7f7df015d590 2023-11-24T06:11:08.050 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:08.045+0000 7f7ded59a640 1 -- 172.21.15.92:0/424162429 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7df00d0280 msgr2=0x7f7df00ce1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:08.050 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:08.045+0000 7f7ded59a640 1 --2- 172.21.15.92:0/424162429 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7df00d0280 0x7f7df00ce1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:08.050 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:08.045+0000 7f7ded59a640 1 -- 172.21.15.92:0/424162429 >> v1:172.21.15.118:6789/0 conn(0x7f7df00ce6f0 legacy=0x7f7df00b6be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:08.050 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:08.045+0000 7f7ded59a640 1 -- 172.21.15.92:0/424162429 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7df00b6100 con 0x7f7df015d590 2023-11-24T06:11:08.050 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:08.045+0000 7f7ded59a640 1 -- 172.21.15.92:0/424162429 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2607661350 0 0) 0x7f7de00019f0 con 0x7f7df015d590 2023-11-24T06:11:08.050 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:08.045+0000 7f7ded59a640 1 -- 172.21.15.92:0/424162429 >> v1:172.21.15.92:6789/0 conn(0x7f7df015d590 legacy=0x7f7df00cfb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:08.050 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:08.045+0000 7f7ded59a640 1 --2- 172.21.15.92:0/424162429 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7ddc0039a0 0x7f7ddc003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:08.050 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:08.045+0000 7f7ded59a640 1 --2- 172.21.15.92:0/424162429 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7ddc0047c0 0x7f7ddc004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:08.051 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:08.045+0000 7f7ded59a640 1 --2- 172.21.15.92:0/424162429 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7df00d0280 0x7f7ddc005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:08.051 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:08.045+0000 7f7ded59a640 1 -- 172.21.15.92:0/424162429 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7df00cef10 con 0x7f7ddc0039a0 2023-11-24T06:11:08.051 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:08.045+0000 7f7ded59a640 1 -- 172.21.15.92:0/424162429 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ddc00bb10 con 0x7f7ddc0047c0 2023-11-24T06:11:08.051 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:08.045+0000 7f7ded59a640 1 -- 172.21.15.92:0/424162429 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ddc00bcd0 con 0x7f7df00d0280 2023-11-24T06:11:08.051 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:08.046+0000 7f7dee59c640 1 --2- 172.21.15.92:0/424162429 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7ddc0039a0 0x7f7ddc003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:08.051 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:08.046+0000 7f7deed9d640 1 --2- 172.21.15.92:0/424162429 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7df00d0280 0x7f7ddc005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:08.051 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:08.046+0000 7f7dedd9b640 1 --2- 172.21.15.92:0/424162429 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7ddc0047c0 0x7f7ddc004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:08.051 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:08.046+0000 7f7dee59c640 1 -- 172.21.15.92:0/424162429 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7df00d0280 msgr2=0x7f7ddc005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:08.051 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:08.046+0000 7f7dee59c640 1 --2- 172.21.15.92:0/424162429 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7df00d0280 0x7f7ddc005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:08.051 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:08.046+0000 7f7dee59c640 1 -- 172.21.15.92:0/424162429 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7ddc0047c0 msgr2=0x7f7ddc004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:08.051 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:08.046+0000 7f7dee59c640 1 --2- 172.21.15.92:0/424162429 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7ddc0047c0 0x7f7ddc004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:08.051 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:08.046+0000 7f7dee59c640 1 -- 172.21.15.92:0/424162429 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ddc003520 con 0x7f7ddc0039a0 2023-11-24T06:11:08.052 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:08.046+0000 7f7deed9d640 1 --2- 172.21.15.92:0/424162429 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7df00d0280 0x7f7ddc005210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:11:08.052 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:08.046+0000 7f7dedd9b640 1 --2- 172.21.15.92:0/424162429 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7ddc0047c0 0x7f7ddc004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:11:08.053 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:08.047+0000 7f7dee59c640 1 --2- 172.21.15.92:0/424162429 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7ddc0039a0 0x7f7ddc003dc0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f7de00013d0 tx=0x7f7ddc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e221cca5298150c9 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:08.053 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:08.047+0000 7f7ded59a640 1 -- 172.21.15.92:0/424162429 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7de0034630 con 0x7f7ddc0039a0 2023-11-24T06:11:08.053 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:08.047+0000 7f7ded59a640 1 -- 172.21.15.92:0/424162429 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7de002f070 con 0x7f7ddc0039a0 2023-11-24T06:11:08.053 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:08.047+0000 7f7ded59a640 1 -- 172.21.15.92:0/424162429 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7de00349d0 con 0x7f7ddc0039a0 2023-11-24T06:11:08.053 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:08.048+0000 7f7df4df1640 1 -- 172.21.15.92:0/424162429 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7ddc0039a0 msgr2=0x7f7ddc003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:08.053 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:08.048+0000 7f7df4df1640 1 --2- 172.21.15.92:0/424162429 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7ddc0039a0 0x7f7ddc003dc0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f7de00013d0 tx=0x7f7ddc0057f0 comp rx=0 tx=0).stop 2023-11-24T06:11:08.053 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:08.048+0000 7f7deed9d640 1 -- 172.21.15.92:0/424162429 reap_dead start 2023-11-24T06:11:08.054 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:08.048+0000 7f7df4df1640 1 -- 172.21.15.92:0/424162429 shutdown_connections 2023-11-24T06:11:08.054 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:08.048+0000 7f7df4df1640 1 -- 172.21.15.92:0/424162429 >> 172.21.15.92:0/424162429 conn(0x7f7df009c540 msgr2=0x7f7df00d3210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:08.054 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:08.048+0000 7f7df4df1640 1 -- 172.21.15.92:0/424162429 shutdown_connections 2023-11-24T06:11:08.054 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:08.048+0000 7f7df4df1640 1 -- 172.21.15.92:0/424162429 wait complete. 2023-11-24T06:11:08.054 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:08.050+0000 7f7df4df1640 1 Processor -- start 2023-11-24T06:11:08.054 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:08.050+0000 7f7df4df1640 1 -- start start 2023-11-24T06:11:08.056 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:08.051+0000 7f7df4df1640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7ddc0039a0 0x7f7df00b4380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:08.056 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:08.051+0000 7f7df4df1640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7df00b48c0 0x7f7df014f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:08.056 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:08.051+0000 7f7df4df1640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7df014fbb0 0x7f7df0150000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:08.056 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:08.051+0000 7f7df4df1640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7df014eed0 con 0x7f7df014fbb0 2023-11-24T06:11:08.056 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:08.051+0000 7f7df4df1640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7df014e2c0 con 0x7f7ddc0039a0 2023-11-24T06:11:08.056 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:08.051+0000 7f7df4df1640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7df014e5a0 con 0x7f7df00b48c0 2023-11-24T06:11:08.056 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:08.051+0000 7f7deed9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7df014fbb0 0x7f7df0150000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:08.056 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:08.051+0000 7f7deed9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7df014fbb0 0x7f7df0150000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53360/0 (socket says 172.21.15.92:53360) 2023-11-24T06:11:08.056 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:08.051+0000 7f7deed9d640 1 -- 172.21.15.92:0/1059768316 learned_addr learned my addr 172.21.15.92:0/1059768316 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:11:08.056 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:08.051+0000 7f7dedd9b640 1 --2- 172.21.15.92:0/1059768316 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7df00b48c0 0x7f7df014f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:08.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:08.051+0000 7f7dee59c640 1 --2- 172.21.15.92:0/1059768316 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7ddc0039a0 0x7f7df00b4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:08.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:08.051+0000 7f7deed9d640 1 -- 172.21.15.92:0/1059768316 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7df00b48c0 msgr2=0x7f7df014f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:08.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:08.051+0000 7f7deed9d640 1 --2- 172.21.15.92:0/1059768316 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7df00b48c0 0x7f7df014f670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:08.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:08.051+0000 7f7deed9d640 1 -- 172.21.15.92:0/1059768316 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7ddc0039a0 msgr2=0x7f7df00b4380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:08.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:08.051+0000 7f7deed9d640 1 --2- 172.21.15.92:0/1059768316 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7ddc0039a0 0x7f7df00b4380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:08.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:08.051+0000 7f7deed9d640 1 -- 172.21.15.92:0/1059768316 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7df014e880 con 0x7f7df014fbb0 2023-11-24T06:11:08.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:08.051+0000 7f7dee59c640 1 --2- 172.21.15.92:0/1059768316 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7ddc0039a0 0x7f7df00b4380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:11:08.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:08.052+0000 7f7deed9d640 1 --2- 172.21.15.92:0/1059768316 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7df014fbb0 0x7f7df0150000 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f7df006d6d0 tx=0x7f7de800d890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:08.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:08.052+0000 7f7de6ffd640 1 -- 172.21.15.92:0/1059768316 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7de800b070 con 0x7f7df014fbb0 2023-11-24T06:11:08.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:08.052+0000 7f7de6ffd640 1 -- 172.21.15.92:0/1059768316 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7de80054a0 con 0x7f7df014fbb0 2023-11-24T06:11:08.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:08.052+0000 7f7de6ffd640 1 -- 172.21.15.92:0/1059768316 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7de8007270 con 0x7f7df014fbb0 2023-11-24T06:11:08.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:08.052+0000 7f7df4df1640 1 -- 172.21.15.92:0/1059768316 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7df014eb60 con 0x7f7df014fbb0 2023-11-24T06:11:08.058 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:08.052+0000 7f7df4df1640 1 -- 172.21.15.92:0/1059768316 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7df0150720 con 0x7f7df014fbb0 2023-11-24T06:11:08.058 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:08.053+0000 7f7df4df1640 1 -- 172.21.15.92:0/1059768316 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7df0150940 con 0x7f7df014fbb0 2023-11-24T06:11:08.058 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:08.054+0000 7f7de6ffd640 1 -- 172.21.15.92:0/1059768316 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f7de80073d0 con 0x7f7df014fbb0 2023-11-24T06:11:08.058 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:08.055+0000 7f7de6ffd640 1 --2- 172.21.15.92:0/1059768316 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f7dd006ae40 0x7f7dd006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:08.059 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:08.055+0000 7f7de6ffd640 1 -- 172.21.15.92:0/1059768316 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7de8089080 con 0x7f7df014fbb0 2023-11-24T06:11:08.059 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:08.055+0000 7f7dee59c640 1 --2- 172.21.15.92:0/1059768316 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f7dd006ae40 0x7f7dd006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:08.059 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:08.056+0000 7f7dee59c640 1 --2- 172.21.15.92:0/1059768316 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f7dd006ae40 0x7f7dd006d300 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f7de00013d0 tx=0x7f7de0009030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:08.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:08.059+0000 7f7de6ffd640 1 -- 172.21.15.92:0/1059768316 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7de8054090 con 0x7f7df014fbb0 2023-11-24T06:11:08.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:08.246+0000 7f7df4df1640 1 -- 172.21.15.92:0/1059768316 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f7df00675d0 con 0x7f7df014fbb0 2023-11-24T06:11:08.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:08.253+0000 7f7de6ffd640 1 -- 172.21.15.92:0/1059768316 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7f7de80576b0 con 0x7f7df014fbb0 2023-11-24T06:11:08.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:08.255+0000 7f7df4df1640 1 -- 172.21.15.92:0/1059768316 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f7dd006ae40 msgr2=0x7f7dd006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:08.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:08.255+0000 7f7df4df1640 1 --2- 172.21.15.92:0/1059768316 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f7dd006ae40 0x7f7dd006d300 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f7de00013d0 tx=0x7f7de0009030 comp rx=0 tx=0).stop 2023-11-24T06:11:08.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:08.256+0000 7f7df4df1640 1 -- 172.21.15.92:0/1059768316 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7df014fbb0 msgr2=0x7f7df0150000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:08.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:08.256+0000 7f7df4df1640 1 --2- 172.21.15.92:0/1059768316 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7df014fbb0 0x7f7df0150000 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f7df006d6d0 tx=0x7f7de800d890 comp rx=0 tx=0).stop 2023-11-24T06:11:08.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:08.256+0000 7f7df4df1640 1 -- 172.21.15.92:0/1059768316 shutdown_connections 2023-11-24T06:11:08.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:08.256+0000 7f7df4df1640 1 --2- 172.21.15.92:0/1059768316 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f7dd006ae40 0x7f7dd006d300 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:08.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:08.256+0000 7f7df4df1640 1 --2- 172.21.15.92:0/1059768316 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7df014fbb0 0x7f7df0150000 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:08.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:08.256+0000 7f7df4df1640 1 --2- 172.21.15.92:0/1059768316 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7df00b48c0 0x7f7df014f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:08.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:08.256+0000 7f7df4df1640 1 --2- 172.21.15.92:0/1059768316 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7ddc0039a0 0x7f7df00b4380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:08.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:08.256+0000 7f7df4df1640 1 -- 172.21.15.92:0/1059768316 >> 172.21.15.92:0/1059768316 conn(0x7f7df009c540 msgr2=0x7f7df015c5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:08.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:08.256+0000 7f7df4df1640 1 -- 172.21.15.92:0/1059768316 shutdown_connections 2023-11-24T06:11:08.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:08.256+0000 7f7df4df1640 1 -- 172.21.15.92:0/1059768316 wait complete. 2023-11-24T06:11:08.277 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-24T06:11:08.278 DEBUG:teuthology.orchestra.run.smithi092:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-24T06:11:08.401 DEBUG:teuthology.orchestra.run.smithi118:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-24T06:11:08.526 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-24T06:11:08.526 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-11-24T06:11:08.526 DEBUG:teuthology.orchestra.run.smithi092:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:11:08.588 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-11-24T06:11:08.588 DEBUG:teuthology.orchestra.run.smithi092:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:11:08.642 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:11:08.849 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:08.846+0000 7f5892a90640 1 Processor -- start 2023-11-24T06:11:08.849 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:08.846+0000 7f5892a90640 1 -- start start 2023-11-24T06:11:08.850 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:08.847+0000 7f5892a90640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f588c143ae0 0x7f588c0cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:08.850 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:08.847+0000 7f5892a90640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f588c0cfc30 con 0x7f588c15e3b0 2023-11-24T06:11:08.851 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:08.847+0000 7f5892a90640 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f588c0b1ee0 con 0x7f588c0cf7d0 2023-11-24T06:11:08.851 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:08.847+0000 7f5892a90640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f588c0b2780 con 0x7f588c143ae0 2023-11-24T06:11:08.851 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:08.848+0000 7f589102d640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f588c15e3b0 0x7f588c0702c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:35638/0 (socket says 172.21.15.92:35638) 2023-11-24T06:11:08.851 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:08.848+0000 7f589102d640 1 -- 172.21.15.92:0/3932453929 learned_addr learned my addr 172.21.15.92:0/3932453929 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-24T06:11:08.851 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:08.848+0000 7f589082c640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f588c143ae0 0x7f588c0cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:08.852 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:08.848+0000 7f588b7fe640 1 -- 172.21.15.92:0/3932453929 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4277661584 0 0) 0x7f588c0cfc30 con 0x7f588c15e3b0 2023-11-24T06:11:08.852 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:08.848+0000 7f589082c640 1 -- 172.21.15.92:0/3932453929 >> v1:172.21.15.118:6789/0 conn(0x7f588c0cf7d0 legacy=0x7f588c15dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:08.852 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:08.848+0000 7f589082c640 1 -- 172.21.15.92:0/3932453929 >> v1:172.21.15.92:6789/0 conn(0x7f588c15e3b0 legacy=0x7f588c0702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:08.852 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:08.848+0000 7f589082c640 1 -- 172.21.15.92:0/3932453929 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f588c15e990 con 0x7f588c143ae0 2023-11-24T06:11:08.852 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:08.849+0000 7f589082c640 1 --2- 172.21.15.92:0/3932453929 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f588c143ae0 0x7f588c0cf290 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f587c0097a0 tx=0x7f587c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ad3800dbbaaedb0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:08.852 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:08.850+0000 7f588b7fe640 1 -- 172.21.15.92:0/3932453929 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f587c031830 con 0x7f588c143ae0 2023-11-24T06:11:08.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:08.850+0000 7f588b7fe640 1 -- 172.21.15.92:0/3932453929 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f587c031990 con 0x7f588c143ae0 2023-11-24T06:11:08.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:08.850+0000 7f588b7fe640 1 -- 172.21.15.92:0/3932453929 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f587c031d20 con 0x7f588c143ae0 2023-11-24T06:11:08.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:08.850+0000 7f5892a90640 1 -- 172.21.15.92:0/3932453929 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f588c143ae0 msgr2=0x7f588c0cf290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:08.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:08.850+0000 7f5892a90640 1 --2- 172.21.15.92:0/3932453929 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f588c143ae0 0x7f588c0cf290 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f587c0097a0 tx=0x7f587c02b8d0 comp rx=0 tx=0).stop 2023-11-24T06:11:08.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:08.850+0000 7f5892a90640 1 -- 172.21.15.92:0/3932453929 shutdown_connections 2023-11-24T06:11:08.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:08.850+0000 7f5892a90640 1 --2- 172.21.15.92:0/3932453929 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f588c143ae0 0x7f588c0cf290 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:08.854 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:08.850+0000 7f5892a90640 1 -- 172.21.15.92:0/3932453929 >> 172.21.15.92:0/3932453929 conn(0x7f588c06a800 msgr2=0x7f588c06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:08.854 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:08.850+0000 7f5892a90640 1 -- 172.21.15.92:0/3932453929 shutdown_connections 2023-11-24T06:11:08.854 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:08.850+0000 7f5892a90640 1 -- 172.21.15.92:0/3932453929 wait complete. 2023-11-24T06:11:08.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:08.852+0000 7f5892a90640 1 Processor -- start 2023-11-24T06:11:08.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:08.852+0000 7f5892a90640 1 -- start start 2023-11-24T06:11:08.856 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:08.853+0000 7f5892a90640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f588c0cf7d0 0x7f588c15a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:08.856 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:08.853+0000 7f5892a90640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f588c143ae0 0x7f588c1f6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:08.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:08.853+0000 7f5892a90640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f588c15e3b0 0x7f588c159540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:08.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:08.853+0000 7f5892a90640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f588c0ad070 con 0x7f588c143ae0 2023-11-24T06:11:08.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:08.853+0000 7f5892a90640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f588c0aca80 con 0x7f588c15e3b0 2023-11-24T06:11:08.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:08.853+0000 7f5892a90640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f588c1f6c90 con 0x7f588c0cf7d0 2023-11-24T06:11:08.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:08.853+0000 7f588bfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f588c143ae0 0x7f588c1f6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:08.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:08.853+0000 7f588bfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f588c143ae0 0x7f588c1f6260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53386/0 (socket says 172.21.15.92:53386) 2023-11-24T06:11:08.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:08.853+0000 7f588bfff640 1 -- 172.21.15.92:0/2135300418 learned_addr learned my addr 172.21.15.92:0/2135300418 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:11:08.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:08.854+0000 7f588bfff640 1 -- 172.21.15.92:0/2135300418 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f588c0cf7d0 msgr2=0x7f588c15a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:08.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:08.854+0000 7f588bfff640 1 --2- 172.21.15.92:0/2135300418 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f588c0cf7d0 0x7f588c15a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:08.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:08.854+0000 7f588bfff640 1 -- 172.21.15.92:0/2135300418 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f588c15e3b0 msgr2=0x7f588c159540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:08.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:08.854+0000 7f588bfff640 1 --2- 172.21.15.92:0/2135300418 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f588c15e3b0 0x7f588c159540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:08.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:08.854+0000 7f588bfff640 1 -- 172.21.15.92:0/2135300418 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f587c009400 con 0x7f588c143ae0 2023-11-24T06:11:08.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:08.854+0000 7f588bfff640 1 --2- 172.21.15.92:0/2135300418 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f588c143ae0 0x7f588c1f6260 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f587400c430 tx=0x7f587400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:08.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:08.854+0000 7f58897fa640 1 -- 172.21.15.92:0/2135300418 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5874011790 con 0x7f588c143ae0 2023-11-24T06:11:08.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:08.854+0000 7f58897fa640 1 -- 172.21.15.92:0/2135300418 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58740118f0 con 0x7f588c143ae0 2023-11-24T06:11:08.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:08.854+0000 7f58897fa640 1 -- 172.21.15.92:0/2135300418 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5874011ce0 con 0x7f588c143ae0 2023-11-24T06:11:08.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:08.854+0000 7f5892a90640 1 -- 172.21.15.92:0/2135300418 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f588c1f6f70 con 0x7f588c143ae0 2023-11-24T06:11:08.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:08.855+0000 7f5892a90640 1 -- 172.21.15.92:0/2135300418 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f588c0cc810 con 0x7f588c143ae0 2023-11-24T06:11:08.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:08.856+0000 7f5892a90640 1 -- 172.21.15.92:0/2135300418 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f588c1f7160 con 0x7f588c143ae0 2023-11-24T06:11:08.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:08.857+0000 7f58897fa640 1 -- 172.21.15.92:0/2135300418 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f587401c460 con 0x7f588c143ae0 2023-11-24T06:11:08.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:08.858+0000 7f58897fa640 1 --2- 172.21.15.92:0/2135300418 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f586c06ae40 0x7f586c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:08.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:08.858+0000 7f58897fa640 1 -- 172.21.15.92:0/2135300418 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f58740894d0 con 0x7f588c143ae0 2023-11-24T06:11:08.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:08.858+0000 7f589082c640 1 --2- 172.21.15.92:0/2135300418 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f586c06ae40 0x7f586c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:08.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:08.859+0000 7f589082c640 1 --2- 172.21.15.92:0/2135300418 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f586c06ae40 0x7f586c06d300 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f587c036000 tx=0x7f587c030000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:08.865 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:08.863+0000 7f58897fa640 1 -- 172.21.15.92:0/2135300418 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f588c1f7160 con 0x7f588c143ae0 2023-11-24T06:11:09.056 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:09.053+0000 7f5892a90640 1 -- 172.21.15.92:0/2135300418 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f588c0cc9a0 con 0x7f588c143ae0 2023-11-24T06:11:09.058 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:09.055+0000 7f58897fa640 1 -- 172.21.15.92:0/2135300418 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16457 (secure 0 0 0) 0x7f5874054610 con 0x7f588c143ae0 2023-11-24T06:11:09.058 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-24T06:11:09.059 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":20,"fsid":"2d6dee5f-8aa3-470f-931a-50ade963b1cb","created":"2023-11-24T06:10:04.707204+0000","modified":"2023-11-24T06:10:26.698733+0000","last_up_change":"2023-11-24T06:10:17.598352+0000","last_in_change":"2023-11-24T06:10:09.040793+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:10:17.733098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:10:22.926445+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T06:10:24.151288+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ff37a9fb-135f-471f-b4b0-f7b5772006c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6803","nonce":20560714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6805","nonce":20560714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6809","nonce":20560714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6807","nonce":20560714}]},"public_addr":"172.21.15.92:6803/20560714","cluster_addr":"172.21.15.92:6805/20560714","heartbeat_back_addr":"172.21.15.92:6809/20560714","heartbeat_front_addr":"172.21.15.92:6807/20560714","state":["exists","up"]},{"osd":1,"uuid":"03cdd353-dcf1-4f38-94fe-a795137c06b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6819","nonce":2451239906}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6821","nonce":2451239906}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6825","nonce":2451239906}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6823","nonce":2451239906}]},"public_addr":"172.21.15.92:6819/2451239906","cluster_addr":"172.21.15.92:6821/2451239906","heartbeat_back_addr":"172.21.15.92:6825/2451239906","heartbeat_front_addr":"172.21.15.92:6823/2451239906","state":["exists","up"]},{"osd":2,"uuid":"a4a20d80-5d57-484b-becd-9aa8070dd3aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6811","nonce":1124979277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6813","nonce":1124979277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6817","nonce":1124979277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6815","nonce":1124979277}]},"public_addr":"172.21.15.92:6811/1124979277","cluster_addr":"172.21.15.92:6813/1124979277","heartbeat_back_addr":"172.21.15.92:6817/1124979277","heartbeat_front_addr":"172.21.15.92:6815/1124979277","state":["exists","up"]},{"osd":3,"uuid":"4648d7ce-9b29-4583-9425-4ba8fd2b1903","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6827","nonce":1734115782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6829","nonce":1734115782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6833","nonce":1734115782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6831","nonce":1734115782}]},"public_addr":"172.21.15.92:6827/1734115782","cluster_addr":"172.21.15.92:6829/1734115782","heartbeat_back_addr":"172.21.15.92:6833/1734115782","heartbeat_front_addr":"172.21.15.92:6831/1734115782","state":["exists","up"]},{"osd":4,"uuid":"3bc9d7eb-c6fa-4fcf-a3ec-ade604e05aac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6819","nonce":3425937229}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6822","nonce":3425937229}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6830","nonce":3425937229}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6826","nonce":3425937229}]},"public_addr":"172.21.15.118:6819/3425937229","cluster_addr":"172.21.15.118:6822/3425937229","heartbeat_back_addr":"172.21.15.118:6830/3425937229","heartbeat_front_addr":"172.21.15.118:6826/3425937229","state":["exists","up"]},{"osd":5,"uuid":"dce9ba6d-59ed-46d1-84fc-a84b13de10c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6801","nonce":827455984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6803","nonce":827455984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6807","nonce":827455984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6805","nonce":827455984}]},"public_addr":"172.21.15.118:6801/827455984","cluster_addr":"172.21.15.118:6803/827455984","heartbeat_back_addr":"172.21.15.118:6807/827455984","heartbeat_front_addr":"172.21.15.118:6805/827455984","state":["exists","up"]},{"osd":6,"uuid":"82aec504-ae58-4709-8d30-46f2b12081b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6809","nonce":2946986141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6811","nonce":2946986141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6815","nonce":2946986141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6813","nonce":2946986141}]},"public_addr":"172.21.15.118:6809/2946986141","cluster_addr":"172.21.15.118:6811/2946986141","heartbeat_back_addr":"172.21.15.118:6815/2946986141","heartbeat_front_addr":"172.21.15.118:6813/2946986141","state":["exists","up"]},{"osd":7,"uuid":"4ed4c624-c88f-4d35-bdb1-bf209b5dffb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6817","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6818","nonce":48122059}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6821","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6823","nonce":48122059}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6829","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6831","nonce":48122059}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6825","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6827","nonce":48122059}]},"public_addr":"172.21.15.118:6818/48122059","cluster_addr":"172.21.15.118:6823/48122059","heartbeat_back_addr":"172.21.15.118:6831/48122059","heartbeat_front_addr":"172.21.15.118:6827/48122059","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.536664+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.368324+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.271201+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.342971+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.621832+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.387110+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.632946+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.416032+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:11:09.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:09.057+0000 7f5892a90640 1 -- 172.21.15.92:0/2135300418 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f586c06ae40 msgr2=0x7f586c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:09.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:09.057+0000 7f5892a90640 1 --2- 172.21.15.92:0/2135300418 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f586c06ae40 0x7f586c06d300 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f587c036000 tx=0x7f587c030000 comp rx=0 tx=0).stop 2023-11-24T06:11:09.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:09.057+0000 7f5892a90640 1 -- 172.21.15.92:0/2135300418 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f588c143ae0 msgr2=0x7f588c1f6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:09.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:09.057+0000 7f5892a90640 1 --2- 172.21.15.92:0/2135300418 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f588c143ae0 0x7f588c1f6260 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f587400c430 tx=0x7f587400c8f0 comp rx=0 tx=0).stop 2023-11-24T06:11:09.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:09.057+0000 7f5892a90640 1 -- 172.21.15.92:0/2135300418 shutdown_connections 2023-11-24T06:11:09.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:09.057+0000 7f5892a90640 1 --2- 172.21.15.92:0/2135300418 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f586c06ae40 0x7f586c06d300 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:09.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:09.057+0000 7f5892a90640 1 --2- 172.21.15.92:0/2135300418 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f588c15e3b0 0x7f588c159540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:09.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:09.057+0000 7f5892a90640 1 --2- 172.21.15.92:0/2135300418 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f588c143ae0 0x7f588c1f6260 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:09.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:09.057+0000 7f5892a90640 1 --2- 172.21.15.92:0/2135300418 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f588c0cf7d0 0x7f588c15a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:09.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:09.057+0000 7f5892a90640 1 -- 172.21.15.92:0/2135300418 >> 172.21.15.92:0/2135300418 conn(0x7f588c06a800 msgr2=0x7f588c140e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:09.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:09.058+0000 7f5892a90640 1 -- 172.21.15.92:0/2135300418 shutdown_connections 2023-11-24T06:11:09.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:09.058+0000 7f5892a90640 1 -- 172.21.15.92:0/2135300418 wait complete. 2023-11-24T06:11:09.080 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T06:10:17.733098+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T06:10:22.926445+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T06:10:24.151288+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T06:11:09.081 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T06:11:09.292 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:09.289+0000 7f4b9a3b0640 1 Processor -- start 2023-11-24T06:11:09.292 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:09.289+0000 7f4b9a3b0640 1 -- start start 2023-11-24T06:11:09.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:09.290+0000 7f4b9a3b0640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f4b9415b2a0 0x7f4b940cf5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:09.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:09.290+0000 7f4b9a3b0640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f4b940ce070 0x7f4b94164f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:09.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:09.290+0000 7f4b9a3b0640 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b940d00a0 con 0x7f4b940cfaf0 2023-11-24T06:11:09.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:09.290+0000 7f4b9a3b0640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b940b4010 con 0x7f4b9415b2a0 2023-11-24T06:11:09.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:09.290+0000 7f4b9a3b0640 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b940b42f0 con 0x7f4b940ce070 2023-11-24T06:11:09.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:09.291+0000 7f4b93fff640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f4b9415b2a0 0x7f4b940cf5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:09.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:09.291+0000 7f4b93fff640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f4b9415b2a0 0x7f4b940cf5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:60968/0 (socket says 172.21.15.92:60968) 2023-11-24T06:11:09.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:09.291+0000 7f4b93fff640 1 -- 172.21.15.92:0/2388251514 learned_addr learned my addr 172.21.15.92:0/2388251514 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:11:09.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:09.291+0000 7f4b93fff640 1 -- 172.21.15.92:0/2388251514 >> v2:172.21.15.118:3300/0 conn(0x7f4b940ce070 msgr2=0x7f4b94164f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:09.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:09.291+0000 7f4b9894d640 1 --2- 172.21.15.92:0/2388251514 >> v2:172.21.15.118:3300/0 conn(0x7f4b940ce070 0x7f4b94164f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:09.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:09.291+0000 7f4b93fff640 1 --2- 172.21.15.92:0/2388251514 >> v2:172.21.15.118:3300/0 conn(0x7f4b940ce070 0x7f4b94164f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:09.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:09.291+0000 7f4b93fff640 1 -- 172.21.15.92:0/2388251514 >> v1:172.21.15.118:6789/0 conn(0x7f4b940cfaf0 legacy=0x7f4b940cd960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:09.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:09.291+0000 7f4b93fff640 1 -- 172.21.15.92:0/2388251514 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b940b45d0 con 0x7f4b9415b2a0 2023-11-24T06:11:09.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:09.291+0000 7f4b9894d640 1 --2- 172.21.15.92:0/2388251514 >> v2:172.21.15.118:3300/0 conn(0x7f4b940ce070 0x7f4b94164f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:11:09.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:09.292+0000 7f4b93fff640 1 --2- 172.21.15.92:0/2388251514 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4b9415b2a0 0x7f4b940cf5b0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f4b94071f20 tx=0x7f4b8402bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6902fab2e15a498 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:09.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:09.293+0000 7f4b92ffd640 1 -- 172.21.15.92:0/2388251514 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b8402ca80 con 0x7f4b9415b2a0 2023-11-24T06:11:09.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:09.293+0000 7f4b92ffd640 1 -- 172.21.15.92:0/2388251514 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b8402cbe0 con 0x7f4b9415b2a0 2023-11-24T06:11:09.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:09.293+0000 7f4b92ffd640 1 -- 172.21.15.92:0/2388251514 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b840343f0 con 0x7f4b9415b2a0 2023-11-24T06:11:09.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:09.293+0000 7f4b9a3b0640 1 -- 172.21.15.92:0/2388251514 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4b9415b2a0 msgr2=0x7f4b940cf5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:09.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:09.293+0000 7f4b9a3b0640 1 --2- 172.21.15.92:0/2388251514 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4b9415b2a0 0x7f4b940cf5b0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f4b94071f20 tx=0x7f4b8402bee0 comp rx=0 tx=0).stop 2023-11-24T06:11:09.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:09.294+0000 7f4b9a3b0640 1 -- 172.21.15.92:0/2388251514 shutdown_connections 2023-11-24T06:11:09.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:09.294+0000 7f4b9a3b0640 1 --2- 172.21.15.92:0/2388251514 >> v2:172.21.15.118:3300/0 conn(0x7f4b940ce070 0x7f4b94164f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:09.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:09.294+0000 7f4b9a3b0640 1 --2- 172.21.15.92:0/2388251514 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4b9415b2a0 0x7f4b940cf5b0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:09.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:09.294+0000 7f4b9a3b0640 1 -- 172.21.15.92:0/2388251514 >> 172.21.15.92:0/2388251514 conn(0x7f4b940d1020 msgr2=0x7f4b940d1430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:09.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:09.294+0000 7f4b9a3b0640 1 -- 172.21.15.92:0/2388251514 shutdown_connections 2023-11-24T06:11:09.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:09.294+0000 7f4b9a3b0640 1 -- 172.21.15.92:0/2388251514 wait complete. 2023-11-24T06:11:09.299 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:09.295+0000 7f4b9a3b0640 1 Processor -- start 2023-11-24T06:11:09.299 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:09.296+0000 7f4b9a3b0640 1 -- start start 2023-11-24T06:11:09.300 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:09.297+0000 7f4b9a3b0640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4b940ce070 0x7f4b9415d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:09.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:09.297+0000 7f4b9a3b0640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4b940cfaf0 0x7f4b940c8520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:09.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:09.297+0000 7f4b9a3b0640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4b9415b2a0 0x7f4b9414e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:09.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:09.297+0000 7f4b9a3b0640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b940c50e0 con 0x7f4b940ce070 2023-11-24T06:11:09.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:09.297+0000 7f4b9a3b0640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b9415d620 con 0x7f4b9415b2a0 2023-11-24T06:11:09.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:09.297+0000 7f4b9a3b0640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b9415d8c0 con 0x7f4b940cfaf0 2023-11-24T06:11:09.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:09.297+0000 7f4b93fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4b940ce070 0x7f4b9415d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:09.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:09.297+0000 7f4b93fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4b940ce070 0x7f4b9415d0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:60978/0 (socket says 172.21.15.92:60978) 2023-11-24T06:11:09.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:09.297+0000 7f4b93fff640 1 -- 172.21.15.92:0/630664654 learned_addr learned my addr 172.21.15.92:0/630664654 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:11:09.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:09.297+0000 7f4b93fff640 1 -- 172.21.15.92:0/630664654 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4b940cfaf0 msgr2=0x7f4b940c8520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:09.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:09.297+0000 7f4b93fff640 1 --2- 172.21.15.92:0/630664654 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4b940cfaf0 0x7f4b940c8520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:09.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:09.297+0000 7f4b93fff640 1 -- 172.21.15.92:0/630664654 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4b9415b2a0 msgr2=0x7f4b9414e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:09.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:09.297+0000 7f4b93fff640 1 --2- 172.21.15.92:0/630664654 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4b9415b2a0 0x7f4b9414e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:09.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:09.297+0000 7f4b93fff640 1 -- 172.21.15.92:0/630664654 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b84009b30 con 0x7f4b940ce070 2023-11-24T06:11:09.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:09.298+0000 7f4b93fff640 1 --2- 172.21.15.92:0/630664654 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4b940ce070 0x7f4b9415d0e0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f4b84000f80 tx=0x7f4b84004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:09.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:09.298+0000 7f4b90ff9640 1 -- 172.21.15.92:0/630664654 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b8403e450 con 0x7f4b940ce070 2023-11-24T06:11:09.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:09.298+0000 7f4b90ff9640 1 -- 172.21.15.92:0/630664654 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b8403e5f0 con 0x7f4b940ce070 2023-11-24T06:11:09.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:09.298+0000 7f4b90ff9640 1 -- 172.21.15.92:0/630664654 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b8403e9a0 con 0x7f4b940ce070 2023-11-24T06:11:09.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:09.298+0000 7f4b9a3b0640 1 -- 172.21.15.92:0/630664654 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b9415dba0 con 0x7f4b940ce070 2023-11-24T06:11:09.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:09.298+0000 7f4b9a3b0640 1 -- 172.21.15.92:0/630664654 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b940c52f0 con 0x7f4b940ce070 2023-11-24T06:11:09.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:09.299+0000 7f4b9a3b0640 1 -- 172.21.15.92:0/630664654 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b9415e0c0 con 0x7f4b940ce070 2023-11-24T06:11:09.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:09.300+0000 7f4b90ff9640 1 -- 172.21.15.92:0/630664654 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f4b8402cd50 con 0x7f4b940ce070 2023-11-24T06:11:09.304 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:09.301+0000 7f4b90ff9640 1 --2- 172.21.15.92:0/630664654 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f4b7406ae70 0x7f4b7406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:09.304 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:09.301+0000 7f4b90ff9640 1 -- 172.21.15.92:0/630664654 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4b840ac1f0 con 0x7f4b940ce070 2023-11-24T06:11:09.304 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:09.301+0000 7f4b937fe640 1 --2- 172.21.15.92:0/630664654 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f4b7406ae70 0x7f4b7406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:09.305 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:09.302+0000 7f4b937fe640 1 --2- 172.21.15.92:0/630664654 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f4b7406ae70 0x7f4b7406d330 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f4b9406e630 tx=0x7f4b7c0079b0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:09.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:09.305+0000 7f4b90ff9640 1 -- 172.21.15.92:0/630664654 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4b84077290 con 0x7f4b940ce070 2023-11-24T06:11:09.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:09.494+0000 7f4b9a3b0640 1 -- 172.21.15.92:0/630664654 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4b940675d0 con 0x7f4b940ce070 2023-11-24T06:11:09.498 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:09.495+0000 7f4b90ff9640 1 -- 172.21.15.92:0/630664654 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4b84033e80 con 0x7f4b940ce070 2023-11-24T06:11:09.498 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 1 2023-11-24T06:11:09.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:09.497+0000 7f4b9a3b0640 1 -- 172.21.15.92:0/630664654 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f4b7406ae70 msgr2=0x7f4b7406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:09.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:09.497+0000 7f4b9a3b0640 1 --2- 172.21.15.92:0/630664654 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f4b7406ae70 0x7f4b7406d330 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f4b9406e630 tx=0x7f4b7c0079b0 comp rx=0 tx=0).stop 2023-11-24T06:11:09.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:09.497+0000 7f4b9a3b0640 1 -- 172.21.15.92:0/630664654 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4b940ce070 msgr2=0x7f4b9415d0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:09.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:09.497+0000 7f4b9a3b0640 1 --2- 172.21.15.92:0/630664654 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4b940ce070 0x7f4b9415d0e0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f4b84000f80 tx=0x7f4b84004a40 comp rx=0 tx=0).stop 2023-11-24T06:11:09.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:09.498+0000 7f4b9a3b0640 1 -- 172.21.15.92:0/630664654 shutdown_connections 2023-11-24T06:11:09.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:09.498+0000 7f4b9a3b0640 1 --2- 172.21.15.92:0/630664654 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f4b7406ae70 0x7f4b7406d330 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:09.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:09.498+0000 7f4b9a3b0640 1 --2- 172.21.15.92:0/630664654 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4b9415b2a0 0x7f4b9414e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:09.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:09.498+0000 7f4b9a3b0640 1 --2- 172.21.15.92:0/630664654 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4b940cfaf0 0x7f4b940c8520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:09.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:09.498+0000 7f4b9a3b0640 1 --2- 172.21.15.92:0/630664654 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4b940ce070 0x7f4b9415d0e0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:09.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:09.498+0000 7f4b9a3b0640 1 -- 172.21.15.92:0/630664654 >> 172.21.15.92:0/630664654 conn(0x7f4b940d1020 msgr2=0x7f4b940b4c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:09.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:09.498+0000 7f4b9a3b0640 1 -- 172.21.15.92:0/630664654 shutdown_connections 2023-11-24T06:11:09.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:09.498+0000 7f4b9a3b0640 1 -- 172.21.15.92:0/630664654 wait complete. 2023-11-24T06:11:09.520 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T06:11:09.724 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:09.721+0000 7f9d9ebc9640 1 Processor -- start 2023-11-24T06:11:09.725 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:09.721+0000 7f9d9ebc9640 1 -- start start 2023-11-24T06:11:09.726 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:09.723+0000 7f9d9ebc9640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f9d980cf9a0 0x7f9d9815dca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:09.726 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:09.723+0000 7f9d9ebc9640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f9d9815e1e0 0x7f9d980702c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:09.726 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:09.723+0000 7f9d9ebc9640 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d980b1ee0 con 0x7f9d98143ae0 2023-11-24T06:11:09.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:09.723+0000 7f9d9ebc9640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d980b2780 con 0x7f9d9815e1e0 2023-11-24T06:11:09.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:09.723+0000 7f9d9ebc9640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d9815e990 con 0x7f9d980cf9a0 2023-11-24T06:11:09.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:09.723+0000 7f9d9d166640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f9d9815e1e0 0x7f9d980702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:09.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:09.723+0000 7f9d9d166640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f9d9815e1e0 0x7f9d980702c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:60982/0 (socket says 172.21.15.92:60982) 2023-11-24T06:11:09.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:09.723+0000 7f9d9d166640 1 -- 172.21.15.92:0/1580362619 learned_addr learned my addr 172.21.15.92:0/1580362619 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:11:09.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:09.724+0000 7f9d97fff640 1 --2- 172.21.15.92:0/1580362619 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f9d980cf9a0 0x7f9d9815dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:09.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:09.724+0000 7f9d9d166640 1 -- 172.21.15.92:0/1580362619 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f9d980cf9a0 msgr2=0x7f9d9815dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:09.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:09.724+0000 7f9d9d166640 1 --2- 172.21.15.92:0/1580362619 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f9d980cf9a0 0x7f9d9815dca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:09.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:09.724+0000 7f9d9d166640 1 -- 172.21.15.92:0/1580362619 >> v1:172.21.15.118:6789/0 conn(0x7f9d98143ae0 legacy=0x7f9d980cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:09.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:09.724+0000 7f9d9d166640 1 -- 172.21.15.92:0/1580362619 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d980b2130 con 0x7f9d9815e1e0 2023-11-24T06:11:09.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:09.725+0000 7f9d9d166640 1 --2- 172.21.15.92:0/1580362619 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9d9815e1e0 0x7f9d980702c0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f9d90009e50 tx=0x7f9d9002bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2af1fbf83de36d09 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:09.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:09.725+0000 7f9d977fe640 1 -- 172.21.15.92:0/1580362619 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d9002cb00 con 0x7f9d9815e1e0 2023-11-24T06:11:09.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:09.725+0000 7f9d977fe640 1 -- 172.21.15.92:0/1580362619 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d9002cc60 con 0x7f9d9815e1e0 2023-11-24T06:11:09.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:09.725+0000 7f9d977fe640 1 -- 172.21.15.92:0/1580362619 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d900343f0 con 0x7f9d9815e1e0 2023-11-24T06:11:09.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:09.726+0000 7f9d9ebc9640 1 -- 172.21.15.92:0/1580362619 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9d9815e1e0 msgr2=0x7f9d980702c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:09.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:09.726+0000 7f9d9ebc9640 1 --2- 172.21.15.92:0/1580362619 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9d9815e1e0 0x7f9d980702c0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f9d90009e50 tx=0x7f9d9002bf60 comp rx=0 tx=0).stop 2023-11-24T06:11:09.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:09.726+0000 7f9d9ebc9640 1 -- 172.21.15.92:0/1580362619 shutdown_connections 2023-11-24T06:11:09.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:09.726+0000 7f9d9ebc9640 1 --2- 172.21.15.92:0/1580362619 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9d9815e1e0 0x7f9d980702c0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:09.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:09.726+0000 7f9d9ebc9640 1 --2- 172.21.15.92:0/1580362619 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f9d980cf9a0 0x7f9d9815dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:09.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:09.726+0000 7f9d9ebc9640 1 -- 172.21.15.92:0/1580362619 >> 172.21.15.92:0/1580362619 conn(0x7f9d9806a800 msgr2=0x7f9d9806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:09.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:09.726+0000 7f9d9ebc9640 1 -- 172.21.15.92:0/1580362619 shutdown_connections 2023-11-24T06:11:09.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:09.726+0000 7f9d9ebc9640 1 -- 172.21.15.92:0/1580362619 wait complete. 2023-11-24T06:11:09.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:09.728+0000 7f9d9ebc9640 1 Processor -- start 2023-11-24T06:11:09.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:09.728+0000 7f9d9ebc9640 1 -- start start 2023-11-24T06:11:09.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:09.729+0000 7f9d9ebc9640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9d980cf9a0 0x7f9d980cb360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:09.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:09.729+0000 7f9d9ebc9640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9d98143ae0 0x7f9d980b0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:09.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:09.729+0000 7f9d9ebc9640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f9d9815e1e0 0x7f9d980b1100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:09.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:09.729+0000 7f9d9ebc9640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d9815a240 con 0x7f9d98143ae0 2023-11-24T06:11:09.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:09.729+0000 7f9d9ebc9640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d980c4180 con 0x7f9d980cf9a0 2023-11-24T06:11:09.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:09.729+0000 7f9d9ebc9640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d98131ec0 con 0x7f9d9815e1e0 2023-11-24T06:11:09.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:09.730+0000 7f9d97fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9d98143ae0 0x7f9d980b0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:09.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:09.730+0000 7f9d97fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9d98143ae0 0x7f9d980b0bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:60992/0 (socket says 172.21.15.92:60992) 2023-11-24T06:11:09.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:09.730+0000 7f9d97fff640 1 -- 172.21.15.92:0/3210671931 learned_addr learned my addr 172.21.15.92:0/3210671931 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:11:09.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:09.730+0000 7f9d9c965640 1 --2- 172.21.15.92:0/3210671931 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9d980cf9a0 0x7f9d980cb360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:09.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:09.730+0000 7f9d9d166640 1 --2- 172.21.15.92:0/3210671931 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f9d9815e1e0 0x7f9d980b1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:09.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:09.730+0000 7f9d97fff640 1 -- 172.21.15.92:0/3210671931 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f9d9815e1e0 msgr2=0x7f9d980b1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:09.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:09.730+0000 7f9d97fff640 1 --2- 172.21.15.92:0/3210671931 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f9d9815e1e0 0x7f9d980b1100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:09.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:09.730+0000 7f9d97fff640 1 -- 172.21.15.92:0/3210671931 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9d980cf9a0 msgr2=0x7f9d980cb360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:09.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:09.730+0000 7f9d97fff640 1 --2- 172.21.15.92:0/3210671931 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9d980cf9a0 0x7f9d980cb360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:09.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:09.730+0000 7f9d97fff640 1 -- 172.21.15.92:0/3210671931 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d90009b30 con 0x7f9d98143ae0 2023-11-24T06:11:09.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:09.730+0000 7f9d9d166640 1 --2- 172.21.15.92:0/3210671931 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f9d9815e1e0 0x7f9d980b1100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:11:09.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:09.731+0000 7f9d97fff640 1 --2- 172.21.15.92:0/3210671931 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9d98143ae0 0x7f9d980b0bc0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f9d8400d800 tx=0x7f9d8400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:09.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:09.731+0000 7f9d957fa640 1 -- 172.21.15.92:0/3210671931 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d8400b6c0 con 0x7f9d98143ae0 2023-11-24T06:11:09.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:09.731+0000 7f9d957fa640 1 -- 172.21.15.92:0/3210671931 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d8400b820 con 0x7f9d98143ae0 2023-11-24T06:11:09.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:09.731+0000 7f9d957fa640 1 -- 172.21.15.92:0/3210671931 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d8400bbb0 con 0x7f9d98143ae0 2023-11-24T06:11:09.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:09.731+0000 7f9d9ebc9640 1 -- 172.21.15.92:0/3210671931 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d981321a0 con 0x7f9d98143ae0 2023-11-24T06:11:09.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:09.731+0000 7f9d9ebc9640 1 -- 172.21.15.92:0/3210671931 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d98132760 con 0x7f9d98143ae0 2023-11-24T06:11:09.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:09.732+0000 7f9d9ebc9640 1 -- 172.21.15.92:0/3210671931 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d980675d0 con 0x7f9d98143ae0 2023-11-24T06:11:09.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:09.733+0000 7f9d957fa640 1 -- 172.21.15.92:0/3210671931 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f9d8400bd10 con 0x7f9d98143ae0 2023-11-24T06:11:09.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:09.735+0000 7f9d957fa640 1 --2- 172.21.15.92:0/3210671931 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f9d7c06ae40 0x7f9d7c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:09.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:09.735+0000 7f9d957fa640 1 -- 172.21.15.92:0/3210671931 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9d84089c40 con 0x7f9d98143ae0 2023-11-24T06:11:09.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:09.735+0000 7f9d9c965640 1 --2- 172.21.15.92:0/3210671931 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f9d7c06ae40 0x7f9d7c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:09.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:09.735+0000 7f9d9c965640 1 --2- 172.21.15.92:0/3210671931 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f9d7c06ae40 0x7f9d7c06d300 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f9d8c001640 tx=0x7f9d8c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:09.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:09.739+0000 7f9d957fa640 1 -- 172.21.15.92:0/3210671931 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9d84054c40 con 0x7f9d98143ae0 2023-11-24T06:11:09.933 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:09.929+0000 7f9d9ebc9640 1 -- 172.21.15.92:0/3210671931 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f9d9815d400 con 0x7f9d98143ae0 2023-11-24T06:11:09.933 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:09.930+0000 7f9d957fa640 1 -- 172.21.15.92:0/3210671931 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f9d84058260 con 0x7f9d98143ae0 2023-11-24T06:11:09.934 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 32 2023-11-24T06:11:09.936 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:09.932+0000 7f9d9ebc9640 1 -- 172.21.15.92:0/3210671931 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f9d7c06ae40 msgr2=0x7f9d7c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:09.936 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:09.932+0000 7f9d9ebc9640 1 --2- 172.21.15.92:0/3210671931 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f9d7c06ae40 0x7f9d7c06d300 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f9d8c001640 tx=0x7f9d8c009040 comp rx=0 tx=0).stop 2023-11-24T06:11:09.936 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:09.932+0000 7f9d9ebc9640 1 -- 172.21.15.92:0/3210671931 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9d98143ae0 msgr2=0x7f9d980b0bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:09.936 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:09.932+0000 7f9d9ebc9640 1 --2- 172.21.15.92:0/3210671931 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9d98143ae0 0x7f9d980b0bc0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f9d8400d800 tx=0x7f9d8400dcc0 comp rx=0 tx=0).stop 2023-11-24T06:11:09.936 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:09.933+0000 7f9d9ebc9640 1 -- 172.21.15.92:0/3210671931 shutdown_connections 2023-11-24T06:11:09.936 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:09.933+0000 7f9d9ebc9640 1 --2- 172.21.15.92:0/3210671931 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f9d7c06ae40 0x7f9d7c06d300 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:09.936 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:09.933+0000 7f9d9ebc9640 1 --2- 172.21.15.92:0/3210671931 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f9d9815e1e0 0x7f9d980b1100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:09.937 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:09.933+0000 7f9d9ebc9640 1 --2- 172.21.15.92:0/3210671931 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9d98143ae0 0x7f9d980b0bc0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:09.937 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:09.933+0000 7f9d9ebc9640 1 --2- 172.21.15.92:0/3210671931 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9d980cf9a0 0x7f9d980cb360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:09.937 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:09.933+0000 7f9d9ebc9640 1 -- 172.21.15.92:0/3210671931 >> 172.21.15.92:0/3210671931 conn(0x7f9d9806a800 msgr2=0x7f9d9815b710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:09.937 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:09.933+0000 7f9d9ebc9640 1 -- 172.21.15.92:0/3210671931 shutdown_connections 2023-11-24T06:11:09.937 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:09.933+0000 7f9d9ebc9640 1 -- 172.21.15.92:0/3210671931 wait complete. 2023-11-24T06:11:09.956 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T06:11:10.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:10.162+0000 7f8dce01e640 1 Processor -- start 2023-11-24T06:11:10.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:10.162+0000 7f8dce01e640 1 -- start start 2023-11-24T06:11:10.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:10.163+0000 7f8dce01e640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f8dc80cf9a0 0x7f8dc815dca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:10.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:10.164+0000 7f8dce01e640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f8dc815e1e0 0x7f8dc80702c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:10.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:10.164+0000 7f8dce01e640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8dc80b1ee0 con 0x7f8dc8143ae0 2023-11-24T06:11:10.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:10.164+0000 7f8dce01e640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8dc80b2780 con 0x7f8dc80cf9a0 2023-11-24T06:11:10.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:10.164+0000 7f8dce01e640 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8dc815e990 con 0x7f8dc815e1e0 2023-11-24T06:11:10.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:10.164+0000 7f8dc6ffd640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f8dc80cf9a0 0x7f8dc815dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:10.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:10.164+0000 7f8dc6ffd640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f8dc80cf9a0 0x7f8dc815dca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:60994/0 (socket says 172.21.15.92:60994) 2023-11-24T06:11:10.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:10.164+0000 7f8dc6ffd640 1 -- 172.21.15.92:0/473964873 learned_addr learned my addr 172.21.15.92:0/473964873 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:11:10.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:10.164+0000 7f8dc77fe640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f8dc8143ae0 0x7f8dc80cf290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:54232/0 (socket says 172.21.15.92:54232) 2023-11-24T06:11:10.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:10.164+0000 7f8dc7fff640 1 --2- 172.21.15.92:0/473964873 >> v2:172.21.15.118:3300/0 conn(0x7f8dc815e1e0 0x7f8dc80702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:10.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:10.164+0000 7f8dc6ffd640 1 -- 172.21.15.92:0/473964873 >> v2:172.21.15.118:3300/0 conn(0x7f8dc815e1e0 msgr2=0x7f8dc80702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:10.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:10.164+0000 7f8dc6ffd640 1 --2- 172.21.15.92:0/473964873 >> v2:172.21.15.118:3300/0 conn(0x7f8dc815e1e0 0x7f8dc80702c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:10.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:10.164+0000 7f8dc6ffd640 1 -- 172.21.15.92:0/473964873 >> v1:172.21.15.92:6789/0 conn(0x7f8dc8143ae0 legacy=0x7f8dc80cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:10.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:10.164+0000 7f8dc6ffd640 1 -- 172.21.15.92:0/473964873 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8dc80b2130 con 0x7f8dc80cf9a0 2023-11-24T06:11:10.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:10.164+0000 7f8dc7fff640 1 --2- 172.21.15.92:0/473964873 >> v2:172.21.15.118:3300/0 conn(0x7f8dc815e1e0 0x7f8dc80702c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:11:10.170 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:10.165+0000 7f8dc6ffd640 1 --2- 172.21.15.92:0/473964873 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8dc80cf9a0 0x7f8dc815dca0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f8dc80ae9d0 tx=0x7f8dbc02e180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f518c5c1065eba55 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:10.170 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:10.166+0000 7f8dc67fc640 1 -- 172.21.15.92:0/473964873 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dbc036440 con 0x7f8dc80cf9a0 2023-11-24T06:11:10.170 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:10.166+0000 7f8dc67fc640 1 -- 172.21.15.92:0/473964873 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8dbc0365a0 con 0x7f8dc80cf9a0 2023-11-24T06:11:10.170 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:10.166+0000 7f8dc67fc640 1 -- 172.21.15.92:0/473964873 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dbc036930 con 0x7f8dc80cf9a0 2023-11-24T06:11:10.170 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:10.166+0000 7f8dce01e640 1 -- 172.21.15.92:0/473964873 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8dc80cf9a0 msgr2=0x7f8dc815dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:10.170 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:10.166+0000 7f8dce01e640 1 --2- 172.21.15.92:0/473964873 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8dc80cf9a0 0x7f8dc815dca0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f8dc80ae9d0 tx=0x7f8dbc02e180 comp rx=0 tx=0).stop 2023-11-24T06:11:10.170 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:10.166+0000 7f8dce01e640 1 -- 172.21.15.92:0/473964873 shutdown_connections 2023-11-24T06:11:10.170 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:10.166+0000 7f8dce01e640 1 --2- 172.21.15.92:0/473964873 >> v2:172.21.15.118:3300/0 conn(0x7f8dc815e1e0 0x7f8dc80702c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:10.170 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:10.166+0000 7f8dce01e640 1 --2- 172.21.15.92:0/473964873 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8dc80cf9a0 0x7f8dc815dca0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:10.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:10.166+0000 7f8dce01e640 1 -- 172.21.15.92:0/473964873 >> 172.21.15.92:0/473964873 conn(0x7f8dc806a800 msgr2=0x7f8dc806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:10.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:10.167+0000 7f8dce01e640 1 -- 172.21.15.92:0/473964873 shutdown_connections 2023-11-24T06:11:10.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:10.167+0000 7f8dce01e640 1 -- 172.21.15.92:0/473964873 wait complete. 2023-11-24T06:11:10.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:10.168+0000 7f8dce01e640 1 Processor -- start 2023-11-24T06:11:10.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:10.168+0000 7f8dce01e640 1 -- start start 2023-11-24T06:11:10.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:10.169+0000 7f8dce01e640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8dc80cf9a0 0x7f8dc8159d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:10.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:10.169+0000 7f8dce01e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8dc8143ae0 0x7f8dc80cd790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:10.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:10.169+0000 7f8dce01e640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8dc815e1e0 0x7f8dc813e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:10.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:10.169+0000 7f8dce01e640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dc813e800 con 0x7f8dc8143ae0 2023-11-24T06:11:10.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:10.169+0000 7f8dce01e640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dc80ce450 con 0x7f8dc815e1e0 2023-11-24T06:11:10.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:10.169+0000 7f8dce01e640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8dc81f2e60 con 0x7f8dc80cf9a0 2023-11-24T06:11:10.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:10.170+0000 7f8dc6ffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8dc8143ae0 0x7f8dc80cd790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:10.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:10.170+0000 7f8dc6ffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8dc8143ae0 0x7f8dc80cd790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:32778/0 (socket says 172.21.15.92:32778) 2023-11-24T06:11:10.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:10.170+0000 7f8dc6ffd640 1 -- 172.21.15.92:0/34874673 learned_addr learned my addr 172.21.15.92:0/34874673 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:11:10.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:10.170+0000 7f8dc6ffd640 1 -- 172.21.15.92:0/34874673 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8dc80cf9a0 msgr2=0x7f8dc8159d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:10.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:10.170+0000 7f8dc6ffd640 1 --2- 172.21.15.92:0/34874673 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8dc80cf9a0 0x7f8dc8159d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:10.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:10.170+0000 7f8dc6ffd640 1 -- 172.21.15.92:0/34874673 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8dc815e1e0 msgr2=0x7f8dc813e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:10.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:10.170+0000 7f8dc6ffd640 1 --2- 172.21.15.92:0/34874673 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8dc815e1e0 0x7f8dc813e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:10.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:10.170+0000 7f8dc6ffd640 1 -- 172.21.15.92:0/34874673 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8dbc00b770 con 0x7f8dc8143ae0 2023-11-24T06:11:10.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:10.170+0000 7f8dc6ffd640 1 --2- 172.21.15.92:0/34874673 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8dc8143ae0 0x7f8dc80cd790 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f8dbc02e1c0 tx=0x7f8dbc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:10.175 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:10.170+0000 7f8daffff640 1 -- 172.21.15.92:0/34874673 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dbc036440 con 0x7f8dc8143ae0 2023-11-24T06:11:10.175 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:10.171+0000 7f8daffff640 1 -- 172.21.15.92:0/34874673 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8dbc034ba0 con 0x7f8dc8143ae0 2023-11-24T06:11:10.175 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:10.171+0000 7f8daffff640 1 -- 172.21.15.92:0/34874673 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dbc041480 con 0x7f8dc8143ae0 2023-11-24T06:11:10.175 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:10.171+0000 7f8dce01e640 1 -- 172.21.15.92:0/34874673 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8dc80cde90 con 0x7f8dc8143ae0 2023-11-24T06:11:10.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:10.171+0000 7f8dce01e640 1 -- 172.21.15.92:0/34874673 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8dc81f2af0 con 0x7f8dc8143ae0 2023-11-24T06:11:10.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:10.172+0000 7f8dce01e640 1 -- 172.21.15.92:0/34874673 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8dc81f2c80 con 0x7f8dc8143ae0 2023-11-24T06:11:10.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:10.173+0000 7f8daffff640 1 -- 172.21.15.92:0/34874673 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f8dbc0415e0 con 0x7f8dc8143ae0 2023-11-24T06:11:10.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:10.174+0000 7f8daffff640 1 --2- 172.21.15.92:0/34874673 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f8db406ae70 0x7f8db406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:10.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:10.174+0000 7f8daffff640 1 -- 172.21.15.92:0/34874673 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8dbc0ac790 con 0x7f8dc8143ae0 2023-11-24T06:11:10.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:10.174+0000 7f8dc77fe640 1 --2- 172.21.15.92:0/34874673 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f8db406ae70 0x7f8db406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:10.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:10.175+0000 7f8dc77fe640 1 --2- 172.21.15.92:0/34874673 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f8db406ae70 0x7f8db406d330 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f8db80015e0 tx=0x7f8db80079c0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:10.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:10.178+0000 7f8daffff640 1 -- 172.21.15.92:0/34874673 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8dbc0778d0 con 0x7f8dc8143ae0 2023-11-24T06:11:10.370 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:10.367+0000 7f8dce01e640 1 -- 172.21.15.92:0/34874673 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f8dc81f2950 con 0x7f8dc8143ae0 2023-11-24T06:11:10.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:10.368+0000 7f8daffff640 1 -- 172.21.15.92:0/34874673 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f8dc81f2950 con 0x7f8dc8143ae0 2023-11-24T06:11:10.371 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 64 2023-11-24T06:11:10.374 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:10.370+0000 7f8dce01e640 1 -- 172.21.15.92:0/34874673 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f8db406ae70 msgr2=0x7f8db406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:10.374 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:10.370+0000 7f8dce01e640 1 --2- 172.21.15.92:0/34874673 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f8db406ae70 0x7f8db406d330 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f8db80015e0 tx=0x7f8db80079c0 comp rx=0 tx=0).stop 2023-11-24T06:11:10.374 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:10.370+0000 7f8dce01e640 1 -- 172.21.15.92:0/34874673 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8dc8143ae0 msgr2=0x7f8dc80cd790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:10.374 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:10.370+0000 7f8dce01e640 1 --2- 172.21.15.92:0/34874673 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8dc8143ae0 0x7f8dc80cd790 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f8dbc02e1c0 tx=0x7f8dbc004730 comp rx=0 tx=0).stop 2023-11-24T06:11:10.375 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:10.371+0000 7f8dce01e640 1 -- 172.21.15.92:0/34874673 shutdown_connections 2023-11-24T06:11:10.375 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:10.371+0000 7f8dce01e640 1 --2- 172.21.15.92:0/34874673 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f8db406ae70 0x7f8db406d330 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:10.375 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:10.371+0000 7f8dce01e640 1 --2- 172.21.15.92:0/34874673 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8dc815e1e0 0x7f8dc813e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:10.375 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:10.371+0000 7f8dce01e640 1 --2- 172.21.15.92:0/34874673 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8dc8143ae0 0x7f8dc80cd790 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:10.375 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:10.371+0000 7f8dce01e640 1 --2- 172.21.15.92:0/34874673 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8dc80cf9a0 0x7f8dc8159d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:10.375 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:10.371+0000 7f8dce01e640 1 -- 172.21.15.92:0/34874673 >> 172.21.15.92:0/34874673 conn(0x7f8dc806a800 msgr2=0x7f8dc8141a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:10.375 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:10.371+0000 7f8dce01e640 1 -- 172.21.15.92:0/34874673 shutdown_connections 2023-11-24T06:11:10.375 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:10.371+0000 7f8dce01e640 1 -- 172.21.15.92:0/34874673 wait complete. 2023-11-24T06:11:10.393 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-24T06:11:10.615 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:10.611+0000 7f5d26615640 1 Processor -- start 2023-11-24T06:11:10.615 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:10.611+0000 7f5d26615640 1 -- start start 2023-11-24T06:11:10.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:10.613+0000 7f5d26615640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f5d2015c300 0x7f5d2015f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:10.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:10.613+0000 7f5d26615640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5d200b34e0 0x7f5d2015f920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:10.618 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:10.613+0000 7f5d26615640 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d2015fe60 con 0x7f5d20159040 2023-11-24T06:11:10.618 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:10.613+0000 7f5d26615640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d200c6060 con 0x7f5d2015c300 2023-11-24T06:11:10.618 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:10.613+0000 7f5d26615640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d200cf710 con 0x7f5d200b34e0 2023-11-24T06:11:10.618 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:10.613+0000 7f5d17fff640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f5d2015c300 0x7f5d2015f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:10.618 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:10.613+0000 7f5d17fff640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f5d2015c300 0x7f5d2015f3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:32788/0 (socket says 172.21.15.92:32788) 2023-11-24T06:11:10.618 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:10.613+0000 7f5d17fff640 1 -- 172.21.15.92:0/1913572254 learned_addr learned my addr 172.21.15.92:0/1913572254 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:11:10.619 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:10.614+0000 7f5d17fff640 1 -- 172.21.15.92:0/1913572254 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5d200b34e0 msgr2=0x7f5d2015f920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:10.619 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:10.614+0000 7f5d17fff640 1 --2- 172.21.15.92:0/1913572254 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5d200b34e0 0x7f5d2015f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:10.619 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:10.614+0000 7f5d17fff640 1 -- 172.21.15.92:0/1913572254 >> v1:172.21.15.118:6789/0 conn(0x7f5d20159040 legacy=0x7f5d200cf000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:10.619 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:10.614+0000 7f5d17fff640 1 -- 172.21.15.92:0/1913572254 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d200c62d0 con 0x7f5d2015c300 2023-11-24T06:11:10.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:10.615+0000 7f5d17fff640 1 --2- 172.21.15.92:0/1913572254 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5d2015c300 0x7f5d2015f3e0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f5d08009870 tx=0x7f5d0802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fcd82953b3c5d1e server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:10.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:10.615+0000 7f5d1f7fe640 1 -- 172.21.15.92:0/1913572254 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d08031640 con 0x7f5d2015c300 2023-11-24T06:11:10.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:10.615+0000 7f5d1f7fe640 1 -- 172.21.15.92:0/1913572254 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d080317a0 con 0x7f5d2015c300 2023-11-24T06:11:10.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:10.615+0000 7f5d1f7fe640 1 -- 172.21.15.92:0/1913572254 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d08031b30 con 0x7f5d2015c300 2023-11-24T06:11:10.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:10.616+0000 7f5d26615640 1 -- 172.21.15.92:0/1913572254 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5d2015c300 msgr2=0x7f5d2015f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:10.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:10.616+0000 7f5d26615640 1 --2- 172.21.15.92:0/1913572254 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5d2015c300 0x7f5d2015f3e0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f5d08009870 tx=0x7f5d0802b8f0 comp rx=0 tx=0).stop 2023-11-24T06:11:10.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:10.616+0000 7f5d26615640 1 -- 172.21.15.92:0/1913572254 shutdown_connections 2023-11-24T06:11:10.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:10.616+0000 7f5d26615640 1 --2- 172.21.15.92:0/1913572254 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5d200b34e0 0x7f5d2015f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:10.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:10.616+0000 7f5d26615640 1 --2- 172.21.15.92:0/1913572254 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5d2015c300 0x7f5d2015f3e0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:10.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:10.616+0000 7f5d26615640 1 -- 172.21.15.92:0/1913572254 >> 172.21.15.92:0/1913572254 conn(0x7f5d20066690 msgr2=0x7f5d2006a3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:10.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:10.616+0000 7f5d26615640 1 -- 172.21.15.92:0/1913572254 shutdown_connections 2023-11-24T06:11:10.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:10.616+0000 7f5d26615640 1 -- 172.21.15.92:0/1913572254 wait complete. 2023-11-24T06:11:10.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:10.618+0000 7f5d26615640 1 Processor -- start 2023-11-24T06:11:10.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:10.618+0000 7f5d26615640 1 -- start start 2023-11-24T06:11:10.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:10.619+0000 7f5d26615640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5d200b34e0 0x7f5d20149160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:10.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:10.619+0000 7f5d26615640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5d20159040 0x7f5d200b5400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:10.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:10.619+0000 7f5d26615640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5d2015c300 0x7f5d2015a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:10.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:10.619+0000 7f5d26615640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d200b6af0 con 0x7f5d200b34e0 2023-11-24T06:11:10.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:10.619+0000 7f5d26615640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d200b5e70 con 0x7f5d2015c300 2023-11-24T06:11:10.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:10.619+0000 7f5d26615640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d200b6150 con 0x7f5d20159040 2023-11-24T06:11:10.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:10.619+0000 7f5d1ffff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5d200b34e0 0x7f5d20149160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:10.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:10.619+0000 7f5d1ffff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5d200b34e0 0x7f5d20149160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:32796/0 (socket says 172.21.15.92:32796) 2023-11-24T06:11:10.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:10.619+0000 7f5d1ffff640 1 -- 172.21.15.92:0/3484908810 learned_addr learned my addr 172.21.15.92:0/3484908810 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:11:10.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:10.620+0000 7f5d17fff640 1 --2- 172.21.15.92:0/3484908810 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5d20159040 0x7f5d200b5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:10.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:10.620+0000 7f5d1ffff640 1 -- 172.21.15.92:0/3484908810 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5d20159040 msgr2=0x7f5d200b5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:10.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:10.620+0000 7f5d1ffff640 1 --2- 172.21.15.92:0/3484908810 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5d20159040 0x7f5d200b5400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:10.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:10.620+0000 7f5d1ffff640 1 -- 172.21.15.92:0/3484908810 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5d2015c300 msgr2=0x7f5d2015a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:10.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:10.620+0000 7f5d24bb2640 1 --2- 172.21.15.92:0/3484908810 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5d2015c300 0x7f5d2015a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:10.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:10.620+0000 7f5d1ffff640 1 --2- 172.21.15.92:0/3484908810 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5d2015c300 0x7f5d2015a970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:10.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:10.620+0000 7f5d1ffff640 1 -- 172.21.15.92:0/3484908810 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d080094d0 con 0x7f5d200b34e0 2023-11-24T06:11:10.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:10.620+0000 7f5d24bb2640 1 --2- 172.21.15.92:0/3484908810 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5d2015c300 0x7f5d2015a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:11:10.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:10.620+0000 7f5d1ffff640 1 --2- 172.21.15.92:0/3484908810 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5d200b34e0 0x7f5d20149160 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f5d1000c430 tx=0x7f5d1000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:10.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:10.620+0000 7f5d1d7fa640 1 -- 172.21.15.92:0/3484908810 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d10011790 con 0x7f5d200b34e0 2023-11-24T06:11:10.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:10.620+0000 7f5d1d7fa640 1 -- 172.21.15.92:0/3484908810 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d100118f0 con 0x7f5d200b34e0 2023-11-24T06:11:10.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:10.620+0000 7f5d1d7fa640 1 -- 172.21.15.92:0/3484908810 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d10011ce0 con 0x7f5d200b34e0 2023-11-24T06:11:10.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:10.620+0000 7f5d26615640 1 -- 172.21.15.92:0/3484908810 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d200b6430 con 0x7f5d200b34e0 2023-11-24T06:11:10.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:10.621+0000 7f5d26615640 1 -- 172.21.15.92:0/3484908810 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d2014ba10 con 0x7f5d200b34e0 2023-11-24T06:11:10.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:10.622+0000 7f5d26615640 1 -- 172.21.15.92:0/3484908810 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d20074a50 con 0x7f5d200b34e0 2023-11-24T06:11:10.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:10.623+0000 7f5d1d7fa640 1 -- 172.21.15.92:0/3484908810 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5d1001c460 con 0x7f5d200b34e0 2023-11-24T06:11:10.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:10.623+0000 7f5d1d7fa640 1 --2- 172.21.15.92:0/3484908810 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f5d0006ae40 0x7f5d0006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:10.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:10.624+0000 7f5d1d7fa640 1 -- 172.21.15.92:0/3484908810 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5883+0+0 (secure 0 0 0) 0x7f5d10089690 con 0x7f5d200b34e0 2023-11-24T06:11:10.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:10.624+0000 7f5d17fff640 1 --2- 172.21.15.92:0/3484908810 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f5d0006ae40 0x7f5d0006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:10.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:10.624+0000 7f5d17fff640 1 --2- 172.21.15.92:0/3484908810 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f5d0006ae40 0x7f5d0006d300 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f5d08036000 tx=0x7f5d0803c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:10.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:10.634+0000 7f5d1d7fa640 1 -- 172.21.15.92:0/3484908810 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5d10054680 con 0x7f5d200b34e0 2023-11-24T06:11:10.844 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:10.841+0000 7f5d26615640 1 -- 172.21.15.92:0/3484908810 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f5d200675d0 con 0x7f5d200b34e0 2023-11-24T06:11:10.844 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:10.841+0000 7f5d1d7fa640 1 -- 172.21.15.92:0/3484908810 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v21) v1 ==== 80+0+0 (secure 0 0 0) 0x7f5d10057ca0 con 0x7f5d200b34e0 2023-11-24T06:11:10.845 INFO:teuthology.orchestra.run.smithi092.stderr:listed 0 entries 2023-11-24T06:11:10.847 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:10.844+0000 7f5d26615640 1 -- 172.21.15.92:0/3484908810 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f5d0006ae40 msgr2=0x7f5d0006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:10.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:10.844+0000 7f5d26615640 1 --2- 172.21.15.92:0/3484908810 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f5d0006ae40 0x7f5d0006d300 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f5d08036000 tx=0x7f5d0803c040 comp rx=0 tx=0).stop 2023-11-24T06:11:10.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:10.844+0000 7f5d26615640 1 -- 172.21.15.92:0/3484908810 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5d200b34e0 msgr2=0x7f5d20149160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:10.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:10.844+0000 7f5d26615640 1 --2- 172.21.15.92:0/3484908810 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5d200b34e0 0x7f5d20149160 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f5d1000c430 tx=0x7f5d1000c8f0 comp rx=0 tx=0).stop 2023-11-24T06:11:10.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:10.844+0000 7f5d26615640 1 -- 172.21.15.92:0/3484908810 shutdown_connections 2023-11-24T06:11:10.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:10.844+0000 7f5d26615640 1 --2- 172.21.15.92:0/3484908810 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f5d0006ae40 0x7f5d0006d300 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:10.849 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:10.844+0000 7f5d26615640 1 --2- 172.21.15.92:0/3484908810 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5d2015c300 0x7f5d2015a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:10.849 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:10.844+0000 7f5d26615640 1 --2- 172.21.15.92:0/3484908810 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5d20159040 0x7f5d200b5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:10.849 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:10.844+0000 7f5d26615640 1 --2- 172.21.15.92:0/3484908810 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5d200b34e0 0x7f5d20149160 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:10.849 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:10.844+0000 7f5d26615640 1 -- 172.21.15.92:0/3484908810 >> 172.21.15.92:0/3484908810 conn(0x7f5d20066690 msgr2=0x7f5d200d0f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:10.849 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:10.844+0000 7f5d26615640 1 -- 172.21.15.92:0/3484908810 shutdown_connections 2023-11-24T06:11:10.849 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:10.844+0000 7f5d26615640 1 -- 172.21.15.92:0/3484908810 wait complete. 2023-11-24T06:11:10.864 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-11-24T06:11:10.864 DEBUG:teuthology.orchestra.run.smithi092:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:11:10.915 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:11:11.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:11.126+0000 7fc782f90640 1 Processor -- start 2023-11-24T06:11:11.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:11.126+0000 7fc782f90640 1 -- start start 2023-11-24T06:11:11.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:11.128+0000 7fc782f90640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc77c13df80 0x7fc77c15bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:11.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:11.128+0000 7fc782f90640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc77c06c470 con 0x7fc77c0cf9a0 2023-11-24T06:11:11.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:11.128+0000 7fc782f90640 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc77c06c750 con 0x7fc77c13e360 2023-11-24T06:11:11.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:11.128+0000 7fc782f90640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc77c06ca30 con 0x7fc77c13df80 2023-11-24T06:11:11.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:11.128+0000 7fc78152d640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fc77c0cf9a0 0x7fc77c0cd840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:54240/0 (socket says 172.21.15.92:54240) 2023-11-24T06:11:11.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:11.128+0000 7fc78152d640 1 -- 172.21.15.92:0/3783313347 learned_addr learned my addr 172.21.15.92:0/3783313347 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-24T06:11:11.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:11.129+0000 7fc780d2c640 1 --2- 172.21.15.92:0/3783313347 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc77c13df80 0x7fc77c15bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:11.138 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:11.129+0000 7fc7737fe640 1 -- 172.21.15.92:0/3783313347 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1662869491 0 0) 0x7fc77c06c470 con 0x7fc77c0cf9a0 2023-11-24T06:11:11.138 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:11.129+0000 7fc7737fe640 1 -- 172.21.15.92:0/3783313347 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc764003520 con 0x7fc77c0cf9a0 2023-11-24T06:11:11.138 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:11.129+0000 7fc7737fe640 1 -- 172.21.15.92:0/3783313347 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1563405654 0 0) 0x7fc764003520 con 0x7fc77c0cf9a0 2023-11-24T06:11:11.138 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:11.129+0000 7fc7737fe640 1 -- 172.21.15.92:0/3783313347 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc77c13df80 msgr2=0x7fc77c15bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:11.138 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:11.129+0000 7fc7737fe640 1 --2- 172.21.15.92:0/3783313347 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc77c13df80 0x7fc77c15bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:11.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:11.129+0000 7fc7737fe640 1 -- 172.21.15.92:0/3783313347 >> v1:172.21.15.118:6789/0 conn(0x7fc77c13e360 legacy=0x7fc77c0cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:11.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:11.129+0000 7fc7737fe640 1 -- 172.21.15.92:0/3783313347 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc77c06cd10 con 0x7fc77c0cf9a0 2023-11-24T06:11:11.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:11.129+0000 7fc7737fe640 1 -- 172.21.15.92:0/3783313347 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2607661350 0 0) 0x7fc7740019f0 con 0x7fc77c0cf9a0 2023-11-24T06:11:11.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:11.129+0000 7fc780d2c640 1 --2- 172.21.15.92:0/3783313347 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc77c13df80 0x7fc77c15bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:11:11.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:11.129+0000 7fc7737fe640 1 -- 172.21.15.92:0/3783313347 >> v1:172.21.15.92:6789/0 conn(0x7fc77c0cf9a0 legacy=0x7fc77c0cd840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:11.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:11.129+0000 7fc7737fe640 1 --2- 172.21.15.92:0/3783313347 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc7640039a0 0x7fc764003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:11.140 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:11.129+0000 7fc7737fe640 1 --2- 172.21.15.92:0/3783313347 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc7640047c0 0x7fc764004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:11.140 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:11.129+0000 7fc7737fe640 1 --2- 172.21.15.92:0/3783313347 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc77c13df80 0x7fc764005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:11.140 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:11.129+0000 7fc7737fe640 1 -- 172.21.15.92:0/3783313347 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc77c06ca30 con 0x7fc7640047c0 2023-11-24T06:11:11.140 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:11.129+0000 7fc7737fe640 1 -- 172.21.15.92:0/3783313347 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc76400ba80 con 0x7fc7640039a0 2023-11-24T06:11:11.140 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:11.129+0000 7fc7737fe640 1 -- 172.21.15.92:0/3783313347 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc76400bc40 con 0x7fc77c13df80 2023-11-24T06:11:11.140 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:11.130+0000 7fc773fff640 1 --2- 172.21.15.92:0/3783313347 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc7640047c0 0x7fc764004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:11.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:11.130+0000 7fc78152d640 1 --2- 172.21.15.92:0/3783313347 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc77c13df80 0x7fc764005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:11.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:11.130+0000 7fc780d2c640 1 --2- 172.21.15.92:0/3783313347 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc7640039a0 0x7fc764003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:11.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:11.130+0000 7fc773fff640 1 -- 172.21.15.92:0/3783313347 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc77c13df80 msgr2=0x7fc764005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:11.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:11.130+0000 7fc773fff640 1 --2- 172.21.15.92:0/3783313347 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc77c13df80 0x7fc764005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:11.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:11.130+0000 7fc773fff640 1 -- 172.21.15.92:0/3783313347 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc7640039a0 msgr2=0x7fc764003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:11.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:11.130+0000 7fc773fff640 1 --2- 172.21.15.92:0/3783313347 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc7640039a0 0x7fc764003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:11.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:11.130+0000 7fc773fff640 1 -- 172.21.15.92:0/3783313347 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc764003520 con 0x7fc7640047c0 2023-11-24T06:11:11.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:11.130+0000 7fc78152d640 1 --2- 172.21.15.92:0/3783313347 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc77c13df80 0x7fc764005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:11:11.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:11.130+0000 7fc780d2c640 1 --2- 172.21.15.92:0/3783313347 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc7640039a0 0x7fc764003dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:11:11.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:11.131+0000 7fc773fff640 1 --2- 172.21.15.92:0/3783313347 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc7640047c0 0x7fc764004c40 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fc76c001750 tx=0x7fc7640057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfe9d35840c0c6d7 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:11.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:11.131+0000 7fc7737fe640 1 -- 172.21.15.92:0/3783313347 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc76c0052d0 con 0x7fc7640047c0 2023-11-24T06:11:11.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:11.131+0000 7fc7737fe640 1 -- 172.21.15.92:0/3783313347 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc76c02e050 con 0x7fc7640047c0 2023-11-24T06:11:11.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:11.131+0000 7fc7737fe640 1 -- 172.21.15.92:0/3783313347 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc76c0035f0 con 0x7fc7640047c0 2023-11-24T06:11:11.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:11.132+0000 7fc782f90640 1 -- 172.21.15.92:0/3783313347 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc7640047c0 msgr2=0x7fc764004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:11.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:11.132+0000 7fc782f90640 1 --2- 172.21.15.92:0/3783313347 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc7640047c0 0x7fc764004c40 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fc76c001750 tx=0x7fc7640057f0 comp rx=0 tx=0).stop 2023-11-24T06:11:11.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:11.132+0000 7fc78152d640 1 -- 172.21.15.92:0/3783313347 reap_dead start 2023-11-24T06:11:11.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:11.132+0000 7fc782f90640 1 -- 172.21.15.92:0/3783313347 shutdown_connections 2023-11-24T06:11:11.144 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:11.132+0000 7fc782f90640 1 -- 172.21.15.92:0/3783313347 >> 172.21.15.92:0/3783313347 conn(0x7fc77c06a800 msgr2=0x7fc77c06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:11.144 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:11.132+0000 7fc782f90640 1 -- 172.21.15.92:0/3783313347 shutdown_connections 2023-11-24T06:11:11.144 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:11.132+0000 7fc782f90640 1 -- 172.21.15.92:0/3783313347 wait complete. 2023-11-24T06:11:11.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:11.134+0000 7fc782f90640 1 Processor -- start 2023-11-24T06:11:11.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:11.135+0000 7fc782f90640 1 -- start start 2023-11-24T06:11:11.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:11.136+0000 7fc782f90640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc7640047c0 0x7fc77c0c4b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:11.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:11.136+0000 7fc782f90640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc77c1efed0 0x7fc77c1f0350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:11.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:11.136+0000 7fc782f90640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc77c0a9fc0 0x7fc77c0aa470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:11.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:11.136+0000 7fc782f90640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc77c1f0890 con 0x7fc77c0a9fc0 2023-11-24T06:11:11.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:11.136+0000 7fc782f90640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc77c0aa9b0 con 0x7fc77c1efed0 2023-11-24T06:11:11.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:11.136+0000 7fc782f90640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc77c0aac80 con 0x7fc7640047c0 2023-11-24T06:11:11.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:11.136+0000 7fc78152d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc77c0a9fc0 0x7fc77c0aa470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:11.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:11.136+0000 7fc78152d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc77c0a9fc0 0x7fc77c0aa470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:32816/0 (socket says 172.21.15.92:32816) 2023-11-24T06:11:11.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:11.136+0000 7fc78152d640 1 -- 172.21.15.92:0/4179497180 learned_addr learned my addr 172.21.15.92:0/4179497180 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:11:11.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:11.136+0000 7fc780d2c640 1 --2- 172.21.15.92:0/4179497180 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc7640047c0 0x7fc77c0c4b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:11.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:11.136+0000 7fc773fff640 1 --2- 172.21.15.92:0/4179497180 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc77c1efed0 0x7fc77c1f0350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:11.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:11.137+0000 7fc78152d640 1 -- 172.21.15.92:0/4179497180 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc7640047c0 msgr2=0x7fc77c0c4b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:11.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:11.137+0000 7fc78152d640 1 --2- 172.21.15.92:0/4179497180 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc7640047c0 0x7fc77c0c4b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:11.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:11.137+0000 7fc78152d640 1 -- 172.21.15.92:0/4179497180 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc77c1efed0 msgr2=0x7fc77c1f0350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:11.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:11.137+0000 7fc78152d640 1 --2- 172.21.15.92:0/4179497180 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc77c1efed0 0x7fc77c1f0350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:11.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:11.137+0000 7fc78152d640 1 -- 172.21.15.92:0/4179497180 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc77c0aaf60 con 0x7fc77c0a9fc0 2023-11-24T06:11:11.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:11.137+0000 7fc78152d640 1 --2- 172.21.15.92:0/4179497180 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc77c0a9fc0 0x7fc77c0aa470 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fc77c0cfea0 tx=0x7fc77400b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:11.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:11.137+0000 7fc7717fa640 1 -- 172.21.15.92:0/4179497180 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc774004d60 con 0x7fc77c0a9fc0 2023-11-24T06:11:11.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:11.137+0000 7fc7717fa640 1 -- 172.21.15.92:0/4179497180 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc77400bb80 con 0x7fc77c0a9fc0 2023-11-24T06:11:11.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:11.137+0000 7fc7717fa640 1 -- 172.21.15.92:0/4179497180 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc774007110 con 0x7fc77c0a9fc0 2023-11-24T06:11:11.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:11.137+0000 7fc782f90640 1 -- 172.21.15.92:0/4179497180 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc77c0ab240 con 0x7fc77c0a9fc0 2023-11-24T06:11:11.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:11.137+0000 7fc782f90640 1 -- 172.21.15.92:0/4179497180 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc77c0ab800 con 0x7fc77c0a9fc0 2023-11-24T06:11:11.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:11.139+0000 7fc782f90640 1 -- 172.21.15.92:0/4179497180 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc77c0ab9f0 con 0x7fc77c0a9fc0 2023-11-24T06:11:11.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:11.140+0000 7fc7717fa640 1 -- 172.21.15.92:0/4179497180 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc774007270 con 0x7fc77c0a9fc0 2023-11-24T06:11:11.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:11.140+0000 7fc7717fa640 1 --2- 172.21.15.92:0/4179497180 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fc76806ae40 0x7fc76806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:11.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:11.141+0000 7fc7717fa640 1 -- 172.21.15.92:0/4179497180 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5883+0+0 (secure 0 0 0) 0x7fc774089840 con 0x7fc77c0a9fc0 2023-11-24T06:11:11.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:11.141+0000 7fc780d2c640 1 --2- 172.21.15.92:0/4179497180 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fc76806ae40 0x7fc76806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:11.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:11.141+0000 7fc780d2c640 1 --2- 172.21.15.92:0/4179497180 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fc76806ae40 0x7fc76806d300 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fc778006fd0 tx=0x7fc778008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:11.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:11.145+0000 7fc7717fa640 1 -- 172.21.15.92:0/4179497180 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc7740548d0 con 0x7fc77c0a9fc0 2023-11-24T06:11:11.343 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:11.340+0000 7fc782f90640 1 -- 172.21.15.92:0/4179497180 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc77c0675d0 con 0x7fc77c0a9fc0 2023-11-24T06:11:11.344 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:11.341+0000 7fc7717fa640 1 -- 172.21.15.92:0/4179497180 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16800 (secure 0 0 0) 0x7fc774057ef0 con 0x7fc77c0a9fc0 2023-11-24T06:11:11.345 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-24T06:11:11.345 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":21,"fsid":"2d6dee5f-8aa3-470f-931a-50ade963b1cb","created":"2023-11-24T06:10:04.707204+0000","modified":"2023-11-24T06:11:10.468963+0000","last_up_change":"2023-11-24T06:10:17.598352+0000","last_in_change":"2023-11-24T06:10:09.040793+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:10:17.733098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:10:22.926445+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T06:10:24.151288+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ff37a9fb-135f-471f-b4b0-f7b5772006c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6803","nonce":20560714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6805","nonce":20560714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6809","nonce":20560714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6807","nonce":20560714}]},"public_addr":"172.21.15.92:6803/20560714","cluster_addr":"172.21.15.92:6805/20560714","heartbeat_back_addr":"172.21.15.92:6809/20560714","heartbeat_front_addr":"172.21.15.92:6807/20560714","state":["exists","up"]},{"osd":1,"uuid":"03cdd353-dcf1-4f38-94fe-a795137c06b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6819","nonce":2451239906}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6821","nonce":2451239906}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6825","nonce":2451239906}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6823","nonce":2451239906}]},"public_addr":"172.21.15.92:6819/2451239906","cluster_addr":"172.21.15.92:6821/2451239906","heartbeat_back_addr":"172.21.15.92:6825/2451239906","heartbeat_front_addr":"172.21.15.92:6823/2451239906","state":["exists","up"]},{"osd":2,"uuid":"a4a20d80-5d57-484b-becd-9aa8070dd3aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6811","nonce":1124979277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6813","nonce":1124979277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6817","nonce":1124979277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6815","nonce":1124979277}]},"public_addr":"172.21.15.92:6811/1124979277","cluster_addr":"172.21.15.92:6813/1124979277","heartbeat_back_addr":"172.21.15.92:6817/1124979277","heartbeat_front_addr":"172.21.15.92:6815/1124979277","state":["exists","up"]},{"osd":3,"uuid":"4648d7ce-9b29-4583-9425-4ba8fd2b1903","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6827","nonce":1734115782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6829","nonce":1734115782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6833","nonce":1734115782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6831","nonce":1734115782}]},"public_addr":"172.21.15.92:6827/1734115782","cluster_addr":"172.21.15.92:6829/1734115782","heartbeat_back_addr":"172.21.15.92:6833/1734115782","heartbeat_front_addr":"172.21.15.92:6831/1734115782","state":["exists","up"]},{"osd":4,"uuid":"3bc9d7eb-c6fa-4fcf-a3ec-ade604e05aac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6819","nonce":3425937229}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6822","nonce":3425937229}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6830","nonce":3425937229}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6826","nonce":3425937229}]},"public_addr":"172.21.15.118:6819/3425937229","cluster_addr":"172.21.15.118:6822/3425937229","heartbeat_back_addr":"172.21.15.118:6830/3425937229","heartbeat_front_addr":"172.21.15.118:6826/3425937229","state":["exists","up"]},{"osd":5,"uuid":"dce9ba6d-59ed-46d1-84fc-a84b13de10c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6801","nonce":827455984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6803","nonce":827455984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6807","nonce":827455984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6805","nonce":827455984}]},"public_addr":"172.21.15.118:6801/827455984","cluster_addr":"172.21.15.118:6803/827455984","heartbeat_back_addr":"172.21.15.118:6807/827455984","heartbeat_front_addr":"172.21.15.118:6805/827455984","state":["exists","up"]},{"osd":6,"uuid":"82aec504-ae58-4709-8d30-46f2b12081b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6809","nonce":2946986141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6811","nonce":2946986141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6815","nonce":2946986141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6813","nonce":2946986141}]},"public_addr":"172.21.15.118:6809/2946986141","cluster_addr":"172.21.15.118:6811/2946986141","heartbeat_back_addr":"172.21.15.118:6815/2946986141","heartbeat_front_addr":"172.21.15.118:6813/2946986141","state":["exists","up"]},{"osd":7,"uuid":"4ed4c624-c88f-4d35-bdb1-bf209b5dffb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6817","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6818","nonce":48122059}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6821","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6823","nonce":48122059}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6829","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6831","nonce":48122059}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6825","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6827","nonce":48122059}]},"public_addr":"172.21.15.118:6818/48122059","cluster_addr":"172.21.15.118:6823/48122059","heartbeat_back_addr":"172.21.15.118:6831/48122059","heartbeat_front_addr":"172.21.15.118:6827/48122059","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.536664+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.368324+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.271201+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.342971+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.621832+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.387110+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.632946+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.416032+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.d","mappings":[{"from":5,"to":1}]},{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"3.29","mappings":[{"from":3,"to":2}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.34","mappings":[{"from":4,"to":2}]},{"pgid":"3.3b","mappings":[{"from":7,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:11:11.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:11.344+0000 7fc782f90640 1 -- 172.21.15.92:0/4179497180 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fc76806ae40 msgr2=0x7fc76806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:11.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:11.344+0000 7fc782f90640 1 --2- 172.21.15.92:0/4179497180 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fc76806ae40 0x7fc76806d300 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fc778006fd0 tx=0x7fc778008040 comp rx=0 tx=0).stop 2023-11-24T06:11:11.348 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:11.344+0000 7fc782f90640 1 -- 172.21.15.92:0/4179497180 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc77c0a9fc0 msgr2=0x7fc77c0aa470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:11.348 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:11.344+0000 7fc782f90640 1 --2- 172.21.15.92:0/4179497180 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc77c0a9fc0 0x7fc77c0aa470 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fc77c0cfea0 tx=0x7fc77400b4a0 comp rx=0 tx=0).stop 2023-11-24T06:11:11.348 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:11.344+0000 7fc782f90640 1 -- 172.21.15.92:0/4179497180 shutdown_connections 2023-11-24T06:11:11.348 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:11.344+0000 7fc782f90640 1 --2- 172.21.15.92:0/4179497180 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fc76806ae40 0x7fc76806d300 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:11.349 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:11.344+0000 7fc782f90640 1 --2- 172.21.15.92:0/4179497180 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc77c0a9fc0 0x7fc77c0aa470 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:11.349 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:11.344+0000 7fc782f90640 1 --2- 172.21.15.92:0/4179497180 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc77c1efed0 0x7fc77c1f0350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:11.349 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:11.344+0000 7fc782f90640 1 --2- 172.21.15.92:0/4179497180 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc7640047c0 0x7fc77c0c4b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:11.349 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:11.344+0000 7fc782f90640 1 -- 172.21.15.92:0/4179497180 >> 172.21.15.92:0/4179497180 conn(0x7fc77c06a800 msgr2=0x7fc77c0cd840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:11.349 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:11.344+0000 7fc782f90640 1 -- 172.21.15.92:0/4179497180 shutdown_connections 2023-11-24T06:11:11.349 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:11.344+0000 7fc782f90640 1 -- 172.21.15.92:0/4179497180 wait complete. 2023-11-24T06:11:11.367 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T06:10:17.733098+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T06:10:22.926445+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T06:10:24.151288+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T06:11:11.367 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T06:11:11.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:11.577+0000 7f7e7214c640 1 Processor -- start 2023-11-24T06:11:11.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:11.577+0000 7f7e7214c640 1 -- start start 2023-11-24T06:11:11.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:11.579+0000 7f7e7214c640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f7e6c13e360 0x7f7e6c0cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:11.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:11.579+0000 7f7e7214c640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e6c15c5f0 con 0x7f7e6c15f9c0 2023-11-24T06:11:11.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:11.579+0000 7f7e7214c640 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e6c06c470 con 0x7f7e6c13df80 2023-11-24T06:11:11.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:11.579+0000 7f7e7214c640 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e6c06c750 con 0x7f7e6c13e360 2023-11-24T06:11:11.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:11.579+0000 7f7e6bfff640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f7e6c15f9c0 0x7f7e6c0cd840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:54244/0 (socket says 172.21.15.92:54244) 2023-11-24T06:11:11.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:11.579+0000 7f7e6bfff640 1 -- 172.21.15.92:0/2553098119 learned_addr learned my addr 172.21.15.92:0/2553098119 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-24T06:11:11.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:11.579+0000 7f7e6a7fc640 1 -- 172.21.15.92:0/2553098119 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4240926239 0 0) 0x7f7e6c15c5f0 con 0x7f7e6c15f9c0 2023-11-24T06:11:11.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:11.580+0000 7f7e6a7fc640 1 -- 172.21.15.92:0/2553098119 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7e54003520 con 0x7f7e6c15f9c0 2023-11-24T06:11:11.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:11.580+0000 7f7e6affd640 1 --2- 172.21.15.92:0/2553098119 >> v2:172.21.15.118:3300/0 conn(0x7f7e6c13e360 0x7f7e6c0cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:11.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:11.580+0000 7f7e6a7fc640 1 -- 172.21.15.92:0/2553098119 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2298677068 0 0) 0x7f7e54003520 con 0x7f7e6c15f9c0 2023-11-24T06:11:11.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:11.580+0000 7f7e6a7fc640 1 -- 172.21.15.92:0/2553098119 >> v2:172.21.15.118:3300/0 conn(0x7f7e6c13e360 msgr2=0x7f7e6c0cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:11.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:11.580+0000 7f7e6a7fc640 1 --2- 172.21.15.92:0/2553098119 >> v2:172.21.15.118:3300/0 conn(0x7f7e6c13e360 0x7f7e6c0cf290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:11.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:11.580+0000 7f7e6a7fc640 1 -- 172.21.15.92:0/2553098119 >> v1:172.21.15.118:6789/0 conn(0x7f7e6c13df80 legacy=0x7f7e6c15bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:11.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:11.580+0000 7f7e6a7fc640 1 -- 172.21.15.92:0/2553098119 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e6c06ca30 con 0x7f7e6c15f9c0 2023-11-24T06:11:11.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:11.580+0000 7f7e6a7fc640 1 -- 172.21.15.92:0/2553098119 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2607661350 0 0) 0x7f7e64001a10 con 0x7f7e6c15f9c0 2023-11-24T06:11:11.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:11.580+0000 7f7e6a7fc640 1 -- 172.21.15.92:0/2553098119 >> v1:172.21.15.92:6789/0 conn(0x7f7e6c15f9c0 legacy=0x7f7e6c0cd840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:11.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:11.580+0000 7f7e6a7fc640 1 --2- 172.21.15.92:0/2553098119 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7e54003970 0x7f7e54003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:11.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:11.580+0000 7f7e6a7fc640 1 --2- 172.21.15.92:0/2553098119 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7e540047c0 0x7f7e54004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:11.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:11.580+0000 7f7e6a7fc640 1 --2- 172.21.15.92:0/2553098119 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7e54005180 0x7f7e5400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:11.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:11.580+0000 7f7e6a7fc640 1 -- 172.21.15.92:0/2553098119 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e6c06c750 con 0x7f7e54005180 2023-11-24T06:11:11.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:11.580+0000 7f7e6a7fc640 1 -- 172.21.15.92:0/2553098119 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e5400c0d0 con 0x7f7e540047c0 2023-11-24T06:11:11.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:11.580+0000 7f7e6a7fc640 1 -- 172.21.15.92:0/2553098119 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e5400c290 con 0x7f7e54003970 2023-11-24T06:11:11.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:11.580+0000 7f7e6bfff640 1 --2- 172.21.15.92:0/2553098119 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7e54005180 0x7f7e5400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:11.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:11.581+0000 7f7e6b7fe640 1 --2- 172.21.15.92:0/2553098119 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7e54003970 0x7f7e54003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:11.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:11.581+0000 7f7e6bfff640 1 -- 172.21.15.92:0/2553098119 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7e54003970 msgr2=0x7f7e54003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:11.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:11.581+0000 7f7e6bfff640 1 --2- 172.21.15.92:0/2553098119 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7e54003970 0x7f7e54003d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:11.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:11.581+0000 7f7e6bfff640 1 -- 172.21.15.92:0/2553098119 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7e540047c0 msgr2=0x7f7e54004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:11.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:11.581+0000 7f7e6affd640 1 --2- 172.21.15.92:0/2553098119 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7e540047c0 0x7f7e54004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:11.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:11.581+0000 7f7e6bfff640 1 --2- 172.21.15.92:0/2553098119 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7e540047c0 0x7f7e54004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:11.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:11.581+0000 7f7e6bfff640 1 -- 172.21.15.92:0/2553098119 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e54003520 con 0x7f7e54005180 2023-11-24T06:11:11.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:11.582+0000 7f7e6bfff640 1 -- 172.21.15.92:0/2553098119 reap_dead start 2023-11-24T06:11:11.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:11.582+0000 7f7e6bfff640 1 --2- 172.21.15.92:0/2553098119 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7e54005180 0x7f7e5400b980 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f7e6c15b6f0 tx=0x7f7e540057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a2a0fa07628f891 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:11.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:11.582+0000 7f7e6a7fc640 1 -- 172.21.15.92:0/2553098119 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e64034630 con 0x7f7e54005180 2023-11-24T06:11:11.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:11.582+0000 7f7e6a7fc640 1 -- 172.21.15.92:0/2553098119 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e6402f070 con 0x7f7e54005180 2023-11-24T06:11:11.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:11.582+0000 7f7e6a7fc640 1 -- 172.21.15.92:0/2553098119 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e64034990 con 0x7f7e54005180 2023-11-24T06:11:11.590 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:11.583+0000 7f7e7214c640 1 -- 172.21.15.92:0/2553098119 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7e54005180 msgr2=0x7f7e5400b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:11.590 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:11.583+0000 7f7e7214c640 1 --2- 172.21.15.92:0/2553098119 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7e54005180 0x7f7e5400b980 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f7e6c15b6f0 tx=0x7f7e540057f0 comp rx=0 tx=0).stop 2023-11-24T06:11:11.591 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:11.583+0000 7f7e7214c640 1 -- 172.21.15.92:0/2553098119 shutdown_connections 2023-11-24T06:11:11.591 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:11.583+0000 7f7e7214c640 1 --2- 172.21.15.92:0/2553098119 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7e54005180 0x7f7e5400b980 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:11.591 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:11.583+0000 7f7e7214c640 1 -- 172.21.15.92:0/2553098119 >> 172.21.15.92:0/2553098119 conn(0x7f7e6c06a800 msgr2=0x7f7e6c06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:11.591 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:11.583+0000 7f7e7214c640 1 -- 172.21.15.92:0/2553098119 shutdown_connections 2023-11-24T06:11:11.591 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:11.583+0000 7f7e7214c640 1 -- 172.21.15.92:0/2553098119 wait complete. 2023-11-24T06:11:11.591 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:11.585+0000 7f7e7214c640 1 Processor -- start 2023-11-24T06:11:11.591 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:11.585+0000 7f7e7214c640 1 -- start start 2023-11-24T06:11:11.591 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:11.586+0000 7f7e7214c640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7e54005180 0x7f7e6c135520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:11.591 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:11.586+0000 7f7e7214c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7e6c0b31a0 0x7f7e6c0a84c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:11.591 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:11.586+0000 7f7e7214c640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7e6c0b3600 0x7f7e6c0a8a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:11.592 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:11.586+0000 7f7e7214c640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e6c1f3010 con 0x7f7e6c0b31a0 2023-11-24T06:11:11.592 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:11.586+0000 7f7e7214c640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e6c152810 con 0x7f7e54005180 2023-11-24T06:11:11.592 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:11.586+0000 7f7e7214c640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e6c152af0 con 0x7f7e6c0b3600 2023-11-24T06:11:11.592 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:11.586+0000 7f7e6affd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7e6c0b31a0 0x7f7e6c0a84c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:11.592 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:11.586+0000 7f7e6affd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7e6c0b31a0 0x7f7e6c0a84c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:32842/0 (socket says 172.21.15.92:32842) 2023-11-24T06:11:11.592 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:11.586+0000 7f7e6affd640 1 -- 172.21.15.92:0/481151818 learned_addr learned my addr 172.21.15.92:0/481151818 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:11:11.592 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:11.586+0000 7f7e6bfff640 1 --2- 172.21.15.92:0/481151818 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7e6c0b3600 0x7f7e6c0a8a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:11.592 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:11.586+0000 7f7e6affd640 1 -- 172.21.15.92:0/481151818 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7e6c0b3600 msgr2=0x7f7e6c0a8a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:11.592 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:11.586+0000 7f7e6affd640 1 --2- 172.21.15.92:0/481151818 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7e6c0b3600 0x7f7e6c0a8a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:11.592 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:11.586+0000 7f7e6affd640 1 -- 172.21.15.92:0/481151818 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7e54005180 msgr2=0x7f7e6c135520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:11.592 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:11.586+0000 7f7e6affd640 1 --2- 172.21.15.92:0/481151818 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7e54005180 0x7f7e6c135520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:11.593 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:11.586+0000 7f7e6affd640 1 -- 172.21.15.92:0/481151818 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e6c152dd0 con 0x7f7e6c0b31a0 2023-11-24T06:11:11.593 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:11.587+0000 7f7e6affd640 1 --2- 172.21.15.92:0/481151818 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7e6c0b31a0 0x7f7e6c0a84c0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f7e6000e5e0 tx=0x7f7e6000eaa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:11.593 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:11.587+0000 7f7e53fff640 1 -- 172.21.15.92:0/481151818 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e60013610 con 0x7f7e6c0b31a0 2023-11-24T06:11:11.593 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:11.587+0000 7f7e53fff640 1 -- 172.21.15.92:0/481151818 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e60013770 con 0x7f7e6c0b31a0 2023-11-24T06:11:11.593 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:11.587+0000 7f7e53fff640 1 -- 172.21.15.92:0/481151818 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e60013b00 con 0x7f7e6c0b31a0 2023-11-24T06:11:11.593 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:11.587+0000 7f7e7214c640 1 -- 172.21.15.92:0/481151818 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e6c1530b0 con 0x7f7e6c0b31a0 2023-11-24T06:11:11.593 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:11.587+0000 7f7e7214c640 1 -- 172.21.15.92:0/481151818 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e6c152430 con 0x7f7e6c0b31a0 2023-11-24T06:11:11.594 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:11.588+0000 7f7e7214c640 1 -- 172.21.15.92:0/481151818 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e6c1532a0 con 0x7f7e6c0b31a0 2023-11-24T06:11:11.594 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:11.589+0000 7f7e53fff640 1 -- 172.21.15.92:0/481151818 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f7e60013c60 con 0x7f7e6c0b31a0 2023-11-24T06:11:11.594 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:11.590+0000 7f7e53fff640 1 --2- 172.21.15.92:0/481151818 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f7e5806ae40 0x7f7e5806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:11.594 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:11.590+0000 7f7e53fff640 1 -- 172.21.15.92:0/481151818 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5883+0+0 (secure 0 0 0) 0x7f7e60089500 con 0x7f7e6c0b31a0 2023-11-24T06:11:11.594 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:11.590+0000 7f7e6b7fe640 1 --2- 172.21.15.92:0/481151818 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f7e5806ae40 0x7f7e5806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:11.594 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:11.591+0000 7f7e6b7fe640 1 --2- 172.21.15.92:0/481151818 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f7e5806ae40 0x7f7e5806d300 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f7e5c000f60 tx=0x7f7e5c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:11.598 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:11.595+0000 7f7e53fff640 1 -- 172.21.15.92:0/481151818 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7e600543d0 con 0x7f7e6c0b31a0 2023-11-24T06:11:11.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:11.785+0000 7f7e7214c640 1 -- 172.21.15.92:0/481151818 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7e6c0675d0 con 0x7f7e6c0b31a0 2023-11-24T06:11:11.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:11.786+0000 7f7e53fff640 1 -- 172.21.15.92:0/481151818 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7e60009350 con 0x7f7e6c0b31a0 2023-11-24T06:11:11.789 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 1 2023-11-24T06:11:11.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:11.789+0000 7f7e7214c640 1 -- 172.21.15.92:0/481151818 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f7e5806ae40 msgr2=0x7f7e5806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:11.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:11.789+0000 7f7e7214c640 1 --2- 172.21.15.92:0/481151818 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f7e5806ae40 0x7f7e5806d300 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f7e5c000f60 tx=0x7f7e5c009040 comp rx=0 tx=0).stop 2023-11-24T06:11:11.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:11.789+0000 7f7e7214c640 1 -- 172.21.15.92:0/481151818 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7e6c0b31a0 msgr2=0x7f7e6c0a84c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:11.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:11.789+0000 7f7e7214c640 1 --2- 172.21.15.92:0/481151818 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7e6c0b31a0 0x7f7e6c0a84c0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f7e6000e5e0 tx=0x7f7e6000eaa0 comp rx=0 tx=0).stop 2023-11-24T06:11:11.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:11.789+0000 7f7e7214c640 1 -- 172.21.15.92:0/481151818 shutdown_connections 2023-11-24T06:11:11.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:11.789+0000 7f7e7214c640 1 --2- 172.21.15.92:0/481151818 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f7e5806ae40 0x7f7e5806d300 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:11.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:11.789+0000 7f7e7214c640 1 --2- 172.21.15.92:0/481151818 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7e6c0b3600 0x7f7e6c0a8a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:11.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:11.789+0000 7f7e7214c640 1 --2- 172.21.15.92:0/481151818 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7e6c0b31a0 0x7f7e6c0a84c0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:11.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:11.789+0000 7f7e7214c640 1 --2- 172.21.15.92:0/481151818 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7e54005180 0x7f7e6c135520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:11.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:11.789+0000 7f7e7214c640 1 -- 172.21.15.92:0/481151818 >> 172.21.15.92:0/481151818 conn(0x7f7e6c06a800 msgr2=0x7f7e6c0af900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:11.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:11.789+0000 7f7e7214c640 1 -- 172.21.15.92:0/481151818 shutdown_connections 2023-11-24T06:11:11.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:11.789+0000 7f7e7214c640 1 -- 172.21.15.92:0/481151818 wait complete. 2023-11-24T06:11:11.811 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T06:11:12.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.024+0000 7f6f53fff640 1 Processor -- start 2023-11-24T06:11:12.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.024+0000 7f6f53fff640 1 -- start start 2023-11-24T06:11:12.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.025+0000 7f6f53fff640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f6f540cf9a0 0x7f6f540cfde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:12.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.025+0000 7f6f53fff640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f540d0320 con 0x7f6f5415d4b0 2023-11-24T06:11:12.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.025+0000 7f6f53fff640 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f540d1c60 con 0x7f6f540b31d0 2023-11-24T06:11:12.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.025+0000 7f6f53fff640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f540b2730 con 0x7f6f540cf9a0 2023-11-24T06:11:12.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.026+0000 7f6f527fc640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f6f540cf9a0 0x7f6f540cfde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:12.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.026+0000 7f6f52ffd640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f6f5415d4b0 0x7f6f540d1550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:54252/0 (socket says 172.21.15.92:54252) 2023-11-24T06:11:12.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.026+0000 7f6f52ffd640 1 -- 172.21.15.92:0/823062515 learned_addr learned my addr 172.21.15.92:0/823062515 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-24T06:11:12.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.026+0000 7f6f527fc640 1 -- 172.21.15.92:0/823062515 >> v1:172.21.15.118:6789/0 conn(0x7f6f540b31d0 legacy=0x7f6f540b3660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:12.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.026+0000 7f6f527fc640 1 -- 172.21.15.92:0/823062515 >> v1:172.21.15.92:6789/0 conn(0x7f6f5415d4b0 legacy=0x7f6f540d1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:12.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.026+0000 7f6f527fc640 1 -- 172.21.15.92:0/823062515 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f540d1ec0 con 0x7f6f540cf9a0 2023-11-24T06:11:12.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.027+0000 7f6f527fc640 1 --2- 172.21.15.92:0/823062515 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6f540cf9a0 0x7f6f540cfde0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f6f3c009d90 tx=0x7f6f3c02beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5c49c63c6df7a9c server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:12.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.027+0000 7f6f51ffb640 1 -- 172.21.15.92:0/823062515 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f3c02cc60 con 0x7f6f540cf9a0 2023-11-24T06:11:12.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.027+0000 7f6f51ffb640 1 -- 172.21.15.92:0/823062515 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f3c02cdc0 con 0x7f6f540cf9a0 2023-11-24T06:11:12.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.027+0000 7f6f51ffb640 1 -- 172.21.15.92:0/823062515 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f3c034660 con 0x7f6f540cf9a0 2023-11-24T06:11:12.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.028+0000 7f6f53fff640 1 -- 172.21.15.92:0/823062515 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6f540cf9a0 msgr2=0x7f6f540cfde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:12.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.028+0000 7f6f53fff640 1 --2- 172.21.15.92:0/823062515 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6f540cf9a0 0x7f6f540cfde0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f6f3c009d90 tx=0x7f6f3c02beb0 comp rx=0 tx=0).stop 2023-11-24T06:11:12.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.028+0000 7f6f53fff640 1 -- 172.21.15.92:0/823062515 shutdown_connections 2023-11-24T06:11:12.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.028+0000 7f6f53fff640 1 --2- 172.21.15.92:0/823062515 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6f540cf9a0 0x7f6f540cfde0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:12.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.028+0000 7f6f53fff640 1 -- 172.21.15.92:0/823062515 >> 172.21.15.92:0/823062515 conn(0x7f6f540d3000 msgr2=0x7f6f540d3410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:12.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.028+0000 7f6f53fff640 1 -- 172.21.15.92:0/823062515 shutdown_connections 2023-11-24T06:11:12.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.028+0000 7f6f53fff640 1 -- 172.21.15.92:0/823062515 wait complete. 2023-11-24T06:11:12.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.030+0000 7f6f53fff640 1 Processor -- start 2023-11-24T06:11:12.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.030+0000 7f6f53fff640 1 -- start start 2023-11-24T06:11:12.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.031+0000 7f6f53fff640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6f540b31d0 0x7f6f54157090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:12.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.031+0000 7f6f53fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6f540cf9a0 0x7f6f541f34e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:12.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.031+0000 7f6f53fff640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6f5415d4b0 0x7f6f54159bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:12.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.031+0000 7f6f53fff640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f541f6850 con 0x7f6f540cf9a0 2023-11-24T06:11:12.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.031+0000 7f6f53fff640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f541575d0 con 0x7f6f540b31d0 2023-11-24T06:11:12.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.031+0000 7f6f53fff640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f540ca200 con 0x7f6f5415d4b0 2023-11-24T06:11:12.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.031+0000 7f6f527fc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6f540cf9a0 0x7f6f541f34e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:12.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.031+0000 7f6f527fc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6f540cf9a0 0x7f6f541f34e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:32860/0 (socket says 172.21.15.92:32860) 2023-11-24T06:11:12.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.031+0000 7f6f527fc640 1 -- 172.21.15.92:0/1720767608 learned_addr learned my addr 172.21.15.92:0/1720767608 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:11:12.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.031+0000 7f6f52ffd640 1 --2- 172.21.15.92:0/1720767608 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6f540b31d0 0x7f6f54157090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:12.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.032+0000 7f6f527fc640 1 -- 172.21.15.92:0/1720767608 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6f5415d4b0 msgr2=0x7f6f54159bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:12.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.032+0000 7f6f527fc640 1 --2- 172.21.15.92:0/1720767608 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6f5415d4b0 0x7f6f54159bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:12.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.032+0000 7f6f527fc640 1 -- 172.21.15.92:0/1720767608 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6f540b31d0 msgr2=0x7f6f54157090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:12.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.032+0000 7f6f527fc640 1 --2- 172.21.15.92:0/1720767608 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6f540b31d0 0x7f6f54157090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:12.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.032+0000 7f6f527fc640 1 -- 172.21.15.92:0/1720767608 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f3c009a70 con 0x7f6f540cf9a0 2023-11-24T06:11:12.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.032+0000 7f6f527fc640 1 --2- 172.21.15.92:0/1720767608 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6f540cf9a0 0x7f6f541f34e0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f6f3c009d90 tx=0x7f6f3c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:12.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.032+0000 7f6f4bfff640 1 -- 172.21.15.92:0/1720767608 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f3c03e610 con 0x7f6f540cf9a0 2023-11-24T06:11:12.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.032+0000 7f6f4bfff640 1 -- 172.21.15.92:0/1720767608 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f3c03e770 con 0x7f6f540cf9a0 2023-11-24T06:11:12.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.032+0000 7f6f4bfff640 1 -- 172.21.15.92:0/1720767608 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f3c03eb20 con 0x7f6f540cf9a0 2023-11-24T06:11:12.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.032+0000 7f6f53fff640 1 -- 172.21.15.92:0/1720767608 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f540ca4e0 con 0x7f6f540cf9a0 2023-11-24T06:11:12.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.033+0000 7f6f53fff640 1 -- 172.21.15.92:0/1720767608 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f540caaa0 con 0x7f6f540cf9a0 2023-11-24T06:11:12.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.034+0000 7f6f53fff640 1 -- 172.21.15.92:0/1720767608 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f54067450 con 0x7f6f540cf9a0 2023-11-24T06:11:12.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.035+0000 7f6f4bfff640 1 -- 172.21.15.92:0/1720767608 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6f3c03b050 con 0x7f6f540cf9a0 2023-11-24T06:11:12.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.035+0000 7f6f4bfff640 1 --2- 172.21.15.92:0/1720767608 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f6f3406ae40 0x7f6f3406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:12.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.036+0000 7f6f4bfff640 1 -- 172.21.15.92:0/1720767608 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5883+0+0 (secure 0 0 0) 0x7f6f3c0ac1d0 con 0x7f6f540cf9a0 2023-11-24T06:11:12.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.036+0000 7f6f52ffd640 1 --2- 172.21.15.92:0/1720767608 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f6f3406ae40 0x7f6f3406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:12.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.036+0000 7f6f52ffd640 1 --2- 172.21.15.92:0/1720767608 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f6f3406ae40 0x7f6f3406d300 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f6f44005880 tx=0x7f6f44009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:12.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.040+0000 7f6f4bfff640 1 -- 172.21.15.92:0/1720767608 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6f3c0772e0 con 0x7f6f540cf9a0 2023-11-24T06:11:12.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.229+0000 7f6f53fff640 1 -- 172.21.15.92:0/1720767608 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f6f540b58a0 con 0x7f6f540cf9a0 2023-11-24T06:11:12.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.230+0000 7f6f4bfff640 1 -- 172.21.15.92:0/1720767608 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7f6f3c07a900 con 0x7f6f540cf9a0 2023-11-24T06:11:12.234 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 32 2023-11-24T06:11:12.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.233+0000 7f6f53fff640 1 -- 172.21.15.92:0/1720767608 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f6f3406ae40 msgr2=0x7f6f3406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:12.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.233+0000 7f6f53fff640 1 --2- 172.21.15.92:0/1720767608 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f6f3406ae40 0x7f6f3406d300 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f6f44005880 tx=0x7f6f44009040 comp rx=0 tx=0).stop 2023-11-24T06:11:12.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.233+0000 7f6f53fff640 1 -- 172.21.15.92:0/1720767608 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6f540cf9a0 msgr2=0x7f6f541f34e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:12.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.233+0000 7f6f53fff640 1 --2- 172.21.15.92:0/1720767608 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6f540cf9a0 0x7f6f541f34e0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f6f3c009d90 tx=0x7f6f3c004a40 comp rx=0 tx=0).stop 2023-11-24T06:11:12.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.233+0000 7f6f53fff640 1 -- 172.21.15.92:0/1720767608 shutdown_connections 2023-11-24T06:11:12.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.233+0000 7f6f53fff640 1 --2- 172.21.15.92:0/1720767608 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f6f3406ae40 0x7f6f3406d300 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:12.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.233+0000 7f6f53fff640 1 --2- 172.21.15.92:0/1720767608 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6f5415d4b0 0x7f6f54159bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:12.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.233+0000 7f6f53fff640 1 --2- 172.21.15.92:0/1720767608 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6f540cf9a0 0x7f6f541f34e0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:12.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.233+0000 7f6f53fff640 1 --2- 172.21.15.92:0/1720767608 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6f540b31d0 0x7f6f54157090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:12.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.233+0000 7f6f53fff640 1 -- 172.21.15.92:0/1720767608 >> 172.21.15.92:0/1720767608 conn(0x7f6f540d3000 msgr2=0x7f6f5406c750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:12.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.233+0000 7f6f53fff640 1 -- 172.21.15.92:0/1720767608 shutdown_connections 2023-11-24T06:11:12.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.233+0000 7f6f53fff640 1 -- 172.21.15.92:0/1720767608 wait complete. 2023-11-24T06:11:12.256 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T06:11:12.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.470+0000 7f1fe0108640 1 Processor -- start 2023-11-24T06:11:12.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.470+0000 7f1fe0108640 1 -- start start 2023-11-24T06:11:12.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.472+0000 7f1fe0108640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f1fd80cd170 0x7f1fd8071070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:12.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.472+0000 7f1fe0108640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1fd8071690 con 0x7f1fd80cf9a0 2023-11-24T06:11:12.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.472+0000 7f1fe0108640 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1fd8071970 con 0x7f1fd814b2c0 2023-11-24T06:11:12.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.472+0000 7f1fe0108640 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1fd814b6a0 con 0x7f1fd80cd170 2023-11-24T06:11:12.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.472+0000 7f1fdd6a3640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f1fd80cf9a0 0x7f1fd8070960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:54264/0 (socket says 172.21.15.92:54264) 2023-11-24T06:11:12.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.472+0000 7f1fdd6a3640 1 -- 172.21.15.92:0/2305068187 learned_addr learned my addr 172.21.15.92:0/2305068187 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-24T06:11:12.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.472+0000 7f1fde6a5640 1 --2- 172.21.15.92:0/2305068187 >> v2:172.21.15.118:3300/0 conn(0x7f1fd80cd170 0x7f1fd8071070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:12.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.472+0000 7f1fdcea2640 1 -- 172.21.15.92:0/2305068187 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3121217275 0 0) 0x7f1fd8071690 con 0x7f1fd80cf9a0 2023-11-24T06:11:12.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.472+0000 7f1fdcea2640 1 -- 172.21.15.92:0/2305068187 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1fc4003520 con 0x7f1fd80cf9a0 2023-11-24T06:11:12.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.473+0000 7f1fdcea2640 1 -- 172.21.15.92:0/2305068187 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 258814370 0 0) 0x7f1fc4003520 con 0x7f1fd80cf9a0 2023-11-24T06:11:12.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.473+0000 7f1fdcea2640 1 -- 172.21.15.92:0/2305068187 >> v2:172.21.15.118:3300/0 conn(0x7f1fd80cd170 msgr2=0x7f1fd8071070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:12.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.473+0000 7f1fdcea2640 1 --2- 172.21.15.92:0/2305068187 >> v2:172.21.15.118:3300/0 conn(0x7f1fd80cd170 0x7f1fd8071070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:12.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.473+0000 7f1fdcea2640 1 -- 172.21.15.92:0/2305068187 >> v1:172.21.15.118:6789/0 conn(0x7f1fd814b2c0 legacy=0x7f1fd80cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:12.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.473+0000 7f1fdcea2640 1 -- 172.21.15.92:0/2305068187 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fd8071c50 con 0x7f1fd80cf9a0 2023-11-24T06:11:12.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.473+0000 7f1fdcea2640 1 -- 172.21.15.92:0/2305068187 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2607661350 0 0) 0x7f1fc8001670 con 0x7f1fd80cf9a0 2023-11-24T06:11:12.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.473+0000 7f1fdcea2640 1 -- 172.21.15.92:0/2305068187 >> v1:172.21.15.92:6789/0 conn(0x7f1fd80cf9a0 legacy=0x7f1fd8070960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:12.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.473+0000 7f1fdcea2640 1 --2- 172.21.15.92:0/2305068187 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1fc40039c0 0x7f1fc4003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:12.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.473+0000 7f1fdcea2640 1 --2- 172.21.15.92:0/2305068187 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1fc40047c0 0x7f1fc4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:12.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.473+0000 7f1fdcea2640 1 --2- 172.21.15.92:0/2305068187 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1fc4005180 0x7f1fc400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:12.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.473+0000 7f1fdcea2640 1 -- 172.21.15.92:0/2305068187 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fd814b6a0 con 0x7f1fc40039c0 2023-11-24T06:11:12.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.473+0000 7f1fdcea2640 1 -- 172.21.15.92:0/2305068187 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fc400c080 con 0x7f1fc4005180 2023-11-24T06:11:12.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.473+0000 7f1fdcea2640 1 -- 172.21.15.92:0/2305068187 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fc400c240 con 0x7f1fc40047c0 2023-11-24T06:11:12.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.473+0000 7f1fddea4640 1 --2- 172.21.15.92:0/2305068187 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1fc40039c0 0x7f1fc4003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:12.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.474+0000 7f1fde6a5640 1 --2- 172.21.15.92:0/2305068187 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1fc4005180 0x7f1fc400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:12.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.474+0000 7f1fdd6a3640 1 --2- 172.21.15.92:0/2305068187 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1fc40047c0 0x7f1fc4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:12.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.474+0000 7f1fddea4640 1 -- 172.21.15.92:0/2305068187 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1fc40047c0 msgr2=0x7f1fc4004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:12.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.474+0000 7f1fddea4640 1 --2- 172.21.15.92:0/2305068187 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1fc40047c0 0x7f1fc4004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:12.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.474+0000 7f1fddea4640 1 -- 172.21.15.92:0/2305068187 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1fc4005180 msgr2=0x7f1fc400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:12.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.474+0000 7f1fddea4640 1 --2- 172.21.15.92:0/2305068187 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1fc4005180 0x7f1fc400b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:12.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.474+0000 7f1fde6a5640 1 -- 172.21.15.92:0/2305068187 reap_dead start 2023-11-24T06:11:12.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.474+0000 7f1fddea4640 1 -- 172.21.15.92:0/2305068187 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fc4003520 con 0x7f1fc40039c0 2023-11-24T06:11:12.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.474+0000 7f1fdd6a3640 1 --2- 172.21.15.92:0/2305068187 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1fc40047c0 0x7f1fc4004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:11:12.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.475+0000 7f1fddea4640 1 --2- 172.21.15.92:0/2305068187 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1fc40039c0 0x7f1fc4003de0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f1fd4001750 tx=0x7f1fc40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7597793b80d43e90 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:12.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.475+0000 7f1fdcea2640 1 -- 172.21.15.92:0/2305068187 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fd40052d0 con 0x7f1fc40039c0 2023-11-24T06:11:12.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.475+0000 7f1fdcea2640 1 -- 172.21.15.92:0/2305068187 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fd402e050 con 0x7f1fc40039c0 2023-11-24T06:11:12.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.475+0000 7f1fdcea2640 1 -- 172.21.15.92:0/2305068187 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fd4003830 con 0x7f1fc40039c0 2023-11-24T06:11:12.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.476+0000 7f1fe0108640 1 -- 172.21.15.92:0/2305068187 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1fc40039c0 msgr2=0x7f1fc4003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:12.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.476+0000 7f1fe0108640 1 --2- 172.21.15.92:0/2305068187 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1fc40039c0 0x7f1fc4003de0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f1fd4001750 tx=0x7f1fc40057f0 comp rx=0 tx=0).stop 2023-11-24T06:11:12.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.476+0000 7f1fe0108640 1 -- 172.21.15.92:0/2305068187 shutdown_connections 2023-11-24T06:11:12.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.476+0000 7f1fe0108640 1 --2- 172.21.15.92:0/2305068187 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1fc40039c0 0x7f1fc4003de0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:12.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.476+0000 7f1fe0108640 1 -- 172.21.15.92:0/2305068187 >> 172.21.15.92:0/2305068187 conn(0x7f1fd806a800 msgr2=0x7f1fd806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:12.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.476+0000 7f1fe0108640 1 -- 172.21.15.92:0/2305068187 shutdown_connections 2023-11-24T06:11:12.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.476+0000 7f1fe0108640 1 -- 172.21.15.92:0/2305068187 wait complete. 2023-11-24T06:11:12.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.479+0000 7f1fe0108640 1 Processor -- start 2023-11-24T06:11:12.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.479+0000 7f1fe0108640 1 -- start start 2023-11-24T06:11:12.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.480+0000 7f1fe0108640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1fc40039c0 0x7f1fd8157f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:12.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.480+0000 7f1fe0108640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1fd813f440 0x7f1fd8163c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:12.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.480+0000 7f1fe0108640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1fd8146580 0x7f1fd8146a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:12.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.480+0000 7f1fe0108640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fd81405a0 con 0x7f1fd8146580 2023-11-24T06:11:12.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.480+0000 7f1fe0108640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fd8146f70 con 0x7f1fd813f440 2023-11-24T06:11:12.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.480+0000 7f1fe0108640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fd81409c0 con 0x7f1fc40039c0 2023-11-24T06:11:12.492 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.480+0000 7f1fde6a5640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1fd8146580 0x7f1fd8146a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:12.492 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.480+0000 7f1fde6a5640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1fd8146580 0x7f1fd8146a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:32886/0 (socket says 172.21.15.92:32886) 2023-11-24T06:11:12.492 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.480+0000 7f1fde6a5640 1 -- 172.21.15.92:0/112511473 learned_addr learned my addr 172.21.15.92:0/112511473 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:11:12.492 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.480+0000 7f1fddea4640 1 --2- 172.21.15.92:0/112511473 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1fc40039c0 0x7f1fd8157f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:12.492 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.480+0000 7f1fde6a5640 1 -- 172.21.15.92:0/112511473 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1fc40039c0 msgr2=0x7f1fd8157f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:12.492 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.480+0000 7f1fde6a5640 1 --2- 172.21.15.92:0/112511473 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1fc40039c0 0x7f1fd8157f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:12.492 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.480+0000 7f1fde6a5640 1 -- 172.21.15.92:0/112511473 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1fd813f440 msgr2=0x7f1fd8163c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:12.492 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.480+0000 7f1fde6a5640 1 --2- 172.21.15.92:0/112511473 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1fd813f440 0x7f1fd8163c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:12.492 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.480+0000 7f1fde6a5640 1 -- 172.21.15.92:0/112511473 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fd813f920 con 0x7f1fd8146580 2023-11-24T06:11:12.492 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.481+0000 7f1fde6a5640 1 --2- 172.21.15.92:0/112511473 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1fd8146580 0x7f1fd8146a30 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f1fd806e2d0 tx=0x7f1fd000f520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:12.492 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.481+0000 7f1fce7fc640 1 -- 172.21.15.92:0/112511473 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fd0016420 con 0x7f1fd8146580 2023-11-24T06:11:12.493 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.481+0000 7f1fce7fc640 1 -- 172.21.15.92:0/112511473 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fd0016580 con 0x7f1fd8146580 2023-11-24T06:11:12.493 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.481+0000 7f1fce7fc640 1 -- 172.21.15.92:0/112511473 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fd0016930 con 0x7f1fd8146580 2023-11-24T06:11:12.493 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.481+0000 7f1fe0108640 1 -- 172.21.15.92:0/112511473 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fd813fc00 con 0x7f1fd8146580 2023-11-24T06:11:12.493 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.481+0000 7f1fe0108640 1 -- 172.21.15.92:0/112511473 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fd81401c0 con 0x7f1fd8146580 2023-11-24T06:11:12.493 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.483+0000 7f1fe0108640 1 -- 172.21.15.92:0/112511473 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1fd81406e0 con 0x7f1fd8146580 2023-11-24T06:11:12.493 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.483+0000 7f1fce7fc640 1 -- 172.21.15.92:0/112511473 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f1fd0016a90 con 0x7f1fd8146580 2023-11-24T06:11:12.493 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.484+0000 7f1fce7fc640 1 --2- 172.21.15.92:0/112511473 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f1fbc06ae40 0x7f1fbc06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:12.493 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.485+0000 7f1fce7fc640 1 -- 172.21.15.92:0/112511473 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5883+0+0 (secure 0 0 0) 0x7f1fd008a5d0 con 0x7f1fd8146580 2023-11-24T06:11:12.493 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.485+0000 7f1fddea4640 1 --2- 172.21.15.92:0/112511473 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f1fbc06ae40 0x7f1fbc06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:12.493 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.485+0000 7f1fddea4640 1 --2- 172.21.15.92:0/112511473 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f1fbc06ae40 0x7f1fbc06d300 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f1fd4033000 tx=0x7f1fd4009030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:12.494 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.489+0000 7f1fce7fc640 1 -- 172.21.15.92:0/112511473 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1fd81406e0 con 0x7f1fd8146580 2023-11-24T06:11:12.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.681+0000 7f1fe0108640 1 -- 172.21.15.92:0/112511473 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f1fd81406e0 con 0x7f1fd8146580 2023-11-24T06:11:12.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.682+0000 7f1fce7fc640 1 -- 172.21.15.92:0/112511473 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7f1fd81406e0 con 0x7f1fd8146580 2023-11-24T06:11:12.685 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 64 2023-11-24T06:11:12.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.684+0000 7f1fe0108640 1 -- 172.21.15.92:0/112511473 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f1fbc06ae40 msgr2=0x7f1fbc06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:12.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.684+0000 7f1fe0108640 1 --2- 172.21.15.92:0/112511473 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f1fbc06ae40 0x7f1fbc06d300 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f1fd4033000 tx=0x7f1fd4009030 comp rx=0 tx=0).stop 2023-11-24T06:11:12.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.684+0000 7f1fe0108640 1 -- 172.21.15.92:0/112511473 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1fd8146580 msgr2=0x7f1fd8146a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:12.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.684+0000 7f1fe0108640 1 --2- 172.21.15.92:0/112511473 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1fd8146580 0x7f1fd8146a30 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f1fd806e2d0 tx=0x7f1fd000f520 comp rx=0 tx=0).stop 2023-11-24T06:11:12.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.685+0000 7f1fe0108640 1 -- 172.21.15.92:0/112511473 shutdown_connections 2023-11-24T06:11:12.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.685+0000 7f1fe0108640 1 --2- 172.21.15.92:0/112511473 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f1fbc06ae40 0x7f1fbc06d300 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:12.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.685+0000 7f1fe0108640 1 --2- 172.21.15.92:0/112511473 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1fd8146580 0x7f1fd8146a30 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:12.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.685+0000 7f1fe0108640 1 --2- 172.21.15.92:0/112511473 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1fd813f440 0x7f1fd8163c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:12.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.685+0000 7f1fe0108640 1 --2- 172.21.15.92:0/112511473 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1fc40039c0 0x7f1fd8157f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:12.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.685+0000 7f1fe0108640 1 -- 172.21.15.92:0/112511473 >> 172.21.15.92:0/112511473 conn(0x7f1fd806a800 msgr2=0x7f1fd80cdd90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:12.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.685+0000 7f1fe0108640 1 -- 172.21.15.92:0/112511473 shutdown_connections 2023-11-24T06:11:12.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.685+0000 7f1fe0108640 1 -- 172.21.15.92:0/112511473 wait complete. 2023-11-24T06:11:12.709 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-24T06:11:12.911 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.907+0000 7f5e1d2ee640 1 Processor -- start 2023-11-24T06:11:12.911 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.908+0000 7f5e1d2ee640 1 -- start start 2023-11-24T06:11:12.912 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.909+0000 7f5e1d2ee640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f5e180cf9a0 0x7f5e18166030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:12.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.909+0000 7f5e1d2ee640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e1815c5f0 con 0x7f5e1813e360 2023-11-24T06:11:12.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.909+0000 7f5e1d2ee640 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e1806c470 con 0x7f5e1813df80 2023-11-24T06:11:12.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.909+0000 7f5e1d2ee640 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e1806c750 con 0x7f5e180cf9a0 2023-11-24T06:11:12.914 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.910+0000 7f5e167fc640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f5e1813e360 0x7f5e180cf290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:54266/0 (socket says 172.21.15.92:54266) 2023-11-24T06:11:12.915 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.910+0000 7f5e167fc640 1 -- 172.21.15.92:0/2979767345 learned_addr learned my addr 172.21.15.92:0/2979767345 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-24T06:11:12.915 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.910+0000 7f5e177fe640 1 --2- 172.21.15.92:0/2979767345 >> v2:172.21.15.118:3300/0 conn(0x7f5e180cf9a0 0x7f5e18166030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:12.915 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.910+0000 7f5e15ffb640 1 -- 172.21.15.92:0/2979767345 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1565970289 0 0) 0x7f5e1815c5f0 con 0x7f5e1813e360 2023-11-24T06:11:12.915 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.910+0000 7f5e15ffb640 1 -- 172.21.15.92:0/2979767345 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5e04003520 con 0x7f5e1813e360 2023-11-24T06:11:12.915 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.911+0000 7f5e15ffb640 1 -- 172.21.15.92:0/2979767345 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 530113612 0 0) 0x7f5e04003520 con 0x7f5e1813e360 2023-11-24T06:11:12.915 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.911+0000 7f5e15ffb640 1 -- 172.21.15.92:0/2979767345 >> v2:172.21.15.118:3300/0 conn(0x7f5e180cf9a0 msgr2=0x7f5e18166030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:12.915 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.911+0000 7f5e15ffb640 1 --2- 172.21.15.92:0/2979767345 >> v2:172.21.15.118:3300/0 conn(0x7f5e180cf9a0 0x7f5e18166030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:12.915 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.911+0000 7f5e15ffb640 1 -- 172.21.15.92:0/2979767345 >> v1:172.21.15.118:6789/0 conn(0x7f5e1813df80 legacy=0x7f5e1815bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:12.915 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.911+0000 7f5e15ffb640 1 -- 172.21.15.92:0/2979767345 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e1806ca30 con 0x7f5e1813e360 2023-11-24T06:11:12.915 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.911+0000 7f5e15ffb640 1 -- 172.21.15.92:0/2979767345 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2607661350 0 0) 0x7f5e00001670 con 0x7f5e1813e360 2023-11-24T06:11:12.915 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.911+0000 7f5e15ffb640 1 -- 172.21.15.92:0/2979767345 >> v1:172.21.15.92:6789/0 conn(0x7f5e1813e360 legacy=0x7f5e180cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:12.916 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.911+0000 7f5e15ffb640 1 --2- 172.21.15.92:0/2979767345 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5e04003970 0x7f5e04003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:12.916 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.911+0000 7f5e15ffb640 1 --2- 172.21.15.92:0/2979767345 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5e040047c0 0x7f5e04004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:12.916 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.911+0000 7f5e15ffb640 1 --2- 172.21.15.92:0/2979767345 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5e04005180 0x7f5e0400fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:12.916 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.911+0000 7f5e15ffb640 1 -- 172.21.15.92:0/2979767345 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e1806c750 con 0x7f5e04003970 2023-11-24T06:11:12.916 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.911+0000 7f5e15ffb640 1 -- 172.21.15.92:0/2979767345 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e040104e0 con 0x7f5e04005180 2023-11-24T06:11:12.916 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.911+0000 7f5e15ffb640 1 -- 172.21.15.92:0/2979767345 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e040106a0 con 0x7f5e040047c0 2023-11-24T06:11:12.916 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.911+0000 7f5e16ffd640 1 --2- 172.21.15.92:0/2979767345 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5e04003970 0x7f5e04003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:12.916 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.911+0000 7f5e177fe640 1 --2- 172.21.15.92:0/2979767345 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5e04005180 0x7f5e0400fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:12.916 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.911+0000 7f5e167fc640 1 --2- 172.21.15.92:0/2979767345 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5e040047c0 0x7f5e04004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:12.916 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.911+0000 7f5e16ffd640 1 -- 172.21.15.92:0/2979767345 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5e040047c0 msgr2=0x7f5e04004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:12.916 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.911+0000 7f5e16ffd640 1 --2- 172.21.15.92:0/2979767345 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5e040047c0 0x7f5e04004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:12.917 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.911+0000 7f5e16ffd640 1 -- 172.21.15.92:0/2979767345 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5e04005180 msgr2=0x7f5e0400fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:12.917 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.911+0000 7f5e16ffd640 1 --2- 172.21.15.92:0/2979767345 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5e04005180 0x7f5e0400fd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:12.917 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.911+0000 7f5e177fe640 1 -- 172.21.15.92:0/2979767345 reap_dead start 2023-11-24T06:11:12.917 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.911+0000 7f5e16ffd640 1 -- 172.21.15.92:0/2979767345 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e04003520 con 0x7f5e04003970 2023-11-24T06:11:12.918 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.912+0000 7f5e16ffd640 1 --2- 172.21.15.92:0/2979767345 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5e04003970 0x7f5e04003d90 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f5e08001e50 tx=0x7f5e040057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=254e8811f857c55 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:12.918 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.913+0000 7f5e15ffb640 1 -- 172.21.15.92:0/2979767345 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e08003b70 con 0x7f5e04003970 2023-11-24T06:11:12.918 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.913+0000 7f5e15ffb640 1 -- 172.21.15.92:0/2979767345 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e08003cd0 con 0x7f5e04003970 2023-11-24T06:11:12.918 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.913+0000 7f5e15ffb640 1 -- 172.21.15.92:0/2979767345 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e08005310 con 0x7f5e04003970 2023-11-24T06:11:12.918 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.913+0000 7f5e1d2ee640 1 -- 172.21.15.92:0/2979767345 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5e04003970 msgr2=0x7f5e04003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:12.918 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.913+0000 7f5e1d2ee640 1 --2- 172.21.15.92:0/2979767345 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5e04003970 0x7f5e04003d90 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f5e08001e50 tx=0x7f5e040057f0 comp rx=0 tx=0).stop 2023-11-24T06:11:12.918 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.913+0000 7f5e1d2ee640 1 -- 172.21.15.92:0/2979767345 shutdown_connections 2023-11-24T06:11:12.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.913+0000 7f5e1d2ee640 1 --2- 172.21.15.92:0/2979767345 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5e04003970 0x7f5e04003d90 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:12.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.913+0000 7f5e1d2ee640 1 -- 172.21.15.92:0/2979767345 >> 172.21.15.92:0/2979767345 conn(0x7f5e1806a800 msgr2=0x7f5e1806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:12.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.914+0000 7f5e1d2ee640 1 -- 172.21.15.92:0/2979767345 shutdown_connections 2023-11-24T06:11:12.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.914+0000 7f5e1d2ee640 1 -- 172.21.15.92:0/2979767345 wait complete. 2023-11-24T06:11:12.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.916+0000 7f5e1d2ee640 1 Processor -- start 2023-11-24T06:11:12.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.916+0000 7f5e1d2ee640 1 -- start start 2023-11-24T06:11:12.921 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.917+0000 7f5e1d2ee640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5e04003970 0x7f5e1814ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:12.921 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.917+0000 7f5e1d2ee640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5e181391c0 0x7f5e18070630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:12.921 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.917+0000 7f5e1d2ee640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5e180b39f0 0x7f5e181389a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:12.921 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.917+0000 7f5e1d2ee640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e180ce2b0 con 0x7f5e181391c0 2023-11-24T06:11:12.921 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.917+0000 7f5e1d2ee640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e180ce7e0 con 0x7f5e180b39f0 2023-11-24T06:11:12.921 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.917+0000 7f5e1d2ee640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e18072310 con 0x7f5e04003970 2023-11-24T06:11:12.922 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.918+0000 7f5e167fc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5e181391c0 0x7f5e18070630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:12.922 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.918+0000 7f5e167fc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5e181391c0 0x7f5e18070630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:32900/0 (socket says 172.21.15.92:32900) 2023-11-24T06:11:12.922 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.918+0000 7f5e167fc640 1 -- 172.21.15.92:0/539730936 learned_addr learned my addr 172.21.15.92:0/539730936 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:11:12.922 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.918+0000 7f5e167fc640 1 -- 172.21.15.92:0/539730936 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5e04003970 msgr2=0x7f5e1814ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:12.922 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.918+0000 7f5e167fc640 1 --2- 172.21.15.92:0/539730936 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5e04003970 0x7f5e1814ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:12.922 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.918+0000 7f5e167fc640 1 -- 172.21.15.92:0/539730936 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5e180b39f0 msgr2=0x7f5e181389a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:12.922 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.918+0000 7f5e167fc640 1 --2- 172.21.15.92:0/539730936 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5e180b39f0 0x7f5e181389a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:12.922 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.918+0000 7f5e167fc640 1 -- 172.21.15.92:0/539730936 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e180ce500 con 0x7f5e181391c0 2023-11-24T06:11:12.923 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.918+0000 7f5e167fc640 1 --2- 172.21.15.92:0/539730936 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5e181391c0 0x7f5e18070630 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f5e00000ff0 tx=0x7f5e0000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:12.923 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.918+0000 7f5e0f7fe640 1 -- 172.21.15.92:0/539730936 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e000072d0 con 0x7f5e181391c0 2023-11-24T06:11:12.923 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.919+0000 7f5e0f7fe640 1 -- 172.21.15.92:0/539730936 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e00007430 con 0x7f5e181391c0 2023-11-24T06:11:12.923 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.919+0000 7f5e0f7fe640 1 -- 172.21.15.92:0/539730936 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e000047e0 con 0x7f5e181391c0 2023-11-24T06:11:12.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.919+0000 7f5e1d2ee640 1 -- 172.21.15.92:0/539730936 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e18138ee0 con 0x7f5e181391c0 2023-11-24T06:11:12.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.919+0000 7f5e1d2ee640 1 -- 172.21.15.92:0/539730936 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e18071f30 con 0x7f5e181391c0 2023-11-24T06:11:12.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.920+0000 7f5e1d2ee640 1 -- 172.21.15.92:0/539730936 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e18070350 con 0x7f5e181391c0 2023-11-24T06:11:12.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.921+0000 7f5e0f7fe640 1 -- 172.21.15.92:0/539730936 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5e00004940 con 0x7f5e181391c0 2023-11-24T06:11:12.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.922+0000 7f5e0f7fe640 1 --2- 172.21.15.92:0/539730936 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f5df806ae70 0x7f5df806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:12.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.922+0000 7f5e0f7fe640 1 -- 172.21.15.92:0/539730936 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5883+0+0 (secure 0 0 0) 0x7f5e00088e90 con 0x7f5e181391c0 2023-11-24T06:11:12.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.922+0000 7f5e16ffd640 1 --2- 172.21.15.92:0/539730936 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f5df806ae70 0x7f5df806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:12.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.923+0000 7f5e16ffd640 1 --2- 172.21.15.92:0/539730936 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f5df806ae70 0x7f5df806d330 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f5e08034000 tx=0x7f5e08030000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:12.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:12.926+0000 7f5e0f7fe640 1 -- 172.21.15.92:0/539730936 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5e18070350 con 0x7f5e181391c0 2023-11-24T06:11:13.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:13.127+0000 7f5e1d2ee640 1 -- 172.21.15.92:0/539730936 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f5e18070350 con 0x7f5e181391c0 2023-11-24T06:11:13.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:13.128+0000 7f5e0f7fe640 1 -- 172.21.15.92:0/539730936 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) v1 ==== 80+0+0 (secure 0 0 0) 0x7f5e18070350 con 0x7f5e181391c0 2023-11-24T06:11:13.131 INFO:teuthology.orchestra.run.smithi092.stderr:listed 0 entries 2023-11-24T06:11:13.133 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:13.130+0000 7f5e1d2ee640 1 -- 172.21.15.92:0/539730936 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f5df806ae70 msgr2=0x7f5df806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:13.133 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:13.130+0000 7f5e1d2ee640 1 --2- 172.21.15.92:0/539730936 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f5df806ae70 0x7f5df806d330 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f5e08034000 tx=0x7f5e08030000 comp rx=0 tx=0).stop 2023-11-24T06:11:13.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:13.130+0000 7f5e1d2ee640 1 -- 172.21.15.92:0/539730936 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5e181391c0 msgr2=0x7f5e18070630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:13.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:13.130+0000 7f5e1d2ee640 1 --2- 172.21.15.92:0/539730936 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5e181391c0 0x7f5e18070630 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f5e00000ff0 tx=0x7f5e0000dc00 comp rx=0 tx=0).stop 2023-11-24T06:11:13.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:13.130+0000 7f5e1d2ee640 1 -- 172.21.15.92:0/539730936 shutdown_connections 2023-11-24T06:11:13.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:13.130+0000 7f5e1d2ee640 1 --2- 172.21.15.92:0/539730936 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f5df806ae70 0x7f5df806d330 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:13.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:13.130+0000 7f5e1d2ee640 1 --2- 172.21.15.92:0/539730936 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5e180b39f0 0x7f5e181389a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:13.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:13.130+0000 7f5e1d2ee640 1 --2- 172.21.15.92:0/539730936 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5e181391c0 0x7f5e18070630 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:13.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:13.130+0000 7f5e1d2ee640 1 --2- 172.21.15.92:0/539730936 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5e04003970 0x7f5e1814ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:13.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:13.130+0000 7f5e1d2ee640 1 -- 172.21.15.92:0/539730936 >> 172.21.15.92:0/539730936 conn(0x7f5e1806a800 msgr2=0x7f5e18151fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:13.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:13.130+0000 7f5e1d2ee640 1 -- 172.21.15.92:0/539730936 shutdown_connections 2023-11-24T06:11:13.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:13.130+0000 7f5e1d2ee640 1 -- 172.21.15.92:0/539730936 wait complete. 2023-11-24T06:11:13.152 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi092.front.sepia.ceph.com... 2023-11-24T06:11:13.152 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:11:13.152 DEBUG:teuthology.orchestra.run.smithi092:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-11-24T06:11:13.252 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T06:11:13.252 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi092.front.sepia.ceph.com 2023-11-24T06:11:13.252 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:11:13.253 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-24T06:11:13.311 DEBUG:teuthology.orchestra.run.smithi118:> set -ex 2023-11-24T06:11:13.312 DEBUG:teuthology.orchestra.run.smithi118:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:11:13.372 DEBUG:teuthology.orchestra.run.smithi118:> set -ex 2023-11-24T06:11:13.372 DEBUG:teuthology.orchestra.run.smithi118:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:11:13.426 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:11:13.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:13.643+0000 7f804e415640 1 Processor -- start 2023-11-24T06:11:13.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:13.643+0000 7f804e415640 1 -- start start 2023-11-24T06:11:13.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:13.645+0000 7f804e415640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f804813e360 0x7f80480cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:13.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:13.645+0000 7f804e415640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f804815c5f0 con 0x7f804813df80 2023-11-24T06:11:13.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:13.645+0000 7f804e415640 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f804806c470 con 0x7f804815f9c0 2023-11-24T06:11:13.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:13.645+0000 7f804e415640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f804806c750 con 0x7f804813e360 2023-11-24T06:11:13.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:13.645+0000 7f8047fff640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f804813df80 0x7f804815bee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:54274/0 (socket says 172.21.15.92:54274) 2023-11-24T06:11:13.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:13.645+0000 7f8047fff640 1 -- 172.21.15.92:0/3420924657 learned_addr learned my addr 172.21.15.92:0/3420924657 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-24T06:11:13.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:13.645+0000 7f803ffff640 1 --2- 172.21.15.92:0/3420924657 >> v2:172.21.15.92:3300/0 conn(0x7f804813e360 0x7f80480cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:13.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:13.646+0000 7f80477fe640 1 -- 172.21.15.92:0/3420924657 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2888915367 0 0) 0x7f804815c5f0 con 0x7f804813df80 2023-11-24T06:11:13.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:13.646+0000 7f80477fe640 1 -- 172.21.15.92:0/3420924657 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8034003520 con 0x7f804813df80 2023-11-24T06:11:13.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:13.646+0000 7f803ffff640 1 -- 172.21.15.92:0/3420924657 >> v1:172.21.15.118:6789/0 conn(0x7f804815f9c0 legacy=0x7f80480cd840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:13.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:13.646+0000 7f803ffff640 1 -- 172.21.15.92:0/3420924657 >> v1:172.21.15.92:6789/0 conn(0x7f804813df80 legacy=0x7f804815bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:13.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:13.646+0000 7f803ffff640 1 -- 172.21.15.92:0/3420924657 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f804806ca30 con 0x7f804813e360 2023-11-24T06:11:13.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:13.647+0000 7f803ffff640 1 --2- 172.21.15.92:0/3420924657 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f804813e360 0x7f80480cf290 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f80300097a0 tx=0x7f803002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7668f40fef3c7e54 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:13.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:13.647+0000 7f80477fe640 1 -- 172.21.15.92:0/3420924657 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8030031830 con 0x7f804813e360 2023-11-24T06:11:13.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:13.647+0000 7f80477fe640 1 -- 172.21.15.92:0/3420924657 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8030031990 con 0x7f804813e360 2023-11-24T06:11:13.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:13.647+0000 7f80477fe640 1 -- 172.21.15.92:0/3420924657 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8030031d20 con 0x7f804813e360 2023-11-24T06:11:13.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:13.648+0000 7f804e415640 1 -- 172.21.15.92:0/3420924657 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f804813e360 msgr2=0x7f80480cf290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:13.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:13.648+0000 7f804e415640 1 --2- 172.21.15.92:0/3420924657 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f804813e360 0x7f80480cf290 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f80300097a0 tx=0x7f803002b8d0 comp rx=0 tx=0).stop 2023-11-24T06:11:13.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:13.648+0000 7f804e415640 1 -- 172.21.15.92:0/3420924657 shutdown_connections 2023-11-24T06:11:13.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:13.648+0000 7f804e415640 1 --2- 172.21.15.92:0/3420924657 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f804813e360 0x7f80480cf290 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:13.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:13.648+0000 7f804e415640 1 -- 172.21.15.92:0/3420924657 >> 172.21.15.92:0/3420924657 conn(0x7f804806a800 msgr2=0x7f804806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:13.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:13.648+0000 7f804e415640 1 -- 172.21.15.92:0/3420924657 shutdown_connections 2023-11-24T06:11:13.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:13.648+0000 7f804e415640 1 -- 172.21.15.92:0/3420924657 wait complete. 2023-11-24T06:11:13.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:13.650+0000 7f804e415640 1 Processor -- start 2023-11-24T06:11:13.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:13.650+0000 7f804e415640 1 -- start start 2023-11-24T06:11:13.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:13.651+0000 7f804e415640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f804813df80 0x7f80480714c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:13.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:13.651+0000 7f804e415640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f804813e360 0x7f804814df60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:13.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:13.651+0000 7f804e415640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f804815f9c0 0x7f80480b3290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:13.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:13.651+0000 7f804e415640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f804814ee20 con 0x7f804813e360 2023-11-24T06:11:13.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:13.651+0000 7f804e415640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8048071030 con 0x7f804813df80 2023-11-24T06:11:13.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:13.651+0000 7f804e415640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f804814e860 con 0x7f804815f9c0 2023-11-24T06:11:13.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:13.651+0000 7f803ffff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f804813e360 0x7f804814df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:13.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:13.651+0000 7f803ffff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f804813e360 0x7f804814df60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:32922/0 (socket says 172.21.15.92:32922) 2023-11-24T06:11:13.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:13.651+0000 7f803ffff640 1 -- 172.21.15.92:0/2938146928 learned_addr learned my addr 172.21.15.92:0/2938146928 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:11:13.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:13.652+0000 7f804c9b2640 1 --2- 172.21.15.92:0/2938146928 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f804815f9c0 0x7f80480b3290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:13.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:13.652+0000 7f8047fff640 1 --2- 172.21.15.92:0/2938146928 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f804813df80 0x7f80480714c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:13.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:13.652+0000 7f803ffff640 1 -- 172.21.15.92:0/2938146928 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f804815f9c0 msgr2=0x7f80480b3290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:13.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:13.652+0000 7f803ffff640 1 --2- 172.21.15.92:0/2938146928 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f804815f9c0 0x7f80480b3290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:13.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:13.652+0000 7f803ffff640 1 -- 172.21.15.92:0/2938146928 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f804813df80 msgr2=0x7f80480714c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:13.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:13.652+0000 7f803ffff640 1 --2- 172.21.15.92:0/2938146928 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f804813df80 0x7f80480714c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:13.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:13.652+0000 7f803ffff640 1 -- 172.21.15.92:0/2938146928 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8030009400 con 0x7f804813e360 2023-11-24T06:11:13.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:13.652+0000 7f804c9b2640 1 --2- 172.21.15.92:0/2938146928 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f804815f9c0 0x7f80480b3290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:11:13.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:13.652+0000 7f8047fff640 1 --2- 172.21.15.92:0/2938146928 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f804813df80 0x7f80480714c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:11:13.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:13.652+0000 7f803ffff640 1 --2- 172.21.15.92:0/2938146928 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f804813e360 0x7f804814df60 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f80300098c0 tx=0x7f80300046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:13.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:13.652+0000 7f80457fa640 1 -- 172.21.15.92:0/2938146928 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f803003a6f0 con 0x7f804813e360 2023-11-24T06:11:13.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:13.652+0000 7f80457fa640 1 -- 172.21.15.92:0/2938146928 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f803003a870 con 0x7f804813e360 2023-11-24T06:11:13.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:13.652+0000 7f804e415640 1 -- 172.21.15.92:0/2938146928 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f804814eb40 con 0x7f804813e360 2023-11-24T06:11:13.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:13.652+0000 7f80457fa640 1 -- 172.21.15.92:0/2938146928 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f803003ac20 con 0x7f804813e360 2023-11-24T06:11:13.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:13.652+0000 7f804e415640 1 -- 172.21.15.92:0/2938146928 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8048071a00 con 0x7f804813e360 2023-11-24T06:11:13.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:13.654+0000 7f804e415640 1 -- 172.21.15.92:0/2938146928 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8048070b20 con 0x7f804813e360 2023-11-24T06:11:13.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:13.655+0000 7f80457fa640 1 -- 172.21.15.92:0/2938146928 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f803004c3b0 con 0x7f804813e360 2023-11-24T06:11:13.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:13.656+0000 7f80457fa640 1 --2- 172.21.15.92:0/2938146928 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f802806ae70 0x7f802806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:13.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:13.656+0000 7f80457fa640 1 -- 172.21.15.92:0/2938146928 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5883+0+0 (secure 0 0 0) 0x7f80300ab300 con 0x7f804813e360 2023-11-24T06:11:13.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:13.656+0000 7f8047fff640 1 --2- 172.21.15.92:0/2938146928 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f802806ae70 0x7f802806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:13.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:13.657+0000 7f8047fff640 1 --2- 172.21.15.92:0/2938146928 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f802806ae70 0x7f802806d330 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f8038005880 tx=0x7f8038009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:13.666 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:13.660+0000 7f80457fa640 1 -- 172.21.15.92:0/2938146928 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8048070b20 con 0x7f804813e360 2023-11-24T06:11:13.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:13.857+0000 7f804e415640 1 -- 172.21.15.92:0/2938146928 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8048070b20 con 0x7f804813e360 2023-11-24T06:11:13.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:13.859+0000 7f80457fa640 1 -- 172.21.15.92:0/2938146928 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16800 (secure 0 0 0) 0x7f8048070b20 con 0x7f804813e360 2023-11-24T06:11:13.863 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-24T06:11:13.863 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":22,"fsid":"2d6dee5f-8aa3-470f-931a-50ade963b1cb","created":"2023-11-24T06:10:04.707204+0000","modified":"2023-11-24T06:11:11.472401+0000","last_up_change":"2023-11-24T06:10:17.598352+0000","last_in_change":"2023-11-24T06:10:09.040793+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:10:17.733098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:10:22.926445+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T06:10:24.151288+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ff37a9fb-135f-471f-b4b0-f7b5772006c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6803","nonce":20560714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6805","nonce":20560714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6809","nonce":20560714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6807","nonce":20560714}]},"public_addr":"172.21.15.92:6803/20560714","cluster_addr":"172.21.15.92:6805/20560714","heartbeat_back_addr":"172.21.15.92:6809/20560714","heartbeat_front_addr":"172.21.15.92:6807/20560714","state":["exists","up"]},{"osd":1,"uuid":"03cdd353-dcf1-4f38-94fe-a795137c06b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6819","nonce":2451239906}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6821","nonce":2451239906}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6825","nonce":2451239906}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6823","nonce":2451239906}]},"public_addr":"172.21.15.92:6819/2451239906","cluster_addr":"172.21.15.92:6821/2451239906","heartbeat_back_addr":"172.21.15.92:6825/2451239906","heartbeat_front_addr":"172.21.15.92:6823/2451239906","state":["exists","up"]},{"osd":2,"uuid":"a4a20d80-5d57-484b-becd-9aa8070dd3aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6811","nonce":1124979277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6813","nonce":1124979277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6817","nonce":1124979277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6815","nonce":1124979277}]},"public_addr":"172.21.15.92:6811/1124979277","cluster_addr":"172.21.15.92:6813/1124979277","heartbeat_back_addr":"172.21.15.92:6817/1124979277","heartbeat_front_addr":"172.21.15.92:6815/1124979277","state":["exists","up"]},{"osd":3,"uuid":"4648d7ce-9b29-4583-9425-4ba8fd2b1903","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6827","nonce":1734115782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6829","nonce":1734115782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6833","nonce":1734115782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6831","nonce":1734115782}]},"public_addr":"172.21.15.92:6827/1734115782","cluster_addr":"172.21.15.92:6829/1734115782","heartbeat_back_addr":"172.21.15.92:6833/1734115782","heartbeat_front_addr":"172.21.15.92:6831/1734115782","state":["exists","up"]},{"osd":4,"uuid":"3bc9d7eb-c6fa-4fcf-a3ec-ade604e05aac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6819","nonce":3425937229}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6822","nonce":3425937229}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6830","nonce":3425937229}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6826","nonce":3425937229}]},"public_addr":"172.21.15.118:6819/3425937229","cluster_addr":"172.21.15.118:6822/3425937229","heartbeat_back_addr":"172.21.15.118:6830/3425937229","heartbeat_front_addr":"172.21.15.118:6826/3425937229","state":["exists","up"]},{"osd":5,"uuid":"dce9ba6d-59ed-46d1-84fc-a84b13de10c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6801","nonce":827455984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6803","nonce":827455984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6807","nonce":827455984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6805","nonce":827455984}]},"public_addr":"172.21.15.118:6801/827455984","cluster_addr":"172.21.15.118:6803/827455984","heartbeat_back_addr":"172.21.15.118:6807/827455984","heartbeat_front_addr":"172.21.15.118:6805/827455984","state":["exists","up"]},{"osd":6,"uuid":"82aec504-ae58-4709-8d30-46f2b12081b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6809","nonce":2946986141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6811","nonce":2946986141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6815","nonce":2946986141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6813","nonce":2946986141}]},"public_addr":"172.21.15.118:6809/2946986141","cluster_addr":"172.21.15.118:6811/2946986141","heartbeat_back_addr":"172.21.15.118:6815/2946986141","heartbeat_front_addr":"172.21.15.118:6813/2946986141","state":["exists","up"]},{"osd":7,"uuid":"4ed4c624-c88f-4d35-bdb1-bf209b5dffb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6817","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6818","nonce":48122059}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6821","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6823","nonce":48122059}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6829","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6831","nonce":48122059}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6825","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6827","nonce":48122059}]},"public_addr":"172.21.15.118:6818/48122059","cluster_addr":"172.21.15.118:6823/48122059","heartbeat_back_addr":"172.21.15.118:6831/48122059","heartbeat_front_addr":"172.21.15.118:6827/48122059","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.536664+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.368324+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.271201+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.342971+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.621832+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.387110+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.632946+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.416032+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.d","mappings":[{"from":5,"to":1}]},{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"3.29","mappings":[{"from":3,"to":2}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.34","mappings":[{"from":4,"to":2}]},{"pgid":"3.3b","mappings":[{"from":7,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:11:13.866 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:13.863+0000 7f804e415640 1 -- 172.21.15.92:0/2938146928 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f802806ae70 msgr2=0x7f802806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:13.866 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:13.863+0000 7f804e415640 1 --2- 172.21.15.92:0/2938146928 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f802806ae70 0x7f802806d330 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f8038005880 tx=0x7f8038009040 comp rx=0 tx=0).stop 2023-11-24T06:11:13.866 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:13.863+0000 7f804e415640 1 -- 172.21.15.92:0/2938146928 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f804813e360 msgr2=0x7f804814df60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:13.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:13.863+0000 7f804e415640 1 --2- 172.21.15.92:0/2938146928 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f804813e360 0x7f804814df60 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f80300098c0 tx=0x7f80300046c0 comp rx=0 tx=0).stop 2023-11-24T06:11:13.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:13.863+0000 7f804e415640 1 -- 172.21.15.92:0/2938146928 shutdown_connections 2023-11-24T06:11:13.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:13.863+0000 7f804e415640 1 --2- 172.21.15.92:0/2938146928 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f802806ae70 0x7f802806d330 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:13.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:13.863+0000 7f804e415640 1 --2- 172.21.15.92:0/2938146928 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f804815f9c0 0x7f80480b3290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:13.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:13.863+0000 7f804e415640 1 --2- 172.21.15.92:0/2938146928 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f804813e360 0x7f804814df60 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:13.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:13.863+0000 7f804e415640 1 --2- 172.21.15.92:0/2938146928 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f804813df80 0x7f80480714c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:13.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:13.863+0000 7f804e415640 1 -- 172.21.15.92:0/2938146928 >> 172.21.15.92:0/2938146928 conn(0x7f804806a800 msgr2=0x7f804815cef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:13.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:13.863+0000 7f804e415640 1 -- 172.21.15.92:0/2938146928 shutdown_connections 2023-11-24T06:11:13.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:13.863+0000 7f804e415640 1 -- 172.21.15.92:0/2938146928 wait complete. 2023-11-24T06:11:13.884 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T06:10:17.733098+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T06:10:22.926445+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T06:10:24.151288+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T06:11:13.885 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T06:11:14.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.088+0000 7f976d2b4640 1 Processor -- start 2023-11-24T06:11:14.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.089+0000 7f976d2b4640 1 -- start start 2023-11-24T06:11:14.094 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.091+0000 7f976d2b4640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f9768071660 0x7f976815dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:14.094 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.091+0000 7f976d2b4640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f9768071a40 0x7f976813d840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:14.094 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.091+0000 7f976d2b4640 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97680cebf0 con 0x7f976806f2a0 2023-11-24T06:11:14.094 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.091+0000 7f976d2b4640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f976815e2a0 con 0x7f9768071660 2023-11-24T06:11:14.094 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.091+0000 7f976d2b4640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f976815e580 con 0x7f9768071a40 2023-11-24T06:11:14.094 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.091+0000 7f9766ffd640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f9768071660 0x7f976815dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:14.094 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.091+0000 7f9766ffd640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f9768071660 0x7f976815dcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:32936/0 (socket says 172.21.15.92:32936) 2023-11-24T06:11:14.095 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.091+0000 7f9766ffd640 1 -- 172.21.15.92:0/1255956474 learned_addr learned my addr 172.21.15.92:0/1255956474 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:11:14.095 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.091+0000 7f975ffff640 1 --2- 172.21.15.92:0/1255956474 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f9768071a40 0x7f976813d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:14.095 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.091+0000 7f9766ffd640 1 -- 172.21.15.92:0/1255956474 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f9768071a40 msgr2=0x7f976813d840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:14.095 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.091+0000 7f9766ffd640 1 --2- 172.21.15.92:0/1255956474 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f9768071a40 0x7f976813d840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:14.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.091+0000 7f9766ffd640 1 -- 172.21.15.92:0/1255956474 >> v1:172.21.15.118:6789/0 conn(0x7f976806f2a0 legacy=0x7f97680cf080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:14.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.091+0000 7f9766ffd640 1 -- 172.21.15.92:0/1255956474 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f976815e860 con 0x7f9768071660 2023-11-24T06:11:14.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.091+0000 7f975ffff640 1 --2- 172.21.15.92:0/1255956474 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f9768071a40 0x7f976813d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:11:14.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.093+0000 7f9766ffd640 1 --2- 172.21.15.92:0/1255956474 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9768071660 0x7f976815dcc0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f9758009870 tx=0x7f975802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5196095c5c2a6486 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:14.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.093+0000 7f97667fc640 1 -- 172.21.15.92:0/1255956474 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9758031640 con 0x7f9768071660 2023-11-24T06:11:14.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.093+0000 7f97667fc640 1 -- 172.21.15.92:0/1255956474 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97580317a0 con 0x7f9768071660 2023-11-24T06:11:14.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.093+0000 7f97667fc640 1 -- 172.21.15.92:0/1255956474 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9758031b30 con 0x7f9768071660 2023-11-24T06:11:14.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.093+0000 7f976d2b4640 1 -- 172.21.15.92:0/1255956474 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9768071660 msgr2=0x7f976815dcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:14.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.093+0000 7f976d2b4640 1 --2- 172.21.15.92:0/1255956474 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9768071660 0x7f976815dcc0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f9758009870 tx=0x7f975802b8f0 comp rx=0 tx=0).stop 2023-11-24T06:11:14.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.094+0000 7f976d2b4640 1 -- 172.21.15.92:0/1255956474 shutdown_connections 2023-11-24T06:11:14.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.094+0000 7f976d2b4640 1 --2- 172.21.15.92:0/1255956474 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f9768071a40 0x7f976813d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:14.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.094+0000 7f976d2b4640 1 --2- 172.21.15.92:0/1255956474 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9768071660 0x7f976815dcc0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:14.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.094+0000 7f976d2b4640 1 -- 172.21.15.92:0/1255956474 >> 172.21.15.92:0/1255956474 conn(0x7f97680b1dd0 msgr2=0x7f97680b21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:14.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.094+0000 7f976d2b4640 1 -- 172.21.15.92:0/1255956474 shutdown_connections 2023-11-24T06:11:14.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.094+0000 7f976d2b4640 1 -- 172.21.15.92:0/1255956474 wait complete. 2023-11-24T06:11:14.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.095+0000 7f976d2b4640 1 Processor -- start 2023-11-24T06:11:14.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.096+0000 7f976d2b4640 1 -- start start 2023-11-24T06:11:14.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.096+0000 7f976d2b4640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f976806f2a0 0x7f976813bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:14.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.096+0000 7f976d2b4640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9768071660 0x7f9768138ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:14.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.096+0000 7f976d2b4640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f9768071a40 0x7f97681f30f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:14.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.096+0000 7f976d2b4640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97680ad4f0 con 0x7f9768071660 2023-11-24T06:11:14.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.097+0000 7f976d2b4640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9768139f70 con 0x7f976806f2a0 2023-11-24T06:11:14.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.097+0000 7f976d2b4640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9768139420 con 0x7f9768071a40 2023-11-24T06:11:14.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.097+0000 7f975ffff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9768071660 0x7f9768138ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:14.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.097+0000 7f975ffff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9768071660 0x7f9768138ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:32952/0 (socket says 172.21.15.92:32952) 2023-11-24T06:11:14.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.097+0000 7f975ffff640 1 -- 172.21.15.92:0/1907071412 learned_addr learned my addr 172.21.15.92:0/1907071412 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:11:14.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.097+0000 7f975ffff640 1 -- 172.21.15.92:0/1907071412 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f9768071a40 msgr2=0x7f97681f30f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:14.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.097+0000 7f975ffff640 1 --2- 172.21.15.92:0/1907071412 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f9768071a40 0x7f97681f30f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:14.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.097+0000 7f975ffff640 1 -- 172.21.15.92:0/1907071412 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f976806f2a0 msgr2=0x7f976813bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:14.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.097+0000 7f975ffff640 1 --2- 172.21.15.92:0/1907071412 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f976806f2a0 0x7f976813bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:14.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.097+0000 7f975ffff640 1 -- 172.21.15.92:0/1907071412 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97580094d0 con 0x7f9768071660 2023-11-24T06:11:14.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.097+0000 7f975ffff640 1 --2- 172.21.15.92:0/1907071412 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9768071660 0x7f9768138ee0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f975000c7f0 tx=0x7f975000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:14.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.098+0000 7f975f7fe640 1 -- 172.21.15.92:0/1907071412 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9750007820 con 0x7f9768071660 2023-11-24T06:11:14.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.098+0000 7f975f7fe640 1 -- 172.21.15.92:0/1907071412 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9750007980 con 0x7f9768071660 2023-11-24T06:11:14.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.098+0000 7f975f7fe640 1 -- 172.21.15.92:0/1907071412 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9750007cf0 con 0x7f9768071660 2023-11-24T06:11:14.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.098+0000 7f976d2b4640 1 -- 172.21.15.92:0/1907071412 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97681396d0 con 0x7f9768071660 2023-11-24T06:11:14.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.098+0000 7f976d2b4640 1 -- 172.21.15.92:0/1907071412 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9768139c90 con 0x7f9768071660 2023-11-24T06:11:14.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.099+0000 7f976d2b4640 1 -- 172.21.15.92:0/1907071412 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97681435a0 con 0x7f9768071660 2023-11-24T06:11:14.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.100+0000 7f975f7fe640 1 -- 172.21.15.92:0/1907071412 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f97500113f0 con 0x7f9768071660 2023-11-24T06:11:14.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.101+0000 7f975f7fe640 1 --2- 172.21.15.92:0/1907071412 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f974806ae40 0x7f974806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:14.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.101+0000 7f975f7fe640 1 -- 172.21.15.92:0/1907071412 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5883+0+0 (secure 0 0 0) 0x7f97500899e0 con 0x7f9768071660 2023-11-24T06:11:14.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.101+0000 7f9766ffd640 1 --2- 172.21.15.92:0/1907071412 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f974806ae40 0x7f974806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:14.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.102+0000 7f9766ffd640 1 --2- 172.21.15.92:0/1907071412 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f974806ae40 0x7f974806d300 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f9758036000 tx=0x7f9758030000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:14.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.105+0000 7f975f7fe640 1 -- 172.21.15.92:0/1907071412 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f97500549d0 con 0x7f9768071660 2023-11-24T06:11:14.291 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.287+0000 7f976d2b4640 1 -- 172.21.15.92:0/1907071412 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f97680675d0 con 0x7f9768071660 2023-11-24T06:11:14.291 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.288+0000 7f975f7fe640 1 -- 172.21.15.92:0/1907071412 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7f975001f0a0 con 0x7f9768071660 2023-11-24T06:11:14.291 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 1 2023-11-24T06:11:14.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.290+0000 7f976d2b4640 1 -- 172.21.15.92:0/1907071412 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f974806ae40 msgr2=0x7f974806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:14.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.290+0000 7f976d2b4640 1 --2- 172.21.15.92:0/1907071412 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f974806ae40 0x7f974806d300 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f9758036000 tx=0x7f9758030000 comp rx=0 tx=0).stop 2023-11-24T06:11:14.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.290+0000 7f976d2b4640 1 -- 172.21.15.92:0/1907071412 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9768071660 msgr2=0x7f9768138ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:14.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.290+0000 7f976d2b4640 1 --2- 172.21.15.92:0/1907071412 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9768071660 0x7f9768138ee0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f975000c7f0 tx=0x7f975000ccb0 comp rx=0 tx=0).stop 2023-11-24T06:11:14.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.291+0000 7f976d2b4640 1 -- 172.21.15.92:0/1907071412 shutdown_connections 2023-11-24T06:11:14.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.291+0000 7f976d2b4640 1 --2- 172.21.15.92:0/1907071412 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f974806ae40 0x7f974806d300 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:14.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.291+0000 7f976d2b4640 1 --2- 172.21.15.92:0/1907071412 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f9768071a40 0x7f97681f30f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:14.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.291+0000 7f976d2b4640 1 --2- 172.21.15.92:0/1907071412 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9768071660 0x7f9768138ee0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:14.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.291+0000 7f976d2b4640 1 --2- 172.21.15.92:0/1907071412 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f976806f2a0 0x7f976813bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:14.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.291+0000 7f976d2b4640 1 -- 172.21.15.92:0/1907071412 >> 172.21.15.92:0/1907071412 conn(0x7f97680b1dd0 msgr2=0x7f97680b40c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:14.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.291+0000 7f976d2b4640 1 -- 172.21.15.92:0/1907071412 shutdown_connections 2023-11-24T06:11:14.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.291+0000 7f976d2b4640 1 -- 172.21.15.92:0/1907071412 wait complete. 2023-11-24T06:11:14.310 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T06:11:14.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.525+0000 7fb40bfff640 1 Processor -- start 2023-11-24T06:11:14.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.525+0000 7fb40bfff640 1 -- start start 2023-11-24T06:11:14.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.526+0000 7fb40bfff640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fb40c06c6d0 0x7fb40c0cd8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:14.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.526+0000 7fb40bfff640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb40c0cde00 0x7fb40c0b0840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:14.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.526+0000 7fb40bfff640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb40c06b670 con 0x7fb40c15d2a0 2023-11-24T06:11:14.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.526+0000 7fb40bfff640 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb40c0ce2b0 con 0x7fb40c06c6d0 2023-11-24T06:11:14.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.526+0000 7fb40bfff640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb40c06b910 con 0x7fb40c0cde00 2023-11-24T06:11:14.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.526+0000 7fb40affd640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fb40c15d2a0 0x7fb40c0cfc10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:54276/0 (socket says 172.21.15.92:54276) 2023-11-24T06:11:14.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.526+0000 7fb40affd640 1 -- 172.21.15.92:0/501296188 learned_addr learned my addr 172.21.15.92:0/501296188 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-24T06:11:14.532 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.527+0000 7fb409ffb640 1 -- 172.21.15.92:0/501296188 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1065436828 0 0) 0x7fb40c06b670 con 0x7fb40c15d2a0 2023-11-24T06:11:14.532 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.527+0000 7fb40b7fe640 1 --2- 172.21.15.92:0/501296188 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb40c0cde00 0x7fb40c0b0840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:14.532 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.527+0000 7fb40a7fc640 1 --2- 172.21.15.92:0/501296188 >> v2:172.21.15.118:3300/0 conn(0x7fb40c06c6d0 0x7fb40c0cd8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:14.532 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.527+0000 7fb409ffb640 1 -- 172.21.15.92:0/501296188 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb3f8003520 con 0x7fb40c15d2a0 2023-11-24T06:11:14.532 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.527+0000 7fb409ffb640 1 -- 172.21.15.92:0/501296188 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3426655806 0 0) 0x7fb3f8003520 con 0x7fb40c15d2a0 2023-11-24T06:11:14.532 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.527+0000 7fb409ffb640 1 -- 172.21.15.92:0/501296188 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb40c0cde00 msgr2=0x7fb40c0b0840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:14.532 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.527+0000 7fb409ffb640 1 --2- 172.21.15.92:0/501296188 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb40c0cde00 0x7fb40c0b0840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:14.532 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.527+0000 7fb409ffb640 1 -- 172.21.15.92:0/501296188 >> v2:172.21.15.118:3300/0 conn(0x7fb40c06c6d0 msgr2=0x7fb40c0cd8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:14.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.527+0000 7fb409ffb640 1 --2- 172.21.15.92:0/501296188 >> v2:172.21.15.118:3300/0 conn(0x7fb40c06c6d0 0x7fb40c0cd8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:14.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.527+0000 7fb409ffb640 1 -- 172.21.15.92:0/501296188 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb40c06bbf0 con 0x7fb40c15d2a0 2023-11-24T06:11:14.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.527+0000 7fb409ffb640 1 -- 172.21.15.92:0/501296188 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2607661350 0 0) 0x7fb3fc0019f0 con 0x7fb40c15d2a0 2023-11-24T06:11:14.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.527+0000 7fb409ffb640 1 -- 172.21.15.92:0/501296188 >> v1:172.21.15.92:6789/0 conn(0x7fb40c15d2a0 legacy=0x7fb40c0cfc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:14.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.527+0000 7fb409ffb640 1 --2- 172.21.15.92:0/501296188 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb3f8003970 0x7fb3f8003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:14.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.527+0000 7fb409ffb640 1 --2- 172.21.15.92:0/501296188 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb40c0cde00 0x7fb3f80047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:14.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.527+0000 7fb409ffb640 1 --2- 172.21.15.92:0/501296188 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb3f8004d00 0x7fb3f8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:14.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.527+0000 7fb409ffb640 1 -- 172.21.15.92:0/501296188 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb40c0ce2b0 con 0x7fb3f8004d00 2023-11-24T06:11:14.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.527+0000 7fb409ffb640 1 -- 172.21.15.92:0/501296188 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb40c06b910 con 0x7fb3f8003970 2023-11-24T06:11:14.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.527+0000 7fb409ffb640 1 -- 172.21.15.92:0/501296188 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3f800b980 con 0x7fb40c0cde00 2023-11-24T06:11:14.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.527+0000 7fb40a7fc640 1 --2- 172.21.15.92:0/501296188 >> v2:172.21.15.118:3300/0 conn(0x7fb40c06c6d0 0x7fb40c0cd8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:11:14.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.528+0000 7fb40b7fe640 1 --2- 172.21.15.92:0/501296188 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb3f8004d00 0x7fb3f8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:14.534 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.528+0000 7fb40b7fe640 1 -- 172.21.15.92:0/501296188 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb40c0cde00 msgr2=0x7fb3f80047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:14.534 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.528+0000 7fb40b7fe640 1 --2- 172.21.15.92:0/501296188 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb40c0cde00 0x7fb3f80047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:14.534 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.528+0000 7fb40b7fe640 1 -- 172.21.15.92:0/501296188 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb3f8003970 msgr2=0x7fb3f8003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:14.534 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.528+0000 7fb40b7fe640 1 --2- 172.21.15.92:0/501296188 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb3f8003970 0x7fb3f8003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:14.534 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.528+0000 7fb40b7fe640 1 -- 172.21.15.92:0/501296188 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3f8003520 con 0x7fb3f8004d00 2023-11-24T06:11:14.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.529+0000 7fb40b7fe640 1 --2- 172.21.15.92:0/501296188 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb3f8004d00 0x7fb3f8005180 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fb404009af0 tx=0x7fb3f80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f701d70d57aadbf server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:14.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.529+0000 7fb409ffb640 1 -- 172.21.15.92:0/501296188 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb40402c710 con 0x7fb3f8004d00 2023-11-24T06:11:14.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.529+0000 7fb409ffb640 1 -- 172.21.15.92:0/501296188 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb40402c870 con 0x7fb3f8004d00 2023-11-24T06:11:14.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.529+0000 7fb409ffb640 1 -- 172.21.15.92:0/501296188 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb40402cc00 con 0x7fb3f8004d00 2023-11-24T06:11:14.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.530+0000 7fb40bfff640 1 -- 172.21.15.92:0/501296188 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb3f8004d00 msgr2=0x7fb3f8005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:14.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.530+0000 7fb40bfff640 1 --2- 172.21.15.92:0/501296188 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb3f8004d00 0x7fb3f8005180 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fb404009af0 tx=0x7fb3f80057f0 comp rx=0 tx=0).stop 2023-11-24T06:11:14.537 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.530+0000 7fb40b7fe640 1 -- 172.21.15.92:0/501296188 reap_dead start 2023-11-24T06:11:14.537 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.530+0000 7fb40bfff640 1 -- 172.21.15.92:0/501296188 shutdown_connections 2023-11-24T06:11:14.537 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.530+0000 7fb40bfff640 1 -- 172.21.15.92:0/501296188 >> 172.21.15.92:0/501296188 conn(0x7fb40c0b2a30 msgr2=0x7fb40c0b2e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:14.537 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.530+0000 7fb40bfff640 1 -- 172.21.15.92:0/501296188 shutdown_connections 2023-11-24T06:11:14.537 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.530+0000 7fb40bfff640 1 -- 172.21.15.92:0/501296188 wait complete. 2023-11-24T06:11:14.537 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.532+0000 7fb40bfff640 1 Processor -- start 2023-11-24T06:11:14.537 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.532+0000 7fb40bfff640 1 -- start start 2023-11-24T06:11:14.539 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.533+0000 7fb40bfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb3f8004d00 0x7fb40c0c6460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:14.539 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.533+0000 7fb40bfff640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb40c1f2c20 0x7fb40c0b6650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:14.539 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.533+0000 7fb40bfff640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb40c09e420 0x7fb40c1f6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:14.539 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.533+0000 7fb40bfff640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb40c0b3cc0 con 0x7fb3f8004d00 2023-11-24T06:11:14.539 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.533+0000 7fb40bfff640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb40c0b3fa0 con 0x7fb40c09e420 2023-11-24T06:11:14.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.533+0000 7fb40bfff640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb40c0b4280 con 0x7fb40c1f2c20 2023-11-24T06:11:14.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.533+0000 7fb40affd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb3f8004d00 0x7fb40c0c6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:14.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.533+0000 7fb40affd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb3f8004d00 0x7fb40c0c6460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:32968/0 (socket says 172.21.15.92:32968) 2023-11-24T06:11:14.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.533+0000 7fb40affd640 1 -- 172.21.15.92:0/1821998257 learned_addr learned my addr 172.21.15.92:0/1821998257 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:11:14.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.534+0000 7fb40b7fe640 1 --2- 172.21.15.92:0/1821998257 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb40c09e420 0x7fb40c1f6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:14.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.534+0000 7fb40a7fc640 1 --2- 172.21.15.92:0/1821998257 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb40c1f2c20 0x7fb40c0b6650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:14.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.534+0000 7fb40affd640 1 -- 172.21.15.92:0/1821998257 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb40c1f2c20 msgr2=0x7fb40c0b6650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:14.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.534+0000 7fb40affd640 1 --2- 172.21.15.92:0/1821998257 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb40c1f2c20 0x7fb40c0b6650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:14.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.534+0000 7fb40affd640 1 -- 172.21.15.92:0/1821998257 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb40c09e420 msgr2=0x7fb40c1f6190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:14.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.534+0000 7fb40affd640 1 --2- 172.21.15.92:0/1821998257 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb40c09e420 0x7fb40c1f6190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:14.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.534+0000 7fb40affd640 1 -- 172.21.15.92:0/1821998257 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb40c0b4560 con 0x7fb3f8004d00 2023-11-24T06:11:14.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.534+0000 7fb40a7fc640 1 --2- 172.21.15.92:0/1821998257 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb40c1f2c20 0x7fb40c0b6650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:11:14.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.534+0000 7fb40b7fe640 1 --2- 172.21.15.92:0/1821998257 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb40c09e420 0x7fb40c1f6190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:11:14.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.534+0000 7fb40affd640 1 --2- 172.21.15.92:0/1821998257 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb3f8004d00 0x7fb40c0c6460 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fb3fc0012e0 tx=0x7fb3fc00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:14.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.534+0000 7fb4037fe640 1 -- 172.21.15.92:0/1821998257 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3fc004d60 con 0x7fb3f8004d00 2023-11-24T06:11:14.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.534+0000 7fb4037fe640 1 -- 172.21.15.92:0/1821998257 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3fc00ab40 con 0x7fb3f8004d00 2023-11-24T06:11:14.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.534+0000 7fb4037fe640 1 -- 172.21.15.92:0/1821998257 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3fc0070d0 con 0x7fb3f8004d00 2023-11-24T06:11:14.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.534+0000 7fb40bfff640 1 -- 172.21.15.92:0/1821998257 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb40c0b4840 con 0x7fb3f8004d00 2023-11-24T06:11:14.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.534+0000 7fb40bfff640 1 -- 172.21.15.92:0/1821998257 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb40c09ea90 con 0x7fb3f8004d00 2023-11-24T06:11:14.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.535+0000 7fb40bfff640 1 -- 172.21.15.92:0/1821998257 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb40c0b4a30 con 0x7fb3f8004d00 2023-11-24T06:11:14.542 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.536+0000 7fb4037fe640 1 -- 172.21.15.92:0/1821998257 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb3fc007230 con 0x7fb3f8004d00 2023-11-24T06:11:14.542 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.537+0000 7fb4037fe640 1 --2- 172.21.15.92:0/1821998257 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fb3ec06ae70 0x7fb3ec06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:14.542 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.537+0000 7fb4037fe640 1 -- 172.21.15.92:0/1821998257 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5883+0+0 (secure 0 0 0) 0x7fb3fc088f20 con 0x7fb3f8004d00 2023-11-24T06:11:14.542 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.537+0000 7fb40a7fc640 1 --2- 172.21.15.92:0/1821998257 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fb3ec06ae70 0x7fb3ec06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:14.543 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.538+0000 7fb40a7fc640 1 --2- 172.21.15.92:0/1821998257 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fb3ec06ae70 0x7fb3ec06d330 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fb40c1f3000 tx=0x7fb3f4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:14.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.542+0000 7fb4037fe640 1 -- 172.21.15.92:0/1821998257 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb3fc053e80 con 0x7fb3f8004d00 2023-11-24T06:11:14.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.735+0000 7fb40bfff640 1 -- 172.21.15.92:0/1821998257 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb40c067450 con 0x7fb3f8004d00 2023-11-24T06:11:14.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.736+0000 7fb4037fe640 1 -- 172.21.15.92:0/1821998257 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb3fc0180a0 con 0x7fb3f8004d00 2023-11-24T06:11:14.739 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 32 2023-11-24T06:11:14.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.738+0000 7fb40bfff640 1 -- 172.21.15.92:0/1821998257 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fb3ec06ae70 msgr2=0x7fb3ec06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:14.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.738+0000 7fb40bfff640 1 --2- 172.21.15.92:0/1821998257 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fb3ec06ae70 0x7fb3ec06d330 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fb40c1f3000 tx=0x7fb3f4008040 comp rx=0 tx=0).stop 2023-11-24T06:11:14.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.738+0000 7fb40bfff640 1 -- 172.21.15.92:0/1821998257 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb3f8004d00 msgr2=0x7fb40c0c6460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:14.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.738+0000 7fb40bfff640 1 --2- 172.21.15.92:0/1821998257 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb3f8004d00 0x7fb40c0c6460 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fb3fc0012e0 tx=0x7fb3fc00a430 comp rx=0 tx=0).stop 2023-11-24T06:11:14.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.739+0000 7fb40bfff640 1 -- 172.21.15.92:0/1821998257 shutdown_connections 2023-11-24T06:11:14.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.739+0000 7fb40bfff640 1 --2- 172.21.15.92:0/1821998257 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fb3ec06ae70 0x7fb3ec06d330 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:14.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.739+0000 7fb40bfff640 1 --2- 172.21.15.92:0/1821998257 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb40c09e420 0x7fb40c1f6190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:14.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.739+0000 7fb40bfff640 1 --2- 172.21.15.92:0/1821998257 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb40c1f2c20 0x7fb40c0b6650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:14.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.739+0000 7fb40bfff640 1 --2- 172.21.15.92:0/1821998257 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb3f8004d00 0x7fb40c0c6460 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:14.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.739+0000 7fb40bfff640 1 -- 172.21.15.92:0/1821998257 >> 172.21.15.92:0/1821998257 conn(0x7fb40c0b2a30 msgr2=0x7fb40c0afd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:14.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.739+0000 7fb40bfff640 1 -- 172.21.15.92:0/1821998257 shutdown_connections 2023-11-24T06:11:14.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.739+0000 7fb40bfff640 1 -- 172.21.15.92:0/1821998257 wait complete. 2023-11-24T06:11:14.760 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T06:11:14.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.959+0000 7fdba1ce7640 1 Processor -- start 2023-11-24T06:11:14.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.959+0000 7fdba1ce7640 1 -- start start 2023-11-24T06:11:14.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.961+0000 7fdba1ce7640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fdb9c0cf9a0 0x7fdb9c15dca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:14.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.961+0000 7fdba1ce7640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fdb9c15e1e0 0x7fdb9c0702c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:14.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.961+0000 7fdba1ce7640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb9c0b1ee0 con 0x7fdb9c143ae0 2023-11-24T06:11:14.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.961+0000 7fdba1ce7640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb9c0b2780 con 0x7fdb9c15e1e0 2023-11-24T06:11:14.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.961+0000 7fdba1ce7640 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb9c15e990 con 0x7fdb9c0cf9a0 2023-11-24T06:11:14.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.961+0000 7fdb9b7fe640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fdb9c143ae0 0x7fdb9c0cf290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:54282/0 (socket says 172.21.15.92:54282) 2023-11-24T06:11:14.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.961+0000 7fdb9b7fe640 1 -- 172.21.15.92:0/3413442566 learned_addr learned my addr 172.21.15.92:0/3413442566 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-24T06:11:14.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.962+0000 7fdb9bfff640 1 --2- 172.21.15.92:0/3413442566 >> v2:172.21.15.92:3300/0 conn(0x7fdb9c15e1e0 0x7fdb9c0702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:14.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.962+0000 7fdb92ffd640 1 --2- 172.21.15.92:0/3413442566 >> v2:172.21.15.118:3300/0 conn(0x7fdb9c0cf9a0 0x7fdb9c15dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:14.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.962+0000 7fdb9affd640 1 -- 172.21.15.92:0/3413442566 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2728204324 0 0) 0x7fdb9c0b1ee0 con 0x7fdb9c143ae0 2023-11-24T06:11:14.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.962+0000 7fdb9affd640 1 -- 172.21.15.92:0/3413442566 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdb88003520 con 0x7fdb9c143ae0 2023-11-24T06:11:14.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.962+0000 7fdb9bfff640 1 -- 172.21.15.92:0/3413442566 >> v2:172.21.15.118:3300/0 conn(0x7fdb9c0cf9a0 msgr2=0x7fdb9c15dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:14.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.962+0000 7fdb9bfff640 1 --2- 172.21.15.92:0/3413442566 >> v2:172.21.15.118:3300/0 conn(0x7fdb9c0cf9a0 0x7fdb9c15dca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:14.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.962+0000 7fdb9bfff640 1 -- 172.21.15.92:0/3413442566 >> v1:172.21.15.92:6789/0 conn(0x7fdb9c143ae0 legacy=0x7fdb9c0cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:14.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.962+0000 7fdb9bfff640 1 -- 172.21.15.92:0/3413442566 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb9c0b2130 con 0x7fdb9c15e1e0 2023-11-24T06:11:14.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.964+0000 7fdb9bfff640 1 --2- 172.21.15.92:0/3413442566 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdb9c15e1e0 0x7fdb9c0702c0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fdb9c0cb830 tx=0x7fdb9402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa6f6976c72edbc4 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:14.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.964+0000 7fdb9affd640 1 -- 172.21.15.92:0/3413442566 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb94031830 con 0x7fdb9c15e1e0 2023-11-24T06:11:14.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.964+0000 7fdb9affd640 1 -- 172.21.15.92:0/3413442566 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb94031990 con 0x7fdb9c15e1e0 2023-11-24T06:11:14.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.964+0000 7fdb9affd640 1 -- 172.21.15.92:0/3413442566 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb94031d20 con 0x7fdb9c15e1e0 2023-11-24T06:11:14.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.965+0000 7fdba1ce7640 1 -- 172.21.15.92:0/3413442566 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdb9c15e1e0 msgr2=0x7fdb9c0702c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:14.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.965+0000 7fdba1ce7640 1 --2- 172.21.15.92:0/3413442566 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdb9c15e1e0 0x7fdb9c0702c0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fdb9c0cb830 tx=0x7fdb9402b8d0 comp rx=0 tx=0).stop 2023-11-24T06:11:14.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.965+0000 7fdba1ce7640 1 -- 172.21.15.92:0/3413442566 shutdown_connections 2023-11-24T06:11:14.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.965+0000 7fdba1ce7640 1 --2- 172.21.15.92:0/3413442566 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdb9c15e1e0 0x7fdb9c0702c0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:14.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.965+0000 7fdba1ce7640 1 --2- 172.21.15.92:0/3413442566 >> v2:172.21.15.118:3300/0 conn(0x7fdb9c0cf9a0 0x7fdb9c15dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:14.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.965+0000 7fdba1ce7640 1 -- 172.21.15.92:0/3413442566 >> 172.21.15.92:0/3413442566 conn(0x7fdb9c06a800 msgr2=0x7fdb9c06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:14.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.965+0000 7fdba1ce7640 1 -- 172.21.15.92:0/3413442566 shutdown_connections 2023-11-24T06:11:14.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.965+0000 7fdba1ce7640 1 -- 172.21.15.92:0/3413442566 wait complete. 2023-11-24T06:11:14.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.966+0000 7fdba1ce7640 1 Processor -- start 2023-11-24T06:11:14.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.967+0000 7fdba1ce7640 1 -- start start 2023-11-24T06:11:14.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.967+0000 7fdba1ce7640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdb9c0cf9a0 0x7fdb9c140d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:14.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.967+0000 7fdba1ce7640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fdb9c143ae0 0x7fdb9c0702c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:14.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.968+0000 7fdba1ce7640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fdb9c15e1e0 0x7fdb9c070800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:14.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.968+0000 7fdba1ce7640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb9c1412b0 con 0x7fdb9c0cf9a0 2023-11-24T06:11:14.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.968+0000 7fdba1ce7640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb9c0b07f0 con 0x7fdb9c15e1e0 2023-11-24T06:11:14.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.968+0000 7fdba1ce7640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb9c0af750 con 0x7fdb9c143ae0 2023-11-24T06:11:14.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.968+0000 7fdb9b7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdb9c0cf9a0 0x7fdb9c140d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:14.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.968+0000 7fdb9b7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdb9c0cf9a0 0x7fdb9c140d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:32978/0 (socket says 172.21.15.92:32978) 2023-11-24T06:11:14.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.968+0000 7fdb9b7fe640 1 -- 172.21.15.92:0/2884154387 learned_addr learned my addr 172.21.15.92:0/2884154387 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:11:14.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.968+0000 7fdb9b7fe640 1 -- 172.21.15.92:0/2884154387 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fdb9c143ae0 msgr2=0x7fdb9c0702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:14.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.968+0000 7fdb9b7fe640 1 --2- 172.21.15.92:0/2884154387 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fdb9c143ae0 0x7fdb9c0702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:14.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.968+0000 7fdb9b7fe640 1 -- 172.21.15.92:0/2884154387 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fdb9c15e1e0 msgr2=0x7fdb9c070800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:14.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.968+0000 7fdb9bfff640 1 --2- 172.21.15.92:0/2884154387 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fdb9c15e1e0 0x7fdb9c070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:14.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.968+0000 7fdb9b7fe640 1 --2- 172.21.15.92:0/2884154387 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fdb9c15e1e0 0x7fdb9c070800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:14.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.968+0000 7fdb9b7fe640 1 -- 172.21.15.92:0/2884154387 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb94009400 con 0x7fdb9c0cf9a0 2023-11-24T06:11:14.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.969+0000 7fdb9b7fe640 1 --2- 172.21.15.92:0/2884154387 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdb9c0cf9a0 0x7fdb9c140d70 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fdb8c00cbc0 tx=0x7fdb8c00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:14.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.969+0000 7fdb98ff9640 1 -- 172.21.15.92:0/2884154387 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb8c00ad40 con 0x7fdb9c0cf9a0 2023-11-24T06:11:14.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.969+0000 7fdb98ff9640 1 -- 172.21.15.92:0/2884154387 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb8c012400 con 0x7fdb9c0cf9a0 2023-11-24T06:11:14.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.969+0000 7fdba1ce7640 1 -- 172.21.15.92:0/2884154387 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb9c0afa30 con 0x7fdb9c0cf9a0 2023-11-24T06:11:14.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.969+0000 7fdb98ff9640 1 -- 172.21.15.92:0/2884154387 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb8c00aea0 con 0x7fdb9c0cf9a0 2023-11-24T06:11:14.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.969+0000 7fdba1ce7640 1 -- 172.21.15.92:0/2884154387 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb9c0afff0 con 0x7fdb9c0cf9a0 2023-11-24T06:11:14.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.970+0000 7fdba1ce7640 1 -- 172.21.15.92:0/2884154387 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb9c0675d0 con 0x7fdb9c0cf9a0 2023-11-24T06:11:14.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.971+0000 7fdb98ff9640 1 -- 172.21.15.92:0/2884154387 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fdb8c007070 con 0x7fdb9c0cf9a0 2023-11-24T06:11:14.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.972+0000 7fdb98ff9640 1 --2- 172.21.15.92:0/2884154387 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fdb7c06ae40 0x7fdb7c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:14.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.972+0000 7fdb98ff9640 1 -- 172.21.15.92:0/2884154387 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5883+0+0 (secure 0 0 0) 0x7fdb8c089690 con 0x7fdb9c0cf9a0 2023-11-24T06:11:14.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.972+0000 7fdb92ffd640 1 --2- 172.21.15.92:0/2884154387 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fdb7c06ae40 0x7fdb7c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:14.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.973+0000 7fdb92ffd640 1 --2- 172.21.15.92:0/2884154387 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fdb7c06ae40 0x7fdb7c06d300 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fdb84006fd0 tx=0x7fdb84008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:14.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:14.977+0000 7fdb98ff9640 1 -- 172.21.15.92:0/2884154387 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fdb8c054730 con 0x7fdb9c0cf9a0 2023-11-24T06:11:15.155 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:15.151+0000 7fdba1ce7640 1 -- 172.21.15.92:0/2884154387 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fdb9c067db0 con 0x7fdb9c0cf9a0 2023-11-24T06:11:15.155 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:15.152+0000 7fdb98ff9640 1 -- 172.21.15.92:0/2884154387 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7fdb8c057d50 con 0x7fdb9c0cf9a0 2023-11-24T06:11:15.155 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 64 2023-11-24T06:11:15.158 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:15.154+0000 7fdba1ce7640 1 -- 172.21.15.92:0/2884154387 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fdb7c06ae40 msgr2=0x7fdb7c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:15.158 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:15.154+0000 7fdba1ce7640 1 --2- 172.21.15.92:0/2884154387 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fdb7c06ae40 0x7fdb7c06d300 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fdb84006fd0 tx=0x7fdb84008040 comp rx=0 tx=0).stop 2023-11-24T06:11:15.158 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:15.154+0000 7fdba1ce7640 1 -- 172.21.15.92:0/2884154387 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdb9c0cf9a0 msgr2=0x7fdb9c140d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:15.158 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:15.154+0000 7fdba1ce7640 1 --2- 172.21.15.92:0/2884154387 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdb9c0cf9a0 0x7fdb9c140d70 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fdb8c00cbc0 tx=0x7fdb8c00a430 comp rx=0 tx=0).stop 2023-11-24T06:11:15.158 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:15.154+0000 7fdba1ce7640 1 -- 172.21.15.92:0/2884154387 shutdown_connections 2023-11-24T06:11:15.158 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:15.154+0000 7fdba1ce7640 1 --2- 172.21.15.92:0/2884154387 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fdb7c06ae40 0x7fdb7c06d300 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:15.158 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:15.154+0000 7fdba1ce7640 1 --2- 172.21.15.92:0/2884154387 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fdb9c15e1e0 0x7fdb9c070800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:15.158 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:15.154+0000 7fdba1ce7640 1 --2- 172.21.15.92:0/2884154387 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fdb9c143ae0 0x7fdb9c0702c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:15.158 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:15.154+0000 7fdba1ce7640 1 --2- 172.21.15.92:0/2884154387 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdb9c0cf9a0 0x7fdb9c140d70 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:15.158 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:15.154+0000 7fdba1ce7640 1 -- 172.21.15.92:0/2884154387 >> 172.21.15.92:0/2884154387 conn(0x7fdb9c06a800 msgr2=0x7fdb9c141a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:15.159 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:15.155+0000 7fdba1ce7640 1 -- 172.21.15.92:0/2884154387 shutdown_connections 2023-11-24T06:11:15.159 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:15.155+0000 7fdba1ce7640 1 -- 172.21.15.92:0/2884154387 wait complete. 2023-11-24T06:11:15.176 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-24T06:11:15.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:15.382+0000 7fed1f568640 1 Processor -- start 2023-11-24T06:11:15.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:15.382+0000 7fed1f568640 1 -- start start 2023-11-24T06:11:15.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:15.383+0000 7fed1f568640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fed1813df80 0x7fed1815bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:15.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:15.383+0000 7fed1f568640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fed1813e360 0x7fed180cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:15.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:15.383+0000 7fed1f568640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed1806c470 con 0x7fed1815f9c0 2023-11-24T06:11:15.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:15.383+0000 7fed1f568640 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed1806c750 con 0x7fed1813e360 2023-11-24T06:11:15.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:15.383+0000 7fed1f568640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed1806ca30 con 0x7fed1813df80 2023-11-24T06:11:15.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:15.384+0000 7fed1db05640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fed1815f9c0 0x7fed181408d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:54286/0 (socket says 172.21.15.92:54286) 2023-11-24T06:11:15.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:15.384+0000 7fed1db05640 1 -- 172.21.15.92:0/1032734958 learned_addr learned my addr 172.21.15.92:0/1032734958 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-24T06:11:15.389 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:15.384+0000 7fed1d304640 1 --2- 172.21.15.92:0/1032734958 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fed1813df80 0x7fed1815bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:15.389 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:15.384+0000 7fed1cb03640 1 --2- 172.21.15.92:0/1032734958 >> v2:172.21.15.118:3300/0 conn(0x7fed1813e360 0x7fed180cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:15.389 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:15.384+0000 7fed0ffff640 1 -- 172.21.15.92:0/1032734958 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4233525736 0 0) 0x7fed1806c470 con 0x7fed1815f9c0 2023-11-24T06:11:15.389 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:15.384+0000 7fed0ffff640 1 -- 172.21.15.92:0/1032734958 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fed04003520 con 0x7fed1815f9c0 2023-11-24T06:11:15.389 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:15.385+0000 7fed0ffff640 1 -- 172.21.15.92:0/1032734958 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3949892667 0 0) 0x7fed04003520 con 0x7fed1815f9c0 2023-11-24T06:11:15.389 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:15.385+0000 7fed0ffff640 1 -- 172.21.15.92:0/1032734958 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fed1813df80 msgr2=0x7fed1815bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:15.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:15.385+0000 7fed0ffff640 1 --2- 172.21.15.92:0/1032734958 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fed1813df80 0x7fed1815bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:15.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:15.385+0000 7fed0ffff640 1 -- 172.21.15.92:0/1032734958 >> v2:172.21.15.118:3300/0 conn(0x7fed1813e360 msgr2=0x7fed180cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:15.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:15.385+0000 7fed0ffff640 1 --2- 172.21.15.92:0/1032734958 >> v2:172.21.15.118:3300/0 conn(0x7fed1813e360 0x7fed180cf290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:15.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:15.385+0000 7fed0ffff640 1 -- 172.21.15.92:0/1032734958 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed1806cd10 con 0x7fed1815f9c0 2023-11-24T06:11:15.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:15.385+0000 7fed0ffff640 1 -- 172.21.15.92:0/1032734958 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2607661350 0 0) 0x7fed100019f0 con 0x7fed1815f9c0 2023-11-24T06:11:15.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:15.385+0000 7fed0ffff640 1 -- 172.21.15.92:0/1032734958 >> v1:172.21.15.92:6789/0 conn(0x7fed1815f9c0 legacy=0x7fed181408d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:15.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:15.385+0000 7fed0ffff640 1 --2- 172.21.15.92:0/1032734958 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fed04003970 0x7fed04003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:15.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:15.385+0000 7fed0ffff640 1 --2- 172.21.15.92:0/1032734958 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fed040047c0 0x7fed04004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:15.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:15.385+0000 7fed0ffff640 1 --2- 172.21.15.92:0/1032734958 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fed1813df80 0x7fed04005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:15.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:15.385+0000 7fed0ffff640 1 -- 172.21.15.92:0/1032734958 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed1806c750 con 0x7fed040047c0 2023-11-24T06:11:15.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:15.385+0000 7fed0ffff640 1 -- 172.21.15.92:0/1032734958 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed1806ca30 con 0x7fed04003970 2023-11-24T06:11:15.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:15.385+0000 7fed0ffff640 1 -- 172.21.15.92:0/1032734958 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed0400b980 con 0x7fed1813df80 2023-11-24T06:11:15.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:15.385+0000 7fed1cb03640 1 --2- 172.21.15.92:0/1032734958 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fed040047c0 0x7fed04004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:15.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:15.385+0000 7fed1d304640 1 --2- 172.21.15.92:0/1032734958 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fed04003970 0x7fed04003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:15.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:15.385+0000 7fed1cb03640 1 -- 172.21.15.92:0/1032734958 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fed1813df80 msgr2=0x7fed04005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:15.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:15.385+0000 7fed1cb03640 1 --2- 172.21.15.92:0/1032734958 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fed1813df80 0x7fed04005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:15.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:15.385+0000 7fed1cb03640 1 -- 172.21.15.92:0/1032734958 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fed04003970 msgr2=0x7fed04003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:15.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:15.385+0000 7fed1cb03640 1 --2- 172.21.15.92:0/1032734958 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fed04003970 0x7fed04003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:15.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:15.385+0000 7fed1cb03640 1 -- 172.21.15.92:0/1032734958 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed04003520 con 0x7fed040047c0 2023-11-24T06:11:15.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:15.385+0000 7fed1d304640 1 --2- 172.21.15.92:0/1032734958 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fed04003970 0x7fed04003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:11:15.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:15.386+0000 7fed1cb03640 1 --2- 172.21.15.92:0/1032734958 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fed040047c0 0x7fed04004c40 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fed08009ac0 tx=0x7fed040057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cdb0b22ae04e2d7 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:15.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:15.387+0000 7fed0ffff640 1 -- 172.21.15.92:0/1032734958 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed0802c6d0 con 0x7fed040047c0 2023-11-24T06:11:15.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:15.387+0000 7fed0ffff640 1 -- 172.21.15.92:0/1032734958 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed0802c830 con 0x7fed040047c0 2023-11-24T06:11:15.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:15.387+0000 7fed0ffff640 1 -- 172.21.15.92:0/1032734958 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed0802cbc0 con 0x7fed040047c0 2023-11-24T06:11:15.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:15.387+0000 7fed1f568640 1 -- 172.21.15.92:0/1032734958 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fed040047c0 msgr2=0x7fed04004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:15.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:15.387+0000 7fed1f568640 1 --2- 172.21.15.92:0/1032734958 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fed040047c0 0x7fed04004c40 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fed08009ac0 tx=0x7fed040057f0 comp rx=0 tx=0).stop 2023-11-24T06:11:15.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:15.387+0000 7fed1db05640 1 -- 172.21.15.92:0/1032734958 reap_dead start 2023-11-24T06:11:15.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:15.387+0000 7fed1f568640 1 -- 172.21.15.92:0/1032734958 shutdown_connections 2023-11-24T06:11:15.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:15.387+0000 7fed1f568640 1 -- 172.21.15.92:0/1032734958 >> 172.21.15.92:0/1032734958 conn(0x7fed1806a800 msgr2=0x7fed1806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:15.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:15.387+0000 7fed1f568640 1 -- 172.21.15.92:0/1032734958 shutdown_connections 2023-11-24T06:11:15.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:15.387+0000 7fed1f568640 1 -- 172.21.15.92:0/1032734958 wait complete. 2023-11-24T06:11:15.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:15.389+0000 7fed1f568640 1 Processor -- start 2023-11-24T06:11:15.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:15.389+0000 7fed1f568640 1 -- start start 2023-11-24T06:11:15.395 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:15.390+0000 7fed1f568640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fed040047c0 0x7fed1814d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:15.395 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:15.390+0000 7fed1f568640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fed1815a540 0x7fed1814ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:15.395 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:15.390+0000 7fed1f568640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fed1814f340 0x7fed181529c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:15.395 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:15.390+0000 7fed1f568640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed18137130 con 0x7fed040047c0 2023-11-24T06:11:15.395 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:15.390+0000 7fed1f568640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed1814d340 con 0x7fed1814f340 2023-11-24T06:11:15.395 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:15.390+0000 7fed1f568640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed1814c730 con 0x7fed1815a540 2023-11-24T06:11:15.395 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:15.391+0000 7fed1d304640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fed040047c0 0x7fed1814d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:15.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:15.391+0000 7fed1d304640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fed040047c0 0x7fed1814d7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:33000/0 (socket says 172.21.15.92:33000) 2023-11-24T06:11:15.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:15.391+0000 7fed1d304640 1 -- 172.21.15.92:0/2465380324 learned_addr learned my addr 172.21.15.92:0/2465380324 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:11:15.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:15.391+0000 7fed1d304640 1 -- 172.21.15.92:0/2465380324 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fed1815a540 msgr2=0x7fed1814ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:15.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:15.391+0000 7fed1d304640 1 --2- 172.21.15.92:0/2465380324 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fed1815a540 0x7fed1814ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:15.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:15.391+0000 7fed1d304640 1 -- 172.21.15.92:0/2465380324 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fed1814f340 msgr2=0x7fed181529c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:15.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:15.391+0000 7fed1d304640 1 --2- 172.21.15.92:0/2465380324 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fed1814f340 0x7fed181529c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:15.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:15.391+0000 7fed1d304640 1 -- 172.21.15.92:0/2465380324 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed1814ca10 con 0x7fed040047c0 2023-11-24T06:11:15.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:15.391+0000 7fed1d304640 1 --2- 172.21.15.92:0/2465380324 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fed040047c0 0x7fed1814d7d0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fed140108b0 tx=0x7fed14010d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:15.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:15.391+0000 7fed0dffb640 1 -- 172.21.15.92:0/2465380324 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed1400eb60 con 0x7fed040047c0 2023-11-24T06:11:15.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:15.392+0000 7fed0dffb640 1 -- 172.21.15.92:0/2465380324 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed1400ecc0 con 0x7fed040047c0 2023-11-24T06:11:15.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:15.392+0000 7fed1f568640 1 -- 172.21.15.92:0/2465380324 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed1814ccf0 con 0x7fed040047c0 2023-11-24T06:11:15.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:15.392+0000 7fed0dffb640 1 -- 172.21.15.92:0/2465380324 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed1401c400 con 0x7fed040047c0 2023-11-24T06:11:15.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:15.392+0000 7fed1f568640 1 -- 172.21.15.92:0/2465380324 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed1814c0a0 con 0x7fed040047c0 2023-11-24T06:11:15.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:15.393+0000 7fed1f568640 1 -- 172.21.15.92:0/2465380324 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed180675d0 con 0x7fed040047c0 2023-11-24T06:11:15.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:15.394+0000 7fed0dffb640 1 -- 172.21.15.92:0/2465380324 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fed1401c560 con 0x7fed040047c0 2023-11-24T06:11:15.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:15.395+0000 7fed0dffb640 1 --2- 172.21.15.92:0/2465380324 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fecfc06ae40 0x7fecfc06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:15.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:15.395+0000 7fed0dffb640 1 -- 172.21.15.92:0/2465380324 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5883+0+0 (secure 0 0 0) 0x7fed14089ce0 con 0x7fed040047c0 2023-11-24T06:11:15.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:15.395+0000 7fed1cb03640 1 --2- 172.21.15.92:0/2465380324 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fecfc06ae40 0x7fecfc06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:15.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:15.396+0000 7fed1cb03640 1 --2- 172.21.15.92:0/2465380324 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fecfc06ae40 0x7fecfc06d300 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fed08036000 tx=0x7fed0802e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:15.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:15.399+0000 7fed0dffb640 1 -- 172.21.15.92:0/2465380324 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fed14054cf0 con 0x7fed040047c0 2023-11-24T06:11:15.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:15.600+0000 7fed1f568640 1 -- 172.21.15.92:0/2465380324 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fed1806df00 con 0x7fed040047c0 2023-11-24T06:11:15.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:15.600+0000 7fed0dffb640 1 -- 172.21.15.92:0/2465380324 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) v1 ==== 80+0+0 (secure 0 0 0) 0x7fed14058310 con 0x7fed040047c0 2023-11-24T06:11:15.604 INFO:teuthology.orchestra.run.smithi092.stderr:listed 0 entries 2023-11-24T06:11:15.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:15.602+0000 7fed1f568640 1 -- 172.21.15.92:0/2465380324 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fecfc06ae40 msgr2=0x7fecfc06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:15.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:15.602+0000 7fed1f568640 1 --2- 172.21.15.92:0/2465380324 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fecfc06ae40 0x7fecfc06d300 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fed08036000 tx=0x7fed0802e000 comp rx=0 tx=0).stop 2023-11-24T06:11:15.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:15.602+0000 7fed1f568640 1 -- 172.21.15.92:0/2465380324 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fed040047c0 msgr2=0x7fed1814d7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:15.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:15.602+0000 7fed1f568640 1 --2- 172.21.15.92:0/2465380324 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fed040047c0 0x7fed1814d7d0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fed140108b0 tx=0x7fed14010d70 comp rx=0 tx=0).stop 2023-11-24T06:11:15.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:15.603+0000 7fed1f568640 1 -- 172.21.15.92:0/2465380324 shutdown_connections 2023-11-24T06:11:15.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:15.603+0000 7fed1f568640 1 --2- 172.21.15.92:0/2465380324 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fecfc06ae40 0x7fecfc06d300 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:15.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:15.603+0000 7fed1f568640 1 --2- 172.21.15.92:0/2465380324 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fed1814f340 0x7fed181529c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:15.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:15.603+0000 7fed1f568640 1 --2- 172.21.15.92:0/2465380324 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fed1815a540 0x7fed1814ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:15.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:15.603+0000 7fed1f568640 1 --2- 172.21.15.92:0/2465380324 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fed040047c0 0x7fed1814d7d0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:15.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:15.603+0000 7fed1f568640 1 -- 172.21.15.92:0/2465380324 >> 172.21.15.92:0/2465380324 conn(0x7fed1806a800 msgr2=0x7fed180cf260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:15.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:15.604+0000 7fed1f568640 1 -- 172.21.15.92:0/2465380324 shutdown_connections 2023-11-24T06:11:15.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:15.604+0000 7fed1f568640 1 -- 172.21.15.92:0/2465380324 wait complete. 2023-11-24T06:11:15.624 DEBUG:teuthology.orchestra.run.smithi118:> set -ex 2023-11-24T06:11:15.624 DEBUG:teuthology.orchestra.run.smithi118:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:11:15.685 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:11:15.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:15.895+0000 7fe95bfff640 1 Processor -- start 2023-11-24T06:11:15.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:15.896+0000 7fe95bfff640 1 -- start start 2023-11-24T06:11:15.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:15.898+0000 7fe95bfff640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fe95c143c40 0x7fe95c071290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:15.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:15.898+0000 7fe95bfff640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe95c0717d0 0x7fe95c0cc500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:15.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:15.898+0000 7fe95bfff640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe95c070e50 con 0x7fe95c0cca40 2023-11-24T06:11:15.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:15.898+0000 7fe95bfff640 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe95c0723c0 con 0x7fe95c143c40 2023-11-24T06:11:15.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:15.898+0000 7fe95bfff640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe95c0cce70 con 0x7fe95c0717d0 2023-11-24T06:11:15.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:15.898+0000 7fe95b7fe640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fe95c0cca40 0x7fe95c15eab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:54292/0 (socket says 172.21.15.92:54292) 2023-11-24T06:11:15.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:15.898+0000 7fe95b7fe640 1 -- 172.21.15.92:0/3153208226 learned_addr learned my addr 172.21.15.92:0/3153208226 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-24T06:11:15.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:15.898+0000 7fe9527fc640 1 --2- 172.21.15.92:0/3153208226 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe95c0717d0 0x7fe95c0cc500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:15.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:15.898+0000 7fe95affd640 1 --2- 172.21.15.92:0/3153208226 >> v2:172.21.15.118:3300/0 conn(0x7fe95c143c40 0x7fe95c071290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:15.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:15.899+0000 7fe95a7fc640 1 -- 172.21.15.92:0/3153208226 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3194720908 0 0) 0x7fe95c070e50 con 0x7fe95c0cca40 2023-11-24T06:11:15.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:15.899+0000 7fe95a7fc640 1 -- 172.21.15.92:0/3153208226 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe948003520 con 0x7fe95c0cca40 2023-11-24T06:11:15.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:15.899+0000 7fe95a7fc640 1 -- 172.21.15.92:0/3153208226 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 483318292 0 0) 0x7fe948003520 con 0x7fe95c0cca40 2023-11-24T06:11:15.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:15.899+0000 7fe95a7fc640 1 -- 172.21.15.92:0/3153208226 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe95c0717d0 msgr2=0x7fe95c0cc500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:15.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:15.899+0000 7fe95a7fc640 1 --2- 172.21.15.92:0/3153208226 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe95c0717d0 0x7fe95c0cc500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:15.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:15.899+0000 7fe95a7fc640 1 -- 172.21.15.92:0/3153208226 >> v2:172.21.15.118:3300/0 conn(0x7fe95c143c40 msgr2=0x7fe95c071290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:15.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:15.899+0000 7fe95a7fc640 1 --2- 172.21.15.92:0/3153208226 >> v2:172.21.15.118:3300/0 conn(0x7fe95c143c40 0x7fe95c071290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:15.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:15.899+0000 7fe95a7fc640 1 -- 172.21.15.92:0/3153208226 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe95c06e170 con 0x7fe95c0cca40 2023-11-24T06:11:15.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:15.899+0000 7fe95a7fc640 1 -- 172.21.15.92:0/3153208226 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2607661350 0 0) 0x7fe9540019f0 con 0x7fe95c0cca40 2023-11-24T06:11:15.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:15.899+0000 7fe95a7fc640 1 -- 172.21.15.92:0/3153208226 >> v1:172.21.15.92:6789/0 conn(0x7fe95c0cca40 legacy=0x7fe95c15eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:15.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:15.899+0000 7fe95a7fc640 1 --2- 172.21.15.92:0/3153208226 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe948003970 0x7fe948003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:15.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:15.899+0000 7fe95a7fc640 1 --2- 172.21.15.92:0/3153208226 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe9480047c0 0x7fe948004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:15.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:15.899+0000 7fe95a7fc640 1 --2- 172.21.15.92:0/3153208226 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe95c0717d0 0x7fe948005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:15.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:15.899+0000 7fe95a7fc640 1 -- 172.21.15.92:0/3153208226 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe95c0723c0 con 0x7fe9480047c0 2023-11-24T06:11:15.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:15.899+0000 7fe95a7fc640 1 -- 172.21.15.92:0/3153208226 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe95c0cce70 con 0x7fe948003970 2023-11-24T06:11:15.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:15.899+0000 7fe95a7fc640 1 -- 172.21.15.92:0/3153208226 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe94800b9b0 con 0x7fe95c0717d0 2023-11-24T06:11:15.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:15.899+0000 7fe9527fc640 1 --2- 172.21.15.92:0/3153208226 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe9480047c0 0x7fe948004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:15.910 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:15.900+0000 7fe95b7fe640 1 --2- 172.21.15.92:0/3153208226 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe95c0717d0 0x7fe948005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:15.910 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:15.900+0000 7fe9527fc640 1 -- 172.21.15.92:0/3153208226 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe95c0717d0 msgr2=0x7fe948005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:15.910 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:15.900+0000 7fe9527fc640 1 --2- 172.21.15.92:0/3153208226 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe95c0717d0 0x7fe948005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:15.910 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:15.900+0000 7fe9527fc640 1 -- 172.21.15.92:0/3153208226 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe948003970 msgr2=0x7fe948003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:15.910 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:15.900+0000 7fe95affd640 1 --2- 172.21.15.92:0/3153208226 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe948003970 0x7fe948003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:15.910 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:15.900+0000 7fe9527fc640 1 --2- 172.21.15.92:0/3153208226 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe948003970 0x7fe948003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:15.911 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:15.900+0000 7fe9527fc640 1 -- 172.21.15.92:0/3153208226 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe948003520 con 0x7fe9480047c0 2023-11-24T06:11:15.911 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:15.900+0000 7fe95affd640 1 --2- 172.21.15.92:0/3153208226 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe948003970 0x7fe948003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:11:15.911 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:15.901+0000 7fe9527fc640 1 --2- 172.21.15.92:0/3153208226 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe9480047c0 0x7fe948004c40 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fe944009ac0 tx=0x7fe9480057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be70450b7fcba85b server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:15.911 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:15.901+0000 7fe95a7fc640 1 -- 172.21.15.92:0/3153208226 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe94402c6b0 con 0x7fe9480047c0 2023-11-24T06:11:15.911 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:15.901+0000 7fe95a7fc640 1 -- 172.21.15.92:0/3153208226 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe94402c810 con 0x7fe9480047c0 2023-11-24T06:11:15.911 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:15.901+0000 7fe95a7fc640 1 -- 172.21.15.92:0/3153208226 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe94402cba0 con 0x7fe9480047c0 2023-11-24T06:11:15.912 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:15.902+0000 7fe95bfff640 1 -- 172.21.15.92:0/3153208226 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe9480047c0 msgr2=0x7fe948004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:15.912 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:15.902+0000 7fe95bfff640 1 --2- 172.21.15.92:0/3153208226 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe9480047c0 0x7fe948004c40 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fe944009ac0 tx=0x7fe9480057f0 comp rx=0 tx=0).stop 2023-11-24T06:11:15.912 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:15.902+0000 7fe95b7fe640 1 -- 172.21.15.92:0/3153208226 reap_dead start 2023-11-24T06:11:15.912 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:15.902+0000 7fe95bfff640 1 -- 172.21.15.92:0/3153208226 shutdown_connections 2023-11-24T06:11:15.912 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:15.902+0000 7fe95bfff640 1 -- 172.21.15.92:0/3153208226 >> 172.21.15.92:0/3153208226 conn(0x7fe95c06a800 msgr2=0x7fe95c06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:15.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:15.902+0000 7fe95bfff640 1 -- 172.21.15.92:0/3153208226 shutdown_connections 2023-11-24T06:11:15.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:15.902+0000 7fe95bfff640 1 -- 172.21.15.92:0/3153208226 wait complete. 2023-11-24T06:11:15.917 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:15.904+0000 7fe95bfff640 1 Processor -- start 2023-11-24T06:11:15.917 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:15.904+0000 7fe95bfff640 1 -- start start 2023-11-24T06:11:15.918 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:15.905+0000 7fe95bfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe9480047c0 0x7fe95c0b27e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:15.918 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:15.905+0000 7fe95bfff640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe95c0a8ed0 0x7fe95c0a9350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:15.918 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:15.905+0000 7fe95bfff640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe95c15a4a0 0x7fe95c15a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:15.918 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:15.905+0000 7fe95bfff640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe95c0ab8b0 con 0x7fe9480047c0 2023-11-24T06:11:15.918 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:15.905+0000 7fe95bfff640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe95c0aba40 con 0x7fe95c0a8ed0 2023-11-24T06:11:15.918 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:15.905+0000 7fe95bfff640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe95c15ae60 con 0x7fe95c15a4a0 2023-11-24T06:11:15.918 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:15.905+0000 7fe95affd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe9480047c0 0x7fe95c0b27e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:15.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:15.905+0000 7fe95affd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe9480047c0 0x7fe95c0b27e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:33022/0 (socket says 172.21.15.92:33022) 2023-11-24T06:11:15.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:15.905+0000 7fe95affd640 1 -- 172.21.15.92:0/2669538229 learned_addr learned my addr 172.21.15.92:0/2669538229 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:11:15.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:15.905+0000 7fe9527fc640 1 --2- 172.21.15.92:0/2669538229 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe95c0a8ed0 0x7fe95c0a9350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:15.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:15.905+0000 7fe95b7fe640 1 --2- 172.21.15.92:0/2669538229 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe95c15a4a0 0x7fe95c15a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:15.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:15.905+0000 7fe95affd640 1 -- 172.21.15.92:0/2669538229 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe95c15a4a0 msgr2=0x7fe95c15a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:15.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:15.905+0000 7fe95affd640 1 --2- 172.21.15.92:0/2669538229 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe95c15a4a0 0x7fe95c15a920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:15.920 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:15.905+0000 7fe95affd640 1 -- 172.21.15.92:0/2669538229 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe95c0a8ed0 msgr2=0x7fe95c0a9350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:15.920 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:15.905+0000 7fe95affd640 1 --2- 172.21.15.92:0/2669538229 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe95c0a8ed0 0x7fe95c0a9350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:15.920 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:15.905+0000 7fe95affd640 1 -- 172.21.15.92:0/2669538229 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe95c0a99d0 con 0x7fe9480047c0 2023-11-24T06:11:15.920 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:15.905+0000 7fe9527fc640 1 --2- 172.21.15.92:0/2669538229 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe95c0a8ed0 0x7fe95c0a9350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:11:15.920 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:15.906+0000 7fe95affd640 1 --2- 172.21.15.92:0/2669538229 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe9480047c0 0x7fe95c0b27e0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fe94c0109d0 tx=0x7fe94c010e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:15.920 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:15.906+0000 7fe958ff9640 1 -- 172.21.15.92:0/2669538229 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe94c00ec20 con 0x7fe9480047c0 2023-11-24T06:11:15.921 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:15.906+0000 7fe958ff9640 1 -- 172.21.15.92:0/2669538229 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe94c00ed80 con 0x7fe9480047c0 2023-11-24T06:11:15.921 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:15.906+0000 7fe958ff9640 1 -- 172.21.15.92:0/2669538229 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe94c01c620 con 0x7fe9480047c0 2023-11-24T06:11:15.921 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:15.906+0000 7fe95bfff640 1 -- 172.21.15.92:0/2669538229 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe95c0a9cb0 con 0x7fe9480047c0 2023-11-24T06:11:15.921 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:15.906+0000 7fe95bfff640 1 -- 172.21.15.92:0/2669538229 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe95c0aa270 con 0x7fe9480047c0 2023-11-24T06:11:15.921 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:15.907+0000 7fe95bfff640 1 -- 172.21.15.92:0/2669538229 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe95c144b10 con 0x7fe9480047c0 2023-11-24T06:11:15.921 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:15.908+0000 7fe958ff9640 1 -- 172.21.15.92:0/2669538229 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe94c01c780 con 0x7fe9480047c0 2023-11-24T06:11:15.922 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:15.910+0000 7fe958ff9640 1 --2- 172.21.15.92:0/2669538229 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fe93c06ae00 0x7fe93c06d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:15.922 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:15.910+0000 7fe958ff9640 1 -- 172.21.15.92:0/2669538229 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5883+0+0 (secure 0 0 0) 0x7fe94c089e10 con 0x7fe9480047c0 2023-11-24T06:11:15.922 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:15.910+0000 7fe9527fc640 1 --2- 172.21.15.92:0/2669538229 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fe93c06ae00 0x7fe93c06d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:15.922 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:15.910+0000 7fe9527fc640 1 --2- 172.21.15.92:0/2669538229 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fe93c06ae00 0x7fe93c06d2c0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fe944036000 tx=0x7fe944030000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:15.922 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:15.914+0000 7fe958ff9640 1 -- 172.21.15.92:0/2669538229 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe94c08d030 con 0x7fe9480047c0 2023-11-24T06:11:16.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:16.105+0000 7fe95bfff640 1 -- 172.21.15.92:0/2669538229 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe95c0aa460 con 0x7fe9480047c0 2023-11-24T06:11:16.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:16.108+0000 7fe958ff9640 1 -- 172.21.15.92:0/2669538229 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16800 (secure 0 0 0) 0x7fe94c054e20 con 0x7fe9480047c0 2023-11-24T06:11:16.112 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-24T06:11:16.112 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":22,"fsid":"2d6dee5f-8aa3-470f-931a-50ade963b1cb","created":"2023-11-24T06:10:04.707204+0000","modified":"2023-11-24T06:11:11.472401+0000","last_up_change":"2023-11-24T06:10:17.598352+0000","last_in_change":"2023-11-24T06:10:09.040793+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:10:17.733098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:10:22.926445+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T06:10:24.151288+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ff37a9fb-135f-471f-b4b0-f7b5772006c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6803","nonce":20560714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6805","nonce":20560714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6809","nonce":20560714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6807","nonce":20560714}]},"public_addr":"172.21.15.92:6803/20560714","cluster_addr":"172.21.15.92:6805/20560714","heartbeat_back_addr":"172.21.15.92:6809/20560714","heartbeat_front_addr":"172.21.15.92:6807/20560714","state":["exists","up"]},{"osd":1,"uuid":"03cdd353-dcf1-4f38-94fe-a795137c06b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6819","nonce":2451239906}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6821","nonce":2451239906}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6825","nonce":2451239906}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6823","nonce":2451239906}]},"public_addr":"172.21.15.92:6819/2451239906","cluster_addr":"172.21.15.92:6821/2451239906","heartbeat_back_addr":"172.21.15.92:6825/2451239906","heartbeat_front_addr":"172.21.15.92:6823/2451239906","state":["exists","up"]},{"osd":2,"uuid":"a4a20d80-5d57-484b-becd-9aa8070dd3aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6811","nonce":1124979277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6813","nonce":1124979277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6817","nonce":1124979277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6815","nonce":1124979277}]},"public_addr":"172.21.15.92:6811/1124979277","cluster_addr":"172.21.15.92:6813/1124979277","heartbeat_back_addr":"172.21.15.92:6817/1124979277","heartbeat_front_addr":"172.21.15.92:6815/1124979277","state":["exists","up"]},{"osd":3,"uuid":"4648d7ce-9b29-4583-9425-4ba8fd2b1903","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6827","nonce":1734115782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6829","nonce":1734115782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6833","nonce":1734115782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6831","nonce":1734115782}]},"public_addr":"172.21.15.92:6827/1734115782","cluster_addr":"172.21.15.92:6829/1734115782","heartbeat_back_addr":"172.21.15.92:6833/1734115782","heartbeat_front_addr":"172.21.15.92:6831/1734115782","state":["exists","up"]},{"osd":4,"uuid":"3bc9d7eb-c6fa-4fcf-a3ec-ade604e05aac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6819","nonce":3425937229}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6822","nonce":3425937229}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6830","nonce":3425937229}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6826","nonce":3425937229}]},"public_addr":"172.21.15.118:6819/3425937229","cluster_addr":"172.21.15.118:6822/3425937229","heartbeat_back_addr":"172.21.15.118:6830/3425937229","heartbeat_front_addr":"172.21.15.118:6826/3425937229","state":["exists","up"]},{"osd":5,"uuid":"dce9ba6d-59ed-46d1-84fc-a84b13de10c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6801","nonce":827455984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6803","nonce":827455984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6807","nonce":827455984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6805","nonce":827455984}]},"public_addr":"172.21.15.118:6801/827455984","cluster_addr":"172.21.15.118:6803/827455984","heartbeat_back_addr":"172.21.15.118:6807/827455984","heartbeat_front_addr":"172.21.15.118:6805/827455984","state":["exists","up"]},{"osd":6,"uuid":"82aec504-ae58-4709-8d30-46f2b12081b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6809","nonce":2946986141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6811","nonce":2946986141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6815","nonce":2946986141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6813","nonce":2946986141}]},"public_addr":"172.21.15.118:6809/2946986141","cluster_addr":"172.21.15.118:6811/2946986141","heartbeat_back_addr":"172.21.15.118:6815/2946986141","heartbeat_front_addr":"172.21.15.118:6813/2946986141","state":["exists","up"]},{"osd":7,"uuid":"4ed4c624-c88f-4d35-bdb1-bf209b5dffb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6817","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6818","nonce":48122059}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6821","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6823","nonce":48122059}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6829","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6831","nonce":48122059}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6825","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6827","nonce":48122059}]},"public_addr":"172.21.15.118:6818/48122059","cluster_addr":"172.21.15.118:6823/48122059","heartbeat_back_addr":"172.21.15.118:6831/48122059","heartbeat_front_addr":"172.21.15.118:6827/48122059","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.536664+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.368324+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.271201+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.342971+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.621832+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.387110+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.632946+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.416032+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.d","mappings":[{"from":5,"to":1}]},{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"3.29","mappings":[{"from":3,"to":2}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.34","mappings":[{"from":4,"to":2}]},{"pgid":"3.3b","mappings":[{"from":7,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:11:16.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:16.110+0000 7fe95bfff640 1 -- 172.21.15.92:0/2669538229 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fe93c06ae00 msgr2=0x7fe93c06d2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:16.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:16.110+0000 7fe95bfff640 1 --2- 172.21.15.92:0/2669538229 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fe93c06ae00 0x7fe93c06d2c0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fe944036000 tx=0x7fe944030000 comp rx=0 tx=0).stop 2023-11-24T06:11:16.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:16.110+0000 7fe95bfff640 1 -- 172.21.15.92:0/2669538229 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe9480047c0 msgr2=0x7fe95c0b27e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:16.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:16.110+0000 7fe95bfff640 1 --2- 172.21.15.92:0/2669538229 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe9480047c0 0x7fe95c0b27e0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fe94c0109d0 tx=0x7fe94c010e90 comp rx=0 tx=0).stop 2023-11-24T06:11:16.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:16.111+0000 7fe95bfff640 1 -- 172.21.15.92:0/2669538229 shutdown_connections 2023-11-24T06:11:16.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:16.111+0000 7fe95bfff640 1 --2- 172.21.15.92:0/2669538229 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fe93c06ae00 0x7fe93c06d2c0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:16.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:16.111+0000 7fe95bfff640 1 --2- 172.21.15.92:0/2669538229 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe95c15a4a0 0x7fe95c15a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:16.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:16.111+0000 7fe95bfff640 1 --2- 172.21.15.92:0/2669538229 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe95c0a8ed0 0x7fe95c0a9350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:16.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:16.111+0000 7fe95bfff640 1 --2- 172.21.15.92:0/2669538229 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe9480047c0 0x7fe95c0b27e0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:16.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:16.111+0000 7fe95bfff640 1 -- 172.21.15.92:0/2669538229 >> 172.21.15.92:0/2669538229 conn(0x7fe95c06a800 msgr2=0x7fe95c1420a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:16.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:16.111+0000 7fe95bfff640 1 -- 172.21.15.92:0/2669538229 shutdown_connections 2023-11-24T06:11:16.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:16.111+0000 7fe95bfff640 1 -- 172.21.15.92:0/2669538229 wait complete. 2023-11-24T06:11:16.133 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T06:10:17.733098+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T06:10:22.926445+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T06:10:24.151288+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T06:11:16.134 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T06:11:16.331 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:16.327+0000 7f0b245ce640 1 Processor -- start 2023-11-24T06:11:16.331 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:16.328+0000 7f0b245ce640 1 -- start start 2023-11-24T06:11:16.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:16.329+0000 7f0b245ce640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f0b1c0b3ae0 0x7f0b1c15e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:16.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:16.329+0000 7f0b245ce640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0b1c0cf290 0x7f0b1c0cf720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:16.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:16.329+0000 7f0b245ce640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b1c070c40 con 0x7f0b1c0ae5a0 2023-11-24T06:11:16.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:16.329+0000 7f0b245ce640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b1c071450 con 0x7f0b1c0b3ae0 2023-11-24T06:11:16.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:16.329+0000 7f0b245ce640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b1c0cfc60 con 0x7f0b1c0cf290 2023-11-24T06:11:16.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:16.329+0000 7f0b2236a640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f0b1c0ae5a0 0x7f0b1c06cbe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:54308/0 (socket says 172.21.15.92:54308) 2023-11-24T06:11:16.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:16.329+0000 7f0b2236a640 1 -- 172.21.15.92:0/3079951350 learned_addr learned my addr 172.21.15.92:0/3079951350 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-24T06:11:16.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:16.330+0000 7f0b21b69640 1 --2- 172.21.15.92:0/3079951350 >> v2:172.21.15.92:3300/0 conn(0x7f0b1c0b3ae0 0x7f0b1c15e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:16.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:16.330+0000 7f0b21368640 1 -- 172.21.15.92:0/3079951350 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4136122573 0 0) 0x7f0b1c070c40 con 0x7f0b1c0ae5a0 2023-11-24T06:11:16.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:16.330+0000 7f0b21368640 1 -- 172.21.15.92:0/3079951350 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0b08003520 con 0x7f0b1c0ae5a0 2023-11-24T06:11:16.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:16.330+0000 7f0b22b6b640 1 --2- 172.21.15.92:0/3079951350 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0b1c0cf290 0x7f0b1c0cf720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:16.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:16.330+0000 7f0b21b69640 1 -- 172.21.15.92:0/3079951350 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0b1c0cf290 msgr2=0x7f0b1c0cf720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:16.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:16.330+0000 7f0b21b69640 1 --2- 172.21.15.92:0/3079951350 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0b1c0cf290 0x7f0b1c0cf720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:16.335 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:16.330+0000 7f0b21b69640 1 -- 172.21.15.92:0/3079951350 >> v1:172.21.15.92:6789/0 conn(0x7f0b1c0ae5a0 legacy=0x7f0b1c06cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:16.335 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:16.330+0000 7f0b21b69640 1 -- 172.21.15.92:0/3079951350 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b1c06c470 con 0x7f0b1c0b3ae0 2023-11-24T06:11:16.335 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:16.331+0000 7f0b21b69640 1 --2- 172.21.15.92:0/3079951350 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0b1c0b3ae0 0x7f0b1c15e430 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f0b100097a0 tx=0x7f0b1002b940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=542635716065e439 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:16.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:16.331+0000 7f0b21368640 1 -- 172.21.15.92:0/3079951350 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b10030800 con 0x7f0b1c0b3ae0 2023-11-24T06:11:16.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:16.332+0000 7f0b21368640 1 -- 172.21.15.92:0/3079951350 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b10030960 con 0x7f0b1c0b3ae0 2023-11-24T06:11:16.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:16.332+0000 7f0b21368640 1 -- 172.21.15.92:0/3079951350 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b10030cf0 con 0x7f0b1c0b3ae0 2023-11-24T06:11:16.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:16.332+0000 7f0b245ce640 1 -- 172.21.15.92:0/3079951350 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0b1c0b3ae0 msgr2=0x7f0b1c15e430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:16.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:16.332+0000 7f0b245ce640 1 --2- 172.21.15.92:0/3079951350 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0b1c0b3ae0 0x7f0b1c15e430 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f0b100097a0 tx=0x7f0b1002b940 comp rx=0 tx=0).stop 2023-11-24T06:11:16.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:16.332+0000 7f0b245ce640 1 -- 172.21.15.92:0/3079951350 shutdown_connections 2023-11-24T06:11:16.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:16.332+0000 7f0b245ce640 1 --2- 172.21.15.92:0/3079951350 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0b1c0cf290 0x7f0b1c0cf720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:16.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:16.332+0000 7f0b245ce640 1 --2- 172.21.15.92:0/3079951350 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0b1c0b3ae0 0x7f0b1c15e430 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:16.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:16.332+0000 7f0b245ce640 1 -- 172.21.15.92:0/3079951350 >> 172.21.15.92:0/3079951350 conn(0x7f0b1c06a800 msgr2=0x7f0b1c06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:16.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:16.332+0000 7f0b245ce640 1 -- 172.21.15.92:0/3079951350 shutdown_connections 2023-11-24T06:11:16.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:16.332+0000 7f0b245ce640 1 -- 172.21.15.92:0/3079951350 wait complete. 2023-11-24T06:11:16.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:16.334+0000 7f0b245ce640 1 Processor -- start 2023-11-24T06:11:16.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:16.334+0000 7f0b245ce640 1 -- start start 2023-11-24T06:11:16.338 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:16.335+0000 7f0b245ce640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0b1c0ae5a0 0x7f0b1c1f5c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:16.338 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:16.335+0000 7f0b245ce640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0b1c0b3ae0 0x7f0b1c15f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:16.338 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:16.335+0000 7f0b245ce640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0b1c0cf290 0x7f0b1c151530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:16.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:16.335+0000 7f0b245ce640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b1c158510 con 0x7f0b1c0cf290 2023-11-24T06:11:16.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:16.335+0000 7f0b245ce640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b1c157640 con 0x7f0b1c0ae5a0 2023-11-24T06:11:16.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:16.335+0000 7f0b245ce640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b1c1f6160 con 0x7f0b1c0b3ae0 2023-11-24T06:11:16.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:16.335+0000 7f0b22b6b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0b1c0cf290 0x7f0b1c151530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:16.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:16.335+0000 7f0b22b6b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0b1c0cf290 0x7f0b1c151530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:33032/0 (socket says 172.21.15.92:33032) 2023-11-24T06:11:16.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:16.335+0000 7f0b22b6b640 1 -- 172.21.15.92:0/1921915382 learned_addr learned my addr 172.21.15.92:0/1921915382 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:11:16.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:16.336+0000 7f0b2236a640 1 --2- 172.21.15.92:0/1921915382 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0b1c0ae5a0 0x7f0b1c1f5c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:16.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:16.336+0000 7f0b22b6b640 1 -- 172.21.15.92:0/1921915382 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0b1c0b3ae0 msgr2=0x7f0b1c15f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:16.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:16.336+0000 7f0b21b69640 1 --2- 172.21.15.92:0/1921915382 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0b1c0b3ae0 0x7f0b1c15f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:16.341 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:16.336+0000 7f0b22b6b640 1 --2- 172.21.15.92:0/1921915382 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0b1c0b3ae0 0x7f0b1c15f620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:16.341 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:16.336+0000 7f0b22b6b640 1 -- 172.21.15.92:0/1921915382 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0b1c0ae5a0 msgr2=0x7f0b1c1f5c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:16.341 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:16.336+0000 7f0b22b6b640 1 --2- 172.21.15.92:0/1921915382 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0b1c0ae5a0 0x7f0b1c1f5c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:16.341 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:16.336+0000 7f0b22b6b640 1 -- 172.21.15.92:0/1921915382 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b10009400 con 0x7f0b1c0cf290 2023-11-24T06:11:16.341 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:16.336+0000 7f0b2236a640 1 --2- 172.21.15.92:0/1921915382 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0b1c0ae5a0 0x7f0b1c1f5c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:11:16.341 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:16.336+0000 7f0b22b6b640 1 --2- 172.21.15.92:0/1921915382 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0b1c0cf290 0x7f0b1c151530 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f0b14007e90 tx=0x7f0b1400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:16.341 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:16.336+0000 7f0b0effd640 1 -- 172.21.15.92:0/1921915382 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b14014420 con 0x7f0b1c0cf290 2023-11-24T06:11:16.341 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:16.336+0000 7f0b0effd640 1 -- 172.21.15.92:0/1921915382 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b14014580 con 0x7f0b1c0cf290 2023-11-24T06:11:16.341 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:16.336+0000 7f0b0effd640 1 -- 172.21.15.92:0/1921915382 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b14014930 con 0x7f0b1c0cf290 2023-11-24T06:11:16.341 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:16.336+0000 7f0b245ce640 1 -- 172.21.15.92:0/1921915382 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b1c157890 con 0x7f0b1c0cf290 2023-11-24T06:11:16.341 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:16.336+0000 7f0b245ce640 1 -- 172.21.15.92:0/1921915382 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b1c157e50 con 0x7f0b1c0cf290 2023-11-24T06:11:16.342 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:16.338+0000 7f0b245ce640 1 -- 172.21.15.92:0/1921915382 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b1c158650 con 0x7f0b1c0cf290 2023-11-24T06:11:16.342 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:16.338+0000 7f0b0effd640 1 -- 172.21.15.92:0/1921915382 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f0b14014a90 con 0x7f0b1c0cf290 2023-11-24T06:11:16.343 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:16.340+0000 7f0b0effd640 1 --2- 172.21.15.92:0/1921915382 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f0b0006ae40 0x7f0b0006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:16.343 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:16.340+0000 7f0b0effd640 1 -- 172.21.15.92:0/1921915382 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5883+0+0 (secure 0 0 0) 0x7f0b14089fe0 con 0x7f0b1c0cf290 2023-11-24T06:11:16.343 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:16.340+0000 7f0b2236a640 1 --2- 172.21.15.92:0/1921915382 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f0b0006ae40 0x7f0b0006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:16.343 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:16.341+0000 7f0b2236a640 1 --2- 172.21.15.92:0/1921915382 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f0b0006ae40 0x7f0b0006d300 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f0b18007fd0 tx=0x7f0b18009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:16.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:16.344+0000 7f0b0effd640 1 -- 172.21.15.92:0/1921915382 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0b1c158650 con 0x7f0b1c0cf290 2023-11-24T06:11:16.452 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T06:11:16.462 DEBUG:teuthology.orchestra.run.smithi118:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T06:11:16.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:16.522+0000 7f0b245ce640 1 -- 172.21.15.92:0/1921915382 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0b1c158650 con 0x7f0b1c0cf290 2023-11-24T06:11:16.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:16.523+0000 7f0b0effd640 1 -- 172.21.15.92:0/1921915382 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0b1c158650 con 0x7f0b1c0cf290 2023-11-24T06:11:16.527 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 1 2023-11-24T06:11:16.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:16.525+0000 7f0b245ce640 1 -- 172.21.15.92:0/1921915382 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f0b0006ae40 msgr2=0x7f0b0006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:16.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:16.525+0000 7f0b245ce640 1 --2- 172.21.15.92:0/1921915382 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f0b0006ae40 0x7f0b0006d300 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f0b18007fd0 tx=0x7f0b18009040 comp rx=0 tx=0).stop 2023-11-24T06:11:16.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:16.525+0000 7f0b245ce640 1 -- 172.21.15.92:0/1921915382 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0b1c0cf290 msgr2=0x7f0b1c151530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:16.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:16.525+0000 7f0b245ce640 1 --2- 172.21.15.92:0/1921915382 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0b1c0cf290 0x7f0b1c151530 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f0b14007e90 tx=0x7f0b1400e4b0 comp rx=0 tx=0).stop 2023-11-24T06:11:16.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:16.525+0000 7f0b245ce640 1 -- 172.21.15.92:0/1921915382 shutdown_connections 2023-11-24T06:11:16.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:16.525+0000 7f0b245ce640 1 --2- 172.21.15.92:0/1921915382 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f0b0006ae40 0x7f0b0006d300 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:16.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:16.525+0000 7f0b245ce640 1 --2- 172.21.15.92:0/1921915382 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0b1c0cf290 0x7f0b1c151530 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:16.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:16.525+0000 7f0b245ce640 1 --2- 172.21.15.92:0/1921915382 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0b1c0b3ae0 0x7f0b1c15f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:16.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:16.525+0000 7f0b245ce640 1 --2- 172.21.15.92:0/1921915382 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0b1c0ae5a0 0x7f0b1c1f5c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:16.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:16.525+0000 7f0b245ce640 1 -- 172.21.15.92:0/1921915382 >> 172.21.15.92:0/1921915382 conn(0x7f0b1c06a800 msgr2=0x7f0b1c0b2b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:16.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:16.525+0000 7f0b245ce640 1 -- 172.21.15.92:0/1921915382 shutdown_connections 2023-11-24T06:11:16.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:16.525+0000 7f0b245ce640 1 -- 172.21.15.92:0/1921915382 wait complete. 2023-11-24T06:11:16.547 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T06:11:16.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:16.748+0000 7fcd6d9c5640 1 Processor -- start 2023-11-24T06:11:16.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:16.748+0000 7fcd6d9c5640 1 -- start start 2023-11-24T06:11:16.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:16.750+0000 7fcd6d9c5640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fcd6813df80 0x7fcd6815bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:16.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:16.750+0000 7fcd6d9c5640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fcd6813e360 0x7fcd680cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:16.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:16.750+0000 7fcd6d9c5640 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd6806c470 con 0x7fcd6815f9c0 2023-11-24T06:11:16.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:16.750+0000 7fcd6d9c5640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd6806c750 con 0x7fcd6813e360 2023-11-24T06:11:16.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:16.750+0000 7fcd6d9c5640 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd6806ca30 con 0x7fcd6813df80 2023-11-24T06:11:16.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:16.750+0000 7fcd667fc640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fcd6813e360 0x7fcd680cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:16.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:16.750+0000 7fcd667fc640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fcd6813e360 0x7fcd680cf290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:33040/0 (socket says 172.21.15.92:33040) 2023-11-24T06:11:16.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:16.750+0000 7fcd667fc640 1 -- 172.21.15.92:0/3887434749 learned_addr learned my addr 172.21.15.92:0/3887434749 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:11:16.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:16.750+0000 7fcd667fc640 1 -- 172.21.15.92:0/3887434749 >> v2:172.21.15.118:3300/0 conn(0x7fcd6813df80 msgr2=0x7fcd6815bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:16.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:16.750+0000 7fcd667fc640 1 --2- 172.21.15.92:0/3887434749 >> v2:172.21.15.118:3300/0 conn(0x7fcd6813df80 0x7fcd6815bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:16.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:16.751+0000 7fcd667fc640 1 -- 172.21.15.92:0/3887434749 >> v1:172.21.15.118:6789/0 conn(0x7fcd6815f9c0 legacy=0x7fcd681408d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:16.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:16.751+0000 7fcd667fc640 1 -- 172.21.15.92:0/3887434749 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd6806cd10 con 0x7fcd6813e360 2023-11-24T06:11:16.756 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:16.752+0000 7fcd667fc640 1 --2- 172.21.15.92:0/3887434749 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcd6813e360 0x7fcd680cf290 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fcd68072770 tx=0x7fcd5c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75043d8efebbb89c server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:16.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:16.752+0000 7fcd65ffb640 1 -- 172.21.15.92:0/3887434749 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd5c031870 con 0x7fcd6813e360 2023-11-24T06:11:16.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:16.752+0000 7fcd65ffb640 1 -- 172.21.15.92:0/3887434749 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd5c0319d0 con 0x7fcd6813e360 2023-11-24T06:11:16.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:16.752+0000 7fcd65ffb640 1 -- 172.21.15.92:0/3887434749 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd5c031d60 con 0x7fcd6813e360 2023-11-24T06:11:16.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:16.753+0000 7fcd6d9c5640 1 -- 172.21.15.92:0/3887434749 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcd6813e360 msgr2=0x7fcd680cf290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:16.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:16.753+0000 7fcd6d9c5640 1 --2- 172.21.15.92:0/3887434749 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcd6813e360 0x7fcd680cf290 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fcd68072770 tx=0x7fcd5c02b910 comp rx=0 tx=0).stop 2023-11-24T06:11:16.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:16.753+0000 7fcd6d9c5640 1 -- 172.21.15.92:0/3887434749 shutdown_connections 2023-11-24T06:11:16.758 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:16.753+0000 7fcd6d9c5640 1 --2- 172.21.15.92:0/3887434749 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcd6813e360 0x7fcd680cf290 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:16.758 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:16.753+0000 7fcd6d9c5640 1 --2- 172.21.15.92:0/3887434749 >> v2:172.21.15.118:3300/0 conn(0x7fcd6813df80 0x7fcd6815bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:16.758 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:16.753+0000 7fcd6d9c5640 1 -- 172.21.15.92:0/3887434749 >> 172.21.15.92:0/3887434749 conn(0x7fcd6806a800 msgr2=0x7fcd6806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:16.758 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:16.753+0000 7fcd6d9c5640 1 -- 172.21.15.92:0/3887434749 shutdown_connections 2023-11-24T06:11:16.758 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:16.753+0000 7fcd6d9c5640 1 -- 172.21.15.92:0/3887434749 wait complete. 2023-11-24T06:11:16.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:16.754+0000 7fcd6d9c5640 1 Processor -- start 2023-11-24T06:11:16.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:16.755+0000 7fcd6d9c5640 1 -- start start 2023-11-24T06:11:16.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:16.755+0000 7fcd6d9c5640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcd6813df80 0x7fcd681e4b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:16.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:16.755+0000 7fcd6d9c5640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fcd6813e360 0x7fcd68139900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:16.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:16.755+0000 7fcd6d9c5640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fcd6815f9c0 0x7fcd6815d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:16.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:16.755+0000 7fcd6d9c5640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd6813b4c0 con 0x7fcd6813df80 2023-11-24T06:11:16.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:16.755+0000 7fcd6d9c5640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd6813b7a0 con 0x7fcd6813e360 2023-11-24T06:11:16.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:16.755+0000 7fcd6d9c5640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd6813ba80 con 0x7fcd6815f9c0 2023-11-24T06:11:16.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:16.756+0000 7fcd66ffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcd6813df80 0x7fcd681e4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:16.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:16.756+0000 7fcd66ffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcd6813df80 0x7fcd681e4b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:33052/0 (socket says 172.21.15.92:33052) 2023-11-24T06:11:16.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:16.756+0000 7fcd66ffd640 1 -- 172.21.15.92:0/688319330 learned_addr learned my addr 172.21.15.92:0/688319330 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:11:16.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:16.756+0000 7fcd66ffd640 1 -- 172.21.15.92:0/688319330 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fcd6815f9c0 msgr2=0x7fcd6815d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:16.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:16.756+0000 7fcd677fe640 1 --2- 172.21.15.92:0/688319330 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fcd6815f9c0 0x7fcd6815d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:16.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:16.756+0000 7fcd667fc640 1 --2- 172.21.15.92:0/688319330 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fcd6813e360 0x7fcd68139900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:16.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:16.756+0000 7fcd66ffd640 1 --2- 172.21.15.92:0/688319330 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fcd6815f9c0 0x7fcd6815d8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:16.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:16.756+0000 7fcd66ffd640 1 -- 172.21.15.92:0/688319330 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fcd6813e360 msgr2=0x7fcd68139900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:16.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:16.756+0000 7fcd66ffd640 1 --2- 172.21.15.92:0/688319330 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fcd6813e360 0x7fcd68139900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:16.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:16.756+0000 7fcd66ffd640 1 -- 172.21.15.92:0/688319330 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd5c0094d0 con 0x7fcd6813df80 2023-11-24T06:11:16.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:16.756+0000 7fcd677fe640 1 --2- 172.21.15.92:0/688319330 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fcd6815f9c0 0x7fcd6815d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:11:16.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:16.756+0000 7fcd66ffd640 1 --2- 172.21.15.92:0/688319330 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcd6813df80 0x7fcd681e4b00 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fcd5800b410 tx=0x7fcd5800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:16.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:16.757+0000 7fcd4ffff640 1 -- 172.21.15.92:0/688319330 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd58011830 con 0x7fcd6813df80 2023-11-24T06:11:16.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:16.757+0000 7fcd4ffff640 1 -- 172.21.15.92:0/688319330 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd580119b0 con 0x7fcd6813df80 2023-11-24T06:11:16.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:16.757+0000 7fcd6d9c5640 1 -- 172.21.15.92:0/688319330 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd6813bd60 con 0x7fcd6813df80 2023-11-24T06:11:16.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:16.757+0000 7fcd4ffff640 1 -- 172.21.15.92:0/688319330 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd58011d60 con 0x7fcd6813df80 2023-11-24T06:11:16.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:16.757+0000 7fcd6d9c5640 1 -- 172.21.15.92:0/688319330 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd681e5040 con 0x7fcd6813df80 2023-11-24T06:11:16.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:16.758+0000 7fcd6d9c5640 1 -- 172.21.15.92:0/688319330 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd6813c280 con 0x7fcd6813df80 2023-11-24T06:11:16.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:16.759+0000 7fcd4ffff640 1 -- 172.21.15.92:0/688319330 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fcd5801a400 con 0x7fcd6813df80 2023-11-24T06:11:16.766 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:16.760+0000 7fcd4ffff640 1 --2- 172.21.15.92:0/688319330 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fcd5406ae40 0x7fcd5406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:16.766 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:16.760+0000 7fcd4ffff640 1 -- 172.21.15.92:0/688319330 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5883+0+0 (secure 0 0 0) 0x7fcd580897b0 con 0x7fcd6813df80 2023-11-24T06:11:16.766 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:16.760+0000 7fcd667fc640 1 --2- 172.21.15.92:0/688319330 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fcd5406ae40 0x7fcd5406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:16.766 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:16.761+0000 7fcd667fc640 1 --2- 172.21.15.92:0/688319330 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fcd5406ae40 0x7fcd5406d300 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fcd5c036000 tx=0x7fcd5c03c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:16.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:16.765+0000 7fcd4ffff640 1 -- 172.21.15.92:0/688319330 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fcd6813c280 con 0x7fcd6813df80 2023-11-24T06:11:16.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:16.954+0000 7fcd6d9c5640 1 -- 172.21.15.92:0/688319330 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fcd6813c280 con 0x7fcd6813df80 2023-11-24T06:11:16.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:16.955+0000 7fcd4ffff640 1 -- 172.21.15.92:0/688319330 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7fcd6813c280 con 0x7fcd6813df80 2023-11-24T06:11:16.959 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 32 2023-11-24T06:11:16.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:16.958+0000 7fcd6d9c5640 1 -- 172.21.15.92:0/688319330 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fcd5406ae40 msgr2=0x7fcd5406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:16.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:16.958+0000 7fcd6d9c5640 1 --2- 172.21.15.92:0/688319330 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fcd5406ae40 0x7fcd5406d300 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fcd5c036000 tx=0x7fcd5c03c040 comp rx=0 tx=0).stop 2023-11-24T06:11:16.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:16.958+0000 7fcd6d9c5640 1 -- 172.21.15.92:0/688319330 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcd6813df80 msgr2=0x7fcd681e4b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:16.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:16.958+0000 7fcd6d9c5640 1 --2- 172.21.15.92:0/688319330 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcd6813df80 0x7fcd681e4b00 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fcd5800b410 tx=0x7fcd5800b8d0 comp rx=0 tx=0).stop 2023-11-24T06:11:16.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:16.958+0000 7fcd6d9c5640 1 -- 172.21.15.92:0/688319330 shutdown_connections 2023-11-24T06:11:16.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:16.958+0000 7fcd6d9c5640 1 --2- 172.21.15.92:0/688319330 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fcd5406ae40 0x7fcd5406d300 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:16.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:16.958+0000 7fcd6d9c5640 1 --2- 172.21.15.92:0/688319330 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fcd6815f9c0 0x7fcd6815d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:16.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:16.958+0000 7fcd6d9c5640 1 --2- 172.21.15.92:0/688319330 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fcd6813e360 0x7fcd68139900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:16.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:16.958+0000 7fcd6d9c5640 1 --2- 172.21.15.92:0/688319330 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcd6813df80 0x7fcd681e4b00 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:16.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:16.958+0000 7fcd6d9c5640 1 -- 172.21.15.92:0/688319330 >> 172.21.15.92:0/688319330 conn(0x7fcd6806a800 msgr2=0x7fcd68154190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:16.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:16.958+0000 7fcd6d9c5640 1 -- 172.21.15.92:0/688319330 shutdown_connections 2023-11-24T06:11:16.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:16.958+0000 7fcd6d9c5640 1 -- 172.21.15.92:0/688319330 wait complete. 2023-11-24T06:11:16.981 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T06:11:17.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:17.186+0000 7f1f4cf68640 1 Processor -- start 2023-11-24T06:11:17.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:17.186+0000 7f1f4cf68640 1 -- start start 2023-11-24T06:11:17.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:17.187+0000 7f1f4cf68640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1f480cf9a0 0x7f1f48166030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:17.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:17.187+0000 7f1f4cf68640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f4815c5f0 con 0x7f1f4813e360 2023-11-24T06:11:17.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:17.187+0000 7f1f4cf68640 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f4806c470 con 0x7f1f4813df80 2023-11-24T06:11:17.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:17.187+0000 7f1f4cf68640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f4806c750 con 0x7f1f480cf9a0 2023-11-24T06:11:17.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:17.187+0000 7f1f45d9b640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f1f4813e360 0x7f1f480cf290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:54324/0 (socket says 172.21.15.92:54324) 2023-11-24T06:11:17.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:17.188+0000 7f1f45d9b640 1 -- 172.21.15.92:0/2944918354 learned_addr learned my addr 172.21.15.92:0/2944918354 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-24T06:11:17.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:17.188+0000 7f1f4559a640 1 -- 172.21.15.92:0/2944918354 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3972185587 0 0) 0x7f1f4815c5f0 con 0x7f1f4813e360 2023-11-24T06:11:17.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:17.188+0000 7f1f4559a640 1 -- 172.21.15.92:0/2944918354 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1f30003520 con 0x7f1f4813e360 2023-11-24T06:11:17.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:17.188+0000 7f1f46d9d640 1 --2- 172.21.15.92:0/2944918354 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1f480cf9a0 0x7f1f48166030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:17.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:17.188+0000 7f1f4559a640 1 -- 172.21.15.92:0/2944918354 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1589201768 0 0) 0x7f1f30003520 con 0x7f1f4813e360 2023-11-24T06:11:17.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:17.188+0000 7f1f4559a640 1 -- 172.21.15.92:0/2944918354 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1f480cf9a0 msgr2=0x7f1f48166030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:17.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:17.188+0000 7f1f4559a640 1 --2- 172.21.15.92:0/2944918354 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1f480cf9a0 0x7f1f48166030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:17.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:17.188+0000 7f1f4559a640 1 -- 172.21.15.92:0/2944918354 >> v1:172.21.15.118:6789/0 conn(0x7f1f4813df80 legacy=0x7f1f4815bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:17.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:17.188+0000 7f1f4559a640 1 -- 172.21.15.92:0/2944918354 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f4806ca30 con 0x7f1f4813e360 2023-11-24T06:11:17.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:17.188+0000 7f1f4559a640 1 -- 172.21.15.92:0/2944918354 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2607661350 0 0) 0x7f1f3c001670 con 0x7f1f4813e360 2023-11-24T06:11:17.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:17.188+0000 7f1f4559a640 1 -- 172.21.15.92:0/2944918354 >> v1:172.21.15.92:6789/0 conn(0x7f1f4813e360 legacy=0x7f1f480cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:17.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:17.188+0000 7f1f4559a640 1 --2- 172.21.15.92:0/2944918354 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1f300039a0 0x7f1f30003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:17.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:17.188+0000 7f1f4559a640 1 --2- 172.21.15.92:0/2944918354 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1f480cf9a0 0x7f1f300047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:17.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:17.188+0000 7f1f4559a640 1 --2- 172.21.15.92:0/2944918354 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1f30004d00 0x7f1f30005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:17.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:17.188+0000 7f1f4559a640 1 -- 172.21.15.92:0/2944918354 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f4806c750 con 0x7f1f300039a0 2023-11-24T06:11:17.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:17.188+0000 7f1f4559a640 1 -- 172.21.15.92:0/2944918354 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f3000bae0 con 0x7f1f30004d00 2023-11-24T06:11:17.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:17.188+0000 7f1f4559a640 1 -- 172.21.15.92:0/2944918354 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f3000bca0 con 0x7f1f480cf9a0 2023-11-24T06:11:17.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:17.189+0000 7f1f4659c640 1 --2- 172.21.15.92:0/2944918354 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1f300039a0 0x7f1f30003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:17.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:17.189+0000 7f1f4659c640 1 -- 172.21.15.92:0/2944918354 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1f480cf9a0 msgr2=0x7f1f300047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:17.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:17.189+0000 7f1f4659c640 1 --2- 172.21.15.92:0/2944918354 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1f480cf9a0 0x7f1f300047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:17.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:17.189+0000 7f1f4659c640 1 -- 172.21.15.92:0/2944918354 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1f30004d00 msgr2=0x7f1f30005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:17.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:17.189+0000 7f1f4659c640 1 --2- 172.21.15.92:0/2944918354 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1f30004d00 0x7f1f30005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:17.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:17.189+0000 7f1f4659c640 1 -- 172.21.15.92:0/2944918354 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f30003520 con 0x7f1f300039a0 2023-11-24T06:11:17.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:17.190+0000 7f1f4659c640 1 --2- 172.21.15.92:0/2944918354 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1f300039a0 0x7f1f30003da0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f1f38003db0 tx=0x7f1f300057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bc2e79dc2df38fd server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:17.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:17.190+0000 7f1f4559a640 1 -- 172.21.15.92:0/2944918354 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f38032600 con 0x7f1f300039a0 2023-11-24T06:11:17.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:17.190+0000 7f1f4559a640 1 -- 172.21.15.92:0/2944918354 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f3802e050 con 0x7f1f300039a0 2023-11-24T06:11:17.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:17.190+0000 7f1f4559a640 1 -- 172.21.15.92:0/2944918354 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f380329a0 con 0x7f1f300039a0 2023-11-24T06:11:17.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:17.191+0000 7f1f4cf68640 1 -- 172.21.15.92:0/2944918354 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1f300039a0 msgr2=0x7f1f30003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:17.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:17.191+0000 7f1f4cf68640 1 --2- 172.21.15.92:0/2944918354 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1f300039a0 0x7f1f30003da0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f1f38003db0 tx=0x7f1f300057f0 comp rx=0 tx=0).stop 2023-11-24T06:11:17.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:17.191+0000 7f1f46d9d640 1 -- 172.21.15.92:0/2944918354 reap_dead start 2023-11-24T06:11:17.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:17.191+0000 7f1f4cf68640 1 -- 172.21.15.92:0/2944918354 shutdown_connections 2023-11-24T06:11:17.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:17.191+0000 7f1f4cf68640 1 -- 172.21.15.92:0/2944918354 >> 172.21.15.92:0/2944918354 conn(0x7f1f4806a800 msgr2=0x7f1f4806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:17.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:17.191+0000 7f1f4cf68640 1 -- 172.21.15.92:0/2944918354 shutdown_connections 2023-11-24T06:11:17.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:17.191+0000 7f1f4cf68640 1 -- 172.21.15.92:0/2944918354 wait complete. 2023-11-24T06:11:17.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:17.193+0000 7f1f4cf68640 1 Processor -- start 2023-11-24T06:11:17.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:17.193+0000 7f1f4cf68640 1 -- start start 2023-11-24T06:11:17.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:17.194+0000 7f1f4cf68640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1f300039a0 0x7f1f480b32d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:17.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:17.194+0000 7f1f4cf68640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1f48132fc0 0x7f1f481f2590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:17.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:17.194+0000 7f1f4cf68640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1f481f2ad0 0x7f1f480a84c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:17.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:17.194+0000 7f1f4cf68640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f481f2f50 con 0x7f1f48132fc0 2023-11-24T06:11:17.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:17.194+0000 7f1f4cf68640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f481337f0 con 0x7f1f481f2ad0 2023-11-24T06:11:17.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:17.194+0000 7f1f4cf68640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f48133ad0 con 0x7f1f300039a0 2023-11-24T06:11:17.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:17.194+0000 7f1f45d9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1f48132fc0 0x7f1f481f2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:17.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:17.194+0000 7f1f45d9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1f48132fc0 0x7f1f481f2590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:33062/0 (socket says 172.21.15.92:33062) 2023-11-24T06:11:17.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:17.194+0000 7f1f45d9b640 1 -- 172.21.15.92:0/2434736241 learned_addr learned my addr 172.21.15.92:0/2434736241 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:11:17.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:17.194+0000 7f1f45d9b640 1 -- 172.21.15.92:0/2434736241 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1f300039a0 msgr2=0x7f1f480b32d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:17.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:17.194+0000 7f1f45d9b640 1 --2- 172.21.15.92:0/2434736241 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1f300039a0 0x7f1f480b32d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:17.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:17.194+0000 7f1f45d9b640 1 -- 172.21.15.92:0/2434736241 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1f481f2ad0 msgr2=0x7f1f480a84c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:17.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:17.194+0000 7f1f45d9b640 1 --2- 172.21.15.92:0/2434736241 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1f481f2ad0 0x7f1f480a84c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:17.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:17.194+0000 7f1f45d9b640 1 -- 172.21.15.92:0/2434736241 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f48133db0 con 0x7f1f48132fc0 2023-11-24T06:11:17.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:17.195+0000 7f1f45d9b640 1 --2- 172.21.15.92:0/2434736241 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1f48132fc0 0x7f1f481f2590 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f1f3c00c840 tx=0x7f1f3c00cd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:17.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:17.195+0000 7f1f36ffd640 1 -- 172.21.15.92:0/2434736241 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f3c0074e0 con 0x7f1f48132fc0 2023-11-24T06:11:17.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:17.195+0000 7f1f36ffd640 1 -- 172.21.15.92:0/2434736241 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f3c007660 con 0x7f1f48132fc0 2023-11-24T06:11:17.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:17.195+0000 7f1f36ffd640 1 -- 172.21.15.92:0/2434736241 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f3c004a20 con 0x7f1f48132fc0 2023-11-24T06:11:17.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:17.195+0000 7f1f4cf68640 1 -- 172.21.15.92:0/2434736241 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f48134090 con 0x7f1f48132fc0 2023-11-24T06:11:17.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:17.195+0000 7f1f4cf68640 1 -- 172.21.15.92:0/2434736241 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f48133440 con 0x7f1f48132fc0 2023-11-24T06:11:17.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:17.196+0000 7f1f4cf68640 1 -- 172.21.15.92:0/2434736241 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f481345b0 con 0x7f1f48132fc0 2023-11-24T06:11:17.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:17.197+0000 7f1f36ffd640 1 -- 172.21.15.92:0/2434736241 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f1f3c004b80 con 0x7f1f48132fc0 2023-11-24T06:11:17.201 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:17.198+0000 7f1f36ffd640 1 --2- 172.21.15.92:0/2434736241 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f1f2806ae40 0x7f1f2806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:17.201 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:17.198+0000 7f1f36ffd640 1 -- 172.21.15.92:0/2434736241 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5883+0+0 (secure 0 0 0) 0x7f1f3c088c30 con 0x7f1f48132fc0 2023-11-24T06:11:17.202 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:17.199+0000 7f1f4659c640 1 --2- 172.21.15.92:0/2434736241 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f1f2806ae40 0x7f1f2806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:17.202 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:17.199+0000 7f1f4659c640 1 --2- 172.21.15.92:0/2434736241 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f1f2806ae40 0x7f1f2806d300 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f1f38034000 tx=0x7f1f3802f000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:17.205 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:17.203+0000 7f1f36ffd640 1 -- 172.21.15.92:0/2434736241 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1f481345b0 con 0x7f1f48132fc0 2023-11-24T06:11:17.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:17.378+0000 7f1f4cf68640 1 -- 172.21.15.92:0/2434736241 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f1f481345b0 con 0x7f1f48132fc0 2023-11-24T06:11:17.382 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:17.379+0000 7f1f36ffd640 1 -- 172.21.15.92:0/2434736241 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7f1f481345b0 con 0x7f1f48132fc0 2023-11-24T06:11:17.382 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 64 2023-11-24T06:11:17.384 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:17.381+0000 7f1f4cf68640 1 -- 172.21.15.92:0/2434736241 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f1f2806ae40 msgr2=0x7f1f2806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:17.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:17.381+0000 7f1f4cf68640 1 --2- 172.21.15.92:0/2434736241 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f1f2806ae40 0x7f1f2806d300 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f1f38034000 tx=0x7f1f3802f000 comp rx=0 tx=0).stop 2023-11-24T06:11:17.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:17.381+0000 7f1f4cf68640 1 -- 172.21.15.92:0/2434736241 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1f48132fc0 msgr2=0x7f1f481f2590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:17.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:17.381+0000 7f1f4cf68640 1 --2- 172.21.15.92:0/2434736241 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1f48132fc0 0x7f1f481f2590 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f1f3c00c840 tx=0x7f1f3c00cd00 comp rx=0 tx=0).stop 2023-11-24T06:11:17.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:17.381+0000 7f1f4cf68640 1 -- 172.21.15.92:0/2434736241 shutdown_connections 2023-11-24T06:11:17.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:17.381+0000 7f1f4cf68640 1 --2- 172.21.15.92:0/2434736241 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f1f2806ae40 0x7f1f2806d300 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:17.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:17.381+0000 7f1f4cf68640 1 --2- 172.21.15.92:0/2434736241 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1f481f2ad0 0x7f1f480a84c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:17.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:17.381+0000 7f1f4cf68640 1 --2- 172.21.15.92:0/2434736241 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1f48132fc0 0x7f1f481f2590 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:17.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:17.381+0000 7f1f4cf68640 1 --2- 172.21.15.92:0/2434736241 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1f300039a0 0x7f1f480b32d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:17.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:17.381+0000 7f1f4cf68640 1 -- 172.21.15.92:0/2434736241 >> 172.21.15.92:0/2434736241 conn(0x7f1f4806a800 msgr2=0x7f1f48070630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:17.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:17.381+0000 7f1f4cf68640 1 -- 172.21.15.92:0/2434736241 shutdown_connections 2023-11-24T06:11:17.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:17.381+0000 7f1f4cf68640 1 -- 172.21.15.92:0/2434736241 wait complete. 2023-11-24T06:11:17.404 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-24T06:11:17.611 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:17.608+0000 7f3653f33640 1 Processor -- start 2023-11-24T06:11:17.612 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:17.608+0000 7f3653f33640 1 -- start start 2023-11-24T06:11:17.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:17.610+0000 7f3653f33640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f364c0cd240 0x7f364c0b3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:17.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:17.610+0000 7f3653f33640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f364c09a170 0x7f364c0cf640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:17.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:17.610+0000 7f3653f33640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f364c06cea0 con 0x7f364c06c6b0 2023-11-24T06:11:17.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:17.610+0000 7f3653f33640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f364c06cbc0 con 0x7f364c0cd240 2023-11-24T06:11:17.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:17.610+0000 7f3653f33640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f364c06f350 con 0x7f364c09a170 2023-11-24T06:11:17.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:17.610+0000 7f3651ccf640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f364c0cd240 0x7f364c0b3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:17.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:17.610+0000 7f36514ce640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f364c06c6b0 0x7f364c099a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:54336/0 (socket says 172.21.15.92:54336) 2023-11-24T06:11:17.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:17.610+0000 7f36514ce640 1 -- 172.21.15.92:0/892089476 learned_addr learned my addr 172.21.15.92:0/892089476 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-24T06:11:17.615 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:17.611+0000 7f36524d0640 1 --2- 172.21.15.92:0/892089476 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f364c09a170 0x7f364c0cf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:17.615 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:17.611+0000 7f3651ccf640 1 -- 172.21.15.92:0/892089476 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f364c09a170 msgr2=0x7f364c0cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:17.615 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:17.611+0000 7f3651ccf640 1 --2- 172.21.15.92:0/892089476 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f364c09a170 0x7f364c0cf640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:17.615 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:17.611+0000 7f3651ccf640 1 -- 172.21.15.92:0/892089476 >> v1:172.21.15.92:6789/0 conn(0x7f364c06c6b0 legacy=0x7f364c099a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:17.615 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:17.611+0000 7f3651ccf640 1 -- 172.21.15.92:0/892089476 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f364c06f630 con 0x7f364c0cd240 2023-11-24T06:11:17.615 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:17.611+0000 7f36524d0640 1 --2- 172.21.15.92:0/892089476 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f364c09a170 0x7f364c0cf640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:11:17.615 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:17.611+0000 7f3650ccd640 1 -- 172.21.15.92:0/892089476 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3056872238 0 0) 0x7f364c06cea0 con 0x7f364c06c6b0 2023-11-24T06:11:17.616 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:17.612+0000 7f3651ccf640 1 --2- 172.21.15.92:0/892089476 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f364c0cd240 0x7f364c0b3820 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f363c0097a0 tx=0x7f363c02b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a4c5b822ce697da server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:17.616 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:17.612+0000 7f3650ccd640 1 -- 172.21.15.92:0/892089476 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f363c02d040 con 0x7f364c0cd240 2023-11-24T06:11:17.616 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:17.612+0000 7f3650ccd640 1 -- 172.21.15.92:0/892089476 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f363c031810 con 0x7f364c0cd240 2023-11-24T06:11:17.616 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:17.612+0000 7f3650ccd640 1 -- 172.21.15.92:0/892089476 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f363c031ba0 con 0x7f364c0cd240 2023-11-24T06:11:17.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:17.613+0000 7f3653f33640 1 -- 172.21.15.92:0/892089476 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f364c0cd240 msgr2=0x7f364c0b3820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:17.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:17.613+0000 7f3653f33640 1 --2- 172.21.15.92:0/892089476 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f364c0cd240 0x7f364c0b3820 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f363c0097a0 tx=0x7f363c02b8b0 comp rx=0 tx=0).stop 2023-11-24T06:11:17.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:17.613+0000 7f3653f33640 1 -- 172.21.15.92:0/892089476 shutdown_connections 2023-11-24T06:11:17.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:17.613+0000 7f3653f33640 1 --2- 172.21.15.92:0/892089476 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f364c09a170 0x7f364c0cf640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:17.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:17.613+0000 7f3653f33640 1 --2- 172.21.15.92:0/892089476 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f364c0cd240 0x7f364c0b3820 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:17.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:17.613+0000 7f3653f33640 1 -- 172.21.15.92:0/892089476 >> 172.21.15.92:0/892089476 conn(0x7f364c0b2d00 msgr2=0x7f364c0b3110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:17.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:17.613+0000 7f3653f33640 1 -- 172.21.15.92:0/892089476 shutdown_connections 2023-11-24T06:11:17.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:17.613+0000 7f3653f33640 1 -- 172.21.15.92:0/892089476 wait complete. 2023-11-24T06:11:17.618 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:17.615+0000 7f3653f33640 1 Processor -- start 2023-11-24T06:11:17.618 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:17.615+0000 7f3653f33640 1 -- start start 2023-11-24T06:11:17.619 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:17.616+0000 7f3653f33640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f364c06c6b0 0x7f364c1f6940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:17.619 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:17.616+0000 7f3653f33640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f364c09a170 0x7f364c151f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:17.619 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:17.616+0000 7f3653f33640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f364c0cd240 0x7f364c06b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:17.619 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:17.616+0000 7f3653f33640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f364c0b6fe0 con 0x7f364c09a170 2023-11-24T06:11:17.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:17.616+0000 7f3653f33640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f364c1524d0 con 0x7f364c06c6b0 2023-11-24T06:11:17.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:17.616+0000 7f3653f33640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f364c0b6d00 con 0x7f364c0cd240 2023-11-24T06:11:17.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:17.616+0000 7f36514ce640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f364c09a170 0x7f364c151f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:17.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:17.616+0000 7f36514ce640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f364c09a170 0x7f364c151f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:33076/0 (socket says 172.21.15.92:33076) 2023-11-24T06:11:17.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:17.616+0000 7f36514ce640 1 -- 172.21.15.92:0/920342166 learned_addr learned my addr 172.21.15.92:0/920342166 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:11:17.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:17.616+0000 7f3651ccf640 1 --2- 172.21.15.92:0/920342166 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f364c06c6b0 0x7f364c1f6940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:17.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:17.616+0000 7f36514ce640 1 -- 172.21.15.92:0/920342166 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f364c0cd240 msgr2=0x7f364c06b4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:17.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:17.616+0000 7f36514ce640 1 --2- 172.21.15.92:0/920342166 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f364c0cd240 0x7f364c06b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:17.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:17.616+0000 7f36514ce640 1 -- 172.21.15.92:0/920342166 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f364c06c6b0 msgr2=0x7f364c1f6940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:17.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:17.616+0000 7f36514ce640 1 --2- 172.21.15.92:0/920342166 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f364c06c6b0 0x7f364c1f6940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:17.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:17.616+0000 7f36514ce640 1 -- 172.21.15.92:0/920342166 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f363c009400 con 0x7f364c09a170 2023-11-24T06:11:17.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:17.617+0000 7f36514ce640 1 --2- 172.21.15.92:0/920342166 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f364c09a170 0x7f364c151f90 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f364000c410 tx=0x7f364000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:17.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:17.617+0000 7f364a7fc640 1 -- 172.21.15.92:0/920342166 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36400074e0 con 0x7f364c09a170 2023-11-24T06:11:17.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:17.617+0000 7f364a7fc640 1 -- 172.21.15.92:0/920342166 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3640007660 con 0x7f364c09a170 2023-11-24T06:11:17.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:17.617+0000 7f364a7fc640 1 -- 172.21.15.92:0/920342166 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3640004a20 con 0x7f364c09a170 2023-11-24T06:11:17.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:17.617+0000 7f3653f33640 1 -- 172.21.15.92:0/920342166 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f364c1f5e60 con 0x7f364c09a170 2023-11-24T06:11:17.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:17.617+0000 7f3653f33640 1 -- 172.21.15.92:0/920342166 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f364c1f6420 con 0x7f364c09a170 2023-11-24T06:11:17.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:17.618+0000 7f3653f33640 1 -- 172.21.15.92:0/920342166 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f364c1f6640 con 0x7f364c09a170 2023-11-24T06:11:17.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:17.619+0000 7f364a7fc640 1 -- 172.21.15.92:0/920342166 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f3640004b80 con 0x7f364c09a170 2023-11-24T06:11:17.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:17.620+0000 7f364a7fc640 1 --2- 172.21.15.92:0/920342166 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f363806ae40 0x7f363806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:17.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:17.620+0000 7f364a7fc640 1 -- 172.21.15.92:0/920342166 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5883+0+0 (secure 0 0 0) 0x7f3640089560 con 0x7f364c09a170 2023-11-24T06:11:17.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:17.620+0000 7f3651ccf640 1 --2- 172.21.15.92:0/920342166 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f363806ae40 0x7f363806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:17.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:17.621+0000 7f3651ccf640 1 --2- 172.21.15.92:0/920342166 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f363806ae40 0x7f363806d300 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f363c036000 tx=0x7f363c03c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:17.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:17.624+0000 7f364a7fc640 1 -- 172.21.15.92:0/920342166 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3640057f50 con 0x7f364c09a170 2023-11-24T06:11:17.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:17.829+0000 7f3653f33640 1 -- 172.21.15.92:0/920342166 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f364c0675d0 con 0x7f364c09a170 2023-11-24T06:11:17.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:17.829+0000 7f364a7fc640 1 -- 172.21.15.92:0/920342166 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) v1 ==== 80+0+0 (secure 0 0 0) 0x7f3640057ae0 con 0x7f364c09a170 2023-11-24T06:11:17.833 INFO:teuthology.orchestra.run.smithi092.stderr:listed 0 entries 2023-11-24T06:11:17.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:17.832+0000 7f3653f33640 1 -- 172.21.15.92:0/920342166 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f363806ae40 msgr2=0x7f363806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:17.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:17.832+0000 7f3653f33640 1 --2- 172.21.15.92:0/920342166 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f363806ae40 0x7f363806d300 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f363c036000 tx=0x7f363c03c040 comp rx=0 tx=0).stop 2023-11-24T06:11:17.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:17.832+0000 7f3653f33640 1 -- 172.21.15.92:0/920342166 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f364c09a170 msgr2=0x7f364c151f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:17.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:17.832+0000 7f3653f33640 1 --2- 172.21.15.92:0/920342166 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f364c09a170 0x7f364c151f90 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f364000c410 tx=0x7f364000c8d0 comp rx=0 tx=0).stop 2023-11-24T06:11:17.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:17.832+0000 7f3653f33640 1 -- 172.21.15.92:0/920342166 shutdown_connections 2023-11-24T06:11:17.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:17.832+0000 7f3653f33640 1 --2- 172.21.15.92:0/920342166 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f363806ae40 0x7f363806d300 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:17.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:17.832+0000 7f3653f33640 1 --2- 172.21.15.92:0/920342166 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f364c0cd240 0x7f364c06b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:17.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:17.832+0000 7f3653f33640 1 --2- 172.21.15.92:0/920342166 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f364c09a170 0x7f364c151f90 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:17.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:17.832+0000 7f3653f33640 1 --2- 172.21.15.92:0/920342166 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f364c06c6b0 0x7f364c1f6940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:17.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:17.832+0000 7f3653f33640 1 -- 172.21.15.92:0/920342166 >> 172.21.15.92:0/920342166 conn(0x7f364c0b2d00 msgr2=0x7f364c0b30e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:17.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:17.832+0000 7f3653f33640 1 -- 172.21.15.92:0/920342166 shutdown_connections 2023-11-24T06:11:17.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:17.832+0000 7f3653f33640 1 -- 172.21.15.92:0/920342166 wait complete. 2023-11-24T06:11:17.854 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi118.front.sepia.ceph.com... 2023-11-24T06:11:17.854 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:11:17.854 DEBUG:teuthology.orchestra.run.smithi118:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2023-11-24T06:11:17.963 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T06:11:17.963 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi118.front.sepia.ceph.com 2023-11-24T06:11:17.963 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:11:17.963 DEBUG:teuthology.orchestra.run.smithi118:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-11-24T06:11:18.030 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-11-24T06:11:18.030 DEBUG:teuthology.orchestra.run.smithi092:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:11:18.091 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-11-24T06:11:18.091 DEBUG:teuthology.orchestra.run.smithi092:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:11:18.144 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:11:18.389 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:18.385+0000 7feb649cf640 1 Processor -- start 2023-11-24T06:11:18.389 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:18.386+0000 7feb649cf640 1 -- start start 2023-11-24T06:11:18.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:18.387+0000 7feb649cf640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7feb60143c40 0x7feb60071290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:18.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:18.387+0000 7feb649cf640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7feb600717d0 0x7feb600cc500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:18.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:18.387+0000 7feb649cf640 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb60070e50 con 0x7feb600cca40 2023-11-24T06:11:18.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:18.387+0000 7feb649cf640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb600723c0 con 0x7feb60143c40 2023-11-24T06:11:18.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:18.387+0000 7feb649cf640 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb600cce40 con 0x7feb600717d0 2023-11-24T06:11:18.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:18.387+0000 7feb5f7fe640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7feb60143c40 0x7feb60071290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:18.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:18.387+0000 7feb5f7fe640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7feb60143c40 0x7feb60071290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:33088/0 (socket says 172.21.15.92:33088) 2023-11-24T06:11:18.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:18.387+0000 7feb5f7fe640 1 -- 172.21.15.92:0/2920809976 learned_addr learned my addr 172.21.15.92:0/2920809976 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:11:18.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:18.388+0000 7feb5f7fe640 1 -- 172.21.15.92:0/2920809976 >> v2:172.21.15.118:3300/0 conn(0x7feb600717d0 msgr2=0x7feb600cc500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:18.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:18.388+0000 7feb5f7fe640 1 --2- 172.21.15.92:0/2920809976 >> v2:172.21.15.118:3300/0 conn(0x7feb600717d0 0x7feb600cc500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:18.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:18.388+0000 7feb5f7fe640 1 -- 172.21.15.92:0/2920809976 >> v1:172.21.15.118:6789/0 conn(0x7feb600cca40 legacy=0x7feb6015eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:18.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:18.388+0000 7feb5f7fe640 1 -- 172.21.15.92:0/2920809976 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb6006e170 con 0x7feb60143c40 2023-11-24T06:11:18.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:18.389+0000 7feb5f7fe640 1 --2- 172.21.15.92:0/2920809976 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feb60143c40 0x7feb60071290 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7feb600b44a0 tx=0x7feb5002be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5363f0fb3f35b756 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:18.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:18.389+0000 7feb5effd640 1 -- 172.21.15.92:0/2920809976 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb5002c9f0 con 0x7feb60143c40 2023-11-24T06:11:18.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:18.389+0000 7feb5effd640 1 -- 172.21.15.92:0/2920809976 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb5002cb50 con 0x7feb60143c40 2023-11-24T06:11:18.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:18.389+0000 7feb5effd640 1 -- 172.21.15.92:0/2920809976 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb500343f0 con 0x7feb60143c40 2023-11-24T06:11:18.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:18.390+0000 7feb649cf640 1 -- 172.21.15.92:0/2920809976 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feb60143c40 msgr2=0x7feb60071290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:18.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:18.390+0000 7feb649cf640 1 --2- 172.21.15.92:0/2920809976 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feb60143c40 0x7feb60071290 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7feb600b44a0 tx=0x7feb5002be50 comp rx=0 tx=0).stop 2023-11-24T06:11:18.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:18.390+0000 7feb649cf640 1 -- 172.21.15.92:0/2920809976 shutdown_connections 2023-11-24T06:11:18.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:18.390+0000 7feb649cf640 1 --2- 172.21.15.92:0/2920809976 >> v2:172.21.15.118:3300/0 conn(0x7feb600717d0 0x7feb600cc500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:18.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:18.390+0000 7feb649cf640 1 --2- 172.21.15.92:0/2920809976 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feb60143c40 0x7feb60071290 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:18.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:18.390+0000 7feb649cf640 1 -- 172.21.15.92:0/2920809976 >> 172.21.15.92:0/2920809976 conn(0x7feb6006a800 msgr2=0x7feb6006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:18.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:18.390+0000 7feb649cf640 1 -- 172.21.15.92:0/2920809976 shutdown_connections 2023-11-24T06:11:18.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:18.390+0000 7feb649cf640 1 -- 172.21.15.92:0/2920809976 wait complete. 2023-11-24T06:11:18.395 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:18.392+0000 7feb649cf640 1 Processor -- start 2023-11-24T06:11:18.395 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:18.392+0000 7feb649cf640 1 -- start start 2023-11-24T06:11:18.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:18.393+0000 7feb649cf640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7feb600717d0 0x7feb601f6c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:18.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:18.393+0000 7feb649cf640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7feb600cca40 0x7feb6015a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:18.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:18.393+0000 7feb649cf640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feb60143c40 0x7feb600c9900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:18.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:18.393+0000 7feb649cf640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb60130120 con 0x7feb60143c40 2023-11-24T06:11:18.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:18.393+0000 7feb649cf640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb600ca1a0 con 0x7feb600cca40 2023-11-24T06:11:18.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:18.393+0000 7feb649cf640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb600ca480 con 0x7feb600717d0 2023-11-24T06:11:18.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:18.393+0000 7feb5ffff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feb60143c40 0x7feb600c9900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:18.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:18.393+0000 7feb5ffff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feb60143c40 0x7feb600c9900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:33094/0 (socket says 172.21.15.92:33094) 2023-11-24T06:11:18.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:18.393+0000 7feb5ffff640 1 -- 172.21.15.92:0/3070674088 learned_addr learned my addr 172.21.15.92:0/3070674088 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:11:18.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:18.394+0000 7feb56ffd640 1 --2- 172.21.15.92:0/3070674088 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7feb600cca40 0x7feb6015a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:18.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:18.394+0000 7feb5ffff640 1 -- 172.21.15.92:0/3070674088 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7feb600717d0 msgr2=0x7feb601f6c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:18.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:18.394+0000 7feb5ffff640 1 --2- 172.21.15.92:0/3070674088 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7feb600717d0 0x7feb601f6c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:18.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:18.394+0000 7feb5ffff640 1 -- 172.21.15.92:0/3070674088 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7feb600cca40 msgr2=0x7feb6015a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:18.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:18.394+0000 7feb5ffff640 1 --2- 172.21.15.92:0/3070674088 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7feb600cca40 0x7feb6015a980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:18.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:18.394+0000 7feb5ffff640 1 -- 172.21.15.92:0/3070674088 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb50009b30 con 0x7feb60143c40 2023-11-24T06:11:18.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:18.394+0000 7feb56ffd640 1 --2- 172.21.15.92:0/3070674088 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7feb600cca40 0x7feb6015a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:11:18.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:18.394+0000 7feb5ffff640 1 --2- 172.21.15.92:0/3070674088 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feb60143c40 0x7feb600c9900 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7feb5800c390 tx=0x7feb5800c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:18.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:18.394+0000 7feb5cff9640 1 -- 172.21.15.92:0/3070674088 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb580035f0 con 0x7feb60143c40 2023-11-24T06:11:18.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:18.394+0000 7feb5cff9640 1 -- 172.21.15.92:0/3070674088 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb58003750 con 0x7feb60143c40 2023-11-24T06:11:18.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:18.394+0000 7feb5cff9640 1 -- 172.21.15.92:0/3070674088 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb58003b00 con 0x7feb60143c40 2023-11-24T06:11:18.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:18.394+0000 7feb649cf640 1 -- 172.21.15.92:0/3070674088 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb600ca760 con 0x7feb60143c40 2023-11-24T06:11:18.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:18.394+0000 7feb649cf640 1 -- 172.21.15.92:0/3070674088 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb6015aec0 con 0x7feb60143c40 2023-11-24T06:11:18.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:18.395+0000 7feb649cf640 1 -- 172.21.15.92:0/3070674088 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb601434a0 con 0x7feb60143c40 2023-11-24T06:11:18.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:18.396+0000 7feb5cff9640 1 -- 172.21.15.92:0/3070674088 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7feb58003c60 con 0x7feb60143c40 2023-11-24T06:11:18.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:18.397+0000 7feb5cff9640 1 --2- 172.21.15.92:0/3070674088 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7feb4006ae70 0x7feb4006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:18.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:18.397+0000 7feb5cff9640 1 -- 172.21.15.92:0/3070674088 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5883+0+0 (secure 0 0 0) 0x7feb58088890 con 0x7feb60143c40 2023-11-24T06:11:18.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:18.398+0000 7feb5f7fe640 1 --2- 172.21.15.92:0/3070674088 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7feb4006ae70 0x7feb4006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:18.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:18.398+0000 7feb5f7fe640 1 --2- 172.21.15.92:0/3070674088 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7feb4006ae70 0x7feb4006d330 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7feb50035000 tx=0x7feb5002e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:18.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:18.402+0000 7feb5cff9640 1 -- 172.21.15.92:0/3070674088 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7feb580537f0 con 0x7feb60143c40 2023-11-24T06:11:18.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:18.578+0000 7feb649cf640 1 -- 172.21.15.92:0/3070674088 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7feb60067450 con 0x7feb60143c40 2023-11-24T06:11:18.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:18.580+0000 7feb5cff9640 1 -- 172.21.15.92:0/3070674088 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16800 (secure 0 0 0) 0x7feb58056e10 con 0x7feb60143c40 2023-11-24T06:11:18.584 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-24T06:11:18.584 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":22,"fsid":"2d6dee5f-8aa3-470f-931a-50ade963b1cb","created":"2023-11-24T06:10:04.707204+0000","modified":"2023-11-24T06:11:11.472401+0000","last_up_change":"2023-11-24T06:10:17.598352+0000","last_in_change":"2023-11-24T06:10:09.040793+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:10:17.733098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:10:22.926445+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T06:10:24.151288+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ff37a9fb-135f-471f-b4b0-f7b5772006c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6803","nonce":20560714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6805","nonce":20560714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6809","nonce":20560714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6807","nonce":20560714}]},"public_addr":"172.21.15.92:6803/20560714","cluster_addr":"172.21.15.92:6805/20560714","heartbeat_back_addr":"172.21.15.92:6809/20560714","heartbeat_front_addr":"172.21.15.92:6807/20560714","state":["exists","up"]},{"osd":1,"uuid":"03cdd353-dcf1-4f38-94fe-a795137c06b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6819","nonce":2451239906}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6821","nonce":2451239906}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6825","nonce":2451239906}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6823","nonce":2451239906}]},"public_addr":"172.21.15.92:6819/2451239906","cluster_addr":"172.21.15.92:6821/2451239906","heartbeat_back_addr":"172.21.15.92:6825/2451239906","heartbeat_front_addr":"172.21.15.92:6823/2451239906","state":["exists","up"]},{"osd":2,"uuid":"a4a20d80-5d57-484b-becd-9aa8070dd3aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6811","nonce":1124979277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6813","nonce":1124979277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6817","nonce":1124979277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6815","nonce":1124979277}]},"public_addr":"172.21.15.92:6811/1124979277","cluster_addr":"172.21.15.92:6813/1124979277","heartbeat_back_addr":"172.21.15.92:6817/1124979277","heartbeat_front_addr":"172.21.15.92:6815/1124979277","state":["exists","up"]},{"osd":3,"uuid":"4648d7ce-9b29-4583-9425-4ba8fd2b1903","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6827","nonce":1734115782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6829","nonce":1734115782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6833","nonce":1734115782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6831","nonce":1734115782}]},"public_addr":"172.21.15.92:6827/1734115782","cluster_addr":"172.21.15.92:6829/1734115782","heartbeat_back_addr":"172.21.15.92:6833/1734115782","heartbeat_front_addr":"172.21.15.92:6831/1734115782","state":["exists","up"]},{"osd":4,"uuid":"3bc9d7eb-c6fa-4fcf-a3ec-ade604e05aac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6819","nonce":3425937229}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6822","nonce":3425937229}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6830","nonce":3425937229}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6826","nonce":3425937229}]},"public_addr":"172.21.15.118:6819/3425937229","cluster_addr":"172.21.15.118:6822/3425937229","heartbeat_back_addr":"172.21.15.118:6830/3425937229","heartbeat_front_addr":"172.21.15.118:6826/3425937229","state":["exists","up"]},{"osd":5,"uuid":"dce9ba6d-59ed-46d1-84fc-a84b13de10c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6801","nonce":827455984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6803","nonce":827455984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6807","nonce":827455984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6805","nonce":827455984}]},"public_addr":"172.21.15.118:6801/827455984","cluster_addr":"172.21.15.118:6803/827455984","heartbeat_back_addr":"172.21.15.118:6807/827455984","heartbeat_front_addr":"172.21.15.118:6805/827455984","state":["exists","up"]},{"osd":6,"uuid":"82aec504-ae58-4709-8d30-46f2b12081b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6809","nonce":2946986141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6811","nonce":2946986141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6815","nonce":2946986141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6813","nonce":2946986141}]},"public_addr":"172.21.15.118:6809/2946986141","cluster_addr":"172.21.15.118:6811/2946986141","heartbeat_back_addr":"172.21.15.118:6815/2946986141","heartbeat_front_addr":"172.21.15.118:6813/2946986141","state":["exists","up"]},{"osd":7,"uuid":"4ed4c624-c88f-4d35-bdb1-bf209b5dffb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6817","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6818","nonce":48122059}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6821","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6823","nonce":48122059}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6829","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6831","nonce":48122059}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6825","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6827","nonce":48122059}]},"public_addr":"172.21.15.118:6818/48122059","cluster_addr":"172.21.15.118:6823/48122059","heartbeat_back_addr":"172.21.15.118:6831/48122059","heartbeat_front_addr":"172.21.15.118:6827/48122059","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.536664+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.368324+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.271201+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.342971+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.621832+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.387110+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.632946+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.416032+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.d","mappings":[{"from":5,"to":1}]},{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"3.29","mappings":[{"from":3,"to":2}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.34","mappings":[{"from":4,"to":2}]},{"pgid":"3.3b","mappings":[{"from":7,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:11:18.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:18.583+0000 7feb649cf640 1 -- 172.21.15.92:0/3070674088 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7feb4006ae70 msgr2=0x7feb4006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:18.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:18.583+0000 7feb649cf640 1 --2- 172.21.15.92:0/3070674088 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7feb4006ae70 0x7feb4006d330 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7feb50035000 tx=0x7feb5002e000 comp rx=0 tx=0).stop 2023-11-24T06:11:18.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:18.583+0000 7feb649cf640 1 -- 172.21.15.92:0/3070674088 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feb60143c40 msgr2=0x7feb600c9900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:18.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:18.583+0000 7feb649cf640 1 --2- 172.21.15.92:0/3070674088 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feb60143c40 0x7feb600c9900 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7feb5800c390 tx=0x7feb5800c850 comp rx=0 tx=0).stop 2023-11-24T06:11:18.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:18.583+0000 7feb649cf640 1 -- 172.21.15.92:0/3070674088 shutdown_connections 2023-11-24T06:11:18.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:18.583+0000 7feb649cf640 1 --2- 172.21.15.92:0/3070674088 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7feb4006ae70 0x7feb4006d330 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:18.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:18.583+0000 7feb649cf640 1 --2- 172.21.15.92:0/3070674088 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feb60143c40 0x7feb600c9900 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:18.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:18.583+0000 7feb649cf640 1 --2- 172.21.15.92:0/3070674088 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7feb600cca40 0x7feb6015a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:18.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:18.583+0000 7feb649cf640 1 --2- 172.21.15.92:0/3070674088 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7feb600717d0 0x7feb601f6c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:18.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:18.583+0000 7feb649cf640 1 -- 172.21.15.92:0/3070674088 >> 172.21.15.92:0/3070674088 conn(0x7feb6006a800 msgr2=0x7feb600aee30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:18.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:18.583+0000 7feb649cf640 1 -- 172.21.15.92:0/3070674088 shutdown_connections 2023-11-24T06:11:18.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:18.583+0000 7feb649cf640 1 -- 172.21.15.92:0/3070674088 wait complete. 2023-11-24T06:11:18.604 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T06:10:17.733098+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T06:10:22.926445+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T06:10:24.151288+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T06:11:18.605 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T06:11:18.812 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:18.808+0000 7f4285d0f640 1 Processor -- start 2023-11-24T06:11:18.812 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:18.809+0000 7f4285d0f640 1 -- start start 2023-11-24T06:11:18.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:18.810+0000 7f4285d0f640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f428015da10 0x7f4280072230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:18.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:18.810+0000 7f4285d0f640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4280071190 con 0x7f42800b2bd0 2023-11-24T06:11:18.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:18.810+0000 7f4285d0f640 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4280071470 con 0x7f4280150760 2023-11-24T06:11:18.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:18.810+0000 7f4285d0f640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42800b3790 con 0x7f428015da10 2023-11-24T06:11:18.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:18.810+0000 7f427ffff640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f42800b2bd0 0x7f42800b3080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:54346/0 (socket says 172.21.15.92:54346) 2023-11-24T06:11:18.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:18.810+0000 7f427ffff640 1 -- 172.21.15.92:0/3328509441 learned_addr learned my addr 172.21.15.92:0/3328509441 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-24T06:11:18.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:18.811+0000 7f4277fff640 1 --2- 172.21.15.92:0/3328509441 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f428015da10 0x7f4280072230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:18.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:18.811+0000 7f427effd640 1 -- 172.21.15.92:0/3328509441 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3261508685 0 0) 0x7f4280071190 con 0x7f42800b2bd0 2023-11-24T06:11:18.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:18.811+0000 7f427effd640 1 -- 172.21.15.92:0/3328509441 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f426c003440 con 0x7f42800b2bd0 2023-11-24T06:11:18.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:18.811+0000 7f427effd640 1 -- 172.21.15.92:0/3328509441 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3680289789 0 0) 0x7f426c003440 con 0x7f42800b2bd0 2023-11-24T06:11:18.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:18.811+0000 7f427effd640 1 -- 172.21.15.92:0/3328509441 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f428015da10 msgr2=0x7f4280072230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:18.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:18.811+0000 7f427effd640 1 --2- 172.21.15.92:0/3328509441 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f428015da10 0x7f4280072230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:18.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:18.811+0000 7f427effd640 1 -- 172.21.15.92:0/3328509441 >> v1:172.21.15.118:6789/0 conn(0x7f4280150760 legacy=0x7f428015d300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:18.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:18.811+0000 7f427effd640 1 -- 172.21.15.92:0/3328509441 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4280071750 con 0x7f42800b2bd0 2023-11-24T06:11:18.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:18.811+0000 7f427effd640 1 -- 172.21.15.92:0/3328509441 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2607661350 0 0) 0x7f42780019f0 con 0x7f42800b2bd0 2023-11-24T06:11:18.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:18.811+0000 7f427effd640 1 -- 172.21.15.92:0/3328509441 >> v1:172.21.15.92:6789/0 conn(0x7f42800b2bd0 legacy=0x7f42800b3080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:18.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:18.811+0000 7f427effd640 1 --2- 172.21.15.92:0/3328509441 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f426c003a90 0x7f426c003eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:18.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:18.811+0000 7f427effd640 1 --2- 172.21.15.92:0/3328509441 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f426c008850 0x7f426c00ac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:18.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:18.811+0000 7f427effd640 1 --2- 172.21.15.92:0/3328509441 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f428015da10 0x7f426c00d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:18.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:18.811+0000 7f427effd640 1 -- 172.21.15.92:0/3328509441 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f426c00dbe0 con 0x7f426c008850 2023-11-24T06:11:18.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:18.811+0000 7f427effd640 1 -- 172.21.15.92:0/3328509441 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f426c00dd50 con 0x7f426c003a90 2023-11-24T06:11:18.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:18.811+0000 7f427effd640 1 -- 172.21.15.92:0/3328509441 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f426c00df10 con 0x7f428015da10 2023-11-24T06:11:18.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:18.812+0000 7f4277fff640 1 --2- 172.21.15.92:0/3328509441 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f426c008850 0x7f426c00ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:18.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:18.812+0000 7f4277fff640 1 -- 172.21.15.92:0/3328509441 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f428015da10 msgr2=0x7f426c00d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:18.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:18.812+0000 7f4277fff640 1 --2- 172.21.15.92:0/3328509441 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f428015da10 0x7f426c00d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:18.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:18.812+0000 7f427f7fe640 1 --2- 172.21.15.92:0/3328509441 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f426c003a90 0x7f426c003eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:18.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:18.812+0000 7f4277fff640 1 -- 172.21.15.92:0/3328509441 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f426c003a90 msgr2=0x7f426c003eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:18.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:18.812+0000 7f4277fff640 1 --2- 172.21.15.92:0/3328509441 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f426c003a90 0x7f426c003eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:18.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:18.812+0000 7f4277fff640 1 -- 172.21.15.92:0/3328509441 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f426c003440 con 0x7f426c008850 2023-11-24T06:11:18.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:18.812+0000 7f427f7fe640 1 --2- 172.21.15.92:0/3328509441 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f426c003a90 0x7f426c003eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:11:18.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:18.813+0000 7f4277fff640 1 --2- 172.21.15.92:0/3328509441 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f426c008850 0x7f426c00ac60 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f4268009ac0 tx=0x7f426c0036f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f09008153de347c server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:18.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:18.813+0000 7f427effd640 1 -- 172.21.15.92:0/3328509441 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f426802c710 con 0x7f426c008850 2023-11-24T06:11:18.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:18.813+0000 7f427effd640 1 -- 172.21.15.92:0/3328509441 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f426802c870 con 0x7f426c008850 2023-11-24T06:11:18.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:18.813+0000 7f427effd640 1 -- 172.21.15.92:0/3328509441 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f426802cc00 con 0x7f426c008850 2023-11-24T06:11:18.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:18.814+0000 7f4285d0f640 1 -- 172.21.15.92:0/3328509441 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f426c008850 msgr2=0x7f426c00ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:18.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:18.814+0000 7f4285d0f640 1 --2- 172.21.15.92:0/3328509441 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f426c008850 0x7f426c00ac60 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f4268009ac0 tx=0x7f426c0036f0 comp rx=0 tx=0).stop 2023-11-24T06:11:18.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:18.814+0000 7f427ffff640 1 -- 172.21.15.92:0/3328509441 reap_dead start 2023-11-24T06:11:18.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:18.814+0000 7f4285d0f640 1 -- 172.21.15.92:0/3328509441 shutdown_connections 2023-11-24T06:11:18.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:18.814+0000 7f4285d0f640 1 -- 172.21.15.92:0/3328509441 >> 172.21.15.92:0/3328509441 conn(0x7f428006a800 msgr2=0x7f428006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:18.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:18.814+0000 7f4285d0f640 1 -- 172.21.15.92:0/3328509441 shutdown_connections 2023-11-24T06:11:18.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:18.814+0000 7f4285d0f640 1 -- 172.21.15.92:0/3328509441 wait complete. 2023-11-24T06:11:18.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:18.816+0000 7f4285d0f640 1 Processor -- start 2023-11-24T06:11:18.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:18.816+0000 7f4285d0f640 1 -- start start 2023-11-24T06:11:18.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:18.817+0000 7f4285d0f640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f426c008850 0x7f428014bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:18.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:18.817+0000 7f4285d0f640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f42800aeb30 0x7f42800aef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:18.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:18.817+0000 7f4285d0f640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4280138250 0x7f42801386d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:18.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:18.817+0000 7f4285d0f640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f428014d130 con 0x7f426c008850 2023-11-24T06:11:18.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:18.817+0000 7f4285d0f640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f428014d550 con 0x7f42800aeb30 2023-11-24T06:11:18.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:18.817+0000 7f4285d0f640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42800af4d0 con 0x7f4280138250 2023-11-24T06:11:18.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:18.817+0000 7f427f7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f426c008850 0x7f428014bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:18.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:18.817+0000 7f427f7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f426c008850 0x7f428014bb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:33120/0 (socket says 172.21.15.92:33120) 2023-11-24T06:11:18.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:18.817+0000 7f427f7fe640 1 -- 172.21.15.92:0/1267842357 learned_addr learned my addr 172.21.15.92:0/1267842357 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:11:18.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:18.817+0000 7f427f7fe640 1 -- 172.21.15.92:0/1267842357 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4280138250 msgr2=0x7f42801386d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:18.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:18.817+0000 7f427f7fe640 1 --2- 172.21.15.92:0/1267842357 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4280138250 0x7f42801386d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:18.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:18.817+0000 7f427f7fe640 1 -- 172.21.15.92:0/1267842357 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f42800aeb30 msgr2=0x7f42800aef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:18.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:18.817+0000 7f4277fff640 1 --2- 172.21.15.92:0/1267842357 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f42800aeb30 0x7f42800aef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:18.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:18.817+0000 7f427f7fe640 1 --2- 172.21.15.92:0/1267842357 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f42800aeb30 0x7f42800aef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:18.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:18.817+0000 7f427f7fe640 1 -- 172.21.15.92:0/1267842357 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4280138c10 con 0x7f426c008850 2023-11-24T06:11:18.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:18.818+0000 7f427f7fe640 1 --2- 172.21.15.92:0/1267842357 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f426c008850 0x7f428014bb80 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f427000d7d0 tx=0x7f427000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:18.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:18.818+0000 7f427cff9640 1 -- 172.21.15.92:0/1267842357 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4270005270 con 0x7f426c008850 2023-11-24T06:11:18.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:18.818+0000 7f427cff9640 1 -- 172.21.15.92:0/1267842357 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42700053d0 con 0x7f426c008850 2023-11-24T06:11:18.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:18.818+0000 7f4285d0f640 1 -- 172.21.15.92:0/1267842357 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42800cd560 con 0x7f426c008850 2023-11-24T06:11:18.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:18.818+0000 7f427cff9640 1 -- 172.21.15.92:0/1267842357 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f427000b8f0 con 0x7f426c008850 2023-11-24T06:11:18.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:18.818+0000 7f4285d0f640 1 -- 172.21.15.92:0/1267842357 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f428014c790 con 0x7f426c008850 2023-11-24T06:11:18.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:18.819+0000 7f4285d0f640 1 -- 172.21.15.92:0/1267842357 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f428014d3a0 con 0x7f426c008850 2023-11-24T06:11:18.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:18.820+0000 7f427cff9640 1 -- 172.21.15.92:0/1267842357 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f427000ba50 con 0x7f426c008850 2023-11-24T06:11:18.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:18.821+0000 7f427cff9640 1 --2- 172.21.15.92:0/1267842357 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f426006ae40 0x7f426006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:18.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:18.821+0000 7f427cff9640 1 -- 172.21.15.92:0/1267842357 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5883+0+0 (secure 0 0 0) 0x7f4270089600 con 0x7f426c008850 2023-11-24T06:11:18.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:18.822+0000 7f4277fff640 1 --2- 172.21.15.92:0/1267842357 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f426006ae40 0x7f426006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:18.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:18.822+0000 7f4277fff640 1 --2- 172.21.15.92:0/1267842357 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f426006ae40 0x7f426006d300 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f4268036000 tx=0x7f4268030000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:18.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:18.826+0000 7f427cff9640 1 -- 172.21.15.92:0/1267842357 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4270054600 con 0x7f426c008850 2023-11-24T06:11:19.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:19.001+0000 7f4285d0f640 1 -- 172.21.15.92:0/1267842357 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f42800675d0 con 0x7f426c008850 2023-11-24T06:11:19.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:19.002+0000 7f427cff9640 1 -- 172.21.15.92:0/1267842357 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4270019310 con 0x7f426c008850 2023-11-24T06:11:19.005 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 1 2023-11-24T06:11:19.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:19.004+0000 7f4285d0f640 1 -- 172.21.15.92:0/1267842357 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f426006ae40 msgr2=0x7f426006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:19.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:19.004+0000 7f4285d0f640 1 --2- 172.21.15.92:0/1267842357 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f426006ae40 0x7f426006d300 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f4268036000 tx=0x7f4268030000 comp rx=0 tx=0).stop 2023-11-24T06:11:19.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:19.004+0000 7f4285d0f640 1 -- 172.21.15.92:0/1267842357 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f426c008850 msgr2=0x7f428014bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:19.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:19.004+0000 7f4285d0f640 1 --2- 172.21.15.92:0/1267842357 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f426c008850 0x7f428014bb80 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f427000d7d0 tx=0x7f427000dc90 comp rx=0 tx=0).stop 2023-11-24T06:11:19.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:19.004+0000 7f4285d0f640 1 -- 172.21.15.92:0/1267842357 shutdown_connections 2023-11-24T06:11:19.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:19.004+0000 7f4285d0f640 1 --2- 172.21.15.92:0/1267842357 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f426006ae40 0x7f426006d300 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:19.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:19.004+0000 7f4285d0f640 1 --2- 172.21.15.92:0/1267842357 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4280138250 0x7f42801386d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:19.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:19.004+0000 7f4285d0f640 1 --2- 172.21.15.92:0/1267842357 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f42800aeb30 0x7f42800aef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:19.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:19.004+0000 7f4285d0f640 1 --2- 172.21.15.92:0/1267842357 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f426c008850 0x7f428014bb80 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:19.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:19.004+0000 7f4285d0f640 1 -- 172.21.15.92:0/1267842357 >> 172.21.15.92:0/1267842357 conn(0x7f428006a800 msgr2=0x7f428015c300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:19.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:19.005+0000 7f4285d0f640 1 -- 172.21.15.92:0/1267842357 shutdown_connections 2023-11-24T06:11:19.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:19.005+0000 7f4285d0f640 1 -- 172.21.15.92:0/1267842357 wait complete. 2023-11-24T06:11:19.026 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T06:11:19.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:19.234+0000 7f01588d3640 1 Processor -- start 2023-11-24T06:11:19.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:19.234+0000 7f01588d3640 1 -- start start 2023-11-24T06:11:19.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:19.236+0000 7f01588d3640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f015413e360 0x7f01540cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:19.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:19.236+0000 7f01588d3640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f015415f9c0 0x7f01540cf7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:19.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:19.236+0000 7f01588d3640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f015415c5f0 con 0x7f015413df80 2023-11-24T06:11:19.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:19.236+0000 7f01588d3640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f015406c470 con 0x7f015413e360 2023-11-24T06:11:19.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:19.236+0000 7f01588d3640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f015406c750 con 0x7f015415f9c0 2023-11-24T06:11:19.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:19.236+0000 7f015259c640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f015413df80 0x7f015415bee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:56166/0 (socket says 172.21.15.92:56166) 2023-11-24T06:11:19.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:19.236+0000 7f014bfff640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f015413e360 0x7f01540cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:19.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:19.236+0000 7f015259c640 1 -- 172.21.15.92:0/610592531 learned_addr learned my addr 172.21.15.92:0/610592531 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-24T06:11:19.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:19.237+0000 7f0152d9d640 1 --2- 172.21.15.92:0/610592531 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f015415f9c0 0x7f01540cf7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:19.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:19.237+0000 7f014bfff640 1 -- 172.21.15.92:0/610592531 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f015415f9c0 msgr2=0x7f01540cf7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:19.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:19.237+0000 7f014bfff640 1 --2- 172.21.15.92:0/610592531 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f015415f9c0 0x7f01540cf7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:19.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:19.237+0000 7f014bfff640 1 -- 172.21.15.92:0/610592531 >> v1:172.21.15.92:6789/0 conn(0x7f015413df80 legacy=0x7f015415bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:19.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:19.237+0000 7f014bfff640 1 -- 172.21.15.92:0/610592531 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f015406ca30 con 0x7f015413e360 2023-11-24T06:11:19.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:19.237+0000 7f0152d9d640 1 --2- 172.21.15.92:0/610592531 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f015415f9c0 0x7f01540cf7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:11:19.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:19.238+0000 7f014bfff640 1 --2- 172.21.15.92:0/610592531 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f015413e360 0x7f01540cf290 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f013c0097a0 tx=0x7f013c02b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0ef97403faa7cd4 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:19.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:19.238+0000 7f0151d9b640 1 -- 172.21.15.92:0/610592531 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f013c02d040 con 0x7f015413e360 2023-11-24T06:11:19.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:19.238+0000 7f0151d9b640 1 -- 172.21.15.92:0/610592531 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f013c031810 con 0x7f015413e360 2023-11-24T06:11:19.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:19.238+0000 7f0151d9b640 1 -- 172.21.15.92:0/610592531 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f013c031ba0 con 0x7f015413e360 2023-11-24T06:11:19.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:19.239+0000 7f01588d3640 1 -- 172.21.15.92:0/610592531 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f015413e360 msgr2=0x7f01540cf290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:19.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:19.239+0000 7f01588d3640 1 --2- 172.21.15.92:0/610592531 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f015413e360 0x7f01540cf290 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f013c0097a0 tx=0x7f013c02b8b0 comp rx=0 tx=0).stop 2023-11-24T06:11:19.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:19.239+0000 7f01588d3640 1 -- 172.21.15.92:0/610592531 shutdown_connections 2023-11-24T06:11:19.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:19.239+0000 7f01588d3640 1 --2- 172.21.15.92:0/610592531 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f015415f9c0 0x7f01540cf7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:19.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:19.239+0000 7f01588d3640 1 --2- 172.21.15.92:0/610592531 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f015413e360 0x7f01540cf290 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:19.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:19.239+0000 7f01588d3640 1 -- 172.21.15.92:0/610592531 >> 172.21.15.92:0/610592531 conn(0x7f015406a800 msgr2=0x7f015406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:19.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:19.239+0000 7f01588d3640 1 -- 172.21.15.92:0/610592531 shutdown_connections 2023-11-24T06:11:19.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:19.239+0000 7f01588d3640 1 -- 172.21.15.92:0/610592531 wait complete. 2023-11-24T06:11:19.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:19.240+0000 7f01588d3640 1 Processor -- start 2023-11-24T06:11:19.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:19.241+0000 7f01588d3640 1 -- start start 2023-11-24T06:11:19.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:19.241+0000 7f01588d3640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f015413df80 0x7f01541f2f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:19.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:19.242+0000 7f01588d3640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f015413e360 0x7f015413d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:19.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:19.242+0000 7f01588d3640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f015415f9c0 0x7f01540b3200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:19.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:19.242+0000 7f01588d3640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01541e6130 con 0x7f015413df80 2023-11-24T06:11:19.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:19.242+0000 7f01588d3640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01541e69d0 con 0x7f015413e360 2023-11-24T06:11:19.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:19.242+0000 7f01588d3640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01541e5bb0 con 0x7f015415f9c0 2023-11-24T06:11:19.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:19.242+0000 7f015259c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f015413df80 0x7f01541f2f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:19.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:19.242+0000 7f015259c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f015413df80 0x7f01541f2f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45012/0 (socket says 172.21.15.92:45012) 2023-11-24T06:11:19.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:19.242+0000 7f015259c640 1 -- 172.21.15.92:0/1299066912 learned_addr learned my addr 172.21.15.92:0/1299066912 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:11:19.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:19.242+0000 7f015259c640 1 -- 172.21.15.92:0/1299066912 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f015415f9c0 msgr2=0x7f01540b3200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:19.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:19.242+0000 7f015259c640 1 --2- 172.21.15.92:0/1299066912 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f015415f9c0 0x7f01540b3200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:19.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:19.242+0000 7f015259c640 1 -- 172.21.15.92:0/1299066912 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f015413e360 msgr2=0x7f015413d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:19.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:19.242+0000 7f015259c640 1 --2- 172.21.15.92:0/1299066912 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f015413e360 0x7f015413d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:19.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:19.242+0000 7f015259c640 1 -- 172.21.15.92:0/1299066912 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f013c009400 con 0x7f015413df80 2023-11-24T06:11:19.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:19.243+0000 7f015259c640 1 --2- 172.21.15.92:0/1299066912 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f015413df80 0x7f01541f2f80 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f014400c400 tx=0x7f014400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:19.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:19.243+0000 7f014affd640 1 -- 172.21.15.92:0/1299066912 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0144005ce0 con 0x7f015413df80 2023-11-24T06:11:19.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:19.243+0000 7f014affd640 1 -- 172.21.15.92:0/1299066912 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0144005e40 con 0x7f015413df80 2023-11-24T06:11:19.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:19.243+0000 7f014affd640 1 -- 172.21.15.92:0/1299066912 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0144007310 con 0x7f015413df80 2023-11-24T06:11:19.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:19.243+0000 7f01588d3640 1 -- 172.21.15.92:0/1299066912 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01541e6380 con 0x7f015413df80 2023-11-24T06:11:19.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:19.243+0000 7f01588d3640 1 -- 172.21.15.92:0/1299066912 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01540b3740 con 0x7f015413df80 2023-11-24T06:11:19.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:19.244+0000 7f01588d3640 1 -- 172.21.15.92:0/1299066912 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01540675d0 con 0x7f015413df80 2023-11-24T06:11:19.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:19.245+0000 7f014affd640 1 -- 172.21.15.92:0/1299066912 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f0144007470 con 0x7f015413df80 2023-11-24T06:11:19.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:19.246+0000 7f014affd640 1 --2- 172.21.15.92:0/1299066912 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f013406ae40 0x7f013406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:19.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:19.246+0000 7f014affd640 1 -- 172.21.15.92:0/1299066912 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5883+0+0 (secure 0 0 0) 0x7f0144088c80 con 0x7f015413df80 2023-11-24T06:11:19.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:19.246+0000 7f014bfff640 1 --2- 172.21.15.92:0/1299066912 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f013406ae40 0x7f013406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:19.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:19.247+0000 7f014bfff640 1 --2- 172.21.15.92:0/1299066912 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f013406ae40 0x7f013406d300 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f013c036000 tx=0x7f013c030000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:19.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:19.250+0000 7f014affd640 1 -- 172.21.15.92:0/1299066912 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0144053be0 con 0x7f015413df80 2023-11-24T06:11:19.427 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:19.424+0000 7f01588d3640 1 -- 172.21.15.92:0/1299066912 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f01540cfad0 con 0x7f015413df80 2023-11-24T06:11:19.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:19.425+0000 7f014affd640 1 -- 172.21.15.92:0/1299066912 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7f0144017020 con 0x7f015413df80 2023-11-24T06:11:19.428 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 32 2023-11-24T06:11:19.430 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:19.427+0000 7f01588d3640 1 -- 172.21.15.92:0/1299066912 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f013406ae40 msgr2=0x7f013406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:19.430 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:19.427+0000 7f01588d3640 1 --2- 172.21.15.92:0/1299066912 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f013406ae40 0x7f013406d300 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f013c036000 tx=0x7f013c030000 comp rx=0 tx=0).stop 2023-11-24T06:11:19.431 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:19.427+0000 7f01588d3640 1 -- 172.21.15.92:0/1299066912 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f015413df80 msgr2=0x7f01541f2f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:19.431 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:19.427+0000 7f01588d3640 1 --2- 172.21.15.92:0/1299066912 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f015413df80 0x7f01541f2f80 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f014400c400 tx=0x7f014400c8c0 comp rx=0 tx=0).stop 2023-11-24T06:11:19.431 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:19.427+0000 7f01588d3640 1 -- 172.21.15.92:0/1299066912 shutdown_connections 2023-11-24T06:11:19.431 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:19.427+0000 7f01588d3640 1 --2- 172.21.15.92:0/1299066912 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f013406ae40 0x7f013406d300 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:19.431 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:19.427+0000 7f01588d3640 1 --2- 172.21.15.92:0/1299066912 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f015415f9c0 0x7f01540b3200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:19.431 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:19.427+0000 7f01588d3640 1 --2- 172.21.15.92:0/1299066912 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f015413e360 0x7f015413d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:19.431 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:19.427+0000 7f01588d3640 1 --2- 172.21.15.92:0/1299066912 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f015413df80 0x7f01541f2f80 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:19.431 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:19.427+0000 7f01588d3640 1 -- 172.21.15.92:0/1299066912 >> 172.21.15.92:0/1299066912 conn(0x7f015406a800 msgr2=0x7f01540aeb50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:19.431 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:19.427+0000 7f01588d3640 1 -- 172.21.15.92:0/1299066912 shutdown_connections 2023-11-24T06:11:19.431 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:19.427+0000 7f01588d3640 1 -- 172.21.15.92:0/1299066912 wait complete. 2023-11-24T06:11:19.447 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T06:11:19.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:19.657+0000 7f3e019f1640 1 Processor -- start 2023-11-24T06:11:19.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:19.657+0000 7f3e019f1640 1 -- start start 2023-11-24T06:11:19.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:19.659+0000 7f3e019f1640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3dfc143ae0 0x7f3dfc0cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:19.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:19.659+0000 7f3e019f1640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f3dfc0cf7d0 0x7f3dfc15dca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:19.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:19.659+0000 7f3e019f1640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3dfc0cfc30 con 0x7f3dfc15e1e0 2023-11-24T06:11:19.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:19.659+0000 7f3e019f1640 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3dfc0b1ee0 con 0x7f3dfc0cf7d0 2023-11-24T06:11:19.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:19.659+0000 7f3e019f1640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3dfc0b2780 con 0x7f3dfc143ae0 2023-11-24T06:11:19.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:19.660+0000 7f3dfb7fe640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f3dfc15e1e0 0x7f3dfc15a670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:56170/0 (socket says 172.21.15.92:56170) 2023-11-24T06:11:19.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:19.660+0000 7f3dfb7fe640 1 -- 172.21.15.92:0/526892220 learned_addr learned my addr 172.21.15.92:0/526892220 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-24T06:11:19.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:19.660+0000 7f3df27fc640 1 --2- 172.21.15.92:0/526892220 >> v2:172.21.15.118:3300/0 conn(0x7f3dfc0cf7d0 0x7f3dfc15dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:19.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:19.660+0000 7f3dfaffd640 1 --2- 172.21.15.92:0/526892220 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3dfc143ae0 0x7f3dfc0cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:19.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:19.660+0000 7f3dfa7fc640 1 -- 172.21.15.92:0/526892220 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3605301327 0 0) 0x7f3dfc0cfc30 con 0x7f3dfc15e1e0 2023-11-24T06:11:19.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:19.660+0000 7f3dfa7fc640 1 -- 172.21.15.92:0/526892220 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3de8003520 con 0x7f3dfc15e1e0 2023-11-24T06:11:19.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:19.660+0000 7f3dfa7fc640 1 -- 172.21.15.92:0/526892220 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3603632094 0 0) 0x7f3de8003520 con 0x7f3dfc15e1e0 2023-11-24T06:11:19.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:19.660+0000 7f3dfa7fc640 1 -- 172.21.15.92:0/526892220 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3dfc143ae0 msgr2=0x7f3dfc0cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:19.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:19.660+0000 7f3dfa7fc640 1 --2- 172.21.15.92:0/526892220 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3dfc143ae0 0x7f3dfc0cf290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:19.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:19.660+0000 7f3dfa7fc640 1 -- 172.21.15.92:0/526892220 >> v2:172.21.15.118:3300/0 conn(0x7f3dfc0cf7d0 msgr2=0x7f3dfc15dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:19.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:19.660+0000 7f3dfa7fc640 1 --2- 172.21.15.92:0/526892220 >> v2:172.21.15.118:3300/0 conn(0x7f3dfc0cf7d0 0x7f3dfc15dca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:19.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:19.660+0000 7f3dfa7fc640 1 -- 172.21.15.92:0/526892220 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dfc15e990 con 0x7f3dfc15e1e0 2023-11-24T06:11:19.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:19.661+0000 7f3dfa7fc640 1 -- 172.21.15.92:0/526892220 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2607661350 0 0) 0x7f3df40019f0 con 0x7f3dfc15e1e0 2023-11-24T06:11:19.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:19.661+0000 7f3dfa7fc640 1 -- 172.21.15.92:0/526892220 >> v1:172.21.15.92:6789/0 conn(0x7f3dfc15e1e0 legacy=0x7f3dfc15a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:19.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:19.661+0000 7f3dfa7fc640 1 --2- 172.21.15.92:0/526892220 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3de8003970 0x7f3de8003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:19.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:19.661+0000 7f3dfa7fc640 1 --2- 172.21.15.92:0/526892220 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3dfc143ae0 0x7f3de80047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:19.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:19.661+0000 7f3dfa7fc640 1 --2- 172.21.15.92:0/526892220 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3de8004d00 0x7f3de8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:19.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:19.661+0000 7f3dfa7fc640 1 -- 172.21.15.92:0/526892220 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dfc0b1ee0 con 0x7f3de8004d00 2023-11-24T06:11:19.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:19.661+0000 7f3dfa7fc640 1 -- 172.21.15.92:0/526892220 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dfc0b2780 con 0x7f3de8003970 2023-11-24T06:11:19.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:19.661+0000 7f3dfa7fc640 1 -- 172.21.15.92:0/526892220 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3de800db90 con 0x7f3dfc143ae0 2023-11-24T06:11:19.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:19.661+0000 7f3dfb7fe640 1 --2- 172.21.15.92:0/526892220 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3de8004d00 0x7f3de8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:19.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:19.661+0000 7f3dfaffd640 1 --2- 172.21.15.92:0/526892220 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3de8003970 0x7f3de8003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:19.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:19.661+0000 7f3dfb7fe640 1 -- 172.21.15.92:0/526892220 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3dfc143ae0 msgr2=0x7f3de80047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:19.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:19.661+0000 7f3dfb7fe640 1 --2- 172.21.15.92:0/526892220 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3dfc143ae0 0x7f3de80047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:19.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:19.661+0000 7f3dfb7fe640 1 -- 172.21.15.92:0/526892220 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3de8003970 msgr2=0x7f3de8003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:19.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:19.661+0000 7f3dfb7fe640 1 --2- 172.21.15.92:0/526892220 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3de8003970 0x7f3de8003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:19.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:19.661+0000 7f3dfb7fe640 1 -- 172.21.15.92:0/526892220 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3de8003520 con 0x7f3de8004d00 2023-11-24T06:11:19.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:19.662+0000 7f3dfb7fe640 1 --2- 172.21.15.92:0/526892220 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3de8004d00 0x7f3de8005180 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f3df400a830 tx=0x7f3de80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4aa13dc7953a4551 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:19.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:19.663+0000 7f3dfa7fc640 1 -- 172.21.15.92:0/526892220 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3df4034630 con 0x7f3de8004d00 2023-11-24T06:11:19.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:19.663+0000 7f3dfa7fc640 1 -- 172.21.15.92:0/526892220 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3df402f070 con 0x7f3de8004d00 2023-11-24T06:11:19.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:19.663+0000 7f3dfa7fc640 1 -- 172.21.15.92:0/526892220 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3df4034a10 con 0x7f3de8004d00 2023-11-24T06:11:19.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:19.663+0000 7f3e019f1640 1 -- 172.21.15.92:0/526892220 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3de8004d00 msgr2=0x7f3de8005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:19.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:19.663+0000 7f3e019f1640 1 --2- 172.21.15.92:0/526892220 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3de8004d00 0x7f3de8005180 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f3df400a830 tx=0x7f3de80057f0 comp rx=0 tx=0).stop 2023-11-24T06:11:19.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:19.663+0000 7f3dfb7fe640 1 -- 172.21.15.92:0/526892220 reap_dead start 2023-11-24T06:11:19.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:19.663+0000 7f3e019f1640 1 -- 172.21.15.92:0/526892220 shutdown_connections 2023-11-24T06:11:19.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:19.663+0000 7f3e019f1640 1 -- 172.21.15.92:0/526892220 >> 172.21.15.92:0/526892220 conn(0x7f3dfc06a800 msgr2=0x7f3dfc06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:19.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:19.663+0000 7f3e019f1640 1 -- 172.21.15.92:0/526892220 shutdown_connections 2023-11-24T06:11:19.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:19.663+0000 7f3e019f1640 1 -- 172.21.15.92:0/526892220 wait complete. 2023-11-24T06:11:19.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:19.665+0000 7f3e019f1640 1 Processor -- start 2023-11-24T06:11:19.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:19.665+0000 7f3e019f1640 1 -- start start 2023-11-24T06:11:19.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:19.666+0000 7f3e019f1640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3de8004d00 0x7f3dfc13a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:19.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:19.666+0000 7f3e019f1640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3dfc1595a0 0x7f3dfc158dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:19.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:19.666+0000 7f3e019f1640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3dfc0cd6a0 0x7f3dfc0cdb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:19.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:19.666+0000 7f3e019f1640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dfc0ce060 con 0x7f3dfc0cd6a0 2023-11-24T06:11:19.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:19.666+0000 7f3e019f1640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dfc159310 con 0x7f3dfc1595a0 2023-11-24T06:11:19.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:19.666+0000 7f3e019f1640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3dfc15d5b0 con 0x7f3de8004d00 2023-11-24T06:11:19.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:19.666+0000 7f3dfb7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3dfc0cd6a0 0x7f3dfc0cdb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:19.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:19.666+0000 7f3dfb7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3dfc0cd6a0 0x7f3dfc0cdb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45034/0 (socket says 172.21.15.92:45034) 2023-11-24T06:11:19.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:19.666+0000 7f3dfb7fe640 1 -- 172.21.15.92:0/2312156610 learned_addr learned my addr 172.21.15.92:0/2312156610 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:11:19.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:19.667+0000 7f3df27fc640 1 --2- 172.21.15.92:0/2312156610 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3dfc1595a0 0x7f3dfc158dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:19.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:19.667+0000 7f3dfaffd640 1 --2- 172.21.15.92:0/2312156610 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3de8004d00 0x7f3dfc13a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:19.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:19.667+0000 7f3dfb7fe640 1 -- 172.21.15.92:0/2312156610 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3de8004d00 msgr2=0x7f3dfc13a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:19.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:19.667+0000 7f3dfb7fe640 1 --2- 172.21.15.92:0/2312156610 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3de8004d00 0x7f3dfc13a3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:19.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:19.667+0000 7f3dfb7fe640 1 -- 172.21.15.92:0/2312156610 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3dfc1595a0 msgr2=0x7f3dfc158dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:19.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:19.667+0000 7f3dfb7fe640 1 --2- 172.21.15.92:0/2312156610 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3dfc1595a0 0x7f3dfc158dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:19.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:19.667+0000 7f3dfb7fe640 1 -- 172.21.15.92:0/2312156610 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dfc15d2d0 con 0x7f3dfc0cd6a0 2023-11-24T06:11:19.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:19.667+0000 7f3df27fc640 1 --2- 172.21.15.92:0/2312156610 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3dfc1595a0 0x7f3dfc158dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:11:19.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:19.667+0000 7f3dfb7fe640 1 --2- 172.21.15.92:0/2312156610 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3dfc0cd6a0 0x7f3dfc0cdb20 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f3df400a830 tx=0x7f3dfc0b26f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:19.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:19.667+0000 7f3df8ff9640 1 -- 172.21.15.92:0/2312156610 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3df4034630 con 0x7f3dfc0cd6a0 2023-11-24T06:11:19.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:19.667+0000 7f3df8ff9640 1 -- 172.21.15.92:0/2312156610 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3df4034790 con 0x7f3dfc0cd6a0 2023-11-24T06:11:19.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:19.667+0000 7f3df8ff9640 1 -- 172.21.15.92:0/2312156610 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3df4034b00 con 0x7f3dfc0cd6a0 2023-11-24T06:11:19.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:19.667+0000 7f3e019f1640 1 -- 172.21.15.92:0/2312156610 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3dfc15a610 con 0x7f3dfc0cd6a0 2023-11-24T06:11:19.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:19.668+0000 7f3e019f1640 1 -- 172.21.15.92:0/2312156610 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3dfc15abd0 con 0x7f3dfc0cd6a0 2023-11-24T06:11:19.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:19.669+0000 7f3e019f1640 1 -- 172.21.15.92:0/2312156610 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3dfc0675d0 con 0x7f3dfc0cd6a0 2023-11-24T06:11:19.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:19.670+0000 7f3df8ff9640 1 -- 172.21.15.92:0/2312156610 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f3df4031030 con 0x7f3dfc0cd6a0 2023-11-24T06:11:19.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:19.670+0000 7f3df8ff9640 1 --2- 172.21.15.92:0/2312156610 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f3ddc06ae70 0x7f3ddc06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:19.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:19.670+0000 7f3df8ff9640 1 -- 172.21.15.92:0/2312156610 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5883+0+0 (secure 0 0 0) 0x7f3df40ab560 con 0x7f3dfc0cd6a0 2023-11-24T06:11:19.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:19.671+0000 7f3dfaffd640 1 --2- 172.21.15.92:0/2312156610 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f3ddc06ae70 0x7f3ddc06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:19.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:19.671+0000 7f3dfaffd640 1 --2- 172.21.15.92:0/2312156610 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f3ddc06ae70 0x7f3ddc06d330 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f3dfc140fa0 tx=0x7f3dec008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:19.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:19.675+0000 7f3df8ff9640 1 -- 172.21.15.92:0/2312156610 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3df4076670 con 0x7f3dfc0cd6a0 2023-11-24T06:11:19.870 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:19.866+0000 7f3e019f1640 1 -- 172.21.15.92:0/2312156610 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f3dfc0675d0 con 0x7f3dfc0cd6a0 2023-11-24T06:11:19.871 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:19.867+0000 7f3df8ff9640 1 -- 172.21.15.92:0/2312156610 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7f3df4001fc0 con 0x7f3dfc0cd6a0 2023-11-24T06:11:19.871 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 64 2023-11-24T06:11:19.873 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:19.870+0000 7f3e019f1640 1 -- 172.21.15.92:0/2312156610 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f3ddc06ae70 msgr2=0x7f3ddc06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:19.874 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:19.870+0000 7f3e019f1640 1 --2- 172.21.15.92:0/2312156610 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f3ddc06ae70 0x7f3ddc06d330 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f3dfc140fa0 tx=0x7f3dec008040 comp rx=0 tx=0).stop 2023-11-24T06:11:19.874 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:19.870+0000 7f3e019f1640 1 -- 172.21.15.92:0/2312156610 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3dfc0cd6a0 msgr2=0x7f3dfc0cdb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:19.874 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:19.870+0000 7f3e019f1640 1 --2- 172.21.15.92:0/2312156610 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3dfc0cd6a0 0x7f3dfc0cdb20 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f3df400a830 tx=0x7f3dfc0b26f0 comp rx=0 tx=0).stop 2023-11-24T06:11:19.874 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:19.870+0000 7f3e019f1640 1 -- 172.21.15.92:0/2312156610 shutdown_connections 2023-11-24T06:11:19.874 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:19.870+0000 7f3e019f1640 1 --2- 172.21.15.92:0/2312156610 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f3ddc06ae70 0x7f3ddc06d330 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:19.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:19.870+0000 7f3e019f1640 1 --2- 172.21.15.92:0/2312156610 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3dfc0cd6a0 0x7f3dfc0cdb20 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:19.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:19.870+0000 7f3e019f1640 1 --2- 172.21.15.92:0/2312156610 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3dfc1595a0 0x7f3dfc158dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:19.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:19.870+0000 7f3e019f1640 1 --2- 172.21.15.92:0/2312156610 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3de8004d00 0x7f3dfc13a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:19.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:19.870+0000 7f3e019f1640 1 -- 172.21.15.92:0/2312156610 >> 172.21.15.92:0/2312156610 conn(0x7f3dfc06a800 msgr2=0x7f3dfc0b0bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:19.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:19.870+0000 7f3e019f1640 1 -- 172.21.15.92:0/2312156610 shutdown_connections 2023-11-24T06:11:19.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:19.870+0000 7f3e019f1640 1 -- 172.21.15.92:0/2312156610 wait complete. 2023-11-24T06:11:19.891 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-24T06:11:20.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:20.103+0000 7fa31c7c9640 1 Processor -- start 2023-11-24T06:11:20.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:20.103+0000 7fa31c7c9640 1 -- start start 2023-11-24T06:11:20.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:20.105+0000 7fa31c7c9640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fa3140cd240 0x7fa3140b3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:20.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:20.105+0000 7fa31c7c9640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fa31406c6b0 0x7fa314099a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:20.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:20.105+0000 7fa31c7c9640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa31406cea0 con 0x7fa314099fa0 2023-11-24T06:11:20.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:20.105+0000 7fa31c7c9640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa31409a410 con 0x7fa31406c6b0 2023-11-24T06:11:20.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:20.105+0000 7fa31c7c9640 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa31406cbc0 con 0x7fa3140cd240 2023-11-24T06:11:20.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:20.105+0000 7fa31ad66640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fa314099fa0 0x7fa31406b4f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:56186/0 (socket says 172.21.15.92:56186) 2023-11-24T06:11:20.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:20.105+0000 7fa31ad66640 1 -- 172.21.15.92:0/136557779 learned_addr learned my addr 172.21.15.92:0/136557779 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-24T06:11:20.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:20.105+0000 7fa319d64640 1 --2- 172.21.15.92:0/136557779 >> v2:172.21.15.92:3300/0 conn(0x7fa31406c6b0 0x7fa314099a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:20.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:20.105+0000 7fa31a565640 1 --2- 172.21.15.92:0/136557779 >> v2:172.21.15.118:3300/0 conn(0x7fa3140cd240 0x7fa3140b3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:20.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:20.106+0000 7fa319563640 1 -- 172.21.15.92:0/136557779 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 847454234 0 0) 0x7fa31406cea0 con 0x7fa314099fa0 2023-11-24T06:11:20.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:20.106+0000 7fa319563640 1 -- 172.21.15.92:0/136557779 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa300003520 con 0x7fa314099fa0 2023-11-24T06:11:20.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:20.106+0000 7fa319d64640 1 -- 172.21.15.92:0/136557779 >> v2:172.21.15.118:3300/0 conn(0x7fa3140cd240 msgr2=0x7fa3140b3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:20.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:20.106+0000 7fa319d64640 1 --2- 172.21.15.92:0/136557779 >> v2:172.21.15.118:3300/0 conn(0x7fa3140cd240 0x7fa3140b3820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:20.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:20.106+0000 7fa319d64640 1 -- 172.21.15.92:0/136557779 >> v1:172.21.15.92:6789/0 conn(0x7fa314099fa0 legacy=0x7fa31406b4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:20.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:20.106+0000 7fa319d64640 1 -- 172.21.15.92:0/136557779 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa31406f350 con 0x7fa31406c6b0 2023-11-24T06:11:20.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:20.107+0000 7fa319d64640 1 --2- 172.21.15.92:0/136557779 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa31406c6b0 0x7fa314099a60 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fa314070a10 tx=0x7fa30402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a74a03a53b5a3cb server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:20.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:20.107+0000 7fa319563640 1 -- 172.21.15.92:0/136557779 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa304031830 con 0x7fa31406c6b0 2023-11-24T06:11:20.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:20.107+0000 7fa319563640 1 -- 172.21.15.92:0/136557779 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa304031990 con 0x7fa31406c6b0 2023-11-24T06:11:20.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:20.107+0000 7fa319563640 1 -- 172.21.15.92:0/136557779 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa304031d20 con 0x7fa31406c6b0 2023-11-24T06:11:20.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:20.108+0000 7fa31c7c9640 1 -- 172.21.15.92:0/136557779 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa31406c6b0 msgr2=0x7fa314099a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:20.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:20.108+0000 7fa31c7c9640 1 --2- 172.21.15.92:0/136557779 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa31406c6b0 0x7fa314099a60 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fa314070a10 tx=0x7fa30402b8d0 comp rx=0 tx=0).stop 2023-11-24T06:11:20.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:20.108+0000 7fa31c7c9640 1 -- 172.21.15.92:0/136557779 shutdown_connections 2023-11-24T06:11:20.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:20.108+0000 7fa31c7c9640 1 --2- 172.21.15.92:0/136557779 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa31406c6b0 0x7fa314099a60 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:20.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:20.108+0000 7fa31c7c9640 1 --2- 172.21.15.92:0/136557779 >> v2:172.21.15.118:3300/0 conn(0x7fa3140cd240 0x7fa3140b3820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:20.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:20.108+0000 7fa31c7c9640 1 -- 172.21.15.92:0/136557779 >> 172.21.15.92:0/136557779 conn(0x7fa3140b2d00 msgr2=0x7fa3140b3110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:20.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:20.108+0000 7fa31c7c9640 1 -- 172.21.15.92:0/136557779 shutdown_connections 2023-11-24T06:11:20.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:20.108+0000 7fa31c7c9640 1 -- 172.21.15.92:0/136557779 wait complete. 2023-11-24T06:11:20.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:20.110+0000 7fa31c7c9640 1 Processor -- start 2023-11-24T06:11:20.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:20.110+0000 7fa31c7c9640 1 -- start start 2023-11-24T06:11:20.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:20.111+0000 7fa31c7c9640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa31406c6b0 0x7fa3140c79d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:20.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:20.111+0000 7fa31c7c9640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa314099fa0 0x7fa314155fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:20.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:20.111+0000 7fa31c7c9640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa3140cd240 0x7fa3140b5f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:20.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:20.111+0000 7fa31c7c9640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3140b5240 con 0x7fa3140cd240 2023-11-24T06:11:20.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:20.111+0000 7fa31c7c9640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3140b6e70 con 0x7fa31406c6b0 2023-11-24T06:11:20.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:20.111+0000 7fa31c7c9640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3140b5450 con 0x7fa314099fa0 2023-11-24T06:11:20.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:20.111+0000 7fa31ad66640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa3140cd240 0x7fa3140b5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:20.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:20.111+0000 7fa31ad66640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa3140cd240 0x7fa3140b5f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45040/0 (socket says 172.21.15.92:45040) 2023-11-24T06:11:20.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:20.111+0000 7fa31ad66640 1 -- 172.21.15.92:0/3959804855 learned_addr learned my addr 172.21.15.92:0/3959804855 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:11:20.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:20.111+0000 7fa31a565640 1 --2- 172.21.15.92:0/3959804855 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa31406c6b0 0x7fa3140c79d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:20.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:20.111+0000 7fa31ad66640 1 -- 172.21.15.92:0/3959804855 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa314099fa0 msgr2=0x7fa314155fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:20.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:20.111+0000 7fa319d64640 1 --2- 172.21.15.92:0/3959804855 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa314099fa0 0x7fa314155fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:20.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:20.111+0000 7fa31ad66640 1 --2- 172.21.15.92:0/3959804855 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa314099fa0 0x7fa314155fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:20.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:20.111+0000 7fa31ad66640 1 -- 172.21.15.92:0/3959804855 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa31406c6b0 msgr2=0x7fa3140c79d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:20.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:20.111+0000 7fa31ad66640 1 --2- 172.21.15.92:0/3959804855 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa31406c6b0 0x7fa3140c79d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:20.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:20.112+0000 7fa31ad66640 1 -- 172.21.15.92:0/3959804855 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa304009400 con 0x7fa3140cd240 2023-11-24T06:11:20.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:20.112+0000 7fa31a565640 1 --2- 172.21.15.92:0/3959804855 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa31406c6b0 0x7fa3140c79d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:11:20.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:20.112+0000 7fa31ad66640 1 --2- 172.21.15.92:0/3959804855 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa3140cd240 0x7fa3140b5f30 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fa30c00cb40 tx=0x7fa30c00a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:20.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:20.112+0000 7fa30affd640 1 -- 172.21.15.92:0/3959804855 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa30c00aca0 con 0x7fa3140cd240 2023-11-24T06:11:20.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:20.112+0000 7fa30affd640 1 -- 172.21.15.92:0/3959804855 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa30c00ae00 con 0x7fa3140cd240 2023-11-24T06:11:20.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:20.112+0000 7fa30affd640 1 -- 172.21.15.92:0/3959804855 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa30c012620 con 0x7fa3140cd240 2023-11-24T06:11:20.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:20.112+0000 7fa31c7c9640 1 -- 172.21.15.92:0/3959804855 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3140b5730 con 0x7fa3140cd240 2023-11-24T06:11:20.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:20.112+0000 7fa31c7c9640 1 -- 172.21.15.92:0/3959804855 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa314156510 con 0x7fa3140cd240 2023-11-24T06:11:20.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:20.113+0000 7fa31c7c9640 1 -- 172.21.15.92:0/3959804855 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3140b5c50 con 0x7fa3140cd240 2023-11-24T06:11:20.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:20.114+0000 7fa30affd640 1 -- 172.21.15.92:0/3959804855 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa30c007070 con 0x7fa3140cd240 2023-11-24T06:11:20.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:20.115+0000 7fa30affd640 1 --2- 172.21.15.92:0/3959804855 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fa2f806ae40 0x7fa2f806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:20.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:20.115+0000 7fa30affd640 1 -- 172.21.15.92:0/3959804855 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5883+0+0 (secure 0 0 0) 0x7fa30c089680 con 0x7fa3140cd240 2023-11-24T06:11:20.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:20.115+0000 7fa31a565640 1 --2- 172.21.15.92:0/3959804855 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fa2f806ae40 0x7fa2f806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:20.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:20.116+0000 7fa31a565640 1 --2- 172.21.15.92:0/3959804855 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fa2f806ae40 0x7fa2f806d300 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fa310009ac0 tx=0x7fa310008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:20.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:20.120+0000 7fa30affd640 1 -- 172.21.15.92:0/3959804855 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa3140b5c50 con 0x7fa3140cd240 2023-11-24T06:11:20.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:20.322+0000 7fa31c7c9640 1 -- 172.21.15.92:0/3959804855 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fa3140b5c50 con 0x7fa3140cd240 2023-11-24T06:11:20.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:20.322+0000 7fa30affd640 1 -- 172.21.15.92:0/3959804855 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) v1 ==== 80+0+0 (secure 0 0 0) 0x7fa3140b5c50 con 0x7fa3140cd240 2023-11-24T06:11:20.325 INFO:teuthology.orchestra.run.smithi092.stderr:listed 0 entries 2023-11-24T06:11:20.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:20.325+0000 7fa31c7c9640 1 -- 172.21.15.92:0/3959804855 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fa2f806ae40 msgr2=0x7fa2f806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:20.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:20.325+0000 7fa31c7c9640 1 --2- 172.21.15.92:0/3959804855 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fa2f806ae40 0x7fa2f806d300 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fa310009ac0 tx=0x7fa310008040 comp rx=0 tx=0).stop 2023-11-24T06:11:20.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:20.325+0000 7fa31c7c9640 1 -- 172.21.15.92:0/3959804855 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa3140cd240 msgr2=0x7fa3140b5f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:20.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:20.325+0000 7fa31c7c9640 1 --2- 172.21.15.92:0/3959804855 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa3140cd240 0x7fa3140b5f30 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fa30c00cb40 tx=0x7fa30c00a060 comp rx=0 tx=0).stop 2023-11-24T06:11:20.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:20.325+0000 7fa31c7c9640 1 -- 172.21.15.92:0/3959804855 shutdown_connections 2023-11-24T06:11:20.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:20.325+0000 7fa31c7c9640 1 --2- 172.21.15.92:0/3959804855 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fa2f806ae40 0x7fa2f806d300 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:20.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:20.325+0000 7fa31c7c9640 1 --2- 172.21.15.92:0/3959804855 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa3140cd240 0x7fa3140b5f30 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:20.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:20.325+0000 7fa31c7c9640 1 --2- 172.21.15.92:0/3959804855 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa314099fa0 0x7fa314155fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:20.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:20.325+0000 7fa31c7c9640 1 --2- 172.21.15.92:0/3959804855 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa31406c6b0 0x7fa3140c79d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:20.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:20.325+0000 7fa31c7c9640 1 -- 172.21.15.92:0/3959804855 >> 172.21.15.92:0/3959804855 conn(0x7fa3140b2d00 msgr2=0x7fa3140d0170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:20.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:20.325+0000 7fa31c7c9640 1 -- 172.21.15.92:0/3959804855 shutdown_connections 2023-11-24T06:11:20.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:20.325+0000 7fa31c7c9640 1 -- 172.21.15.92:0/3959804855 wait complete. 2023-11-24T06:11:20.345 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-11-24T06:11:20.346 DEBUG:teuthology.orchestra.run.smithi092:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:11:20.397 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:11:20.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:20.604+0000 7f38ce2c9640 1 Processor -- start 2023-11-24T06:11:20.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:20.606+0000 7f38ce2c9640 1 -- start start 2023-11-24T06:11:20.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:20.607+0000 7f38ce2c9640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f38c8143ae0 0x7f38c80cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:20.610 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:20.607+0000 7f38ce2c9640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f38c80cf7d0 0x7f38c815dca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:20.610 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:20.607+0000 7f38ce2c9640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38c80cfc30 con 0x7f38c815e1e0 2023-11-24T06:11:20.610 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:20.607+0000 7f38ce2c9640 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38c80b1ee0 con 0x7f38c80cf7d0 2023-11-24T06:11:20.610 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:20.607+0000 7f38ce2c9640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38c80b2780 con 0x7f38c8143ae0 2023-11-24T06:11:20.610 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:20.607+0000 7f38cc866640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f38c815e1e0 0x7f38c815a670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:56194/0 (socket says 172.21.15.92:56194) 2023-11-24T06:11:20.611 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:20.607+0000 7f38cc866640 1 -- 172.21.15.92:0/1416967595 learned_addr learned my addr 172.21.15.92:0/1416967595 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-24T06:11:20.613 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:20.607+0000 7f38bffff640 1 --2- 172.21.15.92:0/1416967595 >> v2:172.21.15.118:3300/0 conn(0x7f38c80cf7d0 0x7f38c815dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:20.613 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:20.607+0000 7f38c7fff640 1 --2- 172.21.15.92:0/1416967595 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f38c8143ae0 0x7f38c80cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:20.613 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:20.607+0000 7f38c77fe640 1 -- 172.21.15.92:0/1416967595 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3948556966 0 0) 0x7f38c80cfc30 con 0x7f38c815e1e0 2023-11-24T06:11:20.613 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:20.607+0000 7f38c77fe640 1 -- 172.21.15.92:0/1416967595 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f38b4003520 con 0x7f38c815e1e0 2023-11-24T06:11:20.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:20.607+0000 7f38c77fe640 1 -- 172.21.15.92:0/1416967595 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1023174339 0 0) 0x7f38b4003520 con 0x7f38c815e1e0 2023-11-24T06:11:20.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:20.607+0000 7f38c77fe640 1 -- 172.21.15.92:0/1416967595 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f38c8143ae0 msgr2=0x7f38c80cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:20.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:20.607+0000 7f38c77fe640 1 --2- 172.21.15.92:0/1416967595 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f38c8143ae0 0x7f38c80cf290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:20.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:20.607+0000 7f38c77fe640 1 -- 172.21.15.92:0/1416967595 >> v2:172.21.15.118:3300/0 conn(0x7f38c80cf7d0 msgr2=0x7f38c815dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:20.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:20.607+0000 7f38c77fe640 1 --2- 172.21.15.92:0/1416967595 >> v2:172.21.15.118:3300/0 conn(0x7f38c80cf7d0 0x7f38c815dca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:20.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:20.607+0000 7f38c77fe640 1 -- 172.21.15.92:0/1416967595 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38c815e990 con 0x7f38c815e1e0 2023-11-24T06:11:20.615 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:20.608+0000 7f38c77fe640 1 -- 172.21.15.92:0/1416967595 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2607661350 0 0) 0x7f38c00019f0 con 0x7f38c815e1e0 2023-11-24T06:11:20.615 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:20.608+0000 7f38c7fff640 1 --2- 172.21.15.92:0/1416967595 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f38c8143ae0 0x7f38c80cf290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:11:20.615 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:20.608+0000 7f38c77fe640 1 -- 172.21.15.92:0/1416967595 >> v1:172.21.15.92:6789/0 conn(0x7f38c815e1e0 legacy=0x7f38c815a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:20.615 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:20.608+0000 7f38c77fe640 1 --2- 172.21.15.92:0/1416967595 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f38b4003970 0x7f38b4003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:20.615 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:20.608+0000 7f38c77fe640 1 --2- 172.21.15.92:0/1416967595 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f38c8143ae0 0x7f38b40047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:20.615 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:20.608+0000 7f38c77fe640 1 --2- 172.21.15.92:0/1416967595 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f38b4004d00 0x7f38b4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:20.615 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:20.608+0000 7f38c77fe640 1 -- 172.21.15.92:0/1416967595 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38c80b1ee0 con 0x7f38b4004d00 2023-11-24T06:11:20.616 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:20.608+0000 7f38c77fe640 1 -- 172.21.15.92:0/1416967595 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38c80b2780 con 0x7f38b4003970 2023-11-24T06:11:20.616 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:20.608+0000 7f38c77fe640 1 -- 172.21.15.92:0/1416967595 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38b400b980 con 0x7f38c8143ae0 2023-11-24T06:11:20.616 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:20.608+0000 7f38cc866640 1 --2- 172.21.15.92:0/1416967595 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f38b4004d00 0x7f38b4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:20.616 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:20.608+0000 7f38c7fff640 1 --2- 172.21.15.92:0/1416967595 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f38b4003970 0x7f38b4003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:20.616 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:20.608+0000 7f38bffff640 1 --2- 172.21.15.92:0/1416967595 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f38c8143ae0 0x7f38b40047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:20.616 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:20.608+0000 7f38cc866640 1 -- 172.21.15.92:0/1416967595 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f38c8143ae0 msgr2=0x7f38b40047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:20.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:20.608+0000 7f38cc866640 1 --2- 172.21.15.92:0/1416967595 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f38c8143ae0 0x7f38b40047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:20.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:20.608+0000 7f38cc866640 1 -- 172.21.15.92:0/1416967595 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f38b4003970 msgr2=0x7f38b4003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:20.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:20.608+0000 7f38cc866640 1 --2- 172.21.15.92:0/1416967595 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f38b4003970 0x7f38b4003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:20.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:20.608+0000 7f38cc866640 1 -- 172.21.15.92:0/1416967595 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38b4003520 con 0x7f38b4004d00 2023-11-24T06:11:20.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:20.609+0000 7f38cc866640 1 --2- 172.21.15.92:0/1416967595 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f38b4004d00 0x7f38b4005180 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f38c000a830 tx=0x7f38b40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f02706376379fb04 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:20.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:20.610+0000 7f38c77fe640 1 -- 172.21.15.92:0/1416967595 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38c0034630 con 0x7f38b4004d00 2023-11-24T06:11:20.618 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:20.610+0000 7f38c77fe640 1 -- 172.21.15.92:0/1416967595 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38c002f070 con 0x7f38b4004d00 2023-11-24T06:11:20.618 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:20.610+0000 7f38c77fe640 1 -- 172.21.15.92:0/1416967595 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38c00349f0 con 0x7f38b4004d00 2023-11-24T06:11:20.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:20.610+0000 7f38ce2c9640 1 -- 172.21.15.92:0/1416967595 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f38b4004d00 msgr2=0x7f38b4005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:20.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:20.610+0000 7f38ce2c9640 1 --2- 172.21.15.92:0/1416967595 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f38b4004d00 0x7f38b4005180 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f38c000a830 tx=0x7f38b40057f0 comp rx=0 tx=0).stop 2023-11-24T06:11:20.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:20.610+0000 7f38cc866640 1 -- 172.21.15.92:0/1416967595 reap_dead start 2023-11-24T06:11:20.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:20.610+0000 7f38ce2c9640 1 -- 172.21.15.92:0/1416967595 shutdown_connections 2023-11-24T06:11:20.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:20.610+0000 7f38ce2c9640 1 -- 172.21.15.92:0/1416967595 >> 172.21.15.92:0/1416967595 conn(0x7f38c806a800 msgr2=0x7f38c806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:20.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:20.610+0000 7f38ce2c9640 1 -- 172.21.15.92:0/1416967595 shutdown_connections 2023-11-24T06:11:20.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:20.611+0000 7f38ce2c9640 1 -- 172.21.15.92:0/1416967595 wait complete. 2023-11-24T06:11:20.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:20.612+0000 7f38ce2c9640 1 Processor -- start 2023-11-24T06:11:20.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:20.613+0000 7f38ce2c9640 1 -- start start 2023-11-24T06:11:20.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:20.614+0000 7f38ce2c9640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f38b4004d00 0x7f38c815b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:20.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:20.614+0000 7f38ce2c9640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f38c815c960 0x7f38c81594b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:20.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:20.614+0000 7f38ce2c9640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f38c80cdd40 0x7f38c80ccee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:20.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:20.614+0000 7f38ce2c9640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38c815c540 con 0x7f38c815c960 2023-11-24T06:11:20.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:20.614+0000 7f38ce2c9640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38c815b8c0 con 0x7f38c80cdd40 2023-11-24T06:11:20.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:20.614+0000 7f38ce2c9640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38c815bba0 con 0x7f38b4004d00 2023-11-24T06:11:20.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:20.614+0000 7f38bffff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f38c815c960 0x7f38c81594b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:20.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:20.614+0000 7f38bffff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f38c815c960 0x7f38c81594b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45048/0 (socket says 172.21.15.92:45048) 2023-11-24T06:11:20.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:20.614+0000 7f38bffff640 1 -- 172.21.15.92:0/1296743263 learned_addr learned my addr 172.21.15.92:0/1296743263 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:11:20.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:20.614+0000 7f38c7fff640 1 --2- 172.21.15.92:0/1296743263 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f38b4004d00 0x7f38c815b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:20.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:20.614+0000 7f38bffff640 1 -- 172.21.15.92:0/1296743263 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f38b4004d00 msgr2=0x7f38c815b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:20.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:20.614+0000 7f38cc866640 1 --2- 172.21.15.92:0/1296743263 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f38c80cdd40 0x7f38c80ccee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:20.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:20.614+0000 7f38bffff640 1 --2- 172.21.15.92:0/1296743263 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f38b4004d00 0x7f38c815b0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:20.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:20.614+0000 7f38bffff640 1 -- 172.21.15.92:0/1296743263 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f38c80cdd40 msgr2=0x7f38c80ccee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:20.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:20.614+0000 7f38bffff640 1 --2- 172.21.15.92:0/1296743263 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f38c80cdd40 0x7f38c80ccee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:20.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:20.614+0000 7f38bffff640 1 -- 172.21.15.92:0/1296743263 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38c815be80 con 0x7f38c815c960 2023-11-24T06:11:20.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:20.614+0000 7f38c7fff640 1 --2- 172.21.15.92:0/1296743263 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f38b4004d00 0x7f38c815b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:11:20.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:20.615+0000 7f38bffff640 1 --2- 172.21.15.92:0/1296743263 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f38c815c960 0x7f38c81594b0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f38b000fd50 tx=0x7f38b000d720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:20.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:20.615+0000 7f38c57fa640 1 -- 172.21.15.92:0/1296743263 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38b000de30 con 0x7f38c815c960 2023-11-24T06:11:20.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:20.615+0000 7f38c57fa640 1 -- 172.21.15.92:0/1296743263 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38b0011660 con 0x7f38c815c960 2023-11-24T06:11:20.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:20.615+0000 7f38c57fa640 1 -- 172.21.15.92:0/1296743263 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38b00119f0 con 0x7f38c815c960 2023-11-24T06:11:20.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:20.615+0000 7f38ce2c9640 1 -- 172.21.15.92:0/1296743263 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38c815c160 con 0x7f38c815c960 2023-11-24T06:11:20.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:20.615+0000 7f38ce2c9640 1 -- 172.21.15.92:0/1296743263 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38c815cde0 con 0x7f38c815c960 2023-11-24T06:11:20.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:20.616+0000 7f38ce2c9640 1 -- 172.21.15.92:0/1296743263 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38c815c680 con 0x7f38c815c960 2023-11-24T06:11:20.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:20.617+0000 7f38c57fa640 1 -- 172.21.15.92:0/1296743263 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f38b0011b50 con 0x7f38c815c960 2023-11-24T06:11:20.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:20.618+0000 7f38c57fa640 1 --2- 172.21.15.92:0/1296743263 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f38a806ae40 0x7f38a806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:20.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:20.618+0000 7f38c57fa640 1 -- 172.21.15.92:0/1296743263 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5883+0+0 (secure 0 0 0) 0x7f38b0015070 con 0x7f38c815c960 2023-11-24T06:11:20.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:20.619+0000 7f38c7fff640 1 --2- 172.21.15.92:0/1296743263 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f38a806ae40 0x7f38a806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:20.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:20.619+0000 7f38c7fff640 1 --2- 172.21.15.92:0/1296743263 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f38a806ae40 0x7f38a806d300 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f38b8006fd0 tx=0x7f38b8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:20.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:20.624+0000 7f38c57fa640 1 -- 172.21.15.92:0/1296743263 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f38c815c680 con 0x7f38c815c960 2023-11-24T06:11:20.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:20.818+0000 7f38ce2c9640 1 -- 172.21.15.92:0/1296743263 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f38c815bce0 con 0x7f38c815c960 2023-11-24T06:11:20.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:20.819+0000 7f38c57fa640 1 -- 172.21.15.92:0/1296743263 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16800 (secure 0 0 0) 0x7f38c815bce0 con 0x7f38c815c960 2023-11-24T06:11:20.822 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-24T06:11:20.822 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":22,"fsid":"2d6dee5f-8aa3-470f-931a-50ade963b1cb","created":"2023-11-24T06:10:04.707204+0000","modified":"2023-11-24T06:11:11.472401+0000","last_up_change":"2023-11-24T06:10:17.598352+0000","last_in_change":"2023-11-24T06:10:09.040793+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:10:17.733098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:10:22.926445+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T06:10:24.151288+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ff37a9fb-135f-471f-b4b0-f7b5772006c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6803","nonce":20560714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6805","nonce":20560714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6809","nonce":20560714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6807","nonce":20560714}]},"public_addr":"172.21.15.92:6803/20560714","cluster_addr":"172.21.15.92:6805/20560714","heartbeat_back_addr":"172.21.15.92:6809/20560714","heartbeat_front_addr":"172.21.15.92:6807/20560714","state":["exists","up"]},{"osd":1,"uuid":"03cdd353-dcf1-4f38-94fe-a795137c06b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6819","nonce":2451239906}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6821","nonce":2451239906}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6825","nonce":2451239906}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6823","nonce":2451239906}]},"public_addr":"172.21.15.92:6819/2451239906","cluster_addr":"172.21.15.92:6821/2451239906","heartbeat_back_addr":"172.21.15.92:6825/2451239906","heartbeat_front_addr":"172.21.15.92:6823/2451239906","state":["exists","up"]},{"osd":2,"uuid":"a4a20d80-5d57-484b-becd-9aa8070dd3aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6811","nonce":1124979277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6813","nonce":1124979277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6817","nonce":1124979277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6815","nonce":1124979277}]},"public_addr":"172.21.15.92:6811/1124979277","cluster_addr":"172.21.15.92:6813/1124979277","heartbeat_back_addr":"172.21.15.92:6817/1124979277","heartbeat_front_addr":"172.21.15.92:6815/1124979277","state":["exists","up"]},{"osd":3,"uuid":"4648d7ce-9b29-4583-9425-4ba8fd2b1903","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6827","nonce":1734115782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6829","nonce":1734115782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6833","nonce":1734115782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6831","nonce":1734115782}]},"public_addr":"172.21.15.92:6827/1734115782","cluster_addr":"172.21.15.92:6829/1734115782","heartbeat_back_addr":"172.21.15.92:6833/1734115782","heartbeat_front_addr":"172.21.15.92:6831/1734115782","state":["exists","up"]},{"osd":4,"uuid":"3bc9d7eb-c6fa-4fcf-a3ec-ade604e05aac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6819","nonce":3425937229}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6822","nonce":3425937229}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6830","nonce":3425937229}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6826","nonce":3425937229}]},"public_addr":"172.21.15.118:6819/3425937229","cluster_addr":"172.21.15.118:6822/3425937229","heartbeat_back_addr":"172.21.15.118:6830/3425937229","heartbeat_front_addr":"172.21.15.118:6826/3425937229","state":["exists","up"]},{"osd":5,"uuid":"dce9ba6d-59ed-46d1-84fc-a84b13de10c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6801","nonce":827455984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6803","nonce":827455984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6807","nonce":827455984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6805","nonce":827455984}]},"public_addr":"172.21.15.118:6801/827455984","cluster_addr":"172.21.15.118:6803/827455984","heartbeat_back_addr":"172.21.15.118:6807/827455984","heartbeat_front_addr":"172.21.15.118:6805/827455984","state":["exists","up"]},{"osd":6,"uuid":"82aec504-ae58-4709-8d30-46f2b12081b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6809","nonce":2946986141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6811","nonce":2946986141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6815","nonce":2946986141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6813","nonce":2946986141}]},"public_addr":"172.21.15.118:6809/2946986141","cluster_addr":"172.21.15.118:6811/2946986141","heartbeat_back_addr":"172.21.15.118:6815/2946986141","heartbeat_front_addr":"172.21.15.118:6813/2946986141","state":["exists","up"]},{"osd":7,"uuid":"4ed4c624-c88f-4d35-bdb1-bf209b5dffb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6817","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6818","nonce":48122059}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6821","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6823","nonce":48122059}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6829","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6831","nonce":48122059}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6825","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6827","nonce":48122059}]},"public_addr":"172.21.15.118:6818/48122059","cluster_addr":"172.21.15.118:6823/48122059","heartbeat_back_addr":"172.21.15.118:6831/48122059","heartbeat_front_addr":"172.21.15.118:6827/48122059","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.536664+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.368324+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.271201+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.342971+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.621832+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.387110+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.632946+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.416032+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.d","mappings":[{"from":5,"to":1}]},{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"3.29","mappings":[{"from":3,"to":2}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.34","mappings":[{"from":4,"to":2}]},{"pgid":"3.3b","mappings":[{"from":7,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:11:20.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:20.822+0000 7f38ce2c9640 1 -- 172.21.15.92:0/1296743263 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f38a806ae40 msgr2=0x7f38a806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:20.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:20.822+0000 7f38ce2c9640 1 --2- 172.21.15.92:0/1296743263 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f38a806ae40 0x7f38a806d300 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f38b8006fd0 tx=0x7f38b8008040 comp rx=0 tx=0).stop 2023-11-24T06:11:20.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:20.822+0000 7f38ce2c9640 1 -- 172.21.15.92:0/1296743263 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f38c815c960 msgr2=0x7f38c81594b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:20.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:20.822+0000 7f38ce2c9640 1 --2- 172.21.15.92:0/1296743263 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f38c815c960 0x7f38c81594b0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f38b000fd50 tx=0x7f38b000d720 comp rx=0 tx=0).stop 2023-11-24T06:11:20.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:20.822+0000 7f38ce2c9640 1 -- 172.21.15.92:0/1296743263 shutdown_connections 2023-11-24T06:11:20.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:20.822+0000 7f38ce2c9640 1 --2- 172.21.15.92:0/1296743263 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f38a806ae40 0x7f38a806d300 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:20.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:20.822+0000 7f38ce2c9640 1 --2- 172.21.15.92:0/1296743263 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f38c80cdd40 0x7f38c80ccee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:20.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:20.822+0000 7f38ce2c9640 1 --2- 172.21.15.92:0/1296743263 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f38c815c960 0x7f38c81594b0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:20.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:20.822+0000 7f38ce2c9640 1 --2- 172.21.15.92:0/1296743263 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f38b4004d00 0x7f38c815b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:20.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:20.822+0000 7f38ce2c9640 1 -- 172.21.15.92:0/1296743263 >> 172.21.15.92:0/1296743263 conn(0x7f38c806a800 msgr2=0x7f38c80b0bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:20.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:20.822+0000 7f38ce2c9640 1 -- 172.21.15.92:0/1296743263 shutdown_connections 2023-11-24T06:11:20.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:20.823+0000 7f38ce2c9640 1 -- 172.21.15.92:0/1296743263 wait complete. 2023-11-24T06:11:20.843 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T06:10:17.733098+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T06:10:22.926445+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T06:10:24.151288+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T06:11:20.843 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T06:11:21.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.055+0000 7f40cdecf640 1 Processor -- start 2023-11-24T06:11:21.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.055+0000 7f40cdecf640 1 -- start start 2023-11-24T06:11:21.059 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.057+0000 7f40cdecf640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f40c80cf9a0 0x7f40c8166030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:21.059 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.057+0000 7f40cdecf640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40c815c5f0 con 0x7f40c813e360 2023-11-24T06:11:21.059 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.057+0000 7f40cdecf640 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40c806c470 con 0x7f40c813df80 2023-11-24T06:11:21.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.057+0000 7f40cdecf640 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40c806c750 con 0x7f40c80cf9a0 2023-11-24T06:11:21.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.057+0000 7f40bffff640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f40c813e360 0x7f40c80cf290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:56202/0 (socket says 172.21.15.92:56202) 2023-11-24T06:11:21.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.057+0000 7f40bffff640 1 -- 172.21.15.92:0/2836256518 learned_addr learned my addr 172.21.15.92:0/2836256518 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-24T06:11:21.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.058+0000 7f40c7fff640 1 --2- 172.21.15.92:0/2836256518 >> v2:172.21.15.118:3300/0 conn(0x7f40c80cf9a0 0x7f40c8166030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:21.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.058+0000 7f40c6ffd640 1 -- 172.21.15.92:0/2836256518 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1417656821 0 0) 0x7f40c815c5f0 con 0x7f40c813e360 2023-11-24T06:11:21.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.058+0000 7f40c6ffd640 1 -- 172.21.15.92:0/2836256518 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f40b4003520 con 0x7f40c813e360 2023-11-24T06:11:21.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.058+0000 7f40c6ffd640 1 -- 172.21.15.92:0/2836256518 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2565791664 0 0) 0x7f40b4003520 con 0x7f40c813e360 2023-11-24T06:11:21.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.058+0000 7f40c6ffd640 1 -- 172.21.15.92:0/2836256518 >> v2:172.21.15.118:3300/0 conn(0x7f40c80cf9a0 msgr2=0x7f40c8166030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:21.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.058+0000 7f40c6ffd640 1 --2- 172.21.15.92:0/2836256518 >> v2:172.21.15.118:3300/0 conn(0x7f40c80cf9a0 0x7f40c8166030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:21.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.058+0000 7f40c6ffd640 1 -- 172.21.15.92:0/2836256518 >> v1:172.21.15.118:6789/0 conn(0x7f40c813df80 legacy=0x7f40c815bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:21.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.058+0000 7f40c6ffd640 1 -- 172.21.15.92:0/2836256518 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40c806ca30 con 0x7f40c813e360 2023-11-24T06:11:21.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.058+0000 7f40c6ffd640 1 -- 172.21.15.92:0/2836256518 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2607661350 0 0) 0x7f40b0001670 con 0x7f40c813e360 2023-11-24T06:11:21.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.058+0000 7f40c6ffd640 1 -- 172.21.15.92:0/2836256518 >> v1:172.21.15.92:6789/0 conn(0x7f40c813e360 legacy=0x7f40c80cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:21.063 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.058+0000 7f40c6ffd640 1 --2- 172.21.15.92:0/2836256518 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f40b40039c0 0x7f40b4003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:21.063 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.058+0000 7f40c6ffd640 1 --2- 172.21.15.92:0/2836256518 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f40b40047c0 0x7f40b4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:21.063 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.058+0000 7f40c6ffd640 1 --2- 172.21.15.92:0/2836256518 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f40b4005180 0x7f40b400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:21.063 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.058+0000 7f40c6ffd640 1 -- 172.21.15.92:0/2836256518 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40c806c750 con 0x7f40b40047c0 2023-11-24T06:11:21.063 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.058+0000 7f40c6ffd640 1 -- 172.21.15.92:0/2836256518 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40b400c080 con 0x7f40b4005180 2023-11-24T06:11:21.063 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.058+0000 7f40c6ffd640 1 -- 172.21.15.92:0/2836256518 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40b400c240 con 0x7f40b40039c0 2023-11-24T06:11:21.063 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.059+0000 7f40bffff640 1 --2- 172.21.15.92:0/2836256518 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f40b40047c0 0x7f40b4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:21.063 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.059+0000 7f40c7fff640 1 --2- 172.21.15.92:0/2836256518 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f40b4005180 0x7f40b400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:21.063 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.059+0000 7f40c77fe640 1 --2- 172.21.15.92:0/2836256518 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f40b40039c0 0x7f40b4003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:21.063 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.059+0000 7f40bffff640 1 -- 172.21.15.92:0/2836256518 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f40b40039c0 msgr2=0x7f40b4003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:21.063 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.059+0000 7f40bffff640 1 --2- 172.21.15.92:0/2836256518 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f40b40039c0 0x7f40b4003de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:21.063 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.059+0000 7f40bffff640 1 -- 172.21.15.92:0/2836256518 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f40b4005180 msgr2=0x7f40b400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:21.064 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.059+0000 7f40bffff640 1 --2- 172.21.15.92:0/2836256518 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f40b4005180 0x7f40b400b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:21.064 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.059+0000 7f40bffff640 1 -- 172.21.15.92:0/2836256518 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40b4003520 con 0x7f40b40047c0 2023-11-24T06:11:21.064 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.059+0000 7f40c7fff640 1 -- 172.21.15.92:0/2836256518 reap_dead start 2023-11-24T06:11:21.065 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.061+0000 7f40bffff640 1 --2- 172.21.15.92:0/2836256518 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f40b40047c0 0x7f40b4004c40 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f40c80cc790 tx=0x7f40b40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49d237d269562cab server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:21.065 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.061+0000 7f40c6ffd640 1 -- 172.21.15.92:0/2836256518 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40b0005ad0 con 0x7f40b40047c0 2023-11-24T06:11:21.065 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.061+0000 7f40c6ffd640 1 -- 172.21.15.92:0/2836256518 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40b0005c30 con 0x7f40b40047c0 2023-11-24T06:11:21.065 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.061+0000 7f40c6ffd640 1 -- 172.21.15.92:0/2836256518 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40b002c9d0 con 0x7f40b40047c0 2023-11-24T06:11:21.065 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.062+0000 7f40cdecf640 1 -- 172.21.15.92:0/2836256518 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f40b40047c0 msgr2=0x7f40b4004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:21.065 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.062+0000 7f40cdecf640 1 --2- 172.21.15.92:0/2836256518 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f40b40047c0 0x7f40b4004c40 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f40c80cc790 tx=0x7f40b40057f0 comp rx=0 tx=0).stop 2023-11-24T06:11:21.065 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.062+0000 7f40cdecf640 1 -- 172.21.15.92:0/2836256518 shutdown_connections 2023-11-24T06:11:21.065 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.062+0000 7f40cdecf640 1 --2- 172.21.15.92:0/2836256518 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f40b40047c0 0x7f40b4004c40 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:21.066 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.062+0000 7f40cdecf640 1 -- 172.21.15.92:0/2836256518 >> 172.21.15.92:0/2836256518 conn(0x7f40c806a800 msgr2=0x7f40c806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:21.066 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.062+0000 7f40cdecf640 1 -- 172.21.15.92:0/2836256518 shutdown_connections 2023-11-24T06:11:21.066 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.062+0000 7f40cdecf640 1 -- 172.21.15.92:0/2836256518 wait complete. 2023-11-24T06:11:21.066 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.064+0000 7f40cdecf640 1 Processor -- start 2023-11-24T06:11:21.066 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.064+0000 7f40cdecf640 1 -- start start 2023-11-24T06:11:21.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.065+0000 7f40cdecf640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f40b40047c0 0x7f40c81482f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:21.069 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.065+0000 7f40cdecf640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f40c81582f0 0x7f40c80cd840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:21.069 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.065+0000 7f40cdecf640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f40c8158750 0x7f40c80cdd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:21.069 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.065+0000 7f40cdecf640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40c8157250 con 0x7f40c81582f0 2023-11-24T06:11:21.069 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.065+0000 7f40cdecf640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40c8157530 con 0x7f40b40047c0 2023-11-24T06:11:21.069 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.065+0000 7f40cdecf640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40c8157810 con 0x7f40c8158750 2023-11-24T06:11:21.069 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.065+0000 7f40bffff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f40c81582f0 0x7f40c80cd840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:21.069 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.065+0000 7f40bffff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f40c81582f0 0x7f40c80cd840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45070/0 (socket says 172.21.15.92:45070) 2023-11-24T06:11:21.069 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.065+0000 7f40bffff640 1 -- 172.21.15.92:0/3864385052 learned_addr learned my addr 172.21.15.92:0/3864385052 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:11:21.070 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.066+0000 7f40c77fe640 1 --2- 172.21.15.92:0/3864385052 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f40b40047c0 0x7f40c81482f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:21.070 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.066+0000 7f40c7fff640 1 --2- 172.21.15.92:0/3864385052 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f40c8158750 0x7f40c80cdd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:21.070 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.066+0000 7f40bffff640 1 -- 172.21.15.92:0/3864385052 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f40c8158750 msgr2=0x7f40c80cdd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:21.070 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.066+0000 7f40bffff640 1 --2- 172.21.15.92:0/3864385052 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f40c8158750 0x7f40c80cdd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:21.070 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.066+0000 7f40bffff640 1 -- 172.21.15.92:0/3864385052 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f40b40047c0 msgr2=0x7f40c81482f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:21.070 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.066+0000 7f40bffff640 1 --2- 172.21.15.92:0/3864385052 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f40b40047c0 0x7f40c81482f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:21.070 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.066+0000 7f40bffff640 1 -- 172.21.15.92:0/3864385052 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40c8157af0 con 0x7f40c81582f0 2023-11-24T06:11:21.070 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.066+0000 7f40bffff640 1 --2- 172.21.15.92:0/3864385052 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f40c81582f0 0x7f40c80cd840 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f40b002c330 tx=0x7f40b002e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:21.070 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.066+0000 7f40c4ff9640 1 -- 172.21.15.92:0/3864385052 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40b003c040 con 0x7f40c81582f0 2023-11-24T06:11:21.071 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.066+0000 7f40c4ff9640 1 -- 172.21.15.92:0/3864385052 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40b002cc80 con 0x7f40c81582f0 2023-11-24T06:11:21.071 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.066+0000 7f40c4ff9640 1 -- 172.21.15.92:0/3864385052 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40b0007070 con 0x7f40c81582f0 2023-11-24T06:11:21.071 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.066+0000 7f40cdecf640 1 -- 172.21.15.92:0/3864385052 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40c8157dd0 con 0x7f40c81582f0 2023-11-24T06:11:21.071 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.066+0000 7f40cdecf640 1 -- 172.21.15.92:0/3864385052 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40c8145f30 con 0x7f40c81582f0 2023-11-24T06:11:21.071 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.068+0000 7f40cdecf640 1 -- 172.21.15.92:0/3864385052 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40c8157f60 con 0x7f40c81582f0 2023-11-24T06:11:21.071 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.068+0000 7f40c4ff9640 1 -- 172.21.15.92:0/3864385052 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f40b0032070 con 0x7f40c81582f0 2023-11-24T06:11:21.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.070+0000 7f40c4ff9640 1 --2- 172.21.15.92:0/3864385052 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f40a806ae40 0x7f40a806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:21.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.070+0000 7f40c4ff9640 1 -- 172.21.15.92:0/3864385052 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5883+0+0 (secure 0 0 0) 0x7f40b00aabb0 con 0x7f40c81582f0 2023-11-24T06:11:21.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.070+0000 7f40c77fe640 1 --2- 172.21.15.92:0/3864385052 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f40a806ae40 0x7f40a806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:21.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.071+0000 7f40c77fe640 1 --2- 172.21.15.92:0/3864385052 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f40a806ae40 0x7f40a806d300 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f40c80695b0 tx=0x7f40b8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:21.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.074+0000 7f40c4ff9640 1 -- 172.21.15.92:0/3864385052 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f40b0075b20 con 0x7f40c81582f0 2023-11-24T06:11:21.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.262+0000 7f40cdecf640 1 -- 172.21.15.92:0/3864385052 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f40c80675d0 con 0x7f40c81582f0 2023-11-24T06:11:21.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.263+0000 7f40c4ff9640 1 -- 172.21.15.92:0/3864385052 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7f40b0079140 con 0x7f40c81582f0 2023-11-24T06:11:21.266 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 1 2023-11-24T06:11:21.269 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.266+0000 7f40cdecf640 1 -- 172.21.15.92:0/3864385052 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f40a806ae40 msgr2=0x7f40a806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:21.269 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.266+0000 7f40cdecf640 1 --2- 172.21.15.92:0/3864385052 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f40a806ae40 0x7f40a806d300 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f40c80695b0 tx=0x7f40b8009040 comp rx=0 tx=0).stop 2023-11-24T06:11:21.269 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.266+0000 7f40cdecf640 1 -- 172.21.15.92:0/3864385052 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f40c81582f0 msgr2=0x7f40c80cd840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:21.269 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.266+0000 7f40cdecf640 1 --2- 172.21.15.92:0/3864385052 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f40c81582f0 0x7f40c80cd840 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f40b002c330 tx=0x7f40b002e070 comp rx=0 tx=0).stop 2023-11-24T06:11:21.269 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.266+0000 7f40cdecf640 1 -- 172.21.15.92:0/3864385052 shutdown_connections 2023-11-24T06:11:21.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.266+0000 7f40cdecf640 1 --2- 172.21.15.92:0/3864385052 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f40a806ae40 0x7f40a806d300 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:21.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.266+0000 7f40cdecf640 1 --2- 172.21.15.92:0/3864385052 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f40c8158750 0x7f40c80cdd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:21.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.266+0000 7f40cdecf640 1 --2- 172.21.15.92:0/3864385052 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f40c81582f0 0x7f40c80cd840 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:21.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.266+0000 7f40cdecf640 1 --2- 172.21.15.92:0/3864385052 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f40b40047c0 0x7f40c81482f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:21.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.266+0000 7f40cdecf640 1 -- 172.21.15.92:0/3864385052 >> 172.21.15.92:0/3864385052 conn(0x7f40c806a800 msgr2=0x7f40c80b3ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:21.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.266+0000 7f40cdecf640 1 -- 172.21.15.92:0/3864385052 shutdown_connections 2023-11-24T06:11:21.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.266+0000 7f40cdecf640 1 -- 172.21.15.92:0/3864385052 wait complete. 2023-11-24T06:11:21.288 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T06:11:21.507 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.505+0000 7ffa85706640 1 Processor -- start 2023-11-24T06:11:21.507 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.505+0000 7ffa85706640 1 -- start start 2023-11-24T06:11:21.508 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.506+0000 7ffa85706640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ffa8015b210 0x7ffa800d1920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:21.508 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.506+0000 7ffa85706640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7ffa800cdf00 0x7ffa800b0c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:21.508 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.506+0000 7ffa85706640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa8015b5f0 con 0x7ffa800b01e0 2023-11-24T06:11:21.509 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.506+0000 7ffa85706640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa800cce60 con 0x7ffa8015b210 2023-11-24T06:11:21.509 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.506+0000 7ffa85706640 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa800cf510 con 0x7ffa800cdf00 2023-11-24T06:11:21.509 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.506+0000 7ffa7f7fe640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7ffa800b01e0 0x7ffa800cd7f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:56212/0 (socket says 172.21.15.92:56212) 2023-11-24T06:11:21.509 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.506+0000 7ffa7f7fe640 1 -- 172.21.15.92:0/2692580548 learned_addr learned my addr 172.21.15.92:0/2692580548 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-24T06:11:21.509 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.506+0000 7ffa7ffff640 1 --2- 172.21.15.92:0/2692580548 >> v2:172.21.15.92:3300/0 conn(0x7ffa8015b210 0x7ffa800d1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:21.510 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.507+0000 7ffa84f05640 1 --2- 172.21.15.92:0/2692580548 >> v2:172.21.15.118:3300/0 conn(0x7ffa800cdf00 0x7ffa800b0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:21.510 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.507+0000 7ffa7effd640 1 -- 172.21.15.92:0/2692580548 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 210325513 0 0) 0x7ffa8015b5f0 con 0x7ffa800b01e0 2023-11-24T06:11:21.510 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.507+0000 7ffa7effd640 1 -- 172.21.15.92:0/2692580548 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffa6c003520 con 0x7ffa800b01e0 2023-11-24T06:11:21.510 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.507+0000 7ffa7ffff640 1 -- 172.21.15.92:0/2692580548 >> v2:172.21.15.118:3300/0 conn(0x7ffa800cdf00 msgr2=0x7ffa800b0c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:21.510 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.507+0000 7ffa7ffff640 1 --2- 172.21.15.92:0/2692580548 >> v2:172.21.15.118:3300/0 conn(0x7ffa800cdf00 0x7ffa800b0c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:21.510 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.507+0000 7ffa7ffff640 1 -- 172.21.15.92:0/2692580548 >> v1:172.21.15.92:6789/0 conn(0x7ffa800b01e0 legacy=0x7ffa800cd7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:21.510 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.507+0000 7ffa7ffff640 1 -- 172.21.15.92:0/2692580548 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa800cf7f0 con 0x7ffa8015b210 2023-11-24T06:11:21.510 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.507+0000 7ffa84f05640 1 --2- 172.21.15.92:0/2692580548 >> v2:172.21.15.118:3300/0 conn(0x7ffa800cdf00 0x7ffa800b0c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:11:21.511 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.508+0000 7ffa7ffff640 1 --2- 172.21.15.92:0/2692580548 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffa8015b210 0x7ffa800d1920 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7ffa800d0b40 tx=0x7ffa7002be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11dae6fcad7a4e74 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:21.512 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.508+0000 7ffa7effd640 1 -- 172.21.15.92:0/2692580548 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa7002cbe0 con 0x7ffa8015b210 2023-11-24T06:11:21.512 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.508+0000 7ffa7effd640 1 -- 172.21.15.92:0/2692580548 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa7002cd40 con 0x7ffa8015b210 2023-11-24T06:11:21.512 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.508+0000 7ffa7effd640 1 -- 172.21.15.92:0/2692580548 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa70034640 con 0x7ffa8015b210 2023-11-24T06:11:21.512 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.509+0000 7ffa85706640 1 -- 172.21.15.92:0/2692580548 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffa8015b210 msgr2=0x7ffa800d1920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:21.512 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.509+0000 7ffa85706640 1 --2- 172.21.15.92:0/2692580548 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffa8015b210 0x7ffa800d1920 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7ffa800d0b40 tx=0x7ffa7002be30 comp rx=0 tx=0).stop 2023-11-24T06:11:21.512 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.509+0000 7ffa85706640 1 -- 172.21.15.92:0/2692580548 shutdown_connections 2023-11-24T06:11:21.512 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.509+0000 7ffa85706640 1 --2- 172.21.15.92:0/2692580548 >> v2:172.21.15.118:3300/0 conn(0x7ffa800cdf00 0x7ffa800b0c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:21.512 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.509+0000 7ffa85706640 1 --2- 172.21.15.92:0/2692580548 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffa8015b210 0x7ffa800d1920 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:21.512 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.509+0000 7ffa85706640 1 -- 172.21.15.92:0/2692580548 >> 172.21.15.92:0/2692580548 conn(0x7ffa800d0e00 msgr2=0x7ffa800d1210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:21.512 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.509+0000 7ffa85706640 1 -- 172.21.15.92:0/2692580548 shutdown_connections 2023-11-24T06:11:21.512 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.509+0000 7ffa85706640 1 -- 172.21.15.92:0/2692580548 wait complete. 2023-11-24T06:11:21.513 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.511+0000 7ffa85706640 1 Processor -- start 2023-11-24T06:11:21.513 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.511+0000 7ffa85706640 1 -- start start 2023-11-24T06:11:21.514 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.512+0000 7ffa85706640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ffa800b01e0 0x7ffa800ca970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:21.514 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.512+0000 7ffa85706640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ffa800cdf00 0x7ffa8014e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:21.514 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.512+0000 7ffa85706640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffa8015b210 0x7ffa8015cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:21.514 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.512+0000 7ffa85706640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa800c8220 con 0x7ffa8015b210 2023-11-24T06:11:21.515 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.512+0000 7ffa85706640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa800caeb0 con 0x7ffa800b01e0 2023-11-24T06:11:21.515 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.512+0000 7ffa85706640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa8014e820 con 0x7ffa800cdf00 2023-11-24T06:11:21.515 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.512+0000 7ffa84f05640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffa8015b210 0x7ffa8015cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:21.515 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.512+0000 7ffa84f05640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffa8015b210 0x7ffa8015cdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45080/0 (socket says 172.21.15.92:45080) 2023-11-24T06:11:21.515 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.512+0000 7ffa84f05640 1 -- 172.21.15.92:0/2616455157 learned_addr learned my addr 172.21.15.92:0/2616455157 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:11:21.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.513+0000 7ffa7ffff640 1 --2- 172.21.15.92:0/2616455157 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ffa800b01e0 0x7ffa800ca970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:21.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.513+0000 7ffa84f05640 1 -- 172.21.15.92:0/2616455157 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ffa800cdf00 msgr2=0x7ffa8014e2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:21.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.513+0000 7ffa84f05640 1 --2- 172.21.15.92:0/2616455157 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ffa800cdf00 0x7ffa8014e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:21.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.513+0000 7ffa84f05640 1 -- 172.21.15.92:0/2616455157 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ffa800b01e0 msgr2=0x7ffa800ca970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:21.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.513+0000 7ffa84f05640 1 --2- 172.21.15.92:0/2616455157 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ffa800b01e0 0x7ffa800ca970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:21.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.513+0000 7ffa84f05640 1 -- 172.21.15.92:0/2616455157 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa70009a70 con 0x7ffa8015b210 2023-11-24T06:11:21.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.513+0000 7ffa7ffff640 1 --2- 172.21.15.92:0/2616455157 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ffa800b01e0 0x7ffa800ca970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:11:21.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.513+0000 7ffa84f05640 1 --2- 172.21.15.92:0/2616455157 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffa8015b210 0x7ffa8015cdd0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7ffa7800c7f0 tx=0x7ffa7800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:21.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.513+0000 7ffa7cff9640 1 -- 172.21.15.92:0/2616455157 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa78007820 con 0x7ffa8015b210 2023-11-24T06:11:21.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.513+0000 7ffa7cff9640 1 -- 172.21.15.92:0/2616455157 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa78007980 con 0x7ffa8015b210 2023-11-24T06:11:21.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.513+0000 7ffa7cff9640 1 -- 172.21.15.92:0/2616455157 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa78007d70 con 0x7ffa8015b210 2023-11-24T06:11:21.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.513+0000 7ffa85706640 1 -- 172.21.15.92:0/2616455157 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa800c7ed0 con 0x7ffa8015b210 2023-11-24T06:11:21.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.513+0000 7ffa85706640 1 -- 172.21.15.92:0/2616455157 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa800c7600 con 0x7ffa8015b210 2023-11-24T06:11:21.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.514+0000 7ffa85706640 1 -- 172.21.15.92:0/2616455157 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa800d04c0 con 0x7ffa8015b210 2023-11-24T06:11:21.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.515+0000 7ffa7cff9640 1 -- 172.21.15.92:0/2616455157 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ffa780113f0 con 0x7ffa8015b210 2023-11-24T06:11:21.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.516+0000 7ffa7cff9640 1 --2- 172.21.15.92:0/2616455157 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7ffa6006ae40 0x7ffa6006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:21.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.516+0000 7ffa7cff9640 1 -- 172.21.15.92:0/2616455157 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5883+0+0 (secure 0 0 0) 0x7ffa78089960 con 0x7ffa8015b210 2023-11-24T06:11:21.519 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.517+0000 7ffa7ffff640 1 --2- 172.21.15.92:0/2616455157 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7ffa6006ae40 0x7ffa6006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:21.519 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.517+0000 7ffa7ffff640 1 --2- 172.21.15.92:0/2616455157 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7ffa6006ae40 0x7ffa6006d300 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7ffa70036000 tx=0x7ffa70030000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:21.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.521+0000 7ffa7cff9640 1 -- 172.21.15.92:0/2616455157 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ffa780549f0 con 0x7ffa8015b210 2023-11-24T06:11:21.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.710+0000 7ffa85706640 1 -- 172.21.15.92:0/2616455157 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ffa800c8360 con 0x7ffa8015b210 2023-11-24T06:11:21.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.711+0000 7ffa7cff9640 1 -- 172.21.15.92:0/2616455157 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7ffa7801f020 con 0x7ffa8015b210 2023-11-24T06:11:21.713 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 32 2023-11-24T06:11:21.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.714+0000 7ffa85706640 1 -- 172.21.15.92:0/2616455157 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7ffa6006ae40 msgr2=0x7ffa6006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:21.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.714+0000 7ffa85706640 1 --2- 172.21.15.92:0/2616455157 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7ffa6006ae40 0x7ffa6006d300 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7ffa70036000 tx=0x7ffa70030000 comp rx=0 tx=0).stop 2023-11-24T06:11:21.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.714+0000 7ffa85706640 1 -- 172.21.15.92:0/2616455157 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffa8015b210 msgr2=0x7ffa8015cdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:21.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.714+0000 7ffa85706640 1 --2- 172.21.15.92:0/2616455157 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffa8015b210 0x7ffa8015cdd0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7ffa7800c7f0 tx=0x7ffa7800ccb0 comp rx=0 tx=0).stop 2023-11-24T06:11:21.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.714+0000 7ffa85706640 1 -- 172.21.15.92:0/2616455157 shutdown_connections 2023-11-24T06:11:21.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.714+0000 7ffa85706640 1 --2- 172.21.15.92:0/2616455157 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7ffa6006ae40 0x7ffa6006d300 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:21.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.714+0000 7ffa85706640 1 --2- 172.21.15.92:0/2616455157 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffa8015b210 0x7ffa8015cdd0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:21.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.714+0000 7ffa85706640 1 --2- 172.21.15.92:0/2616455157 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ffa800cdf00 0x7ffa8014e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:21.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.714+0000 7ffa85706640 1 --2- 172.21.15.92:0/2616455157 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ffa800b01e0 0x7ffa800ca970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:21.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.714+0000 7ffa85706640 1 -- 172.21.15.92:0/2616455157 >> 172.21.15.92:0/2616455157 conn(0x7ffa800d0e00 msgr2=0x7ffa800cba80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:21.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.714+0000 7ffa85706640 1 -- 172.21.15.92:0/2616455157 shutdown_connections 2023-11-24T06:11:21.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.714+0000 7ffa85706640 1 -- 172.21.15.92:0/2616455157 wait complete. 2023-11-24T06:11:21.736 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T06:11:21.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.948+0000 7fc586f97640 1 Processor -- start 2023-11-24T06:11:21.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.948+0000 7fc586f97640 1 -- start start 2023-11-24T06:11:21.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.949+0000 7fc586f97640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fc58013df80 0x7fc58015bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:21.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.949+0000 7fc586f97640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc58006c470 con 0x7fc58013e360 2023-11-24T06:11:21.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.949+0000 7fc586f97640 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc58006c750 con 0x7fc5800cf9a0 2023-11-24T06:11:21.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.949+0000 7fc586f97640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc58006ca30 con 0x7fc58013df80 2023-11-24T06:11:21.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.950+0000 7fc584d33640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fc58013df80 0x7fc58015bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:21.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.950+0000 7fc57ffff640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fc58013e360 0x7fc5800cf290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:56220/0 (socket says 172.21.15.92:56220) 2023-11-24T06:11:21.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.950+0000 7fc57ffff640 1 -- 172.21.15.92:0/3325916415 learned_addr learned my addr 172.21.15.92:0/3325916415 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-24T06:11:21.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.950+0000 7fc584d33640 1 -- 172.21.15.92:0/3325916415 >> v1:172.21.15.118:6789/0 conn(0x7fc5800cf9a0 legacy=0x7fc5800cd840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:21.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.950+0000 7fc584d33640 1 -- 172.21.15.92:0/3325916415 >> v1:172.21.15.92:6789/0 conn(0x7fc58013e360 legacy=0x7fc5800cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:21.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.950+0000 7fc584d33640 1 -- 172.21.15.92:0/3325916415 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc58006cd10 con 0x7fc58013df80 2023-11-24T06:11:21.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.951+0000 7fc584d33640 1 --2- 172.21.15.92:0/3325916415 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc58013df80 0x7fc58015bee0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fc5740097a0 tx=0x7fc57402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3077fa2d67b44eab server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:21.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.952+0000 7fc57f7fe640 1 -- 172.21.15.92:0/3325916415 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc574031830 con 0x7fc58013df80 2023-11-24T06:11:21.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.952+0000 7fc57f7fe640 1 -- 172.21.15.92:0/3325916415 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc574031990 con 0x7fc58013df80 2023-11-24T06:11:21.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.952+0000 7fc57f7fe640 1 -- 172.21.15.92:0/3325916415 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc574031d20 con 0x7fc58013df80 2023-11-24T06:11:21.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.952+0000 7fc586f97640 1 -- 172.21.15.92:0/3325916415 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc58013df80 msgr2=0x7fc58015bee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:21.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.952+0000 7fc586f97640 1 --2- 172.21.15.92:0/3325916415 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc58013df80 0x7fc58015bee0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fc5740097a0 tx=0x7fc57402b8d0 comp rx=0 tx=0).stop 2023-11-24T06:11:21.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.952+0000 7fc586f97640 1 -- 172.21.15.92:0/3325916415 shutdown_connections 2023-11-24T06:11:21.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.952+0000 7fc586f97640 1 --2- 172.21.15.92:0/3325916415 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc58013df80 0x7fc58015bee0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:21.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.952+0000 7fc586f97640 1 -- 172.21.15.92:0/3325916415 >> 172.21.15.92:0/3325916415 conn(0x7fc58006a800 msgr2=0x7fc58006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:21.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.952+0000 7fc586f97640 1 -- 172.21.15.92:0/3325916415 shutdown_connections 2023-11-24T06:11:21.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.952+0000 7fc586f97640 1 -- 172.21.15.92:0/3325916415 wait complete. 2023-11-24T06:11:21.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.954+0000 7fc586f97640 1 Processor -- start 2023-11-24T06:11:21.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.955+0000 7fc586f97640 1 -- start start 2023-11-24T06:11:21.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.956+0000 7fc586f97640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc5800cf9a0 0x7fc580071f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:21.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.956+0000 7fc586f97640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc58013df80 0x7fc58013ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:21.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.956+0000 7fc586f97640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc58013e360 0x7fc5800703e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:21.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.956+0000 7fc586f97640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc580071ab0 con 0x7fc5800cf9a0 2023-11-24T06:11:21.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.956+0000 7fc586f97640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc58013b190 con 0x7fc58013df80 2023-11-24T06:11:21.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.956+0000 7fc586f97640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc580070ea0 con 0x7fc58013e360 2023-11-24T06:11:21.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.956+0000 7fc584d33640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc5800cf9a0 0x7fc580071f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:21.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.956+0000 7fc584d33640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc5800cf9a0 0x7fc580071f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45098/0 (socket says 172.21.15.92:45098) 2023-11-24T06:11:21.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.956+0000 7fc584d33640 1 -- 172.21.15.92:0/3918286668 learned_addr learned my addr 172.21.15.92:0/3918286668 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:11:21.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.957+0000 7fc57ffff640 1 --2- 172.21.15.92:0/3918286668 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc58013df80 0x7fc58013ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:21.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.957+0000 7fc584d33640 1 -- 172.21.15.92:0/3918286668 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc58013e360 msgr2=0x7fc5800703e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:21.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.957+0000 7fc585534640 1 --2- 172.21.15.92:0/3918286668 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc58013e360 0x7fc5800703e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:21.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.957+0000 7fc584d33640 1 --2- 172.21.15.92:0/3918286668 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc58013e360 0x7fc5800703e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:21.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.957+0000 7fc584d33640 1 -- 172.21.15.92:0/3918286668 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc58013df80 msgr2=0x7fc58013ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:21.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.957+0000 7fc584d33640 1 --2- 172.21.15.92:0/3918286668 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc58013df80 0x7fc58013ac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:21.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.957+0000 7fc584d33640 1 -- 172.21.15.92:0/3918286668 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc574009400 con 0x7fc5800cf9a0 2023-11-24T06:11:21.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.957+0000 7fc57ffff640 1 --2- 172.21.15.92:0/3918286668 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc58013df80 0x7fc58013ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:11:21.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.957+0000 7fc584d33640 1 --2- 172.21.15.92:0/3918286668 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc5800cf9a0 0x7fc580071f40 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fc5740098c0 tx=0x7fc5740046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:21.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.957+0000 7fc57d7fa640 1 -- 172.21.15.92:0/3918286668 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc57403ba70 con 0x7fc5800cf9a0 2023-11-24T06:11:21.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.957+0000 7fc57d7fa640 1 -- 172.21.15.92:0/3918286668 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc57403bbd0 con 0x7fc5800cf9a0 2023-11-24T06:11:21.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.957+0000 7fc586f97640 1 -- 172.21.15.92:0/3918286668 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc580071180 con 0x7fc5800cf9a0 2023-11-24T06:11:21.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.957+0000 7fc57d7fa640 1 -- 172.21.15.92:0/3918286668 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5740453f0 con 0x7fc5800cf9a0 2023-11-24T06:11:21.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.958+0000 7fc586f97640 1 -- 172.21.15.92:0/3918286668 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc580071740 con 0x7fc5800cf9a0 2023-11-24T06:11:21.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.959+0000 7fc586f97640 1 -- 172.21.15.92:0/3918286668 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5800675d0 con 0x7fc5800cf9a0 2023-11-24T06:11:21.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.960+0000 7fc57d7fa640 1 -- 172.21.15.92:0/3918286668 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc57404e450 con 0x7fc5800cf9a0 2023-11-24T06:11:21.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.961+0000 7fc57d7fa640 1 --2- 172.21.15.92:0/3918286668 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fc56406ae40 0x7fc56406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:21.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.961+0000 7fc57d7fa640 1 -- 172.21.15.92:0/3918286668 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5883+0+0 (secure 0 0 0) 0x7fc5740aba10 con 0x7fc5800cf9a0 2023-11-24T06:11:21.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.962+0000 7fc57ffff640 1 --2- 172.21.15.92:0/3918286668 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fc56406ae40 0x7fc56406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:21.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.962+0000 7fc57ffff640 1 --2- 172.21.15.92:0/3918286668 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fc56406ae40 0x7fc56406d300 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fc5800cd280 tx=0x7fc56c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:21.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:21.966+0000 7fc57d7fa640 1 -- 172.21.15.92:0/3918286668 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc574076980 con 0x7fc5800cf9a0 2023-11-24T06:11:22.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:22.154+0000 7fc586f97640 1 -- 172.21.15.92:0/3918286668 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc58006df00 con 0x7fc5800cf9a0 2023-11-24T06:11:22.158 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:22.155+0000 7fc57d7fa640 1 -- 172.21.15.92:0/3918286668 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc574079fa0 con 0x7fc5800cf9a0 2023-11-24T06:11:22.158 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 64 2023-11-24T06:11:22.161 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:22.158+0000 7fc586f97640 1 -- 172.21.15.92:0/3918286668 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fc56406ae40 msgr2=0x7fc56406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:22.161 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:22.158+0000 7fc586f97640 1 --2- 172.21.15.92:0/3918286668 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fc56406ae40 0x7fc56406d300 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fc5800cd280 tx=0x7fc56c009040 comp rx=0 tx=0).stop 2023-11-24T06:11:22.161 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:22.158+0000 7fc586f97640 1 -- 172.21.15.92:0/3918286668 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc5800cf9a0 msgr2=0x7fc580071f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:22.161 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:22.158+0000 7fc586f97640 1 --2- 172.21.15.92:0/3918286668 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc5800cf9a0 0x7fc580071f40 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fc5740098c0 tx=0x7fc5740046c0 comp rx=0 tx=0).stop 2023-11-24T06:11:22.161 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:22.158+0000 7fc586f97640 1 -- 172.21.15.92:0/3918286668 shutdown_connections 2023-11-24T06:11:22.161 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:22.158+0000 7fc586f97640 1 --2- 172.21.15.92:0/3918286668 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fc56406ae40 0x7fc56406d300 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:22.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:22.158+0000 7fc586f97640 1 --2- 172.21.15.92:0/3918286668 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc58013e360 0x7fc5800703e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:22.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:22.158+0000 7fc586f97640 1 --2- 172.21.15.92:0/3918286668 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc58013df80 0x7fc58013ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:22.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:22.158+0000 7fc586f97640 1 --2- 172.21.15.92:0/3918286668 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc5800cf9a0 0x7fc580071f40 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:22.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:22.158+0000 7fc586f97640 1 -- 172.21.15.92:0/3918286668 >> 172.21.15.92:0/3918286668 conn(0x7fc58006a800 msgr2=0x7fc58006abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:22.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:22.158+0000 7fc586f97640 1 -- 172.21.15.92:0/3918286668 shutdown_connections 2023-11-24T06:11:22.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:22.158+0000 7fc586f97640 1 -- 172.21.15.92:0/3918286668 wait complete. 2023-11-24T06:11:22.178 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-24T06:11:22.388 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:22.386+0000 7f5633fff640 1 Processor -- start 2023-11-24T06:11:22.388 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:22.386+0000 7f5633fff640 1 -- start start 2023-11-24T06:11:22.389 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:22.387+0000 7f5633fff640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f563413cfa0 0x7f563415afb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:22.389 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:22.387+0000 7f5633fff640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f56340cd530 0x7f563413f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:22.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:22.387+0000 7f5633fff640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56340cdec0 con 0x7f56340cd9b0 2023-11-24T06:11:22.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:22.387+0000 7f5633fff640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56340ce1a0 con 0x7f56340cd530 2023-11-24T06:11:22.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:22.387+0000 7f5633fff640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56340ce480 con 0x7f563413cfa0 2023-11-24T06:11:22.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:22.388+0000 7f56327fc640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f56340cd530 0x7f563413f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:22.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:22.388+0000 7f56337fe640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f56340cd9b0 0x7f563415d9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:56236/0 (socket says 172.21.15.92:56236) 2023-11-24T06:11:22.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:22.388+0000 7f56337fe640 1 -- 172.21.15.92:0/1443623426 learned_addr learned my addr 172.21.15.92:0/1443623426 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-24T06:11:22.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:22.388+0000 7f5632ffd640 1 --2- 172.21.15.92:0/1443623426 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f563413cfa0 0x7f563415afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:22.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:22.388+0000 7f56327fc640 1 -- 172.21.15.92:0/1443623426 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f563413cfa0 msgr2=0x7f563415afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:22.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:22.388+0000 7f56327fc640 1 --2- 172.21.15.92:0/1443623426 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f563413cfa0 0x7f563415afb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:22.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:22.388+0000 7f56327fc640 1 -- 172.21.15.92:0/1443623426 >> v1:172.21.15.92:6789/0 conn(0x7f56340cd9b0 legacy=0x7f563415d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:22.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:22.388+0000 7f56327fc640 1 -- 172.21.15.92:0/1443623426 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56340ce760 con 0x7f56340cd530 2023-11-24T06:11:22.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:22.388+0000 7f5631ffb640 1 -- 172.21.15.92:0/1443623426 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1470482282 0 0) 0x7f56340cdec0 con 0x7f56340cd9b0 2023-11-24T06:11:22.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:22.389+0000 7f56327fc640 1 --2- 172.21.15.92:0/1443623426 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f56340cd530 0x7f563413f5a0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f56280097a0 tx=0x7f562802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=269ad1bee215c776 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:22.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:22.390+0000 7f5631ffb640 1 -- 172.21.15.92:0/1443623426 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f562802e070 con 0x7f56340cd530 2023-11-24T06:11:22.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:22.390+0000 7f5631ffb640 1 -- 172.21.15.92:0/1443623426 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56280315a0 con 0x7f56340cd530 2023-11-24T06:11:22.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:22.390+0000 7f5631ffb640 1 -- 172.21.15.92:0/1443623426 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5628031930 con 0x7f56340cd530 2023-11-24T06:11:22.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:22.390+0000 7f5633fff640 1 -- 172.21.15.92:0/1443623426 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f56340cd530 msgr2=0x7f563413f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:22.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:22.390+0000 7f5633fff640 1 --2- 172.21.15.92:0/1443623426 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f56340cd530 0x7f563413f5a0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f56280097a0 tx=0x7f562802b840 comp rx=0 tx=0).stop 2023-11-24T06:11:22.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:22.390+0000 7f5633fff640 1 -- 172.21.15.92:0/1443623426 shutdown_connections 2023-11-24T06:11:22.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:22.390+0000 7f5633fff640 1 --2- 172.21.15.92:0/1443623426 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f56340cd530 0x7f563413f5a0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:22.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:22.390+0000 7f5633fff640 1 --2- 172.21.15.92:0/1443623426 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f563413cfa0 0x7f563415afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:22.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:22.390+0000 7f5633fff640 1 -- 172.21.15.92:0/1443623426 >> 172.21.15.92:0/1443623426 conn(0x7f563406a800 msgr2=0x7f563406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:22.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:22.390+0000 7f5633fff640 1 -- 172.21.15.92:0/1443623426 shutdown_connections 2023-11-24T06:11:22.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:22.390+0000 7f5633fff640 1 -- 172.21.15.92:0/1443623426 wait complete. 2023-11-24T06:11:22.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:22.392+0000 7f5633fff640 1 Processor -- start 2023-11-24T06:11:22.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:22.392+0000 7f5633fff640 1 -- start start 2023-11-24T06:11:22.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:22.393+0000 7f5633fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f56340cd530 0x7f5634159970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:22.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:22.393+0000 7f5633fff640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f56340cd9b0 0x7f563414f200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:22.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:22.393+0000 7f5633fff640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f563413cfa0 0x7f5634155db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:22.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:22.393+0000 7f5633fff640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56341f7670 con 0x7f56340cd530 2023-11-24T06:11:22.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:22.393+0000 7f5633fff640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f563414e720 con 0x7f56340cd9b0 2023-11-24T06:11:22.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:22.393+0000 7f5633fff640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f563414ea00 con 0x7f563413cfa0 2023-11-24T06:11:22.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:22.393+0000 7f5632ffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f56340cd530 0x7f5634159970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:22.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:22.393+0000 7f5632ffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f56340cd530 0x7f5634159970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45112/0 (socket says 172.21.15.92:45112) 2023-11-24T06:11:22.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:22.393+0000 7f5632ffd640 1 -- 172.21.15.92:0/2476051014 learned_addr learned my addr 172.21.15.92:0/2476051014 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:11:22.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:22.394+0000 7f5632ffd640 1 -- 172.21.15.92:0/2476051014 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f563413cfa0 msgr2=0x7f5634155db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:22.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:22.394+0000 7f5632ffd640 1 --2- 172.21.15.92:0/2476051014 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f563413cfa0 0x7f5634155db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:22.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:22.394+0000 7f5632ffd640 1 -- 172.21.15.92:0/2476051014 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f56340cd9b0 msgr2=0x7f563414f200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:22.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:22.394+0000 7f5632ffd640 1 --2- 172.21.15.92:0/2476051014 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f56340cd9b0 0x7f563414f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:22.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:22.394+0000 7f5632ffd640 1 -- 172.21.15.92:0/2476051014 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5628009400 con 0x7f56340cd530 2023-11-24T06:11:22.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:22.394+0000 7f5632ffd640 1 --2- 172.21.15.92:0/2476051014 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f56340cd530 0x7f5634159970 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f562400d5f0 tx=0x7f562400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:22.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:22.394+0000 7f561b7fe640 1 -- 172.21.15.92:0/2476051014 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56240125f0 con 0x7f56340cd530 2023-11-24T06:11:22.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:22.394+0000 7f561b7fe640 1 -- 172.21.15.92:0/2476051014 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5624012750 con 0x7f56340cd530 2023-11-24T06:11:22.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:22.394+0000 7f561b7fe640 1 -- 172.21.15.92:0/2476051014 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5624012b00 con 0x7f56340cd530 2023-11-24T06:11:22.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:22.394+0000 7f5633fff640 1 -- 172.21.15.92:0/2476051014 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f563414ece0 con 0x7f56340cd530 2023-11-24T06:11:22.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:22.395+0000 7f5633fff640 1 -- 172.21.15.92:0/2476051014 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5634159eb0 con 0x7f56340cd530 2023-11-24T06:11:22.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:22.396+0000 7f5633fff640 1 -- 172.21.15.92:0/2476051014 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5634067450 con 0x7f56340cd530 2023-11-24T06:11:22.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:22.397+0000 7f561b7fe640 1 -- 172.21.15.92:0/2476051014 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5624012c60 con 0x7f56340cd530 2023-11-24T06:11:22.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:22.397+0000 7f561b7fe640 1 --2- 172.21.15.92:0/2476051014 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f562006ae40 0x7f562006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:22.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:22.398+0000 7f561b7fe640 1 -- 172.21.15.92:0/2476051014 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5883+0+0 (secure 0 0 0) 0x7f5624089bb0 con 0x7f56340cd530 2023-11-24T06:11:22.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:22.398+0000 7f56327fc640 1 --2- 172.21.15.92:0/2476051014 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f562006ae40 0x7f562006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:22.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:22.398+0000 7f56327fc640 1 --2- 172.21.15.92:0/2476051014 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f562006ae40 0x7f562006d300 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f5628036000 tx=0x7f5628030000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:22.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:22.402+0000 7f561b7fe640 1 -- 172.21.15.92:0/2476051014 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5624054bb0 con 0x7f56340cd530 2023-11-24T06:11:22.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:22.601+0000 7f5633fff640 1 -- 172.21.15.92:0/2476051014 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f563415aff0 con 0x7f56340cd530 2023-11-24T06:11:22.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:22.602+0000 7f561b7fe640 1 -- 172.21.15.92:0/2476051014 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) v1 ==== 80+0+0 (secure 0 0 0) 0x7f56240581d0 con 0x7f56340cd530 2023-11-24T06:11:22.605 INFO:teuthology.orchestra.run.smithi092.stderr:listed 0 entries 2023-11-24T06:11:22.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:22.605+0000 7f5633fff640 1 -- 172.21.15.92:0/2476051014 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f562006ae40 msgr2=0x7f562006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:22.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:22.605+0000 7f5633fff640 1 --2- 172.21.15.92:0/2476051014 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f562006ae40 0x7f562006d300 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f5628036000 tx=0x7f5628030000 comp rx=0 tx=0).stop 2023-11-24T06:11:22.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:22.605+0000 7f5633fff640 1 -- 172.21.15.92:0/2476051014 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f56340cd530 msgr2=0x7f5634159970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:22.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:22.605+0000 7f5633fff640 1 --2- 172.21.15.92:0/2476051014 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f56340cd530 0x7f5634159970 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f562400d5f0 tx=0x7f562400dab0 comp rx=0 tx=0).stop 2023-11-24T06:11:22.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:22.605+0000 7f5633fff640 1 -- 172.21.15.92:0/2476051014 shutdown_connections 2023-11-24T06:11:22.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:22.605+0000 7f5633fff640 1 --2- 172.21.15.92:0/2476051014 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f562006ae40 0x7f562006d300 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:22.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:22.605+0000 7f5633fff640 1 --2- 172.21.15.92:0/2476051014 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f563413cfa0 0x7f5634155db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:22.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:22.605+0000 7f5633fff640 1 --2- 172.21.15.92:0/2476051014 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f56340cd9b0 0x7f563414f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:22.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:22.605+0000 7f5633fff640 1 --2- 172.21.15.92:0/2476051014 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f56340cd530 0x7f5634159970 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:22.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:22.605+0000 7f5633fff640 1 -- 172.21.15.92:0/2476051014 >> 172.21.15.92:0/2476051014 conn(0x7f563406a800 msgr2=0x7f56340af250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:22.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:22.605+0000 7f5633fff640 1 -- 172.21.15.92:0/2476051014 shutdown_connections 2023-11-24T06:11:22.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:22.605+0000 7f5633fff640 1 -- 172.21.15.92:0/2476051014 wait complete. 2023-11-24T06:11:22.623 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi092.front.sepia.ceph.com... 2023-11-24T06:11:22.623 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:11:22.624 DEBUG:teuthology.orchestra.run.smithi092:> sudo fusermount -u /home/ubuntu/cephtest/mnt.2 2023-11-24T06:11:22.709 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T06:11:22.709 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi092.front.sepia.ceph.com 2023-11-24T06:11:22.710 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:11:22.710 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-11-24T06:11:22.759 DEBUG:teuthology.orchestra.run.smithi118:> set -ex 2023-11-24T06:11:22.759 DEBUG:teuthology.orchestra.run.smithi118:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:11:22.820 DEBUG:teuthology.orchestra.run.smithi118:> set -ex 2023-11-24T06:11:22.820 DEBUG:teuthology.orchestra.run.smithi118:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:11:22.876 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:11:23.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.094+0000 7ffab571d640 1 Processor -- start 2023-11-24T06:11:23.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.094+0000 7ffab571d640 1 -- start start 2023-11-24T06:11:23.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.096+0000 7ffab571d640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ffab00ae5a0 0x7ffab015e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:23.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.096+0000 7ffab571d640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ffab00b3ae0 0x7ffab006cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:23.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.096+0000 7ffab571d640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7ffab006d120 0x7ffab00cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:23.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.096+0000 7ffab571d640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffab0071450 con 0x7ffab00ae5a0 2023-11-24T06:11:23.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.096+0000 7ffab571d640 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffab0070c40 con 0x7ffab006d120 2023-11-24T06:11:23.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.096+0000 7ffab571d640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffab006c470 con 0x7ffab00b3ae0 2023-11-24T06:11:23.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.097+0000 7ffaaeffd640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ffab00ae5a0 0x7ffab015e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:23.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.097+0000 7ffaaeffd640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ffab00ae5a0 0x7ffab015e430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45128/0 (socket says 172.21.15.92:45128) 2023-11-24T06:11:23.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.097+0000 7ffaaeffd640 1 -- 172.21.15.92:0/3622798032 learned_addr learned my addr 172.21.15.92:0/3622798032 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:11:23.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.097+0000 7ffaa67fc640 1 --2- 172.21.15.92:0/3622798032 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ffab00b3ae0 0x7ffab006cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:23.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.097+0000 7ffaaf7fe640 1 --2- 172.21.15.92:0/3622798032 >> v2:172.21.15.118:3300/0 conn(0x7ffab006d120 0x7ffab00cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:23.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.097+0000 7ffaaeffd640 1 -- 172.21.15.92:0/3622798032 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ffab00b3ae0 msgr2=0x7ffab006cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:23.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.097+0000 7ffaaeffd640 1 --2- 172.21.15.92:0/3622798032 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ffab00b3ae0 0x7ffab006cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:23.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.097+0000 7ffaaeffd640 1 -- 172.21.15.92:0/3622798032 >> v2:172.21.15.118:3300/0 conn(0x7ffab006d120 msgr2=0x7ffab00cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:23.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.097+0000 7ffaaeffd640 1 --2- 172.21.15.92:0/3622798032 >> v2:172.21.15.118:3300/0 conn(0x7ffab006d120 0x7ffab00cf290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:23.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.097+0000 7ffaaeffd640 1 -- 172.21.15.92:0/3622798032 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffab006c750 con 0x7ffab00ae5a0 2023-11-24T06:11:23.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.099+0000 7ffaaeffd640 1 --2- 172.21.15.92:0/3622798032 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffab00ae5a0 0x7ffab015e430 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7ffab006e230 tx=0x7ffaa002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be886fc364a429a4 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:23.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.099+0000 7ffaae7fc640 1 -- 172.21.15.92:0/3622798032 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaa0031870 con 0x7ffab00ae5a0 2023-11-24T06:11:23.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.099+0000 7ffaae7fc640 1 -- 172.21.15.92:0/3622798032 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffaa00319d0 con 0x7ffab00ae5a0 2023-11-24T06:11:23.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.099+0000 7ffaae7fc640 1 -- 172.21.15.92:0/3622798032 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaa0031d60 con 0x7ffab00ae5a0 2023-11-24T06:11:23.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.100+0000 7ffab571d640 1 -- 172.21.15.92:0/3622798032 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffab00ae5a0 msgr2=0x7ffab015e430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:23.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.100+0000 7ffab571d640 1 --2- 172.21.15.92:0/3622798032 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffab00ae5a0 0x7ffab015e430 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7ffab006e230 tx=0x7ffaa002b910 comp rx=0 tx=0).stop 2023-11-24T06:11:23.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.100+0000 7ffab571d640 1 -- 172.21.15.92:0/3622798032 shutdown_connections 2023-11-24T06:11:23.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.100+0000 7ffab571d640 1 --2- 172.21.15.92:0/3622798032 >> v2:172.21.15.118:3300/0 conn(0x7ffab006d120 0x7ffab00cf290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:23.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.100+0000 7ffab571d640 1 --2- 172.21.15.92:0/3622798032 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ffab00b3ae0 0x7ffab006cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:23.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.100+0000 7ffab571d640 1 --2- 172.21.15.92:0/3622798032 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffab00ae5a0 0x7ffab015e430 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:23.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.100+0000 7ffab571d640 1 -- 172.21.15.92:0/3622798032 >> 172.21.15.92:0/3622798032 conn(0x7ffab006a800 msgr2=0x7ffab006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:23.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.100+0000 7ffab571d640 1 -- 172.21.15.92:0/3622798032 shutdown_connections 2023-11-24T06:11:23.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.100+0000 7ffab571d640 1 -- 172.21.15.92:0/3622798032 wait complete. 2023-11-24T06:11:23.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.102+0000 7ffab571d640 1 Processor -- start 2023-11-24T06:11:23.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.102+0000 7ffab571d640 1 -- start start 2023-11-24T06:11:23.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.103+0000 7ffab571d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffab006d120 0x7ffab014a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:23.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.103+0000 7ffab571d640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ffab00ae5a0 0x7ffab00b2f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:23.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.103+0000 7ffab571d640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ffab00b3ae0 0x7ffab00cd6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:23.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.103+0000 7ffab571d640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffab00b3440 con 0x7ffab006d120 2023-11-24T06:11:23.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.103+0000 7ffab571d640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffab00cdbe0 con 0x7ffab00ae5a0 2023-11-24T06:11:23.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.103+0000 7ffab571d640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffab00cde30 con 0x7ffab00b3ae0 2023-11-24T06:11:23.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.103+0000 7ffaaeffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffab006d120 0x7ffab014a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:23.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.103+0000 7ffaaeffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffab006d120 0x7ffab014a780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45138/0 (socket says 172.21.15.92:45138) 2023-11-24T06:11:23.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.103+0000 7ffaaeffd640 1 -- 172.21.15.92:0/2040273395 learned_addr learned my addr 172.21.15.92:0/2040273395 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:11:23.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.103+0000 7ffaaf7fe640 1 --2- 172.21.15.92:0/2040273395 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ffab00b3ae0 0x7ffab00cd6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:23.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.103+0000 7ffaa67fc640 1 --2- 172.21.15.92:0/2040273395 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ffab00ae5a0 0x7ffab00b2f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:23.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.103+0000 7ffaaeffd640 1 -- 172.21.15.92:0/2040273395 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ffab00b3ae0 msgr2=0x7ffab00cd6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:23.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.103+0000 7ffaaeffd640 1 --2- 172.21.15.92:0/2040273395 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ffab00b3ae0 0x7ffab00cd6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:23.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.103+0000 7ffaaeffd640 1 -- 172.21.15.92:0/2040273395 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ffab00ae5a0 msgr2=0x7ffab00b2f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:23.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.103+0000 7ffaaeffd640 1 --2- 172.21.15.92:0/2040273395 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ffab00ae5a0 0x7ffab00b2f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:23.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.103+0000 7ffaaeffd640 1 -- 172.21.15.92:0/2040273395 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffaa00094d0 con 0x7ffab006d120 2023-11-24T06:11:23.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.104+0000 7ffaa67fc640 1 --2- 172.21.15.92:0/2040273395 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ffab00ae5a0 0x7ffab00b2f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:11:23.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.104+0000 7ffaaeffd640 1 --2- 172.21.15.92:0/2040273395 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffab006d120 0x7ffab014a780 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7ffaa0000c00 tx=0x7ffaa00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:23.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.104+0000 7ffaa7fff640 1 -- 172.21.15.92:0/2040273395 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaa003abe0 con 0x7ffab006d120 2023-11-24T06:11:23.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.104+0000 7ffaa7fff640 1 -- 172.21.15.92:0/2040273395 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffaa003ad40 con 0x7ffab006d120 2023-11-24T06:11:23.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.104+0000 7ffab571d640 1 -- 172.21.15.92:0/2040273395 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffab00ce110 con 0x7ffab006d120 2023-11-24T06:11:23.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.104+0000 7ffaa7fff640 1 -- 172.21.15.92:0/2040273395 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaa0044610 con 0x7ffab006d120 2023-11-24T06:11:23.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.104+0000 7ffab571d640 1 -- 172.21.15.92:0/2040273395 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffab00ce6d0 con 0x7ffab006d120 2023-11-24T06:11:23.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.105+0000 7ffab571d640 1 -- 172.21.15.92:0/2040273395 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffab00675d0 con 0x7ffab006d120 2023-11-24T06:11:23.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.106+0000 7ffaa7fff640 1 -- 172.21.15.92:0/2040273395 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ffaa004d450 con 0x7ffab006d120 2023-11-24T06:11:23.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.107+0000 7ffaa7fff640 1 --2- 172.21.15.92:0/2040273395 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7ffa9006ae40 0x7ffa9006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:23.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.107+0000 7ffaa7fff640 1 -- 172.21.15.92:0/2040273395 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5883+0+0 (secure 0 0 0) 0x7ffaa002d040 con 0x7ffab006d120 2023-11-24T06:11:23.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.107+0000 7ffaa67fc640 1 --2- 172.21.15.92:0/2040273395 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7ffa9006ae40 0x7ffa9006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:23.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.108+0000 7ffaa67fc640 1 --2- 172.21.15.92:0/2040273395 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7ffa9006ae40 0x7ffa9006d300 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7ffa98006fd0 tx=0x7ffa98008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:23.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.112+0000 7ffaa7fff640 1 -- 172.21.15.92:0/2040273395 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ffaa0076d10 con 0x7ffab006d120 2023-11-24T06:11:23.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.301+0000 7ffab571d640 1 -- 172.21.15.92:0/2040273395 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ffab006b3f0 con 0x7ffab006d120 2023-11-24T06:11:23.305 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.304+0000 7ffaa7fff640 1 -- 172.21.15.92:0/2040273395 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16800 (secure 0 0 0) 0x7ffaa007a330 con 0x7ffab006d120 2023-11-24T06:11:23.306 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-24T06:11:23.306 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":22,"fsid":"2d6dee5f-8aa3-470f-931a-50ade963b1cb","created":"2023-11-24T06:10:04.707204+0000","modified":"2023-11-24T06:11:11.472401+0000","last_up_change":"2023-11-24T06:10:17.598352+0000","last_in_change":"2023-11-24T06:10:09.040793+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:10:17.733098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:10:22.926445+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T06:10:24.151288+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ff37a9fb-135f-471f-b4b0-f7b5772006c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6803","nonce":20560714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6805","nonce":20560714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6809","nonce":20560714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6807","nonce":20560714}]},"public_addr":"172.21.15.92:6803/20560714","cluster_addr":"172.21.15.92:6805/20560714","heartbeat_back_addr":"172.21.15.92:6809/20560714","heartbeat_front_addr":"172.21.15.92:6807/20560714","state":["exists","up"]},{"osd":1,"uuid":"03cdd353-dcf1-4f38-94fe-a795137c06b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6819","nonce":2451239906}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6821","nonce":2451239906}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6825","nonce":2451239906}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6823","nonce":2451239906}]},"public_addr":"172.21.15.92:6819/2451239906","cluster_addr":"172.21.15.92:6821/2451239906","heartbeat_back_addr":"172.21.15.92:6825/2451239906","heartbeat_front_addr":"172.21.15.92:6823/2451239906","state":["exists","up"]},{"osd":2,"uuid":"a4a20d80-5d57-484b-becd-9aa8070dd3aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6811","nonce":1124979277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6813","nonce":1124979277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6817","nonce":1124979277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6815","nonce":1124979277}]},"public_addr":"172.21.15.92:6811/1124979277","cluster_addr":"172.21.15.92:6813/1124979277","heartbeat_back_addr":"172.21.15.92:6817/1124979277","heartbeat_front_addr":"172.21.15.92:6815/1124979277","state":["exists","up"]},{"osd":3,"uuid":"4648d7ce-9b29-4583-9425-4ba8fd2b1903","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6827","nonce":1734115782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6829","nonce":1734115782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6833","nonce":1734115782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6831","nonce":1734115782}]},"public_addr":"172.21.15.92:6827/1734115782","cluster_addr":"172.21.15.92:6829/1734115782","heartbeat_back_addr":"172.21.15.92:6833/1734115782","heartbeat_front_addr":"172.21.15.92:6831/1734115782","state":["exists","up"]},{"osd":4,"uuid":"3bc9d7eb-c6fa-4fcf-a3ec-ade604e05aac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6819","nonce":3425937229}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6822","nonce":3425937229}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6830","nonce":3425937229}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6826","nonce":3425937229}]},"public_addr":"172.21.15.118:6819/3425937229","cluster_addr":"172.21.15.118:6822/3425937229","heartbeat_back_addr":"172.21.15.118:6830/3425937229","heartbeat_front_addr":"172.21.15.118:6826/3425937229","state":["exists","up"]},{"osd":5,"uuid":"dce9ba6d-59ed-46d1-84fc-a84b13de10c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6801","nonce":827455984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6803","nonce":827455984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6807","nonce":827455984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6805","nonce":827455984}]},"public_addr":"172.21.15.118:6801/827455984","cluster_addr":"172.21.15.118:6803/827455984","heartbeat_back_addr":"172.21.15.118:6807/827455984","heartbeat_front_addr":"172.21.15.118:6805/827455984","state":["exists","up"]},{"osd":6,"uuid":"82aec504-ae58-4709-8d30-46f2b12081b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6809","nonce":2946986141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6811","nonce":2946986141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6815","nonce":2946986141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6813","nonce":2946986141}]},"public_addr":"172.21.15.118:6809/2946986141","cluster_addr":"172.21.15.118:6811/2946986141","heartbeat_back_addr":"172.21.15.118:6815/2946986141","heartbeat_front_addr":"172.21.15.118:6813/2946986141","state":["exists","up"]},{"osd":7,"uuid":"4ed4c624-c88f-4d35-bdb1-bf209b5dffb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6817","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6818","nonce":48122059}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6821","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6823","nonce":48122059}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6829","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6831","nonce":48122059}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6825","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6827","nonce":48122059}]},"public_addr":"172.21.15.118:6818/48122059","cluster_addr":"172.21.15.118:6823/48122059","heartbeat_back_addr":"172.21.15.118:6831/48122059","heartbeat_front_addr":"172.21.15.118:6827/48122059","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.536664+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.368324+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.271201+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.342971+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.621832+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.387110+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.632946+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.416032+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.d","mappings":[{"from":5,"to":1}]},{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"3.29","mappings":[{"from":3,"to":2}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.34","mappings":[{"from":4,"to":2}]},{"pgid":"3.3b","mappings":[{"from":7,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:11:23.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.306+0000 7ffab571d640 1 -- 172.21.15.92:0/2040273395 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7ffa9006ae40 msgr2=0x7ffa9006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:23.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.306+0000 7ffab571d640 1 --2- 172.21.15.92:0/2040273395 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7ffa9006ae40 0x7ffa9006d300 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7ffa98006fd0 tx=0x7ffa98008040 comp rx=0 tx=0).stop 2023-11-24T06:11:23.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.306+0000 7ffab571d640 1 -- 172.21.15.92:0/2040273395 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffab006d120 msgr2=0x7ffab014a780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:23.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.306+0000 7ffab571d640 1 --2- 172.21.15.92:0/2040273395 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffab006d120 0x7ffab014a780 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7ffaa0000c00 tx=0x7ffaa00046c0 comp rx=0 tx=0).stop 2023-11-24T06:11:23.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.306+0000 7ffab571d640 1 -- 172.21.15.92:0/2040273395 shutdown_connections 2023-11-24T06:11:23.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.306+0000 7ffab571d640 1 --2- 172.21.15.92:0/2040273395 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7ffa9006ae40 0x7ffa9006d300 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:23.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.306+0000 7ffab571d640 1 --2- 172.21.15.92:0/2040273395 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ffab00b3ae0 0x7ffab00cd6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:23.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.306+0000 7ffab571d640 1 --2- 172.21.15.92:0/2040273395 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ffab00ae5a0 0x7ffab00b2f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:23.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.306+0000 7ffab571d640 1 --2- 172.21.15.92:0/2040273395 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffab006d120 0x7ffab014a780 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:23.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.306+0000 7ffab571d640 1 -- 172.21.15.92:0/2040273395 >> 172.21.15.92:0/2040273395 conn(0x7ffab006a800 msgr2=0x7ffab015a9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:23.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.306+0000 7ffab571d640 1 -- 172.21.15.92:0/2040273395 shutdown_connections 2023-11-24T06:11:23.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.306+0000 7ffab571d640 1 -- 172.21.15.92:0/2040273395 wait complete. 2023-11-24T06:11:23.328 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T06:10:17.733098+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T06:10:22.926445+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T06:10:24.151288+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T06:11:23.328 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T06:11:23.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.538+0000 7f7175fc5640 1 Processor -- start 2023-11-24T06:11:23.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.538+0000 7f7175fc5640 1 -- start start 2023-11-24T06:11:23.542 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.539+0000 7f7175fc5640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f717013e360 0x7f71700cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:23.542 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.539+0000 7f7175fc5640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f717015f9c0 0x7f71700cf7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:23.543 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.539+0000 7f7175fc5640 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f717015c5f0 con 0x7f717013df80 2023-11-24T06:11:23.543 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.539+0000 7f7175fc5640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f717006c470 con 0x7f717015f9c0 2023-11-24T06:11:23.543 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.539+0000 7f7175fc5640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f717006c750 con 0x7f717013e360 2023-11-24T06:11:23.543 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.540+0000 7f716ffff640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f717015f9c0 0x7f71700cf7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:23.543 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.540+0000 7f716ffff640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f717015f9c0 0x7f71700cf7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45148/0 (socket says 172.21.15.92:45148) 2023-11-24T06:11:23.543 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.540+0000 7f716ffff640 1 -- 172.21.15.92:0/4267927148 learned_addr learned my addr 172.21.15.92:0/4267927148 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:11:23.543 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.540+0000 7f7167fff640 1 --2- 172.21.15.92:0/4267927148 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f717013e360 0x7f71700cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:23.543 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.540+0000 7f716ffff640 1 -- 172.21.15.92:0/4267927148 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f717013e360 msgr2=0x7f71700cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:23.543 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.540+0000 7f716ffff640 1 --2- 172.21.15.92:0/4267927148 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f717013e360 0x7f71700cf290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:23.543 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.540+0000 7f716ffff640 1 -- 172.21.15.92:0/4267927148 >> v1:172.21.15.118:6789/0 conn(0x7f717013df80 legacy=0x7f717015bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:23.543 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.540+0000 7f716ffff640 1 -- 172.21.15.92:0/4267927148 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f717006ca30 con 0x7f717015f9c0 2023-11-24T06:11:23.544 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.540+0000 7f7167fff640 1 --2- 172.21.15.92:0/4267927148 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f717013e360 0x7f71700cf290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:11:23.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.541+0000 7f716ffff640 1 --2- 172.21.15.92:0/4267927148 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f717015f9c0 0x7f71700cf7d0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f7168009e50 tx=0x7f716802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6f39f994d1b1bd9 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:23.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.541+0000 7f716effd640 1 -- 172.21.15.92:0/4267927148 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f716802ca70 con 0x7f717015f9c0 2023-11-24T06:11:23.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.542+0000 7f716effd640 1 -- 172.21.15.92:0/4267927148 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f716802cbd0 con 0x7f717015f9c0 2023-11-24T06:11:23.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.542+0000 7f716effd640 1 -- 172.21.15.92:0/4267927148 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71680343f0 con 0x7f717015f9c0 2023-11-24T06:11:23.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.542+0000 7f7175fc5640 1 -- 172.21.15.92:0/4267927148 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f717015f9c0 msgr2=0x7f71700cf7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:23.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.542+0000 7f7175fc5640 1 --2- 172.21.15.92:0/4267927148 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f717015f9c0 0x7f71700cf7d0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f7168009e50 tx=0x7f716802bed0 comp rx=0 tx=0).stop 2023-11-24T06:11:23.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.542+0000 7f7175fc5640 1 -- 172.21.15.92:0/4267927148 shutdown_connections 2023-11-24T06:11:23.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.542+0000 7f7175fc5640 1 --2- 172.21.15.92:0/4267927148 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f717015f9c0 0x7f71700cf7d0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:23.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.542+0000 7f7175fc5640 1 --2- 172.21.15.92:0/4267927148 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f717013e360 0x7f71700cf290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:23.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.542+0000 7f7175fc5640 1 -- 172.21.15.92:0/4267927148 >> 172.21.15.92:0/4267927148 conn(0x7f717006a800 msgr2=0x7f717006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:23.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.542+0000 7f7175fc5640 1 -- 172.21.15.92:0/4267927148 shutdown_connections 2023-11-24T06:11:23.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.542+0000 7f7175fc5640 1 -- 172.21.15.92:0/4267927148 wait complete. 2023-11-24T06:11:23.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.544+0000 7f7175fc5640 1 Processor -- start 2023-11-24T06:11:23.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.544+0000 7f7175fc5640 1 -- start start 2023-11-24T06:11:23.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.545+0000 7f7175fc5640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f717013df80 0x7f717014e110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:23.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.545+0000 7f7175fc5640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f717013e360 0x7f71700b2f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:23.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.545+0000 7f7175fc5640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f717015f9c0 0x7f717013d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:23.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.545+0000 7f7175fc5640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71701f6950 con 0x7f717013df80 2023-11-24T06:11:23.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.545+0000 7f7175fc5640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71701f6340 con 0x7f717013e360 2023-11-24T06:11:23.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.545+0000 7f7175fc5640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f717014f280 con 0x7f717015f9c0 2023-11-24T06:11:23.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.545+0000 7f716f7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f717013df80 0x7f717014e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:23.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.545+0000 7f716f7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f717013df80 0x7f717014e110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45154/0 (socket says 172.21.15.92:45154) 2023-11-24T06:11:23.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.545+0000 7f716f7fe640 1 -- 172.21.15.92:0/1459825265 learned_addr learned my addr 172.21.15.92:0/1459825265 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:11:23.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.546+0000 7f716f7fe640 1 -- 172.21.15.92:0/1459825265 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f717015f9c0 msgr2=0x7f717013d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:23.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.546+0000 7f716f7fe640 1 --2- 172.21.15.92:0/1459825265 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f717015f9c0 0x7f717013d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:23.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.546+0000 7f716f7fe640 1 -- 172.21.15.92:0/1459825265 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f717013e360 msgr2=0x7f71700b2f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:23.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.546+0000 7f716f7fe640 1 --2- 172.21.15.92:0/1459825265 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f717013e360 0x7f71700b2f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:23.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.546+0000 7f716f7fe640 1 -- 172.21.15.92:0/1459825265 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7168009b30 con 0x7f717013df80 2023-11-24T06:11:23.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.546+0000 7f716f7fe640 1 --2- 172.21.15.92:0/1459825265 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f717013df80 0x7f717014e110 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f716000b3f0 tx=0x7f716000b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:23.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.546+0000 7f716cff9640 1 -- 172.21.15.92:0/1459825265 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7160011830 con 0x7f717013df80 2023-11-24T06:11:23.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.546+0000 7f716cff9640 1 -- 172.21.15.92:0/1459825265 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7160011990 con 0x7f717013df80 2023-11-24T06:11:23.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.546+0000 7f716cff9640 1 -- 172.21.15.92:0/1459825265 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7160011d40 con 0x7f717013df80 2023-11-24T06:11:23.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.546+0000 7f7175fc5640 1 -- 172.21.15.92:0/1459825265 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f717014efa0 con 0x7f717013df80 2023-11-24T06:11:23.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.546+0000 7f7175fc5640 1 -- 172.21.15.92:0/1459825265 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f717014e650 con 0x7f717013df80 2023-11-24T06:11:23.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.548+0000 7f7175fc5640 1 -- 172.21.15.92:0/1459825265 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71701f61a0 con 0x7f717013df80 2023-11-24T06:11:23.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.549+0000 7f716cff9640 1 -- 172.21.15.92:0/1459825265 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f716001a400 con 0x7f717013df80 2023-11-24T06:11:23.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.549+0000 7f716cff9640 1 --2- 172.21.15.92:0/1459825265 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f715006ae40 0x7f715006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:23.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.550+0000 7f716cff9640 1 -- 172.21.15.92:0/1459825265 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5883+0+0 (secure 0 0 0) 0x7f7160089500 con 0x7f717013df80 2023-11-24T06:11:23.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.550+0000 7f7167fff640 1 --2- 172.21.15.92:0/1459825265 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f715006ae40 0x7f715006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:23.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.550+0000 7f7167fff640 1 --2- 172.21.15.92:0/1459825265 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f715006ae40 0x7f715006d300 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f7158006fd0 tx=0x7f7158008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:23.556 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.554+0000 7f716cff9640 1 -- 172.21.15.92:0/1459825265 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f71701f61a0 con 0x7f717013df80 2023-11-24T06:11:23.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.745+0000 7f7175fc5640 1 -- 172.21.15.92:0/1459825265 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f71701f61a0 con 0x7f717013df80 2023-11-24T06:11:23.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.746+0000 7f716cff9640 1 -- 172.21.15.92:0/1459825265 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7f71701f61a0 con 0x7f717013df80 2023-11-24T06:11:23.748 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 1 2023-11-24T06:11:23.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.749+0000 7f7175fc5640 1 -- 172.21.15.92:0/1459825265 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f715006ae40 msgr2=0x7f715006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:23.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.749+0000 7f7175fc5640 1 --2- 172.21.15.92:0/1459825265 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f715006ae40 0x7f715006d300 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f7158006fd0 tx=0x7f7158008040 comp rx=0 tx=0).stop 2023-11-24T06:11:23.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.749+0000 7f7175fc5640 1 -- 172.21.15.92:0/1459825265 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f717013df80 msgr2=0x7f717014e110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:23.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.749+0000 7f7175fc5640 1 --2- 172.21.15.92:0/1459825265 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f717013df80 0x7f717014e110 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f716000b3f0 tx=0x7f716000b8b0 comp rx=0 tx=0).stop 2023-11-24T06:11:23.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.749+0000 7f7175fc5640 1 -- 172.21.15.92:0/1459825265 shutdown_connections 2023-11-24T06:11:23.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.749+0000 7f7175fc5640 1 --2- 172.21.15.92:0/1459825265 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f715006ae40 0x7f715006d300 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:23.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.749+0000 7f7175fc5640 1 --2- 172.21.15.92:0/1459825265 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f717015f9c0 0x7f717013d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:23.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.749+0000 7f7175fc5640 1 --2- 172.21.15.92:0/1459825265 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f717013e360 0x7f71700b2f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:23.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.749+0000 7f7175fc5640 1 --2- 172.21.15.92:0/1459825265 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f717013df80 0x7f717014e110 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:23.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.749+0000 7f7175fc5640 1 -- 172.21.15.92:0/1459825265 >> 172.21.15.92:0/1459825265 conn(0x7f717006a800 msgr2=0x7f717015d060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:23.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.750+0000 7f7175fc5640 1 -- 172.21.15.92:0/1459825265 shutdown_connections 2023-11-24T06:11:23.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.750+0000 7f7175fc5640 1 -- 172.21.15.92:0/1459825265 wait complete. 2023-11-24T06:11:23.771 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T06:11:23.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.972+0000 7f42e4690640 1 Processor -- start 2023-11-24T06:11:23.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.973+0000 7f42e4690640 1 -- start start 2023-11-24T06:11:23.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.974+0000 7f42e4690640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f42dc0d3910 0x7f42dc0d1100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:23.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.974+0000 7f42e4690640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f42dc160e10 0x7f42dc0d1640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:23.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.974+0000 7f42e4690640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42dc0d4160 con 0x7f42dc0cfcf0 2023-11-24T06:11:23.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.974+0000 7f42e4690640 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42dc0ceff0 con 0x7f42dc160e10 2023-11-24T06:11:23.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.974+0000 7f42e4690640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42dc0cf410 con 0x7f42dc0d3910 2023-11-24T06:11:23.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.974+0000 7f42e242c640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f42dc0cfcf0 0x7f42dc160700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:56240/0 (socket says 172.21.15.92:56240) 2023-11-24T06:11:23.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.974+0000 7f42e242c640 1 -- 172.21.15.92:0/1720267886 learned_addr learned my addr 172.21.15.92:0/1720267886 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-24T06:11:23.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.975+0000 7f42e142a640 1 -- 172.21.15.92:0/1720267886 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2831344033 0 0) 0x7f42dc0d4160 con 0x7f42dc0cfcf0 2023-11-24T06:11:23.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.975+0000 7f42e142a640 1 -- 172.21.15.92:0/1720267886 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f42c8003520 con 0x7f42dc0cfcf0 2023-11-24T06:11:23.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.975+0000 7f42e2c2d640 1 --2- 172.21.15.92:0/1720267886 >> v2:172.21.15.118:3300/0 conn(0x7f42dc160e10 0x7f42dc0d1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:23.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.975+0000 7f42e1c2b640 1 --2- 172.21.15.92:0/1720267886 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f42dc0d3910 0x7f42dc0d1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:23.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.975+0000 7f42e142a640 1 -- 172.21.15.92:0/1720267886 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1821223497 0 0) 0x7f42c8003520 con 0x7f42dc0cfcf0 2023-11-24T06:11:23.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.975+0000 7f42e142a640 1 -- 172.21.15.92:0/1720267886 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f42dc0d3910 msgr2=0x7f42dc0d1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:23.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.975+0000 7f42e142a640 1 --2- 172.21.15.92:0/1720267886 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f42dc0d3910 0x7f42dc0d1100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:23.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.975+0000 7f42e142a640 1 -- 172.21.15.92:0/1720267886 >> v2:172.21.15.118:3300/0 conn(0x7f42dc160e10 msgr2=0x7f42dc0d1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:23.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.975+0000 7f42e142a640 1 --2- 172.21.15.92:0/1720267886 >> v2:172.21.15.118:3300/0 conn(0x7f42dc160e10 0x7f42dc0d1640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:23.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.975+0000 7f42e142a640 1 -- 172.21.15.92:0/1720267886 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42dc0b2420 con 0x7f42dc0cfcf0 2023-11-24T06:11:23.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.975+0000 7f42e142a640 1 -- 172.21.15.92:0/1720267886 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2607661350 0 0) 0x7f42d8001670 con 0x7f42dc0cfcf0 2023-11-24T06:11:23.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.975+0000 7f42e142a640 1 -- 172.21.15.92:0/1720267886 >> v1:172.21.15.92:6789/0 conn(0x7f42dc0cfcf0 legacy=0x7f42dc160700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:23.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.976+0000 7f42e142a640 1 --2- 172.21.15.92:0/1720267886 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f42c80039a0 0x7f42c8003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:23.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.976+0000 7f42e142a640 1 --2- 172.21.15.92:0/1720267886 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f42c80047c0 0x7f42c8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:23.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.976+0000 7f42e142a640 1 --2- 172.21.15.92:0/1720267886 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f42dc0d3910 0x7f42c8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:23.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.976+0000 7f42e142a640 1 -- 172.21.15.92:0/1720267886 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42dc0ceff0 con 0x7f42c80039a0 2023-11-24T06:11:23.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.976+0000 7f42e142a640 1 -- 172.21.15.92:0/1720267886 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42dc0cf410 con 0x7f42c80047c0 2023-11-24T06:11:23.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.976+0000 7f42e142a640 1 -- 172.21.15.92:0/1720267886 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42c800bb00 con 0x7f42dc0d3910 2023-11-24T06:11:23.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.976+0000 7f42e242c640 1 --2- 172.21.15.92:0/1720267886 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f42c80039a0 0x7f42c8003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:23.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.976+0000 7f42e1c2b640 1 --2- 172.21.15.92:0/1720267886 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f42c80047c0 0x7f42c8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:23.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.976+0000 7f42e2c2d640 1 --2- 172.21.15.92:0/1720267886 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f42dc0d3910 0x7f42c8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:23.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.976+0000 7f42e242c640 1 -- 172.21.15.92:0/1720267886 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f42dc0d3910 msgr2=0x7f42c8005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:23.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.976+0000 7f42e242c640 1 --2- 172.21.15.92:0/1720267886 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f42dc0d3910 0x7f42c8005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:23.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.976+0000 7f42e242c640 1 -- 172.21.15.92:0/1720267886 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f42c80047c0 msgr2=0x7f42c8004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:23.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.976+0000 7f42e242c640 1 --2- 172.21.15.92:0/1720267886 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f42c80047c0 0x7f42c8004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:23.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.976+0000 7f42e242c640 1 -- 172.21.15.92:0/1720267886 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42c8003520 con 0x7f42c80039a0 2023-11-24T06:11:23.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.977+0000 7f42e242c640 1 --2- 172.21.15.92:0/1720267886 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f42c80039a0 0x7f42c8003dc0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f42d800c240 tx=0x7f42c80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=663c2a99d7328dfb server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:23.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.978+0000 7f42e142a640 1 -- 172.21.15.92:0/1720267886 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42d8005ad0 con 0x7f42c80039a0 2023-11-24T06:11:23.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.978+0000 7f42e142a640 1 -- 172.21.15.92:0/1720267886 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42d8005c30 con 0x7f42c80039a0 2023-11-24T06:11:23.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.978+0000 7f42e142a640 1 -- 172.21.15.92:0/1720267886 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42d802c8b0 con 0x7f42c80039a0 2023-11-24T06:11:23.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.978+0000 7f42e4690640 1 -- 172.21.15.92:0/1720267886 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f42c80039a0 msgr2=0x7f42c8003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:23.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.978+0000 7f42e4690640 1 --2- 172.21.15.92:0/1720267886 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f42c80039a0 0x7f42c8003dc0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f42d800c240 tx=0x7f42c80057f0 comp rx=0 tx=0).stop 2023-11-24T06:11:23.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.978+0000 7f42e2c2d640 1 -- 172.21.15.92:0/1720267886 reap_dead start 2023-11-24T06:11:23.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.979+0000 7f42e4690640 1 -- 172.21.15.92:0/1720267886 shutdown_connections 2023-11-24T06:11:23.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.979+0000 7f42e4690640 1 -- 172.21.15.92:0/1720267886 >> 172.21.15.92:0/1720267886 conn(0x7f42dc09c530 msgr2=0x7f42dc0d3200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:23.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.979+0000 7f42e4690640 1 -- 172.21.15.92:0/1720267886 shutdown_connections 2023-11-24T06:11:23.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.979+0000 7f42e4690640 1 -- 172.21.15.92:0/1720267886 wait complete. 2023-11-24T06:11:23.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.980+0000 7f42e4690640 1 Processor -- start 2023-11-24T06:11:23.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.981+0000 7f42e4690640 1 -- start start 2023-11-24T06:11:23.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.982+0000 7f42e4690640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f42c80039a0 0x7f42dc14f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:23.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.982+0000 7f42e4690640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f42dc0c6ca0 0x7f42dc0b8030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:23.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.982+0000 7f42e4690640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f42dc0c82c0 0x7f42dc0b8570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:23.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.982+0000 7f42e4690640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42dc158820 con 0x7f42dc0c82c0 2023-11-24T06:11:23.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.982+0000 7f42e4690640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42dc0c7220 con 0x7f42dc0c6ca0 2023-11-24T06:11:23.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.982+0000 7f42e4690640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42dc0c7500 con 0x7f42c80039a0 2023-11-24T06:11:23.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.982+0000 7f42e2c2d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f42dc0c82c0 0x7f42dc0b8570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:23.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.982+0000 7f42e2c2d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f42dc0c82c0 0x7f42dc0b8570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45164/0 (socket says 172.21.15.92:45164) 2023-11-24T06:11:23.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.982+0000 7f42e2c2d640 1 -- 172.21.15.92:0/1469574984 learned_addr learned my addr 172.21.15.92:0/1469574984 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:11:23.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.982+0000 7f42e242c640 1 --2- 172.21.15.92:0/1469574984 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f42c80039a0 0x7f42dc14f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:23.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.982+0000 7f42e2c2d640 1 -- 172.21.15.92:0/1469574984 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f42c80039a0 msgr2=0x7f42dc14f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:23.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.982+0000 7f42e2c2d640 1 --2- 172.21.15.92:0/1469574984 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f42c80039a0 0x7f42dc14f4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:23.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.982+0000 7f42e2c2d640 1 -- 172.21.15.92:0/1469574984 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f42dc0c6ca0 msgr2=0x7f42dc0b8030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:23.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.982+0000 7f42e2c2d640 1 --2- 172.21.15.92:0/1469574984 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f42dc0c6ca0 0x7f42dc0b8030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:23.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.982+0000 7f42e2c2d640 1 -- 172.21.15.92:0/1469574984 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42dc0c77e0 con 0x7f42dc0c82c0 2023-11-24T06:11:23.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.983+0000 7f42e242c640 1 --2- 172.21.15.92:0/1469574984 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f42c80039a0 0x7f42dc14f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:11:23.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.983+0000 7f42e2c2d640 1 --2- 172.21.15.92:0/1469574984 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f42dc0c82c0 0x7f42dc0b8570 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f42dc08b880 tx=0x7f42d4009d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:23.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.983+0000 7f42d2ffd640 1 -- 172.21.15.92:0/1469574984 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42d4007cf0 con 0x7f42dc0c82c0 2023-11-24T06:11:23.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.983+0000 7f42d2ffd640 1 -- 172.21.15.92:0/1469574984 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42d4007e70 con 0x7f42dc0c82c0 2023-11-24T06:11:23.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.983+0000 7f42d2ffd640 1 -- 172.21.15.92:0/1469574984 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42d4014620 con 0x7f42dc0c82c0 2023-11-24T06:11:23.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.983+0000 7f42e4690640 1 -- 172.21.15.92:0/1469574984 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42dc0c7ac0 con 0x7f42dc0c82c0 2023-11-24T06:11:23.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.983+0000 7f42e4690640 1 -- 172.21.15.92:0/1469574984 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42dc14f9e0 con 0x7f42dc0c82c0 2023-11-24T06:11:23.988 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.984+0000 7f42e4690640 1 -- 172.21.15.92:0/1469574984 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42dc0c7fe0 con 0x7f42dc0c82c0 2023-11-24T06:11:23.988 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.985+0000 7f42d2ffd640 1 -- 172.21.15.92:0/1469574984 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f42d401d3b0 con 0x7f42dc0c82c0 2023-11-24T06:11:23.988 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.986+0000 7f42d2ffd640 1 --2- 172.21.15.92:0/1469574984 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f42c006ae40 0x7f42c006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:23.988 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.986+0000 7f42d2ffd640 1 -- 172.21.15.92:0/1469574984 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5883+0+0 (secure 0 0 0) 0x7f42d4089380 con 0x7f42dc0c82c0 2023-11-24T06:11:23.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.986+0000 7f42e242c640 1 --2- 172.21.15.92:0/1469574984 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f42c006ae40 0x7f42c006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:23.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.987+0000 7f42e242c640 1 --2- 172.21.15.92:0/1469574984 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f42c006ae40 0x7f42c006d300 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f42d8034000 tx=0x7f42d8009030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:23.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:23.990+0000 7f42d2ffd640 1 -- 172.21.15.92:0/1469574984 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f42dc0c7fe0 con 0x7f42dc0c82c0 2023-11-24T06:11:24.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:24.180+0000 7f42e4690640 1 -- 172.21.15.92:0/1469574984 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f42dc0ce8e0 con 0x7f42dc0c82c0 2023-11-24T06:11:24.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:24.181+0000 7f42d2ffd640 1 -- 172.21.15.92:0/1469574984 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7f42d4054370 con 0x7f42dc0c82c0 2023-11-24T06:11:24.184 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 32 2023-11-24T06:11:24.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:24.184+0000 7f42e4690640 1 -- 172.21.15.92:0/1469574984 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f42c006ae40 msgr2=0x7f42c006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:24.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:24.184+0000 7f42e4690640 1 --2- 172.21.15.92:0/1469574984 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f42c006ae40 0x7f42c006d300 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f42d8034000 tx=0x7f42d8009030 comp rx=0 tx=0).stop 2023-11-24T06:11:24.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:24.184+0000 7f42e4690640 1 -- 172.21.15.92:0/1469574984 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f42dc0c82c0 msgr2=0x7f42dc0b8570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:24.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:24.184+0000 7f42e4690640 1 --2- 172.21.15.92:0/1469574984 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f42dc0c82c0 0x7f42dc0b8570 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f42dc08b880 tx=0x7f42d4009d40 comp rx=0 tx=0).stop 2023-11-24T06:11:24.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:24.184+0000 7f42e4690640 1 -- 172.21.15.92:0/1469574984 shutdown_connections 2023-11-24T06:11:24.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:24.184+0000 7f42e4690640 1 --2- 172.21.15.92:0/1469574984 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f42c006ae40 0x7f42c006d300 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:24.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:24.184+0000 7f42e4690640 1 --2- 172.21.15.92:0/1469574984 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f42dc0c82c0 0x7f42dc0b8570 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:24.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:24.184+0000 7f42e4690640 1 --2- 172.21.15.92:0/1469574984 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f42dc0c6ca0 0x7f42dc0b8030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:24.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:24.184+0000 7f42e4690640 1 --2- 172.21.15.92:0/1469574984 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f42c80039a0 0x7f42dc14f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:24.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:24.184+0000 7f42e4690640 1 -- 172.21.15.92:0/1469574984 >> 172.21.15.92:0/1469574984 conn(0x7f42dc09c530 msgr2=0x7f42dc0b4cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:24.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:24.184+0000 7f42e4690640 1 -- 172.21.15.92:0/1469574984 shutdown_connections 2023-11-24T06:11:24.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:24.184+0000 7f42e4690640 1 -- 172.21.15.92:0/1469574984 wait complete. 2023-11-24T06:11:24.205 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T06:11:24.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:24.407+0000 7f63a0eca640 1 Processor -- start 2023-11-24T06:11:24.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:24.408+0000 7f63a0eca640 1 -- start start 2023-11-24T06:11:24.412 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:24.409+0000 7f63a0eca640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f639c15de80 0x7f639c0ae170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:24.412 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:24.409+0000 7f63a0eca640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f639c06c730 con 0x7f639c071e40 2023-11-24T06:11:24.412 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:24.409+0000 7f63a0eca640 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f639c0724d0 con 0x7f639c143810 2023-11-24T06:11:24.412 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:24.409+0000 7f63a0eca640 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f639c143bf0 con 0x7f639c15de80 2023-11-24T06:11:24.412 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:24.410+0000 7f6393fff640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f639c071e40 0x7f639c15d770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:56246/0 (socket says 172.21.15.92:56246) 2023-11-24T06:11:24.412 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:24.410+0000 7f6393fff640 1 -- 172.21.15.92:0/2282484955 learned_addr learned my addr 172.21.15.92:0/2282484955 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-24T06:11:24.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:24.410+0000 7f639affd640 1 -- 172.21.15.92:0/2282484955 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2951208303 0 0) 0x7f639c06c730 con 0x7f639c071e40 2023-11-24T06:11:24.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:24.410+0000 7f639affd640 1 -- 172.21.15.92:0/2282484955 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6388003520 con 0x7f639c071e40 2023-11-24T06:11:24.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:24.410+0000 7f639bfff640 1 --2- 172.21.15.92:0/2282484955 >> v2:172.21.15.118:3300/0 conn(0x7f639c15de80 0x7f639c0ae170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:24.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:24.410+0000 7f639affd640 1 -- 172.21.15.92:0/2282484955 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 268969927 0 0) 0x7f6388003520 con 0x7f639c071e40 2023-11-24T06:11:24.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:24.410+0000 7f639affd640 1 -- 172.21.15.92:0/2282484955 >> v2:172.21.15.118:3300/0 conn(0x7f639c15de80 msgr2=0x7f639c0ae170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:24.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:24.410+0000 7f639affd640 1 --2- 172.21.15.92:0/2282484955 >> v2:172.21.15.118:3300/0 conn(0x7f639c15de80 0x7f639c0ae170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:24.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:24.410+0000 7f639affd640 1 -- 172.21.15.92:0/2282484955 >> v1:172.21.15.118:6789/0 conn(0x7f639c143810 legacy=0x7f639c15e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:24.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:24.410+0000 7f639affd640 1 -- 172.21.15.92:0/2282484955 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f639c06c9f0 con 0x7f639c071e40 2023-11-24T06:11:24.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:24.410+0000 7f639affd640 1 -- 172.21.15.92:0/2282484955 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2607661350 0 0) 0x7f6384001670 con 0x7f639c071e40 2023-11-24T06:11:24.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:24.410+0000 7f639affd640 1 -- 172.21.15.92:0/2282484955 >> v1:172.21.15.92:6789/0 conn(0x7f639c071e40 legacy=0x7f639c15d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:24.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:24.411+0000 7f639affd640 1 --2- 172.21.15.92:0/2282484955 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6388003970 0x7f6388003d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:24.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:24.411+0000 7f639affd640 1 --2- 172.21.15.92:0/2282484955 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f63880047c0 0x7f6388004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:24.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:24.411+0000 7f639affd640 1 --2- 172.21.15.92:0/2282484955 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f63880052a0 0x7f638800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:24.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:24.411+0000 7f639affd640 1 -- 172.21.15.92:0/2282484955 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f639c143bf0 con 0x7f63880047c0 2023-11-24T06:11:24.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:24.411+0000 7f639affd640 1 -- 172.21.15.92:0/2282484955 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f638800c170 con 0x7f63880052a0 2023-11-24T06:11:24.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:24.411+0000 7f639affd640 1 -- 172.21.15.92:0/2282484955 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f638800c330 con 0x7f6388003970 2023-11-24T06:11:24.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:24.411+0000 7f6393fff640 1 --2- 172.21.15.92:0/2282484955 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f63880047c0 0x7f6388004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:24.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:24.411+0000 7f6393fff640 1 -- 172.21.15.92:0/2282484955 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6388003970 msgr2=0x7f6388003d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:24.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:24.411+0000 7f639b7fe640 1 --2- 172.21.15.92:0/2282484955 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6388003970 0x7f6388003d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:24.417 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:24.411+0000 7f6393fff640 1 --2- 172.21.15.92:0/2282484955 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6388003970 0x7f6388003d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:24.417 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:24.411+0000 7f639bfff640 1 --2- 172.21.15.92:0/2282484955 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f63880052a0 0x7f638800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:24.417 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:24.411+0000 7f6393fff640 1 -- 172.21.15.92:0/2282484955 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f63880052a0 msgr2=0x7f638800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:24.417 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:24.411+0000 7f6393fff640 1 --2- 172.21.15.92:0/2282484955 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f63880052a0 0x7f638800b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:24.417 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:24.411+0000 7f6393fff640 1 -- 172.21.15.92:0/2282484955 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6388003520 con 0x7f63880047c0 2023-11-24T06:11:24.417 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:24.411+0000 7f639bfff640 1 -- 172.21.15.92:0/2282484955 reap_dead start 2023-11-24T06:11:24.417 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:24.413+0000 7f6393fff640 1 --2- 172.21.15.92:0/2282484955 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f63880047c0 0x7f6388004c40 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f6384005070 tx=0x7f63880057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d53d55b6069deaf server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:24.417 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:24.413+0000 7f639affd640 1 -- 172.21.15.92:0/2282484955 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6384005b90 con 0x7f63880047c0 2023-11-24T06:11:24.417 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:24.413+0000 7f639affd640 1 -- 172.21.15.92:0/2282484955 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f638402f070 con 0x7f63880047c0 2023-11-24T06:11:24.417 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:24.413+0000 7f639affd640 1 -- 172.21.15.92:0/2282484955 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63840080d0 con 0x7f63880047c0 2023-11-24T06:11:24.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:24.413+0000 7f63a0eca640 1 -- 172.21.15.92:0/2282484955 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f63880047c0 msgr2=0x7f6388004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:24.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:24.413+0000 7f63a0eca640 1 --2- 172.21.15.92:0/2282484955 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f63880047c0 0x7f6388004c40 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f6384005070 tx=0x7f63880057f0 comp rx=0 tx=0).stop 2023-11-24T06:11:24.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:24.413+0000 7f63a0eca640 1 -- 172.21.15.92:0/2282484955 shutdown_connections 2023-11-24T06:11:24.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:24.413+0000 7f63a0eca640 1 --2- 172.21.15.92:0/2282484955 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f63880047c0 0x7f6388004c40 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:24.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:24.414+0000 7f63a0eca640 1 -- 172.21.15.92:0/2282484955 >> 172.21.15.92:0/2282484955 conn(0x7f639c06a7d0 msgr2=0x7f639c06abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:24.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:24.414+0000 7f63a0eca640 1 -- 172.21.15.92:0/2282484955 shutdown_connections 2023-11-24T06:11:24.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:24.414+0000 7f63a0eca640 1 -- 172.21.15.92:0/2282484955 wait complete. 2023-11-24T06:11:24.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:24.415+0000 7f63a0eca640 1 Processor -- start 2023-11-24T06:11:24.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:24.415+0000 7f63a0eca640 1 -- start start 2023-11-24T06:11:24.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:24.416+0000 7f63a0eca640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f63880047c0 0x7f639c15c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:24.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:24.416+0000 7f63a0eca640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f639c0ae9e0 0x7f639c144490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:24.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:24.416+0000 7f63a0eca640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f639c141e50 0x7f639c1422d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:24.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:24.417+0000 7f63a0eca640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f639c0722f0 con 0x7f639c141e50 2023-11-24T06:11:24.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:24.417+0000 7f63a0eca640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f639c1f2a90 con 0x7f63880047c0 2023-11-24T06:11:24.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:24.417+0000 7f63a0eca640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f639c142810 con 0x7f639c0ae9e0 2023-11-24T06:11:24.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:24.417+0000 7f639bfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f639c141e50 0x7f639c1422d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:24.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:24.417+0000 7f639bfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f639c141e50 0x7f639c1422d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45176/0 (socket says 172.21.15.92:45176) 2023-11-24T06:11:24.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:24.417+0000 7f639bfff640 1 -- 172.21.15.92:0/1205328783 learned_addr learned my addr 172.21.15.92:0/1205328783 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:11:24.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:24.417+0000 7f639bfff640 1 -- 172.21.15.92:0/1205328783 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f639c0ae9e0 msgr2=0x7f639c144490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:24.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:24.417+0000 7f639bfff640 1 --2- 172.21.15.92:0/1205328783 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f639c0ae9e0 0x7f639c144490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:24.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:24.417+0000 7f639bfff640 1 -- 172.21.15.92:0/1205328783 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f63880047c0 msgr2=0x7f639c15c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:24.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:24.417+0000 7f639b7fe640 1 --2- 172.21.15.92:0/1205328783 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f63880047c0 0x7f639c15c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:24.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:24.417+0000 7f639bfff640 1 --2- 172.21.15.92:0/1205328783 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f63880047c0 0x7f639c15c740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:24.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:24.417+0000 7f639bfff640 1 -- 172.21.15.92:0/1205328783 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f639c145580 con 0x7f639c141e50 2023-11-24T06:11:24.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:24.417+0000 7f639bfff640 1 --2- 172.21.15.92:0/1205328783 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f639c141e50 0x7f639c1422d0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f639c0ad100 tx=0x7f6394009d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:24.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:24.418+0000 7f6398ff9640 1 -- 172.21.15.92:0/1205328783 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6394007c90 con 0x7f639c141e50 2023-11-24T06:11:24.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:24.418+0000 7f6398ff9640 1 -- 172.21.15.92:0/1205328783 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6394007df0 con 0x7f639c141e50 2023-11-24T06:11:24.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:24.418+0000 7f6398ff9640 1 -- 172.21.15.92:0/1205328783 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6394014640 con 0x7f639c141e50 2023-11-24T06:11:24.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:24.418+0000 7f63a0eca640 1 -- 172.21.15.92:0/1205328783 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f639c0afff0 con 0x7f639c141e50 2023-11-24T06:11:24.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:24.418+0000 7f63a0eca640 1 -- 172.21.15.92:0/1205328783 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f639c144fc0 con 0x7f639c141e50 2023-11-24T06:11:24.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:24.419+0000 7f63a0eca640 1 -- 172.21.15.92:0/1205328783 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f639c0b0180 con 0x7f639c141e50 2023-11-24T06:11:24.423 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:24.420+0000 7f6398ff9640 1 -- 172.21.15.92:0/1205328783 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f639401d3b0 con 0x7f639c141e50 2023-11-24T06:11:24.423 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:24.421+0000 7f6398ff9640 1 --2- 172.21.15.92:0/1205328783 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f637c06ae70 0x7f637c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:24.423 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:24.421+0000 7f6398ff9640 1 -- 172.21.15.92:0/1205328783 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5883+0+0 (secure 0 0 0) 0x7f6394089400 con 0x7f639c141e50 2023-11-24T06:11:24.423 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:24.421+0000 7f639b7fe640 1 --2- 172.21.15.92:0/1205328783 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f637c06ae70 0x7f637c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:24.424 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:24.422+0000 7f639b7fe640 1 --2- 172.21.15.92:0/1205328783 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f637c06ae70 0x7f637c06d330 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f638c003c90 tx=0x7f638c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:24.427 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:24.425+0000 7f6398ff9640 1 -- 172.21.15.92:0/1205328783 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6394054490 con 0x7f639c141e50 2023-11-24T06:11:24.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:24.615+0000 7f63a0eca640 1 -- 172.21.15.92:0/1205328783 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f639c067450 con 0x7f639c141e50 2023-11-24T06:11:24.618 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:24.616+0000 7f6398ff9640 1 -- 172.21.15.92:0/1205328783 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7f6394057ab0 con 0x7f639c141e50 2023-11-24T06:11:24.619 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 64 2023-11-24T06:11:24.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:24.619+0000 7f63a0eca640 1 -- 172.21.15.92:0/1205328783 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f637c06ae70 msgr2=0x7f637c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:24.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:24.619+0000 7f63a0eca640 1 --2- 172.21.15.92:0/1205328783 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f637c06ae70 0x7f637c06d330 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f638c003c90 tx=0x7f638c009040 comp rx=0 tx=0).stop 2023-11-24T06:11:24.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:24.619+0000 7f63a0eca640 1 -- 172.21.15.92:0/1205328783 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f639c141e50 msgr2=0x7f639c1422d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:24.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:24.619+0000 7f63a0eca640 1 --2- 172.21.15.92:0/1205328783 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f639c141e50 0x7f639c1422d0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f639c0ad100 tx=0x7f6394009d20 comp rx=0 tx=0).stop 2023-11-24T06:11:24.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:24.619+0000 7f63a0eca640 1 -- 172.21.15.92:0/1205328783 shutdown_connections 2023-11-24T06:11:24.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:24.619+0000 7f63a0eca640 1 --2- 172.21.15.92:0/1205328783 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f637c06ae70 0x7f637c06d330 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:24.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:24.619+0000 7f63a0eca640 1 --2- 172.21.15.92:0/1205328783 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f639c141e50 0x7f639c1422d0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:24.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:24.619+0000 7f63a0eca640 1 --2- 172.21.15.92:0/1205328783 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f639c0ae9e0 0x7f639c144490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:24.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:24.619+0000 7f63a0eca640 1 --2- 172.21.15.92:0/1205328783 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f63880047c0 0x7f639c15c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:24.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:24.619+0000 7f63a0eca640 1 -- 172.21.15.92:0/1205328783 >> 172.21.15.92:0/1205328783 conn(0x7f639c06a7d0 msgr2=0x7f639c070160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:24.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:24.619+0000 7f63a0eca640 1 -- 172.21.15.92:0/1205328783 shutdown_connections 2023-11-24T06:11:24.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:24.619+0000 7f63a0eca640 1 -- 172.21.15.92:0/1205328783 wait complete. 2023-11-24T06:11:24.640 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-24T06:11:24.854 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:24.852+0000 7fb853278640 1 Processor -- start 2023-11-24T06:11:24.854 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:24.852+0000 7fb853278640 1 -- start start 2023-11-24T06:11:24.856 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:24.854+0000 7fb853278640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb84c15afb0 0x7fb84c13f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:24.856 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:24.854+0000 7fb853278640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb84c13ff30 con 0x7fb84c13fae0 2023-11-24T06:11:24.856 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:24.854+0000 7fb853278640 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb84c0cdc40 con 0x7fb84c13cfa0 2023-11-24T06:11:24.856 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:24.854+0000 7fb853278640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb84c0cdec0 con 0x7fb84c15afb0 2023-11-24T06:11:24.856 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:24.854+0000 7fb852a77640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fb84c13fae0 0x7fb84c0b2040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:56248/0 (socket says 172.21.15.92:56248) 2023-11-24T06:11:24.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:24.854+0000 7fb851a75640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb84c15afb0 0x7fb84c13f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:24.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:24.854+0000 7fb852a77640 1 -- 172.21.15.92:0/3632034152 learned_addr learned my addr 172.21.15.92:0/3632034152 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-24T06:11:24.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:24.854+0000 7fb851274640 1 -- 172.21.15.92:0/3632034152 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3210340409 0 0) 0x7fb844001490 con 0x7fb84c13fae0 2023-11-24T06:11:24.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:24.855+0000 7fb851274640 1 -- 172.21.15.92:0/3632034152 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb838005730 con 0x7fb84c13fae0 2023-11-24T06:11:24.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:24.855+0000 7fb851a75640 1 -- 172.21.15.92:0/3632034152 >> v1:172.21.15.118:6789/0 conn(0x7fb84c13cfa0 legacy=0x7fb84c0cd530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:24.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:24.855+0000 7fb851a75640 1 -- 172.21.15.92:0/3632034152 >> v1:172.21.15.92:6789/0 conn(0x7fb84c13fae0 legacy=0x7fb84c0b2040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:24.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:24.855+0000 7fb851a75640 1 -- 172.21.15.92:0/3632034152 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb84c0ce1a0 con 0x7fb84c15afb0 2023-11-24T06:11:24.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:24.856+0000 7fb851a75640 1 --2- 172.21.15.92:0/3632034152 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb84c15afb0 0x7fb84c13f5a0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fb83c0097a0 tx=0x7fb83c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26bbdf1cd57abec4 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:24.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:24.856+0000 7fb851274640 1 -- 172.21.15.92:0/3632034152 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb83c031830 con 0x7fb84c15afb0 2023-11-24T06:11:24.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:24.856+0000 7fb851274640 1 -- 172.21.15.92:0/3632034152 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb83c031990 con 0x7fb84c15afb0 2023-11-24T06:11:24.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:24.856+0000 7fb851274640 1 -- 172.21.15.92:0/3632034152 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb83c031d20 con 0x7fb84c15afb0 2023-11-24T06:11:24.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:24.857+0000 7fb853278640 1 -- 172.21.15.92:0/3632034152 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb84c15afb0 msgr2=0x7fb84c13f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:24.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:24.857+0000 7fb853278640 1 --2- 172.21.15.92:0/3632034152 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb84c15afb0 0x7fb84c13f5a0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fb83c0097a0 tx=0x7fb83c02b8d0 comp rx=0 tx=0).stop 2023-11-24T06:11:24.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:24.857+0000 7fb853278640 1 -- 172.21.15.92:0/3632034152 shutdown_connections 2023-11-24T06:11:24.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:24.857+0000 7fb853278640 1 --2- 172.21.15.92:0/3632034152 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb84c15afb0 0x7fb84c13f5a0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:24.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:24.857+0000 7fb853278640 1 -- 172.21.15.92:0/3632034152 >> 172.21.15.92:0/3632034152 conn(0x7fb84c06a800 msgr2=0x7fb84c06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:24.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:24.857+0000 7fb853278640 1 -- 172.21.15.92:0/3632034152 shutdown_connections 2023-11-24T06:11:24.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:24.857+0000 7fb853278640 1 -- 172.21.15.92:0/3632034152 wait complete. 2023-11-24T06:11:24.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:24.858+0000 7fb853278640 1 Processor -- start 2023-11-24T06:11:24.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:24.859+0000 7fb853278640 1 -- start start 2023-11-24T06:11:24.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:24.859+0000 7fb853278640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb84c13cfa0 0x7fb84c13d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:24.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:24.859+0000 7fb853278640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb84c13fae0 0x7fb84c15b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:24.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:24.859+0000 7fb853278640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb84c15afb0 0x7fb84c13e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:24.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:24.859+0000 7fb853278640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb84c15c920 con 0x7fb84c13cfa0 2023-11-24T06:11:24.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:24.859+0000 7fb853278640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb84c15bca0 con 0x7fb84c15afb0 2023-11-24T06:11:24.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:24.859+0000 7fb853278640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb84c15bf80 con 0x7fb84c13fae0 2023-11-24T06:11:24.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:24.860+0000 7fb852276640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb84c13cfa0 0x7fb84c13d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:24.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:24.860+0000 7fb852276640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb84c13cfa0 0x7fb84c13d4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45194/0 (socket says 172.21.15.92:45194) 2023-11-24T06:11:24.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:24.860+0000 7fb852276640 1 -- 172.21.15.92:0/2815876026 learned_addr learned my addr 172.21.15.92:0/2815876026 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:11:24.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:24.860+0000 7fb852276640 1 -- 172.21.15.92:0/2815876026 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb84c13fae0 msgr2=0x7fb84c15b5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:24.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:24.860+0000 7fb851a75640 1 --2- 172.21.15.92:0/2815876026 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb84c13fae0 0x7fb84c15b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:24.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:24.860+0000 7fb852a77640 1 --2- 172.21.15.92:0/2815876026 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb84c15afb0 0x7fb84c13e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:24.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:24.860+0000 7fb852276640 1 --2- 172.21.15.92:0/2815876026 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb84c13fae0 0x7fb84c15b5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:24.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:24.860+0000 7fb852276640 1 -- 172.21.15.92:0/2815876026 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb84c15afb0 msgr2=0x7fb84c13e3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:24.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:24.860+0000 7fb852276640 1 --2- 172.21.15.92:0/2815876026 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb84c15afb0 0x7fb84c13e3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:24.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:24.860+0000 7fb852276640 1 -- 172.21.15.92:0/2815876026 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb83c009400 con 0x7fb84c13cfa0 2023-11-24T06:11:24.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:24.861+0000 7fb852276640 1 --2- 172.21.15.92:0/2815876026 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb84c13cfa0 0x7fb84c13d4f0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fb84800c450 tx=0x7fb84800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:24.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:24.861+0000 7fb842ffd640 1 -- 172.21.15.92:0/2815876026 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8480052d0 con 0x7fb84c13cfa0 2023-11-24T06:11:24.865 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:24.861+0000 7fb842ffd640 1 -- 172.21.15.92:0/2815876026 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb848005430 con 0x7fb84c13cfa0 2023-11-24T06:11:24.865 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:24.861+0000 7fb842ffd640 1 -- 172.21.15.92:0/2815876026 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb848012870 con 0x7fb84c13cfa0 2023-11-24T06:11:24.865 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:24.861+0000 7fb853278640 1 -- 172.21.15.92:0/2815876026 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb84c15c260 con 0x7fb84c13cfa0 2023-11-24T06:11:24.865 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:24.861+0000 7fb853278640 1 -- 172.21.15.92:0/2815876026 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb84c15cd40 con 0x7fb84c13cfa0 2023-11-24T06:11:24.865 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:24.862+0000 7fb853278640 1 -- 172.21.15.92:0/2815876026 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb84c15ca60 con 0x7fb84c13cfa0 2023-11-24T06:11:24.865 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:24.863+0000 7fb842ffd640 1 -- 172.21.15.92:0/2815876026 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb8480129d0 con 0x7fb84c13cfa0 2023-11-24T06:11:24.866 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:24.864+0000 7fb842ffd640 1 --2- 172.21.15.92:0/2815876026 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fb83006ae40 0x7fb83006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:24.866 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:24.864+0000 7fb842ffd640 1 -- 172.21.15.92:0/2815876026 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5883+0+0 (secure 0 0 0) 0x7fb848089210 con 0x7fb84c13cfa0 2023-11-24T06:11:24.866 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:24.864+0000 7fb851a75640 1 --2- 172.21.15.92:0/2815876026 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fb83006ae40 0x7fb83006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:24.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:24.865+0000 7fb851a75640 1 --2- 172.21.15.92:0/2815876026 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fb83006ae40 0x7fb83006d300 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fb83c036000 tx=0x7fb83c03c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:24.870 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:24.868+0000 7fb842ffd640 1 -- 172.21.15.92:0/2815876026 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb84c15ca60 con 0x7fb84c13cfa0 2023-11-24T06:11:25.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:25.058+0000 7fb853278640 1 -- 172.21.15.92:0/2815876026 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fb84c067450 con 0x7fb84c13cfa0 2023-11-24T06:11:25.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:25.059+0000 7fb842ffd640 1 -- 172.21.15.92:0/2815876026 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) v1 ==== 80+0+0 (secure 0 0 0) 0x7fb848054200 con 0x7fb84c13cfa0 2023-11-24T06:11:25.061 INFO:teuthology.orchestra.run.smithi092.stderr:listed 0 entries 2023-11-24T06:11:25.063 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:25.061+0000 7fb853278640 1 -- 172.21.15.92:0/2815876026 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fb83006ae40 msgr2=0x7fb83006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:25.063 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:25.061+0000 7fb853278640 1 --2- 172.21.15.92:0/2815876026 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fb83006ae40 0x7fb83006d300 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fb83c036000 tx=0x7fb83c03c040 comp rx=0 tx=0).stop 2023-11-24T06:11:25.063 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:25.061+0000 7fb853278640 1 -- 172.21.15.92:0/2815876026 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb84c13cfa0 msgr2=0x7fb84c13d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:25.064 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:25.061+0000 7fb853278640 1 --2- 172.21.15.92:0/2815876026 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb84c13cfa0 0x7fb84c13d4f0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fb84800c450 tx=0x7fb84800c910 comp rx=0 tx=0).stop 2023-11-24T06:11:25.064 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:25.061+0000 7fb853278640 1 -- 172.21.15.92:0/2815876026 shutdown_connections 2023-11-24T06:11:25.064 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:25.061+0000 7fb853278640 1 --2- 172.21.15.92:0/2815876026 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fb83006ae40 0x7fb83006d300 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:25.064 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:25.061+0000 7fb853278640 1 --2- 172.21.15.92:0/2815876026 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb84c15afb0 0x7fb84c13e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:25.064 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:25.061+0000 7fb853278640 1 --2- 172.21.15.92:0/2815876026 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb84c13fae0 0x7fb84c15b5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:25.064 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:25.061+0000 7fb853278640 1 --2- 172.21.15.92:0/2815876026 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb84c13cfa0 0x7fb84c13d4f0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:25.064 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:25.061+0000 7fb853278640 1 -- 172.21.15.92:0/2815876026 >> 172.21.15.92:0/2815876026 conn(0x7fb84c06a800 msgr2=0x7fb84c0713e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:25.064 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:25.061+0000 7fb853278640 1 -- 172.21.15.92:0/2815876026 shutdown_connections 2023-11-24T06:11:25.064 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:25.061+0000 7fb853278640 1 -- 172.21.15.92:0/2815876026 wait complete. 2023-11-24T06:11:25.082 DEBUG:teuthology.orchestra.run.smithi118:> set -ex 2023-11-24T06:11:25.082 DEBUG:teuthology.orchestra.run.smithi118:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:11:25.141 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:11:25.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:25.354+0000 7fb022d89640 1 Processor -- start 2023-11-24T06:11:25.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:25.354+0000 7fb022d89640 1 -- start start 2023-11-24T06:11:25.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:25.356+0000 7fb022d89640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb01c0ce8f0 0x7fb01c0ced30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:25.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:25.356+0000 7fb022d89640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fb01c0b2a00 0x7fb01c0b2e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:25.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:25.356+0000 7fb022d89640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb01c0d0de0 con 0x7fb01c160f80 2023-11-24T06:11:25.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:25.356+0000 7fb022d89640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb01c0d10c0 con 0x7fb01c0ce8f0 2023-11-24T06:11:25.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:25.356+0000 7fb022d89640 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb01c0d13a0 con 0x7fb01c0b2a00 2023-11-24T06:11:25.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:25.357+0000 7fb020b25640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fb01c160f80 0x7fb01c161360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:56250/0 (socket says 172.21.15.92:56250) 2023-11-24T06:11:25.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:25.357+0000 7fb020b25640 1 -- 172.21.15.92:0/1549373552 learned_addr learned my addr 172.21.15.92:0/1549373552 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-24T06:11:25.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:25.357+0000 7fb01bfff640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb01c0ce8f0 0x7fb01c0ced30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:25.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:25.357+0000 7fb021326640 1 --2- 172.21.15.92:0/1549373552 >> v2:172.21.15.118:3300/0 conn(0x7fb01c0b2a00 0x7fb01c0b2e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:25.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:25.357+0000 7fb01b7fe640 1 -- 172.21.15.92:0/1549373552 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1983081773 0 0) 0x7fb01c0d0de0 con 0x7fb01c160f80 2023-11-24T06:11:25.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:25.357+0000 7fb01b7fe640 1 -- 172.21.15.92:0/1549373552 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb00c003520 con 0x7fb01c160f80 2023-11-24T06:11:25.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:25.357+0000 7fb01bfff640 1 -- 172.21.15.92:0/1549373552 >> v2:172.21.15.118:3300/0 conn(0x7fb01c0b2a00 msgr2=0x7fb01c0b2e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:25.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:25.357+0000 7fb01bfff640 1 --2- 172.21.15.92:0/1549373552 >> v2:172.21.15.118:3300/0 conn(0x7fb01c0b2a00 0x7fb01c0b2e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:25.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:25.357+0000 7fb01bfff640 1 -- 172.21.15.92:0/1549373552 >> v1:172.21.15.92:6789/0 conn(0x7fb01c160f80 legacy=0x7fb01c161360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:25.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:25.357+0000 7fb01bfff640 1 -- 172.21.15.92:0/1549373552 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb01c0d1680 con 0x7fb01c0ce8f0 2023-11-24T06:11:25.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:25.359+0000 7fb01bfff640 1 --2- 172.21.15.92:0/1549373552 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb01c0ce8f0 0x7fb01c0ced30 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fb01c0b4cb0 tx=0x7fb00802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f211691ae1f06da server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:25.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:25.359+0000 7fb01b7fe640 1 -- 172.21.15.92:0/1549373552 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb008031830 con 0x7fb01c0ce8f0 2023-11-24T06:11:25.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:25.359+0000 7fb01b7fe640 1 -- 172.21.15.92:0/1549373552 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb008031990 con 0x7fb01c0ce8f0 2023-11-24T06:11:25.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:25.359+0000 7fb01b7fe640 1 -- 172.21.15.92:0/1549373552 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb008031d00 con 0x7fb01c0ce8f0 2023-11-24T06:11:25.362 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:25.359+0000 7fb022d89640 1 -- 172.21.15.92:0/1549373552 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb01c0ce8f0 msgr2=0x7fb01c0ced30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:25.362 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:25.359+0000 7fb022d89640 1 --2- 172.21.15.92:0/1549373552 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb01c0ce8f0 0x7fb01c0ced30 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fb01c0b4cb0 tx=0x7fb00802b8d0 comp rx=0 tx=0).stop 2023-11-24T06:11:25.362 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:25.360+0000 7fb022d89640 1 -- 172.21.15.92:0/1549373552 shutdown_connections 2023-11-24T06:11:25.362 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:25.360+0000 7fb022d89640 1 --2- 172.21.15.92:0/1549373552 >> v2:172.21.15.118:3300/0 conn(0x7fb01c0b2a00 0x7fb01c0b2e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:25.363 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:25.360+0000 7fb022d89640 1 --2- 172.21.15.92:0/1549373552 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb01c0ce8f0 0x7fb01c0ced30 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:25.363 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:25.360+0000 7fb022d89640 1 -- 172.21.15.92:0/1549373552 >> 172.21.15.92:0/1549373552 conn(0x7fb01c09c550 msgr2=0x7fb01c0d3210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:25.363 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:25.360+0000 7fb022d89640 1 -- 172.21.15.92:0/1549373552 shutdown_connections 2023-11-24T06:11:25.363 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:25.360+0000 7fb022d89640 1 -- 172.21.15.92:0/1549373552 wait complete. 2023-11-24T06:11:25.363 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:25.361+0000 7fb022d89640 1 Processor -- start 2023-11-24T06:11:25.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:25.362+0000 7fb022d89640 1 -- start start 2023-11-24T06:11:25.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:25.363+0000 7fb022d89640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb01c0b2a00 0x7fb01c150aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:25.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:25.363+0000 7fb022d89640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb01c0ce8f0 0x7fb01c15f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:25.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:25.363+0000 7fb022d89640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb01c160f80 0x7fb01c15f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:25.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:25.363+0000 7fb022d89640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb01c0cc2c0 con 0x7fb01c0b2a00 2023-11-24T06:11:25.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:25.363+0000 7fb022d89640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb01c0b65e0 con 0x7fb01c160f80 2023-11-24T06:11:25.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:25.363+0000 7fb022d89640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb01c0b5fd0 con 0x7fb01c0ce8f0 2023-11-24T06:11:25.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:25.363+0000 7fb020b25640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb01c0b2a00 0x7fb01c150aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:25.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:25.363+0000 7fb020b25640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb01c0b2a00 0x7fb01c150aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45220/0 (socket says 172.21.15.92:45220) 2023-11-24T06:11:25.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:25.363+0000 7fb020b25640 1 -- 172.21.15.92:0/962528860 learned_addr learned my addr 172.21.15.92:0/962528860 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:11:25.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:25.363+0000 7fb020b25640 1 -- 172.21.15.92:0/962528860 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb01c0ce8f0 msgr2=0x7fb01c15f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:25.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:25.363+0000 7fb020b25640 1 --2- 172.21.15.92:0/962528860 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb01c0ce8f0 0x7fb01c15f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:25.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:25.363+0000 7fb020b25640 1 -- 172.21.15.92:0/962528860 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb01c160f80 msgr2=0x7fb01c15f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:25.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:25.363+0000 7fb020b25640 1 --2- 172.21.15.92:0/962528860 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb01c160f80 0x7fb01c15f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:25.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:25.363+0000 7fb020b25640 1 -- 172.21.15.92:0/962528860 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb008009400 con 0x7fb01c0b2a00 2023-11-24T06:11:25.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:25.364+0000 7fb020b25640 1 --2- 172.21.15.92:0/962528860 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb01c0b2a00 0x7fb01c150aa0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fb01000c390 tx=0x7fb01000c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:25.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:25.364+0000 7fb0197fa640 1 -- 172.21.15.92:0/962528860 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb010005ce0 con 0x7fb01c0b2a00 2023-11-24T06:11:25.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:25.364+0000 7fb0197fa640 1 -- 172.21.15.92:0/962528860 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb010005e40 con 0x7fb01c0b2a00 2023-11-24T06:11:25.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:25.364+0000 7fb0197fa640 1 -- 172.21.15.92:0/962528860 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0100072f0 con 0x7fb01c0b2a00 2023-11-24T06:11:25.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:25.364+0000 7fb022d89640 1 -- 172.21.15.92:0/962528860 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb01c0cbfe0 con 0x7fb01c0b2a00 2023-11-24T06:11:25.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:25.364+0000 7fb022d89640 1 -- 172.21.15.92:0/962528860 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb01c150fe0 con 0x7fb01c0b2a00 2023-11-24T06:11:25.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:25.365+0000 7fb022d89640 1 -- 172.21.15.92:0/962528860 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb01c0675d0 con 0x7fb01c0b2a00 2023-11-24T06:11:25.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:25.366+0000 7fb0197fa640 1 -- 172.21.15.92:0/962528860 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb010007450 con 0x7fb01c0b2a00 2023-11-24T06:11:25.369 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:25.367+0000 7fb0197fa640 1 --2- 172.21.15.92:0/962528860 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fb00006ae40 0x7fb00006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:25.369 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:25.367+0000 7fb0197fa640 1 -- 172.21.15.92:0/962528860 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5883+0+0 (secure 0 0 0) 0x7fb010088ca0 con 0x7fb01c0b2a00 2023-11-24T06:11:25.369 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:25.367+0000 7fb01bfff640 1 --2- 172.21.15.92:0/962528860 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fb00006ae40 0x7fb00006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:25.370 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:25.368+0000 7fb01bfff640 1 --2- 172.21.15.92:0/962528860 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fb00006ae40 0x7fb00006d300 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fb008036000 tx=0x7fb008030000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:25.374 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:25.372+0000 7fb0197fa640 1 -- 172.21.15.92:0/962528860 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb010053c90 con 0x7fb01c0b2a00 2023-11-24T06:11:25.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:25.563+0000 7fb022d89640 1 -- 172.21.15.92:0/962528860 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb01c0c6a40 con 0x7fb01c0b2a00 2023-11-24T06:11:25.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:25.565+0000 7fb0197fa640 1 -- 172.21.15.92:0/962528860 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16800 (secure 0 0 0) 0x7fb010017020 con 0x7fb01c0b2a00 2023-11-24T06:11:25.568 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-24T06:11:25.568 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":22,"fsid":"2d6dee5f-8aa3-470f-931a-50ade963b1cb","created":"2023-11-24T06:10:04.707204+0000","modified":"2023-11-24T06:11:11.472401+0000","last_up_change":"2023-11-24T06:10:17.598352+0000","last_in_change":"2023-11-24T06:10:09.040793+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:10:17.733098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:10:22.926445+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T06:10:24.151288+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ff37a9fb-135f-471f-b4b0-f7b5772006c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6803","nonce":20560714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6805","nonce":20560714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6809","nonce":20560714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6807","nonce":20560714}]},"public_addr":"172.21.15.92:6803/20560714","cluster_addr":"172.21.15.92:6805/20560714","heartbeat_back_addr":"172.21.15.92:6809/20560714","heartbeat_front_addr":"172.21.15.92:6807/20560714","state":["exists","up"]},{"osd":1,"uuid":"03cdd353-dcf1-4f38-94fe-a795137c06b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6819","nonce":2451239906}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6821","nonce":2451239906}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6825","nonce":2451239906}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6823","nonce":2451239906}]},"public_addr":"172.21.15.92:6819/2451239906","cluster_addr":"172.21.15.92:6821/2451239906","heartbeat_back_addr":"172.21.15.92:6825/2451239906","heartbeat_front_addr":"172.21.15.92:6823/2451239906","state":["exists","up"]},{"osd":2,"uuid":"a4a20d80-5d57-484b-becd-9aa8070dd3aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6811","nonce":1124979277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6813","nonce":1124979277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6817","nonce":1124979277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6815","nonce":1124979277}]},"public_addr":"172.21.15.92:6811/1124979277","cluster_addr":"172.21.15.92:6813/1124979277","heartbeat_back_addr":"172.21.15.92:6817/1124979277","heartbeat_front_addr":"172.21.15.92:6815/1124979277","state":["exists","up"]},{"osd":3,"uuid":"4648d7ce-9b29-4583-9425-4ba8fd2b1903","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6827","nonce":1734115782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6829","nonce":1734115782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6833","nonce":1734115782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6831","nonce":1734115782}]},"public_addr":"172.21.15.92:6827/1734115782","cluster_addr":"172.21.15.92:6829/1734115782","heartbeat_back_addr":"172.21.15.92:6833/1734115782","heartbeat_front_addr":"172.21.15.92:6831/1734115782","state":["exists","up"]},{"osd":4,"uuid":"3bc9d7eb-c6fa-4fcf-a3ec-ade604e05aac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6819","nonce":3425937229}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6822","nonce":3425937229}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6830","nonce":3425937229}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6826","nonce":3425937229}]},"public_addr":"172.21.15.118:6819/3425937229","cluster_addr":"172.21.15.118:6822/3425937229","heartbeat_back_addr":"172.21.15.118:6830/3425937229","heartbeat_front_addr":"172.21.15.118:6826/3425937229","state":["exists","up"]},{"osd":5,"uuid":"dce9ba6d-59ed-46d1-84fc-a84b13de10c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6801","nonce":827455984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6803","nonce":827455984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6807","nonce":827455984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6805","nonce":827455984}]},"public_addr":"172.21.15.118:6801/827455984","cluster_addr":"172.21.15.118:6803/827455984","heartbeat_back_addr":"172.21.15.118:6807/827455984","heartbeat_front_addr":"172.21.15.118:6805/827455984","state":["exists","up"]},{"osd":6,"uuid":"82aec504-ae58-4709-8d30-46f2b12081b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6809","nonce":2946986141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6811","nonce":2946986141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6815","nonce":2946986141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6813","nonce":2946986141}]},"public_addr":"172.21.15.118:6809/2946986141","cluster_addr":"172.21.15.118:6811/2946986141","heartbeat_back_addr":"172.21.15.118:6815/2946986141","heartbeat_front_addr":"172.21.15.118:6813/2946986141","state":["exists","up"]},{"osd":7,"uuid":"4ed4c624-c88f-4d35-bdb1-bf209b5dffb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6817","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6818","nonce":48122059}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6821","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6823","nonce":48122059}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6829","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6831","nonce":48122059}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6825","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6827","nonce":48122059}]},"public_addr":"172.21.15.118:6818/48122059","cluster_addr":"172.21.15.118:6823/48122059","heartbeat_back_addr":"172.21.15.118:6831/48122059","heartbeat_front_addr":"172.21.15.118:6827/48122059","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.536664+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.368324+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.271201+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.342971+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.621832+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.387110+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.632946+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.416032+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.d","mappings":[{"from":5,"to":1}]},{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"3.29","mappings":[{"from":3,"to":2}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.34","mappings":[{"from":4,"to":2}]},{"pgid":"3.3b","mappings":[{"from":7,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:11:25.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:25.568+0000 7fb022d89640 1 -- 172.21.15.92:0/962528860 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fb00006ae40 msgr2=0x7fb00006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:25.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:25.568+0000 7fb022d89640 1 --2- 172.21.15.92:0/962528860 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fb00006ae40 0x7fb00006d300 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fb008036000 tx=0x7fb008030000 comp rx=0 tx=0).stop 2023-11-24T06:11:25.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:25.568+0000 7fb022d89640 1 -- 172.21.15.92:0/962528860 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb01c0b2a00 msgr2=0x7fb01c150aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:25.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:25.568+0000 7fb022d89640 1 --2- 172.21.15.92:0/962528860 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb01c0b2a00 0x7fb01c150aa0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fb01000c390 tx=0x7fb01000c850 comp rx=0 tx=0).stop 2023-11-24T06:11:25.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:25.568+0000 7fb022d89640 1 -- 172.21.15.92:0/962528860 shutdown_connections 2023-11-24T06:11:25.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:25.568+0000 7fb022d89640 1 --2- 172.21.15.92:0/962528860 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fb00006ae40 0x7fb00006d300 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:25.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:25.568+0000 7fb022d89640 1 --2- 172.21.15.92:0/962528860 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb01c160f80 0x7fb01c15f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:25.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:25.568+0000 7fb022d89640 1 --2- 172.21.15.92:0/962528860 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb01c0ce8f0 0x7fb01c15f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:25.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:25.568+0000 7fb022d89640 1 --2- 172.21.15.92:0/962528860 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb01c0b2a00 0x7fb01c150aa0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:25.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:25.568+0000 7fb022d89640 1 -- 172.21.15.92:0/962528860 >> 172.21.15.92:0/962528860 conn(0x7fb01c09c550 msgr2=0x7fb01c06c650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:25.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:25.568+0000 7fb022d89640 1 -- 172.21.15.92:0/962528860 shutdown_connections 2023-11-24T06:11:25.572 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:25.569+0000 7fb022d89640 1 -- 172.21.15.92:0/962528860 wait complete. 2023-11-24T06:11:25.591 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T06:10:17.733098+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T06:10:22.926445+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T06:10:24.151288+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T06:11:25.591 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T06:11:25.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:25.804+0000 7f8263f33640 1 Processor -- start 2023-11-24T06:11:25.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:25.804+0000 7f8263f33640 1 -- start start 2023-11-24T06:11:25.808 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:25.806+0000 7f8263f33640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f825c06a3e0 0x7f825c0ac8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:25.808 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:25.806+0000 7f8263f33640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f825c0aede0 0x7f825c06f080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:25.808 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:25.806+0000 7f8263f33640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f825c071de0 con 0x7f825c0d0b60 2023-11-24T06:11:25.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:25.806+0000 7f8263f33640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f825c0af270 con 0x7f825c06a3e0 2023-11-24T06:11:25.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:25.806+0000 7f8263f33640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f825c072030 con 0x7f825c0aede0 2023-11-24T06:11:25.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:25.806+0000 7f8261ccf640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f825c06a3e0 0x7f825c0ac8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:25.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:25.806+0000 7f82614ce640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f825c0d0b60 0x7f825c06a7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:56256/0 (socket says 172.21.15.92:56256) 2023-11-24T06:11:25.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:25.806+0000 7f82614ce640 1 -- 172.21.15.92:0/1948115373 learned_addr learned my addr 172.21.15.92:0/1948115373 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-24T06:11:25.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:25.807+0000 7f82624d0640 1 --2- 172.21.15.92:0/1948115373 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f825c0aede0 0x7f825c06f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:25.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:25.807+0000 7f8261ccf640 1 -- 172.21.15.92:0/1948115373 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f825c0aede0 msgr2=0x7f825c06f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:25.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:25.807+0000 7f8261ccf640 1 --2- 172.21.15.92:0/1948115373 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f825c0aede0 0x7f825c06f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:25.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:25.807+0000 7f8261ccf640 1 -- 172.21.15.92:0/1948115373 >> v1:172.21.15.92:6789/0 conn(0x7f825c0d0b60 legacy=0x7f825c06a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:25.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:25.807+0000 7f8261ccf640 1 -- 172.21.15.92:0/1948115373 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f825c072310 con 0x7f825c06a3e0 2023-11-24T06:11:25.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:25.808+0000 7f8261ccf640 1 --2- 172.21.15.92:0/1948115373 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f825c06a3e0 0x7f825c0ac8d0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f82580097a0 tx=0x7f825802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea7163f9caf3553c server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:25.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:25.808+0000 7f8260ccd640 1 -- 172.21.15.92:0/1948115373 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8258031830 con 0x7f825c06a3e0 2023-11-24T06:11:25.811 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:25.808+0000 7f8260ccd640 1 -- 172.21.15.92:0/1948115373 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8258031990 con 0x7f825c06a3e0 2023-11-24T06:11:25.811 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:25.808+0000 7f8260ccd640 1 -- 172.21.15.92:0/1948115373 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8258031d20 con 0x7f825c06a3e0 2023-11-24T06:11:25.811 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:25.809+0000 7f8263f33640 1 -- 172.21.15.92:0/1948115373 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f825c06a3e0 msgr2=0x7f825c0ac8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:25.812 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:25.809+0000 7f8263f33640 1 --2- 172.21.15.92:0/1948115373 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f825c06a3e0 0x7f825c0ac8d0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f82580097a0 tx=0x7f825802b8d0 comp rx=0 tx=0).stop 2023-11-24T06:11:25.812 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:25.809+0000 7f8263f33640 1 -- 172.21.15.92:0/1948115373 shutdown_connections 2023-11-24T06:11:25.812 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:25.809+0000 7f8263f33640 1 --2- 172.21.15.92:0/1948115373 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f825c0aede0 0x7f825c06f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:25.812 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:25.809+0000 7f8263f33640 1 --2- 172.21.15.92:0/1948115373 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f825c06a3e0 0x7f825c0ac8d0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:25.812 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:25.809+0000 7f8263f33640 1 -- 172.21.15.92:0/1948115373 >> 172.21.15.92:0/1948115373 conn(0x7f825c0cfef0 msgr2=0x7f825c0d0300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:25.812 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:25.809+0000 7f8263f33640 1 -- 172.21.15.92:0/1948115373 shutdown_connections 2023-11-24T06:11:25.812 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:25.809+0000 7f8263f33640 1 -- 172.21.15.92:0/1948115373 wait complete. 2023-11-24T06:11:25.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:25.811+0000 7f8263f33640 1 Processor -- start 2023-11-24T06:11:25.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:25.812+0000 7f8263f33640 1 -- start start 2023-11-24T06:11:25.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:25.813+0000 7f8263f33640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f825c06a3e0 0x7f825c153100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:25.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:25.813+0000 7f8263f33640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f825c0aede0 0x7f825c1f30c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:25.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:25.813+0000 7f8263f33640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f825c0d0b60 0x7f825c15c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:25.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:25.813+0000 7f8263f33640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f825c0ce970 con 0x7f825c06a3e0 2023-11-24T06:11:25.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:25.813+0000 7f8263f33640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f825c153640 con 0x7f825c0d0b60 2023-11-24T06:11:25.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:25.813+0000 7f8263f33640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f825c0cdd60 con 0x7f825c0aede0 2023-11-24T06:11:25.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:25.813+0000 7f8261ccf640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f825c06a3e0 0x7f825c153100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:25.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:25.813+0000 7f8261ccf640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f825c06a3e0 0x7f825c153100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45242/0 (socket says 172.21.15.92:45242) 2023-11-24T06:11:25.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:25.813+0000 7f8261ccf640 1 -- 172.21.15.92:0/598882218 learned_addr learned my addr 172.21.15.92:0/598882218 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:11:25.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:25.814+0000 7f8261ccf640 1 -- 172.21.15.92:0/598882218 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f825c0aede0 msgr2=0x7f825c1f30c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:25.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:25.814+0000 7f8261ccf640 1 --2- 172.21.15.92:0/598882218 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f825c0aede0 0x7f825c1f30c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:25.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:25.814+0000 7f8261ccf640 1 -- 172.21.15.92:0/598882218 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f825c0d0b60 msgr2=0x7f825c15c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:25.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:25.814+0000 7f8261ccf640 1 --2- 172.21.15.92:0/598882218 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f825c0d0b60 0x7f825c15c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:25.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:25.814+0000 7f8261ccf640 1 -- 172.21.15.92:0/598882218 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8258009400 con 0x7f825c06a3e0 2023-11-24T06:11:25.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:25.814+0000 7f8261ccf640 1 --2- 172.21.15.92:0/598882218 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f825c06a3e0 0x7f825c153100 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f8258009890 tx=0x7f82580046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:25.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:25.814+0000 7f82527fc640 1 -- 172.21.15.92:0/598882218 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f825803b980 con 0x7f825c06a3e0 2023-11-24T06:11:25.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:25.814+0000 7f82527fc640 1 -- 172.21.15.92:0/598882218 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f825803bb00 con 0x7f825c06a3e0 2023-11-24T06:11:25.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:25.814+0000 7f8263f33640 1 -- 172.21.15.92:0/598882218 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f825c0ce040 con 0x7f825c06a3e0 2023-11-24T06:11:25.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:25.814+0000 7f82527fc640 1 -- 172.21.15.92:0/598882218 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8258045400 con 0x7f825c06a3e0 2023-11-24T06:11:25.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:25.814+0000 7f8263f33640 1 -- 172.21.15.92:0/598882218 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f825c0ce600 con 0x7f825c06a3e0 2023-11-24T06:11:25.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:25.815+0000 7f8263f33640 1 -- 172.21.15.92:0/598882218 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f825c0675d0 con 0x7f825c06a3e0 2023-11-24T06:11:25.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:25.816+0000 7f82527fc640 1 -- 172.21.15.92:0/598882218 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f825804e430 con 0x7f825c06a3e0 2023-11-24T06:11:25.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:25.817+0000 7f82527fc640 1 --2- 172.21.15.92:0/598882218 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f824006ae70 0x7f824006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:25.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:25.817+0000 7f82527fc640 1 -- 172.21.15.92:0/598882218 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5883+0+0 (secure 0 0 0) 0x7f82580aba90 con 0x7f825c06a3e0 2023-11-24T06:11:25.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:25.818+0000 7f82614ce640 1 --2- 172.21.15.92:0/598882218 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f824006ae70 0x7f824006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:25.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:25.818+0000 7f82614ce640 1 --2- 172.21.15.92:0/598882218 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f824006ae70 0x7f824006d330 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f825c1f65f0 tx=0x7f824c007a40 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:25.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:25.822+0000 7f82527fc640 1 -- 172.21.15.92:0/598882218 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8258076960 con 0x7f825c06a3e0 2023-11-24T06:11:26.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:26.012+0000 7f8263f33640 1 -- 172.21.15.92:0/598882218 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f825c0b40f0 con 0x7f825c06a3e0 2023-11-24T06:11:26.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:26.013+0000 7f82527fc640 1 -- 172.21.15.92:0/598882218 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7f82580350a0 con 0x7f825c06a3e0 2023-11-24T06:11:26.015 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 1 2023-11-24T06:11:26.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:26.016+0000 7f8263f33640 1 -- 172.21.15.92:0/598882218 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f824006ae70 msgr2=0x7f824006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:26.019 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:26.016+0000 7f8263f33640 1 --2- 172.21.15.92:0/598882218 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f824006ae70 0x7f824006d330 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f825c1f65f0 tx=0x7f824c007a40 comp rx=0 tx=0).stop 2023-11-24T06:11:26.019 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:26.016+0000 7f8263f33640 1 -- 172.21.15.92:0/598882218 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f825c06a3e0 msgr2=0x7f825c153100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:26.019 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:26.016+0000 7f8263f33640 1 --2- 172.21.15.92:0/598882218 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f825c06a3e0 0x7f825c153100 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f8258009890 tx=0x7f82580046c0 comp rx=0 tx=0).stop 2023-11-24T06:11:26.019 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:26.016+0000 7f8263f33640 1 -- 172.21.15.92:0/598882218 shutdown_connections 2023-11-24T06:11:26.019 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:26.016+0000 7f8263f33640 1 --2- 172.21.15.92:0/598882218 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f824006ae70 0x7f824006d330 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:26.019 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:26.016+0000 7f8263f33640 1 --2- 172.21.15.92:0/598882218 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f825c0d0b60 0x7f825c15c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:26.019 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:26.016+0000 7f8263f33640 1 --2- 172.21.15.92:0/598882218 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f825c0aede0 0x7f825c1f30c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:26.019 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:26.016+0000 7f8263f33640 1 --2- 172.21.15.92:0/598882218 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f825c06a3e0 0x7f825c153100 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:26.019 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:26.016+0000 7f8263f33640 1 -- 172.21.15.92:0/598882218 >> 172.21.15.92:0/598882218 conn(0x7f825c0cfef0 msgr2=0x7f825c0b0290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:26.019 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:26.016+0000 7f8263f33640 1 -- 172.21.15.92:0/598882218 shutdown_connections 2023-11-24T06:11:26.019 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:26.016+0000 7f8263f33640 1 -- 172.21.15.92:0/598882218 wait complete. 2023-11-24T06:11:26.038 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T06:11:26.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:26.244+0000 7f1b8dd16640 1 Processor -- start 2023-11-24T06:11:26.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:26.244+0000 7f1b8dd16640 1 -- start start 2023-11-24T06:11:26.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:26.246+0000 7f1b8dd16640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f1b8813e360 0x7f1b880cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:26.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:26.246+0000 7f1b8dd16640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f1b8815f9c0 0x7f1b880cf7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:26.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:26.246+0000 7f1b8dd16640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b8815c5f0 con 0x7f1b8813df80 2023-11-24T06:11:26.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:26.246+0000 7f1b8dd16640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b8806c470 con 0x7f1b8813e360 2023-11-24T06:11:26.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:26.246+0000 7f1b8dd16640 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b8806c750 con 0x7f1b8815f9c0 2023-11-24T06:11:26.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:26.246+0000 7f1b86ffd640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f1b8813e360 0x7f1b880cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:26.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:26.246+0000 7f1b877fe640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f1b8813df80 0x7f1b8815bee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:56268/0 (socket says 172.21.15.92:56268) 2023-11-24T06:11:26.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:26.246+0000 7f1b877fe640 1 -- 172.21.15.92:0/3417746524 learned_addr learned my addr 172.21.15.92:0/3417746524 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-24T06:11:26.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:26.246+0000 7f1b87fff640 1 --2- 172.21.15.92:0/3417746524 >> v2:172.21.15.118:3300/0 conn(0x7f1b8815f9c0 0x7f1b880cf7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:26.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:26.247+0000 7f1b86ffd640 1 -- 172.21.15.92:0/3417746524 >> v2:172.21.15.118:3300/0 conn(0x7f1b8815f9c0 msgr2=0x7f1b880cf7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:26.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:26.247+0000 7f1b86ffd640 1 --2- 172.21.15.92:0/3417746524 >> v2:172.21.15.118:3300/0 conn(0x7f1b8815f9c0 0x7f1b880cf7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:26.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:26.247+0000 7f1b86ffd640 1 -- 172.21.15.92:0/3417746524 >> v1:172.21.15.92:6789/0 conn(0x7f1b8813df80 legacy=0x7f1b8815bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:26.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:26.247+0000 7f1b86ffd640 1 -- 172.21.15.92:0/3417746524 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b8806ca30 con 0x7f1b8813e360 2023-11-24T06:11:26.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:26.247+0000 7f1b867fc640 1 -- 172.21.15.92:0/3417746524 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2325175196 0 0) 0x7f1b8815c5f0 con 0x7f1b8813df80 2023-11-24T06:11:26.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:26.248+0000 7f1b86ffd640 1 --2- 172.21.15.92:0/3417746524 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b8813e360 0x7f1b880cf290 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f1b880ccae0 tx=0x7f1b7c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e1c4979adc81266 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:26.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:26.248+0000 7f1b867fc640 1 -- 172.21.15.92:0/3417746524 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b7c031830 con 0x7f1b8813e360 2023-11-24T06:11:26.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:26.248+0000 7f1b867fc640 1 -- 172.21.15.92:0/3417746524 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b7c031990 con 0x7f1b8813e360 2023-11-24T06:11:26.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:26.248+0000 7f1b867fc640 1 -- 172.21.15.92:0/3417746524 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b7c031d20 con 0x7f1b8813e360 2023-11-24T06:11:26.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:26.249+0000 7f1b8dd16640 1 -- 172.21.15.92:0/3417746524 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b8813e360 msgr2=0x7f1b880cf290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:26.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:26.249+0000 7f1b8dd16640 1 --2- 172.21.15.92:0/3417746524 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b8813e360 0x7f1b880cf290 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f1b880ccae0 tx=0x7f1b7c02b8d0 comp rx=0 tx=0).stop 2023-11-24T06:11:26.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:26.249+0000 7f1b8dd16640 1 -- 172.21.15.92:0/3417746524 shutdown_connections 2023-11-24T06:11:26.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:26.249+0000 7f1b8dd16640 1 --2- 172.21.15.92:0/3417746524 >> v2:172.21.15.118:3300/0 conn(0x7f1b8815f9c0 0x7f1b880cf7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:26.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:26.249+0000 7f1b8dd16640 1 --2- 172.21.15.92:0/3417746524 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b8813e360 0x7f1b880cf290 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:26.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:26.249+0000 7f1b8dd16640 1 -- 172.21.15.92:0/3417746524 >> 172.21.15.92:0/3417746524 conn(0x7f1b8806a800 msgr2=0x7f1b8806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:26.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:26.249+0000 7f1b8dd16640 1 -- 172.21.15.92:0/3417746524 shutdown_connections 2023-11-24T06:11:26.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:26.249+0000 7f1b8dd16640 1 -- 172.21.15.92:0/3417746524 wait complete. 2023-11-24T06:11:26.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:26.250+0000 7f1b8dd16640 1 Processor -- start 2023-11-24T06:11:26.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:26.251+0000 7f1b8dd16640 1 -- start start 2023-11-24T06:11:26.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:26.251+0000 7f1b8dd16640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1b8813df80 0x7f1b8815a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:26.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:26.251+0000 7f1b8dd16640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b8813e360 0x7f1b880af1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:26.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:26.251+0000 7f1b8dd16640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1b8815f9c0 0x7f1b8813d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:26.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:26.251+0000 7f1b8dd16640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b880715b0 con 0x7f1b8813e360 2023-11-24T06:11:26.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:26.251+0000 7f1b8dd16640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b88071890 con 0x7f1b8813df80 2023-11-24T06:11:26.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:26.251+0000 7f1b8dd16640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b88071b70 con 0x7f1b8815f9c0 2023-11-24T06:11:26.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:26.252+0000 7f1b86ffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b8813e360 0x7f1b880af1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:26.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:26.252+0000 7f1b86ffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b8813e360 0x7f1b880af1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45246/0 (socket says 172.21.15.92:45246) 2023-11-24T06:11:26.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:26.252+0000 7f1b86ffd640 1 -- 172.21.15.92:0/1364629565 learned_addr learned my addr 172.21.15.92:0/1364629565 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:11:26.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:26.252+0000 7f1b86ffd640 1 -- 172.21.15.92:0/1364629565 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1b8815f9c0 msgr2=0x7f1b8813d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:26.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:26.252+0000 7f1b86ffd640 1 --2- 172.21.15.92:0/1364629565 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1b8815f9c0 0x7f1b8813d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:26.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:26.252+0000 7f1b86ffd640 1 -- 172.21.15.92:0/1364629565 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1b8813df80 msgr2=0x7f1b8815a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:26.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:26.252+0000 7f1b877fe640 1 --2- 172.21.15.92:0/1364629565 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1b8813df80 0x7f1b8815a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:26.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:26.252+0000 7f1b86ffd640 1 --2- 172.21.15.92:0/1364629565 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1b8813df80 0x7f1b8815a1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:26.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:26.252+0000 7f1b86ffd640 1 -- 172.21.15.92:0/1364629565 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b7c009400 con 0x7f1b8813e360 2023-11-24T06:11:26.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:26.252+0000 7f1b877fe640 1 --2- 172.21.15.92:0/1364629565 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1b8813df80 0x7f1b8815a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:11:26.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:26.252+0000 7f1b86ffd640 1 --2- 172.21.15.92:0/1364629565 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b8813e360 0x7f1b880af1d0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f1b7c000c00 tx=0x7f1b7c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:26.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:26.253+0000 7f1b6ffff640 1 -- 172.21.15.92:0/1364629565 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b7c004230 con 0x7f1b8813e360 2023-11-24T06:11:26.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:26.253+0000 7f1b6ffff640 1 -- 172.21.15.92:0/1364629565 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b7c0043b0 con 0x7f1b8813e360 2023-11-24T06:11:26.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:26.253+0000 7f1b6ffff640 1 -- 172.21.15.92:0/1364629565 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b7c004520 con 0x7f1b8813e360 2023-11-24T06:11:26.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:26.253+0000 7f1b8dd16640 1 -- 172.21.15.92:0/1364629565 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b88071e50 con 0x7f1b8813e360 2023-11-24T06:11:26.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:26.253+0000 7f1b8dd16640 1 -- 172.21.15.92:0/1364629565 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b8815a910 con 0x7f1b8813e360 2023-11-24T06:11:26.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:26.254+0000 7f1b8dd16640 1 -- 172.21.15.92:0/1364629565 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b88072070 con 0x7f1b8813e360 2023-11-24T06:11:26.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:26.255+0000 7f1b6ffff640 1 -- 172.21.15.92:0/1364629565 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f1b7c03a8a0 con 0x7f1b8813e360 2023-11-24T06:11:26.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:26.256+0000 7f1b6ffff640 1 --2- 172.21.15.92:0/1364629565 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f1b7406ae40 0x7f1b7406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:26.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:26.256+0000 7f1b6ffff640 1 -- 172.21.15.92:0/1364629565 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5883+0+0 (secure 0 0 0) 0x7f1b7c0abdc0 con 0x7f1b8813e360 2023-11-24T06:11:26.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:26.256+0000 7f1b877fe640 1 --2- 172.21.15.92:0/1364629565 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f1b7406ae40 0x7f1b7406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:26.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:26.257+0000 7f1b877fe640 1 --2- 172.21.15.92:0/1364629565 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f1b7406ae40 0x7f1b7406d300 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f1b78000f60 tx=0x7f1b78009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:26.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:26.260+0000 7f1b6ffff640 1 -- 172.21.15.92:0/1364629565 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1b7c076ec0 con 0x7f1b8813e360 2023-11-24T06:11:26.450 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:26.447+0000 7f1b8dd16640 1 -- 172.21.15.92:0/1364629565 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f1b8815a910 con 0x7f1b8813e360 2023-11-24T06:11:26.450 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:26.448+0000 7f1b6ffff640 1 -- 172.21.15.92:0/1364629565 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7f1b8815a910 con 0x7f1b8813e360 2023-11-24T06:11:26.451 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 32 2023-11-24T06:11:26.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:26.450+0000 7f1b8dd16640 1 -- 172.21.15.92:0/1364629565 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f1b7406ae40 msgr2=0x7f1b7406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:26.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:26.450+0000 7f1b8dd16640 1 --2- 172.21.15.92:0/1364629565 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f1b7406ae40 0x7f1b7406d300 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f1b78000f60 tx=0x7f1b78009040 comp rx=0 tx=0).stop 2023-11-24T06:11:26.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:26.450+0000 7f1b8dd16640 1 -- 172.21.15.92:0/1364629565 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b8813e360 msgr2=0x7f1b880af1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:26.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:26.450+0000 7f1b8dd16640 1 --2- 172.21.15.92:0/1364629565 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b8813e360 0x7f1b880af1d0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f1b7c000c00 tx=0x7f1b7c0046c0 comp rx=0 tx=0).stop 2023-11-24T06:11:26.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:26.451+0000 7f1b8dd16640 1 -- 172.21.15.92:0/1364629565 shutdown_connections 2023-11-24T06:11:26.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:26.451+0000 7f1b8dd16640 1 --2- 172.21.15.92:0/1364629565 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f1b7406ae40 0x7f1b7406d300 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:26.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:26.451+0000 7f1b8dd16640 1 --2- 172.21.15.92:0/1364629565 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1b8815f9c0 0x7f1b8813d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:26.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:26.451+0000 7f1b8dd16640 1 --2- 172.21.15.92:0/1364629565 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b8813e360 0x7f1b880af1d0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:26.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:26.451+0000 7f1b8dd16640 1 --2- 172.21.15.92:0/1364629565 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1b8813df80 0x7f1b8815a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:26.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:26.451+0000 7f1b8dd16640 1 -- 172.21.15.92:0/1364629565 >> 172.21.15.92:0/1364629565 conn(0x7f1b8806a800 msgr2=0x7f1b880cf260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:26.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:26.451+0000 7f1b8dd16640 1 -- 172.21.15.92:0/1364629565 shutdown_connections 2023-11-24T06:11:26.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:26.451+0000 7f1b8dd16640 1 -- 172.21.15.92:0/1364629565 wait complete. 2023-11-24T06:11:26.471 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T06:11:26.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:26.681+0000 7f230a716640 1 Processor -- start 2023-11-24T06:11:26.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:26.682+0000 7f230a716640 1 -- start start 2023-11-24T06:11:26.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:26.683+0000 7f230a716640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f230413df80 0x7f230415bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:26.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:26.683+0000 7f230a716640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f230413e360 0x7f23040cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:26.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:26.683+0000 7f230a716640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f230406c470 con 0x7f230415f9c0 2023-11-24T06:11:26.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:26.683+0000 7f230a716640 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f230406c750 con 0x7f230413e360 2023-11-24T06:11:26.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:26.683+0000 7f230a716640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f230406ca30 con 0x7f230413df80 2023-11-24T06:11:26.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:26.684+0000 7f2308cb3640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f230415f9c0 0x7f23041408d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:56284/0 (socket says 172.21.15.92:56284) 2023-11-24T06:11:26.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:26.684+0000 7f2308cb3640 1 -- 172.21.15.92:0/4123854589 learned_addr learned my addr 172.21.15.92:0/4123854589 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-24T06:11:26.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:26.684+0000 7f22fbfff640 1 --2- 172.21.15.92:0/4123854589 >> v2:172.21.15.118:3300/0 conn(0x7f230413e360 0x7f23040cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:26.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:26.684+0000 7f2303fff640 1 --2- 172.21.15.92:0/4123854589 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f230413df80 0x7f230415bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:26.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:26.684+0000 7f23037fe640 1 -- 172.21.15.92:0/4123854589 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2379814181 0 0) 0x7f230406c470 con 0x7f230415f9c0 2023-11-24T06:11:26.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:26.684+0000 7f23037fe640 1 -- 172.21.15.92:0/4123854589 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f22f0003520 con 0x7f230415f9c0 2023-11-24T06:11:26.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:26.685+0000 7f23037fe640 1 -- 172.21.15.92:0/4123854589 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1345843573 0 0) 0x7f22f0003520 con 0x7f230415f9c0 2023-11-24T06:11:26.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:26.685+0000 7f23037fe640 1 -- 172.21.15.92:0/4123854589 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f230413df80 msgr2=0x7f230415bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:26.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:26.685+0000 7f23037fe640 1 --2- 172.21.15.92:0/4123854589 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f230413df80 0x7f230415bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:26.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:26.685+0000 7f23037fe640 1 -- 172.21.15.92:0/4123854589 >> v2:172.21.15.118:3300/0 conn(0x7f230413e360 msgr2=0x7f23040cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:26.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:26.685+0000 7f23037fe640 1 --2- 172.21.15.92:0/4123854589 >> v2:172.21.15.118:3300/0 conn(0x7f230413e360 0x7f23040cf290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:26.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:26.685+0000 7f23037fe640 1 -- 172.21.15.92:0/4123854589 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f230406cd10 con 0x7f230415f9c0 2023-11-24T06:11:26.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:26.685+0000 7f23037fe640 1 -- 172.21.15.92:0/4123854589 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2607661350 0 0) 0x7f22fc0019f0 con 0x7f230415f9c0 2023-11-24T06:11:26.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:26.685+0000 7f23037fe640 1 -- 172.21.15.92:0/4123854589 >> v1:172.21.15.92:6789/0 conn(0x7f230415f9c0 legacy=0x7f23041408d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:26.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:26.685+0000 7f23037fe640 1 --2- 172.21.15.92:0/4123854589 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f230413df80 0x7f22f0003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:26.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:26.685+0000 7f23037fe640 1 --2- 172.21.15.92:0/4123854589 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f22f0003f40 0x7f22f00047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:26.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:26.685+0000 7f23037fe640 1 --2- 172.21.15.92:0/4123854589 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f22f0004d00 0x7f22f0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:26.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:26.685+0000 7f23037fe640 1 -- 172.21.15.92:0/4123854589 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f230406c750 con 0x7f22f0003f40 2023-11-24T06:11:26.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:26.685+0000 7f23037fe640 1 -- 172.21.15.92:0/4123854589 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f230406ca30 con 0x7f22f0004d00 2023-11-24T06:11:26.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:26.685+0000 7f23037fe640 1 -- 172.21.15.92:0/4123854589 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22f000b980 con 0x7f230413df80 2023-11-24T06:11:26.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:26.685+0000 7f22fbfff640 1 --2- 172.21.15.92:0/4123854589 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f22f0003f40 0x7f22f00047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:26.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:26.685+0000 7f2308cb3640 1 --2- 172.21.15.92:0/4123854589 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f22f0004d00 0x7f22f0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:26.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:26.685+0000 7f22fbfff640 1 -- 172.21.15.92:0/4123854589 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f230413df80 msgr2=0x7f22f0003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:26.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:26.685+0000 7f2303fff640 1 --2- 172.21.15.92:0/4123854589 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f230413df80 0x7f22f0003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:26.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:26.685+0000 7f22fbfff640 1 --2- 172.21.15.92:0/4123854589 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f230413df80 0x7f22f0003970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:26.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:26.685+0000 7f22fbfff640 1 -- 172.21.15.92:0/4123854589 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f22f0004d00 msgr2=0x7f22f0005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:26.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:26.685+0000 7f22fbfff640 1 --2- 172.21.15.92:0/4123854589 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f22f0004d00 0x7f22f0005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:26.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:26.685+0000 7f22fbfff640 1 -- 172.21.15.92:0/4123854589 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22f0003520 con 0x7f22f0003f40 2023-11-24T06:11:26.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:26.685+0000 7f2308cb3640 1 --2- 172.21.15.92:0/4123854589 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f22f0004d00 0x7f22f0005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:11:26.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:26.687+0000 7f22fbfff640 1 --2- 172.21.15.92:0/4123854589 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f22f0003f40 0x7f22f00047c0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f22ec009af0 tx=0x7f22f00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7acfe5f3e2f81c4 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:26.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:26.687+0000 7f23037fe640 1 -- 172.21.15.92:0/4123854589 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22ec02c710 con 0x7f22f0003f40 2023-11-24T06:11:26.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:26.687+0000 7f23037fe640 1 -- 172.21.15.92:0/4123854589 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22ec02c870 con 0x7f22f0003f40 2023-11-24T06:11:26.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:26.687+0000 7f23037fe640 1 -- 172.21.15.92:0/4123854589 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22ec02cc00 con 0x7f22f0003f40 2023-11-24T06:11:26.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:26.688+0000 7f230a716640 1 -- 172.21.15.92:0/4123854589 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f22f0003f40 msgr2=0x7f22f00047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:26.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:26.688+0000 7f230a716640 1 --2- 172.21.15.92:0/4123854589 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f22f0003f40 0x7f22f00047c0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f22ec009af0 tx=0x7f22f00057f0 comp rx=0 tx=0).stop 2023-11-24T06:11:26.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:26.688+0000 7f2308cb3640 1 -- 172.21.15.92:0/4123854589 reap_dead start 2023-11-24T06:11:26.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:26.688+0000 7f230a716640 1 -- 172.21.15.92:0/4123854589 shutdown_connections 2023-11-24T06:11:26.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:26.688+0000 7f230a716640 1 -- 172.21.15.92:0/4123854589 >> 172.21.15.92:0/4123854589 conn(0x7f230406a800 msgr2=0x7f230406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:26.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:26.688+0000 7f230a716640 1 -- 172.21.15.92:0/4123854589 shutdown_connections 2023-11-24T06:11:26.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:26.688+0000 7f230a716640 1 -- 172.21.15.92:0/4123854589 wait complete. 2023-11-24T06:11:26.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:26.690+0000 7f230a716640 1 Processor -- start 2023-11-24T06:11:26.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:26.690+0000 7f230a716640 1 -- start start 2023-11-24T06:11:26.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:26.691+0000 7f230a716640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f22f0003f40 0x7f23041f0c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:26.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:26.691+0000 7f230a716640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2304151510 0x7f230414e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:26.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:26.691+0000 7f230a716640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f230413d180 0x7f230413d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:26.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:26.691+0000 7f230a716640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f230413db70 con 0x7f2304151510 2023-11-24T06:11:26.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:26.691+0000 7f230a716640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2304136120 con 0x7f230413d180 2023-11-24T06:11:26.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:26.691+0000 7f230a716640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f230414ee90 con 0x7f22f0003f40 2023-11-24T06:11:26.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:26.692+0000 7f22fbfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2304151510 0x7f230414e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:26.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:26.692+0000 7f22fbfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2304151510 0x7f230414e610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45256/0 (socket says 172.21.15.92:45256) 2023-11-24T06:11:26.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:26.692+0000 7f22fbfff640 1 -- 172.21.15.92:0/585672088 learned_addr learned my addr 172.21.15.92:0/585672088 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:11:26.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:26.692+0000 7f2303fff640 1 --2- 172.21.15.92:0/585672088 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f22f0003f40 0x7f23041f0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:26.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:26.692+0000 7f2308cb3640 1 --2- 172.21.15.92:0/585672088 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f230413d180 0x7f230413d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:26.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:26.692+0000 7f22fbfff640 1 -- 172.21.15.92:0/585672088 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f22f0003f40 msgr2=0x7f23041f0c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:26.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:26.692+0000 7f22fbfff640 1 --2- 172.21.15.92:0/585672088 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f22f0003f40 0x7f23041f0c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:26.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:26.692+0000 7f22fbfff640 1 -- 172.21.15.92:0/585672088 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f230413d180 msgr2=0x7f230413d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:26.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:26.692+0000 7f22fbfff640 1 --2- 172.21.15.92:0/585672088 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f230413d180 0x7f230413d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:26.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:26.692+0000 7f22fbfff640 1 -- 172.21.15.92:0/585672088 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f230414f170 con 0x7f2304151510 2023-11-24T06:11:26.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:26.692+0000 7f2308cb3640 1 --2- 172.21.15.92:0/585672088 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f230413d180 0x7f230413d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:11:26.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:26.693+0000 7f22fbfff640 1 --2- 172.21.15.92:0/585672088 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2304151510 0x7f230414e610 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f22ec037040 tx=0x7f22ec004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:26.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:26.693+0000 7f23017fa640 1 -- 172.21.15.92:0/585672088 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22ec003810 con 0x7f2304151510 2023-11-24T06:11:26.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:26.693+0000 7f23017fa640 1 -- 172.21.15.92:0/585672088 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22ec02de90 con 0x7f2304151510 2023-11-24T06:11:26.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:26.693+0000 7f23017fa640 1 -- 172.21.15.92:0/585672088 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22ec003b70 con 0x7f2304151510 2023-11-24T06:11:26.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:26.693+0000 7f230a716640 1 -- 172.21.15.92:0/585672088 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f230414bd40 con 0x7f2304151510 2023-11-24T06:11:26.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:26.693+0000 7f230a716640 1 -- 172.21.15.92:0/585672088 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f230414c300 con 0x7f2304151510 2023-11-24T06:11:26.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:26.694+0000 7f230a716640 1 -- 172.21.15.92:0/585672088 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f230414bba0 con 0x7f2304151510 2023-11-24T06:11:26.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:26.695+0000 7f23017fa640 1 -- 172.21.15.92:0/585672088 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f22ec003cd0 con 0x7f2304151510 2023-11-24T06:11:26.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:26.696+0000 7f23017fa640 1 --2- 172.21.15.92:0/585672088 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f22e406ae70 0x7f22e406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:26.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:26.696+0000 7f23017fa640 1 -- 172.21.15.92:0/585672088 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5883+0+0 (secure 0 0 0) 0x7f22ec0ab6e0 con 0x7f2304151510 2023-11-24T06:11:26.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:26.696+0000 7f2303fff640 1 --2- 172.21.15.92:0/585672088 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f22e406ae70 0x7f22e406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:26.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:26.697+0000 7f2303fff640 1 --2- 172.21.15.92:0/585672088 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f22e406ae70 0x7f22e406d330 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f22f4009930 tx=0x7f22f4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:26.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:26.700+0000 7f23017fa640 1 -- 172.21.15.92:0/585672088 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f230414bba0 con 0x7f2304151510 2023-11-24T06:11:26.892 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:26.890+0000 7f230a716640 1 -- 172.21.15.92:0/585672088 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f230414bba0 con 0x7f2304151510 2023-11-24T06:11:26.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:26.891+0000 7f23017fa640 1 -- 172.21.15.92:0/585672088 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7f230414bba0 con 0x7f2304151510 2023-11-24T06:11:26.893 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 64 2023-11-24T06:11:26.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:26.893+0000 7f230a716640 1 -- 172.21.15.92:0/585672088 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f22e406ae70 msgr2=0x7f22e406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:26.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:26.893+0000 7f230a716640 1 --2- 172.21.15.92:0/585672088 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f22e406ae70 0x7f22e406d330 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f22f4009930 tx=0x7f22f4008040 comp rx=0 tx=0).stop 2023-11-24T06:11:26.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:26.893+0000 7f230a716640 1 -- 172.21.15.92:0/585672088 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2304151510 msgr2=0x7f230414e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:26.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:26.893+0000 7f230a716640 1 --2- 172.21.15.92:0/585672088 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2304151510 0x7f230414e610 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f22ec037040 tx=0x7f22ec004730 comp rx=0 tx=0).stop 2023-11-24T06:11:26.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:26.893+0000 7f230a716640 1 -- 172.21.15.92:0/585672088 shutdown_connections 2023-11-24T06:11:26.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:26.894+0000 7f230a716640 1 --2- 172.21.15.92:0/585672088 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f22e406ae70 0x7f22e406d330 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:26.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:26.894+0000 7f230a716640 1 --2- 172.21.15.92:0/585672088 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f230413d180 0x7f230413d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:26.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:26.894+0000 7f230a716640 1 --2- 172.21.15.92:0/585672088 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2304151510 0x7f230414e610 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:26.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:26.894+0000 7f230a716640 1 --2- 172.21.15.92:0/585672088 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f22f0003f40 0x7f23041f0c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:26.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:26.894+0000 7f230a716640 1 -- 172.21.15.92:0/585672088 >> 172.21.15.92:0/585672088 conn(0x7f230406a800 msgr2=0x7f230415d930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:26.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:26.894+0000 7f230a716640 1 -- 172.21.15.92:0/585672088 shutdown_connections 2023-11-24T06:11:26.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:26.894+0000 7f230a716640 1 -- 172.21.15.92:0/585672088 wait complete. 2023-11-24T06:11:26.916 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-24T06:11:27.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:27.127+0000 7f652a1e3640 1 Processor -- start 2023-11-24T06:11:27.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:27.127+0000 7f652a1e3640 1 -- start start 2023-11-24T06:11:27.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:27.129+0000 7f652a1e3640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f652413df80 0x7f652415bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:27.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:27.129+0000 7f652a1e3640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f652406c470 con 0x7f652413e360 2023-11-24T06:11:27.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:27.129+0000 7f652a1e3640 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f652406c750 con 0x7f65240cf9a0 2023-11-24T06:11:27.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:27.129+0000 7f652a1e3640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f652406ca30 con 0x7f652413df80 2023-11-24T06:11:27.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:27.129+0000 7f65237fe640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f652413df80 0x7f652415bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:27.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:27.129+0000 7f651bfff640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f652413e360 0x7f65240cf290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:56298/0 (socket says 172.21.15.92:56298) 2023-11-24T06:11:27.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:27.129+0000 7f651bfff640 1 -- 172.21.15.92:0/4285434616 learned_addr learned my addr 172.21.15.92:0/4285434616 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-24T06:11:27.132 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:27.129+0000 7f65237fe640 1 -- 172.21.15.92:0/4285434616 >> v1:172.21.15.118:6789/0 conn(0x7f65240cf9a0 legacy=0x7f65240cd840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:27.132 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:27.129+0000 7f65237fe640 1 -- 172.21.15.92:0/4285434616 >> v1:172.21.15.92:6789/0 conn(0x7f652413e360 legacy=0x7f65240cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:27.132 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:27.130+0000 7f65237fe640 1 -- 172.21.15.92:0/4285434616 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f652406cd10 con 0x7f652413df80 2023-11-24T06:11:27.133 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:27.130+0000 7f6522ffd640 1 -- 172.21.15.92:0/4285434616 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1743101089 0 0) 0x7f652406c470 con 0x7f652413e360 2023-11-24T06:11:27.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:27.131+0000 7f65237fe640 1 --2- 172.21.15.92:0/4285434616 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f652413df80 0x7f652415bee0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f65140097a0 tx=0x7f651402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=754d0f2357074b54 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:27.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:27.131+0000 7f6522ffd640 1 -- 172.21.15.92:0/4285434616 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6514031830 con 0x7f652413df80 2023-11-24T06:11:27.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:27.131+0000 7f6522ffd640 1 -- 172.21.15.92:0/4285434616 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6514031990 con 0x7f652413df80 2023-11-24T06:11:27.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:27.131+0000 7f6522ffd640 1 -- 172.21.15.92:0/4285434616 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6514031d20 con 0x7f652413df80 2023-11-24T06:11:27.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:27.131+0000 7f652a1e3640 1 -- 172.21.15.92:0/4285434616 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f652413df80 msgr2=0x7f652415bee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:27.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:27.131+0000 7f652a1e3640 1 --2- 172.21.15.92:0/4285434616 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f652413df80 0x7f652415bee0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f65140097a0 tx=0x7f651402b8d0 comp rx=0 tx=0).stop 2023-11-24T06:11:27.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:27.132+0000 7f652a1e3640 1 -- 172.21.15.92:0/4285434616 shutdown_connections 2023-11-24T06:11:27.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:27.132+0000 7f652a1e3640 1 --2- 172.21.15.92:0/4285434616 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f652413df80 0x7f652415bee0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:27.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:27.132+0000 7f652a1e3640 1 -- 172.21.15.92:0/4285434616 >> 172.21.15.92:0/4285434616 conn(0x7f652406a800 msgr2=0x7f652406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:27.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:27.132+0000 7f652a1e3640 1 -- 172.21.15.92:0/4285434616 shutdown_connections 2023-11-24T06:11:27.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:27.132+0000 7f652a1e3640 1 -- 172.21.15.92:0/4285434616 wait complete. 2023-11-24T06:11:27.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:27.133+0000 7f652a1e3640 1 Processor -- start 2023-11-24T06:11:27.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:27.133+0000 7f652a1e3640 1 -- start start 2023-11-24T06:11:27.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:27.134+0000 7f652a1e3640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f65240cf9a0 0x7f65240b3500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:27.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:27.134+0000 7f652a1e3640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f652413df80 0x7f65241f5b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:27.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:27.134+0000 7f652a1e3640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f652413e360 0x7f65241e6430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:27.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:27.134+0000 7f652a1e3640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65241f5710 con 0x7f652413df80 2023-11-24T06:11:27.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:27.134+0000 7f652a1e3640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65241f4a90 con 0x7f65240cf9a0 2023-11-24T06:11:27.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:27.134+0000 7f652a1e3640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65241f4d70 con 0x7f652413e360 2023-11-24T06:11:27.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:27.135+0000 7f651bfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f652413df80 0x7f65241f5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:27.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:27.135+0000 7f651bfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f652413df80 0x7f65241f5b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45272/0 (socket says 172.21.15.92:45272) 2023-11-24T06:11:27.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:27.135+0000 7f651bfff640 1 -- 172.21.15.92:0/3609619109 learned_addr learned my addr 172.21.15.92:0/3609619109 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:11:27.138 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:27.135+0000 7f651bfff640 1 -- 172.21.15.92:0/3609619109 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f652413e360 msgr2=0x7f65241e6430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:27.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:27.135+0000 7f651bfff640 1 --2- 172.21.15.92:0/3609619109 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f652413e360 0x7f65241e6430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:27.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:27.135+0000 7f651bfff640 1 -- 172.21.15.92:0/3609619109 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f65240cf9a0 msgr2=0x7f65240b3500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:27.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:27.135+0000 7f651bfff640 1 --2- 172.21.15.92:0/3609619109 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f65240cf9a0 0x7f65240b3500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:27.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:27.135+0000 7f651bfff640 1 -- 172.21.15.92:0/3609619109 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6514009400 con 0x7f652413df80 2023-11-24T06:11:27.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:27.135+0000 7f651bfff640 1 --2- 172.21.15.92:0/3609619109 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f652413df80 0x7f65241f5b30 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f650c00c4b0 tx=0x7f650c00c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:27.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:27.136+0000 7f6520ff9640 1 -- 172.21.15.92:0/3609619109 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f650c0074e0 con 0x7f652413df80 2023-11-24T06:11:27.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:27.136+0000 7f6520ff9640 1 -- 172.21.15.92:0/3609619109 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f650c007660 con 0x7f652413df80 2023-11-24T06:11:27.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:27.136+0000 7f6520ff9640 1 -- 172.21.15.92:0/3609619109 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f650c004a20 con 0x7f652413df80 2023-11-24T06:11:27.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:27.136+0000 7f652a1e3640 1 -- 172.21.15.92:0/3609619109 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65241f5050 con 0x7f652413df80 2023-11-24T06:11:27.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:27.136+0000 7f652a1e3640 1 -- 172.21.15.92:0/3609619109 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65240b3a40 con 0x7f652413df80 2023-11-24T06:11:27.140 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:27.137+0000 7f652a1e3640 1 -- 172.21.15.92:0/3609619109 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65240b3c30 con 0x7f652413df80 2023-11-24T06:11:27.140 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:27.138+0000 7f6520ff9640 1 -- 172.21.15.92:0/3609619109 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f650c004b80 con 0x7f652413df80 2023-11-24T06:11:27.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:27.139+0000 7f6520ff9640 1 --2- 172.21.15.92:0/3609619109 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f650406ae40 0x7f650406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:27.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:27.139+0000 7f6520ff9640 1 -- 172.21.15.92:0/3609619109 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5883+0+0 (secure 0 0 0) 0x7f650c088930 con 0x7f652413df80 2023-11-24T06:11:27.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:27.139+0000 7f65237fe640 1 --2- 172.21.15.92:0/3609619109 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f650406ae40 0x7f650406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:27.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:27.140+0000 7f65237fe640 1 --2- 172.21.15.92:0/3609619109 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f650406ae40 0x7f650406d300 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f6514036000 tx=0x7f6514030000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:27.146 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:27.144+0000 7f6520ff9640 1 -- 172.21.15.92:0/3609619109 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f650c053950 con 0x7f652413df80 2023-11-24T06:11:27.349 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:27.347+0000 7f652a1e3640 1 -- 172.21.15.92:0/3609619109 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f65240675d0 con 0x7f652413df80 2023-11-24T06:11:27.350 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:27.347+0000 7f6520ff9640 1 -- 172.21.15.92:0/3609619109 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) v1 ==== 80+0+0 (secure 0 0 0) 0x7f650c018020 con 0x7f652413df80 2023-11-24T06:11:27.350 INFO:teuthology.orchestra.run.smithi092.stderr:listed 0 entries 2023-11-24T06:11:27.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:27.350+0000 7f652a1e3640 1 -- 172.21.15.92:0/3609619109 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f650406ae40 msgr2=0x7f650406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:27.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:27.350+0000 7f652a1e3640 1 --2- 172.21.15.92:0/3609619109 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f650406ae40 0x7f650406d300 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f6514036000 tx=0x7f6514030000 comp rx=0 tx=0).stop 2023-11-24T06:11:27.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:27.350+0000 7f652a1e3640 1 -- 172.21.15.92:0/3609619109 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f652413df80 msgr2=0x7f65241f5b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:27.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:27.350+0000 7f652a1e3640 1 --2- 172.21.15.92:0/3609619109 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f652413df80 0x7f65241f5b30 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f650c00c4b0 tx=0x7f650c00c970 comp rx=0 tx=0).stop 2023-11-24T06:11:27.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:27.350+0000 7f652a1e3640 1 -- 172.21.15.92:0/3609619109 shutdown_connections 2023-11-24T06:11:27.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:27.350+0000 7f652a1e3640 1 --2- 172.21.15.92:0/3609619109 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f650406ae40 0x7f650406d300 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:27.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:27.350+0000 7f652a1e3640 1 --2- 172.21.15.92:0/3609619109 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f652413e360 0x7f65241e6430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:27.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:27.350+0000 7f652a1e3640 1 --2- 172.21.15.92:0/3609619109 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f652413df80 0x7f65241f5b30 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:27.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:27.350+0000 7f652a1e3640 1 --2- 172.21.15.92:0/3609619109 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f65240cf9a0 0x7f65240b3500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:27.354 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:27.350+0000 7f652a1e3640 1 -- 172.21.15.92:0/3609619109 >> 172.21.15.92:0/3609619109 conn(0x7f652406a800 msgr2=0x7f65240aeb50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:27.354 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:27.351+0000 7f652a1e3640 1 -- 172.21.15.92:0/3609619109 shutdown_connections 2023-11-24T06:11:27.354 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:27.351+0000 7f652a1e3640 1 -- 172.21.15.92:0/3609619109 wait complete. 2023-11-24T06:11:27.371 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi118.front.sepia.ceph.com... 2023-11-24T06:11:27.371 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:11:27.371 DEBUG:teuthology.orchestra.run.smithi118:> sudo fusermount -u /home/ubuntu/cephtest/mnt.3 2023-11-24T06:11:27.474 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T06:11:27.474 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi118.front.sepia.ceph.com 2023-11-24T06:11:27.474 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:11:27.474 DEBUG:teuthology.orchestra.run.smithi118:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-11-24T06:11:27.526 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-11-24T06:11:27.526 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T06:11:27.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:27.746+0000 7fdc721f1640 1 Processor -- start 2023-11-24T06:11:27.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:27.747+0000 7fdc721f1640 1 -- start start 2023-11-24T06:11:27.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:27.748+0000 7fdc721f1640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fdc6c15c3a0 0x7fdc6c06a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:27.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:27.748+0000 7fdc721f1640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fdc6c06f550 0x7fdc6c06f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:27.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:27.748+0000 7fdc721f1640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc6c06eb00 con 0x7fdc6c0cd960 2023-11-24T06:11:27.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:27.748+0000 7fdc721f1640 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc6c15c820 con 0x7fdc6c15c3a0 2023-11-24T06:11:27.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:27.748+0000 7fdc721f1640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc6c06f0c0 con 0x7fdc6c06f550 2023-11-24T06:11:27.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:27.748+0000 7fdc6b7fe640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fdc6c0cd960 0x7fdc6c0cdd40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:56312/0 (socket says 172.21.15.92:56312) 2023-11-24T06:11:27.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:27.748+0000 7fdc6b7fe640 1 -- 172.21.15.92:0/2477231702 learned_addr learned my addr 172.21.15.92:0/2477231702 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-24T06:11:27.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:27.749+0000 7fdc6affd640 1 --2- 172.21.15.92:0/2477231702 >> v2:172.21.15.118:3300/0 conn(0x7fdc6c15c3a0 0x7fdc6c06a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:27.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:27.749+0000 7fdc6a7fc640 1 -- 172.21.15.92:0/2477231702 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1044878057 0 0) 0x7fdc6c06eb00 con 0x7fdc6c0cd960 2023-11-24T06:11:27.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:27.749+0000 7fdc6bfff640 1 --2- 172.21.15.92:0/2477231702 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fdc6c06f550 0x7fdc6c06f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:27.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:27.749+0000 7fdc6a7fc640 1 -- 172.21.15.92:0/2477231702 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdc58003520 con 0x7fdc6c0cd960 2023-11-24T06:11:27.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:27.749+0000 7fdc6a7fc640 1 -- 172.21.15.92:0/2477231702 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2497650182 0 0) 0x7fdc58003520 con 0x7fdc6c0cd960 2023-11-24T06:11:27.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:27.749+0000 7fdc6a7fc640 1 -- 172.21.15.92:0/2477231702 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fdc6c06f550 msgr2=0x7fdc6c06f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:27.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:27.749+0000 7fdc6a7fc640 1 --2- 172.21.15.92:0/2477231702 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fdc6c06f550 0x7fdc6c06f9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:27.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:27.749+0000 7fdc6a7fc640 1 -- 172.21.15.92:0/2477231702 >> v2:172.21.15.118:3300/0 conn(0x7fdc6c15c3a0 msgr2=0x7fdc6c06a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:27.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:27.749+0000 7fdc6a7fc640 1 --2- 172.21.15.92:0/2477231702 >> v2:172.21.15.118:3300/0 conn(0x7fdc6c15c3a0 0x7fdc6c06a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:27.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:27.749+0000 7fdc6a7fc640 1 -- 172.21.15.92:0/2477231702 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc6c06a920 con 0x7fdc6c0cd960 2023-11-24T06:11:27.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:27.749+0000 7fdc6a7fc640 1 -- 172.21.15.92:0/2477231702 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2607661350 0 0) 0x7fdc54001670 con 0x7fdc6c0cd960 2023-11-24T06:11:27.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:27.749+0000 7fdc6a7fc640 1 -- 172.21.15.92:0/2477231702 >> v1:172.21.15.92:6789/0 conn(0x7fdc6c0cd960 legacy=0x7fdc6c0cdd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:27.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:27.749+0000 7fdc6a7fc640 1 --2- 172.21.15.92:0/2477231702 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fdc58003970 0x7fdc58003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:27.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:27.749+0000 7fdc6a7fc640 1 --2- 172.21.15.92:0/2477231702 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fdc6c06f550 0x7fdc580047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:27.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:27.750+0000 7fdc6a7fc640 1 --2- 172.21.15.92:0/2477231702 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdc58004d00 0x7fdc58005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:27.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:27.750+0000 7fdc6a7fc640 1 -- 172.21.15.92:0/2477231702 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc6c15c820 con 0x7fdc58004d00 2023-11-24T06:11:27.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:27.750+0000 7fdc6a7fc640 1 -- 172.21.15.92:0/2477231702 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc6c06f0c0 con 0x7fdc58003970 2023-11-24T06:11:27.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:27.750+0000 7fdc6a7fc640 1 -- 172.21.15.92:0/2477231702 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc5800b9b0 con 0x7fdc6c06f550 2023-11-24T06:11:27.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:27.750+0000 7fdc6bfff640 1 --2- 172.21.15.92:0/2477231702 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdc58004d00 0x7fdc58005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:27.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:27.750+0000 7fdc6b7fe640 1 --2- 172.21.15.92:0/2477231702 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fdc58003970 0x7fdc58003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:27.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:27.750+0000 7fdc6bfff640 1 -- 172.21.15.92:0/2477231702 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fdc6c06f550 msgr2=0x7fdc580047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:27.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:27.750+0000 7fdc6bfff640 1 --2- 172.21.15.92:0/2477231702 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fdc6c06f550 0x7fdc580047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:27.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:27.750+0000 7fdc6bfff640 1 -- 172.21.15.92:0/2477231702 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fdc58003970 msgr2=0x7fdc58003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:27.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:27.750+0000 7fdc6bfff640 1 --2- 172.21.15.92:0/2477231702 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fdc58003970 0x7fdc58003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:27.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:27.750+0000 7fdc6bfff640 1 -- 172.21.15.92:0/2477231702 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc58003520 con 0x7fdc58004d00 2023-11-24T06:11:27.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:27.751+0000 7fdc6bfff640 1 --2- 172.21.15.92:0/2477231702 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdc58004d00 0x7fdc58005180 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fdc64009ac0 tx=0x7fdc580057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a07c0f2edfa3e0bf server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:27.756 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:27.751+0000 7fdc6a7fc640 1 -- 172.21.15.92:0/2477231702 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc6402c710 con 0x7fdc58004d00 2023-11-24T06:11:27.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:27.752+0000 7fdc6a7fc640 1 -- 172.21.15.92:0/2477231702 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc6402c870 con 0x7fdc58004d00 2023-11-24T06:11:27.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:27.752+0000 7fdc6a7fc640 1 -- 172.21.15.92:0/2477231702 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc6402cc00 con 0x7fdc58004d00 2023-11-24T06:11:27.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:27.752+0000 7fdc721f1640 1 -- 172.21.15.92:0/2477231702 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdc58004d00 msgr2=0x7fdc58005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:27.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:27.752+0000 7fdc721f1640 1 --2- 172.21.15.92:0/2477231702 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdc58004d00 0x7fdc58005180 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fdc64009ac0 tx=0x7fdc580057f0 comp rx=0 tx=0).stop 2023-11-24T06:11:27.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:27.752+0000 7fdc6bfff640 1 -- 172.21.15.92:0/2477231702 reap_dead start 2023-11-24T06:11:27.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:27.752+0000 7fdc721f1640 1 -- 172.21.15.92:0/2477231702 shutdown_connections 2023-11-24T06:11:27.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:27.752+0000 7fdc721f1640 1 -- 172.21.15.92:0/2477231702 >> 172.21.15.92:0/2477231702 conn(0x7fdc6c0b1dd0 msgr2=0x7fdc6c0b21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:27.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:27.752+0000 7fdc721f1640 1 -- 172.21.15.92:0/2477231702 shutdown_connections 2023-11-24T06:11:27.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:27.752+0000 7fdc721f1640 1 -- 172.21.15.92:0/2477231702 wait complete. 2023-11-24T06:11:27.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:27.754+0000 7fdc721f1640 1 Processor -- start 2023-11-24T06:11:27.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:27.754+0000 7fdc721f1640 1 -- start start 2023-11-24T06:11:27.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:27.755+0000 7fdc721f1640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fdc6c150730 0x7fdc6c13c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:27.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:27.755+0000 7fdc721f1640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fdc6c140c70 0x7fdc6c1efc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:27.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:27.755+0000 7fdc721f1640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdc6c150b10 0x7fdc6c1f0170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:27.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:27.755+0000 7fdc721f1640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc6c1f06b0 con 0x7fdc6c150b10 2023-11-24T06:11:27.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:27.755+0000 7fdc721f1640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc6c13c9f0 con 0x7fdc6c150730 2023-11-24T06:11:27.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:27.755+0000 7fdc721f1640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc6c13ccd0 con 0x7fdc6c140c70 2023-11-24T06:11:27.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:27.755+0000 7fdc6bfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdc6c150b10 0x7fdc6c1f0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:27.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:27.756+0000 7fdc6bfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdc6c150b10 0x7fdc6c1f0170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45286/0 (socket says 172.21.15.92:45286) 2023-11-24T06:11:27.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:27.756+0000 7fdc6bfff640 1 -- 172.21.15.92:0/3882282379 learned_addr learned my addr 172.21.15.92:0/3882282379 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:11:27.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:27.756+0000 7fdc6b7fe640 1 --2- 172.21.15.92:0/3882282379 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fdc6c150730 0x7fdc6c13c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:27.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:27.756+0000 7fdc6affd640 1 --2- 172.21.15.92:0/3882282379 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fdc6c140c70 0x7fdc6c1efc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:27.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:27.756+0000 7fdc6bfff640 1 -- 172.21.15.92:0/3882282379 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fdc6c140c70 msgr2=0x7fdc6c1efc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:27.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:27.756+0000 7fdc6bfff640 1 --2- 172.21.15.92:0/3882282379 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fdc6c140c70 0x7fdc6c1efc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:27.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:27.756+0000 7fdc6bfff640 1 -- 172.21.15.92:0/3882282379 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fdc6c150730 msgr2=0x7fdc6c13c0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:27.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:27.756+0000 7fdc6bfff640 1 --2- 172.21.15.92:0/3882282379 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fdc6c150730 0x7fdc6c13c0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:27.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:27.756+0000 7fdc6bfff640 1 -- 172.21.15.92:0/3882282379 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc6c13cfb0 con 0x7fdc6c150b10 2023-11-24T06:11:27.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:27.756+0000 7fdc6bfff640 1 --2- 172.21.15.92:0/3882282379 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdc6c150b10 0x7fdc6c1f0170 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fdc64009ac0 tx=0x7fdc64004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:27.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:27.757+0000 7fdc63fff640 1 -- 172.21.15.92:0/3882282379 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc6402c710 con 0x7fdc6c150b10 2023-11-24T06:11:27.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:27.757+0000 7fdc63fff640 1 -- 172.21.15.92:0/3882282379 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc64003810 con 0x7fdc6c150b10 2023-11-24T06:11:27.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:27.757+0000 7fdc63fff640 1 -- 172.21.15.92:0/3882282379 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc64003bc0 con 0x7fdc6c150b10 2023-11-24T06:11:27.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:27.757+0000 7fdc721f1640 1 -- 172.21.15.92:0/3882282379 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc6c13dd30 con 0x7fdc6c150b10 2023-11-24T06:11:27.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:27.757+0000 7fdc721f1640 1 -- 172.21.15.92:0/3882282379 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc6c13c630 con 0x7fdc6c150b10 2023-11-24T06:11:27.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:27.758+0000 7fdc721f1640 1 -- 172.21.15.92:0/3882282379 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc6c13c7c0 con 0x7fdc6c150b10 2023-11-24T06:11:27.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:27.759+0000 7fdc63fff640 1 -- 172.21.15.92:0/3882282379 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fdc64003d20 con 0x7fdc6c150b10 2023-11-24T06:11:27.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:27.759+0000 7fdc63fff640 1 --2- 172.21.15.92:0/3882282379 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fdc4c06ae40 0x7fdc4c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:27.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:27.760+0000 7fdc63fff640 1 -- 172.21.15.92:0/3882282379 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5883+0+0 (secure 0 0 0) 0x7fdc640aa660 con 0x7fdc6c150b10 2023-11-24T06:11:27.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:27.760+0000 7fdc6b7fe640 1 --2- 172.21.15.92:0/3882282379 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fdc4c06ae40 0x7fdc4c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:27.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:27.760+0000 7fdc6b7fe640 1 --2- 172.21.15.92:0/3882282379 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fdc4c06ae40 0x7fdc4c06d300 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fdc54001eb0 tx=0x7fdc54009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:27.766 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:27.764+0000 7fdc63fff640 1 -- 172.21.15.92:0/3882282379 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fdc640755c0 con 0x7fdc6c150b10 2023-11-24T06:11:27.990 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:27.988+0000 7fdc721f1640 1 -- 172.21.15.92:0/3882282379 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdc6c13e250 con 0x7fdc6c150b10 2023-11-24T06:11:27.991 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:27.989+0000 7fdc63fff640 1 -- 172.21.15.92:0/3882282379 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4910 (secure 0 0 0) 0x7fdc6c13e250 con 0x7fdc6c150b10 2023-11-24T06:11:27.991 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-24T06:11:27.991 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4236,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6836/437604699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":437604699},{"type":"v1","addr":"172.21.15.92:6836","nonce":437604699}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6835/3368711442","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":3368711442},{"type":"v1","addr":"172.21.15.118:6835","nonce":3368711442}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4255,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/2767290380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":2767290380},{"type":"v1","addr":"172.21.15.92:6837","nonce":2767290380}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T06:10:25.115612+0000","modified":"2023-11-24T06:10:28.210438+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4264},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4264":{"gid":4264,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.118:6833/1146052262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":1146052262},{"type":"v1","addr":"172.21.15.118:6833","nonce":1146052262}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-24T06:11:27.991 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 8 2023-11-24T06:11:27.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:27.991+0000 7fdc721f1640 1 -- 172.21.15.92:0/3882282379 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fdc4c06ae40 msgr2=0x7fdc4c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:27.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:27.991+0000 7fdc721f1640 1 --2- 172.21.15.92:0/3882282379 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fdc4c06ae40 0x7fdc4c06d300 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fdc54001eb0 tx=0x7fdc54009040 comp rx=0 tx=0).stop 2023-11-24T06:11:27.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:27.991+0000 7fdc721f1640 1 -- 172.21.15.92:0/3882282379 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdc6c150b10 msgr2=0x7fdc6c1f0170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:27.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:27.991+0000 7fdc721f1640 1 --2- 172.21.15.92:0/3882282379 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdc6c150b10 0x7fdc6c1f0170 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fdc64009ac0 tx=0x7fdc64004730 comp rx=0 tx=0).stop 2023-11-24T06:11:27.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:27.991+0000 7fdc721f1640 1 -- 172.21.15.92:0/3882282379 shutdown_connections 2023-11-24T06:11:27.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:27.991+0000 7fdc721f1640 1 --2- 172.21.15.92:0/3882282379 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fdc4c06ae40 0x7fdc4c06d300 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:27.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:27.991+0000 7fdc721f1640 1 --2- 172.21.15.92:0/3882282379 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdc6c150b10 0x7fdc6c1f0170 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:27.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:27.991+0000 7fdc721f1640 1 --2- 172.21.15.92:0/3882282379 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fdc6c140c70 0x7fdc6c1efc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:27.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:27.991+0000 7fdc721f1640 1 --2- 172.21.15.92:0/3882282379 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fdc6c150730 0x7fdc6c13c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:27.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:27.991+0000 7fdc721f1640 1 -- 172.21.15.92:0/3882282379 >> 172.21.15.92:0/3882282379 conn(0x7fdc6c0b1dd0 msgr2=0x7fdc6c1594e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:27.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:27.991+0000 7fdc721f1640 1 -- 172.21.15.92:0/3882282379 shutdown_connections 2023-11-24T06:11:27.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:27.991+0000 7fdc721f1640 1 -- 172.21.15.92:0/3882282379 wait complete. 2023-11-24T06:11:28.012 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:11:28.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:28.218+0000 7fbcfa21c640 1 Processor -- start 2023-11-24T06:11:28.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:28.218+0000 7fbcfa21c640 1 -- start start 2023-11-24T06:11:28.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:28.220+0000 7fbcfa21c640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fbcf414b2c0 0x7fbcf40cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:28.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:28.220+0000 7fbcfa21c640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fbcf40cf7d0 0x7fbcf4070960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:28.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:28.220+0000 7fbcfa21c640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fbcf4070ea0 0x7fbcf40b1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:28.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:28.220+0000 7fbcfa21c640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbcf414b6a0 con 0x7fbcf414b2c0 2023-11-24T06:11:28.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:28.220+0000 7fbcfa21c640 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbcf40cfbf0 con 0x7fbcf40cf7d0 2023-11-24T06:11:28.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:28.220+0000 7fbcfa21c640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcf4071690 con 0x7fbcf4070ea0 2023-11-24T06:11:28.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:28.220+0000 7fbcf37fe640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fbcf414b2c0 0x7fbcf40cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:28.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:28.221+0000 7fbcf37fe640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fbcf414b2c0 0x7fbcf40cf290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45296/0 (socket says 172.21.15.92:45296) 2023-11-24T06:11:28.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:28.221+0000 7fbcf37fe640 1 -- 172.21.15.92:0/1729459622 learned_addr learned my addr 172.21.15.92:0/1729459622 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:11:28.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:28.221+0000 7fbcf3fff640 1 --2- 172.21.15.92:0/1729459622 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fbcf4070ea0 0x7fbcf40b1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:28.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:28.221+0000 7fbceaffd640 1 --2- 172.21.15.92:0/1729459622 >> v2:172.21.15.118:3300/0 conn(0x7fbcf40cf7d0 0x7fbcf4070960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:28.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:28.221+0000 7fbcf37fe640 1 -- 172.21.15.92:0/1729459622 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fbcf4070ea0 msgr2=0x7fbcf40b1ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:28.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:28.221+0000 7fbcf37fe640 1 --2- 172.21.15.92:0/1729459622 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fbcf4070ea0 0x7fbcf40b1ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:28.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:28.221+0000 7fbcf37fe640 1 -- 172.21.15.92:0/1729459622 >> v2:172.21.15.118:3300/0 conn(0x7fbcf40cf7d0 msgr2=0x7fbcf4070960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:28.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:28.221+0000 7fbcf37fe640 1 --2- 172.21.15.92:0/1729459622 >> v2:172.21.15.118:3300/0 conn(0x7fbcf40cf7d0 0x7fbcf4070960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:28.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:28.221+0000 7fbcf37fe640 1 -- 172.21.15.92:0/1729459622 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcf4071970 con 0x7fbcf414b2c0 2023-11-24T06:11:28.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:28.222+0000 7fbcf37fe640 1 --2- 172.21.15.92:0/1729459622 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbcf414b2c0 0x7fbcf40cf290 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fbcf40cfea0 tx=0x7fbce402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abd1b1e878aa2424 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:28.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:28.223+0000 7fbcf2ffd640 1 -- 172.21.15.92:0/1729459622 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbce4031870 con 0x7fbcf414b2c0 2023-11-24T06:11:28.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:28.223+0000 7fbcf2ffd640 1 -- 172.21.15.92:0/1729459622 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbce40319d0 con 0x7fbcf414b2c0 2023-11-24T06:11:28.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:28.223+0000 7fbcf2ffd640 1 -- 172.21.15.92:0/1729459622 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbce4031d60 con 0x7fbcf414b2c0 2023-11-24T06:11:28.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:28.223+0000 7fbcfa21c640 1 -- 172.21.15.92:0/1729459622 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbcf414b2c0 msgr2=0x7fbcf40cf290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:28.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:28.223+0000 7fbcfa21c640 1 --2- 172.21.15.92:0/1729459622 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbcf414b2c0 0x7fbcf40cf290 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fbcf40cfea0 tx=0x7fbce402b910 comp rx=0 tx=0).stop 2023-11-24T06:11:28.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:28.223+0000 7fbcfa21c640 1 -- 172.21.15.92:0/1729459622 shutdown_connections 2023-11-24T06:11:28.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:28.223+0000 7fbcfa21c640 1 --2- 172.21.15.92:0/1729459622 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fbcf4070ea0 0x7fbcf40b1ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:28.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:28.223+0000 7fbcfa21c640 1 --2- 172.21.15.92:0/1729459622 >> v2:172.21.15.118:3300/0 conn(0x7fbcf40cf7d0 0x7fbcf4070960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:28.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:28.223+0000 7fbcfa21c640 1 --2- 172.21.15.92:0/1729459622 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbcf414b2c0 0x7fbcf40cf290 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:28.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:28.223+0000 7fbcfa21c640 1 -- 172.21.15.92:0/1729459622 >> 172.21.15.92:0/1729459622 conn(0x7fbcf406a800 msgr2=0x7fbcf406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:28.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:28.224+0000 7fbcfa21c640 1 -- 172.21.15.92:0/1729459622 shutdown_connections 2023-11-24T06:11:28.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:28.224+0000 7fbcfa21c640 1 -- 172.21.15.92:0/1729459622 wait complete. 2023-11-24T06:11:28.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:28.225+0000 7fbcfa21c640 1 Processor -- start 2023-11-24T06:11:28.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:28.225+0000 7fbcfa21c640 1 -- start start 2023-11-24T06:11:28.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:28.226+0000 7fbcfa21c640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbcf4070ea0 0x7fbcf41fa530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:28.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:28.226+0000 7fbcfa21c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbcf40cf7d0 0x7fbcf41faa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:28.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:28.226+0000 7fbcfa21c640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fbcf414b2c0 0x7fbcf415ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:28.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:28.226+0000 7fbcfa21c640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcf415e310 con 0x7fbcf40cf7d0 2023-11-24T06:11:28.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:28.226+0000 7fbcfa21c640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcf415d790 con 0x7fbcf4070ea0 2023-11-24T06:11:28.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:28.226+0000 7fbcfa21c640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcf415da70 con 0x7fbcf414b2c0 2023-11-24T06:11:28.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:28.227+0000 7fbceaffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbcf40cf7d0 0x7fbcf41faa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:28.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:28.227+0000 7fbceaffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbcf40cf7d0 0x7fbcf41faa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45306/0 (socket says 172.21.15.92:45306) 2023-11-24T06:11:28.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:28.227+0000 7fbceaffd640 1 -- 172.21.15.92:0/3517136725 learned_addr learned my addr 172.21.15.92:0/3517136725 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:11:28.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:28.227+0000 7fbcf37fe640 1 --2- 172.21.15.92:0/3517136725 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbcf4070ea0 0x7fbcf41fa530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:28.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:28.227+0000 7fbcf3fff640 1 --2- 172.21.15.92:0/3517136725 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fbcf414b2c0 0x7fbcf415ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:28.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:28.227+0000 7fbceaffd640 1 -- 172.21.15.92:0/3517136725 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fbcf414b2c0 msgr2=0x7fbcf415ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:28.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:28.227+0000 7fbceaffd640 1 --2- 172.21.15.92:0/3517136725 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fbcf414b2c0 0x7fbcf415ca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:28.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:28.227+0000 7fbceaffd640 1 -- 172.21.15.92:0/3517136725 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbcf4070ea0 msgr2=0x7fbcf41fa530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:28.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:28.227+0000 7fbceaffd640 1 --2- 172.21.15.92:0/3517136725 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbcf4070ea0 0x7fbcf41fa530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:28.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:28.227+0000 7fbceaffd640 1 -- 172.21.15.92:0/3517136725 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbce40094d0 con 0x7fbcf40cf7d0 2023-11-24T06:11:28.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:28.227+0000 7fbcf3fff640 1 --2- 172.21.15.92:0/3517136725 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fbcf414b2c0 0x7fbcf415ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:11:28.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:28.227+0000 7fbceaffd640 1 --2- 172.21.15.92:0/3517136725 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbcf40cf7d0 0x7fbcf41faa70 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fbcdc00d5f0 tx=0x7fbcdc00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:28.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:28.228+0000 7fbcf0ff9640 1 -- 172.21.15.92:0/3517136725 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcdc0125f0 con 0x7fbcf40cf7d0 2023-11-24T06:11:28.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:28.228+0000 7fbcf0ff9640 1 -- 172.21.15.92:0/3517136725 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcdc012750 con 0x7fbcf40cf7d0 2023-11-24T06:11:28.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:28.228+0000 7fbcf0ff9640 1 -- 172.21.15.92:0/3517136725 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcdc012b00 con 0x7fbcf40cf7d0 2023-11-24T06:11:28.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:28.228+0000 7fbcfa21c640 1 -- 172.21.15.92:0/3517136725 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbcf415dd50 con 0x7fbcf40cf7d0 2023-11-24T06:11:28.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:28.228+0000 7fbcfa21c640 1 -- 172.21.15.92:0/3517136725 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcf41f6dd0 con 0x7fbcf40cf7d0 2023-11-24T06:11:28.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:28.229+0000 7fbcfa21c640 1 -- 172.21.15.92:0/3517136725 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbcf40675d0 con 0x7fbcf40cf7d0 2023-11-24T06:11:28.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:28.230+0000 7fbcf0ff9640 1 -- 172.21.15.92:0/3517136725 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fbcdc012c60 con 0x7fbcf40cf7d0 2023-11-24T06:11:28.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:28.230+0000 7fbcf0ff9640 1 --2- 172.21.15.92:0/3517136725 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fbcd406ae40 0x7fbcd406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:28.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:28.231+0000 7fbcf0ff9640 1 -- 172.21.15.92:0/3517136725 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5883+0+0 (secure 0 0 0) 0x7fbcdc089bb0 con 0x7fbcf40cf7d0 2023-11-24T06:11:28.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:28.231+0000 7fbcf37fe640 1 --2- 172.21.15.92:0/3517136725 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fbcd406ae40 0x7fbcd406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:28.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:28.231+0000 7fbcf37fe640 1 --2- 172.21.15.92:0/3517136725 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fbcd406ae40 0x7fbcd406d300 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fbce4036000 tx=0x7fbce403c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:28.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:28.235+0000 7fbcf0ff9640 1 -- 172.21.15.92:0/3517136725 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fbcdc054bb0 con 0x7fbcf40cf7d0 2023-11-24T06:11:28.427 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:28.425+0000 7fbcfa21c640 1 -- 172.21.15.92:0/3517136725 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbcf4145cd0 con 0x7fbcf40cf7d0 2023-11-24T06:11:28.430 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:28.428+0000 7fbcf0ff9640 1 -- 172.21.15.92:0/3517136725 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16800 (secure 0 0 0) 0x7fbcdc0581d0 con 0x7fbcf40cf7d0 2023-11-24T06:11:28.431 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-24T06:11:28.431 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":22,"fsid":"2d6dee5f-8aa3-470f-931a-50ade963b1cb","created":"2023-11-24T06:10:04.707204+0000","modified":"2023-11-24T06:11:11.472401+0000","last_up_change":"2023-11-24T06:10:17.598352+0000","last_in_change":"2023-11-24T06:10:09.040793+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:10:17.733098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:10:22.926445+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T06:10:24.151288+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ff37a9fb-135f-471f-b4b0-f7b5772006c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6803","nonce":20560714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6805","nonce":20560714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6809","nonce":20560714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6807","nonce":20560714}]},"public_addr":"172.21.15.92:6803/20560714","cluster_addr":"172.21.15.92:6805/20560714","heartbeat_back_addr":"172.21.15.92:6809/20560714","heartbeat_front_addr":"172.21.15.92:6807/20560714","state":["exists","up"]},{"osd":1,"uuid":"03cdd353-dcf1-4f38-94fe-a795137c06b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6819","nonce":2451239906}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6821","nonce":2451239906}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6825","nonce":2451239906}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6823","nonce":2451239906}]},"public_addr":"172.21.15.92:6819/2451239906","cluster_addr":"172.21.15.92:6821/2451239906","heartbeat_back_addr":"172.21.15.92:6825/2451239906","heartbeat_front_addr":"172.21.15.92:6823/2451239906","state":["exists","up"]},{"osd":2,"uuid":"a4a20d80-5d57-484b-becd-9aa8070dd3aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6811","nonce":1124979277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6813","nonce":1124979277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6817","nonce":1124979277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6815","nonce":1124979277}]},"public_addr":"172.21.15.92:6811/1124979277","cluster_addr":"172.21.15.92:6813/1124979277","heartbeat_back_addr":"172.21.15.92:6817/1124979277","heartbeat_front_addr":"172.21.15.92:6815/1124979277","state":["exists","up"]},{"osd":3,"uuid":"4648d7ce-9b29-4583-9425-4ba8fd2b1903","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6827","nonce":1734115782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6829","nonce":1734115782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6833","nonce":1734115782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6831","nonce":1734115782}]},"public_addr":"172.21.15.92:6827/1734115782","cluster_addr":"172.21.15.92:6829/1734115782","heartbeat_back_addr":"172.21.15.92:6833/1734115782","heartbeat_front_addr":"172.21.15.92:6831/1734115782","state":["exists","up"]},{"osd":4,"uuid":"3bc9d7eb-c6fa-4fcf-a3ec-ade604e05aac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6819","nonce":3425937229}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6822","nonce":3425937229}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6830","nonce":3425937229}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6826","nonce":3425937229}]},"public_addr":"172.21.15.118:6819/3425937229","cluster_addr":"172.21.15.118:6822/3425937229","heartbeat_back_addr":"172.21.15.118:6830/3425937229","heartbeat_front_addr":"172.21.15.118:6826/3425937229","state":["exists","up"]},{"osd":5,"uuid":"dce9ba6d-59ed-46d1-84fc-a84b13de10c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6801","nonce":827455984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6803","nonce":827455984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6807","nonce":827455984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6805","nonce":827455984}]},"public_addr":"172.21.15.118:6801/827455984","cluster_addr":"172.21.15.118:6803/827455984","heartbeat_back_addr":"172.21.15.118:6807/827455984","heartbeat_front_addr":"172.21.15.118:6805/827455984","state":["exists","up"]},{"osd":6,"uuid":"82aec504-ae58-4709-8d30-46f2b12081b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6809","nonce":2946986141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6811","nonce":2946986141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6815","nonce":2946986141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6813","nonce":2946986141}]},"public_addr":"172.21.15.118:6809/2946986141","cluster_addr":"172.21.15.118:6811/2946986141","heartbeat_back_addr":"172.21.15.118:6815/2946986141","heartbeat_front_addr":"172.21.15.118:6813/2946986141","state":["exists","up"]},{"osd":7,"uuid":"4ed4c624-c88f-4d35-bdb1-bf209b5dffb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6817","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6818","nonce":48122059}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6821","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6823","nonce":48122059}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6829","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6831","nonce":48122059}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6825","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6827","nonce":48122059}]},"public_addr":"172.21.15.118:6818/48122059","cluster_addr":"172.21.15.118:6823/48122059","heartbeat_back_addr":"172.21.15.118:6831/48122059","heartbeat_front_addr":"172.21.15.118:6827/48122059","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.536664+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.368324+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.271201+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.342971+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.621832+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.387110+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.632946+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.416032+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.d","mappings":[{"from":5,"to":1}]},{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"3.29","mappings":[{"from":3,"to":2}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.34","mappings":[{"from":4,"to":2}]},{"pgid":"3.3b","mappings":[{"from":7,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:11:28.433 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:28.430+0000 7fbcfa21c640 1 -- 172.21.15.92:0/3517136725 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fbcd406ae40 msgr2=0x7fbcd406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:28.433 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:28.430+0000 7fbcfa21c640 1 --2- 172.21.15.92:0/3517136725 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fbcd406ae40 0x7fbcd406d300 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fbce4036000 tx=0x7fbce403c040 comp rx=0 tx=0).stop 2023-11-24T06:11:28.433 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:28.430+0000 7fbcfa21c640 1 -- 172.21.15.92:0/3517136725 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbcf40cf7d0 msgr2=0x7fbcf41faa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:28.433 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:28.430+0000 7fbcfa21c640 1 --2- 172.21.15.92:0/3517136725 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbcf40cf7d0 0x7fbcf41faa70 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fbcdc00d5f0 tx=0x7fbcdc00dab0 comp rx=0 tx=0).stop 2023-11-24T06:11:28.433 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:28.430+0000 7fbcfa21c640 1 -- 172.21.15.92:0/3517136725 shutdown_connections 2023-11-24T06:11:28.433 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:28.430+0000 7fbcfa21c640 1 --2- 172.21.15.92:0/3517136725 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fbcd406ae40 0x7fbcd406d300 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:28.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:28.430+0000 7fbcfa21c640 1 --2- 172.21.15.92:0/3517136725 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fbcf414b2c0 0x7fbcf415ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:28.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:28.430+0000 7fbcfa21c640 1 --2- 172.21.15.92:0/3517136725 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbcf40cf7d0 0x7fbcf41faa70 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:28.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:28.430+0000 7fbcfa21c640 1 --2- 172.21.15.92:0/3517136725 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbcf4070ea0 0x7fbcf41fa530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:28.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:28.430+0000 7fbcfa21c640 1 -- 172.21.15.92:0/3517136725 >> 172.21.15.92:0/3517136725 conn(0x7fbcf406a800 msgr2=0x7fbcf4149bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:28.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:28.431+0000 7fbcfa21c640 1 -- 172.21.15.92:0/3517136725 shutdown_connections 2023-11-24T06:11:28.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:28.431+0000 7fbcfa21c640 1 -- 172.21.15.92:0/3517136725 wait complete. 2023-11-24T06:11:28.452 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T06:10:17.733098+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T06:10:22.926445+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T06:10:24.151288+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T06:11:28.452 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T06:11:28.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:28.664+0000 7feea2683640 1 Processor -- start 2023-11-24T06:11:28.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:28.664+0000 7feea2683640 1 -- start start 2023-11-24T06:11:28.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:28.666+0000 7feea2683640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fee9c13e360 0x7fee9c0cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:28.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:28.666+0000 7feea2683640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fee9c15f9c0 0x7fee9c0cf7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:28.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:28.666+0000 7feea2683640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee9c15c5f0 con 0x7fee9c13df80 2023-11-24T06:11:28.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:28.666+0000 7feea2683640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee9c06c470 con 0x7fee9c13e360 2023-11-24T06:11:28.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:28.666+0000 7feea2683640 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee9c06c750 con 0x7fee9c15f9c0 2023-11-24T06:11:28.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:28.666+0000 7fee9b7fe640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fee9c13e360 0x7fee9c0cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:28.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:28.666+0000 7fee9bfff640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fee9c13df80 0x7fee9c15bee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:56318/0 (socket says 172.21.15.92:56318) 2023-11-24T06:11:28.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:28.666+0000 7fee9b7fe640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fee9c13e360 0x7fee9c0cf290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45314/0 (socket says 172.21.15.92:45314) 2023-11-24T06:11:28.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:28.666+0000 7fee9bfff640 1 -- 172.21.15.92:0/4266248519 learned_addr learned my addr 172.21.15.92:0/4266248519 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-24T06:11:28.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:28.667+0000 7fee9b7fe640 1 -- 172.21.15.92:0/4266248519 >> v2:172.21.15.118:3300/0 conn(0x7fee9c15f9c0 msgr2=0x7fee9c0cf7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:28.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:28.667+0000 7feea0c20640 1 --2- 172.21.15.92:0/4266248519 >> v2:172.21.15.118:3300/0 conn(0x7fee9c15f9c0 0x7fee9c0cf7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:28.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:28.667+0000 7fee9b7fe640 1 --2- 172.21.15.92:0/4266248519 >> v2:172.21.15.118:3300/0 conn(0x7fee9c15f9c0 0x7fee9c0cf7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:28.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:28.667+0000 7fee9b7fe640 1 -- 172.21.15.92:0/4266248519 >> v1:172.21.15.92:6789/0 conn(0x7fee9c13df80 legacy=0x7fee9c15bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:28.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:28.667+0000 7fee9b7fe640 1 -- 172.21.15.92:0/4266248519 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee9c06ca30 con 0x7fee9c13e360 2023-11-24T06:11:28.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:28.667+0000 7fee9affd640 1 -- 172.21.15.92:0/4266248519 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3255014760 0 0) 0x7fee9c15c5f0 con 0x7fee9c13df80 2023-11-24T06:11:28.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:28.667+0000 7feea0c20640 1 --2- 172.21.15.92:0/4266248519 >> v2:172.21.15.118:3300/0 conn(0x7fee9c15f9c0 0x7fee9c0cf7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:11:28.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:28.668+0000 7fee9b7fe640 1 --2- 172.21.15.92:0/4266248519 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fee9c13e360 0x7fee9c0cf290 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fee9c0ccae0 tx=0x7fee9002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7e9efa23f41e441 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:28.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:28.668+0000 7fee9affd640 1 -- 172.21.15.92:0/4266248519 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee90031830 con 0x7fee9c13e360 2023-11-24T06:11:28.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:28.668+0000 7fee9affd640 1 -- 172.21.15.92:0/4266248519 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee90031990 con 0x7fee9c13e360 2023-11-24T06:11:28.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:28.668+0000 7fee9affd640 1 -- 172.21.15.92:0/4266248519 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee90031d20 con 0x7fee9c13e360 2023-11-24T06:11:28.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:28.668+0000 7feea2683640 1 -- 172.21.15.92:0/4266248519 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fee9c13e360 msgr2=0x7fee9c0cf290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:28.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:28.668+0000 7feea2683640 1 --2- 172.21.15.92:0/4266248519 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fee9c13e360 0x7fee9c0cf290 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fee9c0ccae0 tx=0x7fee9002b8d0 comp rx=0 tx=0).stop 2023-11-24T06:11:28.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:28.669+0000 7feea2683640 1 -- 172.21.15.92:0/4266248519 shutdown_connections 2023-11-24T06:11:28.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:28.669+0000 7feea2683640 1 --2- 172.21.15.92:0/4266248519 >> v2:172.21.15.118:3300/0 conn(0x7fee9c15f9c0 0x7fee9c0cf7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:28.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:28.669+0000 7feea2683640 1 --2- 172.21.15.92:0/4266248519 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fee9c13e360 0x7fee9c0cf290 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:28.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:28.669+0000 7feea2683640 1 -- 172.21.15.92:0/4266248519 >> 172.21.15.92:0/4266248519 conn(0x7fee9c06a800 msgr2=0x7fee9c06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:28.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:28.669+0000 7feea2683640 1 -- 172.21.15.92:0/4266248519 shutdown_connections 2023-11-24T06:11:28.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:28.669+0000 7feea2683640 1 -- 172.21.15.92:0/4266248519 wait complete. 2023-11-24T06:11:28.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:28.670+0000 7feea2683640 1 Processor -- start 2023-11-24T06:11:28.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:28.671+0000 7feea2683640 1 -- start start 2023-11-24T06:11:28.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:28.671+0000 7feea2683640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fee9c13df80 0x7fee9c1501b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:28.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:28.671+0000 7feea2683640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fee9c13e360 0x7fee9c1f6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:28.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:28.671+0000 7feea2683640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fee9c15f9c0 0x7fee9c131b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:28.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:28.671+0000 7feea2683640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee9c131680 con 0x7fee9c13e360 2023-11-24T06:11:28.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:28.671+0000 7feea2683640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee9c131380 con 0x7fee9c15f9c0 2023-11-24T06:11:28.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:28.671+0000 7feea2683640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee9c1506f0 con 0x7fee9c13df80 2023-11-24T06:11:28.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:28.672+0000 7fee9b7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fee9c13e360 0x7fee9c1f6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:28.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:28.672+0000 7fee9b7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fee9c13e360 0x7fee9c1f6240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45324/0 (socket says 172.21.15.92:45324) 2023-11-24T06:11:28.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:28.672+0000 7fee9b7fe640 1 -- 172.21.15.92:0/1017565970 learned_addr learned my addr 172.21.15.92:0/1017565970 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:11:28.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:28.672+0000 7fee9b7fe640 1 -- 172.21.15.92:0/1017565970 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fee9c13df80 msgr2=0x7fee9c1501b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:28.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:28.672+0000 7fee9b7fe640 1 --2- 172.21.15.92:0/1017565970 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fee9c13df80 0x7fee9c1501b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:28.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:28.672+0000 7fee9b7fe640 1 -- 172.21.15.92:0/1017565970 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fee9c15f9c0 msgr2=0x7fee9c131b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:28.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:28.672+0000 7fee9b7fe640 1 --2- 172.21.15.92:0/1017565970 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fee9c15f9c0 0x7fee9c131b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:28.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:28.672+0000 7fee9b7fe640 1 -- 172.21.15.92:0/1017565970 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee90009400 con 0x7fee9c13e360 2023-11-24T06:11:28.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:28.672+0000 7fee9b7fe640 1 --2- 172.21.15.92:0/1017565970 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fee9c13e360 0x7fee9c1f6240 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fee90009890 tx=0x7fee900046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:28.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:28.673+0000 7fee98ff9640 1 -- 172.21.15.92:0/1017565970 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee9003ba70 con 0x7fee9c13e360 2023-11-24T06:11:28.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:28.673+0000 7fee98ff9640 1 -- 172.21.15.92:0/1017565970 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee9003bbf0 con 0x7fee9c13e360 2023-11-24T06:11:28.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:28.673+0000 7feea2683640 1 -- 172.21.15.92:0/1017565970 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee9c1f6a90 con 0x7fee9c13e360 2023-11-24T06:11:28.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:28.673+0000 7fee98ff9640 1 -- 172.21.15.92:0/1017565970 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee90045400 con 0x7fee9c13e360 2023-11-24T06:11:28.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:28.673+0000 7feea2683640 1 -- 172.21.15.92:0/1017565970 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee9c1f6780 con 0x7fee9c13e360 2023-11-24T06:11:28.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:28.674+0000 7feea2683640 1 -- 172.21.15.92:0/1017565970 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee9c0675d0 con 0x7fee9c13e360 2023-11-24T06:11:28.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:28.675+0000 7fee98ff9640 1 -- 172.21.15.92:0/1017565970 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fee9004e440 con 0x7fee9c13e360 2023-11-24T06:11:28.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:28.676+0000 7fee98ff9640 1 --2- 172.21.15.92:0/1017565970 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fee8806ae70 0x7fee8806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:28.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:28.676+0000 7fee98ff9640 1 -- 172.21.15.92:0/1017565970 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5883+0+0 (secure 0 0 0) 0x7fee900ab9f0 con 0x7fee9c13e360 2023-11-24T06:11:28.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:28.676+0000 7fee9bfff640 1 --2- 172.21.15.92:0/1017565970 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fee8806ae70 0x7fee8806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:28.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:28.677+0000 7fee9bfff640 1 --2- 172.21.15.92:0/1017565970 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fee8806ae70 0x7fee8806d330 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fee8c005880 tx=0x7fee8c007a00 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:28.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:28.681+0000 7fee98ff9640 1 -- 172.21.15.92:0/1017565970 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fee900769d0 con 0x7fee9c13e360 2023-11-24T06:11:28.873 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:28.871+0000 7feea2683640 1 -- 172.21.15.92:0/1017565970 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fee9c1317c0 con 0x7fee9c13e360 2023-11-24T06:11:28.874 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:28.872+0000 7fee98ff9640 1 -- 172.21.15.92:0/1017565970 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7fee900350a0 con 0x7fee9c13e360 2023-11-24T06:11:28.875 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 1 2023-11-24T06:11:28.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:28.874+0000 7feea2683640 1 -- 172.21.15.92:0/1017565970 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fee8806ae70 msgr2=0x7fee8806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:28.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:28.874+0000 7feea2683640 1 --2- 172.21.15.92:0/1017565970 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fee8806ae70 0x7fee8806d330 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fee8c005880 tx=0x7fee8c007a00 comp rx=0 tx=0).stop 2023-11-24T06:11:28.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:28.874+0000 7feea2683640 1 -- 172.21.15.92:0/1017565970 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fee9c13e360 msgr2=0x7fee9c1f6240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:28.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:28.874+0000 7feea2683640 1 --2- 172.21.15.92:0/1017565970 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fee9c13e360 0x7fee9c1f6240 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fee90009890 tx=0x7fee900046c0 comp rx=0 tx=0).stop 2023-11-24T06:11:28.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:28.875+0000 7feea2683640 1 -- 172.21.15.92:0/1017565970 shutdown_connections 2023-11-24T06:11:28.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:28.875+0000 7feea2683640 1 --2- 172.21.15.92:0/1017565970 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fee8806ae70 0x7fee8806d330 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:28.878 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:28.875+0000 7feea2683640 1 --2- 172.21.15.92:0/1017565970 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fee9c15f9c0 0x7fee9c131b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:28.878 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:28.875+0000 7feea2683640 1 --2- 172.21.15.92:0/1017565970 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fee9c13e360 0x7fee9c1f6240 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:28.878 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:28.875+0000 7feea2683640 1 --2- 172.21.15.92:0/1017565970 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fee9c13df80 0x7fee9c1501b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:28.878 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:28.875+0000 7feea2683640 1 -- 172.21.15.92:0/1017565970 >> 172.21.15.92:0/1017565970 conn(0x7fee9c06a800 msgr2=0x7fee9c0b0340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:28.878 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:28.875+0000 7feea2683640 1 -- 172.21.15.92:0/1017565970 shutdown_connections 2023-11-24T06:11:28.878 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:28.875+0000 7feea2683640 1 -- 172.21.15.92:0/1017565970 wait complete. 2023-11-24T06:11:28.895 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T06:11:29.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:29.113+0000 7f19d90bb640 1 Processor -- start 2023-11-24T06:11:29.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:29.113+0000 7f19d90bb640 1 -- start start 2023-11-24T06:11:29.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:29.114+0000 7f19d90bb640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f19d40ae5a0 0x7f19d415e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:29.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:29.114+0000 7f19d90bb640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f19d40b3ae0 0x7f19d406cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:29.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:29.114+0000 7f19d90bb640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f19d406d120 0x7f19d40cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:29.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:29.114+0000 7f19d90bb640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19d4071450 con 0x7f19d40b3ae0 2023-11-24T06:11:29.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:29.114+0000 7f19d90bb640 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19d4070c40 con 0x7f19d40ae5a0 2023-11-24T06:11:29.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:29.114+0000 7f19d90bb640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19d406c470 con 0x7f19d406d120 2023-11-24T06:11:29.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:29.115+0000 7f19cbfff640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f19d40b3ae0 0x7f19d406cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:29.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:29.115+0000 7f19cbfff640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f19d40b3ae0 0x7f19d406cbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:57762/0 (socket says 172.21.15.92:57762) 2023-11-24T06:11:29.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:29.115+0000 7f19cbfff640 1 -- 172.21.15.92:0/504223367 learned_addr learned my addr 172.21.15.92:0/504223367 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:11:29.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:29.115+0000 7f19d2d9d640 1 --2- 172.21.15.92:0/504223367 >> v2:172.21.15.118:3300/0 conn(0x7f19d40ae5a0 0x7f19d415e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:29.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:29.115+0000 7f19d359e640 1 --2- 172.21.15.92:0/504223367 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f19d406d120 0x7f19d40cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:29.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:29.115+0000 7f19cbfff640 1 -- 172.21.15.92:0/504223367 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f19d406d120 msgr2=0x7f19d40cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:29.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:29.115+0000 7f19cbfff640 1 --2- 172.21.15.92:0/504223367 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f19d406d120 0x7f19d40cf290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:29.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:29.115+0000 7f19cbfff640 1 -- 172.21.15.92:0/504223367 >> v2:172.21.15.118:3300/0 conn(0x7f19d40ae5a0 msgr2=0x7f19d415e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:29.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:29.115+0000 7f19cbfff640 1 --2- 172.21.15.92:0/504223367 >> v2:172.21.15.118:3300/0 conn(0x7f19d40ae5a0 0x7f19d415e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:29.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:29.115+0000 7f19cbfff640 1 -- 172.21.15.92:0/504223367 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19d406c750 con 0x7f19d40b3ae0 2023-11-24T06:11:29.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:29.116+0000 7f19cbfff640 1 --2- 172.21.15.92:0/504223367 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f19d40b3ae0 0x7f19d406cbe0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f19d40cfe20 tx=0x7f19bc02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=538de56a436b2c97 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:29.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:29.117+0000 7f19d259c640 1 -- 172.21.15.92:0/504223367 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19bc031870 con 0x7f19d40b3ae0 2023-11-24T06:11:29.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:29.117+0000 7f19d259c640 1 -- 172.21.15.92:0/504223367 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19bc0319d0 con 0x7f19d40b3ae0 2023-11-24T06:11:29.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:29.117+0000 7f19d259c640 1 -- 172.21.15.92:0/504223367 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19bc031d60 con 0x7f19d40b3ae0 2023-11-24T06:11:29.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:29.117+0000 7f19d90bb640 1 -- 172.21.15.92:0/504223367 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f19d40b3ae0 msgr2=0x7f19d406cbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:29.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:29.117+0000 7f19d90bb640 1 --2- 172.21.15.92:0/504223367 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f19d40b3ae0 0x7f19d406cbe0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f19d40cfe20 tx=0x7f19bc02b910 comp rx=0 tx=0).stop 2023-11-24T06:11:29.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:29.117+0000 7f19d90bb640 1 -- 172.21.15.92:0/504223367 shutdown_connections 2023-11-24T06:11:29.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:29.117+0000 7f19d90bb640 1 --2- 172.21.15.92:0/504223367 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f19d406d120 0x7f19d40cf290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:29.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:29.117+0000 7f19d90bb640 1 --2- 172.21.15.92:0/504223367 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f19d40b3ae0 0x7f19d406cbe0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:29.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:29.117+0000 7f19d90bb640 1 --2- 172.21.15.92:0/504223367 >> v2:172.21.15.118:3300/0 conn(0x7f19d40ae5a0 0x7f19d415e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:29.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:29.117+0000 7f19d90bb640 1 -- 172.21.15.92:0/504223367 >> 172.21.15.92:0/504223367 conn(0x7f19d406a800 msgr2=0x7f19d406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:29.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:29.118+0000 7f19d90bb640 1 -- 172.21.15.92:0/504223367 shutdown_connections 2023-11-24T06:11:29.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:29.118+0000 7f19d90bb640 1 -- 172.21.15.92:0/504223367 wait complete. 2023-11-24T06:11:29.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:29.119+0000 7f19d90bb640 1 Processor -- start 2023-11-24T06:11:29.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:29.120+0000 7f19d90bb640 1 -- start start 2023-11-24T06:11:29.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:29.120+0000 7f19d90bb640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f19d406d120 0x7f19d415bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:29.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:29.120+0000 7f19d90bb640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f19d40ae5a0 0x7f19d40ce110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:29.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:29.120+0000 7f19d90bb640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f19d40b3ae0 0x7f19d41f2310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:29.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:29.121+0000 7f19d90bb640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19d4148840 con 0x7f19d40ae5a0 2023-11-24T06:11:29.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:29.121+0000 7f19d90bb640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19d415c2d0 con 0x7f19d40b3ae0 2023-11-24T06:11:29.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:29.121+0000 7f19d90bb640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19d415c5b0 con 0x7f19d406d120 2023-11-24T06:11:29.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:29.121+0000 7f19cbfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f19d40ae5a0 0x7f19d40ce110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:29.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:29.121+0000 7f19cbfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f19d40ae5a0 0x7f19d40ce110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:57774/0 (socket says 172.21.15.92:57774) 2023-11-24T06:11:29.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:29.121+0000 7f19cbfff640 1 -- 172.21.15.92:0/2366151119 learned_addr learned my addr 172.21.15.92:0/2366151119 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:11:29.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:29.121+0000 7f19cbfff640 1 -- 172.21.15.92:0/2366151119 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f19d406d120 msgr2=0x7f19d415bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:29.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:29.121+0000 7f19d359e640 1 --2- 172.21.15.92:0/2366151119 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f19d40b3ae0 0x7f19d41f2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:29.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:29.121+0000 7f19d2d9d640 1 --2- 172.21.15.92:0/2366151119 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f19d406d120 0x7f19d415bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:29.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:29.121+0000 7f19cbfff640 1 --2- 172.21.15.92:0/2366151119 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f19d406d120 0x7f19d415bcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:29.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:29.121+0000 7f19cbfff640 1 -- 172.21.15.92:0/2366151119 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f19d40b3ae0 msgr2=0x7f19d41f2310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:29.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:29.121+0000 7f19cbfff640 1 --2- 172.21.15.92:0/2366151119 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f19d40b3ae0 0x7f19d41f2310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:29.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:29.121+0000 7f19cbfff640 1 -- 172.21.15.92:0/2366151119 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19bc0094d0 con 0x7f19d40ae5a0 2023-11-24T06:11:29.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:29.121+0000 7f19d359e640 1 --2- 172.21.15.92:0/2366151119 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f19d40b3ae0 0x7f19d41f2310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:11:29.132 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:29.122+0000 7f19cbfff640 1 --2- 172.21.15.92:0/2366151119 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f19d40ae5a0 0x7f19d40ce110 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f19bc02b950 tx=0x7f19bc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:29.132 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:29.122+0000 7f19cb7fe640 1 -- 172.21.15.92:0/2366151119 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19bc03acb0 con 0x7f19d40ae5a0 2023-11-24T06:11:29.132 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:29.122+0000 7f19cb7fe640 1 -- 172.21.15.92:0/2366151119 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19bc03ae30 con 0x7f19d40ae5a0 2023-11-24T06:11:29.132 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:29.122+0000 7f19d90bb640 1 -- 172.21.15.92:0/2366151119 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19d415c890 con 0x7f19d40ae5a0 2023-11-24T06:11:29.132 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:29.122+0000 7f19cb7fe640 1 -- 172.21.15.92:0/2366151119 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19bc044630 con 0x7f19d40ae5a0 2023-11-24T06:11:29.133 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:29.122+0000 7f19d90bb640 1 -- 172.21.15.92:0/2366151119 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19d415cb70 con 0x7f19d40ae5a0 2023-11-24T06:11:29.133 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:29.123+0000 7f19d90bb640 1 -- 172.21.15.92:0/2366151119 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19d41489b0 con 0x7f19d40ae5a0 2023-11-24T06:11:29.133 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:29.124+0000 7f19cb7fe640 1 -- 172.21.15.92:0/2366151119 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f19bc04d450 con 0x7f19d40ae5a0 2023-11-24T06:11:29.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:29.125+0000 7f19cb7fe640 1 --2- 172.21.15.92:0/2366151119 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f19b406ae40 0x7f19b406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:29.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:29.125+0000 7f19cb7fe640 1 -- 172.21.15.92:0/2366151119 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5883+0+0 (secure 0 0 0) 0x7f19bc02d040 con 0x7f19d40ae5a0 2023-11-24T06:11:29.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:29.125+0000 7f19d2d9d640 1 --2- 172.21.15.92:0/2366151119 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f19b406ae40 0x7f19b406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:29.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:29.126+0000 7f19d2d9d640 1 --2- 172.21.15.92:0/2366151119 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f19b406ae40 0x7f19b406d300 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f19c4006fd0 tx=0x7f19c4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:29.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:29.130+0000 7f19cb7fe640 1 -- 172.21.15.92:0/2366151119 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f19bc076d10 con 0x7f19d40ae5a0 2023-11-24T06:11:29.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:29.320+0000 7f19d90bb640 1 -- 172.21.15.92:0/2366151119 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f19d40675d0 con 0x7f19d40ae5a0 2023-11-24T06:11:29.323 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:29.321+0000 7f19cb7fe640 1 -- 172.21.15.92:0/2366151119 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7f19bc07a330 con 0x7f19d40ae5a0 2023-11-24T06:11:29.323 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 32 2023-11-24T06:11:29.326 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:29.323+0000 7f19d90bb640 1 -- 172.21.15.92:0/2366151119 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f19b406ae40 msgr2=0x7f19b406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:29.326 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:29.323+0000 7f19d90bb640 1 --2- 172.21.15.92:0/2366151119 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f19b406ae40 0x7f19b406d300 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f19c4006fd0 tx=0x7f19c4008040 comp rx=0 tx=0).stop 2023-11-24T06:11:29.326 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:29.323+0000 7f19d90bb640 1 -- 172.21.15.92:0/2366151119 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f19d40ae5a0 msgr2=0x7f19d40ce110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:29.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:29.323+0000 7f19d90bb640 1 --2- 172.21.15.92:0/2366151119 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f19d40ae5a0 0x7f19d40ce110 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f19bc02b950 tx=0x7f19bc0046c0 comp rx=0 tx=0).stop 2023-11-24T06:11:29.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:29.324+0000 7f19d90bb640 1 -- 172.21.15.92:0/2366151119 shutdown_connections 2023-11-24T06:11:29.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:29.324+0000 7f19d90bb640 1 --2- 172.21.15.92:0/2366151119 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f19b406ae40 0x7f19b406d300 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:29.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:29.324+0000 7f19d90bb640 1 --2- 172.21.15.92:0/2366151119 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f19d40b3ae0 0x7f19d41f2310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:29.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:29.324+0000 7f19d90bb640 1 --2- 172.21.15.92:0/2366151119 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f19d40ae5a0 0x7f19d40ce110 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:29.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:29.324+0000 7f19d90bb640 1 --2- 172.21.15.92:0/2366151119 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f19d406d120 0x7f19d415bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:29.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:29.324+0000 7f19d90bb640 1 -- 172.21.15.92:0/2366151119 >> 172.21.15.92:0/2366151119 conn(0x7f19d406a800 msgr2=0x7f19d406abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:29.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:29.324+0000 7f19d90bb640 1 -- 172.21.15.92:0/2366151119 shutdown_connections 2023-11-24T06:11:29.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:29.324+0000 7f19d90bb640 1 -- 172.21.15.92:0/2366151119 wait complete. 2023-11-24T06:11:29.345 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T06:11:29.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:29.561+0000 7f44ace56640 1 Processor -- start 2023-11-24T06:11:29.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:29.561+0000 7f44ace56640 1 -- start start 2023-11-24T06:11:29.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:29.563+0000 7f44ace56640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f44a80cf9a0 0x7f44a815dca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:29.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:29.563+0000 7f44ace56640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44a80b1ee0 con 0x7f44a815e1e0 2023-11-24T06:11:29.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:29.563+0000 7f44ace56640 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44a80b2780 con 0x7f44a8143ae0 2023-11-24T06:11:29.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:29.563+0000 7f44ace56640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44a815e990 con 0x7f44a80cf9a0 2023-11-24T06:11:29.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:29.563+0000 7f44a6d9d640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f44a815e1e0 0x7f44a80702c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:35344/0 (socket says 172.21.15.92:35344) 2023-11-24T06:11:29.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:29.563+0000 7f44a6d9d640 1 -- 172.21.15.92:0/1521444096 learned_addr learned my addr 172.21.15.92:0/1521444096 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-24T06:11:29.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:29.563+0000 7f44a5d9b640 1 --2- 172.21.15.92:0/1521444096 >> v2:172.21.15.92:3300/0 conn(0x7f44a80cf9a0 0x7f44a815dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:29.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:29.563+0000 7f44a559a640 1 -- 172.21.15.92:0/1521444096 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1174241795 0 0) 0x7f44a80b1ee0 con 0x7f44a815e1e0 2023-11-24T06:11:29.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:29.564+0000 7f44a559a640 1 -- 172.21.15.92:0/1521444096 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4490003520 con 0x7f44a815e1e0 2023-11-24T06:11:29.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:29.564+0000 7f44a5d9b640 1 -- 172.21.15.92:0/1521444096 >> v1:172.21.15.118:6789/0 conn(0x7f44a8143ae0 legacy=0x7f44a80cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:29.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:29.564+0000 7f44a5d9b640 1 -- 172.21.15.92:0/1521444096 >> v1:172.21.15.92:6789/0 conn(0x7f44a815e1e0 legacy=0x7f44a80702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:29.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:29.564+0000 7f44a5d9b640 1 -- 172.21.15.92:0/1521444096 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44a80b2130 con 0x7f44a80cf9a0 2023-11-24T06:11:29.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:29.565+0000 7f44a5d9b640 1 --2- 172.21.15.92:0/1521444096 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f44a80cf9a0 0x7f44a815dca0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f449c0097a0 tx=0x7f449c02b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6797a94b0d7a032f server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:29.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:29.565+0000 7f44a559a640 1 -- 172.21.15.92:0/1521444096 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f449c030610 con 0x7f44a80cf9a0 2023-11-24T06:11:29.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:29.565+0000 7f44a559a640 1 -- 172.21.15.92:0/1521444096 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f449c030770 con 0x7f44a80cf9a0 2023-11-24T06:11:29.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:29.565+0000 7f44a559a640 1 -- 172.21.15.92:0/1521444096 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f449c030b00 con 0x7f44a80cf9a0 2023-11-24T06:11:29.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:29.566+0000 7f44ace56640 1 -- 172.21.15.92:0/1521444096 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f44a80cf9a0 msgr2=0x7f44a815dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:29.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:29.566+0000 7f44ace56640 1 --2- 172.21.15.92:0/1521444096 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f44a80cf9a0 0x7f44a815dca0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f449c0097a0 tx=0x7f449c02b960 comp rx=0 tx=0).stop 2023-11-24T06:11:29.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:29.566+0000 7f44ace56640 1 -- 172.21.15.92:0/1521444096 shutdown_connections 2023-11-24T06:11:29.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:29.566+0000 7f44ace56640 1 --2- 172.21.15.92:0/1521444096 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f44a80cf9a0 0x7f44a815dca0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:29.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:29.566+0000 7f44ace56640 1 -- 172.21.15.92:0/1521444096 >> 172.21.15.92:0/1521444096 conn(0x7f44a806a800 msgr2=0x7f44a806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:29.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:29.566+0000 7f44ace56640 1 -- 172.21.15.92:0/1521444096 shutdown_connections 2023-11-24T06:11:29.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:29.566+0000 7f44ace56640 1 -- 172.21.15.92:0/1521444096 wait complete. 2023-11-24T06:11:29.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:29.568+0000 7f44ace56640 1 Processor -- start 2023-11-24T06:11:29.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:29.568+0000 7f44ace56640 1 -- start start 2023-11-24T06:11:29.572 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:29.569+0000 7f44ace56640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f44a80cf9a0 0x7f44a81ee190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:29.572 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:29.569+0000 7f44ace56640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f44a8143ae0 0x7f44a80ad1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:29.572 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:29.569+0000 7f44ace56640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f44a815e1e0 0x7f44a80ad730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:29.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:29.569+0000 7f44ace56640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44a80add70 con 0x7f44a815e1e0 2023-11-24T06:11:29.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:29.569+0000 7f44ace56640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44a80ae050 con 0x7f44a80cf9a0 2023-11-24T06:11:29.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:29.569+0000 7f44ace56640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44a80ae330 con 0x7f44a8143ae0 2023-11-24T06:11:29.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:29.569+0000 7f44a6d9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f44a815e1e0 0x7f44a80ad730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:29.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:29.569+0000 7f44a6d9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f44a815e1e0 0x7f44a80ad730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:57782/0 (socket says 172.21.15.92:57782) 2023-11-24T06:11:29.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:29.569+0000 7f44a6d9d640 1 -- 172.21.15.92:0/2501204787 learned_addr learned my addr 172.21.15.92:0/2501204787 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:11:29.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:29.570+0000 7f44a659c640 1 --2- 172.21.15.92:0/2501204787 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f44a80cf9a0 0x7f44a81ee190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:29.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:29.570+0000 7f44a5d9b640 1 --2- 172.21.15.92:0/2501204787 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f44a8143ae0 0x7f44a80ad1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:29.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:29.570+0000 7f44a6d9d640 1 -- 172.21.15.92:0/2501204787 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f44a8143ae0 msgr2=0x7f44a80ad1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:29.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:29.570+0000 7f44a6d9d640 1 --2- 172.21.15.92:0/2501204787 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f44a8143ae0 0x7f44a80ad1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:29.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:29.570+0000 7f44a6d9d640 1 -- 172.21.15.92:0/2501204787 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f44a80cf9a0 msgr2=0x7f44a81ee190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:29.574 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:29.570+0000 7f44a6d9d640 1 --2- 172.21.15.92:0/2501204787 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f44a80cf9a0 0x7f44a81ee190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:29.574 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:29.570+0000 7f44a6d9d640 1 -- 172.21.15.92:0/2501204787 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f449c009400 con 0x7f44a815e1e0 2023-11-24T06:11:29.574 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:29.570+0000 7f44a659c640 1 --2- 172.21.15.92:0/2501204787 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f44a80cf9a0 0x7f44a81ee190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:11:29.574 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:29.570+0000 7f44a6d9d640 1 --2- 172.21.15.92:0/2501204787 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f44a815e1e0 0x7f44a80ad730 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f44a000caa0 tx=0x7f44a000cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:29.574 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:29.570+0000 7f4496ffd640 1 -- 172.21.15.92:0/2501204787 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44a000ac00 con 0x7f44a815e1e0 2023-11-24T06:11:29.574 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:29.570+0000 7f4496ffd640 1 -- 172.21.15.92:0/2501204787 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44a000ad60 con 0x7f44a815e1e0 2023-11-24T06:11:29.574 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:29.570+0000 7f4496ffd640 1 -- 172.21.15.92:0/2501204787 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44a0012620 con 0x7f44a815e1e0 2023-11-24T06:11:29.574 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:29.570+0000 7f44ace56640 1 -- 172.21.15.92:0/2501204787 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44a812ef90 con 0x7f44a815e1e0 2023-11-24T06:11:29.574 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:29.570+0000 7f44ace56640 1 -- 172.21.15.92:0/2501204787 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44a806cdf0 con 0x7f44a815e1e0 2023-11-24T06:11:29.575 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:29.571+0000 7f44ace56640 1 -- 172.21.15.92:0/2501204787 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44a812f4b0 con 0x7f44a815e1e0 2023-11-24T06:11:29.575 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:29.572+0000 7f4496ffd640 1 -- 172.21.15.92:0/2501204787 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f44a0007070 con 0x7f44a815e1e0 2023-11-24T06:11:29.575 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:29.573+0000 7f4496ffd640 1 --2- 172.21.15.92:0/2501204787 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f448806ae40 0x7f448806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:29.576 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:29.573+0000 7f4496ffd640 1 -- 172.21.15.92:0/2501204787 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5883+0+0 (secure 0 0 0) 0x7f44a00896b0 con 0x7f44a815e1e0 2023-11-24T06:11:29.576 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:29.573+0000 7f44a659c640 1 --2- 172.21.15.92:0/2501204787 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f448806ae40 0x7f448806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:29.576 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:29.574+0000 7f44a659c640 1 --2- 172.21.15.92:0/2501204787 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f448806ae40 0x7f448806d300 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f4498000f60 tx=0x7f4498009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:29.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:29.578+0000 7f4496ffd640 1 -- 172.21.15.92:0/2501204787 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f44a812f4b0 con 0x7f44a815e1e0 2023-11-24T06:11:29.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:29.772+0000 7f44ace56640 1 -- 172.21.15.92:0/2501204787 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f44a812f4b0 con 0x7f44a815e1e0 2023-11-24T06:11:29.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:29.772+0000 7f4496ffd640 1 -- 172.21.15.92:0/2501204787 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7f44a812f4b0 con 0x7f44a815e1e0 2023-11-24T06:11:29.775 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 64 2023-11-24T06:11:29.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:29.775+0000 7f44ace56640 1 -- 172.21.15.92:0/2501204787 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f448806ae40 msgr2=0x7f448806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:29.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:29.775+0000 7f44ace56640 1 --2- 172.21.15.92:0/2501204787 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f448806ae40 0x7f448806d300 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f4498000f60 tx=0x7f4498009040 comp rx=0 tx=0).stop 2023-11-24T06:11:29.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:29.775+0000 7f44ace56640 1 -- 172.21.15.92:0/2501204787 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f44a815e1e0 msgr2=0x7f44a80ad730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:29.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:29.775+0000 7f44ace56640 1 --2- 172.21.15.92:0/2501204787 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f44a815e1e0 0x7f44a80ad730 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f44a000caa0 tx=0x7f44a000cf60 comp rx=0 tx=0).stop 2023-11-24T06:11:29.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:29.775+0000 7f44ace56640 1 -- 172.21.15.92:0/2501204787 shutdown_connections 2023-11-24T06:11:29.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:29.775+0000 7f44ace56640 1 --2- 172.21.15.92:0/2501204787 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f448806ae40 0x7f448806d300 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:29.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:29.775+0000 7f44ace56640 1 --2- 172.21.15.92:0/2501204787 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f44a815e1e0 0x7f44a80ad730 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:29.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:29.775+0000 7f44ace56640 1 --2- 172.21.15.92:0/2501204787 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f44a8143ae0 0x7f44a80ad1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:29.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:29.775+0000 7f44ace56640 1 --2- 172.21.15.92:0/2501204787 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f44a80cf9a0 0x7f44a81ee190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:29.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:29.775+0000 7f44ace56640 1 -- 172.21.15.92:0/2501204787 >> 172.21.15.92:0/2501204787 conn(0x7f44a806a800 msgr2=0x7f44a80ce6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:29.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:29.775+0000 7f44ace56640 1 -- 172.21.15.92:0/2501204787 shutdown_connections 2023-11-24T06:11:29.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:29.775+0000 7f44ace56640 1 -- 172.21.15.92:0/2501204787 wait complete. 2023-11-24T06:11:29.796 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T06:11:30.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:30.001+0000 7fe895471640 1 Processor -- start 2023-11-24T06:11:30.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:30.001+0000 7fe895471640 1 -- start start 2023-11-24T06:11:30.006 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:30.003+0000 7fe895471640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fe890143c40 0x7fe890071290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:30.006 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:30.003+0000 7fe895471640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe8900717d0 0x7fe8900cc500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:30.006 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:30.003+0000 7fe895471640 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe890070e50 con 0x7fe8900cca40 2023-11-24T06:11:30.006 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:30.003+0000 7fe895471640 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8900723c0 con 0x7fe890143c40 2023-11-24T06:11:30.006 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:30.003+0000 7fe895471640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8900cce70 con 0x7fe8900717d0 2023-11-24T06:11:30.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:30.003+0000 7fe887fff640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe8900717d0 0x7fe8900cc500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:30.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:30.004+0000 7fe887fff640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe8900717d0 0x7fe8900cc500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3301/0 says I am v2:172.21.15.92:54708/0 (socket says 172.21.15.92:54708) 2023-11-24T06:11:30.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:30.004+0000 7fe88ffff640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fe890143c40 0x7fe890071290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:30.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:30.004+0000 7fe887fff640 1 -- 172.21.15.92:0/274076016 learned_addr learned my addr 172.21.15.92:0/274076016 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:11:30.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:30.004+0000 7fe894c70640 1 --1- 172.21.15.92:0/274076016 >> v1:172.21.15.118:6789/0 conn(0x7fe8900cca40 0x7fe89015eab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.118:6789/0 says I am v1:172.21.15.92:37042/0 (socket says 172.21.15.92:37042) 2023-11-24T06:11:30.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:30.004+0000 7fe88ffff640 1 -- 172.21.15.92:0/274076016 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe8900717d0 msgr2=0x7fe8900cc500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:30.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:30.004+0000 7fe88ffff640 1 --2- 172.21.15.92:0/274076016 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe8900717d0 0x7fe8900cc500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:30.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:30.004+0000 7fe88ffff640 1 -- 172.21.15.92:0/274076016 >> v1:172.21.15.118:6789/0 conn(0x7fe8900cca40 legacy=0x7fe89015eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:30.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:30.004+0000 7fe88ffff640 1 -- 172.21.15.92:0/274076016 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe89006e170 con 0x7fe890143c40 2023-11-24T06:11:30.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:30.006+0000 7fe88ffff640 1 --2- 172.21.15.92:0/274076016 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe890143c40 0x7fe890071290 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fe8800097a0 tx=0x7fe88002b820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6024b22b91c96df0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:30.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:30.006+0000 7fe88f7fe640 1 -- 172.21.15.92:0/274076016 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe880031780 con 0x7fe890143c40 2023-11-24T06:11:30.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:30.006+0000 7fe88f7fe640 1 -- 172.21.15.92:0/274076016 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8800318e0 con 0x7fe890143c40 2023-11-24T06:11:30.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:30.006+0000 7fe88f7fe640 1 -- 172.21.15.92:0/274076016 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe880031c70 con 0x7fe890143c40 2023-11-24T06:11:30.010 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:30.007+0000 7fe895471640 1 -- 172.21.15.92:0/274076016 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe890143c40 msgr2=0x7fe890071290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:30.010 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:30.007+0000 7fe895471640 1 --2- 172.21.15.92:0/274076016 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe890143c40 0x7fe890071290 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fe8800097a0 tx=0x7fe88002b820 comp rx=0 tx=0).stop 2023-11-24T06:11:30.010 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:30.007+0000 7fe895471640 1 -- 172.21.15.92:0/274076016 shutdown_connections 2023-11-24T06:11:30.010 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:30.007+0000 7fe895471640 1 --2- 172.21.15.92:0/274076016 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe8900717d0 0x7fe8900cc500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:30.010 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:30.007+0000 7fe895471640 1 --2- 172.21.15.92:0/274076016 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe890143c40 0x7fe890071290 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:30.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:30.007+0000 7fe895471640 1 -- 172.21.15.92:0/274076016 >> 172.21.15.92:0/274076016 conn(0x7fe89006a800 msgr2=0x7fe89006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:30.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:30.007+0000 7fe895471640 1 -- 172.21.15.92:0/274076016 shutdown_connections 2023-11-24T06:11:30.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:30.007+0000 7fe895471640 1 -- 172.21.15.92:0/274076016 wait complete. 2023-11-24T06:11:30.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:30.009+0000 7fe895471640 1 Processor -- start 2023-11-24T06:11:30.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:30.009+0000 7fe895471640 1 -- start start 2023-11-24T06:11:30.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:30.010+0000 7fe895471640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe8900717d0 0x7fe890158a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:30.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:30.010+0000 7fe895471640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe8900cca40 0x7fe8900ab990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:30.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:30.010+0000 7fe895471640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe890143c40 0x7fe89013bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:30.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:30.010+0000 7fe895471640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8900abed0 con 0x7fe890143c40 2023-11-24T06:11:30.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:30.010+0000 7fe895471640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe89013a470 con 0x7fe8900cca40 2023-11-24T06:11:30.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:30.010+0000 7fe895471640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe89013a750 con 0x7fe8900717d0 2023-11-24T06:11:30.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:30.010+0000 7fe894c70640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe890143c40 0x7fe89013bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:30.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:30.010+0000 7fe894c70640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe890143c40 0x7fe89013bbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:57794/0 (socket says 172.21.15.92:57794) 2023-11-24T06:11:30.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:30.010+0000 7fe894c70640 1 -- 172.21.15.92:0/1808136462 learned_addr learned my addr 172.21.15.92:0/1808136462 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:11:30.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:30.010+0000 7fe88ffff640 1 --2- 172.21.15.92:0/1808136462 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe8900717d0 0x7fe890158a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:30.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:30.010+0000 7fe887fff640 1 --2- 172.21.15.92:0/1808136462 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe8900cca40 0x7fe8900ab990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:30.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:30.011+0000 7fe894c70640 1 -- 172.21.15.92:0/1808136462 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe8900717d0 msgr2=0x7fe890158a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:30.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:30.011+0000 7fe894c70640 1 --2- 172.21.15.92:0/1808136462 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe8900717d0 0x7fe890158a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:30.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:30.011+0000 7fe894c70640 1 -- 172.21.15.92:0/1808136462 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe8900cca40 msgr2=0x7fe8900ab990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:30.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:30.011+0000 7fe894c70640 1 --2- 172.21.15.92:0/1808136462 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe8900cca40 0x7fe8900ab990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:30.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:30.011+0000 7fe894c70640 1 -- 172.21.15.92:0/1808136462 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe880009400 con 0x7fe890143c40 2023-11-24T06:11:30.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:30.011+0000 7fe887fff640 1 --2- 172.21.15.92:0/1808136462 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe8900cca40 0x7fe8900ab990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:11:30.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:30.011+0000 7fe88ffff640 1 --2- 172.21.15.92:0/1808136462 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe8900717d0 0x7fe890158a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:11:30.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:30.011+0000 7fe894c70640 1 --2- 172.21.15.92:0/1808136462 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe890143c40 0x7fe89013bbc0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fe88800c780 tx=0x7fe88800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:30.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:30.011+0000 7fe88d7fa640 1 -- 172.21.15.92:0/1808136462 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe88800ce90 con 0x7fe890143c40 2023-11-24T06:11:30.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:30.011+0000 7fe88d7fa640 1 -- 172.21.15.92:0/1808136462 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe88800a640 con 0x7fe890143c40 2023-11-24T06:11:30.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:30.011+0000 7fe895471640 1 -- 172.21.15.92:0/1808136462 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe89013aa30 con 0x7fe890143c40 2023-11-24T06:11:30.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:30.011+0000 7fe88d7fa640 1 -- 172.21.15.92:0/1808136462 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe88800a9f0 con 0x7fe890143c40 2023-11-24T06:11:30.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:30.011+0000 7fe895471640 1 -- 172.21.15.92:0/1808136462 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe89013c100 con 0x7fe890143c40 2023-11-24T06:11:30.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:30.013+0000 7fe895471640 1 -- 172.21.15.92:0/1808136462 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe890067450 con 0x7fe890143c40 2023-11-24T06:11:30.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:30.013+0000 7fe88d7fa640 1 -- 172.21.15.92:0/1808136462 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe88800ab50 con 0x7fe890143c40 2023-11-24T06:11:30.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:30.014+0000 7fe88d7fa640 1 --2- 172.21.15.92:0/1808136462 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fe87006ae70 0x7fe87006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:30.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:30.014+0000 7fe88d7fa640 1 -- 172.21.15.92:0/1808136462 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5883+0+0 (secure 0 0 0) 0x7fe888088df0 con 0x7fe890143c40 2023-11-24T06:11:30.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:30.015+0000 7fe88ffff640 1 --2- 172.21.15.92:0/1808136462 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fe87006ae70 0x7fe87006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:30.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:30.015+0000 7fe88ffff640 1 --2- 172.21.15.92:0/1808136462 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fe87006ae70 0x7fe87006d330 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fe880036000 tx=0x7fe88003d040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:30.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:30.019+0000 7fe88d7fa640 1 -- 172.21.15.92:0/1808136462 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe888053ea0 con 0x7fe890143c40 2023-11-24T06:11:30.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:30.228+0000 7fe895471640 1 -- 172.21.15.92:0/1808136462 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe8900ceb30 con 0x7fe890143c40 2023-11-24T06:11:30.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:30.229+0000 7fe88d7fa640 1 -- 172.21.15.92:0/1808136462 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4910 (secure 0 0 0) 0x7fe888017020 con 0x7fe890143c40 2023-11-24T06:11:30.232 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-24T06:11:30.232 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4236,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6836/437604699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":437604699},{"type":"v1","addr":"172.21.15.92:6836","nonce":437604699}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6835/3368711442","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":3368711442},{"type":"v1","addr":"172.21.15.118:6835","nonce":3368711442}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4255,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/2767290380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":2767290380},{"type":"v1","addr":"172.21.15.92:6837","nonce":2767290380}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T06:10:25.115612+0000","modified":"2023-11-24T06:10:28.210438+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4264},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4264":{"gid":4264,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.118:6833/1146052262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":1146052262},{"type":"v1","addr":"172.21.15.118:6833","nonce":1146052262}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-24T06:11:30.233 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 8 2023-11-24T06:11:30.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:30.231+0000 7fe895471640 1 -- 172.21.15.92:0/1808136462 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fe87006ae70 msgr2=0x7fe87006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:30.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:30.231+0000 7fe895471640 1 --2- 172.21.15.92:0/1808136462 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fe87006ae70 0x7fe87006d330 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fe880036000 tx=0x7fe88003d040 comp rx=0 tx=0).stop 2023-11-24T06:11:30.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:30.231+0000 7fe895471640 1 -- 172.21.15.92:0/1808136462 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe890143c40 msgr2=0x7fe89013bbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:30.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:30.231+0000 7fe895471640 1 --2- 172.21.15.92:0/1808136462 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe890143c40 0x7fe89013bbc0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fe88800c780 tx=0x7fe88800cc40 comp rx=0 tx=0).stop 2023-11-24T06:11:30.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:30.231+0000 7fe895471640 1 -- 172.21.15.92:0/1808136462 shutdown_connections 2023-11-24T06:11:30.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:30.231+0000 7fe895471640 1 --2- 172.21.15.92:0/1808136462 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fe87006ae70 0x7fe87006d330 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:30.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:30.231+0000 7fe895471640 1 --2- 172.21.15.92:0/1808136462 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe890143c40 0x7fe89013bbc0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:30.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:30.231+0000 7fe895471640 1 --2- 172.21.15.92:0/1808136462 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe8900cca40 0x7fe8900ab990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:30.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:30.231+0000 7fe895471640 1 --2- 172.21.15.92:0/1808136462 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe8900717d0 0x7fe890158a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:30.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:30.231+0000 7fe895471640 1 -- 172.21.15.92:0/1808136462 >> 172.21.15.92:0/1808136462 conn(0x7fe89006a800 msgr2=0x7fe89015a8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:30.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:30.231+0000 7fe895471640 1 -- 172.21.15.92:0/1808136462 shutdown_connections 2023-11-24T06:11:30.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:30.231+0000 7fe895471640 1 -- 172.21.15.92:0/1808136462 wait complete. 2023-11-24T06:11:30.252 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:11:30.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:30.454+0000 7ff8247b0640 1 Processor -- start 2023-11-24T06:11:30.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:30.454+0000 7ff8247b0640 1 -- start start 2023-11-24T06:11:30.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:30.456+0000 7ff8247b0640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff81c13df80 0x7ff81c15bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:30.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:30.456+0000 7ff8247b0640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7ff81c13e360 0x7ff81c0cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:30.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:30.456+0000 7ff8247b0640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ff81c15f9c0 0x7ff81c0cf7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:30.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:30.456+0000 7ff8247b0640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff81c06c470 con 0x7ff81c15f9c0 2023-11-24T06:11:30.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:30.456+0000 7ff8247b0640 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff81c06c750 con 0x7ff81c13e360 2023-11-24T06:11:30.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:30.456+0000 7ff8247b0640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff81c06ca30 con 0x7ff81c13df80 2023-11-24T06:11:30.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:30.456+0000 7ff822d4d640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ff81c15f9c0 0x7ff81c0cf7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:30.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:30.456+0000 7ff822d4d640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ff81c15f9c0 0x7ff81c0cf7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:57796/0 (socket says 172.21.15.92:57796) 2023-11-24T06:11:30.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:30.456+0000 7ff822d4d640 1 -- 172.21.15.92:0/1820891404 learned_addr learned my addr 172.21.15.92:0/1820891404 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:11:30.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:30.456+0000 7ff82254c640 1 --2- 172.21.15.92:0/1820891404 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff81c13df80 0x7ff81c15bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:30.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:30.456+0000 7ff821d4b640 1 --2- 172.21.15.92:0/1820891404 >> v2:172.21.15.118:3300/0 conn(0x7ff81c13e360 0x7ff81c0cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:30.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:30.457+0000 7ff822d4d640 1 -- 172.21.15.92:0/1820891404 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff81c13df80 msgr2=0x7ff81c15bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:30.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:30.457+0000 7ff822d4d640 1 --2- 172.21.15.92:0/1820891404 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff81c13df80 0x7ff81c15bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:30.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:30.457+0000 7ff822d4d640 1 -- 172.21.15.92:0/1820891404 >> v2:172.21.15.118:3300/0 conn(0x7ff81c13e360 msgr2=0x7ff81c0cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:30.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:30.457+0000 7ff822d4d640 1 --2- 172.21.15.92:0/1820891404 >> v2:172.21.15.118:3300/0 conn(0x7ff81c13e360 0x7ff81c0cf290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:30.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:30.457+0000 7ff822d4d640 1 -- 172.21.15.92:0/1820891404 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff81c06cd10 con 0x7ff81c15f9c0 2023-11-24T06:11:30.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:30.458+0000 7ff822d4d640 1 --2- 172.21.15.92:0/1820891404 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff81c15f9c0 0x7ff81c0cf7d0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7ff81c0cfed0 tx=0x7ff81402bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=570bef4fc4ea7d03 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:30.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:30.458+0000 7ff82154a640 1 -- 172.21.15.92:0/1820891404 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff81402ca80 con 0x7ff81c15f9c0 2023-11-24T06:11:30.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:30.458+0000 7ff82154a640 1 -- 172.21.15.92:0/1820891404 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff81402cbe0 con 0x7ff81c15f9c0 2023-11-24T06:11:30.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:30.458+0000 7ff82154a640 1 -- 172.21.15.92:0/1820891404 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8140343f0 con 0x7ff81c15f9c0 2023-11-24T06:11:30.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:30.459+0000 7ff8247b0640 1 -- 172.21.15.92:0/1820891404 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff81c15f9c0 msgr2=0x7ff81c0cf7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:30.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:30.459+0000 7ff8247b0640 1 --2- 172.21.15.92:0/1820891404 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff81c15f9c0 0x7ff81c0cf7d0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7ff81c0cfed0 tx=0x7ff81402bee0 comp rx=0 tx=0).stop 2023-11-24T06:11:30.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:30.459+0000 7ff8247b0640 1 -- 172.21.15.92:0/1820891404 shutdown_connections 2023-11-24T06:11:30.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:30.459+0000 7ff8247b0640 1 --2- 172.21.15.92:0/1820891404 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff81c15f9c0 0x7ff81c0cf7d0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:30.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:30.459+0000 7ff8247b0640 1 --2- 172.21.15.92:0/1820891404 >> v2:172.21.15.118:3300/0 conn(0x7ff81c13e360 0x7ff81c0cf290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:30.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:30.459+0000 7ff8247b0640 1 --2- 172.21.15.92:0/1820891404 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff81c13df80 0x7ff81c15bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:30.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:30.459+0000 7ff8247b0640 1 -- 172.21.15.92:0/1820891404 >> 172.21.15.92:0/1820891404 conn(0x7ff81c06a800 msgr2=0x7ff81c06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:30.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:30.459+0000 7ff8247b0640 1 -- 172.21.15.92:0/1820891404 shutdown_connections 2023-11-24T06:11:30.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:30.459+0000 7ff8247b0640 1 -- 172.21.15.92:0/1820891404 wait complete. 2023-11-24T06:11:30.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:30.461+0000 7ff8247b0640 1 Processor -- start 2023-11-24T06:11:30.466 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:30.461+0000 7ff8247b0640 1 -- start start 2023-11-24T06:11:30.466 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:30.462+0000 7ff8247b0640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff81c13df80 0x7ff81c1541b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:30.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:30.462+0000 7ff8247b0640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff81c13e360 0x7ff81c0b1ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:30.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:30.462+0000 7ff8247b0640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff81c15f9c0 0x7ff81c0b21e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:30.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:30.462+0000 7ff8247b0640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff81c148510 con 0x7ff81c15f9c0 2023-11-24T06:11:30.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:30.462+0000 7ff8247b0640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff81c0b3df0 con 0x7ff81c13e360 2023-11-24T06:11:30.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:30.462+0000 7ff8247b0640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff81c148720 con 0x7ff81c13df80 2023-11-24T06:11:30.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:30.462+0000 7ff822d4d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff81c15f9c0 0x7ff81c0b21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:30.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:30.462+0000 7ff822d4d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff81c15f9c0 0x7ff81c0b21e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:57808/0 (socket says 172.21.15.92:57808) 2023-11-24T06:11:30.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:30.462+0000 7ff822d4d640 1 -- 172.21.15.92:0/3200557611 learned_addr learned my addr 172.21.15.92:0/3200557611 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:11:30.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:30.463+0000 7ff82254c640 1 --2- 172.21.15.92:0/3200557611 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff81c13df80 0x7ff81c1541b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:30.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:30.463+0000 7ff822d4d640 1 -- 172.21.15.92:0/3200557611 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff81c13df80 msgr2=0x7ff81c1541b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:30.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:30.463+0000 7ff822d4d640 1 --2- 172.21.15.92:0/3200557611 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff81c13df80 0x7ff81c1541b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:30.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:30.463+0000 7ff822d4d640 1 -- 172.21.15.92:0/3200557611 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff81c13e360 msgr2=0x7ff81c0b1ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:30.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:30.463+0000 7ff822d4d640 1 --2- 172.21.15.92:0/3200557611 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff81c13e360 0x7ff81c0b1ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:30.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:30.463+0000 7ff822d4d640 1 -- 172.21.15.92:0/3200557611 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff814009b30 con 0x7ff81c15f9c0 2023-11-24T06:11:30.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:30.463+0000 7ff822d4d640 1 --2- 172.21.15.92:0/3200557611 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff81c15f9c0 0x7ff81c0b21e0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7ff814009f40 tx=0x7ff814004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:30.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:30.463+0000 7ff812ffd640 1 -- 172.21.15.92:0/3200557611 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff814009510 con 0x7ff81c15f9c0 2023-11-24T06:11:30.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:30.463+0000 7ff812ffd640 1 -- 172.21.15.92:0/3200557611 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff81403e450 con 0x7ff81c15f9c0 2023-11-24T06:11:30.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:30.463+0000 7ff812ffd640 1 -- 172.21.15.92:0/3200557611 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff81403e800 con 0x7ff81c15f9c0 2023-11-24T06:11:30.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:30.463+0000 7ff8247b0640 1 -- 172.21.15.92:0/3200557611 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff81c148a00 con 0x7ff81c15f9c0 2023-11-24T06:11:30.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:30.463+0000 7ff8247b0640 1 -- 172.21.15.92:0/3200557611 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff81c148fc0 con 0x7ff81c15f9c0 2023-11-24T06:11:30.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:30.464+0000 7ff8247b0640 1 -- 172.21.15.92:0/3200557611 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff81c0675d0 con 0x7ff81c15f9c0 2023-11-24T06:11:30.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:30.465+0000 7ff812ffd640 1 -- 172.21.15.92:0/3200557611 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ff81402cd50 con 0x7ff81c15f9c0 2023-11-24T06:11:30.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:30.466+0000 7ff812ffd640 1 --2- 172.21.15.92:0/3200557611 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7ff80006ae40 0x7ff80006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:30.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:30.466+0000 7ff812ffd640 1 -- 172.21.15.92:0/3200557611 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5883+0+0 (secure 0 0 0) 0x7ff8140ac5a0 con 0x7ff81c15f9c0 2023-11-24T06:11:30.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:30.466+0000 7ff82254c640 1 --2- 172.21.15.92:0/3200557611 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7ff80006ae40 0x7ff80006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:30.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:30.467+0000 7ff82254c640 1 --2- 172.21.15.92:0/3200557611 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7ff80006ae40 0x7ff80006d300 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7ff818006fd0 tx=0x7ff818008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:30.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:30.470+0000 7ff812ffd640 1 -- 172.21.15.92:0/3200557611 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff814077500 con 0x7ff81c15f9c0 2023-11-24T06:11:30.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:30.662+0000 7ff8247b0640 1 -- 172.21.15.92:0/3200557611 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff81c06df00 con 0x7ff81c15f9c0 2023-11-24T06:11:30.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:30.665+0000 7ff812ffd640 1 -- 172.21.15.92:0/3200557611 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16800 (secure 0 0 0) 0x7ff81407ab20 con 0x7ff81c15f9c0 2023-11-24T06:11:30.669 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-24T06:11:30.669 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":22,"fsid":"2d6dee5f-8aa3-470f-931a-50ade963b1cb","created":"2023-11-24T06:10:04.707204+0000","modified":"2023-11-24T06:11:11.472401+0000","last_up_change":"2023-11-24T06:10:17.598352+0000","last_in_change":"2023-11-24T06:10:09.040793+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:10:17.733098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:10:22.926445+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T06:10:24.151288+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ff37a9fb-135f-471f-b4b0-f7b5772006c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6803","nonce":20560714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6805","nonce":20560714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6809","nonce":20560714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6807","nonce":20560714}]},"public_addr":"172.21.15.92:6803/20560714","cluster_addr":"172.21.15.92:6805/20560714","heartbeat_back_addr":"172.21.15.92:6809/20560714","heartbeat_front_addr":"172.21.15.92:6807/20560714","state":["exists","up"]},{"osd":1,"uuid":"03cdd353-dcf1-4f38-94fe-a795137c06b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6819","nonce":2451239906}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6821","nonce":2451239906}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6825","nonce":2451239906}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6823","nonce":2451239906}]},"public_addr":"172.21.15.92:6819/2451239906","cluster_addr":"172.21.15.92:6821/2451239906","heartbeat_back_addr":"172.21.15.92:6825/2451239906","heartbeat_front_addr":"172.21.15.92:6823/2451239906","state":["exists","up"]},{"osd":2,"uuid":"a4a20d80-5d57-484b-becd-9aa8070dd3aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6811","nonce":1124979277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6813","nonce":1124979277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6817","nonce":1124979277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6815","nonce":1124979277}]},"public_addr":"172.21.15.92:6811/1124979277","cluster_addr":"172.21.15.92:6813/1124979277","heartbeat_back_addr":"172.21.15.92:6817/1124979277","heartbeat_front_addr":"172.21.15.92:6815/1124979277","state":["exists","up"]},{"osd":3,"uuid":"4648d7ce-9b29-4583-9425-4ba8fd2b1903","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6827","nonce":1734115782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6829","nonce":1734115782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6833","nonce":1734115782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6831","nonce":1734115782}]},"public_addr":"172.21.15.92:6827/1734115782","cluster_addr":"172.21.15.92:6829/1734115782","heartbeat_back_addr":"172.21.15.92:6833/1734115782","heartbeat_front_addr":"172.21.15.92:6831/1734115782","state":["exists","up"]},{"osd":4,"uuid":"3bc9d7eb-c6fa-4fcf-a3ec-ade604e05aac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6819","nonce":3425937229}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6822","nonce":3425937229}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6830","nonce":3425937229}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6826","nonce":3425937229}]},"public_addr":"172.21.15.118:6819/3425937229","cluster_addr":"172.21.15.118:6822/3425937229","heartbeat_back_addr":"172.21.15.118:6830/3425937229","heartbeat_front_addr":"172.21.15.118:6826/3425937229","state":["exists","up"]},{"osd":5,"uuid":"dce9ba6d-59ed-46d1-84fc-a84b13de10c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6801","nonce":827455984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6803","nonce":827455984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6807","nonce":827455984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6805","nonce":827455984}]},"public_addr":"172.21.15.118:6801/827455984","cluster_addr":"172.21.15.118:6803/827455984","heartbeat_back_addr":"172.21.15.118:6807/827455984","heartbeat_front_addr":"172.21.15.118:6805/827455984","state":["exists","up"]},{"osd":6,"uuid":"82aec504-ae58-4709-8d30-46f2b12081b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6809","nonce":2946986141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6811","nonce":2946986141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6815","nonce":2946986141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6813","nonce":2946986141}]},"public_addr":"172.21.15.118:6809/2946986141","cluster_addr":"172.21.15.118:6811/2946986141","heartbeat_back_addr":"172.21.15.118:6815/2946986141","heartbeat_front_addr":"172.21.15.118:6813/2946986141","state":["exists","up"]},{"osd":7,"uuid":"4ed4c624-c88f-4d35-bdb1-bf209b5dffb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6817","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6818","nonce":48122059}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6821","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6823","nonce":48122059}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6829","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6831","nonce":48122059}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6825","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6827","nonce":48122059}]},"public_addr":"172.21.15.118:6818/48122059","cluster_addr":"172.21.15.118:6823/48122059","heartbeat_back_addr":"172.21.15.118:6831/48122059","heartbeat_front_addr":"172.21.15.118:6827/48122059","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.536664+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.368324+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.271201+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.342971+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.621832+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.387110+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.632946+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.416032+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.d","mappings":[{"from":5,"to":1}]},{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"3.29","mappings":[{"from":3,"to":2}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.34","mappings":[{"from":4,"to":2}]},{"pgid":"3.3b","mappings":[{"from":7,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:11:30.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:30.667+0000 7ff8247b0640 1 -- 172.21.15.92:0/3200557611 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7ff80006ae40 msgr2=0x7ff80006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:30.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:30.667+0000 7ff8247b0640 1 --2- 172.21.15.92:0/3200557611 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7ff80006ae40 0x7ff80006d300 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7ff818006fd0 tx=0x7ff818008040 comp rx=0 tx=0).stop 2023-11-24T06:11:30.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:30.667+0000 7ff8247b0640 1 -- 172.21.15.92:0/3200557611 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff81c15f9c0 msgr2=0x7ff81c0b21e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:30.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:30.667+0000 7ff8247b0640 1 --2- 172.21.15.92:0/3200557611 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff81c15f9c0 0x7ff81c0b21e0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7ff814009f40 tx=0x7ff814004a40 comp rx=0 tx=0).stop 2023-11-24T06:11:30.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:30.668+0000 7ff8247b0640 1 -- 172.21.15.92:0/3200557611 shutdown_connections 2023-11-24T06:11:30.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:30.668+0000 7ff8247b0640 1 --2- 172.21.15.92:0/3200557611 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7ff80006ae40 0x7ff80006d300 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:30.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:30.668+0000 7ff8247b0640 1 --2- 172.21.15.92:0/3200557611 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff81c15f9c0 0x7ff81c0b21e0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:30.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:30.668+0000 7ff8247b0640 1 --2- 172.21.15.92:0/3200557611 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff81c13e360 0x7ff81c0b1ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:30.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:30.668+0000 7ff8247b0640 1 --2- 172.21.15.92:0/3200557611 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff81c13df80 0x7ff81c1541b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:30.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:30.668+0000 7ff8247b0640 1 -- 172.21.15.92:0/3200557611 >> 172.21.15.92:0/3200557611 conn(0x7ff81c06a800 msgr2=0x7ff81c06d230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:30.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:30.668+0000 7ff8247b0640 1 -- 172.21.15.92:0/3200557611 shutdown_connections 2023-11-24T06:11:30.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:30.668+0000 7ff8247b0640 1 -- 172.21.15.92:0/3200557611 wait complete. 2023-11-24T06:11:30.688 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-24T06:11:30.689 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T06:11:30.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:30.896+0000 7f0bf3f83640 1 Processor -- start 2023-11-24T06:11:30.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:30.896+0000 7f0bf3f83640 1 -- start start 2023-11-24T06:11:30.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:30.898+0000 7f0bf3f83640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0bec143ae0 0x7f0bec0cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:30.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:30.898+0000 7f0bf3f83640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f0bec0cf7d0 0x7f0bec15dca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:30.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:30.898+0000 7f0bf3f83640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f0bec15e1e0 0x7f0bec15a670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:30.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:30.898+0000 7f0bf3f83640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0bec0cfc30 con 0x7f0bec15e1e0 2023-11-24T06:11:30.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:30.898+0000 7f0bf3f83640 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0bec0b1ee0 con 0x7f0bec0cf7d0 2023-11-24T06:11:30.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:30.898+0000 7f0bf3f83640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bec0b2780 con 0x7f0bec143ae0 2023-11-24T06:11:30.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:30.898+0000 7f0bf2520640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f0bec15e1e0 0x7f0bec15a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:30.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:30.899+0000 7f0bf2520640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f0bec15e1e0 0x7f0bec15a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:57810/0 (socket says 172.21.15.92:57810) 2023-11-24T06:11:30.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:30.899+0000 7f0bf2520640 1 -- 172.21.15.92:0/1914785092 learned_addr learned my addr 172.21.15.92:0/1914785092 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:11:30.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:30.899+0000 7f0bf2520640 1 -- 172.21.15.92:0/1914785092 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0bec143ae0 msgr2=0x7f0bec0cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:30.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:30.899+0000 7f0bf2520640 1 --2- 172.21.15.92:0/1914785092 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0bec143ae0 0x7f0bec0cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:30.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:30.899+0000 7f0bf2520640 1 -- 172.21.15.92:0/1914785092 >> v2:172.21.15.118:3300/0 conn(0x7f0bec0cf7d0 msgr2=0x7f0bec15dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:30.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:30.899+0000 7f0bf2520640 1 --2- 172.21.15.92:0/1914785092 >> v2:172.21.15.118:3300/0 conn(0x7f0bec0cf7d0 0x7f0bec15dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:30.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:30.899+0000 7f0bf2520640 1 -- 172.21.15.92:0/1914785092 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bec15e990 con 0x7f0bec15e1e0 2023-11-24T06:11:30.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:30.900+0000 7f0bf2520640 1 --2- 172.21.15.92:0/1914785092 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0bec15e1e0 0x7f0bec15a670 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f0bec06e350 tx=0x7f0be402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79923dd079e954b7 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:30.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:30.900+0000 7f0bf0d1d640 1 -- 172.21.15.92:0/1914785092 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0be402ca70 con 0x7f0bec15e1e0 2023-11-24T06:11:30.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:30.900+0000 7f0bf0d1d640 1 -- 172.21.15.92:0/1914785092 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0be402cbd0 con 0x7f0bec15e1e0 2023-11-24T06:11:30.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:30.900+0000 7f0bf0d1d640 1 -- 172.21.15.92:0/1914785092 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0be40343f0 con 0x7f0bec15e1e0 2023-11-24T06:11:30.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:30.901+0000 7f0bf3f83640 1 -- 172.21.15.92:0/1914785092 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0bec15e1e0 msgr2=0x7f0bec15a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:30.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:30.901+0000 7f0bf3f83640 1 --2- 172.21.15.92:0/1914785092 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0bec15e1e0 0x7f0bec15a670 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f0bec06e350 tx=0x7f0be402bed0 comp rx=0 tx=0).stop 2023-11-24T06:11:30.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:30.901+0000 7f0bf3f83640 1 -- 172.21.15.92:0/1914785092 shutdown_connections 2023-11-24T06:11:30.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:30.901+0000 7f0bf3f83640 1 --2- 172.21.15.92:0/1914785092 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0bec15e1e0 0x7f0bec15a670 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:30.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:30.901+0000 7f0bf3f83640 1 --2- 172.21.15.92:0/1914785092 >> v2:172.21.15.118:3300/0 conn(0x7f0bec0cf7d0 0x7f0bec15dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:30.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:30.901+0000 7f0bf3f83640 1 --2- 172.21.15.92:0/1914785092 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0bec143ae0 0x7f0bec0cf290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:30.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:30.901+0000 7f0bf3f83640 1 -- 172.21.15.92:0/1914785092 >> 172.21.15.92:0/1914785092 conn(0x7f0bec06a800 msgr2=0x7f0bec06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:30.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:30.901+0000 7f0bf3f83640 1 -- 172.21.15.92:0/1914785092 shutdown_connections 2023-11-24T06:11:30.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:30.901+0000 7f0bf3f83640 1 -- 172.21.15.92:0/1914785092 wait complete. 2023-11-24T06:11:30.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:30.903+0000 7f0bf3f83640 1 Processor -- start 2023-11-24T06:11:30.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:30.904+0000 7f0bf3f83640 1 -- start start 2023-11-24T06:11:30.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:30.905+0000 7f0bf3f83640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0bec0cf7d0 0x7f0bec0aea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:30.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:30.905+0000 7f0bf3f83640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0bec143ae0 0x7f0bec0aef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:30.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:30.905+0000 7f0bf3f83640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0bec15e1e0 0x7f0bec0cd120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:30.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:30.905+0000 7f0bf3f83640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bec1f3e10 con 0x7f0bec0cf7d0 2023-11-24T06:11:30.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:30.905+0000 7f0bf3f83640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bec0cdea0 con 0x7f0bec143ae0 2023-11-24T06:11:30.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:30.905+0000 7f0bf3f83640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bec0aca00 con 0x7f0bec15e1e0 2023-11-24T06:11:30.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:30.905+0000 7f0bf1d1f640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0bec0cf7d0 0x7f0bec0aea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:30.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:30.905+0000 7f0bf1d1f640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0bec0cf7d0 0x7f0bec0aea20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:57812/0 (socket says 172.21.15.92:57812) 2023-11-24T06:11:30.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:30.905+0000 7f0bf1d1f640 1 -- 172.21.15.92:0/2283307257 learned_addr learned my addr 172.21.15.92:0/2283307257 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:11:30.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:30.905+0000 7f0bf1d1f640 1 -- 172.21.15.92:0/2283307257 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0bec15e1e0 msgr2=0x7f0bec0cd120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:30.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:30.905+0000 7f0bf1d1f640 1 --2- 172.21.15.92:0/2283307257 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0bec15e1e0 0x7f0bec0cd120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:30.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:30.905+0000 7f0bf1d1f640 1 -- 172.21.15.92:0/2283307257 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0bec143ae0 msgr2=0x7f0bec0aef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:30.910 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:30.905+0000 7f0bf1d1f640 1 --2- 172.21.15.92:0/2283307257 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0bec143ae0 0x7f0bec0aef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:30.910 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:30.905+0000 7f0bf1d1f640 1 -- 172.21.15.92:0/2283307257 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0be4009b30 con 0x7f0bec0cf7d0 2023-11-24T06:11:30.910 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:30.906+0000 7f0bf1d1f640 1 --2- 172.21.15.92:0/2283307257 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0bec0cf7d0 0x7f0bec0aea20 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f0be800b4a0 tx=0x7f0be800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:30.910 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:30.906+0000 7f0be27fc640 1 -- 172.21.15.92:0/2283307257 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0be8010610 con 0x7f0bec0cf7d0 2023-11-24T06:11:30.910 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:30.906+0000 7f0be27fc640 1 -- 172.21.15.92:0/2283307257 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0be8010770 con 0x7f0bec0cf7d0 2023-11-24T06:11:30.911 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:30.906+0000 7f0bf3f83640 1 -- 172.21.15.92:0/2283307257 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0bec0acce0 con 0x7f0bec0cf7d0 2023-11-24T06:11:30.911 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:30.906+0000 7f0be27fc640 1 -- 172.21.15.92:0/2283307257 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0be8010b20 con 0x7f0bec0cf7d0 2023-11-24T06:11:30.911 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:30.906+0000 7f0bf3f83640 1 -- 172.21.15.92:0/2283307257 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bec0ce180 con 0x7f0bec0cf7d0 2023-11-24T06:11:30.911 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:30.907+0000 7f0bf3f83640 1 -- 172.21.15.92:0/2283307257 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0bec0b2bf0 con 0x7f0bec0cf7d0 2023-11-24T06:11:30.911 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:30.908+0000 7f0be27fc640 1 -- 172.21.15.92:0/2283307257 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f0be8010c80 con 0x7f0bec0cf7d0 2023-11-24T06:11:30.912 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:30.909+0000 7f0be27fc640 1 --2- 172.21.15.92:0/2283307257 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f0bd006ae40 0x7f0bd006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:30.912 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:30.909+0000 7f0be27fc640 1 -- 172.21.15.92:0/2283307257 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5883+0+0 (secure 0 0 0) 0x7f0be80892a0 con 0x7f0bec0cf7d0 2023-11-24T06:11:30.912 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:30.909+0000 7f0bf151e640 1 --2- 172.21.15.92:0/2283307257 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f0bd006ae40 0x7f0bd006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:30.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:30.910+0000 7f0bf151e640 1 --2- 172.21.15.92:0/2283307257 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f0bd006ae40 0x7f0bd006d300 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f0bec0c3f00 tx=0x7f0bdc006c90 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:30.916 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:30.914+0000 7f0be27fc640 1 -- 172.21.15.92:0/2283307257 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0bec0b2bf0 con 0x7f0bec0cf7d0 2023-11-24T06:11:31.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:31.126+0000 7f0bf3f83640 1 -- 172.21.15.92:0/2283307257 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0bec0ad200 con 0x7f0bec0cf7d0 2023-11-24T06:11:31.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:31.128+0000 7f0be27fc640 1 -- 172.21.15.92:0/2283307257 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4910 (secure 0 0 0) 0x7f0bec0ad200 con 0x7f0bec0cf7d0 2023-11-24T06:11:31.131 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-24T06:11:31.131 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4236,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6836/437604699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":437604699},{"type":"v1","addr":"172.21.15.92:6836","nonce":437604699}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6835/3368711442","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":3368711442},{"type":"v1","addr":"172.21.15.118:6835","nonce":3368711442}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4255,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/2767290380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":2767290380},{"type":"v1","addr":"172.21.15.92:6837","nonce":2767290380}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T06:10:25.115612+0000","modified":"2023-11-24T06:10:28.210438+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4264},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4264":{"gid":4264,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.118:6833/1146052262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":1146052262},{"type":"v1","addr":"172.21.15.118:6833","nonce":1146052262}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-24T06:11:31.132 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 8 2023-11-24T06:11:31.133 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:31.130+0000 7f0bf3f83640 1 -- 172.21.15.92:0/2283307257 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f0bd006ae40 msgr2=0x7f0bd006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:31.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:31.130+0000 7f0bf3f83640 1 --2- 172.21.15.92:0/2283307257 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f0bd006ae40 0x7f0bd006d300 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f0bec0c3f00 tx=0x7f0bdc006c90 comp rx=0 tx=0).stop 2023-11-24T06:11:31.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:31.130+0000 7f0bf3f83640 1 -- 172.21.15.92:0/2283307257 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0bec0cf7d0 msgr2=0x7f0bec0aea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:31.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:31.130+0000 7f0bf3f83640 1 --2- 172.21.15.92:0/2283307257 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0bec0cf7d0 0x7f0bec0aea20 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f0be800b4a0 tx=0x7f0be800b960 comp rx=0 tx=0).stop 2023-11-24T06:11:31.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:31.130+0000 7f0bf3f83640 1 -- 172.21.15.92:0/2283307257 shutdown_connections 2023-11-24T06:11:31.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:31.130+0000 7f0bf3f83640 1 --2- 172.21.15.92:0/2283307257 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f0bd006ae40 0x7f0bd006d300 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:31.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:31.130+0000 7f0bf3f83640 1 --2- 172.21.15.92:0/2283307257 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0bec15e1e0 0x7f0bec0cd120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:31.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:31.130+0000 7f0bf3f83640 1 --2- 172.21.15.92:0/2283307257 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0bec143ae0 0x7f0bec0aef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:31.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:31.130+0000 7f0bf3f83640 1 --2- 172.21.15.92:0/2283307257 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0bec0cf7d0 0x7f0bec0aea20 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:31.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:31.130+0000 7f0bf3f83640 1 -- 172.21.15.92:0/2283307257 >> 172.21.15.92:0/2283307257 conn(0x7f0bec06a800 msgr2=0x7f0bec0b0bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:31.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:31.130+0000 7f0bf3f83640 1 -- 172.21.15.92:0/2283307257 shutdown_connections 2023-11-24T06:11:31.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:31.131+0000 7f0bf3f83640 1 -- 172.21.15.92:0/2283307257 wait complete. 2023-11-24T06:11:31.151 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T06:11:31.343 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:31.340+0000 7fbed6064640 1 Processor -- start 2023-11-24T06:11:31.344 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:31.340+0000 7fbed6064640 1 -- start start 2023-11-24T06:11:31.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:31.341+0000 7fbed6064640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fbed00cf9a0 0x7fbed0166030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:31.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:31.341+0000 7fbed6064640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbed015c5f0 con 0x7fbed013e360 2023-11-24T06:11:31.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:31.341+0000 7fbed6064640 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbed006c470 con 0x7fbed013df80 2023-11-24T06:11:31.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:31.341+0000 7fbed6064640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbed006c750 con 0x7fbed00cf9a0 2023-11-24T06:11:31.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:31.342+0000 7fbecffff640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fbed00cf9a0 0x7fbed0166030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:31.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:31.342+0000 7fbec7fff640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fbed013e360 0x7fbed00cf290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:35354/0 (socket says 172.21.15.92:35354) 2023-11-24T06:11:31.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:31.342+0000 7fbec7fff640 1 -- 172.21.15.92:0/618816044 learned_addr learned my addr 172.21.15.92:0/618816044 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-24T06:11:31.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:31.342+0000 7fbecffff640 1 -- 172.21.15.92:0/618816044 >> v1:172.21.15.118:6789/0 conn(0x7fbed013df80 legacy=0x7fbed015bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:31.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:31.342+0000 7fbecffff640 1 -- 172.21.15.92:0/618816044 >> v1:172.21.15.92:6789/0 conn(0x7fbed013e360 legacy=0x7fbed00cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:31.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:31.342+0000 7fbecffff640 1 -- 172.21.15.92:0/618816044 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbed006ca30 con 0x7fbed00cf9a0 2023-11-24T06:11:31.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:31.342+0000 7fbeceffd640 1 -- 172.21.15.92:0/618816044 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1711488501 0 0) 0x7fbed015c5f0 con 0x7fbed013e360 2023-11-24T06:11:31.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:31.343+0000 7fbecffff640 1 --2- 172.21.15.92:0/618816044 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbed00cf9a0 0x7fbed0166030 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fbec8009d90 tx=0x7fbec802bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e2ba0e4ebf40e92 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:31.348 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:31.344+0000 7fbeceffd640 1 -- 172.21.15.92:0/618816044 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbec802cd00 con 0x7fbed00cf9a0 2023-11-24T06:11:31.348 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:31.344+0000 7fbeceffd640 1 -- 172.21.15.92:0/618816044 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbec802ce60 con 0x7fbed00cf9a0 2023-11-24T06:11:31.348 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:31.344+0000 7fbeceffd640 1 -- 172.21.15.92:0/618816044 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbec8034660 con 0x7fbed00cf9a0 2023-11-24T06:11:31.348 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:31.344+0000 7fbed6064640 1 -- 172.21.15.92:0/618816044 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbed00cf9a0 msgr2=0x7fbed0166030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:31.348 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:31.344+0000 7fbed6064640 1 --2- 172.21.15.92:0/618816044 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbed00cf9a0 0x7fbed0166030 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fbec8009d90 tx=0x7fbec802bf50 comp rx=0 tx=0).stop 2023-11-24T06:11:31.348 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:31.344+0000 7fbed6064640 1 -- 172.21.15.92:0/618816044 shutdown_connections 2023-11-24T06:11:31.348 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:31.344+0000 7fbed6064640 1 --2- 172.21.15.92:0/618816044 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbed00cf9a0 0x7fbed0166030 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:31.348 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:31.344+0000 7fbed6064640 1 -- 172.21.15.92:0/618816044 >> 172.21.15.92:0/618816044 conn(0x7fbed006a800 msgr2=0x7fbed006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:31.348 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:31.344+0000 7fbed6064640 1 -- 172.21.15.92:0/618816044 shutdown_connections 2023-11-24T06:11:31.348 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:31.344+0000 7fbed6064640 1 -- 172.21.15.92:0/618816044 wait complete. 2023-11-24T06:11:31.349 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:31.346+0000 7fbed6064640 1 Processor -- start 2023-11-24T06:11:31.349 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:31.346+0000 7fbed6064640 1 -- start start 2023-11-24T06:11:31.351 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:31.347+0000 7fbed6064640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbed00cf9a0 0x7fbed013d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:31.351 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:31.347+0000 7fbed6064640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fbed013df80 0x7fbed01589d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:31.351 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:31.347+0000 7fbed6064640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbed013e360 0x7fbed015a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:31.351 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:31.347+0000 7fbed6064640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbed013ddb0 con 0x7fbed00cf9a0 2023-11-24T06:11:31.351 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:31.347+0000 7fbed6064640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbed01f3d20 con 0x7fbed013e360 2023-11-24T06:11:31.351 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:31.347+0000 7fbed6064640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbed013cd90 con 0x7fbed013df80 2023-11-24T06:11:31.351 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:31.347+0000 7fbecf7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbed00cf9a0 0x7fbed013d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:31.351 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:31.347+0000 7fbecf7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbed00cf9a0 0x7fbed013d870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:57830/0 (socket says 172.21.15.92:57830) 2023-11-24T06:11:31.352 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:31.347+0000 7fbecf7fe640 1 -- 172.21.15.92:0/1037678435 learned_addr learned my addr 172.21.15.92:0/1037678435 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:11:31.352 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:31.348+0000 7fbecf7fe640 1 -- 172.21.15.92:0/1037678435 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fbed013df80 msgr2=0x7fbed01589d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:31.352 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:31.348+0000 7fbecf7fe640 1 --2- 172.21.15.92:0/1037678435 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fbed013df80 0x7fbed01589d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:31.352 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:31.348+0000 7fbecf7fe640 1 -- 172.21.15.92:0/1037678435 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbed013e360 msgr2=0x7fbed015a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:31.352 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:31.348+0000 7fbecf7fe640 1 --2- 172.21.15.92:0/1037678435 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbed013e360 0x7fbed015a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:31.352 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:31.348+0000 7fbecf7fe640 1 -- 172.21.15.92:0/1037678435 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbec8009a70 con 0x7fbed00cf9a0 2023-11-24T06:11:31.352 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:31.348+0000 7fbecf7fe640 1 --2- 172.21.15.92:0/1037678435 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbed00cf9a0 0x7fbed013d870 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fbec000c400 tx=0x7fbec000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:31.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:31.348+0000 7fbeccff9640 1 -- 172.21.15.92:0/1037678435 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbec00052d0 con 0x7fbed00cf9a0 2023-11-24T06:11:31.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:31.348+0000 7fbeccff9640 1 -- 172.21.15.92:0/1037678435 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbec0005430 con 0x7fbed00cf9a0 2023-11-24T06:11:31.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:31.348+0000 7fbeccff9640 1 -- 172.21.15.92:0/1037678435 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbec0003830 con 0x7fbed00cf9a0 2023-11-24T06:11:31.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:31.348+0000 7fbed6064640 1 -- 172.21.15.92:0/1037678435 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbed013d070 con 0x7fbed00cf9a0 2023-11-24T06:11:31.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:31.348+0000 7fbed6064640 1 -- 172.21.15.92:0/1037678435 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbed0158f10 con 0x7fbed00cf9a0 2023-11-24T06:11:31.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:31.350+0000 7fbed6064640 1 -- 172.21.15.92:0/1037678435 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbed00675d0 con 0x7fbed00cf9a0 2023-11-24T06:11:31.354 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:31.350+0000 7fbeccff9640 1 -- 172.21.15.92:0/1037678435 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fbec0003990 con 0x7fbed00cf9a0 2023-11-24T06:11:31.354 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:31.351+0000 7fbeccff9640 1 --2- 172.21.15.92:0/1037678435 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fbeb006ae40 0x7fbeb006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:31.354 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:31.351+0000 7fbeccff9640 1 -- 172.21.15.92:0/1037678435 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5883+0+0 (secure 0 0 0) 0x7fbec0088cc0 con 0x7fbed00cf9a0 2023-11-24T06:11:31.355 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:31.352+0000 7fbec7fff640 1 --2- 172.21.15.92:0/1037678435 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fbeb006ae40 0x7fbeb006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:31.355 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:31.353+0000 7fbec7fff640 1 --2- 172.21.15.92:0/1037678435 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fbeb006ae40 0x7fbeb006d300 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fbeb8007fd0 tx=0x7fbeb8007900 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:31.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:31.364+0000 7fbeccff9640 1 -- 172.21.15.92:0/1037678435 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fbec008f080 con 0x7fbed00cf9a0 2023-11-24T06:11:31.598 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:31.595+0000 7fbed6064640 1 -- 172.21.15.92:0/1037678435 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbed006df00 con 0x7fbed00cf9a0 2023-11-24T06:11:31.599 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:31.596+0000 7fbeccff9640 1 -- 172.21.15.92:0/1037678435 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4910 (secure 0 0 0) 0x7fbec0017090 con 0x7fbed00cf9a0 2023-11-24T06:11:31.600 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-24T06:11:31.600 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4236,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6836/437604699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":437604699},{"type":"v1","addr":"172.21.15.92:6836","nonce":437604699}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6835/3368711442","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":3368711442},{"type":"v1","addr":"172.21.15.118:6835","nonce":3368711442}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4255,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/2767290380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":2767290380},{"type":"v1","addr":"172.21.15.92:6837","nonce":2767290380}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T06:10:25.115612+0000","modified":"2023-11-24T06:10:28.210438+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4264},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4264":{"gid":4264,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.118:6833/1146052262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":1146052262},{"type":"v1","addr":"172.21.15.118:6833","nonce":1146052262}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-24T06:11:31.600 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 8 2023-11-24T06:11:31.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:31.599+0000 7fbed6064640 1 -- 172.21.15.92:0/1037678435 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fbeb006ae40 msgr2=0x7fbeb006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:31.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:31.599+0000 7fbed6064640 1 --2- 172.21.15.92:0/1037678435 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fbeb006ae40 0x7fbeb006d300 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fbeb8007fd0 tx=0x7fbeb8007900 comp rx=0 tx=0).stop 2023-11-24T06:11:31.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:31.599+0000 7fbed6064640 1 -- 172.21.15.92:0/1037678435 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbed00cf9a0 msgr2=0x7fbed013d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:31.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:31.599+0000 7fbed6064640 1 --2- 172.21.15.92:0/1037678435 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbed00cf9a0 0x7fbed013d870 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fbec000c400 tx=0x7fbec000c8c0 comp rx=0 tx=0).stop 2023-11-24T06:11:31.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:31.599+0000 7fbed6064640 1 -- 172.21.15.92:0/1037678435 shutdown_connections 2023-11-24T06:11:31.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:31.599+0000 7fbed6064640 1 --2- 172.21.15.92:0/1037678435 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fbeb006ae40 0x7fbeb006d300 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:31.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:31.599+0000 7fbed6064640 1 --2- 172.21.15.92:0/1037678435 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbed013e360 0x7fbed015a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:31.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:31.599+0000 7fbed6064640 1 --2- 172.21.15.92:0/1037678435 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fbed013df80 0x7fbed01589d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:31.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:31.599+0000 7fbed6064640 1 --2- 172.21.15.92:0/1037678435 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbed00cf9a0 0x7fbed013d870 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:31.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:31.599+0000 7fbed6064640 1 -- 172.21.15.92:0/1037678435 >> 172.21.15.92:0/1037678435 conn(0x7fbed006a800 msgr2=0x7fbed006abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:31.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:31.599+0000 7fbed6064640 1 -- 172.21.15.92:0/1037678435 shutdown_connections 2023-11-24T06:11:31.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:31.599+0000 7fbed6064640 1 -- 172.21.15.92:0/1037678435 wait complete. 2023-11-24T06:11:31.621 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:11:31.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:31.830+0000 7fadbeeff640 1 Processor -- start 2023-11-24T06:11:31.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:31.831+0000 7fadbeeff640 1 -- start start 2023-11-24T06:11:31.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:31.832+0000 7fadbeeff640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fadb8160920 0x7fadb80b3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:31.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:31.832+0000 7fadbeeff640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fadb80d0a10 con 0x7fadb80cf9f0 2023-11-24T06:11:31.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:31.832+0000 7fadbeeff640 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fadb8161e30 con 0x7fadb80cfe70 2023-11-24T06:11:31.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:31.832+0000 7fadbeeff640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadb80d1f50 con 0x7fadb8160920 2023-11-24T06:11:31.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:31.833+0000 7fadbd6fc640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fadb80cf9f0 0x7fadb8161010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:35366/0 (socket says 172.21.15.92:35366) 2023-11-24T06:11:31.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:31.833+0000 7fadbd6fc640 1 -- 172.21.15.92:0/1989361560 learned_addr learned my addr 172.21.15.92:0/1989361560 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-24T06:11:31.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:31.833+0000 7fadbcefb640 1 -- 172.21.15.92:0/1989361560 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2858523430 0 0) 0x7fadb80d0a10 con 0x7fadb80cf9f0 2023-11-24T06:11:31.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:31.833+0000 7fadbcefb640 1 -- 172.21.15.92:0/1989361560 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fada0003520 con 0x7fadb80cf9f0 2023-11-24T06:11:31.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:31.833+0000 7fadbdefd640 1 --2- 172.21.15.92:0/1989361560 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fadb8160920 0x7fadb80b3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:31.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:31.833+0000 7fadbcefb640 1 -- 172.21.15.92:0/1989361560 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3377400364 0 0) 0x7fada0003520 con 0x7fadb80cf9f0 2023-11-24T06:11:31.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:31.833+0000 7fadbcefb640 1 -- 172.21.15.92:0/1989361560 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fadb8160920 msgr2=0x7fadb80b3f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:31.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:31.833+0000 7fadbcefb640 1 --2- 172.21.15.92:0/1989361560 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fadb8160920 0x7fadb80b3f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:31.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:31.833+0000 7fadbcefb640 1 -- 172.21.15.92:0/1989361560 >> v1:172.21.15.118:6789/0 conn(0x7fadb80cfe70 legacy=0x7fadb8161720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:31.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:31.833+0000 7fadbcefb640 1 -- 172.21.15.92:0/1989361560 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadb80cf080 con 0x7fadb80cf9f0 2023-11-24T06:11:31.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:31.833+0000 7fadbcefb640 1 -- 172.21.15.92:0/1989361560 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2607661350 0 0) 0x7fadac001670 con 0x7fadb80cf9f0 2023-11-24T06:11:31.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:31.833+0000 7fadbcefb640 1 -- 172.21.15.92:0/1989361560 >> v1:172.21.15.92:6789/0 conn(0x7fadb80cf9f0 legacy=0x7fadb8161010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:31.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:31.834+0000 7fadbcefb640 1 --2- 172.21.15.92:0/1989361560 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fada00039a0 0x7fada0003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:31.839 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:31.834+0000 7fadbcefb640 1 --2- 172.21.15.92:0/1989361560 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fada00047c0 0x7fada0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:31.839 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:31.834+0000 7fadbcefb640 1 --2- 172.21.15.92:0/1989361560 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fadb8160920 0x7fada0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:31.839 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:31.834+0000 7fadbcefb640 1 -- 172.21.15.92:0/1989361560 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadb80d1f50 con 0x7fada00039a0 2023-11-24T06:11:31.839 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:31.834+0000 7fadbcefb640 1 -- 172.21.15.92:0/1989361560 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fada000ba30 con 0x7fada00047c0 2023-11-24T06:11:31.839 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:31.834+0000 7fadbcefb640 1 -- 172.21.15.92:0/1989361560 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fada000bbf0 con 0x7fadb8160920 2023-11-24T06:11:31.839 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:31.834+0000 7fadbdefd640 1 --2- 172.21.15.92:0/1989361560 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fada00039a0 0x7fada0003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:31.839 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:31.834+0000 7fadbdefd640 1 -- 172.21.15.92:0/1989361560 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fadb8160920 msgr2=0x7fada0005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:31.839 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:31.834+0000 7fadbdefd640 1 --2- 172.21.15.92:0/1989361560 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fadb8160920 0x7fada0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:31.839 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:31.834+0000 7fadbdefd640 1 -- 172.21.15.92:0/1989361560 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fada00047c0 msgr2=0x7fada0004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:31.839 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:31.834+0000 7fadbdefd640 1 --2- 172.21.15.92:0/1989361560 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fada00047c0 0x7fada0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:31.839 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:31.834+0000 7fadbdefd640 1 -- 172.21.15.92:0/1989361560 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fada0003520 con 0x7fada00039a0 2023-11-24T06:11:31.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:31.835+0000 7fadbdefd640 1 --2- 172.21.15.92:0/1989361560 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fada00039a0 0x7fada0003dc0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fada8009f90 tx=0x7fada00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a802d81e0586311 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:31.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:31.835+0000 7fadbcefb640 1 -- 172.21.15.92:0/1989361560 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fada802cbf0 con 0x7fada00039a0 2023-11-24T06:11:31.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:31.836+0000 7fadbcefb640 1 -- 172.21.15.92:0/1989361560 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fada800f040 con 0x7fada00039a0 2023-11-24T06:11:31.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:31.836+0000 7fadbcefb640 1 -- 172.21.15.92:0/1989361560 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fada8034440 con 0x7fada00039a0 2023-11-24T06:11:31.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:31.836+0000 7fadbeeff640 1 -- 172.21.15.92:0/1989361560 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fada00039a0 msgr2=0x7fada0003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:31.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:31.836+0000 7fadbeeff640 1 --2- 172.21.15.92:0/1989361560 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fada00039a0 0x7fada0003dc0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fada8009f90 tx=0x7fada00057f0 comp rx=0 tx=0).stop 2023-11-24T06:11:31.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:31.836+0000 7fadbe6fe640 1 -- 172.21.15.92:0/1989361560 reap_dead start 2023-11-24T06:11:31.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:31.836+0000 7fadbeeff640 1 -- 172.21.15.92:0/1989361560 shutdown_connections 2023-11-24T06:11:31.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:31.836+0000 7fadbeeff640 1 -- 172.21.15.92:0/1989361560 >> 172.21.15.92:0/1989361560 conn(0x7fadb80d3090 msgr2=0x7fadb80d34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:31.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:31.836+0000 7fadbeeff640 1 -- 172.21.15.92:0/1989361560 shutdown_connections 2023-11-24T06:11:31.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:31.836+0000 7fadbeeff640 1 -- 172.21.15.92:0/1989361560 wait complete. 2023-11-24T06:11:31.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:31.838+0000 7fadbeeff640 1 Processor -- start 2023-11-24T06:11:31.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:31.838+0000 7fadbeeff640 1 -- start start 2023-11-24T06:11:31.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:31.839+0000 7fadbeeff640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fada00039a0 0x7fadb81f2970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:31.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:31.839+0000 7fadbeeff640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fadb80cce00 0x7fadb80b6770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:31.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:31.839+0000 7fadbeeff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fadb80b6cb0 0x7fadb815b820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:31.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:31.839+0000 7fadbeeff640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadb80cc9e0 con 0x7fadb80b6cb0 2023-11-24T06:11:31.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:31.839+0000 7fadbeeff640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadb8151070 con 0x7fada00039a0 2023-11-24T06:11:31.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:31.839+0000 7fadbeeff640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadb80b7230 con 0x7fadb80cce00 2023-11-24T06:11:31.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:31.839+0000 7fadbe6fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fadb80b6cb0 0x7fadb815b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:31.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:31.839+0000 7fadbe6fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fadb80b6cb0 0x7fadb815b820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:57842/0 (socket says 172.21.15.92:57842) 2023-11-24T06:11:31.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:31.839+0000 7fadbe6fe640 1 -- 172.21.15.92:0/2874535497 learned_addr learned my addr 172.21.15.92:0/2874535497 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:11:31.844 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:31.840+0000 7fadbe6fe640 1 -- 172.21.15.92:0/2874535497 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fadb80cce00 msgr2=0x7fadb80b6770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:31.844 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:31.840+0000 7fadbe6fe640 1 --2- 172.21.15.92:0/2874535497 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fadb80cce00 0x7fadb80b6770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:31.844 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:31.840+0000 7fadbe6fe640 1 -- 172.21.15.92:0/2874535497 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fada00039a0 msgr2=0x7fadb81f2970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:31.844 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:31.840+0000 7fadbe6fe640 1 --2- 172.21.15.92:0/2874535497 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fada00039a0 0x7fadb81f2970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:31.844 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:31.840+0000 7fadbe6fe640 1 -- 172.21.15.92:0/2874535497 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadb80cc320 con 0x7fadb80b6cb0 2023-11-24T06:11:31.844 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:31.840+0000 7fadbe6fe640 1 --2- 172.21.15.92:0/2874535497 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fadb80b6cb0 0x7fadb815b820 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fadb808c060 tx=0x7fadb000c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:31.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:31.840+0000 7fadb67fc640 1 -- 172.21.15.92:0/2874535497 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadb00072e0 con 0x7fadb80b6cb0 2023-11-24T06:11:31.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:31.840+0000 7fadb67fc640 1 -- 172.21.15.92:0/2874535497 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadb0007440 con 0x7fadb80b6cb0 2023-11-24T06:11:31.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:31.840+0000 7fadb67fc640 1 -- 172.21.15.92:0/2874535497 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadb0004020 con 0x7fadb80b6cb0 2023-11-24T06:11:31.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:31.840+0000 7fadbeeff640 1 -- 172.21.15.92:0/2874535497 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fadb80cc600 con 0x7fadb80b6cb0 2023-11-24T06:11:31.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:31.840+0000 7fadbeeff640 1 -- 172.21.15.92:0/2874535497 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadb80b8ba0 con 0x7fadb80b6cb0 2023-11-24T06:11:31.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:31.842+0000 7fadbeeff640 1 -- 172.21.15.92:0/2874535497 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fadb8067450 con 0x7fadb80b6cb0 2023-11-24T06:11:31.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:31.843+0000 7fadb67fc640 1 -- 172.21.15.92:0/2874535497 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fadb00041a0 con 0x7fadb80b6cb0 2023-11-24T06:11:31.846 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:31.843+0000 7fadb67fc640 1 --2- 172.21.15.92:0/2874535497 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fada406ae70 0x7fada406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:31.846 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:31.844+0000 7fadb67fc640 1 -- 172.21.15.92:0/2874535497 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5883+0+0 (secure 0 0 0) 0x7fadb00887e0 con 0x7fadb80b6cb0 2023-11-24T06:11:31.847 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:31.844+0000 7fadbdefd640 1 --2- 172.21.15.92:0/2874535497 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fada406ae70 0x7fada406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:31.847 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:31.844+0000 7fadbdefd640 1 --2- 172.21.15.92:0/2874535497 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fada406ae70 0x7fada406d330 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fada8036000 tx=0x7fada802e070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:31.851 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:31.848+0000 7fadb67fc640 1 -- 172.21.15.92:0/2874535497 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fadb00537e0 con 0x7fadb80b6cb0 2023-11-24T06:11:32.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:32.038+0000 7fadbeeff640 1 -- 172.21.15.92:0/2874535497 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fadb808c7e0 con 0x7fadb80b6cb0 2023-11-24T06:11:32.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:32.041+0000 7fadb67fc640 1 -- 172.21.15.92:0/2874535497 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16800 (secure 0 0 0) 0x7fadb0056e00 con 0x7fadb80b6cb0 2023-11-24T06:11:32.044 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-24T06:11:32.045 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":22,"fsid":"2d6dee5f-8aa3-470f-931a-50ade963b1cb","created":"2023-11-24T06:10:04.707204+0000","modified":"2023-11-24T06:11:11.472401+0000","last_up_change":"2023-11-24T06:10:17.598352+0000","last_in_change":"2023-11-24T06:10:09.040793+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:10:17.733098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:10:22.926445+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T06:10:24.151288+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ff37a9fb-135f-471f-b4b0-f7b5772006c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6803","nonce":20560714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6805","nonce":20560714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6809","nonce":20560714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6807","nonce":20560714}]},"public_addr":"172.21.15.92:6803/20560714","cluster_addr":"172.21.15.92:6805/20560714","heartbeat_back_addr":"172.21.15.92:6809/20560714","heartbeat_front_addr":"172.21.15.92:6807/20560714","state":["exists","up"]},{"osd":1,"uuid":"03cdd353-dcf1-4f38-94fe-a795137c06b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6819","nonce":2451239906}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6821","nonce":2451239906}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6825","nonce":2451239906}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6823","nonce":2451239906}]},"public_addr":"172.21.15.92:6819/2451239906","cluster_addr":"172.21.15.92:6821/2451239906","heartbeat_back_addr":"172.21.15.92:6825/2451239906","heartbeat_front_addr":"172.21.15.92:6823/2451239906","state":["exists","up"]},{"osd":2,"uuid":"a4a20d80-5d57-484b-becd-9aa8070dd3aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6811","nonce":1124979277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6813","nonce":1124979277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6817","nonce":1124979277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6815","nonce":1124979277}]},"public_addr":"172.21.15.92:6811/1124979277","cluster_addr":"172.21.15.92:6813/1124979277","heartbeat_back_addr":"172.21.15.92:6817/1124979277","heartbeat_front_addr":"172.21.15.92:6815/1124979277","state":["exists","up"]},{"osd":3,"uuid":"4648d7ce-9b29-4583-9425-4ba8fd2b1903","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6827","nonce":1734115782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6829","nonce":1734115782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6833","nonce":1734115782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6831","nonce":1734115782}]},"public_addr":"172.21.15.92:6827/1734115782","cluster_addr":"172.21.15.92:6829/1734115782","heartbeat_back_addr":"172.21.15.92:6833/1734115782","heartbeat_front_addr":"172.21.15.92:6831/1734115782","state":["exists","up"]},{"osd":4,"uuid":"3bc9d7eb-c6fa-4fcf-a3ec-ade604e05aac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6819","nonce":3425937229}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6822","nonce":3425937229}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6830","nonce":3425937229}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6826","nonce":3425937229}]},"public_addr":"172.21.15.118:6819/3425937229","cluster_addr":"172.21.15.118:6822/3425937229","heartbeat_back_addr":"172.21.15.118:6830/3425937229","heartbeat_front_addr":"172.21.15.118:6826/3425937229","state":["exists","up"]},{"osd":5,"uuid":"dce9ba6d-59ed-46d1-84fc-a84b13de10c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6801","nonce":827455984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6803","nonce":827455984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6807","nonce":827455984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6805","nonce":827455984}]},"public_addr":"172.21.15.118:6801/827455984","cluster_addr":"172.21.15.118:6803/827455984","heartbeat_back_addr":"172.21.15.118:6807/827455984","heartbeat_front_addr":"172.21.15.118:6805/827455984","state":["exists","up"]},{"osd":6,"uuid":"82aec504-ae58-4709-8d30-46f2b12081b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6809","nonce":2946986141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6811","nonce":2946986141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6815","nonce":2946986141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6813","nonce":2946986141}]},"public_addr":"172.21.15.118:6809/2946986141","cluster_addr":"172.21.15.118:6811/2946986141","heartbeat_back_addr":"172.21.15.118:6815/2946986141","heartbeat_front_addr":"172.21.15.118:6813/2946986141","state":["exists","up"]},{"osd":7,"uuid":"4ed4c624-c88f-4d35-bdb1-bf209b5dffb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6817","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6818","nonce":48122059}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6821","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6823","nonce":48122059}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6829","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6831","nonce":48122059}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6825","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6827","nonce":48122059}]},"public_addr":"172.21.15.118:6818/48122059","cluster_addr":"172.21.15.118:6823/48122059","heartbeat_back_addr":"172.21.15.118:6831/48122059","heartbeat_front_addr":"172.21.15.118:6827/48122059","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.536664+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.368324+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.271201+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.342971+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.621832+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.387110+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.632946+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.416032+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.d","mappings":[{"from":5,"to":1}]},{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"3.29","mappings":[{"from":3,"to":2}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.34","mappings":[{"from":4,"to":2}]},{"pgid":"3.3b","mappings":[{"from":7,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:11:32.046 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:32.043+0000 7fadbeeff640 1 -- 172.21.15.92:0/2874535497 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fada406ae70 msgr2=0x7fada406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:32.046 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:32.043+0000 7fadbeeff640 1 --2- 172.21.15.92:0/2874535497 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fada406ae70 0x7fada406d330 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fada8036000 tx=0x7fada802e070 comp rx=0 tx=0).stop 2023-11-24T06:11:32.046 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:32.043+0000 7fadbeeff640 1 -- 172.21.15.92:0/2874535497 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fadb80b6cb0 msgr2=0x7fadb815b820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:32.047 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:32.043+0000 7fadbeeff640 1 --2- 172.21.15.92:0/2874535497 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fadb80b6cb0 0x7fadb815b820 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fadb808c060 tx=0x7fadb000c820 comp rx=0 tx=0).stop 2023-11-24T06:11:32.047 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:32.043+0000 7fadbeeff640 1 -- 172.21.15.92:0/2874535497 shutdown_connections 2023-11-24T06:11:32.047 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:32.043+0000 7fadbeeff640 1 --2- 172.21.15.92:0/2874535497 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fada406ae70 0x7fada406d330 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:32.047 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:32.043+0000 7fadbeeff640 1 --2- 172.21.15.92:0/2874535497 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fadb80b6cb0 0x7fadb815b820 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:32.047 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:32.043+0000 7fadbeeff640 1 --2- 172.21.15.92:0/2874535497 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fadb80cce00 0x7fadb80b6770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:32.047 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:32.043+0000 7fadbeeff640 1 --2- 172.21.15.92:0/2874535497 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fada00039a0 0x7fadb81f2970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:32.047 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:32.043+0000 7fadbeeff640 1 -- 172.21.15.92:0/2874535497 >> 172.21.15.92:0/2874535497 conn(0x7fadb80d3090 msgr2=0x7fadb80d34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:32.047 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:32.044+0000 7fadbeeff640 1 -- 172.21.15.92:0/2874535497 shutdown_connections 2023-11-24T06:11:32.048 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:32.044+0000 7fadbeeff640 1 -- 172.21.15.92:0/2874535497 wait complete. 2023-11-24T06:11:32.066 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-24T06:11:32.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:32.275+0000 7f211d251640 1 Processor -- start 2023-11-24T06:11:32.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:32.276+0000 7f211d251640 1 -- start start 2023-11-24T06:11:32.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:32.277+0000 7f211d251640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f21180ae5a0 0x7f211815e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:32.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:32.277+0000 7f211d251640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2118070c40 con 0x7f21180cf290 2023-11-24T06:11:32.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:32.277+0000 7f211d251640 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f211806c470 con 0x7f21180b3ae0 2023-11-24T06:11:32.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:32.277+0000 7f211d251640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2118071450 con 0x7f21180ae5a0 2023-11-24T06:11:32.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:32.277+0000 7f211759e640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f21180cf290 0x7f21180b1fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:35376/0 (socket says 172.21.15.92:35376) 2023-11-24T06:11:32.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:32.278+0000 7f211759e640 1 -- 172.21.15.92:0/2168326880 learned_addr learned my addr 172.21.15.92:0/2168326880 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-24T06:11:32.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:32.278+0000 7f2116d9d640 1 --2- 172.21.15.92:0/2168326880 >> v2:172.21.15.92:3300/0 conn(0x7f21180ae5a0 0x7f211815e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:32.282 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:32.278+0000 7f211659c640 1 -- 172.21.15.92:0/2168326880 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1165844104 0 0) 0x7f2118070c40 con 0x7f21180cf290 2023-11-24T06:11:32.283 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:32.278+0000 7f211659c640 1 -- 172.21.15.92:0/2168326880 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2104003520 con 0x7f21180cf290 2023-11-24T06:11:32.283 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:32.278+0000 7f2116d9d640 1 -- 172.21.15.92:0/2168326880 >> v1:172.21.15.118:6789/0 conn(0x7f21180b3ae0 legacy=0x7f211806cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:32.283 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:32.278+0000 7f2116d9d640 1 -- 172.21.15.92:0/2168326880 >> v1:172.21.15.92:6789/0 conn(0x7f21180cf290 legacy=0x7f21180b1fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:32.283 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:32.278+0000 7f2116d9d640 1 -- 172.21.15.92:0/2168326880 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f211806c750 con 0x7f21180ae5a0 2023-11-24T06:11:32.283 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:32.279+0000 7f2116d9d640 1 --2- 172.21.15.92:0/2168326880 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f21180ae5a0 0x7f211815e430 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f21080097a0 tx=0x7f210802b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=236338ad00af909b server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:32.283 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:32.279+0000 7f211659c640 1 -- 172.21.15.92:0/2168326880 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2108030610 con 0x7f21180ae5a0 2023-11-24T06:11:32.283 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:32.280+0000 7f211659c640 1 -- 172.21.15.92:0/2168326880 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2108030770 con 0x7f21180ae5a0 2023-11-24T06:11:32.283 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:32.280+0000 7f211659c640 1 -- 172.21.15.92:0/2168326880 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2108030b00 con 0x7f21180ae5a0 2023-11-24T06:11:32.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:32.280+0000 7f211d251640 1 -- 172.21.15.92:0/2168326880 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f21180ae5a0 msgr2=0x7f211815e430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:32.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:32.280+0000 7f211d251640 1 --2- 172.21.15.92:0/2168326880 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f21180ae5a0 0x7f211815e430 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f21080097a0 tx=0x7f210802b960 comp rx=0 tx=0).stop 2023-11-24T06:11:32.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:32.280+0000 7f211d251640 1 -- 172.21.15.92:0/2168326880 shutdown_connections 2023-11-24T06:11:32.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:32.280+0000 7f211d251640 1 --2- 172.21.15.92:0/2168326880 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f21180ae5a0 0x7f211815e430 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:32.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:32.280+0000 7f211d251640 1 -- 172.21.15.92:0/2168326880 >> 172.21.15.92:0/2168326880 conn(0x7f211806a800 msgr2=0x7f211806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:32.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:32.280+0000 7f211d251640 1 -- 172.21.15.92:0/2168326880 shutdown_connections 2023-11-24T06:11:32.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:32.280+0000 7f211d251640 1 -- 172.21.15.92:0/2168326880 wait complete. 2023-11-24T06:11:32.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:32.282+0000 7f211d251640 1 Processor -- start 2023-11-24T06:11:32.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:32.282+0000 7f211d251640 1 -- start start 2023-11-24T06:11:32.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:32.283+0000 7f211d251640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f21180ae5a0 0x7f21181450b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:32.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:32.283+0000 7f211d251640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f21180b3ae0 0x7f211815d820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:32.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:32.283+0000 7f211d251640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f21180cf290 0x7f211815dd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:32.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:32.283+0000 7f211d251640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2118147930 con 0x7f21180cf290 2023-11-24T06:11:32.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:32.283+0000 7f211d251640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2118147c10 con 0x7f21180b3ae0 2023-11-24T06:11:32.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:32.283+0000 7f211d251640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2118147ef0 con 0x7f21180ae5a0 2023-11-24T06:11:32.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:32.283+0000 7f211759e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f21180cf290 0x7f211815dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:32.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:32.284+0000 7f211759e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f21180cf290 0x7f211815dd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:57870/0 (socket says 172.21.15.92:57870) 2023-11-24T06:11:32.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:32.284+0000 7f211759e640 1 -- 172.21.15.92:0/1116122820 learned_addr learned my addr 172.21.15.92:0/1116122820 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:11:32.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:32.284+0000 7f211759e640 1 -- 172.21.15.92:0/1116122820 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f21180ae5a0 msgr2=0x7f21181450b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:32.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:32.284+0000 7f211759e640 1 --2- 172.21.15.92:0/1116122820 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f21180ae5a0 0x7f21181450b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:32.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:32.284+0000 7f211759e640 1 -- 172.21.15.92:0/1116122820 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f21180b3ae0 msgr2=0x7f211815d820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:32.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:32.284+0000 7f211759e640 1 --2- 172.21.15.92:0/1116122820 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f21180b3ae0 0x7f211815d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:32.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:32.284+0000 7f211759e640 1 -- 172.21.15.92:0/1116122820 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2108009400 con 0x7f21180cf290 2023-11-24T06:11:32.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:32.284+0000 7f211759e640 1 --2- 172.21.15.92:0/1116122820 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f21180cf290 0x7f211815dd60 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f211000caa0 tx=0x7f211000cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:32.289 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:32.284+0000 7f210f7fe640 1 -- 172.21.15.92:0/1116122820 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f211000ac00 con 0x7f21180cf290 2023-11-24T06:11:32.289 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:32.284+0000 7f210f7fe640 1 -- 172.21.15.92:0/1116122820 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f211000ad60 con 0x7f21180cf290 2023-11-24T06:11:32.289 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:32.284+0000 7f210f7fe640 1 -- 172.21.15.92:0/1116122820 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2110012620 con 0x7f21180cf290 2023-11-24T06:11:32.289 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:32.284+0000 7f211d251640 1 -- 172.21.15.92:0/1116122820 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21181481d0 con 0x7f21180cf290 2023-11-24T06:11:32.289 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:32.285+0000 7f211d251640 1 -- 172.21.15.92:0/1116122820 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2118156de0 con 0x7f21180cf290 2023-11-24T06:11:32.289 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:32.286+0000 7f211d251640 1 -- 172.21.15.92:0/1116122820 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2118071590 con 0x7f21180cf290 2023-11-24T06:11:32.290 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:32.287+0000 7f210f7fe640 1 -- 172.21.15.92:0/1116122820 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2110007070 con 0x7f21180cf290 2023-11-24T06:11:32.291 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:32.288+0000 7f210f7fe640 1 --2- 172.21.15.92:0/1116122820 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f20f806ae40 0x7f20f806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:32.291 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:32.288+0000 7f210f7fe640 1 -- 172.21.15.92:0/1116122820 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5883+0+0 (secure 0 0 0) 0x7f2110089730 con 0x7f21180cf290 2023-11-24T06:11:32.291 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:32.288+0000 7f2116d9d640 1 --2- 172.21.15.92:0/1116122820 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f20f806ae40 0x7f20f806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:32.292 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:32.289+0000 7f2116d9d640 1 --2- 172.21.15.92:0/1116122820 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f20f806ae40 0x7f20f806d300 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f2108036000 tx=0x7f210803b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:32.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:32.293+0000 7f210f7fe640 1 -- 172.21.15.92:0/1116122820 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2110054690 con 0x7f21180cf290 2023-11-24T06:11:32.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:32.526+0000 7f211d251640 1 -- 172.21.15.92:0/1116122820 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f21181486f0 con 0x7f21180cf290 2023-11-24T06:11:32.531 INFO:tasks.ceph.mon.a.smithi092.stderr:2023-11-24T06:11:32.528+0000 7f83f3ce4640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-24T06:11:32.534 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:32.531+0000 7f210f7fe640 1 -- 172.21.15.92:0/1116122820 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v9) v1 ==== 161+0+0 (secure 0 0 0) 0x7f21181486f0 con 0x7f21180cf290 2023-11-24T06:11:32.534 INFO:teuthology.orchestra.run.smithi092.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-24T06:11:32.538 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:32.534+0000 7f211d251640 1 -- 172.21.15.92:0/1116122820 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f20f806ae40 msgr2=0x7f20f806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:32.538 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:32.534+0000 7f211d251640 1 --2- 172.21.15.92:0/1116122820 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f20f806ae40 0x7f20f806d300 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f2108036000 tx=0x7f210803b040 comp rx=0 tx=0).stop 2023-11-24T06:11:32.538 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:32.534+0000 7f211d251640 1 -- 172.21.15.92:0/1116122820 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f21180cf290 msgr2=0x7f211815dd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:32.538 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:32.534+0000 7f211d251640 1 --2- 172.21.15.92:0/1116122820 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f21180cf290 0x7f211815dd60 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f211000caa0 tx=0x7f211000cf60 comp rx=0 tx=0).stop 2023-11-24T06:11:32.538 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:32.535+0000 7f211d251640 1 -- 172.21.15.92:0/1116122820 shutdown_connections 2023-11-24T06:11:32.538 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:32.535+0000 7f211d251640 1 --2- 172.21.15.92:0/1116122820 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f20f806ae40 0x7f20f806d300 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:32.538 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:32.535+0000 7f211d251640 1 --2- 172.21.15.92:0/1116122820 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f21180cf290 0x7f211815dd60 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:32.538 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:32.535+0000 7f211d251640 1 --2- 172.21.15.92:0/1116122820 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f21180b3ae0 0x7f211815d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:32.538 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:32.535+0000 7f211d251640 1 --2- 172.21.15.92:0/1116122820 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f21180ae5a0 0x7f21181450b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:32.539 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:32.535+0000 7f211d251640 1 -- 172.21.15.92:0/1116122820 >> 172.21.15.92:0/1116122820 conn(0x7f211806a800 msgr2=0x7f2118160f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:32.539 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:32.535+0000 7f211d251640 1 -- 172.21.15.92:0/1116122820 shutdown_connections 2023-11-24T06:11:32.539 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:32.535+0000 7f211d251640 1 -- 172.21.15.92:0/1116122820 wait complete. 2023-11-24T06:11:32.556 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-11-24T06:11:32.563 INFO:tasks.ceph.mds.d.smithi118.stderr: -3624> 2023-11-24T06:10:28.936+0000 7fb1594bf640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-24T06:11:32.660 INFO:tasks.ceph.mds.d.smithi118.stdout:starting mds.d at 2023-11-24T06:11:32.766 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:32.762+0000 7fa977844640 1 Processor -- start 2023-11-24T06:11:32.766 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:32.763+0000 7fa977844640 1 -- start start 2023-11-24T06:11:32.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:32.764+0000 7fa977844640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fa9700b3ae0 0x7fa97015e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:32.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:32.764+0000 7fa977844640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa970070c40 con 0x7fa9700ae5a0 2023-11-24T06:11:32.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:32.764+0000 7fa977844640 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa97006c470 con 0x7fa9700cf290 2023-11-24T06:11:32.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:32.764+0000 7fa977844640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa970071450 con 0x7fa9700b3ae0 2023-11-24T06:11:32.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:32.765+0000 7fa974ddf640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fa9700b3ae0 0x7fa97015e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:32.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:32.765+0000 7fa9755e0640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fa9700ae5a0 0x7fa97006cbe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:35384/0 (socket says 172.21.15.92:35384) 2023-11-24T06:11:32.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:32.765+0000 7fa9755e0640 1 -- 172.21.15.92:0/1429105721 learned_addr learned my addr 172.21.15.92:0/1429105721 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-24T06:11:32.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:32.765+0000 7fa974ddf640 1 -- 172.21.15.92:0/1429105721 >> v1:172.21.15.118:6789/0 conn(0x7fa9700cf290 legacy=0x7fa9700b1fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:32.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:32.765+0000 7fa967fff640 1 -- 172.21.15.92:0/1429105721 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2413707823 0 0) 0x7fa970070c40 con 0x7fa9700ae5a0 2023-11-24T06:11:32.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:32.765+0000 7fa974ddf640 1 -- 172.21.15.92:0/1429105721 >> v1:172.21.15.92:6789/0 conn(0x7fa9700ae5a0 legacy=0x7fa97006cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:32.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:32.765+0000 7fa974ddf640 1 -- 172.21.15.92:0/1429105721 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa97006c750 con 0x7fa9700b3ae0 2023-11-24T06:11:32.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:32.766+0000 7fa974ddf640 1 --2- 172.21.15.92:0/1429105721 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa9700b3ae0 0x7fa97015e430 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fa9600097a0 tx=0x7fa96002b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9ccaac2508d5d70 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:32.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:32.766+0000 7fa967fff640 1 -- 172.21.15.92:0/1429105721 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa960030610 con 0x7fa9700b3ae0 2023-11-24T06:11:32.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:32.766+0000 7fa967fff640 1 -- 172.21.15.92:0/1429105721 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa960030770 con 0x7fa9700b3ae0 2023-11-24T06:11:32.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:32.767+0000 7fa967fff640 1 -- 172.21.15.92:0/1429105721 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa960030b00 con 0x7fa9700b3ae0 2023-11-24T06:11:32.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:32.767+0000 7fa977844640 1 -- 172.21.15.92:0/1429105721 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa9700b3ae0 msgr2=0x7fa97015e430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:32.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:32.767+0000 7fa977844640 1 --2- 172.21.15.92:0/1429105721 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa9700b3ae0 0x7fa97015e430 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fa9600097a0 tx=0x7fa96002b960 comp rx=0 tx=0).stop 2023-11-24T06:11:32.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:32.767+0000 7fa977844640 1 -- 172.21.15.92:0/1429105721 shutdown_connections 2023-11-24T06:11:32.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:32.767+0000 7fa977844640 1 --2- 172.21.15.92:0/1429105721 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa9700b3ae0 0x7fa97015e430 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:32.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:32.767+0000 7fa977844640 1 -- 172.21.15.92:0/1429105721 >> 172.21.15.92:0/1429105721 conn(0x7fa97006a800 msgr2=0x7fa97006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:32.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:32.767+0000 7fa977844640 1 -- 172.21.15.92:0/1429105721 shutdown_connections 2023-11-24T06:11:32.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:32.767+0000 7fa977844640 1 -- 172.21.15.92:0/1429105721 wait complete. 2023-11-24T06:11:32.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:32.769+0000 7fa977844640 1 Processor -- start 2023-11-24T06:11:32.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:32.769+0000 7fa977844640 1 -- start start 2023-11-24T06:11:32.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:32.770+0000 7fa977844640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa9700ae5a0 0x7fa970156b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:32.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:32.770+0000 7fa977844640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa9700b3ae0 0x7fa9701584e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:32.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:32.770+0000 7fa977844640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa9700cf290 0x7fa97015c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:32.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:32.770+0000 7fa977844640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa970158d30 con 0x7fa9700ae5a0 2023-11-24T06:11:32.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:32.770+0000 7fa977844640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa970159010 con 0x7fa9700b3ae0 2023-11-24T06:11:32.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:32.770+0000 7fa977844640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9701592f0 con 0x7fa9700cf290 2023-11-24T06:11:32.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:32.770+0000 7fa9755e0640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa9700ae5a0 0x7fa970156b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:32.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:32.770+0000 7fa9755e0640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa9700ae5a0 0x7fa970156b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:57888/0 (socket says 172.21.15.92:57888) 2023-11-24T06:11:32.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:32.770+0000 7fa9755e0640 1 -- 172.21.15.92:0/501090257 learned_addr learned my addr 172.21.15.92:0/501090257 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:11:32.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:32.771+0000 7fa974ddf640 1 --2- 172.21.15.92:0/501090257 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa9700b3ae0 0x7fa9701584e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:32.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:32.771+0000 7fa975de1640 1 --2- 172.21.15.92:0/501090257 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa9700cf290 0x7fa97015c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:32.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:32.771+0000 7fa9755e0640 1 -- 172.21.15.92:0/501090257 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa9700cf290 msgr2=0x7fa97015c720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:32.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:32.771+0000 7fa9755e0640 1 --2- 172.21.15.92:0/501090257 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa9700cf290 0x7fa97015c720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:32.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:32.771+0000 7fa9755e0640 1 -- 172.21.15.92:0/501090257 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa9700b3ae0 msgr2=0x7fa9701584e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:32.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:32.771+0000 7fa9755e0640 1 --2- 172.21.15.92:0/501090257 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa9700b3ae0 0x7fa9701584e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:32.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:32.771+0000 7fa9755e0640 1 -- 172.21.15.92:0/501090257 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa960009400 con 0x7fa9700ae5a0 2023-11-24T06:11:32.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:32.771+0000 7fa9755e0640 1 --2- 172.21.15.92:0/501090257 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa9700ae5a0 0x7fa970156b80 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fa96c00c4d0 tx=0x7fa96c00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:32.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:32.771+0000 7fa965ffb640 1 -- 172.21.15.92:0/501090257 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa96c0074e0 con 0x7fa9700ae5a0 2023-11-24T06:11:32.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:32.771+0000 7fa965ffb640 1 -- 172.21.15.92:0/501090257 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa96c007660 con 0x7fa9700ae5a0 2023-11-24T06:11:32.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:32.771+0000 7fa965ffb640 1 -- 172.21.15.92:0/501090257 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa96c004a20 con 0x7fa9700ae5a0 2023-11-24T06:11:32.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:32.771+0000 7fa977844640 1 -- 172.21.15.92:0/501090257 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9700cb8a0 con 0x7fa9700ae5a0 2023-11-24T06:11:32.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:32.772+0000 7fa977844640 1 -- 172.21.15.92:0/501090257 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9701570c0 con 0x7fa9700ae5a0 2023-11-24T06:11:32.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:32.772+0000 7fa977844640 1 -- 172.21.15.92:0/501090257 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9700cbdc0 con 0x7fa9700ae5a0 2023-11-24T06:11:32.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:32.774+0000 7fa965ffb640 1 -- 172.21.15.92:0/501090257 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa96c004b80 con 0x7fa9700ae5a0 2023-11-24T06:11:32.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:32.774+0000 7fa965ffb640 1 --2- 172.21.15.92:0/501090257 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fa95406ae70 0x7fa95406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:32.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:32.774+0000 7fa965ffb640 1 -- 172.21.15.92:0/501090257 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5969+0+0 (secure 0 0 0) 0x7fa96c088840 con 0x7fa9700ae5a0 2023-11-24T06:11:32.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:32.775+0000 7fa974ddf640 1 --2- 172.21.15.92:0/501090257 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fa95406ae70 0x7fa95406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:32.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:32.775+0000 7fa974ddf640 1 --2- 172.21.15.92:0/501090257 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fa95406ae70 0x7fa95406d330 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fa960036000 tx=0x7fa96003b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:32.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:32.779+0000 7fa965ffb640 1 -- 172.21.15.92:0/501090257 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa96c053870 con 0x7fa9700ae5a0 2023-11-24T06:11:33.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:33.010+0000 7fa977844640 1 -- 172.21.15.92:0/501090257 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fa9700675d0 con 0x7fa9700ae5a0 2023-11-24T06:11:33.538 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:33.534+0000 7fa965ffb640 1 -- 172.21.15.92:0/501090257 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v10) v1 ==== 104+0+0 (secure 0 0 0) 0x7fa96c0180a0 con 0x7fa9700ae5a0 2023-11-24T06:11:33.542 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:33.538+0000 7fa977844640 1 -- 172.21.15.92:0/501090257 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fa95406ae70 msgr2=0x7fa95406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:33.542 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:33.538+0000 7fa977844640 1 --2- 172.21.15.92:0/501090257 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fa95406ae70 0x7fa95406d330 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fa960036000 tx=0x7fa96003b040 comp rx=0 tx=0).stop 2023-11-24T06:11:33.542 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:33.538+0000 7fa977844640 1 -- 172.21.15.92:0/501090257 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa9700ae5a0 msgr2=0x7fa970156b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:33.542 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:33.538+0000 7fa977844640 1 --2- 172.21.15.92:0/501090257 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa9700ae5a0 0x7fa970156b80 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fa96c00c4d0 tx=0x7fa96c00c990 comp rx=0 tx=0).stop 2023-11-24T06:11:33.542 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:33.538+0000 7fa977844640 1 -- 172.21.15.92:0/501090257 shutdown_connections 2023-11-24T06:11:33.542 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:33.538+0000 7fa977844640 1 --2- 172.21.15.92:0/501090257 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fa95406ae70 0x7fa95406d330 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:33.542 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:33.538+0000 7fa977844640 1 --2- 172.21.15.92:0/501090257 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa9700cf290 0x7fa97015c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:33.542 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:33.538+0000 7fa977844640 1 --2- 172.21.15.92:0/501090257 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa9700b3ae0 0x7fa9701584e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:33.542 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:33.538+0000 7fa977844640 1 --2- 172.21.15.92:0/501090257 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa9700ae5a0 0x7fa970156b80 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:33.543 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:33.538+0000 7fa977844640 1 -- 172.21.15.92:0/501090257 >> 172.21.15.92:0/501090257 conn(0x7fa97006a800 msgr2=0x7fa97006cbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:33.543 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:33.538+0000 7fa977844640 1 -- 172.21.15.92:0/501090257 shutdown_connections 2023-11-24T06:11:33.543 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:33.538+0000 7fa977844640 1 -- 172.21.15.92:0/501090257 wait complete. 2023-11-24T06:11:33.562 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-11-24T06:11:33.766 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:33.762+0000 7f28d6933640 1 Processor -- start 2023-11-24T06:11:33.766 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:33.763+0000 7f28d6933640 1 -- start start 2023-11-24T06:11:33.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:33.764+0000 7f28d6933640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f28d013e360 0x7f28d00cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:33.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:33.764+0000 7f28d6933640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28d015c5f0 con 0x7f28d013df80 2023-11-24T06:11:33.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:33.764+0000 7f28d6933640 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28d006c470 con 0x7f28d015f9c0 2023-11-24T06:11:33.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:33.764+0000 7f28d6933640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28d006c750 con 0x7f28d013e360 2023-11-24T06:11:33.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:33.765+0000 7f28cffff640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f28d013df80 0x7f28d015bee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:35390/0 (socket says 172.21.15.92:35390) 2023-11-24T06:11:33.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:33.765+0000 7f28cffff640 1 -- 172.21.15.92:0/841270811 learned_addr learned my addr 172.21.15.92:0/841270811 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-24T06:11:33.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:33.765+0000 7f28cf7fe640 1 --2- 172.21.15.92:0/841270811 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f28d013e360 0x7f28d00cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:33.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:33.765+0000 7f28ceffd640 1 -- 172.21.15.92:0/841270811 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4082781386 0 0) 0x7f28d015c5f0 con 0x7f28d013df80 2023-11-24T06:11:33.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:33.765+0000 7f28ceffd640 1 -- 172.21.15.92:0/841270811 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f28bc003520 con 0x7f28d013df80 2023-11-24T06:11:33.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:33.766+0000 7f28ceffd640 1 -- 172.21.15.92:0/841270811 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4279091642 0 0) 0x7f28bc003520 con 0x7f28d013df80 2023-11-24T06:11:33.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:33.766+0000 7f28ceffd640 1 -- 172.21.15.92:0/841270811 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f28d013e360 msgr2=0x7f28d00cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:33.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:33.766+0000 7f28ceffd640 1 --2- 172.21.15.92:0/841270811 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f28d013e360 0x7f28d00cf290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:33.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:33.766+0000 7f28ceffd640 1 -- 172.21.15.92:0/841270811 >> v1:172.21.15.118:6789/0 conn(0x7f28d015f9c0 legacy=0x7f28d00cd840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:33.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:33.766+0000 7f28ceffd640 1 -- 172.21.15.92:0/841270811 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28d006ca30 con 0x7f28d013df80 2023-11-24T06:11:33.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:33.766+0000 7f28ceffd640 1 -- 172.21.15.92:0/841270811 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2607661350 0 0) 0x7f28c00019f0 con 0x7f28d013df80 2023-11-24T06:11:33.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:33.766+0000 7f28ceffd640 1 -- 172.21.15.92:0/841270811 >> v1:172.21.15.92:6789/0 conn(0x7f28d013df80 legacy=0x7f28d015bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:33.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:33.766+0000 7f28ceffd640 1 --2- 172.21.15.92:0/841270811 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f28bc0039a0 0x7f28bc003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:33.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:33.766+0000 7f28ceffd640 1 --2- 172.21.15.92:0/841270811 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f28d013e360 0x7f28bc0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:33.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:33.766+0000 7f28ceffd640 1 --2- 172.21.15.92:0/841270811 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f28bc004d00 0x7f28bc005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:33.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:33.766+0000 7f28ceffd640 1 -- 172.21.15.92:0/841270811 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28d006c750 con 0x7f28bc0039a0 2023-11-24T06:11:33.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:33.766+0000 7f28ceffd640 1 -- 172.21.15.92:0/841270811 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28bc00bb10 con 0x7f28bc004d00 2023-11-24T06:11:33.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:33.766+0000 7f28ceffd640 1 -- 172.21.15.92:0/841270811 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28bc00bcd0 con 0x7f28d013e360 2023-11-24T06:11:33.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:33.766+0000 7f28cffff640 1 --2- 172.21.15.92:0/841270811 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f28bc0039a0 0x7f28bc003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:33.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:33.766+0000 7f28cf7fe640 1 --2- 172.21.15.92:0/841270811 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f28d013e360 0x7f28bc0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:33.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:33.766+0000 7f28cffff640 1 -- 172.21.15.92:0/841270811 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f28d013e360 msgr2=0x7f28bc0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:33.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:33.766+0000 7f28cffff640 1 --2- 172.21.15.92:0/841270811 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f28d013e360 0x7f28bc0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:33.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:33.766+0000 7f28d4ed0640 1 --2- 172.21.15.92:0/841270811 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f28bc004d00 0x7f28bc005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:33.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:33.767+0000 7f28cffff640 1 -- 172.21.15.92:0/841270811 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f28bc004d00 msgr2=0x7f28bc005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:33.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:33.767+0000 7f28cffff640 1 --2- 172.21.15.92:0/841270811 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f28bc004d00 0x7f28bc005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:33.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:33.767+0000 7f28cffff640 1 -- 172.21.15.92:0/841270811 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28bc003520 con 0x7f28bc0039a0 2023-11-24T06:11:33.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:33.767+0000 7f28cf7fe640 1 --2- 172.21.15.92:0/841270811 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f28d013e360 0x7f28bc0047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:11:33.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:33.768+0000 7f28cffff640 1 --2- 172.21.15.92:0/841270811 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f28bc0039a0 0x7f28bc003dc0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f28c000a830 tx=0x7f28bc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31380c3a68762e32 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:33.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:33.768+0000 7f28ceffd640 1 -- 172.21.15.92:0/841270811 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28c00072d0 con 0x7f28bc0039a0 2023-11-24T06:11:33.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:33.768+0000 7f28ceffd640 1 -- 172.21.15.92:0/841270811 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28c002f070 con 0x7f28bc0039a0 2023-11-24T06:11:33.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:33.768+0000 7f28ceffd640 1 -- 172.21.15.92:0/841270811 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28c0007650 con 0x7f28bc0039a0 2023-11-24T06:11:33.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:33.768+0000 7f28d6933640 1 -- 172.21.15.92:0/841270811 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f28bc0039a0 msgr2=0x7f28bc003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:33.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:33.768+0000 7f28d6933640 1 --2- 172.21.15.92:0/841270811 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f28bc0039a0 0x7f28bc003dc0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f28c000a830 tx=0x7f28bc0057f0 comp rx=0 tx=0).stop 2023-11-24T06:11:33.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:33.768+0000 7f28d4ed0640 1 -- 172.21.15.92:0/841270811 reap_dead start 2023-11-24T06:11:33.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:33.769+0000 7f28d6933640 1 -- 172.21.15.92:0/841270811 shutdown_connections 2023-11-24T06:11:33.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:33.769+0000 7f28d6933640 1 -- 172.21.15.92:0/841270811 >> 172.21.15.92:0/841270811 conn(0x7f28d006a800 msgr2=0x7f28d006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:33.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:33.769+0000 7f28d6933640 1 -- 172.21.15.92:0/841270811 shutdown_connections 2023-11-24T06:11:33.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:33.769+0000 7f28d6933640 1 -- 172.21.15.92:0/841270811 wait complete. 2023-11-24T06:11:33.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:33.770+0000 7f28d6933640 1 Processor -- start 2023-11-24T06:11:33.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:33.771+0000 7f28d6933640 1 -- start start 2023-11-24T06:11:33.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:33.772+0000 7f28d6933640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f28bc0039a0 0x7f28d013d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:33.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:33.772+0000 7f28d6933640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f28d015a140 0x7f28d015a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:33.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:33.772+0000 7f28d6933640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f28d01f3090 0x7f28d01f34f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:33.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:33.772+0000 7f28d6933640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28d015ab00 con 0x7f28d015a140 2023-11-24T06:11:33.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:33.772+0000 7f28d6933640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28d014c3f0 con 0x7f28d01f3090 2023-11-24T06:11:33.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:33.772+0000 7f28d6933640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28d014b580 con 0x7f28bc0039a0 2023-11-24T06:11:33.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:33.772+0000 7f28cf7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f28d015a140 0x7f28d015a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:33.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:33.772+0000 7f28cf7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f28d015a140 0x7f28d015a5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:57916/0 (socket says 172.21.15.92:57916) 2023-11-24T06:11:33.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:33.772+0000 7f28cf7fe640 1 -- 172.21.15.92:0/3003221539 learned_addr learned my addr 172.21.15.92:0/3003221539 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:11:33.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:33.772+0000 7f28cffff640 1 --2- 172.21.15.92:0/3003221539 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f28bc0039a0 0x7f28d013d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:33.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:33.772+0000 7f28cf7fe640 1 -- 172.21.15.92:0/3003221539 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f28bc0039a0 msgr2=0x7f28d013d360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:33.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:33.772+0000 7f28cf7fe640 1 --2- 172.21.15.92:0/3003221539 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f28bc0039a0 0x7f28d013d360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:33.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:33.772+0000 7f28cf7fe640 1 -- 172.21.15.92:0/3003221539 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f28d01f3090 msgr2=0x7f28d01f34f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:33.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:33.772+0000 7f28cf7fe640 1 --2- 172.21.15.92:0/3003221539 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f28d01f3090 0x7f28d01f34f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:33.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:33.772+0000 7f28cf7fe640 1 -- 172.21.15.92:0/3003221539 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28d014c880 con 0x7f28d015a140 2023-11-24T06:11:33.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:33.772+0000 7f28cffff640 1 --2- 172.21.15.92:0/3003221539 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f28bc0039a0 0x7f28d013d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:11:33.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:33.773+0000 7f28cf7fe640 1 --2- 172.21.15.92:0/3003221539 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f28d015a140 0x7f28d015a5c0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f28b800e7e0 tx=0x7f28b800eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:33.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:33.773+0000 7f28ccff9640 1 -- 172.21.15.92:0/3003221539 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28b800c6d0 con 0x7f28d015a140 2023-11-24T06:11:33.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:33.773+0000 7f28ccff9640 1 -- 172.21.15.92:0/3003221539 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28b800c830 con 0x7f28d015a140 2023-11-24T06:11:33.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:33.773+0000 7f28ccff9640 1 -- 172.21.15.92:0/3003221539 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28b800cba0 con 0x7f28d015a140 2023-11-24T06:11:33.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:33.773+0000 7f28d6933640 1 -- 172.21.15.92:0/3003221539 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28d014b7e0 con 0x7f28d015a140 2023-11-24T06:11:33.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:33.773+0000 7f28d6933640 1 -- 172.21.15.92:0/3003221539 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28d014bda0 con 0x7f28d015a140 2023-11-24T06:11:33.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:33.774+0000 7f28d6933640 1 -- 172.21.15.92:0/3003221539 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28d00675d0 con 0x7f28d015a140 2023-11-24T06:11:33.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:33.775+0000 7f28ccff9640 1 -- 172.21.15.92:0/3003221539 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f28b800cd00 con 0x7f28d015a140 2023-11-24T06:11:33.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:33.777+0000 7f28ccff9640 1 --2- 172.21.15.92:0/3003221539 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f28b006ae70 0x7f28b006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:33.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:33.777+0000 7f28ccff9640 1 -- 172.21.15.92:0/3003221539 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5969+0+0 (secure 0 0 0) 0x7f28b80893e0 con 0x7f28d015a140 2023-11-24T06:11:33.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:33.777+0000 7f28cffff640 1 --2- 172.21.15.92:0/3003221539 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f28b006ae70 0x7f28b006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:33.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:33.778+0000 7f28cffff640 1 --2- 172.21.15.92:0/3003221539 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f28b006ae70 0x7f28b006d330 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f28c000a830 tx=0x7f28c0007000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:33.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:33.781+0000 7f28ccff9640 1 -- 172.21.15.92:0/3003221539 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f28b8054360 con 0x7f28d015a140 2023-11-24T06:11:33.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:33.973+0000 7f28d6933640 1 -- 172.21.15.92:0/3003221539 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f28d0155ae0 con 0x7f28d015a140 2023-11-24T06:11:34.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:34.947+0000 7f28ccff9640 1 -- 172.21.15.92:0/3003221539 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v24) v1 ==== 181+0+0 (secure 0 0 0) 0x7f28b8057980 con 0x7f28d015a140 2023-11-24T06:11:34.951 INFO:teuthology.orchestra.run.smithi092.stderr:pool 'cephfs_metadata' removed 2023-11-24T06:11:34.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:34.951+0000 7f28d6933640 1 -- 172.21.15.92:0/3003221539 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f28b006ae70 msgr2=0x7f28b006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:34.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:34.951+0000 7f28d6933640 1 --2- 172.21.15.92:0/3003221539 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f28b006ae70 0x7f28b006d330 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f28c000a830 tx=0x7f28c0007000 comp rx=0 tx=0).stop 2023-11-24T06:11:34.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:34.951+0000 7f28d6933640 1 -- 172.21.15.92:0/3003221539 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f28d015a140 msgr2=0x7f28d015a5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:34.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:34.951+0000 7f28d6933640 1 --2- 172.21.15.92:0/3003221539 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f28d015a140 0x7f28d015a5c0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f28b800e7e0 tx=0x7f28b800eca0 comp rx=0 tx=0).stop 2023-11-24T06:11:34.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:34.951+0000 7f28d6933640 1 -- 172.21.15.92:0/3003221539 shutdown_connections 2023-11-24T06:11:34.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:34.951+0000 7f28d6933640 1 --2- 172.21.15.92:0/3003221539 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f28b006ae70 0x7f28b006d330 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:34.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:34.951+0000 7f28d6933640 1 --2- 172.21.15.92:0/3003221539 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f28d01f3090 0x7f28d01f34f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:34.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:34.951+0000 7f28d6933640 1 --2- 172.21.15.92:0/3003221539 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f28d015a140 0x7f28d015a5c0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:34.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:34.951+0000 7f28d6933640 1 --2- 172.21.15.92:0/3003221539 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f28bc0039a0 0x7f28d013d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:34.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:34.951+0000 7f28d6933640 1 -- 172.21.15.92:0/3003221539 >> 172.21.15.92:0/3003221539 conn(0x7f28d006a800 msgr2=0x7f28d015ce70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:34.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:34.952+0000 7f28d6933640 1 -- 172.21.15.92:0/3003221539 shutdown_connections 2023-11-24T06:11:34.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:34.952+0000 7f28d6933640 1 -- 172.21.15.92:0/3003221539 wait complete. 2023-11-24T06:11:34.978 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-11-24T06:11:35.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:35.187+0000 7faa5359e640 1 Processor -- start 2023-11-24T06:11:35.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:35.187+0000 7faa5359e640 1 -- start start 2023-11-24T06:11:35.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:35.189+0000 7faa5359e640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7faa540d0100 0x7faa540b3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:35.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:35.189+0000 7faa5359e640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa540d0a10 con 0x7faa54160920 2023-11-24T06:11:35.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:35.189+0000 7faa5359e640 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa540cf080 con 0x7faa540d0f40 2023-11-24T06:11:35.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:35.189+0000 7faa5359e640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa540c6be0 con 0x7faa540d0100 2023-11-24T06:11:35.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:35.190+0000 7faa5259c640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7faa54160920 0x7faa540cf9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:35402/0 (socket says 172.21.15.92:35402) 2023-11-24T06:11:35.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:35.190+0000 7faa5259c640 1 -- 172.21.15.92:0/636688428 learned_addr learned my addr 172.21.15.92:0/636688428 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-24T06:11:35.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:35.190+0000 7faa5159a640 1 -- 172.21.15.92:0/636688428 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2925879345 0 0) 0x7faa540d0a10 con 0x7faa54160920 2023-11-24T06:11:35.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:35.190+0000 7faa5159a640 1 -- 172.21.15.92:0/636688428 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faa3c003520 con 0x7faa54160920 2023-11-24T06:11:35.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:35.190+0000 7faa51d9b640 1 --2- 172.21.15.92:0/636688428 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7faa540d0100 0x7faa540b3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:35.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:35.190+0000 7faa5159a640 1 -- 172.21.15.92:0/636688428 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2761733541 0 0) 0x7faa3c003520 con 0x7faa54160920 2023-11-24T06:11:35.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:35.190+0000 7faa5159a640 1 -- 172.21.15.92:0/636688428 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7faa540d0100 msgr2=0x7faa540b3f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:35.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:35.190+0000 7faa5159a640 1 --2- 172.21.15.92:0/636688428 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7faa540d0100 0x7faa540b3f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:35.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:35.190+0000 7faa5159a640 1 -- 172.21.15.92:0/636688428 >> v1:172.21.15.118:6789/0 conn(0x7faa540d0f40 legacy=0x7faa540d13f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:35.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:35.190+0000 7faa5159a640 1 -- 172.21.15.92:0/636688428 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa540ce7e0 con 0x7faa54160920 2023-11-24T06:11:35.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:35.190+0000 7faa5159a640 1 -- 172.21.15.92:0/636688428 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2607661350 0 0) 0x7faa44001670 con 0x7faa54160920 2023-11-24T06:11:35.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:35.190+0000 7faa5159a640 1 -- 172.21.15.92:0/636688428 >> v1:172.21.15.92:6789/0 conn(0x7faa54160920 legacy=0x7faa540cf9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:35.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:35.190+0000 7faa5159a640 1 --2- 172.21.15.92:0/636688428 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7faa540d0100 0x7faa3c0039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:35.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:35.190+0000 7faa5159a640 1 --2- 172.21.15.92:0/636688428 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7faa3c004000 0x7faa3c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:35.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:35.191+0000 7faa5159a640 1 --2- 172.21.15.92:0/636688428 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faa3c004d00 0x7faa3c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:35.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:35.191+0000 7faa5159a640 1 -- 172.21.15.92:0/636688428 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa540c6be0 con 0x7faa3c004d00 2023-11-24T06:11:35.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:35.191+0000 7faa5159a640 1 -- 172.21.15.92:0/636688428 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa3c00bb70 con 0x7faa3c004000 2023-11-24T06:11:35.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:35.191+0000 7faa5159a640 1 -- 172.21.15.92:0/636688428 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa3c00bd30 con 0x7faa540d0100 2023-11-24T06:11:35.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:35.191+0000 7faa52d9d640 1 --2- 172.21.15.92:0/636688428 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faa3c004d00 0x7faa3c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:35.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:35.191+0000 7faa5259c640 1 --2- 172.21.15.92:0/636688428 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7faa540d0100 0x7faa3c0039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:35.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:35.191+0000 7faa51d9b640 1 --2- 172.21.15.92:0/636688428 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7faa3c004000 0x7faa3c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:35.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:35.191+0000 7faa52d9d640 1 -- 172.21.15.92:0/636688428 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7faa540d0100 msgr2=0x7faa3c0039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:35.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:35.191+0000 7faa52d9d640 1 --2- 172.21.15.92:0/636688428 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7faa540d0100 0x7faa3c0039a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:35.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:35.191+0000 7faa52d9d640 1 -- 172.21.15.92:0/636688428 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7faa3c004000 msgr2=0x7faa3c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:35.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:35.191+0000 7faa52d9d640 1 --2- 172.21.15.92:0/636688428 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7faa3c004000 0x7faa3c0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:35.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:35.191+0000 7faa52d9d640 1 -- 172.21.15.92:0/636688428 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa3c003520 con 0x7faa3c004d00 2023-11-24T06:11:35.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:35.191+0000 7faa51d9b640 1 --2- 172.21.15.92:0/636688428 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7faa3c004000 0x7faa3c0047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:11:35.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:35.192+0000 7faa52d9d640 1 --2- 172.21.15.92:0/636688428 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faa3c004d00 0x7faa3c005180 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7faa4c0044b0 tx=0x7faa3c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3e2e5eb3b411059 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:35.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:35.192+0000 7faa5159a640 1 -- 172.21.15.92:0/636688428 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa4c030620 con 0x7faa3c004d00 2023-11-24T06:11:35.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:35.193+0000 7faa5159a640 1 -- 172.21.15.92:0/636688428 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa4c030780 con 0x7faa3c004d00 2023-11-24T06:11:35.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:35.193+0000 7faa5159a640 1 -- 172.21.15.92:0/636688428 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa4c030b70 con 0x7faa3c004d00 2023-11-24T06:11:35.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:35.193+0000 7faa5359e640 1 -- 172.21.15.92:0/636688428 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faa3c004d00 msgr2=0x7faa3c005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:35.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:35.193+0000 7faa5359e640 1 --2- 172.21.15.92:0/636688428 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faa3c004d00 0x7faa3c005180 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7faa4c0044b0 tx=0x7faa3c0057f0 comp rx=0 tx=0).stop 2023-11-24T06:11:35.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:35.193+0000 7faa52d9d640 1 -- 172.21.15.92:0/636688428 reap_dead start 2023-11-24T06:11:35.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:35.193+0000 7faa5359e640 1 -- 172.21.15.92:0/636688428 shutdown_connections 2023-11-24T06:11:35.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:35.193+0000 7faa5359e640 1 -- 172.21.15.92:0/636688428 >> 172.21.15.92:0/636688428 conn(0x7faa540d3090 msgr2=0x7faa540d34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:35.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:35.193+0000 7faa5359e640 1 -- 172.21.15.92:0/636688428 shutdown_connections 2023-11-24T06:11:35.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:35.193+0000 7faa5359e640 1 -- 172.21.15.92:0/636688428 wait complete. 2023-11-24T06:11:35.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:35.195+0000 7faa5359e640 1 Processor -- start 2023-11-24T06:11:35.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:35.195+0000 7faa5359e640 1 -- start start 2023-11-24T06:11:35.201 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:35.196+0000 7faa5359e640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7faa3c004d00 0x7faa540b7720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:35.201 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:35.196+0000 7faa5359e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faa540b35c0 0x7faa541f6c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:35.201 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:35.196+0000 7faa5359e640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7faa5415acc0 0x7faa5415cf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:35.201 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:35.196+0000 7faa5359e640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa541f2b00 con 0x7faa540b35c0 2023-11-24T06:11:35.201 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:35.196+0000 7faa5359e640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa541f7160 con 0x7faa5415acc0 2023-11-24T06:11:35.201 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:35.196+0000 7faa5359e640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa541f61d0 con 0x7faa3c004d00 2023-11-24T06:11:35.201 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:35.196+0000 7faa51d9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faa540b35c0 0x7faa541f6c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:35.202 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:35.196+0000 7faa51d9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faa540b35c0 0x7faa541f6c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:57936/0 (socket says 172.21.15.92:57936) 2023-11-24T06:11:35.202 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:35.196+0000 7faa51d9b640 1 -- 172.21.15.92:0/2048527671 learned_addr learned my addr 172.21.15.92:0/2048527671 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:11:35.202 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:35.197+0000 7faa51d9b640 1 -- 172.21.15.92:0/2048527671 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7faa3c004d00 msgr2=0x7faa540b7720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:35.202 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:35.197+0000 7faa51d9b640 1 --2- 172.21.15.92:0/2048527671 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7faa3c004d00 0x7faa540b7720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:35.202 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:35.197+0000 7faa51d9b640 1 -- 172.21.15.92:0/2048527671 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7faa5415acc0 msgr2=0x7faa5415cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:35.202 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:35.197+0000 7faa51d9b640 1 --2- 172.21.15.92:0/2048527671 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7faa5415acc0 0x7faa5415cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:35.202 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:35.197+0000 7faa51d9b640 1 -- 172.21.15.92:0/2048527671 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa541f6790 con 0x7faa540b35c0 2023-11-24T06:11:35.202 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:35.197+0000 7faa51d9b640 1 --2- 172.21.15.92:0/2048527671 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faa540b35c0 0x7faa541f6c20 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7faa4800e7e0 tx=0x7faa4800eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:35.202 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:35.197+0000 7faa42ffd640 1 -- 172.21.15.92:0/2048527671 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa4800c650 con 0x7faa540b35c0 2023-11-24T06:11:35.202 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:35.197+0000 7faa42ffd640 1 -- 172.21.15.92:0/2048527671 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa4800c7b0 con 0x7faa540b35c0 2023-11-24T06:11:35.202 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:35.197+0000 7faa42ffd640 1 -- 172.21.15.92:0/2048527671 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa4800cb60 con 0x7faa540b35c0 2023-11-24T06:11:35.203 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:35.197+0000 7faa5359e640 1 -- 172.21.15.92:0/2048527671 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa540b7e00 con 0x7faa540b35c0 2023-11-24T06:11:35.203 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:35.198+0000 7faa5359e640 1 -- 172.21.15.92:0/2048527671 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa541f6420 con 0x7faa540b35c0 2023-11-24T06:11:35.203 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:35.199+0000 7faa5359e640 1 -- 172.21.15.92:0/2048527671 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa540b8320 con 0x7faa540b35c0 2023-11-24T06:11:35.203 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:35.200+0000 7faa42ffd640 1 -- 172.21.15.92:0/2048527671 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7faa4800ccc0 con 0x7faa540b35c0 2023-11-24T06:11:35.203 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:35.200+0000 7faa42ffd640 1 --2- 172.21.15.92:0/2048527671 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7faa3406ae70 0x7faa3406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:35.204 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:35.201+0000 7faa42ffd640 1 -- 172.21.15.92:0/2048527671 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5531+0+0 (secure 0 0 0) 0x7faa480892d0 con 0x7faa540b35c0 2023-11-24T06:11:35.204 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:35.201+0000 7faa5259c640 1 --2- 172.21.15.92:0/2048527671 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7faa3406ae70 0x7faa3406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:35.204 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:35.201+0000 7faa5259c640 1 --2- 172.21.15.92:0/2048527671 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7faa3406ae70 0x7faa3406d330 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7faa44001eb0 tx=0x7faa44009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:35.208 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:35.205+0000 7faa42ffd640 1 -- 172.21.15.92:0/2048527671 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7faa48054390 con 0x7faa540b35c0 2023-11-24T06:11:35.412 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:35.409+0000 7faa5359e640 1 -- 172.21.15.92:0/2048527671 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7faa540d3f00 con 0x7faa540b35c0 2023-11-24T06:11:35.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:35.951+0000 7faa42ffd640 1 -- 172.21.15.92:0/2048527671 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v25) v1 ==== 169+0+0 (secure 0 0 0) 0x7faa480579b0 con 0x7faa540b35c0 2023-11-24T06:11:35.954 INFO:teuthology.orchestra.run.smithi092.stderr:pool 'cephfs_data' removed 2023-11-24T06:11:35.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:35.956+0000 7faa5359e640 1 -- 172.21.15.92:0/2048527671 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7faa3406ae70 msgr2=0x7faa3406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:35.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:35.956+0000 7faa5359e640 1 --2- 172.21.15.92:0/2048527671 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7faa3406ae70 0x7faa3406d330 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7faa44001eb0 tx=0x7faa44009040 comp rx=0 tx=0).stop 2023-11-24T06:11:35.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:35.956+0000 7faa5359e640 1 -- 172.21.15.92:0/2048527671 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faa540b35c0 msgr2=0x7faa541f6c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:35.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:35.956+0000 7faa5359e640 1 --2- 172.21.15.92:0/2048527671 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faa540b35c0 0x7faa541f6c20 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7faa4800e7e0 tx=0x7faa4800eca0 comp rx=0 tx=0).stop 2023-11-24T06:11:35.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:35.956+0000 7faa5359e640 1 -- 172.21.15.92:0/2048527671 shutdown_connections 2023-11-24T06:11:35.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:35.956+0000 7faa5359e640 1 --2- 172.21.15.92:0/2048527671 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7faa3406ae70 0x7faa3406d330 secure :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7faa44001eb0 tx=0x7faa44009040 comp rx=0 tx=0).stop 2023-11-24T06:11:35.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:35.956+0000 7faa5359e640 1 --2- 172.21.15.92:0/2048527671 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7faa5415acc0 0x7faa5415cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:35.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:35.956+0000 7faa5359e640 1 --2- 172.21.15.92:0/2048527671 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faa540b35c0 0x7faa541f6c20 secure :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7faa4800e7e0 tx=0x7faa4800eca0 comp rx=0 tx=0).stop 2023-11-24T06:11:35.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:35.956+0000 7faa5359e640 1 --2- 172.21.15.92:0/2048527671 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7faa3c004d00 0x7faa540b7720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:35.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:35.956+0000 7faa5359e640 1 -- 172.21.15.92:0/2048527671 >> 172.21.15.92:0/2048527671 conn(0x7faa540d3090 msgr2=0x7faa5406aeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:35.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:35.958+0000 7faa5359e640 1 -- 172.21.15.92:0/2048527671 shutdown_connections 2023-11-24T06:11:35.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:35.958+0000 7faa5359e640 1 -- 172.21.15.92:0/2048527671 wait complete. 2023-11-24T06:11:35.985 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-24T06:11:35.985 INFO:tasks.ceph.mds.a:Stopping old one... 2023-11-24T06:11:35.986 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-24T06:11:35.986 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T06:11:35.986 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-24T06:11:35.986 INFO:tasks.ceph.mds.c:Stopping old one... 2023-11-24T06:11:35.986 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-11-24T06:11:35.986 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T06:11:35.987 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-24T06:11:35.987 INFO:tasks.ceph.mds.b:Stopping old one... 2023-11-24T06:11:35.987 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-24T06:11:35.987 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T06:11:35.987 INFO:tasks.ceph.mds.d:Restarting daemon 2023-11-24T06:11:35.987 INFO:tasks.ceph.mds.d:Stopping old one... 2023-11-24T06:11:35.987 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-11-24T06:11:35.988 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T06:11:36.003 INFO:tasks.ceph.mds.a:Stopped 2023-11-24T06:11:36.003 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-24T06:11:36.008 INFO:tasks.ceph.mds.c:Stopped 2023-11-24T06:11:36.008 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-24T06:11:36.014 INFO:tasks.ceph.mds.b:Stopped 2023-11-24T06:11:36.014 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-24T06:11:36.016 INFO:tasks.ceph.mds.d:Stopped 2023-11-24T06:11:36.016 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-11-24T06:11:36.056 INFO:tasks.ceph.mds.a:Started 2023-11-24T06:11:36.056 DEBUG:teuthology.parallel:result is None 2023-11-24T06:11:36.063 INFO:tasks.ceph.mds.c:Started 2023-11-24T06:11:36.064 DEBUG:teuthology.parallel:result is None 2023-11-24T06:11:36.064 INFO:tasks.ceph.mds.b:Started 2023-11-24T06:11:36.065 DEBUG:teuthology.parallel:result is None 2023-11-24T06:11:36.072 INFO:tasks.ceph.mds.d:Started 2023-11-24T06:11:36.072 DEBUG:teuthology.parallel:result is None 2023-11-24T06:11:36.072 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-11-24T06:11:36.227 INFO:tasks.ceph.mds.a.smithi092.stdout:starting mds.a at 2023-11-24T06:11:36.228 INFO:tasks.ceph.mds.c.smithi092.stdout:starting mds.c at 2023-11-24T06:11:36.233 INFO:tasks.ceph.mds.b.smithi118.stdout:starting mds.b at 2023-11-24T06:11:36.233 INFO:tasks.ceph.mds.d.smithi118.stdout:starting mds.d at 2023-11-24T06:11:36.273 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:36.269+0000 7fc8536d4640 1 Processor -- start 2023-11-24T06:11:36.273 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:36.270+0000 7fc8536d4640 1 -- start start 2023-11-24T06:11:36.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:36.271+0000 7fc8536d4640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fc84c0ae5a0 0x7fc84c15e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:36.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:36.271+0000 7fc8536d4640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc84c0cf290 0x7fc84c0cf720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:36.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:36.271+0000 7fc8536d4640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc84c070c40 con 0x7fc84c0b3ae0 2023-11-24T06:11:36.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:36.271+0000 7fc8536d4640 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc84c071450 con 0x7fc84c0ae5a0 2023-11-24T06:11:36.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:36.271+0000 7fc8536d4640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc84c0cfc60 con 0x7fc84c0cf290 2023-11-24T06:11:36.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:36.271+0000 7fc850c6f640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fc84c0b3ae0 0x7fc84c06cbe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:35420/0 (socket says 172.21.15.92:35420) 2023-11-24T06:11:36.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:36.271+0000 7fc850c6f640 1 -- 172.21.15.92:0/587731436 learned_addr learned my addr 172.21.15.92:0/587731436 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-24T06:11:36.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:36.271+0000 7fc851470640 1 --2- 172.21.15.92:0/587731436 >> v2:172.21.15.118:3300/0 conn(0x7fc84c0ae5a0 0x7fc84c15e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:36.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:36.271+0000 7fc851c71640 1 --2- 172.21.15.92:0/587731436 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc84c0cf290 0x7fc84c0cf720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:36.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:36.272+0000 7fc851470640 1 -- 172.21.15.92:0/587731436 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc84c0cf290 msgr2=0x7fc84c0cf720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:36.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:36.272+0000 7fc851470640 1 --2- 172.21.15.92:0/587731436 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc84c0cf290 0x7fc84c0cf720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:36.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:36.272+0000 7fc851470640 1 -- 172.21.15.92:0/587731436 >> v1:172.21.15.92:6789/0 conn(0x7fc84c0b3ae0 legacy=0x7fc84c06cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:36.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:36.272+0000 7fc851470640 1 -- 172.21.15.92:0/587731436 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc84c06c470 con 0x7fc84c0ae5a0 2023-11-24T06:11:36.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:36.273+0000 7fc851470640 1 --2- 172.21.15.92:0/587731436 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc84c0ae5a0 0x7fc84c15e430 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fc8480097a0 tx=0x7fc84802b940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5f91b2992e839580 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:36.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:36.273+0000 7fc843fff640 1 -- 172.21.15.92:0/587731436 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc848030800 con 0x7fc84c0ae5a0 2023-11-24T06:11:36.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:36.273+0000 7fc843fff640 1 -- 172.21.15.92:0/587731436 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc848030960 con 0x7fc84c0ae5a0 2023-11-24T06:11:36.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:36.273+0000 7fc843fff640 1 -- 172.21.15.92:0/587731436 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc848030cf0 con 0x7fc84c0ae5a0 2023-11-24T06:11:36.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:36.273+0000 7fc8536d4640 1 -- 172.21.15.92:0/587731436 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc84c0ae5a0 msgr2=0x7fc84c15e430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:36.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:36.273+0000 7fc8536d4640 1 --2- 172.21.15.92:0/587731436 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc84c0ae5a0 0x7fc84c15e430 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fc8480097a0 tx=0x7fc84802b940 comp rx=0 tx=0).stop 2023-11-24T06:11:36.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:36.274+0000 7fc8536d4640 1 -- 172.21.15.92:0/587731436 shutdown_connections 2023-11-24T06:11:36.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:36.274+0000 7fc8536d4640 1 --2- 172.21.15.92:0/587731436 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc84c0cf290 0x7fc84c0cf720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:36.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:36.274+0000 7fc8536d4640 1 --2- 172.21.15.92:0/587731436 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc84c0ae5a0 0x7fc84c15e430 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:36.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:36.274+0000 7fc8536d4640 1 -- 172.21.15.92:0/587731436 >> 172.21.15.92:0/587731436 conn(0x7fc84c06a800 msgr2=0x7fc84c06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:36.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:36.274+0000 7fc8536d4640 1 -- 172.21.15.92:0/587731436 shutdown_connections 2023-11-24T06:11:36.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:36.274+0000 7fc8536d4640 1 -- 172.21.15.92:0/587731436 wait complete. 2023-11-24T06:11:36.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:36.275+0000 7fc8536d4640 1 Processor -- start 2023-11-24T06:11:36.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:36.276+0000 7fc8536d4640 1 -- start start 2023-11-24T06:11:36.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:36.276+0000 7fc8536d4640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc84c0ae5a0 0x7fc84c1611a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:36.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:36.276+0000 7fc8536d4640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc84c0b3ae0 0x7fc84c15eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:36.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:36.276+0000 7fc8536d4640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc84c0cf290 0x7fc84c15f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:36.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:36.276+0000 7fc8536d4640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc84c1603b0 con 0x7fc84c0cf290 2023-11-24T06:11:36.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:36.276+0000 7fc8536d4640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc84c1531b0 con 0x7fc84c0ae5a0 2023-11-24T06:11:36.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:36.276+0000 7fc8536d4640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc84c15f730 con 0x7fc84c0b3ae0 2023-11-24T06:11:36.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:36.277+0000 7fc851c71640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc84c0cf290 0x7fc84c15f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:36.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:36.277+0000 7fc851c71640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc84c0cf290 0x7fc84c15f090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:57976/0 (socket says 172.21.15.92:57976) 2023-11-24T06:11:36.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:36.277+0000 7fc851c71640 1 -- 172.21.15.92:0/3187842926 learned_addr learned my addr 172.21.15.92:0/3187842926 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:11:36.282 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:36.277+0000 7fc850c6f640 1 --2- 172.21.15.92:0/3187842926 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc84c0b3ae0 0x7fc84c15eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:36.282 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:36.277+0000 7fc851470640 1 --2- 172.21.15.92:0/3187842926 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc84c0ae5a0 0x7fc84c1611a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:36.282 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:36.277+0000 7fc851c71640 1 -- 172.21.15.92:0/3187842926 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc84c0b3ae0 msgr2=0x7fc84c15eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:36.282 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:36.277+0000 7fc851c71640 1 --2- 172.21.15.92:0/3187842926 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc84c0b3ae0 0x7fc84c15eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:36.282 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:36.277+0000 7fc851c71640 1 -- 172.21.15.92:0/3187842926 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc84c0ae5a0 msgr2=0x7fc84c1611a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:36.282 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:36.277+0000 7fc851c71640 1 --2- 172.21.15.92:0/3187842926 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc84c0ae5a0 0x7fc84c1611a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:36.282 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:36.277+0000 7fc851c71640 1 -- 172.21.15.92:0/3187842926 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc848009400 con 0x7fc84c0cf290 2023-11-24T06:11:36.282 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:36.278+0000 7fc851c71640 1 --2- 172.21.15.92:0/3187842926 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc84c0cf290 0x7fc84c15f090 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fc84400efc0 tx=0x7fc84400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:36.282 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:36.278+0000 7fc841ffb640 1 -- 172.21.15.92:0/3187842926 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc844009ba0 con 0x7fc84c0cf290 2023-11-24T06:11:36.282 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:36.278+0000 7fc841ffb640 1 -- 172.21.15.92:0/3187842926 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc84400f040 con 0x7fc84c0cf290 2023-11-24T06:11:36.282 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:36.278+0000 7fc8536d4640 1 -- 172.21.15.92:0/3187842926 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc84c15fa10 con 0x7fc84c0cf290 2023-11-24T06:11:36.282 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:36.278+0000 7fc841ffb640 1 -- 172.21.15.92:0/3187842926 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8440146c0 con 0x7fc84c0cf290 2023-11-24T06:11:36.283 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:36.278+0000 7fc8536d4640 1 -- 172.21.15.92:0/3187842926 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc84c15ffd0 con 0x7fc84c0cf290 2023-11-24T06:11:36.283 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:36.279+0000 7fc8536d4640 1 -- 172.21.15.92:0/3187842926 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc84c0675d0 con 0x7fc84c0cf290 2023-11-24T06:11:36.283 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:36.280+0000 7fc841ffb640 1 -- 172.21.15.92:0/3187842926 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc844014820 con 0x7fc84c0cf290 2023-11-24T06:11:36.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:36.281+0000 7fc841ffb640 1 --2- 172.21.15.92:0/3187842926 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fc83006ae40 0x7fc83006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:36.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:36.281+0000 7fc841ffb640 1 -- 172.21.15.92:0/3187842926 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fc844089fc0 con 0x7fc84c0cf290 2023-11-24T06:11:36.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:36.281+0000 7fc851470640 1 --2- 172.21.15.92:0/3187842926 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fc83006ae40 0x7fc83006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:36.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:36.282+0000 7fc851470640 1 --2- 172.21.15.92:0/3187842926 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fc83006ae40 0x7fc83006d300 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fc848036000 tx=0x7fc848008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:36.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:36.285+0000 7fc841ffb640 1 -- 172.21.15.92:0/3187842926 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc844055410 con 0x7fc84c0cf290 2023-11-24T06:11:36.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:36.486+0000 7fc8536d4640 1 -- 172.21.15.92:0/3187842926 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fc84c0cb620 con 0x7fc84c0cf290 2023-11-24T06:11:36.990 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:36.987+0000 7fc841ffb640 1 -- 172.21.15.92:0/3187842926 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v27) v1 ==== 97+0+0 (secure 0 0 0) 0x7fc844058a30 con 0x7fc84c0cf290 2023-11-24T06:11:36.995 INFO:teuthology.orchestra.run.smithi092.stderr: removed all blocklist entries 2023-11-24T06:11:36.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:36.990+0000 7fc8536d4640 1 -- 172.21.15.92:0/3187842926 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fc83006ae40 msgr2=0x7fc83006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:36.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:36.990+0000 7fc8536d4640 1 --2- 172.21.15.92:0/3187842926 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fc83006ae40 0x7fc83006d300 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fc848036000 tx=0x7fc848008030 comp rx=0 tx=0).stop 2023-11-24T06:11:36.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:36.990+0000 7fc8536d4640 1 -- 172.21.15.92:0/3187842926 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc84c0cf290 msgr2=0x7fc84c15f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:36.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:36.990+0000 7fc8536d4640 1 --2- 172.21.15.92:0/3187842926 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc84c0cf290 0x7fc84c15f090 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fc84400efc0 tx=0x7fc84400c490 comp rx=0 tx=0).stop 2023-11-24T06:11:36.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:36.991+0000 7fc8536d4640 1 -- 172.21.15.92:0/3187842926 shutdown_connections 2023-11-24T06:11:36.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:36.991+0000 7fc8536d4640 1 --2- 172.21.15.92:0/3187842926 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fc83006ae40 0x7fc83006d300 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:36.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:36.991+0000 7fc8536d4640 1 --2- 172.21.15.92:0/3187842926 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc84c0cf290 0x7fc84c15f090 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:36.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:36.991+0000 7fc8536d4640 1 --2- 172.21.15.92:0/3187842926 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc84c0b3ae0 0x7fc84c15eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:36.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:36.991+0000 7fc8536d4640 1 --2- 172.21.15.92:0/3187842926 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc84c0ae5a0 0x7fc84c1611a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:36.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:36.991+0000 7fc8536d4640 1 -- 172.21.15.92:0/3187842926 >> 172.21.15.92:0/3187842926 conn(0x7fc84c06a800 msgr2=0x7fc84c0b2530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:36.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:36.991+0000 7fc8536d4640 1 -- 172.21.15.92:0/3187842926 shutdown_connections 2023-11-24T06:11:36.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:36.991+0000 7fc8536d4640 1 -- 172.21.15.92:0/3187842926 wait complete. 2023-11-24T06:11:37.015 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-11-24T06:11:37.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:37.213+0000 7fc9e8bcb640 1 Processor -- start 2023-11-24T06:11:37.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:37.213+0000 7fc9e8bcb640 1 -- start start 2023-11-24T06:11:37.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:37.214+0000 7fc9e8bcb640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc9e413cda0 0x7fc9e415da60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:37.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:37.215+0000 7fc9e8bcb640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fc9e40ce0c0 0x7fc9e4070b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:37.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:37.215+0000 7fc9e8bcb640 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9e40700f0 con 0x7fc9e415dfa0 2023-11-24T06:11:37.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:37.215+0000 7fc9e8bcb640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9e415e3e0 con 0x7fc9e40ce0c0 2023-11-24T06:11:37.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:37.215+0000 7fc9e8bcb640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9e40ce720 con 0x7fc9e413cda0 2023-11-24T06:11:37.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:37.215+0000 7fc9e3fff640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fc9e40ce0c0 0x7fc9e4070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:37.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:37.215+0000 7fc9e3fff640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fc9e40ce0c0 0x7fc9e4070b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:57992/0 (socket says 172.21.15.92:57992) 2023-11-24T06:11:37.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:37.215+0000 7fc9e3fff640 1 -- 172.21.15.92:0/2810583095 learned_addr learned my addr 172.21.15.92:0/2810583095 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:11:37.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:37.215+0000 7fc9e37fe640 1 --2- 172.21.15.92:0/2810583095 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc9e413cda0 0x7fc9e415da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:37.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:37.215+0000 7fc9e3fff640 1 -- 172.21.15.92:0/2810583095 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc9e413cda0 msgr2=0x7fc9e415da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:37.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:37.215+0000 7fc9e3fff640 1 --2- 172.21.15.92:0/2810583095 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc9e413cda0 0x7fc9e415da60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:37.220 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:37.215+0000 7fc9e3fff640 1 -- 172.21.15.92:0/2810583095 >> v1:172.21.15.118:6789/0 conn(0x7fc9e415dfa0 legacy=0x7fc9e40cd9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:37.220 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:37.215+0000 7fc9e3fff640 1 -- 172.21.15.92:0/2810583095 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9e40cea00 con 0x7fc9e40ce0c0 2023-11-24T06:11:37.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:37.216+0000 7fc9e3fff640 1 --2- 172.21.15.92:0/2810583095 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc9e40ce0c0 0x7fc9e4070b40 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fc9dc009e50 tx=0x7fc9dc02be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5dc120bfed971e44 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:37.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:37.217+0000 7fc9e2ffd640 1 -- 172.21.15.92:0/2810583095 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9dc02c9e0 con 0x7fc9e40ce0c0 2023-11-24T06:11:37.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:37.217+0000 7fc9e2ffd640 1 -- 172.21.15.92:0/2810583095 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9dc02cb40 con 0x7fc9e40ce0c0 2023-11-24T06:11:37.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:37.217+0000 7fc9e2ffd640 1 -- 172.21.15.92:0/2810583095 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9dc0343f0 con 0x7fc9e40ce0c0 2023-11-24T06:11:37.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:37.217+0000 7fc9e8bcb640 1 -- 172.21.15.92:0/2810583095 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc9e40ce0c0 msgr2=0x7fc9e4070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:37.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:37.217+0000 7fc9e8bcb640 1 --2- 172.21.15.92:0/2810583095 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc9e40ce0c0 0x7fc9e4070b40 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fc9dc009e50 tx=0x7fc9dc02be40 comp rx=0 tx=0).stop 2023-11-24T06:11:37.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:37.217+0000 7fc9e8bcb640 1 -- 172.21.15.92:0/2810583095 shutdown_connections 2023-11-24T06:11:37.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:37.217+0000 7fc9e8bcb640 1 --2- 172.21.15.92:0/2810583095 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc9e40ce0c0 0x7fc9e4070b40 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:37.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:37.217+0000 7fc9e8bcb640 1 --2- 172.21.15.92:0/2810583095 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc9e413cda0 0x7fc9e415da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:37.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:37.217+0000 7fc9e8bcb640 1 -- 172.21.15.92:0/2810583095 >> 172.21.15.92:0/2810583095 conn(0x7fc9e406a3e0 msgr2=0x7fc9e406a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:37.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:37.217+0000 7fc9e8bcb640 1 -- 172.21.15.92:0/2810583095 shutdown_connections 2023-11-24T06:11:37.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:37.218+0000 7fc9e8bcb640 1 -- 172.21.15.92:0/2810583095 wait complete. 2023-11-24T06:11:37.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:37.219+0000 7fc9e8bcb640 1 Processor -- start 2023-11-24T06:11:37.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:37.219+0000 7fc9e8bcb640 1 -- start start 2023-11-24T06:11:37.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:37.220+0000 7fc9e8bcb640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc9e40ce0c0 0x7fc9e4150490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:37.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:37.220+0000 7fc9e8bcb640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc9e413cda0 0x7fc9e40acd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:37.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:37.220+0000 7fc9e8bcb640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc9e415dfa0 0x7fc9e413b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:37.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:37.220+0000 7fc9e8bcb640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9e41509d0 con 0x7fc9e413cda0 2023-11-24T06:11:37.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:37.220+0000 7fc9e8bcb640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9e40b2430 con 0x7fc9e40ce0c0 2023-11-24T06:11:37.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:37.220+0000 7fc9e8bcb640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9e40b2710 con 0x7fc9e415dfa0 2023-11-24T06:11:37.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:37.221+0000 7fc9dbfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc9e413cda0 0x7fc9e40acd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:37.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:37.221+0000 7fc9dbfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc9e413cda0 0x7fc9e40acd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:57998/0 (socket says 172.21.15.92:57998) 2023-11-24T06:11:37.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:37.221+0000 7fc9dbfff640 1 -- 172.21.15.92:0/1019870429 learned_addr learned my addr 172.21.15.92:0/1019870429 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:11:37.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:37.221+0000 7fc9e37fe640 1 --2- 172.21.15.92:0/1019870429 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc9e40ce0c0 0x7fc9e4150490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:37.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:37.221+0000 7fc9e3fff640 1 --2- 172.21.15.92:0/1019870429 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc9e415dfa0 0x7fc9e413b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:37.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:37.221+0000 7fc9dbfff640 1 -- 172.21.15.92:0/1019870429 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc9e415dfa0 msgr2=0x7fc9e413b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:37.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:37.221+0000 7fc9dbfff640 1 --2- 172.21.15.92:0/1019870429 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc9e415dfa0 0x7fc9e413b720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:37.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:37.221+0000 7fc9dbfff640 1 -- 172.21.15.92:0/1019870429 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc9e40ce0c0 msgr2=0x7fc9e4150490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:37.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:37.221+0000 7fc9dbfff640 1 --2- 172.21.15.92:0/1019870429 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc9e40ce0c0 0x7fc9e4150490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:37.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:37.221+0000 7fc9dbfff640 1 -- 172.21.15.92:0/1019870429 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9dc009b30 con 0x7fc9e413cda0 2023-11-24T06:11:37.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:37.222+0000 7fc9dbfff640 1 --2- 172.21.15.92:0/1019870429 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc9e413cda0 0x7fc9e40acd10 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fc9cc00c430 tx=0x7fc9cc00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:37.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:37.222+0000 7fc9e0ff9640 1 -- 172.21.15.92:0/1019870429 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9cc011790 con 0x7fc9e413cda0 2023-11-24T06:11:37.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:37.222+0000 7fc9e0ff9640 1 -- 172.21.15.92:0/1019870429 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9cc0118f0 con 0x7fc9e413cda0 2023-11-24T06:11:37.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:37.222+0000 7fc9e0ff9640 1 -- 172.21.15.92:0/1019870429 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9cc011ce0 con 0x7fc9e413cda0 2023-11-24T06:11:37.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:37.222+0000 7fc9e8bcb640 1 -- 172.21.15.92:0/1019870429 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9e40b29f0 con 0x7fc9e413cda0 2023-11-24T06:11:37.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:37.222+0000 7fc9e8bcb640 1 -- 172.21.15.92:0/1019870429 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9e40b2fb0 con 0x7fc9e413cda0 2023-11-24T06:11:37.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:37.223+0000 7fc9e8bcb640 1 -- 172.21.15.92:0/1019870429 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9e40b3170 con 0x7fc9e413cda0 2023-11-24T06:11:37.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:37.224+0000 7fc9e0ff9640 1 -- 172.21.15.92:0/1019870429 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc9cc01c460 con 0x7fc9e413cda0 2023-11-24T06:11:37.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:37.225+0000 7fc9e0ff9640 1 --2- 172.21.15.92:0/1019870429 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fc9c406ae40 0x7fc9c406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:37.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:37.225+0000 7fc9e0ff9640 1 -- 172.21.15.92:0/1019870429 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fc9cc089320 con 0x7fc9e413cda0 2023-11-24T06:11:37.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:37.225+0000 7fc9e37fe640 1 --2- 172.21.15.92:0/1019870429 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fc9c406ae40 0x7fc9c406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:37.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:37.226+0000 7fc9e37fe640 1 --2- 172.21.15.92:0/1019870429 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fc9c406ae40 0x7fc9c406d300 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fc9d4006fd0 tx=0x7fc9d4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:37.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:37.229+0000 7fc9e0ff9640 1 -- 172.21.15.92:0/1019870429 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc9cc054680 con 0x7fc9e413cda0 2023-11-24T06:11:37.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:37.465+0000 7fc9e8bcb640 1 -- 172.21.15.92:0/1019870429 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fc9e4067450 con 0x7fc9e413cda0 2023-11-24T06:11:37.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:37.466+0000 7fc9e0ff9640 1 -- 172.21.15.92:0/1019870429 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v2) v1 ==== 80+0+6645 (secure 0 0 0) 0x7fc9cc057ca0 con 0x7fc9e413cda0 2023-11-24T06:11:37.470 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-24T06:11:37.470 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-24T06:11:37.470 INFO:teuthology.orchestra.run.smithi092.stdout: "auth_dump": [ 2023-11-24T06:11:37.470 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-24T06:11:37.470 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "mds.a", 2023-11-24T06:11:37.470 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQAQPmBl501xBxAAEtXTFSRjPeIoKJpcPILQwA==", 2023-11-24T06:11:37.470 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-11-24T06:11:37.470 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-11-24T06:11:37.471 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow *", 2023-11-24T06:11:37.471 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow *", 2023-11-24T06:11:37.471 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-11-24T06:11:37.471 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-24T06:11:37.471 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-24T06:11:37.471 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-24T06:11:37.471 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "mds.b", 2023-11-24T06:11:37.471 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQAQPmBlaY/AKhAArTolS+ALdzAB8M0lQ3/Mjg==", 2023-11-24T06:11:37.471 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-11-24T06:11:37.471 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-11-24T06:11:37.471 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow *", 2023-11-24T06:11:37.471 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow *", 2023-11-24T06:11:37.472 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-11-24T06:11:37.472 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-24T06:11:37.472 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-24T06:11:37.472 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-24T06:11:37.472 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "mds.c", 2023-11-24T06:11:37.472 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQAQPmBl4VbbGBAAkJwrHXAC4d5S2hdPIcE/Cg==", 2023-11-24T06:11:37.472 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-11-24T06:11:37.472 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-11-24T06:11:37.472 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow *", 2023-11-24T06:11:37.472 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow *", 2023-11-24T06:11:37.473 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-11-24T06:11:37.473 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-24T06:11:37.473 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-24T06:11:37.473 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-24T06:11:37.473 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "mds.d", 2023-11-24T06:11:37.473 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQARPmBl00MmABAAvqEvZ0bL8D1oHg4gcKXEyA==", 2023-11-24T06:11:37.473 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-11-24T06:11:37.473 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-11-24T06:11:37.473 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow *", 2023-11-24T06:11:37.473 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow *", 2023-11-24T06:11:37.473 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-11-24T06:11:37.473 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-24T06:11:37.473 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-24T06:11:37.474 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-24T06:11:37.474 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.0", 2023-11-24T06:11:37.474 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQAVPmBlr+4mLRAAgsX9kvv5frsFc+E1VoO8Fw==", 2023-11-24T06:11:37.474 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-11-24T06:11:37.474 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-11-24T06:11:37.474 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-11-24T06:11:37.474 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-11-24T06:11:37.474 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-24T06:11:37.474 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-24T06:11:37.474 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-24T06:11:37.474 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.1", 2023-11-24T06:11:37.474 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQAZPmBl7IUIChAA/DL645CltqHdzRS8r4Lksg==", 2023-11-24T06:11:37.475 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-11-24T06:11:37.475 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-11-24T06:11:37.475 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-11-24T06:11:37.475 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-11-24T06:11:37.475 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-24T06:11:37.475 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-24T06:11:37.475 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-24T06:11:37.475 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.2", 2023-11-24T06:11:37.475 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQAcPmBlJ5VbIhAAEnc+QX3whpwM0Ok9WD8dcA==", 2023-11-24T06:11:37.475 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-11-24T06:11:37.475 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-11-24T06:11:37.476 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-11-24T06:11:37.476 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-11-24T06:11:37.476 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-24T06:11:37.476 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-24T06:11:37.476 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-24T06:11:37.476 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.3", 2023-11-24T06:11:37.476 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQAfPmBlKdS9KxAAbdaBTd1CEf/A6RyKxc9t6A==", 2023-11-24T06:11:37.476 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-11-24T06:11:37.476 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-11-24T06:11:37.476 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-11-24T06:11:37.476 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-11-24T06:11:37.476 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-24T06:11:37.477 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-24T06:11:37.477 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-24T06:11:37.477 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.4", 2023-11-24T06:11:37.477 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQAkPmBlnOJ6KxAAuLTyFpOBStxC5FevOc4pKw==", 2023-11-24T06:11:37.477 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-11-24T06:11:37.477 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-11-24T06:11:37.477 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-11-24T06:11:37.477 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-11-24T06:11:37.477 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-24T06:11:37.477 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-24T06:11:37.477 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-24T06:11:37.477 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.5", 2023-11-24T06:11:37.477 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQAoPmBlrkDXCBAAwv3u79yJeeTjreYWfr/yvQ==", 2023-11-24T06:11:37.478 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-11-24T06:11:37.478 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-11-24T06:11:37.478 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-11-24T06:11:37.478 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-11-24T06:11:37.478 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-24T06:11:37.478 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-24T06:11:37.478 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-24T06:11:37.478 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.6", 2023-11-24T06:11:37.479 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQArPmBlP1AoIRAA8D8nUCwXZRHZmB9d73fYbQ==", 2023-11-24T06:11:37.479 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-11-24T06:11:37.479 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-11-24T06:11:37.479 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-11-24T06:11:37.479 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-11-24T06:11:37.479 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-24T06:11:37.479 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-24T06:11:37.479 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-24T06:11:37.479 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.7", 2023-11-24T06:11:37.479 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQAuPmBlUBKMKRAA3obYjr9s13SuYnbFt+Y0bQ==", 2023-11-24T06:11:37.479 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-11-24T06:11:37.479 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-11-24T06:11:37.479 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-11-24T06:11:37.480 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-11-24T06:11:37.480 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-24T06:11:37.480 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-24T06:11:37.480 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-24T06:11:37.480 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "client.0", 2023-11-24T06:11:37.480 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQARPmBlGMUsDhAAlgrJvA/TT5hpUYd5o9OzIQ==", 2023-11-24T06:11:37.480 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-11-24T06:11:37.480 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-11-24T06:11:37.480 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow r", 2023-11-24T06:11:37.480 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow rw", 2023-11-24T06:11:37.480 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow rwx" 2023-11-24T06:11:37.480 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-24T06:11:37.480 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-24T06:11:37.481 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-24T06:11:37.481 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "client.1", 2023-11-24T06:11:37.481 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQARPmBlOeviIxAA/egv9DYEDJ+1yKDFbsY1pw==", 2023-11-24T06:11:37.481 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-11-24T06:11:37.481 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-11-24T06:11:37.481 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow r", 2023-11-24T06:11:37.481 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow rw", 2023-11-24T06:11:37.481 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow rwx" 2023-11-24T06:11:37.481 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-24T06:11:37.481 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-24T06:11:37.481 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-24T06:11:37.481 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "client.2", 2023-11-24T06:11:37.481 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQARPmBlIqJ9GBAA46UmMl1hl/522waZSndXhw==", 2023-11-24T06:11:37.482 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-11-24T06:11:37.482 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-11-24T06:11:37.482 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow r", 2023-11-24T06:11:37.482 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow rw", 2023-11-24T06:11:37.482 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow rwx" 2023-11-24T06:11:37.482 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-24T06:11:37.482 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-24T06:11:37.482 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-24T06:11:37.482 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "client.3", 2023-11-24T06:11:37.482 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQARPmBlQRCDLhAA80Oc0yKHPMD5EjVxqurfPw==", 2023-11-24T06:11:37.482 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-11-24T06:11:37.482 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-11-24T06:11:37.483 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow r", 2023-11-24T06:11:37.483 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow rw", 2023-11-24T06:11:37.483 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow rwx" 2023-11-24T06:11:37.483 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-24T06:11:37.483 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-24T06:11:37.483 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-24T06:11:37.483 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "client.admin", 2023-11-24T06:11:37.483 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQAOPmBl/8h5MxAAs//puGUeKqNyBdlTaWY/1w==", 2023-11-24T06:11:37.484 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-11-24T06:11:37.484 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow *", 2023-11-24T06:11:37.484 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow *", 2023-11-24T06:11:37.484 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow *", 2023-11-24T06:11:37.484 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-11-24T06:11:37.484 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-24T06:11:37.484 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-24T06:11:37.484 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-24T06:11:37.485 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "client.bootstrap-mds", 2023-11-24T06:11:37.485 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQA8PmBla0BEKhAAH03WDOFEYRBNn6l1r9X50g==", 2023-11-24T06:11:37.485 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-11-24T06:11:37.485 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile bootstrap-mds" 2023-11-24T06:11:37.485 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-24T06:11:37.485 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-24T06:11:37.485 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-24T06:11:37.485 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "client.bootstrap-mgr", 2023-11-24T06:11:37.485 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQA8PmBluF9EKhAAfVudH5hWKms3xKN5l6PCNQ==", 2023-11-24T06:11:37.486 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-11-24T06:11:37.486 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile bootstrap-mgr" 2023-11-24T06:11:37.486 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-24T06:11:37.486 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-24T06:11:37.486 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-24T06:11:37.486 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "client.bootstrap-osd", 2023-11-24T06:11:37.486 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQA8PmBlg3lEKhAAPi7KGdbUkzIGcuCl16BMVg==", 2023-11-24T06:11:37.486 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-11-24T06:11:37.486 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile bootstrap-osd" 2023-11-24T06:11:37.486 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-24T06:11:37.487 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-24T06:11:37.487 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-24T06:11:37.487 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "client.bootstrap-rbd", 2023-11-24T06:11:37.487 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQA8PmBlApNEKhAAwPGE3p++yAFfpnSkGEzOGA==", 2023-11-24T06:11:37.487 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-11-24T06:11:37.487 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile bootstrap-rbd" 2023-11-24T06:11:37.487 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-24T06:11:37.487 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-24T06:11:37.487 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-24T06:11:37.487 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "client.bootstrap-rbd-mirror", 2023-11-24T06:11:37.487 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQA8PmBldqxEKhAAO0ufQrITw/4a04x4C9G5PA==", 2023-11-24T06:11:37.487 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-11-24T06:11:37.488 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2023-11-24T06:11:37.488 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-24T06:11:37.488 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-24T06:11:37.488 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-24T06:11:37.488 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "client.bootstrap-rgw", 2023-11-24T06:11:37.488 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQA8PmBlxsVEKhAAYMFVwLu6WJS1n+AKLUwEKQ==", 2023-11-24T06:11:37.488 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-11-24T06:11:37.488 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile bootstrap-rgw" 2023-11-24T06:11:37.488 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-24T06:11:37.488 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-24T06:11:37.488 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-24T06:11:37.488 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "mgr.x", 2023-11-24T06:11:37.488 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQAPPmBl1LwlNxAAsk8YjJh9jW0OCcGdfnBA1w==", 2023-11-24T06:11:37.489 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-11-24T06:11:37.489 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow *", 2023-11-24T06:11:37.489 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile mgr", 2023-11-24T06:11:37.489 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-11-24T06:11:37.489 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-24T06:11:37.489 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-24T06:11:37.489 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-24T06:11:37.489 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "mgr.y", 2023-11-24T06:11:37.489 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQAPPmBlyGxUKhAAWOTBKusZ4JOcJ1SaHJ6ZmQ==", 2023-11-24T06:11:37.489 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-11-24T06:11:37.489 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow *", 2023-11-24T06:11:37.489 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile mgr", 2023-11-24T06:11:37.490 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-11-24T06:11:37.490 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-24T06:11:37.490 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-24T06:11:37.490 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-24T06:11:37.490 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-24T06:11:37.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:37.469+0000 7fc9e8bcb640 1 -- 172.21.15.92:0/1019870429 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fc9c406ae40 msgr2=0x7fc9c406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:37.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:37.469+0000 7fc9e8bcb640 1 --2- 172.21.15.92:0/1019870429 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fc9c406ae40 0x7fc9c406d300 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fc9d4006fd0 tx=0x7fc9d4008040 comp rx=0 tx=0).stop 2023-11-24T06:11:37.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:37.469+0000 7fc9e8bcb640 1 -- 172.21.15.92:0/1019870429 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc9e413cda0 msgr2=0x7fc9e40acd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:37.492 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:37.469+0000 7fc9e8bcb640 1 --2- 172.21.15.92:0/1019870429 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc9e413cda0 0x7fc9e40acd10 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fc9cc00c430 tx=0x7fc9cc00c8f0 comp rx=0 tx=0).stop 2023-11-24T06:11:37.492 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:37.469+0000 7fc9e8bcb640 1 -- 172.21.15.92:0/1019870429 shutdown_connections 2023-11-24T06:11:37.492 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:37.469+0000 7fc9e8bcb640 1 --2- 172.21.15.92:0/1019870429 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fc9c406ae40 0x7fc9c406d300 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:37.492 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:37.469+0000 7fc9e8bcb640 1 --2- 172.21.15.92:0/1019870429 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc9e415dfa0 0x7fc9e413b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:37.492 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:37.469+0000 7fc9e8bcb640 1 --2- 172.21.15.92:0/1019870429 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc9e413cda0 0x7fc9e40acd10 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:37.492 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:37.469+0000 7fc9e8bcb640 1 --2- 172.21.15.92:0/1019870429 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc9e40ce0c0 0x7fc9e4150490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:37.492 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:37.469+0000 7fc9e8bcb640 1 -- 172.21.15.92:0/1019870429 >> 172.21.15.92:0/1019870429 conn(0x7fc9e406a3e0 msgr2=0x7fc9e40b02a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:37.492 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:37.469+0000 7fc9e8bcb640 1 -- 172.21.15.92:0/1019870429 shutdown_connections 2023-11-24T06:11:37.492 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:37.469+0000 7fc9e8bcb640 1 -- 172.21.15.92:0/1019870429 wait complete. 2023-11-24T06:11:37.493 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2023-11-24T06:11:37.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:37.700+0000 7f9258bfb640 1 Processor -- start 2023-11-24T06:11:37.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:37.700+0000 7f9258bfb640 1 -- start start 2023-11-24T06:11:37.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:37.702+0000 7f9258bfb640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f925413df80 0x7f925415bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:37.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:37.702+0000 7f9258bfb640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f925413e360 0x7f92540cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:37.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:37.702+0000 7f9258bfb640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f925415f9c0 0x7f92540cf7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:37.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:37.702+0000 7f9258bfb640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f925406c470 con 0x7f925413df80 2023-11-24T06:11:37.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:37.702+0000 7f9258bfb640 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f925406c750 con 0x7f925413e360 2023-11-24T06:11:37.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:37.702+0000 7f9258bfb640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f925406ca30 con 0x7f925415f9c0 2023-11-24T06:11:37.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:37.702+0000 7f925259c640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f925413df80 0x7f925415bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:37.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:37.702+0000 7f925259c640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f925413df80 0x7f925415bee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58010/0 (socket says 172.21.15.92:58010) 2023-11-24T06:11:37.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:37.702+0000 7f925259c640 1 -- 172.21.15.92:0/961208061 learned_addr learned my addr 172.21.15.92:0/961208061 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:11:37.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:37.702+0000 7f9251d9b640 1 --2- 172.21.15.92:0/961208061 >> v2:172.21.15.118:3300/0 conn(0x7f925413e360 0x7f92540cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:37.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:37.702+0000 7f9252d9d640 1 --2- 172.21.15.92:0/961208061 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f925415f9c0 0x7f92540cf7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:37.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:37.702+0000 7f925259c640 1 -- 172.21.15.92:0/961208061 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f925415f9c0 msgr2=0x7f92540cf7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:37.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:37.702+0000 7f925259c640 1 --2- 172.21.15.92:0/961208061 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f925415f9c0 0x7f92540cf7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:37.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:37.702+0000 7f925259c640 1 -- 172.21.15.92:0/961208061 >> v2:172.21.15.118:3300/0 conn(0x7f925413e360 msgr2=0x7f92540cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:37.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:37.702+0000 7f925259c640 1 --2- 172.21.15.92:0/961208061 >> v2:172.21.15.118:3300/0 conn(0x7f925413e360 0x7f92540cf290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:37.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:37.702+0000 7f925259c640 1 -- 172.21.15.92:0/961208061 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f925406cd10 con 0x7f925413df80 2023-11-24T06:11:37.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:37.703+0000 7f9251d9b640 1 --2- 172.21.15.92:0/961208061 >> v2:172.21.15.118:3300/0 conn(0x7f925413e360 0x7f92540cf290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:11:37.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:37.704+0000 7f925259c640 1 --2- 172.21.15.92:0/961208061 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f925413df80 0x7f925415bee0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f925406ded0 tx=0x7f924402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92242e42c224ced server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:37.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:37.704+0000 7f925159a640 1 -- 172.21.15.92:0/961208061 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9244031870 con 0x7f925413df80 2023-11-24T06:11:37.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:37.704+0000 7f925159a640 1 -- 172.21.15.92:0/961208061 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92440319d0 con 0x7f925413df80 2023-11-24T06:11:37.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:37.704+0000 7f925159a640 1 -- 172.21.15.92:0/961208061 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9244031d60 con 0x7f925413df80 2023-11-24T06:11:37.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:37.705+0000 7f9258bfb640 1 -- 172.21.15.92:0/961208061 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f925413df80 msgr2=0x7f925415bee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:37.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:37.705+0000 7f9258bfb640 1 --2- 172.21.15.92:0/961208061 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f925413df80 0x7f925415bee0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f925406ded0 tx=0x7f924402b910 comp rx=0 tx=0).stop 2023-11-24T06:11:37.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:37.705+0000 7f9258bfb640 1 -- 172.21.15.92:0/961208061 shutdown_connections 2023-11-24T06:11:37.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:37.705+0000 7f9258bfb640 1 --2- 172.21.15.92:0/961208061 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f925415f9c0 0x7f92540cf7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:37.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:37.705+0000 7f9258bfb640 1 --2- 172.21.15.92:0/961208061 >> v2:172.21.15.118:3300/0 conn(0x7f925413e360 0x7f92540cf290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:37.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:37.705+0000 7f9258bfb640 1 --2- 172.21.15.92:0/961208061 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f925413df80 0x7f925415bee0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:37.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:37.705+0000 7f9258bfb640 1 -- 172.21.15.92:0/961208061 >> 172.21.15.92:0/961208061 conn(0x7f925406a800 msgr2=0x7f925406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:37.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:37.705+0000 7f9258bfb640 1 -- 172.21.15.92:0/961208061 shutdown_connections 2023-11-24T06:11:37.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:37.705+0000 7f9258bfb640 1 -- 172.21.15.92:0/961208061 wait complete. 2023-11-24T06:11:37.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:37.707+0000 7f9258bfb640 1 Processor -- start 2023-11-24T06:11:37.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:37.707+0000 7f9258bfb640 1 -- start start 2023-11-24T06:11:37.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:37.708+0000 7f9258bfb640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f925413df80 0x7f92540b3a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:37.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:37.708+0000 7f9258bfb640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f925413e360 0x7f92541f3040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:37.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:37.708+0000 7f9258bfb640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f925415f9c0 0x7f925413b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:37.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:37.708+0000 7f9258bfb640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f925413a2e0 con 0x7f925413df80 2023-11-24T06:11:37.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:37.708+0000 7f9258bfb640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f925413a5c0 con 0x7f925415f9c0 2023-11-24T06:11:37.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:37.708+0000 7f9258bfb640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f925413a8a0 con 0x7f925413e360 2023-11-24T06:11:37.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:37.708+0000 7f925259c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f925413df80 0x7f92540b3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:37.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:37.708+0000 7f925259c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f925413df80 0x7f92540b3a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58018/0 (socket says 172.21.15.92:58018) 2023-11-24T06:11:37.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:37.708+0000 7f925259c640 1 -- 172.21.15.92:0/1874243920 learned_addr learned my addr 172.21.15.92:0/1874243920 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:11:37.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:37.709+0000 7f925259c640 1 -- 172.21.15.92:0/1874243920 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f925413e360 msgr2=0x7f92541f3040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:37.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:37.709+0000 7f925259c640 1 --2- 172.21.15.92:0/1874243920 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f925413e360 0x7f92541f3040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:37.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:37.709+0000 7f925259c640 1 -- 172.21.15.92:0/1874243920 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f925415f9c0 msgr2=0x7f925413b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:37.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:37.709+0000 7f925259c640 1 --2- 172.21.15.92:0/1874243920 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f925415f9c0 0x7f925413b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:37.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:37.709+0000 7f925259c640 1 -- 172.21.15.92:0/1874243920 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92440094d0 con 0x7f925413df80 2023-11-24T06:11:37.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:37.709+0000 7f925259c640 1 --2- 172.21.15.92:0/1874243920 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f925413df80 0x7f92540b3a90 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f924402b950 tx=0x7f92440046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:37.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:37.709+0000 7f9242ffd640 1 -- 172.21.15.92:0/1874243920 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9244031860 con 0x7f925413df80 2023-11-24T06:11:37.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:37.709+0000 7f9242ffd640 1 -- 172.21.15.92:0/1874243920 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92440443f0 con 0x7f925413df80 2023-11-24T06:11:37.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:37.709+0000 7f9242ffd640 1 -- 172.21.15.92:0/1874243920 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92440447a0 con 0x7f925413df80 2023-11-24T06:11:37.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:37.709+0000 7f9258bfb640 1 -- 172.21.15.92:0/1874243920 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f925413ab80 con 0x7f925413df80 2023-11-24T06:11:37.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:37.710+0000 7f9258bfb640 1 -- 172.21.15.92:0/1874243920 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92540b3fd0 con 0x7f925413df80 2023-11-24T06:11:37.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:37.711+0000 7f9258bfb640 1 -- 172.21.15.92:0/1874243920 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f925413b0a0 con 0x7f925413df80 2023-11-24T06:11:37.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:37.712+0000 7f9242ffd640 1 -- 172.21.15.92:0/1874243920 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f924404d450 con 0x7f925413df80 2023-11-24T06:11:37.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:37.712+0000 7f9242ffd640 1 --2- 172.21.15.92:0/1874243920 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f923406ae40 0x7f923406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:37.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:37.713+0000 7f9242ffd640 1 -- 172.21.15.92:0/1874243920 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f924402d040 con 0x7f925413df80 2023-11-24T06:11:37.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:37.713+0000 7f9251d9b640 1 --2- 172.21.15.92:0/1874243920 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f923406ae40 0x7f923406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:37.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:37.713+0000 7f9251d9b640 1 --2- 172.21.15.92:0/1874243920 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f923406ae40 0x7f923406d300 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f9248006fd0 tx=0x7f9248008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:37.720 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:37.717+0000 7f9242ffd640 1 -- 172.21.15.92:0/1874243920 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9244076e20 con 0x7f925413df80 2023-11-24T06:11:37.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:37.956+0000 7f9258bfb640 1 -- 172.21.15.92:0/1874243920 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) v1 -- 0x7f925415c1e0 con 0x7f925413df80 2023-11-24T06:11:37.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:37.961+0000 7f9242ffd640 1 -- 172.21.15.92:0/1874243920 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v3) v1 ==== 90+0+0 (secure 0 0 0) 0x7f9244035070 con 0x7f925413df80 2023-11-24T06:11:37.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:37.963+0000 7f9258bfb640 1 -- 172.21.15.92:0/1874243920 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f923406ae40 msgr2=0x7f923406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:37.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:37.963+0000 7f9258bfb640 1 --2- 172.21.15.92:0/1874243920 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f923406ae40 0x7f923406d300 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f9248006fd0 tx=0x7f9248008040 comp rx=0 tx=0).stop 2023-11-24T06:11:37.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:37.963+0000 7f9258bfb640 1 -- 172.21.15.92:0/1874243920 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f925413df80 msgr2=0x7f92540b3a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:37.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:37.963+0000 7f9258bfb640 1 --2- 172.21.15.92:0/1874243920 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f925413df80 0x7f92540b3a90 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f924402b950 tx=0x7f92440046c0 comp rx=0 tx=0).stop 2023-11-24T06:11:37.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:37.963+0000 7f9258bfb640 1 -- 172.21.15.92:0/1874243920 shutdown_connections 2023-11-24T06:11:37.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:37.963+0000 7f9258bfb640 1 --2- 172.21.15.92:0/1874243920 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f923406ae40 0x7f923406d300 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:37.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:37.963+0000 7f9258bfb640 1 --2- 172.21.15.92:0/1874243920 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f925415f9c0 0x7f925413b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:37.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:37.963+0000 7f9258bfb640 1 --2- 172.21.15.92:0/1874243920 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f925413e360 0x7f92541f3040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:37.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:37.963+0000 7f9258bfb640 1 --2- 172.21.15.92:0/1874243920 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f925413df80 0x7f92540b3a90 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:37.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:37.963+0000 7f9258bfb640 1 -- 172.21.15.92:0/1874243920 >> 172.21.15.92:0/1874243920 conn(0x7f925406a800 msgr2=0x7f925415a180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:37.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:37.963+0000 7f9258bfb640 1 -- 172.21.15.92:0/1874243920 shutdown_connections 2023-11-24T06:11:37.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:37.964+0000 7f9258bfb640 1 -- 172.21.15.92:0/1874243920 wait complete. 2023-11-24T06:11:37.984 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2023-11-24T06:11:38.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:38.188+0000 7fbc43581640 1 Processor -- start 2023-11-24T06:11:38.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:38.188+0000 7fbc43581640 1 -- start start 2023-11-24T06:11:38.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:38.190+0000 7fbc43581640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fbc3c0d0150 0x7fbc3c0d1f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:38.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:38.190+0000 7fbc43581640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc3c0b2c70 con 0x7fbc3c15d3c0 2023-11-24T06:11:38.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:38.190+0000 7fbc43581640 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc3c0cf6d0 con 0x7fbc3c0ceea0 2023-11-24T06:11:38.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:38.190+0000 7fbc43581640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc3c0cf3f0 con 0x7fbc3c0d0150 2023-11-24T06:11:38.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:38.190+0000 7fbc4257f640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fbc3c15d3c0 0x7fbc3c0cfa40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:35426/0 (socket says 172.21.15.92:35426) 2023-11-24T06:11:38.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:38.190+0000 7fbc4257f640 1 -- 172.21.15.92:0/2432410815 learned_addr learned my addr 172.21.15.92:0/2432410815 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-24T06:11:38.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:38.190+0000 7fbc41d7e640 1 --2- 172.21.15.92:0/2432410815 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fbc3c0d0150 0x7fbc3c0d1f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:38.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:38.190+0000 7fbc4157d640 1 -- 172.21.15.92:0/2432410815 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2293466025 0 0) 0x7fbc3c0b2c70 con 0x7fbc3c15d3c0 2023-11-24T06:11:38.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:38.191+0000 7fbc4157d640 1 -- 172.21.15.92:0/2432410815 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbc28003520 con 0x7fbc3c15d3c0 2023-11-24T06:11:38.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:38.191+0000 7fbc4157d640 1 -- 172.21.15.92:0/2432410815 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2947766259 0 0) 0x7fbc28003520 con 0x7fbc3c15d3c0 2023-11-24T06:11:38.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:38.191+0000 7fbc4157d640 1 -- 172.21.15.92:0/2432410815 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fbc3c0d0150 msgr2=0x7fbc3c0d1f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:38.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:38.191+0000 7fbc4157d640 1 --2- 172.21.15.92:0/2432410815 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fbc3c0d0150 0x7fbc3c0d1f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:38.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:38.191+0000 7fbc4157d640 1 -- 172.21.15.92:0/2432410815 >> v1:172.21.15.118:6789/0 conn(0x7fbc3c0ceea0 legacy=0x7fbc3c0b3100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:38.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:38.191+0000 7fbc4157d640 1 -- 172.21.15.92:0/2432410815 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc3c0b2620 con 0x7fbc3c15d3c0 2023-11-24T06:11:38.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:38.191+0000 7fbc4157d640 1 -- 172.21.15.92:0/2432410815 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2607661350 0 0) 0x7fbc38001670 con 0x7fbc3c15d3c0 2023-11-24T06:11:38.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:38.191+0000 7fbc4157d640 1 -- 172.21.15.92:0/2432410815 >> v1:172.21.15.92:6789/0 conn(0x7fbc3c15d3c0 legacy=0x7fbc3c0cfa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:38.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:38.191+0000 7fbc4157d640 1 --2- 172.21.15.92:0/2432410815 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbc280039a0 0x7fbc28003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:38.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:38.191+0000 7fbc4157d640 1 --2- 172.21.15.92:0/2432410815 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fbc3c0d0150 0x7fbc280047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:38.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:38.191+0000 7fbc4157d640 1 --2- 172.21.15.92:0/2432410815 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbc28004d00 0x7fbc28005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:38.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:38.191+0000 7fbc4157d640 1 -- 172.21.15.92:0/2432410815 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc3c0cf3f0 con 0x7fbc28004d00 2023-11-24T06:11:38.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:38.191+0000 7fbc4157d640 1 -- 172.21.15.92:0/2432410815 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc2800bac0 con 0x7fbc280039a0 2023-11-24T06:11:38.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:38.191+0000 7fbc4157d640 1 -- 172.21.15.92:0/2432410815 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc2800bc80 con 0x7fbc3c0d0150 2023-11-24T06:11:38.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:38.192+0000 7fbc42d80640 1 --2- 172.21.15.92:0/2432410815 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbc28004d00 0x7fbc28005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:38.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:38.192+0000 7fbc41d7e640 1 --2- 172.21.15.92:0/2432410815 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fbc3c0d0150 0x7fbc280047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:38.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:38.192+0000 7fbc4257f640 1 --2- 172.21.15.92:0/2432410815 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbc280039a0 0x7fbc28003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:38.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:38.192+0000 7fbc42d80640 1 -- 172.21.15.92:0/2432410815 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fbc3c0d0150 msgr2=0x7fbc280047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:38.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:38.192+0000 7fbc42d80640 1 --2- 172.21.15.92:0/2432410815 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fbc3c0d0150 0x7fbc280047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:38.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:38.192+0000 7fbc42d80640 1 -- 172.21.15.92:0/2432410815 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbc280039a0 msgr2=0x7fbc28003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:38.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:38.192+0000 7fbc42d80640 1 --2- 172.21.15.92:0/2432410815 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbc280039a0 0x7fbc28003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:38.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:38.192+0000 7fbc42d80640 1 -- 172.21.15.92:0/2432410815 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc28003520 con 0x7fbc28004d00 2023-11-24T06:11:38.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:38.192+0000 7fbc4257f640 1 --2- 172.21.15.92:0/2432410815 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbc280039a0 0x7fbc28003dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:11:38.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:38.193+0000 7fbc42d80640 1 --2- 172.21.15.92:0/2432410815 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbc28004d00 0x7fbc28005180 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fbc34001e50 tx=0x7fbc280057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2575174fbefd311 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:38.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:38.193+0000 7fbc4157d640 1 -- 172.21.15.92:0/2432410815 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc34003b70 con 0x7fbc28004d00 2023-11-24T06:11:38.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:38.193+0000 7fbc4157d640 1 -- 172.21.15.92:0/2432410815 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc34003cd0 con 0x7fbc28004d00 2023-11-24T06:11:38.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:38.193+0000 7fbc4157d640 1 -- 172.21.15.92:0/2432410815 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc34005310 con 0x7fbc28004d00 2023-11-24T06:11:38.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:38.194+0000 7fbc43581640 1 -- 172.21.15.92:0/2432410815 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbc28004d00 msgr2=0x7fbc28005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:38.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:38.194+0000 7fbc43581640 1 --2- 172.21.15.92:0/2432410815 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbc28004d00 0x7fbc28005180 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fbc34001e50 tx=0x7fbc280057f0 comp rx=0 tx=0).stop 2023-11-24T06:11:38.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:38.194+0000 7fbc42d80640 1 -- 172.21.15.92:0/2432410815 reap_dead start 2023-11-24T06:11:38.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:38.194+0000 7fbc43581640 1 -- 172.21.15.92:0/2432410815 shutdown_connections 2023-11-24T06:11:38.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:38.194+0000 7fbc43581640 1 -- 172.21.15.92:0/2432410815 >> 172.21.15.92:0/2432410815 conn(0x7fbc3c0d3050 msgr2=0x7fbc3c0d3460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:38.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:38.194+0000 7fbc43581640 1 -- 172.21.15.92:0/2432410815 shutdown_connections 2023-11-24T06:11:38.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:38.194+0000 7fbc43581640 1 -- 172.21.15.92:0/2432410815 wait complete. 2023-11-24T06:11:38.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:38.196+0000 7fbc43581640 1 Processor -- start 2023-11-24T06:11:38.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:38.196+0000 7fbc43581640 1 -- start start 2023-11-24T06:11:38.201 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:38.197+0000 7fbc43581640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fbc28004d00 0x7fbc3c0b6db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:38.202 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:38.197+0000 7fbc43581640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbc3c0b7870 0x7fbc3c14f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:38.202 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:38.197+0000 7fbc43581640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbc3c0b7cd0 0x7fbc3c14f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:38.202 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:38.197+0000 7fbc43581640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc3c0b72f0 con 0x7fbc3c0b7cd0 2023-11-24T06:11:38.202 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:38.197+0000 7fbc43581640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc3c0b5d10 con 0x7fbc3c0b7870 2023-11-24T06:11:38.202 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:38.197+0000 7fbc43581640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc3c0b5ff0 con 0x7fbc28004d00 2023-11-24T06:11:38.202 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:38.197+0000 7fbc42d80640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbc3c0b7cd0 0x7fbc3c14f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:38.202 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:38.197+0000 7fbc42d80640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbc3c0b7cd0 0x7fbc3c14f5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58034/0 (socket says 172.21.15.92:58034) 2023-11-24T06:11:38.202 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:38.197+0000 7fbc42d80640 1 -- 172.21.15.92:0/621741854 learned_addr learned my addr 172.21.15.92:0/621741854 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:11:38.202 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:38.197+0000 7fbc4257f640 1 --2- 172.21.15.92:0/621741854 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fbc28004d00 0x7fbc3c0b6db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:38.202 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:38.197+0000 7fbc41d7e640 1 --2- 172.21.15.92:0/621741854 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbc3c0b7870 0x7fbc3c14f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:38.203 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:38.198+0000 7fbc42d80640 1 -- 172.21.15.92:0/621741854 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fbc28004d00 msgr2=0x7fbc3c0b6db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:38.203 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:38.198+0000 7fbc42d80640 1 --2- 172.21.15.92:0/621741854 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fbc28004d00 0x7fbc3c0b6db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:38.203 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:38.198+0000 7fbc42d80640 1 -- 172.21.15.92:0/621741854 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbc3c0b7870 msgr2=0x7fbc3c14f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:38.203 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:38.198+0000 7fbc42d80640 1 --2- 172.21.15.92:0/621741854 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbc3c0b7870 0x7fbc3c14f060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:38.203 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:38.198+0000 7fbc42d80640 1 -- 172.21.15.92:0/621741854 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc3c0b62d0 con 0x7fbc3c0b7cd0 2023-11-24T06:11:38.203 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:38.198+0000 7fbc42d80640 1 --2- 172.21.15.92:0/621741854 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbc3c0b7cd0 0x7fbc3c14f5a0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fbc34001f70 tx=0x7fbc340014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:38.203 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:38.198+0000 7fbc32ffd640 1 -- 172.21.15.92:0/621741854 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc3403e070 con 0x7fbc3c0b7cd0 2023-11-24T06:11:38.203 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:38.198+0000 7fbc32ffd640 1 -- 172.21.15.92:0/621741854 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc34003e40 con 0x7fbc3c0b7cd0 2023-11-24T06:11:38.203 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:38.198+0000 7fbc32ffd640 1 -- 172.21.15.92:0/621741854 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc34006d90 con 0x7fbc3c0b7cd0 2023-11-24T06:11:38.203 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:38.198+0000 7fbc43581640 1 -- 172.21.15.92:0/621741854 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc3c0b65b0 con 0x7fbc3c0b7cd0 2023-11-24T06:11:38.203 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:38.198+0000 7fbc43581640 1 -- 172.21.15.92:0/621741854 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc3c0cc400 con 0x7fbc3c0b7cd0 2023-11-24T06:11:38.204 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:38.200+0000 7fbc43581640 1 -- 172.21.15.92:0/621741854 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc3c0b6ad0 con 0x7fbc3c0b7cd0 2023-11-24T06:11:38.204 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:38.200+0000 7fbc32ffd640 1 -- 172.21.15.92:0/621741854 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fbc34007030 con 0x7fbc3c0b7cd0 2023-11-24T06:11:38.205 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:38.202+0000 7fbc32ffd640 1 --2- 172.21.15.92:0/621741854 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fbc2006ae40 0x7fbc2006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:38.205 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:38.202+0000 7fbc32ffd640 1 -- 172.21.15.92:0/621741854 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fbc340aa530 con 0x7fbc3c0b7cd0 2023-11-24T06:11:38.205 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:38.202+0000 7fbc4257f640 1 --2- 172.21.15.92:0/621741854 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fbc2006ae40 0x7fbc2006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:38.206 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:38.203+0000 7fbc4257f640 1 --2- 172.21.15.92:0/621741854 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fbc2006ae40 0x7fbc2006d300 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fbc38001eb0 tx=0x7fbc38009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:38.209 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:38.206+0000 7fbc32ffd640 1 -- 172.21.15.92:0/621741854 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fbc3c0b6ad0 con 0x7fbc3c0b7cd0 2023-11-24T06:11:38.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:38.466+0000 7fbc43581640 1 -- 172.21.15.92:0/621741854 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) v1 -- 0x7fbc3c0b6ad0 con 0x7fbc3c0b7cd0 2023-11-24T06:11:38.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:38.470+0000 7fbc32ffd640 1 -- 172.21.15.92:0/621741854 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v4) v1 ==== 90+0+0 (secure 0 0 0) 0x7fbc3c0b6ad0 con 0x7fbc3c0b7cd0 2023-11-24T06:11:38.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:38.473+0000 7fbc43581640 1 -- 172.21.15.92:0/621741854 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fbc2006ae40 msgr2=0x7fbc2006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:38.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:38.473+0000 7fbc43581640 1 --2- 172.21.15.92:0/621741854 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fbc2006ae40 0x7fbc2006d300 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fbc38001eb0 tx=0x7fbc38009040 comp rx=0 tx=0).stop 2023-11-24T06:11:38.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:38.473+0000 7fbc43581640 1 -- 172.21.15.92:0/621741854 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbc3c0b7cd0 msgr2=0x7fbc3c14f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:38.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:38.473+0000 7fbc43581640 1 --2- 172.21.15.92:0/621741854 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbc3c0b7cd0 0x7fbc3c14f5a0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fbc34001f70 tx=0x7fbc340014a0 comp rx=0 tx=0).stop 2023-11-24T06:11:38.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:38.473+0000 7fbc43581640 1 -- 172.21.15.92:0/621741854 shutdown_connections 2023-11-24T06:11:38.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:38.473+0000 7fbc43581640 1 --2- 172.21.15.92:0/621741854 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fbc2006ae40 0x7fbc2006d300 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:38.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:38.473+0000 7fbc43581640 1 --2- 172.21.15.92:0/621741854 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbc3c0b7cd0 0x7fbc3c14f5a0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:38.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:38.473+0000 7fbc43581640 1 --2- 172.21.15.92:0/621741854 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbc3c0b7870 0x7fbc3c14f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:38.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:38.473+0000 7fbc43581640 1 --2- 172.21.15.92:0/621741854 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fbc28004d00 0x7fbc3c0b6db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:38.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:38.473+0000 7fbc43581640 1 -- 172.21.15.92:0/621741854 >> 172.21.15.92:0/621741854 conn(0x7fbc3c0d3050 msgr2=0x7fbc3c0cf960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:38.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:38.473+0000 7fbc43581640 1 -- 172.21.15.92:0/621741854 shutdown_connections 2023-11-24T06:11:38.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:38.473+0000 7fbc43581640 1 -- 172.21.15.92:0/621741854 wait complete. 2023-11-24T06:11:38.495 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2023-11-24T06:11:38.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:38.704+0000 7fb7edf48640 1 Processor -- start 2023-11-24T06:11:38.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:38.705+0000 7fb7edf48640 1 -- start start 2023-11-24T06:11:38.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:38.706+0000 7fb7edf48640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb7e80ae5a0 0x7fb7e815e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:38.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:38.706+0000 7fb7edf48640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb7e80cf290 0x7fb7e80cf720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:38.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:38.706+0000 7fb7edf48640 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7e8070c40 con 0x7fb7e80b3ae0 2023-11-24T06:11:38.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:38.706+0000 7fb7edf48640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7e8071450 con 0x7fb7e80cf290 2023-11-24T06:11:38.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:38.706+0000 7fb7edf48640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7e80cfc60 con 0x7fb7e80ae5a0 2023-11-24T06:11:38.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:38.707+0000 7fb7e7fff640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb7e80cf290 0x7fb7e80cf720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:38.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:38.707+0000 7fb7e7fff640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb7e80cf290 0x7fb7e80cf720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58040/0 (socket says 172.21.15.92:58040) 2023-11-24T06:11:38.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:38.707+0000 7fb7e7fff640 1 -- 172.21.15.92:0/2254136783 learned_addr learned my addr 172.21.15.92:0/2254136783 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:11:38.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:38.707+0000 7fb7e77fe640 1 --2- 172.21.15.92:0/2254136783 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb7e80ae5a0 0x7fb7e815e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:38.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:38.707+0000 7fb7e7fff640 1 -- 172.21.15.92:0/2254136783 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb7e80ae5a0 msgr2=0x7fb7e815e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:38.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:38.707+0000 7fb7e7fff640 1 --2- 172.21.15.92:0/2254136783 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb7e80ae5a0 0x7fb7e815e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:38.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:38.707+0000 7fb7e7fff640 1 -- 172.21.15.92:0/2254136783 >> v1:172.21.15.118:6789/0 conn(0x7fb7e80b3ae0 legacy=0x7fb7e806cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:38.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:38.707+0000 7fb7e7fff640 1 -- 172.21.15.92:0/2254136783 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7e806c470 con 0x7fb7e80cf290 2023-11-24T06:11:38.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:38.708+0000 7fb7e7fff640 1 --2- 172.21.15.92:0/2254136783 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb7e80cf290 0x7fb7e80cf720 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fb7e0009e50 tx=0x7fb7e002bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9b9a2258da4caa0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:38.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:38.709+0000 7fb7e6ffd640 1 -- 172.21.15.92:0/2254136783 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7e002cb00 con 0x7fb7e80cf290 2023-11-24T06:11:38.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:38.709+0000 7fb7e6ffd640 1 -- 172.21.15.92:0/2254136783 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7e002cc60 con 0x7fb7e80cf290 2023-11-24T06:11:38.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:38.709+0000 7fb7e6ffd640 1 -- 172.21.15.92:0/2254136783 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7e00343f0 con 0x7fb7e80cf290 2023-11-24T06:11:38.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:38.709+0000 7fb7edf48640 1 -- 172.21.15.92:0/2254136783 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb7e80cf290 msgr2=0x7fb7e80cf720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:38.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:38.709+0000 7fb7edf48640 1 --2- 172.21.15.92:0/2254136783 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb7e80cf290 0x7fb7e80cf720 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fb7e0009e50 tx=0x7fb7e002bf60 comp rx=0 tx=0).stop 2023-11-24T06:11:38.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:38.709+0000 7fb7edf48640 1 -- 172.21.15.92:0/2254136783 shutdown_connections 2023-11-24T06:11:38.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:38.709+0000 7fb7edf48640 1 --2- 172.21.15.92:0/2254136783 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb7e80cf290 0x7fb7e80cf720 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:38.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:38.709+0000 7fb7edf48640 1 --2- 172.21.15.92:0/2254136783 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb7e80ae5a0 0x7fb7e815e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:38.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:38.710+0000 7fb7edf48640 1 -- 172.21.15.92:0/2254136783 >> 172.21.15.92:0/2254136783 conn(0x7fb7e806a800 msgr2=0x7fb7e806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:38.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:38.710+0000 7fb7edf48640 1 -- 172.21.15.92:0/2254136783 shutdown_connections 2023-11-24T06:11:38.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:38.710+0000 7fb7edf48640 1 -- 172.21.15.92:0/2254136783 wait complete. 2023-11-24T06:11:38.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:38.712+0000 7fb7edf48640 1 Processor -- start 2023-11-24T06:11:38.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:38.712+0000 7fb7edf48640 1 -- start start 2023-11-24T06:11:38.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:38.713+0000 7fb7edf48640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb7e80ae5a0 0x7fb7e815cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:38.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:38.713+0000 7fb7edf48640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb7e80b3ae0 0x7fb7e8149990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:38.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:38.713+0000 7fb7edf48640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb7e80cf290 0x7fb7e815fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:38.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:38.713+0000 7fb7edf48640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7e8157240 con 0x7fb7e80ae5a0 2023-11-24T06:11:38.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:38.713+0000 7fb7edf48640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7e8149ed0 con 0x7fb7e80cf290 2023-11-24T06:11:38.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:38.713+0000 7fb7edf48640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7e814f3b0 con 0x7fb7e80b3ae0 2023-11-24T06:11:38.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:38.713+0000 7fb7e77fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb7e80ae5a0 0x7fb7e815cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:38.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:38.713+0000 7fb7e77fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb7e80ae5a0 0x7fb7e815cfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58054/0 (socket says 172.21.15.92:58054) 2023-11-24T06:11:38.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:38.713+0000 7fb7e77fe640 1 -- 172.21.15.92:0/4034546592 learned_addr learned my addr 172.21.15.92:0/4034546592 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:11:38.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:38.714+0000 7fb7e7fff640 1 --2- 172.21.15.92:0/4034546592 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb7e80cf290 0x7fb7e815fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:38.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:38.714+0000 7fb7dffff640 1 --2- 172.21.15.92:0/4034546592 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb7e80b3ae0 0x7fb7e8149990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:38.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:38.714+0000 7fb7e77fe640 1 -- 172.21.15.92:0/4034546592 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb7e80b3ae0 msgr2=0x7fb7e8149990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:38.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:38.714+0000 7fb7e77fe640 1 --2- 172.21.15.92:0/4034546592 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb7e80b3ae0 0x7fb7e8149990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:38.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:38.714+0000 7fb7e77fe640 1 -- 172.21.15.92:0/4034546592 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb7e80cf290 msgr2=0x7fb7e815fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:38.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:38.714+0000 7fb7e77fe640 1 --2- 172.21.15.92:0/4034546592 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb7e80cf290 0x7fb7e815fba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:38.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:38.714+0000 7fb7e77fe640 1 -- 172.21.15.92:0/4034546592 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7e0009b30 con 0x7fb7e80ae5a0 2023-11-24T06:11:38.720 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:38.714+0000 7fb7e77fe640 1 --2- 172.21.15.92:0/4034546592 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb7e80ae5a0 0x7fb7e815cfd0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fb7d800d800 tx=0x7fb7d800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:38.720 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:38.714+0000 7fb7e4ff9640 1 -- 172.21.15.92:0/4034546592 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7d800b6c0 con 0x7fb7e80ae5a0 2023-11-24T06:11:38.720 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:38.714+0000 7fb7e4ff9640 1 -- 172.21.15.92:0/4034546592 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7d800b820 con 0x7fb7e80ae5a0 2023-11-24T06:11:38.720 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:38.714+0000 7fb7edf48640 1 -- 172.21.15.92:0/4034546592 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7e814f690 con 0x7fb7e80ae5a0 2023-11-24T06:11:38.720 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:38.714+0000 7fb7e4ff9640 1 -- 172.21.15.92:0/4034546592 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7d800bbb0 con 0x7fb7e80ae5a0 2023-11-24T06:11:38.720 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:38.715+0000 7fb7edf48640 1 -- 172.21.15.92:0/4034546592 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7e8156990 con 0x7fb7e80ae5a0 2023-11-24T06:11:38.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:38.716+0000 7fb7edf48640 1 -- 172.21.15.92:0/4034546592 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7e8156d30 con 0x7fb7e80ae5a0 2023-11-24T06:11:38.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:38.716+0000 7fb7e4ff9640 1 -- 172.21.15.92:0/4034546592 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb7d800bd10 con 0x7fb7e80ae5a0 2023-11-24T06:11:38.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:38.717+0000 7fb7e4ff9640 1 --2- 172.21.15.92:0/4034546592 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fb7c806ae40 0x7fb7c806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:38.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:38.717+0000 7fb7e4ff9640 1 -- 172.21.15.92:0/4034546592 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fb7d80897d0 con 0x7fb7e80ae5a0 2023-11-24T06:11:38.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:38.718+0000 7fb7dffff640 1 --2- 172.21.15.92:0/4034546592 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fb7c806ae40 0x7fb7c806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:38.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:38.718+0000 7fb7dffff640 1 --2- 172.21.15.92:0/4034546592 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fb7c806ae40 0x7fb7c806d300 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fb7e815e320 tx=0x7fb7d0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:38.726 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:38.723+0000 7fb7e4ff9640 1 -- 172.21.15.92:0/4034546592 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb7e8156d30 con 0x7fb7e80ae5a0 2023-11-24T06:11:38.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:38.961+0000 7fb7edf48640 1 -- 172.21.15.92:0/4034546592 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) v1 -- 0x7fb7e8156d30 con 0x7fb7e80ae5a0 2023-11-24T06:11:38.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:38.964+0000 7fb7e4ff9640 1 -- 172.21.15.92:0/4034546592 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v5) v1 ==== 90+0+0 (secure 0 0 0) 0x7fb7e8156d30 con 0x7fb7e80ae5a0 2023-11-24T06:11:38.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:38.967+0000 7fb7edf48640 1 -- 172.21.15.92:0/4034546592 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fb7c806ae40 msgr2=0x7fb7c806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:38.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:38.967+0000 7fb7edf48640 1 --2- 172.21.15.92:0/4034546592 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fb7c806ae40 0x7fb7c806d300 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fb7e815e320 tx=0x7fb7d0009040 comp rx=0 tx=0).stop 2023-11-24T06:11:38.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:38.967+0000 7fb7edf48640 1 -- 172.21.15.92:0/4034546592 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb7e80ae5a0 msgr2=0x7fb7e815cfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:38.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:38.967+0000 7fb7edf48640 1 --2- 172.21.15.92:0/4034546592 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb7e80ae5a0 0x7fb7e815cfd0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fb7d800d800 tx=0x7fb7d800dcc0 comp rx=0 tx=0).stop 2023-11-24T06:11:38.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:38.967+0000 7fb7edf48640 1 -- 172.21.15.92:0/4034546592 shutdown_connections 2023-11-24T06:11:38.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:38.967+0000 7fb7edf48640 1 --2- 172.21.15.92:0/4034546592 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fb7c806ae40 0x7fb7c806d300 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:38.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:38.967+0000 7fb7edf48640 1 --2- 172.21.15.92:0/4034546592 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb7e80cf290 0x7fb7e815fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:38.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:38.967+0000 7fb7edf48640 1 --2- 172.21.15.92:0/4034546592 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb7e80b3ae0 0x7fb7e8149990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:38.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:38.967+0000 7fb7edf48640 1 --2- 172.21.15.92:0/4034546592 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb7e80ae5a0 0x7fb7e815cfd0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:38.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:38.967+0000 7fb7edf48640 1 -- 172.21.15.92:0/4034546592 >> 172.21.15.92:0/4034546592 conn(0x7fb7e806a800 msgr2=0x7fb7e815a9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:38.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:38.967+0000 7fb7edf48640 1 -- 172.21.15.92:0/4034546592 shutdown_connections 2023-11-24T06:11:38.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:38.967+0000 7fb7edf48640 1 -- 172.21.15.92:0/4034546592 wait complete. 2023-11-24T06:11:38.987 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2023-11-24T06:11:39.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:39.192+0000 7fbec717a640 1 Processor -- start 2023-11-24T06:11:39.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:39.193+0000 7fbec717a640 1 -- start start 2023-11-24T06:11:39.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:39.194+0000 7fbec717a640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fbec015e170 0x7fbec00cd9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:39.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:39.194+0000 7fbec717a640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbec00700f0 con 0x7fbec00cdef0 2023-11-24T06:11:39.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:39.194+0000 7fbec717a640 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbec00ce720 con 0x7fbec013cda0 2023-11-24T06:11:39.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:39.194+0000 7fbec717a640 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbec00cea00 con 0x7fbec015e170 2023-11-24T06:11:39.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:39.195+0000 7fbec6979640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fbec00cdef0 0x7fbec0070b40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:59672/0 (socket says 172.21.15.92:59672) 2023-11-24T06:11:39.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:39.195+0000 7fbec6979640 1 -- 172.21.15.92:0/1467417113 learned_addr learned my addr 172.21.15.92:0/1467417113 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-24T06:11:39.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:39.195+0000 7fbec5977640 1 --2- 172.21.15.92:0/1467417113 >> v2:172.21.15.118:3300/0 conn(0x7fbec015e170 0x7fbec00cd9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:39.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:39.195+0000 7fbec5176640 1 -- 172.21.15.92:0/1467417113 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1627696919 0 0) 0x7fbec00700f0 con 0x7fbec00cdef0 2023-11-24T06:11:39.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:39.195+0000 7fbec5176640 1 -- 172.21.15.92:0/1467417113 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbeac005730 con 0x7fbec00cdef0 2023-11-24T06:11:39.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:39.196+0000 7fbec5176640 1 -- 172.21.15.92:0/1467417113 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3418769013 0 0) 0x7fbeac005730 con 0x7fbec00cdef0 2023-11-24T06:11:39.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:39.196+0000 7fbec5176640 1 -- 172.21.15.92:0/1467417113 >> v2:172.21.15.118:3300/0 conn(0x7fbec015e170 msgr2=0x7fbec00cd9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:39.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:39.196+0000 7fbec5176640 1 --2- 172.21.15.92:0/1467417113 >> v2:172.21.15.118:3300/0 conn(0x7fbec015e170 0x7fbec00cd9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:39.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:39.196+0000 7fbec5176640 1 -- 172.21.15.92:0/1467417113 >> v1:172.21.15.118:6789/0 conn(0x7fbec013cda0 legacy=0x7fbec015da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:39.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:39.196+0000 7fbec5176640 1 -- 172.21.15.92:0/1467417113 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbec0070340 con 0x7fbec00cdef0 2023-11-24T06:11:39.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:39.196+0000 7fbec5176640 1 -- 172.21.15.92:0/1467417113 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2607661350 0 0) 0x7fbeb80019f0 con 0x7fbec00cdef0 2023-11-24T06:11:39.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:39.196+0000 7fbec5176640 1 -- 172.21.15.92:0/1467417113 >> v1:172.21.15.92:6789/0 conn(0x7fbec00cdef0 legacy=0x7fbec0070b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:39.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:39.196+0000 7fbec5176640 1 --2- 172.21.15.92:0/1467417113 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fbeac005b80 0x7fbeac005fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:39.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:39.196+0000 7fbec5176640 1 --2- 172.21.15.92:0/1467417113 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbeac0069d0 0x7fbeac006e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:39.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:39.196+0000 7fbec5176640 1 --2- 172.21.15.92:0/1467417113 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbeac007390 0x7fbeac00db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:39.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:39.196+0000 7fbec5176640 1 -- 172.21.15.92:0/1467417113 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbec00cea00 con 0x7fbeac0069d0 2023-11-24T06:11:39.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:39.196+0000 7fbec5176640 1 -- 172.21.15.92:0/1467417113 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbeac00e330 con 0x7fbeac007390 2023-11-24T06:11:39.201 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:39.196+0000 7fbec5176640 1 -- 172.21.15.92:0/1467417113 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbeac00e4f0 con 0x7fbeac005b80 2023-11-24T06:11:39.201 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:39.196+0000 7fbec5977640 1 --2- 172.21.15.92:0/1467417113 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbeac0069d0 0x7fbeac006e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:39.202 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:39.196+0000 7fbec5977640 1 -- 172.21.15.92:0/1467417113 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fbeac005b80 msgr2=0x7fbeac005fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:39.202 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:39.196+0000 7fbec5977640 1 --2- 172.21.15.92:0/1467417113 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fbeac005b80 0x7fbeac005fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:39.202 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:39.196+0000 7fbec5977640 1 -- 172.21.15.92:0/1467417113 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbeac007390 msgr2=0x7fbeac00db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:39.202 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:39.196+0000 7fbec6979640 1 --2- 172.21.15.92:0/1467417113 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbeac007390 0x7fbeac00db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:39.202 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:39.196+0000 7fbec5977640 1 --2- 172.21.15.92:0/1467417113 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbeac007390 0x7fbeac00db90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:39.202 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:39.196+0000 7fbec6979640 1 -- 172.21.15.92:0/1467417113 reap_dead start 2023-11-24T06:11:39.202 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:39.196+0000 7fbec5977640 1 -- 172.21.15.92:0/1467417113 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbeac005730 con 0x7fbeac0069d0 2023-11-24T06:11:39.202 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:39.198+0000 7fbec5977640 1 --2- 172.21.15.92:0/1467417113 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbeac0069d0 0x7fbeac006e50 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fbeb0009ac0 tx=0x7fbeac007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee070bcf69185ec6 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:39.203 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:39.198+0000 7fbec5176640 1 -- 172.21.15.92:0/1467417113 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbeb002c6b0 con 0x7fbeac0069d0 2023-11-24T06:11:39.203 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:39.198+0000 7fbec5176640 1 -- 172.21.15.92:0/1467417113 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbeb002c810 con 0x7fbeac0069d0 2023-11-24T06:11:39.203 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:39.198+0000 7fbec5176640 1 -- 172.21.15.92:0/1467417113 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbeb002cba0 con 0x7fbeac0069d0 2023-11-24T06:11:39.203 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:39.198+0000 7fbec717a640 1 -- 172.21.15.92:0/1467417113 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbeac0069d0 msgr2=0x7fbeac006e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:39.203 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:39.198+0000 7fbec717a640 1 --2- 172.21.15.92:0/1467417113 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbeac0069d0 0x7fbeac006e50 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fbeb0009ac0 tx=0x7fbeac007a00 comp rx=0 tx=0).stop 2023-11-24T06:11:39.203 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:39.198+0000 7fbec717a640 1 -- 172.21.15.92:0/1467417113 shutdown_connections 2023-11-24T06:11:39.203 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:39.198+0000 7fbec717a640 1 --2- 172.21.15.92:0/1467417113 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbeac0069d0 0x7fbeac006e50 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:39.203 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:39.198+0000 7fbec717a640 1 -- 172.21.15.92:0/1467417113 >> 172.21.15.92:0/1467417113 conn(0x7fbec006a3e0 msgr2=0x7fbec006a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:39.203 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:39.198+0000 7fbec717a640 1 -- 172.21.15.92:0/1467417113 shutdown_connections 2023-11-24T06:11:39.203 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:39.199+0000 7fbec717a640 1 -- 172.21.15.92:0/1467417113 wait complete. 2023-11-24T06:11:39.204 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:39.200+0000 7fbec717a640 1 Processor -- start 2023-11-24T06:11:39.204 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:39.200+0000 7fbec717a640 1 -- start start 2023-11-24T06:11:39.205 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:39.201+0000 7fbec717a640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fbeac0069d0 0x7fbec015c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:39.205 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:39.201+0000 7fbec717a640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbec013ea50 0x7fbec0156310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:39.205 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:39.201+0000 7fbec717a640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbec013eeb0 0x7fbec0156850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:39.205 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:39.201+0000 7fbec717a640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbec0158000 con 0x7fbec013ea50 2023-11-24T06:11:39.205 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:39.201+0000 7fbec717a640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbec00ae110 con 0x7fbec013eeb0 2023-11-24T06:11:39.205 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:39.201+0000 7fbec717a640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbec01582c0 con 0x7fbeac0069d0 2023-11-24T06:11:39.205 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:39.202+0000 7fbec5977640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbec013ea50 0x7fbec0156310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:39.205 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:39.202+0000 7fbec5977640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbec013ea50 0x7fbec0156310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40286/0 (socket says 172.21.15.92:40286) 2023-11-24T06:11:39.205 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:39.202+0000 7fbec5977640 1 -- 172.21.15.92:0/1343007336 learned_addr learned my addr 172.21.15.92:0/1343007336 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:11:39.206 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:39.202+0000 7fbec5977640 1 -- 172.21.15.92:0/1343007336 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fbeac0069d0 msgr2=0x7fbec015c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:39.207 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:39.202+0000 7fbec6178640 1 --2- 172.21.15.92:0/1343007336 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fbeac0069d0 0x7fbec015c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:39.207 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:39.202+0000 7fbec5977640 1 --2- 172.21.15.92:0/1343007336 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fbeac0069d0 0x7fbec015c870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:39.207 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:39.202+0000 7fbec5977640 1 -- 172.21.15.92:0/1343007336 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbec013eeb0 msgr2=0x7fbec0156850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:39.207 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:39.202+0000 7fbec5977640 1 --2- 172.21.15.92:0/1343007336 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbec013eeb0 0x7fbec0156850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:39.207 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:39.202+0000 7fbec5977640 1 -- 172.21.15.92:0/1343007336 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbec01585a0 con 0x7fbec013ea50 2023-11-24T06:11:39.207 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:39.202+0000 7fbec5977640 1 --2- 172.21.15.92:0/1343007336 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbec013ea50 0x7fbec0156310 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fbeb002c250 tx=0x7fbeb0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:39.207 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:39.203+0000 7fbeb6ffd640 1 -- 172.21.15.92:0/1343007336 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbeb0003810 con 0x7fbec013ea50 2023-11-24T06:11:39.207 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:39.203+0000 7fbeb6ffd640 1 -- 172.21.15.92:0/1343007336 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbeb0003970 con 0x7fbec013ea50 2023-11-24T06:11:39.207 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:39.203+0000 7fbeb6ffd640 1 -- 172.21.15.92:0/1343007336 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbeb0003d00 con 0x7fbec013ea50 2023-11-24T06:11:39.207 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:39.203+0000 7fbec717a640 1 -- 172.21.15.92:0/1343007336 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbec0158880 con 0x7fbec013ea50 2023-11-24T06:11:39.207 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:39.203+0000 7fbec717a640 1 -- 172.21.15.92:0/1343007336 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbec0158f80 con 0x7fbec013ea50 2023-11-24T06:11:39.208 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:39.204+0000 7fbec717a640 1 -- 172.21.15.92:0/1343007336 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbec0067450 con 0x7fbec013ea50 2023-11-24T06:11:39.208 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:39.205+0000 7fbeb6ffd640 1 -- 172.21.15.92:0/1343007336 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fbeb0034440 con 0x7fbec013ea50 2023-11-24T06:11:39.209 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:39.206+0000 7fbeb6ffd640 1 --2- 172.21.15.92:0/1343007336 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fbea406ae70 0x7fbea406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:39.209 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:39.206+0000 7fbeb6ffd640 1 -- 172.21.15.92:0/1343007336 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fbeb00ab570 con 0x7fbec013ea50 2023-11-24T06:11:39.209 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:39.206+0000 7fbec6178640 1 --2- 172.21.15.92:0/1343007336 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fbea406ae70 0x7fbea406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:39.210 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:39.207+0000 7fbec6178640 1 --2- 172.21.15.92:0/1343007336 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fbea406ae70 0x7fbea406d330 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fbebc001610 tx=0x7fbebc009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:39.213 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:39.211+0000 7fbeb6ffd640 1 -- 172.21.15.92:0/1343007336 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fbeb0076a10 con 0x7fbec013ea50 2023-11-24T06:11:39.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:39.469+0000 7fbec717a640 1 -- 172.21.15.92:0/1343007336 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) v1 -- 0x7fbec0072060 con 0x7fbec013ea50 2023-11-24T06:11:39.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:39.474+0000 7fbeb6ffd640 1 -- 172.21.15.92:0/1343007336 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v6) v1 ==== 90+0+0 (secure 0 0 0) 0x7fbeb00ab030 con 0x7fbec013ea50 2023-11-24T06:11:39.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:39.477+0000 7fbec717a640 1 -- 172.21.15.92:0/1343007336 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fbea406ae70 msgr2=0x7fbea406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:39.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:39.477+0000 7fbec717a640 1 --2- 172.21.15.92:0/1343007336 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fbea406ae70 0x7fbea406d330 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fbebc001610 tx=0x7fbebc009040 comp rx=0 tx=0).stop 2023-11-24T06:11:39.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:39.477+0000 7fbec717a640 1 -- 172.21.15.92:0/1343007336 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbec013ea50 msgr2=0x7fbec0156310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:39.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:39.477+0000 7fbec717a640 1 --2- 172.21.15.92:0/1343007336 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbec013ea50 0x7fbec0156310 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fbeb002c250 tx=0x7fbeb0004730 comp rx=0 tx=0).stop 2023-11-24T06:11:39.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:39.477+0000 7fbec717a640 1 -- 172.21.15.92:0/1343007336 shutdown_connections 2023-11-24T06:11:39.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:39.477+0000 7fbec717a640 1 --2- 172.21.15.92:0/1343007336 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fbea406ae70 0x7fbea406d330 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:39.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:39.477+0000 7fbec717a640 1 --2- 172.21.15.92:0/1343007336 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbec013eeb0 0x7fbec0156850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:39.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:39.477+0000 7fbec717a640 1 --2- 172.21.15.92:0/1343007336 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbec013ea50 0x7fbec0156310 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:39.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:39.477+0000 7fbec717a640 1 --2- 172.21.15.92:0/1343007336 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fbeac0069d0 0x7fbec015c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:39.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:39.477+0000 7fbec717a640 1 -- 172.21.15.92:0/1343007336 >> 172.21.15.92:0/1343007336 conn(0x7fbec006a3e0 msgr2=0x7fbec006a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:39.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:39.477+0000 7fbec717a640 1 -- 172.21.15.92:0/1343007336 shutdown_connections 2023-11-24T06:11:39.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:39.477+0000 7fbec717a640 1 -- 172.21.15.92:0/1343007336 wait complete. 2023-11-24T06:11:39.500 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2023-11-24T06:11:39.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:39.712+0000 7f8857007640 1 Processor -- start 2023-11-24T06:11:39.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:39.712+0000 7f8857007640 1 -- start start 2023-11-24T06:11:39.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:39.713+0000 7f8857007640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f885013e360 0x7f88500cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:39.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:39.713+0000 7f8857007640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f885015f9c0 0x7f88500cf7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:39.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:39.713+0000 7f8857007640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f885015c5f0 con 0x7f885013df80 2023-11-24T06:11:39.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:39.713+0000 7f8857007640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f885006c470 con 0x7f885015f9c0 2023-11-24T06:11:39.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:39.713+0000 7f8857007640 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f885006c750 con 0x7f885013e360 2023-11-24T06:11:39.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:39.714+0000 7f88555a4640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f885015f9c0 0x7f88500cf7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:39.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:39.714+0000 7f88555a4640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f885015f9c0 0x7f88500cf7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40290/0 (socket says 172.21.15.92:40290) 2023-11-24T06:11:39.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:39.714+0000 7f88555a4640 1 -- 172.21.15.92:0/1514173098 learned_addr learned my addr 172.21.15.92:0/1514173098 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:11:39.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:39.714+0000 7f884ffff640 1 --2- 172.21.15.92:0/1514173098 >> v2:172.21.15.118:3300/0 conn(0x7f885013e360 0x7f88500cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:39.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:39.714+0000 7f88555a4640 1 -- 172.21.15.92:0/1514173098 >> v2:172.21.15.118:3300/0 conn(0x7f885013e360 msgr2=0x7f88500cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:39.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:39.714+0000 7f88555a4640 1 --2- 172.21.15.92:0/1514173098 >> v2:172.21.15.118:3300/0 conn(0x7f885013e360 0x7f88500cf290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:39.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:39.714+0000 7f88555a4640 1 -- 172.21.15.92:0/1514173098 >> v1:172.21.15.92:6789/0 conn(0x7f885013df80 legacy=0x7f885015bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:39.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:39.714+0000 7f88555a4640 1 -- 172.21.15.92:0/1514173098 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f885006ca30 con 0x7f885015f9c0 2023-11-24T06:11:39.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:39.714+0000 7f884ffff640 1 --2- 172.21.15.92:0/1514173098 >> v2:172.21.15.118:3300/0 conn(0x7f885013e360 0x7f88500cf290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:11:39.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:39.715+0000 7f88555a4640 1 --2- 172.21.15.92:0/1514173098 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f885015f9c0 0x7f88500cf7d0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f885006ded0 tx=0x7f884802bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad1cb5c6bb53dda3 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:39.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:39.715+0000 7f884f7fe640 1 -- 172.21.15.92:0/1514173098 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f884802cb20 con 0x7f885015f9c0 2023-11-24T06:11:39.720 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:39.716+0000 7f884f7fe640 1 -- 172.21.15.92:0/1514173098 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f884802cc80 con 0x7f885015f9c0 2023-11-24T06:11:39.720 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:39.716+0000 7f884f7fe640 1 -- 172.21.15.92:0/1514173098 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88480343f0 con 0x7f885015f9c0 2023-11-24T06:11:39.720 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:39.716+0000 7f8857007640 1 -- 172.21.15.92:0/1514173098 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f885015f9c0 msgr2=0x7f88500cf7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:39.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:39.716+0000 7f8857007640 1 --2- 172.21.15.92:0/1514173098 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f885015f9c0 0x7f88500cf7d0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f885006ded0 tx=0x7f884802bf80 comp rx=0 tx=0).stop 2023-11-24T06:11:39.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:39.716+0000 7f8857007640 1 -- 172.21.15.92:0/1514173098 shutdown_connections 2023-11-24T06:11:39.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:39.716+0000 7f8857007640 1 --2- 172.21.15.92:0/1514173098 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f885015f9c0 0x7f88500cf7d0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:39.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:39.716+0000 7f8857007640 1 --2- 172.21.15.92:0/1514173098 >> v2:172.21.15.118:3300/0 conn(0x7f885013e360 0x7f88500cf290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:39.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:39.716+0000 7f8857007640 1 -- 172.21.15.92:0/1514173098 >> 172.21.15.92:0/1514173098 conn(0x7f885006a800 msgr2=0x7f885006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:39.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:39.716+0000 7f8857007640 1 -- 172.21.15.92:0/1514173098 shutdown_connections 2023-11-24T06:11:39.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:39.716+0000 7f8857007640 1 -- 172.21.15.92:0/1514173098 wait complete. 2023-11-24T06:11:39.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:39.718+0000 7f8857007640 1 Processor -- start 2023-11-24T06:11:39.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:39.718+0000 7f8857007640 1 -- start start 2023-11-24T06:11:39.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:39.719+0000 7f8857007640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f885013df80 0x7f885014f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:39.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:39.719+0000 7f8857007640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f885013e360 0x7f88500b2f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:39.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:39.719+0000 7f8857007640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f885015f9c0 0x7f8850150950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:39.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:39.719+0000 7f8857007640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8850151510 con 0x7f885013df80 2023-11-24T06:11:39.724 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:39.719+0000 7f8857007640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f885014ec80 con 0x7f885013e360 2023-11-24T06:11:39.724 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:39.719+0000 7f8857007640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f885014e2e0 con 0x7f885015f9c0 2023-11-24T06:11:39.724 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:39.719+0000 7f8854da3640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f885013df80 0x7f885014f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:39.724 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:39.719+0000 7f8854da3640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f885013df80 0x7f885014f0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40294/0 (socket says 172.21.15.92:40294) 2023-11-24T06:11:39.724 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:39.719+0000 7f8854da3640 1 -- 172.21.15.92:0/2232759561 learned_addr learned my addr 172.21.15.92:0/2232759561 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:11:39.724 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:39.720+0000 7f8854da3640 1 -- 172.21.15.92:0/2232759561 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f885015f9c0 msgr2=0x7f8850150950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:39.724 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:39.720+0000 7f8854da3640 1 --2- 172.21.15.92:0/2232759561 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f885015f9c0 0x7f8850150950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:39.724 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:39.720+0000 7f8854da3640 1 -- 172.21.15.92:0/2232759561 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f885013e360 msgr2=0x7f88500b2f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:39.724 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:39.720+0000 7f8854da3640 1 --2- 172.21.15.92:0/2232759561 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f885013e360 0x7f88500b2f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:39.724 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:39.720+0000 7f8854da3640 1 -- 172.21.15.92:0/2232759561 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8848009b30 con 0x7f885013df80 2023-11-24T06:11:39.725 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:39.720+0000 7f8854da3640 1 --2- 172.21.15.92:0/2232759561 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f885013df80 0x7f885014f0a0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f884400c390 tx=0x7f884400c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:39.726 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:39.720+0000 7f884d7fa640 1 -- 172.21.15.92:0/2232759561 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88440052d0 con 0x7f885013df80 2023-11-24T06:11:39.726 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:39.720+0000 7f884d7fa640 1 -- 172.21.15.92:0/2232759561 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8844005430 con 0x7f885013df80 2023-11-24T06:11:39.726 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:39.720+0000 7f884d7fa640 1 -- 172.21.15.92:0/2232759561 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8844003830 con 0x7f885013df80 2023-11-24T06:11:39.726 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:39.720+0000 7f8857007640 1 -- 172.21.15.92:0/2232759561 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f885014e5c0 con 0x7f885013df80 2023-11-24T06:11:39.726 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:39.720+0000 7f8857007640 1 -- 172.21.15.92:0/2232759561 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f885014bbd0 con 0x7f885013df80 2023-11-24T06:11:39.726 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:39.722+0000 7f8857007640 1 -- 172.21.15.92:0/2232759561 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f885014edc0 con 0x7f885013df80 2023-11-24T06:11:39.726 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:39.723+0000 7f884d7fa640 1 -- 172.21.15.92:0/2232759561 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f8844003990 con 0x7f885013df80 2023-11-24T06:11:39.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:39.724+0000 7f884d7fa640 1 --2- 172.21.15.92:0/2232759561 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f883406ae40 0x7f883406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:39.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:39.724+0000 7f884d7fa640 1 -- 172.21.15.92:0/2232759561 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f88440887e0 con 0x7f885013df80 2023-11-24T06:11:39.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:39.724+0000 7f884ffff640 1 --2- 172.21.15.92:0/2232759561 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f883406ae40 0x7f883406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:39.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:39.725+0000 7f884ffff640 1 --2- 172.21.15.92:0/2232759561 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f883406ae40 0x7f883406d300 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f883c006fd0 tx=0x7f883c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:39.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:39.729+0000 7f884d7fa640 1 -- 172.21.15.92:0/2232759561 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f885014edc0 con 0x7f885013df80 2023-11-24T06:11:39.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:39.984+0000 7f8857007640 1 -- 172.21.15.92:0/2232759561 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) v1 -- 0x7f885014edc0 con 0x7f885013df80 2023-11-24T06:11:39.999 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:39.996+0000 7f884d7fa640 1 -- 172.21.15.92:0/2232759561 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v7) v1 ==== 97+0+0 (secure 0 0 0) 0x7f885014edc0 con 0x7f885013df80 2023-11-24T06:11:40.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:39.998+0000 7f8857007640 1 -- 172.21.15.92:0/2232759561 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f883406ae40 msgr2=0x7f883406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:40.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:39.998+0000 7f8857007640 1 --2- 172.21.15.92:0/2232759561 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f883406ae40 0x7f883406d300 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f883c006fd0 tx=0x7f883c008040 comp rx=0 tx=0).stop 2023-11-24T06:11:40.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:39.998+0000 7f8857007640 1 -- 172.21.15.92:0/2232759561 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f885013df80 msgr2=0x7f885014f0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:40.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:39.998+0000 7f8857007640 1 --2- 172.21.15.92:0/2232759561 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f885013df80 0x7f885014f0a0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f884400c390 tx=0x7f884400c850 comp rx=0 tx=0).stop 2023-11-24T06:11:40.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:39.999+0000 7f8857007640 1 -- 172.21.15.92:0/2232759561 shutdown_connections 2023-11-24T06:11:40.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:39.999+0000 7f8857007640 1 --2- 172.21.15.92:0/2232759561 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f883406ae40 0x7f883406d300 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:40.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:39.999+0000 7f8857007640 1 --2- 172.21.15.92:0/2232759561 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f885015f9c0 0x7f8850150950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:40.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:39.999+0000 7f8857007640 1 --2- 172.21.15.92:0/2232759561 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f885013e360 0x7f88500b2f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:40.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:39.999+0000 7f8857007640 1 --2- 172.21.15.92:0/2232759561 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f885013df80 0x7f885014f0a0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:40.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:39.999+0000 7f8857007640 1 -- 172.21.15.92:0/2232759561 >> 172.21.15.92:0/2232759561 conn(0x7f885006a800 msgr2=0x7f885015df30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:40.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:39.999+0000 7f8857007640 1 -- 172.21.15.92:0/2232759561 shutdown_connections 2023-11-24T06:11:40.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:39.999+0000 7f8857007640 1 -- 172.21.15.92:0/2232759561 wait complete. 2023-11-24T06:11:40.021 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2023-11-24T06:11:40.215 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:40.212+0000 7fc4da948640 1 Processor -- start 2023-11-24T06:11:40.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:40.213+0000 7fc4da948640 1 -- start start 2023-11-24T06:11:40.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:40.214+0000 7fc4da948640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc4d415d610 0x7fc4d40ce820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:40.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:40.214+0000 7fc4da948640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4d40b3400 con 0x7fc4d40ced60 2023-11-24T06:11:40.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:40.214+0000 7fc4da948640 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4d40b27f0 con 0x7fc4d40d1ef0 2023-11-24T06:11:40.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:40.214+0000 7fc4da948640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4d40d1390 con 0x7fc4d415d610 2023-11-24T06:11:40.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:40.214+0000 7fc4cbfff640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fc4d40ced60 0x7fc4d40d17e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:59686/0 (socket says 172.21.15.92:59686) 2023-11-24T06:11:40.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:40.214+0000 7fc4cbfff640 1 -- 172.21.15.92:0/1752395266 learned_addr learned my addr 172.21.15.92:0/1752395266 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-24T06:11:40.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:40.215+0000 7fc4d37fe640 1 -- 172.21.15.92:0/1752395266 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2160433220 0 0) 0x7fc4d40b3400 con 0x7fc4d40ced60 2023-11-24T06:11:40.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:40.215+0000 7fc4d37fe640 1 -- 172.21.15.92:0/1752395266 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc4c0003520 con 0x7fc4d40ced60 2023-11-24T06:11:40.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:40.215+0000 7fc4d3fff640 1 --2- 172.21.15.92:0/1752395266 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc4d415d610 0x7fc4d40ce820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:40.220 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:40.215+0000 7fc4d37fe640 1 -- 172.21.15.92:0/1752395266 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4222601715 0 0) 0x7fc4c0003520 con 0x7fc4d40ced60 2023-11-24T06:11:40.220 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:40.215+0000 7fc4d37fe640 1 -- 172.21.15.92:0/1752395266 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc4d415d610 msgr2=0x7fc4d40ce820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:40.220 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:40.215+0000 7fc4d37fe640 1 --2- 172.21.15.92:0/1752395266 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc4d415d610 0x7fc4d40ce820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:40.220 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:40.215+0000 7fc4d37fe640 1 -- 172.21.15.92:0/1752395266 >> v1:172.21.15.118:6789/0 conn(0x7fc4d40d1ef0 legacy=0x7fc4d40b3890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:40.220 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:40.215+0000 7fc4d37fe640 1 -- 172.21.15.92:0/1752395266 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4d40b2ad0 con 0x7fc4d40ced60 2023-11-24T06:11:40.220 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:40.215+0000 7fc4d37fe640 1 -- 172.21.15.92:0/1752395266 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2607661350 0 0) 0x7fc4bc001670 con 0x7fc4d40ced60 2023-11-24T06:11:40.220 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:40.215+0000 7fc4d37fe640 1 -- 172.21.15.92:0/1752395266 >> v1:172.21.15.92:6789/0 conn(0x7fc4d40ced60 legacy=0x7fc4d40d17e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:40.220 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:40.216+0000 7fc4d37fe640 1 --2- 172.21.15.92:0/1752395266 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc4c00039a0 0x7fc4c0003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:40.220 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:40.216+0000 7fc4d37fe640 1 --2- 172.21.15.92:0/1752395266 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc4d415d610 0x7fc4c00047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:40.220 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:40.216+0000 7fc4d37fe640 1 --2- 172.21.15.92:0/1752395266 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc4c0004d00 0x7fc4c0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:40.220 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:40.216+0000 7fc4d37fe640 1 -- 172.21.15.92:0/1752395266 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4d40d1390 con 0x7fc4c0004d00 2023-11-24T06:11:40.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:40.216+0000 7fc4d37fe640 1 -- 172.21.15.92:0/1752395266 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4c000ba30 con 0x7fc4c00039a0 2023-11-24T06:11:40.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:40.216+0000 7fc4d37fe640 1 -- 172.21.15.92:0/1752395266 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4c000bbf0 con 0x7fc4d415d610 2023-11-24T06:11:40.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:40.216+0000 7fc4d8ee5640 1 --2- 172.21.15.92:0/1752395266 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc4c0004d00 0x7fc4c0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:40.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:40.216+0000 7fc4d3fff640 1 --2- 172.21.15.92:0/1752395266 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc4c00039a0 0x7fc4c0003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:40.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:40.216+0000 7fc4d8ee5640 1 -- 172.21.15.92:0/1752395266 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc4d415d610 msgr2=0x7fc4c00047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:40.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:40.216+0000 7fc4cbfff640 1 --2- 172.21.15.92:0/1752395266 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc4d415d610 0x7fc4c00047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:40.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:40.216+0000 7fc4d8ee5640 1 --2- 172.21.15.92:0/1752395266 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc4d415d610 0x7fc4c00047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:40.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:40.216+0000 7fc4d8ee5640 1 -- 172.21.15.92:0/1752395266 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc4c00039a0 msgr2=0x7fc4c0003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:40.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:40.216+0000 7fc4d8ee5640 1 --2- 172.21.15.92:0/1752395266 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc4c00039a0 0x7fc4c0003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:40.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:40.216+0000 7fc4d8ee5640 1 -- 172.21.15.92:0/1752395266 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4c0003520 con 0x7fc4c0004d00 2023-11-24T06:11:40.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:40.217+0000 7fc4d8ee5640 1 --2- 172.21.15.92:0/1752395266 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc4c0004d00 0x7fc4c0005180 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fc4d408bb40 tx=0x7fc4c00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59383df991f9f8bc server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:40.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:40.218+0000 7fc4d37fe640 1 -- 172.21.15.92:0/1752395266 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4cc003b70 con 0x7fc4c0004d00 2023-11-24T06:11:40.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:40.218+0000 7fc4d37fe640 1 -- 172.21.15.92:0/1752395266 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4cc003cd0 con 0x7fc4c0004d00 2023-11-24T06:11:40.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:40.218+0000 7fc4d37fe640 1 -- 172.21.15.92:0/1752395266 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4cc005310 con 0x7fc4c0004d00 2023-11-24T06:11:40.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:40.218+0000 7fc4da948640 1 -- 172.21.15.92:0/1752395266 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc4c0004d00 msgr2=0x7fc4c0005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:40.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:40.218+0000 7fc4da948640 1 --2- 172.21.15.92:0/1752395266 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc4c0004d00 0x7fc4c0005180 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fc4d408bb40 tx=0x7fc4c00057f0 comp rx=0 tx=0).stop 2023-11-24T06:11:40.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:40.218+0000 7fc4d8ee5640 1 -- 172.21.15.92:0/1752395266 reap_dead start 2023-11-24T06:11:40.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:40.218+0000 7fc4da948640 1 -- 172.21.15.92:0/1752395266 shutdown_connections 2023-11-24T06:11:40.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:40.218+0000 7fc4da948640 1 -- 172.21.15.92:0/1752395266 >> 172.21.15.92:0/1752395266 conn(0x7fc4d409c540 msgr2=0x7fc4d40d3200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:40.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:40.218+0000 7fc4da948640 1 -- 172.21.15.92:0/1752395266 shutdown_connections 2023-11-24T06:11:40.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:40.218+0000 7fc4da948640 1 -- 172.21.15.92:0/1752395266 wait complete. 2023-11-24T06:11:40.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:40.220+0000 7fc4da948640 1 Processor -- start 2023-11-24T06:11:40.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:40.220+0000 7fc4da948640 1 -- start start 2023-11-24T06:11:40.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:40.221+0000 7fc4da948640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc4c0004d00 0x7fc4d41f2d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:40.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:40.221+0000 7fc4da948640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc4d40b6d90 0x7fc4d415b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:40.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:40.221+0000 7fc4da948640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc4d40b81b0 0x7fc4d41600d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:40.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:40.221+0000 7fc4da948640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4d40b8a70 con 0x7fc4c0004d00 2023-11-24T06:11:40.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:40.221+0000 7fc4da948640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4d41f3250 con 0x7fc4d40b81b0 2023-11-24T06:11:40.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:40.221+0000 7fc4da948640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4d415db70 con 0x7fc4d40b6d90 2023-11-24T06:11:40.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:40.221+0000 7fc4d3fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc4c0004d00 0x7fc4d41f2d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:40.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:40.221+0000 7fc4d3fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc4c0004d00 0x7fc4d41f2d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40304/0 (socket says 172.21.15.92:40304) 2023-11-24T06:11:40.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:40.221+0000 7fc4d3fff640 1 -- 172.21.15.92:0/2769703516 learned_addr learned my addr 172.21.15.92:0/2769703516 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:11:40.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:40.222+0000 7fc4d3fff640 1 -- 172.21.15.92:0/2769703516 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc4d40b6d90 msgr2=0x7fc4d415b560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:40.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:40.222+0000 7fc4d3fff640 1 --2- 172.21.15.92:0/2769703516 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc4d40b6d90 0x7fc4d415b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:40.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:40.222+0000 7fc4d3fff640 1 -- 172.21.15.92:0/2769703516 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc4d40b81b0 msgr2=0x7fc4d41600d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:40.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:40.222+0000 7fc4d8ee5640 1 --2- 172.21.15.92:0/2769703516 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc4d40b81b0 0x7fc4d41600d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:40.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:40.222+0000 7fc4d3fff640 1 --2- 172.21.15.92:0/2769703516 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc4d40b81b0 0x7fc4d41600d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:40.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:40.222+0000 7fc4d3fff640 1 -- 172.21.15.92:0/2769703516 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4d415baa0 con 0x7fc4c0004d00 2023-11-24T06:11:40.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:40.222+0000 7fc4d3fff640 1 --2- 172.21.15.92:0/2769703516 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc4c0004d00 0x7fc4d41f2d10 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fc4c4011660 tx=0x7fc4c4011b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:40.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:40.222+0000 7fc4d17fa640 1 -- 172.21.15.92:0/2769703516 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4c400f960 con 0x7fc4c0004d00 2023-11-24T06:11:40.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:40.222+0000 7fc4d17fa640 1 -- 172.21.15.92:0/2769703516 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4c400fac0 con 0x7fc4c0004d00 2023-11-24T06:11:40.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:40.222+0000 7fc4da948640 1 -- 172.21.15.92:0/2769703516 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4d415ddc0 con 0x7fc4c0004d00 2023-11-24T06:11:40.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:40.222+0000 7fc4d17fa640 1 -- 172.21.15.92:0/2769703516 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4c400fe30 con 0x7fc4c0004d00 2023-11-24T06:11:40.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:40.222+0000 7fc4da948640 1 -- 172.21.15.92:0/2769703516 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4d415e380 con 0x7fc4c0004d00 2023-11-24T06:11:40.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:40.223+0000 7fc4da948640 1 -- 172.21.15.92:0/2769703516 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4d40675d0 con 0x7fc4c0004d00 2023-11-24T06:11:40.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:40.224+0000 7fc4d17fa640 1 -- 172.21.15.92:0/2769703516 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc4c4015460 con 0x7fc4c0004d00 2023-11-24T06:11:40.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:40.225+0000 7fc4d17fa640 1 --2- 172.21.15.92:0/2769703516 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fc4b406ae40 0x7fc4b406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:40.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:40.225+0000 7fc4d17fa640 1 -- 172.21.15.92:0/2769703516 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fc4c4089bd0 con 0x7fc4c0004d00 2023-11-24T06:11:40.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:40.225+0000 7fc4cbfff640 1 --2- 172.21.15.92:0/2769703516 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fc4b406ae40 0x7fc4b406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:40.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:40.226+0000 7fc4cbfff640 1 --2- 172.21.15.92:0/2769703516 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fc4b406ae40 0x7fc4b406d300 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fc4bc0017a0 tx=0x7fc4bc009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:40.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:40.230+0000 7fc4d17fa640 1 -- 172.21.15.92:0/2769703516 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc4c4054f30 con 0x7fc4c0004d00 2023-11-24T06:11:40.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:40.486+0000 7fc4da948640 1 -- 172.21.15.92:0/2769703516 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) v1 -- 0x7fc4d40d05e0 con 0x7fc4c0004d00 2023-11-24T06:11:40.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:40.492+0000 7fc4d17fa640 1 -- 172.21.15.92:0/2769703516 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v8) v1 ==== 90+0+0 (secure 0 0 0) 0x7fc4c4058550 con 0x7fc4c0004d00 2023-11-24T06:11:40.498 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:40.495+0000 7fc4da948640 1 -- 172.21.15.92:0/2769703516 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fc4b406ae40 msgr2=0x7fc4b406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:40.498 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:40.495+0000 7fc4da948640 1 --2- 172.21.15.92:0/2769703516 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fc4b406ae40 0x7fc4b406d300 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fc4bc0017a0 tx=0x7fc4bc009040 comp rx=0 tx=0).stop 2023-11-24T06:11:40.498 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:40.495+0000 7fc4da948640 1 -- 172.21.15.92:0/2769703516 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc4c0004d00 msgr2=0x7fc4d41f2d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:40.498 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:40.495+0000 7fc4da948640 1 --2- 172.21.15.92:0/2769703516 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc4c0004d00 0x7fc4d41f2d10 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fc4c4011660 tx=0x7fc4c4011b20 comp rx=0 tx=0).stop 2023-11-24T06:11:40.499 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:40.495+0000 7fc4da948640 1 -- 172.21.15.92:0/2769703516 shutdown_connections 2023-11-24T06:11:40.499 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:40.495+0000 7fc4da948640 1 --2- 172.21.15.92:0/2769703516 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fc4b406ae40 0x7fc4b406d300 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:40.499 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:40.495+0000 7fc4da948640 1 --2- 172.21.15.92:0/2769703516 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc4d40b81b0 0x7fc4d41600d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:40.499 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:40.495+0000 7fc4da948640 1 --2- 172.21.15.92:0/2769703516 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc4d40b6d90 0x7fc4d415b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:40.499 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:40.495+0000 7fc4da948640 1 --2- 172.21.15.92:0/2769703516 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc4c0004d00 0x7fc4d41f2d10 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:40.499 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:40.495+0000 7fc4da948640 1 -- 172.21.15.92:0/2769703516 >> 172.21.15.92:0/2769703516 conn(0x7fc4d409c540 msgr2=0x7fc4d40cc2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:40.499 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:40.495+0000 7fc4da948640 1 -- 172.21.15.92:0/2769703516 shutdown_connections 2023-11-24T06:11:40.499 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:40.495+0000 7fc4da948640 1 -- 172.21.15.92:0/2769703516 wait complete. 2023-11-24T06:11:40.516 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:11:40.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:40.720+0000 7fb67a71d640 1 Processor -- start 2023-11-24T06:11:40.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:40.720+0000 7fb67a71d640 1 -- start start 2023-11-24T06:11:40.725 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:40.722+0000 7fb67a71d640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fb67415e3b0 0x7fb674164de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:40.725 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:40.722+0000 7fb67a71d640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6740b1ee0 con 0x7fb674143ae0 2023-11-24T06:11:40.725 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:40.722+0000 7fb67a71d640 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6740b2780 con 0x7fb6740cf9a0 2023-11-24T06:11:40.725 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:40.722+0000 7fb67a71d640 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb67415e990 con 0x7fb67415e3b0 2023-11-24T06:11:40.725 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:40.722+0000 7fb673fff640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fb674143ae0 0x7fb6740cf290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:59690/0 (socket says 172.21.15.92:59690) 2023-11-24T06:11:40.725 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:40.722+0000 7fb673fff640 1 -- 172.21.15.92:0/332753535 learned_addr learned my addr 172.21.15.92:0/332753535 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-24T06:11:40.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:40.722+0000 7fb672ffd640 1 -- 172.21.15.92:0/332753535 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3205030416 0 0) 0x7fb6740b1ee0 con 0x7fb674143ae0 2023-11-24T06:11:40.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:40.723+0000 7fb672ffd640 1 -- 172.21.15.92:0/332753535 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb65c003520 con 0x7fb674143ae0 2023-11-24T06:11:40.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:40.723+0000 7fb678cba640 1 --2- 172.21.15.92:0/332753535 >> v2:172.21.15.118:3300/0 conn(0x7fb67415e3b0 0x7fb674164de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:40.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:40.723+0000 7fb672ffd640 1 -- 172.21.15.92:0/332753535 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1048099605 0 0) 0x7fb65c003520 con 0x7fb674143ae0 2023-11-24T06:11:40.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:40.723+0000 7fb672ffd640 1 -- 172.21.15.92:0/332753535 >> v2:172.21.15.118:3300/0 conn(0x7fb67415e3b0 msgr2=0x7fb674164de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:40.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:40.723+0000 7fb672ffd640 1 --2- 172.21.15.92:0/332753535 >> v2:172.21.15.118:3300/0 conn(0x7fb67415e3b0 0x7fb674164de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:40.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:40.723+0000 7fb672ffd640 1 -- 172.21.15.92:0/332753535 >> v1:172.21.15.118:6789/0 conn(0x7fb6740cf9a0 legacy=0x7fb67415dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:40.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:40.723+0000 7fb672ffd640 1 -- 172.21.15.92:0/332753535 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6740b2130 con 0x7fb674143ae0 2023-11-24T06:11:40.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:40.723+0000 7fb672ffd640 1 -- 172.21.15.92:0/332753535 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2607661350 0 0) 0x7fb664001670 con 0x7fb674143ae0 2023-11-24T06:11:40.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:40.723+0000 7fb672ffd640 1 -- 172.21.15.92:0/332753535 >> v1:172.21.15.92:6789/0 conn(0x7fb674143ae0 legacy=0x7fb6740cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:40.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:40.723+0000 7fb672ffd640 1 --2- 172.21.15.92:0/332753535 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb65c0039c0 0x7fb65c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:40.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:40.723+0000 7fb672ffd640 1 --2- 172.21.15.92:0/332753535 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb65c0047c0 0x7fb65c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:40.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:40.723+0000 7fb672ffd640 1 --2- 172.21.15.92:0/332753535 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb65c0051d0 0x7fb65c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:40.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:40.723+0000 7fb672ffd640 1 -- 172.21.15.92:0/332753535 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb67415e990 con 0x7fb65c0051d0 2023-11-24T06:11:40.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:40.723+0000 7fb672ffd640 1 -- 172.21.15.92:0/332753535 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb65c00c100 con 0x7fb65c0039c0 2023-11-24T06:11:40.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:40.723+0000 7fb672ffd640 1 -- 172.21.15.92:0/332753535 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb65c00e4d0 con 0x7fb65c0047c0 2023-11-24T06:11:40.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:40.723+0000 7fb678cba640 1 --2- 172.21.15.92:0/332753535 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb65c0051d0 0x7fb65c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:40.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:40.724+0000 7fb678cba640 1 -- 172.21.15.92:0/332753535 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb65c0047c0 msgr2=0x7fb65c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:40.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:40.724+0000 7fb678cba640 1 --2- 172.21.15.92:0/332753535 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb65c0047c0 0x7fb65c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:40.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:40.724+0000 7fb673fff640 1 --2- 172.21.15.92:0/332753535 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb65c0039c0 0x7fb65c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:40.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:40.724+0000 7fb678cba640 1 -- 172.21.15.92:0/332753535 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb65c0039c0 msgr2=0x7fb65c003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:40.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:40.724+0000 7fb678cba640 1 --2- 172.21.15.92:0/332753535 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb65c0039c0 0x7fb65c003dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:40.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:40.724+0000 7fb678cba640 1 -- 172.21.15.92:0/332753535 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb65c003520 con 0x7fb65c0051d0 2023-11-24T06:11:40.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:40.725+0000 7fb678cba640 1 -- 172.21.15.92:0/332753535 reap_dead start 2023-11-24T06:11:40.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:40.725+0000 7fb678cba640 1 --2- 172.21.15.92:0/332753535 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb65c0051d0 0x7fb65c00b980 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fb66c00ba00 tx=0x7fb65c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11230d5a63f77ee4 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:40.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:40.725+0000 7fb672ffd640 1 -- 172.21.15.92:0/332753535 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb66c004020 con 0x7fb65c0051d0 2023-11-24T06:11:40.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:40.725+0000 7fb672ffd640 1 -- 172.21.15.92:0/332753535 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb66c004180 con 0x7fb65c0051d0 2023-11-24T06:11:40.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:40.725+0000 7fb672ffd640 1 -- 172.21.15.92:0/332753535 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb66c003610 con 0x7fb65c0051d0 2023-11-24T06:11:40.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:40.725+0000 7fb67a71d640 1 -- 172.21.15.92:0/332753535 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb65c0051d0 msgr2=0x7fb65c00b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:40.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:40.725+0000 7fb67a71d640 1 --2- 172.21.15.92:0/332753535 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb65c0051d0 0x7fb65c00b980 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fb66c00ba00 tx=0x7fb65c0057f0 comp rx=0 tx=0).stop 2023-11-24T06:11:40.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:40.725+0000 7fb67a71d640 1 -- 172.21.15.92:0/332753535 shutdown_connections 2023-11-24T06:11:40.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:40.725+0000 7fb67a71d640 1 --2- 172.21.15.92:0/332753535 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb65c0051d0 0x7fb65c00b980 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:40.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:40.725+0000 7fb67a71d640 1 -- 172.21.15.92:0/332753535 >> 172.21.15.92:0/332753535 conn(0x7fb67406a800 msgr2=0x7fb67406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:40.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:40.726+0000 7fb67a71d640 1 -- 172.21.15.92:0/332753535 shutdown_connections 2023-11-24T06:11:40.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:40.726+0000 7fb67a71d640 1 -- 172.21.15.92:0/332753535 wait complete. 2023-11-24T06:11:40.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:40.727+0000 7fb67a71d640 1 Processor -- start 2023-11-24T06:11:40.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:40.727+0000 7fb67a71d640 1 -- start start 2023-11-24T06:11:40.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:40.728+0000 7fb67a71d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb65c0051d0 0x7fb6740aba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:40.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:40.728+0000 7fb67a71d640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb6741408f0 0x7fb6740b0240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:40.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:40.728+0000 7fb67a71d640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb674131830 0x7fb674131cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:40.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:40.728+0000 7fb67a71d640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb674132c90 con 0x7fb65c0051d0 2023-11-24T06:11:40.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:40.728+0000 7fb67a71d640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6740ac610 con 0x7fb6741408f0 2023-11-24T06:11:40.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:40.728+0000 7fb67a71d640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6740ac330 con 0x7fb674131830 2023-11-24T06:11:40.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:40.729+0000 7fb673fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb65c0051d0 0x7fb6740aba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:40.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:40.729+0000 7fb673fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb65c0051d0 0x7fb6740aba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40334/0 (socket says 172.21.15.92:40334) 2023-11-24T06:11:40.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:40.729+0000 7fb673fff640 1 -- 172.21.15.92:0/1628641509 learned_addr learned my addr 172.21.15.92:0/1628641509 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:11:40.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:40.729+0000 7fb678cba640 1 --2- 172.21.15.92:0/1628641509 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb674131830 0x7fb674131cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:40.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:40.729+0000 7fb673fff640 1 -- 172.21.15.92:0/1628641509 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb674131830 msgr2=0x7fb674131cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:40.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:40.729+0000 7fb673fff640 1 --2- 172.21.15.92:0/1628641509 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb674131830 0x7fb674131cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:40.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:40.729+0000 7fb673fff640 1 -- 172.21.15.92:0/1628641509 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb6741408f0 msgr2=0x7fb6740b0240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:40.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:40.729+0000 7fb6737fe640 1 --2- 172.21.15.92:0/1628641509 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb6741408f0 0x7fb6740b0240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:40.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:40.729+0000 7fb673fff640 1 --2- 172.21.15.92:0/1628641509 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb6741408f0 0x7fb6740b0240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:40.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:40.729+0000 7fb673fff640 1 -- 172.21.15.92:0/1628641509 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb67412f940 con 0x7fb65c0051d0 2023-11-24T06:11:40.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:40.729+0000 7fb673fff640 1 --2- 172.21.15.92:0/1628641509 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb65c0051d0 0x7fb6740aba20 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fb66400d7a0 tx=0x7fb66400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:40.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:40.729+0000 7fb670ff9640 1 -- 172.21.15.92:0/1628641509 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6640074e0 con 0x7fb65c0051d0 2023-11-24T06:11:40.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:40.730+0000 7fb670ff9640 1 -- 172.21.15.92:0/1628641509 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb664007640 con 0x7fb65c0051d0 2023-11-24T06:11:40.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:40.730+0000 7fb670ff9640 1 -- 172.21.15.92:0/1628641509 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6640049e0 con 0x7fb65c0051d0 2023-11-24T06:11:40.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:40.730+0000 7fb67a71d640 1 -- 172.21.15.92:0/1628641509 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb67412fc20 con 0x7fb65c0051d0 2023-11-24T06:11:40.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:40.730+0000 7fb67a71d640 1 -- 172.21.15.92:0/1628641509 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6740abf60 con 0x7fb65c0051d0 2023-11-24T06:11:40.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:40.731+0000 7fb67a71d640 1 -- 172.21.15.92:0/1628641509 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb67412f7a0 con 0x7fb65c0051d0 2023-11-24T06:11:40.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:40.732+0000 7fb670ff9640 1 -- 172.21.15.92:0/1628641509 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb664004b40 con 0x7fb65c0051d0 2023-11-24T06:11:40.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:40.733+0000 7fb670ff9640 1 --2- 172.21.15.92:0/1628641509 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fb66006ae40 0x7fb66006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:40.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:40.733+0000 7fb670ff9640 1 -- 172.21.15.92:0/1628641509 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fb664088b10 con 0x7fb65c0051d0 2023-11-24T06:11:40.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:40.733+0000 7fb6737fe640 1 --2- 172.21.15.92:0/1628641509 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fb66006ae40 0x7fb66006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:40.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:40.734+0000 7fb6737fe640 1 --2- 172.21.15.92:0/1628641509 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fb66006ae40 0x7fb66006d300 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fb668007fd0 tx=0x7fb668009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:40.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:40.737+0000 7fb670ff9640 1 -- 172.21.15.92:0/1628641509 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb67412f7a0 con 0x7fb65c0051d0 2023-11-24T06:11:40.932 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:40.929+0000 7fb67a71d640 1 -- 172.21.15.92:0/1628641509 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb674130140 con 0x7fb65c0051d0 2023-11-24T06:11:40.933 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:40.930+0000 7fb670ff9640 1 -- 172.21.15.92:0/1628641509 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) v1 ==== 74+0+12809 (secure 0 0 0) 0x7fb674130140 con 0x7fb65c0051d0 2023-11-24T06:11:40.934 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-24T06:11:40.934 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":29,"fsid":"2d6dee5f-8aa3-470f-931a-50ade963b1cb","created":"2023-11-24T06:10:04.707204+0000","modified":"2023-11-24T06:11:37.001827+0000","last_up_change":"2023-11-24T06:10:17.598352+0000","last_in_change":"2023-11-24T06:10:09.040793+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:10:17.733098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ff37a9fb-135f-471f-b4b0-f7b5772006c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6803","nonce":20560714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6805","nonce":20560714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6809","nonce":20560714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6807","nonce":20560714}]},"public_addr":"172.21.15.92:6803/20560714","cluster_addr":"172.21.15.92:6805/20560714","heartbeat_back_addr":"172.21.15.92:6809/20560714","heartbeat_front_addr":"172.21.15.92:6807/20560714","state":["exists","up"]},{"osd":1,"uuid":"03cdd353-dcf1-4f38-94fe-a795137c06b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6819","nonce":2451239906}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6821","nonce":2451239906}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6825","nonce":2451239906}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6823","nonce":2451239906}]},"public_addr":"172.21.15.92:6819/2451239906","cluster_addr":"172.21.15.92:6821/2451239906","heartbeat_back_addr":"172.21.15.92:6825/2451239906","heartbeat_front_addr":"172.21.15.92:6823/2451239906","state":["exists","up"]},{"osd":2,"uuid":"a4a20d80-5d57-484b-becd-9aa8070dd3aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6811","nonce":1124979277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6813","nonce":1124979277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6817","nonce":1124979277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6815","nonce":1124979277}]},"public_addr":"172.21.15.92:6811/1124979277","cluster_addr":"172.21.15.92:6813/1124979277","heartbeat_back_addr":"172.21.15.92:6817/1124979277","heartbeat_front_addr":"172.21.15.92:6815/1124979277","state":["exists","up"]},{"osd":3,"uuid":"4648d7ce-9b29-4583-9425-4ba8fd2b1903","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6827","nonce":1734115782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6829","nonce":1734115782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6833","nonce":1734115782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6831","nonce":1734115782}]},"public_addr":"172.21.15.92:6827/1734115782","cluster_addr":"172.21.15.92:6829/1734115782","heartbeat_back_addr":"172.21.15.92:6833/1734115782","heartbeat_front_addr":"172.21.15.92:6831/1734115782","state":["exists","up"]},{"osd":4,"uuid":"3bc9d7eb-c6fa-4fcf-a3ec-ade604e05aac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6819","nonce":3425937229}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6822","nonce":3425937229}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6830","nonce":3425937229}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6826","nonce":3425937229}]},"public_addr":"172.21.15.118:6819/3425937229","cluster_addr":"172.21.15.118:6822/3425937229","heartbeat_back_addr":"172.21.15.118:6830/3425937229","heartbeat_front_addr":"172.21.15.118:6826/3425937229","state":["exists","up"]},{"osd":5,"uuid":"dce9ba6d-59ed-46d1-84fc-a84b13de10c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6801","nonce":827455984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6803","nonce":827455984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6807","nonce":827455984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6805","nonce":827455984}]},"public_addr":"172.21.15.118:6801/827455984","cluster_addr":"172.21.15.118:6803/827455984","heartbeat_back_addr":"172.21.15.118:6807/827455984","heartbeat_front_addr":"172.21.15.118:6805/827455984","state":["exists","up"]},{"osd":6,"uuid":"82aec504-ae58-4709-8d30-46f2b12081b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6809","nonce":2946986141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6811","nonce":2946986141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6815","nonce":2946986141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6813","nonce":2946986141}]},"public_addr":"172.21.15.118:6809/2946986141","cluster_addr":"172.21.15.118:6811/2946986141","heartbeat_back_addr":"172.21.15.118:6815/2946986141","heartbeat_front_addr":"172.21.15.118:6813/2946986141","state":["exists","up"]},{"osd":7,"uuid":"4ed4c624-c88f-4d35-bdb1-bf209b5dffb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6817","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6818","nonce":48122059}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6821","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6823","nonce":48122059}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6829","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6831","nonce":48122059}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6825","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6827","nonce":48122059}]},"public_addr":"172.21.15.118:6818/48122059","cluster_addr":"172.21.15.118:6823/48122059","heartbeat_back_addr":"172.21.15.118:6831/48122059","heartbeat_front_addr":"172.21.15.118:6827/48122059","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.536664+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.368324+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.271201+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.342971+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.621832+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.387110+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.632946+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.416032+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:11:40.936 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:40.933+0000 7fb67a71d640 1 -- 172.21.15.92:0/1628641509 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fb66006ae40 msgr2=0x7fb66006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:40.936 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:40.933+0000 7fb67a71d640 1 --2- 172.21.15.92:0/1628641509 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fb66006ae40 0x7fb66006d300 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fb668007fd0 tx=0x7fb668009040 comp rx=0 tx=0).stop 2023-11-24T06:11:40.936 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:40.933+0000 7fb67a71d640 1 -- 172.21.15.92:0/1628641509 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb65c0051d0 msgr2=0x7fb6740aba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:40.936 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:40.933+0000 7fb67a71d640 1 --2- 172.21.15.92:0/1628641509 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb65c0051d0 0x7fb6740aba20 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fb66400d7a0 tx=0x7fb66400dc60 comp rx=0 tx=0).stop 2023-11-24T06:11:40.937 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:40.933+0000 7fb67a71d640 1 -- 172.21.15.92:0/1628641509 shutdown_connections 2023-11-24T06:11:40.937 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:40.933+0000 7fb67a71d640 1 --2- 172.21.15.92:0/1628641509 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fb66006ae40 0x7fb66006d300 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:40.937 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:40.933+0000 7fb67a71d640 1 --2- 172.21.15.92:0/1628641509 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb674131830 0x7fb674131cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:40.937 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:40.933+0000 7fb67a71d640 1 --2- 172.21.15.92:0/1628641509 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb6741408f0 0x7fb6740b0240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:40.937 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:40.933+0000 7fb67a71d640 1 --2- 172.21.15.92:0/1628641509 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb65c0051d0 0x7fb6740aba20 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:40.937 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:40.933+0000 7fb67a71d640 1 -- 172.21.15.92:0/1628641509 >> 172.21.15.92:0/1628641509 conn(0x7fb67406a800 msgr2=0x7fb67406c470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:40.937 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:40.933+0000 7fb67a71d640 1 -- 172.21.15.92:0/1628641509 shutdown_connections 2023-11-24T06:11:40.937 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:40.933+0000 7fb67a71d640 1 -- 172.21.15.92:0/1628641509 wait complete. 2023-11-24T06:11:40.954 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T06:10:17.733098+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T06:11:40.954 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T06:11:41.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:41.152+0000 7f80360d1640 1 Processor -- start 2023-11-24T06:11:41.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:41.153+0000 7f80360d1640 1 -- start start 2023-11-24T06:11:41.158 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:41.154+0000 7f80360d1640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f80300ccb50 0x7f8030070a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:41.158 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:41.154+0000 7f80360d1640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f803015bf10 0x7f8030143d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:41.158 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:41.154+0000 7f80360d1640 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f803015f4e0 con 0x7f80300adf20 2023-11-24T06:11:41.158 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:41.154+0000 7f80360d1640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f803015f970 con 0x7f80300ccb50 2023-11-24T06:11:41.158 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:41.154+0000 7f80360d1640 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8030071130 con 0x7f803015bf10 2023-11-24T06:11:41.158 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:41.155+0000 7f80348ce640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f80300ccb50 0x7f8030070a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:41.158 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:41.155+0000 7f80348ce640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f80300ccb50 0x7f8030070a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40340/0 (socket says 172.21.15.92:40340) 2023-11-24T06:11:41.159 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:41.155+0000 7f80348ce640 1 -- 172.21.15.92:0/713066105 learned_addr learned my addr 172.21.15.92:0/713066105 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:11:41.160 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:41.155+0000 7f80358d0640 1 --2- 172.21.15.92:0/713066105 >> v2:172.21.15.118:3300/0 conn(0x7f803015bf10 0x7f8030143d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:41.160 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:41.155+0000 7f80348ce640 1 -- 172.21.15.92:0/713066105 >> v2:172.21.15.118:3300/0 conn(0x7f803015bf10 msgr2=0x7f8030143d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:41.160 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:41.155+0000 7f80348ce640 1 --2- 172.21.15.92:0/713066105 >> v2:172.21.15.118:3300/0 conn(0x7f803015bf10 0x7f8030143d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:41.160 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:41.155+0000 7f80348ce640 1 -- 172.21.15.92:0/713066105 >> v1:172.21.15.118:6789/0 conn(0x7f80300adf20 legacy=0x7f80300cc440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:41.160 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:41.155+0000 7f80348ce640 1 -- 172.21.15.92:0/713066105 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f803015ebb0 con 0x7f80300ccb50 2023-11-24T06:11:41.160 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:41.156+0000 7f80348ce640 1 --2- 172.21.15.92:0/713066105 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f80300ccb50 0x7f8030070a30 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f803006ff20 tx=0x7f802002b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=504f4867c8fc92e server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:41.160 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:41.156+0000 7f8027fff640 1 -- 172.21.15.92:0/713066105 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8020031800 con 0x7f80300ccb50 2023-11-24T06:11:41.161 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:41.156+0000 7f8027fff640 1 -- 172.21.15.92:0/713066105 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8020031960 con 0x7f80300ccb50 2023-11-24T06:11:41.161 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:41.156+0000 7f8027fff640 1 -- 172.21.15.92:0/713066105 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8020031cf0 con 0x7f80300ccb50 2023-11-24T06:11:41.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:41.157+0000 7f80360d1640 1 -- 172.21.15.92:0/713066105 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f80300ccb50 msgr2=0x7f8030070a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:41.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:41.157+0000 7f80360d1640 1 --2- 172.21.15.92:0/713066105 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f80300ccb50 0x7f8030070a30 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f803006ff20 tx=0x7f802002b8a0 comp rx=0 tx=0).stop 2023-11-24T06:11:41.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:41.157+0000 7f80360d1640 1 -- 172.21.15.92:0/713066105 shutdown_connections 2023-11-24T06:11:41.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:41.157+0000 7f80360d1640 1 --2- 172.21.15.92:0/713066105 >> v2:172.21.15.118:3300/0 conn(0x7f803015bf10 0x7f8030143d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:41.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:41.157+0000 7f80360d1640 1 --2- 172.21.15.92:0/713066105 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f80300ccb50 0x7f8030070a30 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:41.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:41.157+0000 7f80360d1640 1 -- 172.21.15.92:0/713066105 >> 172.21.15.92:0/713066105 conn(0x7f803006a7d0 msgr2=0x7f803006abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:41.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:41.157+0000 7f80360d1640 1 -- 172.21.15.92:0/713066105 shutdown_connections 2023-11-24T06:11:41.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:41.157+0000 7f80360d1640 1 -- 172.21.15.92:0/713066105 wait complete. 2023-11-24T06:11:41.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:41.159+0000 7f80360d1640 1 Processor -- start 2023-11-24T06:11:41.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:41.159+0000 7f80360d1640 1 -- start start 2023-11-24T06:11:41.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:41.160+0000 7f80360d1640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f80300adf20 0x7f80300b3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:41.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:41.160+0000 7f80360d1640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f80300ccb50 0x7f803013dff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:41.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:41.160+0000 7f80360d1640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f803015bf10 0x7f80300afa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:41.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:41.160+0000 7f80360d1640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f803013e6f0 con 0x7f80300adf20 2023-11-24T06:11:41.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:41.160+0000 7f80360d1640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f803013e9d0 con 0x7f80300ccb50 2023-11-24T06:11:41.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:41.160+0000 7f80360d1640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f803013ecb0 con 0x7f803015bf10 2023-11-24T06:11:41.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:41.160+0000 7f80350cf640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f80300adf20 0x7f80300b3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:41.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:41.160+0000 7f80350cf640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f80300adf20 0x7f80300b3ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40344/0 (socket says 172.21.15.92:40344) 2023-11-24T06:11:41.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:41.160+0000 7f80350cf640 1 -- 172.21.15.92:0/1606460425 learned_addr learned my addr 172.21.15.92:0/1606460425 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:11:41.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:41.160+0000 7f80350cf640 1 -- 172.21.15.92:0/1606460425 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f803015bf10 msgr2=0x7f80300afa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:41.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:41.160+0000 7f80350cf640 1 --2- 172.21.15.92:0/1606460425 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f803015bf10 0x7f80300afa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:41.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:41.160+0000 7f80350cf640 1 -- 172.21.15.92:0/1606460425 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f80300ccb50 msgr2=0x7f803013dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:41.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:41.160+0000 7f80348ce640 1 --2- 172.21.15.92:0/1606460425 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f80300ccb50 0x7f803013dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:41.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:41.160+0000 7f80350cf640 1 --2- 172.21.15.92:0/1606460425 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f80300ccb50 0x7f803013dff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:41.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:41.161+0000 7f80350cf640 1 -- 172.21.15.92:0/1606460425 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80200094d0 con 0x7f80300adf20 2023-11-24T06:11:41.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:41.161+0000 7f80350cf640 1 --2- 172.21.15.92:0/1606460425 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f80300adf20 0x7f80300b3ac0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f802c00c410 tx=0x7f802c00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:41.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:41.161+0000 7f8025ffb640 1 -- 172.21.15.92:0/1606460425 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f802c0035f0 con 0x7f80300adf20 2023-11-24T06:11:41.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:41.161+0000 7f8025ffb640 1 -- 172.21.15.92:0/1606460425 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f802c003750 con 0x7f80300adf20 2023-11-24T06:11:41.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:41.161+0000 7f80360d1640 1 -- 172.21.15.92:0/1606460425 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f803013ef90 con 0x7f80300adf20 2023-11-24T06:11:41.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:41.161+0000 7f8025ffb640 1 -- 172.21.15.92:0/1606460425 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f802c003b40 con 0x7f80300adf20 2023-11-24T06:11:41.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:41.161+0000 7f80360d1640 1 -- 172.21.15.92:0/1606460425 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80301405d0 con 0x7f80300adf20 2023-11-24T06:11:41.170 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:41.162+0000 7f80360d1640 1 -- 172.21.15.92:0/1606460425 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8030067450 con 0x7f80300adf20 2023-11-24T06:11:41.170 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:41.163+0000 7f8025ffb640 1 -- 172.21.15.92:0/1606460425 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f802c003ca0 con 0x7f80300adf20 2023-11-24T06:11:41.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:41.164+0000 7f8025ffb640 1 --2- 172.21.15.92:0/1606460425 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f801406ae40 0x7f801406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:41.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:41.164+0000 7f8025ffb640 1 -- 172.21.15.92:0/1606460425 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f802c088950 con 0x7f80300adf20 2023-11-24T06:11:41.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:41.164+0000 7f80348ce640 1 --2- 172.21.15.92:0/1606460425 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f801406ae40 0x7f801406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:41.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:41.165+0000 7f80348ce640 1 --2- 172.21.15.92:0/1606460425 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f801406ae40 0x7f801406d300 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f8020036000 tx=0x7f802003c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:41.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:41.169+0000 7f8025ffb640 1 -- 172.21.15.92:0/1606460425 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f802c053d50 con 0x7f80300adf20 2023-11-24T06:11:41.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:41.343+0000 7f80360d1640 1 -- 172.21.15.92:0/1606460425 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f80300cd750 con 0x7f80300adf20 2023-11-24T06:11:41.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:41.343+0000 7f8025ffb640 1 -- 172.21.15.92:0/1606460425 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v29) v1 ==== 93+0+10 (secure 0 0 0) 0x7f802c017020 con 0x7f80300adf20 2023-11-24T06:11:41.347 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 1 2023-11-24T06:11:41.350 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:41.346+0000 7f80360d1640 1 -- 172.21.15.92:0/1606460425 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f801406ae40 msgr2=0x7f801406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:41.350 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:41.346+0000 7f80360d1640 1 --2- 172.21.15.92:0/1606460425 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f801406ae40 0x7f801406d300 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f8020036000 tx=0x7f802003c040 comp rx=0 tx=0).stop 2023-11-24T06:11:41.350 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:41.346+0000 7f80360d1640 1 -- 172.21.15.92:0/1606460425 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f80300adf20 msgr2=0x7f80300b3ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:41.350 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:41.346+0000 7f80360d1640 1 --2- 172.21.15.92:0/1606460425 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f80300adf20 0x7f80300b3ac0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f802c00c410 tx=0x7f802c00c8d0 comp rx=0 tx=0).stop 2023-11-24T06:11:41.351 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:41.346+0000 7f80360d1640 1 -- 172.21.15.92:0/1606460425 shutdown_connections 2023-11-24T06:11:41.351 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:41.346+0000 7f80360d1640 1 --2- 172.21.15.92:0/1606460425 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f801406ae40 0x7f801406d300 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:41.351 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:41.346+0000 7f80360d1640 1 --2- 172.21.15.92:0/1606460425 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f803015bf10 0x7f80300afa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:41.351 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:41.346+0000 7f80360d1640 1 --2- 172.21.15.92:0/1606460425 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f80300ccb50 0x7f803013dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:41.351 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:41.346+0000 7f80360d1640 1 --2- 172.21.15.92:0/1606460425 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f80300adf20 0x7f80300b3ac0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:41.351 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:41.346+0000 7f80360d1640 1 -- 172.21.15.92:0/1606460425 >> 172.21.15.92:0/1606460425 conn(0x7f803006a7d0 msgr2=0x7f803006cb30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:41.351 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:41.346+0000 7f80360d1640 1 -- 172.21.15.92:0/1606460425 shutdown_connections 2023-11-24T06:11:41.351 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:41.346+0000 7f80360d1640 1 -- 172.21.15.92:0/1606460425 wait complete. 2023-11-24T06:11:41.367 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-11-24T06:11:41.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:41.568+0000 7f88ba373640 1 Processor -- start 2023-11-24T06:11:41.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:41.568+0000 7f88ba373640 1 -- start start 2023-11-24T06:11:41.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:41.570+0000 7f88ba373640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f88b413df80 0x7f88b415bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:41.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:41.570+0000 7f88ba373640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f88b40cf9a0 0x7f88b40cd840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:41.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:41.570+0000 7f88ba373640 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88b406c470 con 0x7f88b413e360 2023-11-24T06:11:41.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:41.570+0000 7f88ba373640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88b406c750 con 0x7f88b40cf9a0 2023-11-24T06:11:41.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:41.570+0000 7f88ba373640 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88b406ca30 con 0x7f88b413df80 2023-11-24T06:11:41.574 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:41.570+0000 7f88b8910640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f88b40cf9a0 0x7f88b40cd840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:41.574 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:41.570+0000 7f88b8910640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f88b40cf9a0 0x7f88b40cd840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40346/0 (socket says 172.21.15.92:40346) 2023-11-24T06:11:41.574 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:41.570+0000 7f88b8910640 1 -- 172.21.15.92:0/894778123 learned_addr learned my addr 172.21.15.92:0/894778123 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:11:41.574 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:41.570+0000 7f88b3fff640 1 --2- 172.21.15.92:0/894778123 >> v2:172.21.15.118:3300/0 conn(0x7f88b413df80 0x7f88b415bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:41.575 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:41.571+0000 7f88b8910640 1 -- 172.21.15.92:0/894778123 >> v2:172.21.15.118:3300/0 conn(0x7f88b413df80 msgr2=0x7f88b415bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:41.575 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:41.571+0000 7f88b8910640 1 --2- 172.21.15.92:0/894778123 >> v2:172.21.15.118:3300/0 conn(0x7f88b413df80 0x7f88b415bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:41.575 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:41.571+0000 7f88b8910640 1 -- 172.21.15.92:0/894778123 >> v1:172.21.15.118:6789/0 conn(0x7f88b413e360 legacy=0x7f88b40cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:41.575 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:41.571+0000 7f88b8910640 1 -- 172.21.15.92:0/894778123 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88b406cd10 con 0x7f88b40cf9a0 2023-11-24T06:11:41.575 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:41.572+0000 7f88b8910640 1 --2- 172.21.15.92:0/894778123 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f88b40cf9a0 0x7f88b40cd840 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f88b4072770 tx=0x7f88ac02bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf9bbdd4352bc0b9 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:41.575 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:41.572+0000 7f88b2ffd640 1 -- 172.21.15.92:0/894778123 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88ac02cb20 con 0x7f88b40cf9a0 2023-11-24T06:11:41.576 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:41.572+0000 7f88b2ffd640 1 -- 172.21.15.92:0/894778123 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88ac02cc80 con 0x7f88b40cf9a0 2023-11-24T06:11:41.576 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:41.572+0000 7f88b2ffd640 1 -- 172.21.15.92:0/894778123 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88ac0343f0 con 0x7f88b40cf9a0 2023-11-24T06:11:41.576 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:41.573+0000 7f88ba373640 1 -- 172.21.15.92:0/894778123 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f88b40cf9a0 msgr2=0x7f88b40cd840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:41.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:41.573+0000 7f88ba373640 1 --2- 172.21.15.92:0/894778123 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f88b40cf9a0 0x7f88b40cd840 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f88b4072770 tx=0x7f88ac02bf80 comp rx=0 tx=0).stop 2023-11-24T06:11:41.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:41.573+0000 7f88ba373640 1 -- 172.21.15.92:0/894778123 shutdown_connections 2023-11-24T06:11:41.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:41.573+0000 7f88ba373640 1 --2- 172.21.15.92:0/894778123 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f88b40cf9a0 0x7f88b40cd840 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:41.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:41.573+0000 7f88ba373640 1 --2- 172.21.15.92:0/894778123 >> v2:172.21.15.118:3300/0 conn(0x7f88b413df80 0x7f88b415bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:41.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:41.573+0000 7f88ba373640 1 -- 172.21.15.92:0/894778123 >> 172.21.15.92:0/894778123 conn(0x7f88b406a800 msgr2=0x7f88b406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:41.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:41.573+0000 7f88ba373640 1 -- 172.21.15.92:0/894778123 shutdown_connections 2023-11-24T06:11:41.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:41.573+0000 7f88ba373640 1 -- 172.21.15.92:0/894778123 wait complete. 2023-11-24T06:11:41.578 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:41.575+0000 7f88ba373640 1 Processor -- start 2023-11-24T06:11:41.578 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:41.575+0000 7f88ba373640 1 -- start start 2023-11-24T06:11:41.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:41.576+0000 7f88ba373640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f88b40cf9a0 0x7f88b40b2fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:41.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:41.576+0000 7f88ba373640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f88b413df80 0x7f88b4153330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:41.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:41.576+0000 7f88ba373640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f88b413e360 0x7f88b40a8c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:41.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:41.576+0000 7f88ba373640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88b41f3090 con 0x7f88b413e360 2023-11-24T06:11:41.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:41.576+0000 7f88ba373640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88b4153870 con 0x7f88b413df80 2023-11-24T06:11:41.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:41.576+0000 7f88ba373640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88b41f31f0 con 0x7f88b40cf9a0 2023-11-24T06:11:41.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:41.576+0000 7f88b8910640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f88b413e360 0x7f88b40a8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:41.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:41.576+0000 7f88b8910640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f88b413e360 0x7f88b40a8c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40348/0 (socket says 172.21.15.92:40348) 2023-11-24T06:11:41.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:41.576+0000 7f88b8910640 1 -- 172.21.15.92:0/1636306383 learned_addr learned my addr 172.21.15.92:0/1636306383 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:11:41.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:41.576+0000 7f88b37fe640 1 --2- 172.21.15.92:0/1636306383 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f88b413df80 0x7f88b4153330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:41.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:41.576+0000 7f88b8910640 1 -- 172.21.15.92:0/1636306383 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f88b40cf9a0 msgr2=0x7f88b40b2fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:41.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:41.576+0000 7f88b3fff640 1 --2- 172.21.15.92:0/1636306383 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f88b40cf9a0 0x7f88b40b2fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:41.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:41.576+0000 7f88b8910640 1 --2- 172.21.15.92:0/1636306383 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f88b40cf9a0 0x7f88b40b2fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:41.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:41.576+0000 7f88b8910640 1 -- 172.21.15.92:0/1636306383 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f88b413df80 msgr2=0x7f88b4153330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:41.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:41.576+0000 7f88b8910640 1 --2- 172.21.15.92:0/1636306383 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f88b413df80 0x7f88b4153330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:41.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:41.576+0000 7f88b8910640 1 -- 172.21.15.92:0/1636306383 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88ac009b30 con 0x7f88b413e360 2023-11-24T06:11:41.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:41.576+0000 7f88b37fe640 1 --2- 172.21.15.92:0/1636306383 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f88b413df80 0x7f88b4153330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:11:41.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:41.577+0000 7f88b8910640 1 --2- 172.21.15.92:0/1636306383 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f88b413e360 0x7f88b40a8c40 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f88ac036040 tx=0x7f88ac004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:41.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:41.577+0000 7f88b0ff9640 1 -- 172.21.15.92:0/1636306383 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88ac007e70 con 0x7f88b413e360 2023-11-24T06:11:41.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:41.577+0000 7f88b0ff9640 1 -- 172.21.15.92:0/1636306383 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88ac009510 con 0x7f88b413e360 2023-11-24T06:11:41.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:41.577+0000 7f88ba373640 1 -- 172.21.15.92:0/1636306383 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88b40b3640 con 0x7f88b413e360 2023-11-24T06:11:41.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:41.577+0000 7f88b0ff9640 1 -- 172.21.15.92:0/1636306383 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88ac03e690 con 0x7f88b413e360 2023-11-24T06:11:41.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:41.577+0000 7f88ba373640 1 -- 172.21.15.92:0/1636306383 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88b40b3c00 con 0x7f88b413e360 2023-11-24T06:11:41.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:41.578+0000 7f88ba373640 1 -- 172.21.15.92:0/1636306383 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88b40675d0 con 0x7f88b413e360 2023-11-24T06:11:41.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:41.579+0000 7f88b0ff9640 1 -- 172.21.15.92:0/1636306383 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f88ac03e7f0 con 0x7f88b413e360 2023-11-24T06:11:41.584 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:41.581+0000 7f88b0ff9640 1 --2- 172.21.15.92:0/1636306383 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f88a006ae40 0x7f88a006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:41.584 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:41.581+0000 7f88b0ff9640 1 -- 172.21.15.92:0/1636306383 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f88ac0abea0 con 0x7f88b413e360 2023-11-24T06:11:41.584 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:41.581+0000 7f88b3fff640 1 --2- 172.21.15.92:0/1636306383 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f88a006ae40 0x7f88a006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:41.584 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:41.581+0000 7f88b3fff640 1 --2- 172.21.15.92:0/1636306383 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f88a006ae40 0x7f88a006d300 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f88a4006fd0 tx=0x7f88a4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:41.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:41.585+0000 7f88b0ff9640 1 -- 172.21.15.92:0/1636306383 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f88ac077290 con 0x7f88b413e360 2023-11-24T06:11:41.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:41.764+0000 7f88ba373640 1 -- 172.21.15.92:0/1636306383 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f88b41f3330 con 0x7f88b413e360 2023-11-24T06:11:41.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:41.765+0000 7f88b0ff9640 1 -- 172.21.15.92:0/1636306383 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v29) v1 ==== 84+0+65 (secure 0 0 0) 0x7f88ac07a8b0 con 0x7f88b413e360 2023-11-24T06:11:41.768 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-24T06:11:41.768 INFO:teuthology.orchestra.run.smithi092.stdout:[ 2023-11-24T06:11:41.768 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-24T06:11:41.768 INFO:teuthology.orchestra.run.smithi092.stdout: "poolnum": 1, 2023-11-24T06:11:41.769 INFO:teuthology.orchestra.run.smithi092.stdout: "poolname": ".mgr" 2023-11-24T06:11:41.769 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-24T06:11:41.769 INFO:teuthology.orchestra.run.smithi092.stdout:] 2023-11-24T06:11:41.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:41.767+0000 7f88ba373640 1 -- 172.21.15.92:0/1636306383 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f88a006ae40 msgr2=0x7f88a006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:41.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:41.767+0000 7f88ba373640 1 --2- 172.21.15.92:0/1636306383 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f88a006ae40 0x7f88a006d300 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f88a4006fd0 tx=0x7f88a4008040 comp rx=0 tx=0).stop 2023-11-24T06:11:41.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:41.767+0000 7f88ba373640 1 -- 172.21.15.92:0/1636306383 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f88b413e360 msgr2=0x7f88b40a8c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:41.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:41.767+0000 7f88ba373640 1 --2- 172.21.15.92:0/1636306383 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f88b413e360 0x7f88b40a8c40 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f88ac036040 tx=0x7f88ac004a40 comp rx=0 tx=0).stop 2023-11-24T06:11:41.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:41.767+0000 7f88ba373640 1 -- 172.21.15.92:0/1636306383 shutdown_connections 2023-11-24T06:11:41.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:41.767+0000 7f88ba373640 1 --2- 172.21.15.92:0/1636306383 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f88a006ae40 0x7f88a006d300 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:41.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:41.767+0000 7f88ba373640 1 --2- 172.21.15.92:0/1636306383 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f88b413e360 0x7f88b40a8c40 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:41.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:41.767+0000 7f88ba373640 1 --2- 172.21.15.92:0/1636306383 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f88b413df80 0x7f88b4153330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:41.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:41.767+0000 7f88ba373640 1 --2- 172.21.15.92:0/1636306383 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f88b40cf9a0 0x7f88b40b2fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:41.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:41.767+0000 7f88ba373640 1 -- 172.21.15.92:0/1636306383 >> 172.21.15.92:0/1636306383 conn(0x7f88b406a800 msgr2=0x7f88b40b0250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:41.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:41.768+0000 7f88ba373640 1 -- 172.21.15.92:0/1636306383 shutdown_connections 2023-11-24T06:11:41.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:41.768+0000 7f88ba373640 1 -- 172.21.15.92:0/1636306383 wait complete. 2023-11-24T06:11:41.789 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-11-24T06:11:41.789 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-11-24T06:11:41.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:41.995+0000 7f98f55f5640 1 Processor -- start 2023-11-24T06:11:41.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:41.995+0000 7f98f55f5640 1 -- start start 2023-11-24T06:11:42.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:41.996+0000 7f98f55f5640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f98f00cf9a0 0x7f98f015dca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:42.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:41.997+0000 7f98f55f5640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98f00b1ee0 con 0x7f98f0143ae0 2023-11-24T06:11:42.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:41.997+0000 7f98f55f5640 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98f00b2780 con 0x7f98f015e1e0 2023-11-24T06:11:42.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:41.997+0000 7f98f55f5640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98f015e990 con 0x7f98f00cf9a0 2023-11-24T06:11:42.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:41.997+0000 7f98eeffd640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f98f0143ae0 0x7f98f00cf290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:59692/0 (socket says 172.21.15.92:59692) 2023-11-24T06:11:42.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:41.997+0000 7f98eeffd640 1 -- 172.21.15.92:0/2347313818 learned_addr learned my addr 172.21.15.92:0/2347313818 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-24T06:11:42.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:41.997+0000 7f98e67fc640 1 --2- 172.21.15.92:0/2347313818 >> v2:172.21.15.92:3300/0 conn(0x7f98f00cf9a0 0x7f98f015dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:42.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:41.997+0000 7f98ee7fc640 1 -- 172.21.15.92:0/2347313818 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1196810447 0 0) 0x7f98f00b1ee0 con 0x7f98f0143ae0 2023-11-24T06:11:42.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:41.997+0000 7f98ee7fc640 1 -- 172.21.15.92:0/2347313818 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f98dc003520 con 0x7f98f0143ae0 2023-11-24T06:11:42.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:41.998+0000 7f98e67fc640 1 -- 172.21.15.92:0/2347313818 >> v1:172.21.15.118:6789/0 conn(0x7f98f015e1e0 legacy=0x7f98f00702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:42.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:41.998+0000 7f98e67fc640 1 -- 172.21.15.92:0/2347313818 >> v1:172.21.15.92:6789/0 conn(0x7f98f0143ae0 legacy=0x7f98f00cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:42.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:41.998+0000 7f98e67fc640 1 -- 172.21.15.92:0/2347313818 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98f00b2130 con 0x7f98f00cf9a0 2023-11-24T06:11:42.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:41.999+0000 7f98e67fc640 1 --2- 172.21.15.92:0/2347313818 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f98f00cf9a0 0x7f98f015dca0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f98d80097a0 tx=0x7f98d802b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fa4c56e16a90ff7 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:42.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:41.999+0000 7f98ee7fc640 1 -- 172.21.15.92:0/2347313818 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98d8030610 con 0x7f98f00cf9a0 2023-11-24T06:11:42.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:41.999+0000 7f98ee7fc640 1 -- 172.21.15.92:0/2347313818 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98d8030770 con 0x7f98f00cf9a0 2023-11-24T06:11:42.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:41.999+0000 7f98ee7fc640 1 -- 172.21.15.92:0/2347313818 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98d8030b00 con 0x7f98f00cf9a0 2023-11-24T06:11:42.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:42.000+0000 7f98f55f5640 1 -- 172.21.15.92:0/2347313818 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f98f00cf9a0 msgr2=0x7f98f015dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:42.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:42.000+0000 7f98f55f5640 1 --2- 172.21.15.92:0/2347313818 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f98f00cf9a0 0x7f98f015dca0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f98d80097a0 tx=0x7f98d802b960 comp rx=0 tx=0).stop 2023-11-24T06:11:42.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:42.000+0000 7f98f55f5640 1 -- 172.21.15.92:0/2347313818 shutdown_connections 2023-11-24T06:11:42.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:42.000+0000 7f98f55f5640 1 --2- 172.21.15.92:0/2347313818 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f98f00cf9a0 0x7f98f015dca0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:42.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:42.000+0000 7f98f55f5640 1 -- 172.21.15.92:0/2347313818 >> 172.21.15.92:0/2347313818 conn(0x7f98f006a800 msgr2=0x7f98f006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:42.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:42.000+0000 7f98f55f5640 1 -- 172.21.15.92:0/2347313818 shutdown_connections 2023-11-24T06:11:42.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:42.000+0000 7f98f55f5640 1 -- 172.21.15.92:0/2347313818 wait complete. 2023-11-24T06:11:42.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:42.002+0000 7f98f55f5640 1 Processor -- start 2023-11-24T06:11:42.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:42.002+0000 7f98f55f5640 1 -- start start 2023-11-24T06:11:42.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:42.003+0000 7f98f55f5640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f98f00cf9a0 0x7f98f0159870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:42.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:42.003+0000 7f98f55f5640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f98f0143ae0 0x7f98f00adae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:42.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:42.003+0000 7f98f55f5640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f98f015e1e0 0x7f98f013c190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:42.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:42.003+0000 7f98f55f5640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98f0138500 con 0x7f98f00cf9a0 2023-11-24T06:11:42.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:42.003+0000 7f98f55f5640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98f01378f0 con 0x7f98f0143ae0 2023-11-24T06:11:42.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:42.003+0000 7f98f55f5640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98f0137bd0 con 0x7f98f015e1e0 2023-11-24T06:11:42.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:42.003+0000 7f98eeffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f98f00cf9a0 0x7f98f0159870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:42.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:42.003+0000 7f98eeffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f98f00cf9a0 0x7f98f0159870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40370/0 (socket says 172.21.15.92:40370) 2023-11-24T06:11:42.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:42.003+0000 7f98eeffd640 1 -- 172.21.15.92:0/925748090 learned_addr learned my addr 172.21.15.92:0/925748090 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:11:42.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:42.004+0000 7f98eeffd640 1 -- 172.21.15.92:0/925748090 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f98f015e1e0 msgr2=0x7f98f013c190 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-11-24T06:11:42.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:42.004+0000 7f98eeffd640 1 --2- 172.21.15.92:0/925748090 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f98f015e1e0 0x7f98f013c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:42.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:42.004+0000 7f98eeffd640 1 -- 172.21.15.92:0/925748090 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f98f0143ae0 msgr2=0x7f98f00adae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:42.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:42.004+0000 7f98eeffd640 1 --2- 172.21.15.92:0/925748090 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f98f0143ae0 0x7f98f00adae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:42.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:42.004+0000 7f98eeffd640 1 -- 172.21.15.92:0/925748090 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98d8009400 con 0x7f98f00cf9a0 2023-11-24T06:11:42.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:42.004+0000 7f98eeffd640 1 --2- 172.21.15.92:0/925748090 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f98f00cf9a0 0x7f98f0159870 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f98e000cbc0 tx=0x7f98e000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:42.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:42.004+0000 7f98e7fff640 1 -- 172.21.15.92:0/925748090 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98e000ad40 con 0x7f98f00cf9a0 2023-11-24T06:11:42.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:42.004+0000 7f98e7fff640 1 -- 172.21.15.92:0/925748090 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98e0012400 con 0x7f98f00cf9a0 2023-11-24T06:11:42.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:42.004+0000 7f98e7fff640 1 -- 172.21.15.92:0/925748090 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98e000aea0 con 0x7f98f00cf9a0 2023-11-24T06:11:42.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:42.004+0000 7f98f55f5640 1 -- 172.21.15.92:0/925748090 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98f0137eb0 con 0x7f98f00cf9a0 2023-11-24T06:11:42.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:42.005+0000 7f98f55f5640 1 -- 172.21.15.92:0/925748090 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98f0159db0 con 0x7f98f00cf9a0 2023-11-24T06:11:42.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:42.006+0000 7f98f55f5640 1 -- 172.21.15.92:0/925748090 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98f00675d0 con 0x7f98f00cf9a0 2023-11-24T06:11:42.010 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:42.007+0000 7f98e7fff640 1 -- 172.21.15.92:0/925748090 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f98e0007070 con 0x7f98f00cf9a0 2023-11-24T06:11:42.010 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:42.007+0000 7f98e7fff640 1 --2- 172.21.15.92:0/925748090 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f98d006ae40 0x7f98d006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:42.010 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:42.008+0000 7f98e7fff640 1 -- 172.21.15.92:0/925748090 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f98e0089280 con 0x7f98f00cf9a0 2023-11-24T06:11:42.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:42.008+0000 7f98e67fc640 1 --2- 172.21.15.92:0/925748090 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f98d006ae40 0x7f98d006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:42.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:42.008+0000 7f98e67fc640 1 --2- 172.21.15.92:0/925748090 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f98d006ae40 0x7f98d006d300 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f98d8036000 tx=0x7f98d803b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:42.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:42.012+0000 7f98e7fff640 1 -- 172.21.15.92:0/925748090 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f98e0054730 con 0x7f98f00cf9a0 2023-11-24T06:11:42.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:42.187+0000 7f98f55f5640 1 -- 172.21.15.92:0/925748090 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f98f0073c30 con 0x7f98f00cf9a0 2023-11-24T06:11:43.020 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:43.016+0000 7f98e7fff640 1 -- 172.21.15.92:0/925748090 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v30) v1 ==== 138+0+0 (secure 0 0 0) 0x7f98f0073c30 con 0x7f98f00cf9a0 2023-11-24T06:11:43.020 INFO:teuthology.orchestra.run.smithi092.stderr:pool 'cephfs_metadata' created 2023-11-24T06:11:43.023 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:43.020+0000 7f98f55f5640 1 -- 172.21.15.92:0/925748090 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f98d006ae40 msgr2=0x7f98d006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:43.023 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:43.020+0000 7f98f55f5640 1 --2- 172.21.15.92:0/925748090 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f98d006ae40 0x7f98d006d300 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f98d8036000 tx=0x7f98d803b040 comp rx=0 tx=0).stop 2023-11-24T06:11:43.023 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:43.020+0000 7f98f55f5640 1 -- 172.21.15.92:0/925748090 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f98f00cf9a0 msgr2=0x7f98f0159870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:43.024 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:43.020+0000 7f98f55f5640 1 --2- 172.21.15.92:0/925748090 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f98f00cf9a0 0x7f98f0159870 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f98e000cbc0 tx=0x7f98e000a430 comp rx=0 tx=0).stop 2023-11-24T06:11:43.024 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:43.020+0000 7f98f55f5640 1 -- 172.21.15.92:0/925748090 shutdown_connections 2023-11-24T06:11:43.024 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:43.020+0000 7f98f55f5640 1 --2- 172.21.15.92:0/925748090 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f98d006ae40 0x7f98d006d300 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:43.024 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:43.020+0000 7f98f55f5640 1 --2- 172.21.15.92:0/925748090 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f98f015e1e0 0x7f98f013c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:43.024 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:43.020+0000 7f98f55f5640 1 --2- 172.21.15.92:0/925748090 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f98f0143ae0 0x7f98f00adae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:43.024 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:43.020+0000 7f98f55f5640 1 --2- 172.21.15.92:0/925748090 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f98f00cf9a0 0x7f98f0159870 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:43.024 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:43.020+0000 7f98f55f5640 1 -- 172.21.15.92:0/925748090 >> 172.21.15.92:0/925748090 conn(0x7f98f006a800 msgr2=0x7f98f015a4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:43.024 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:43.020+0000 7f98f55f5640 1 -- 172.21.15.92:0/925748090 shutdown_connections 2023-11-24T06:11:43.024 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:43.021+0000 7f98f55f5640 1 -- 172.21.15.92:0/925748090 wait complete. 2023-11-24T06:11:43.048 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-11-24T06:11:43.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:43.253+0000 7f3340f53640 1 Processor -- start 2023-11-24T06:11:43.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:43.254+0000 7f3340f53640 1 -- start start 2023-11-24T06:11:43.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:43.255+0000 7f3340f53640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f333c160c70 0x7f333c0cfbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:43.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:43.255+0000 7f3340f53640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f333c0b2cc0 con 0x7f333c0d1550 2023-11-24T06:11:43.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:43.255+0000 7f3340f53640 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f333c0d0530 con 0x7f333c0d0100 2023-11-24T06:11:43.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:43.255+0000 7f3340f53640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f333c0cf4c0 con 0x7f333c160c70 2023-11-24T06:11:43.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:43.255+0000 7f333a59c640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f333c160c70 0x7f333c0cfbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:43.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:43.256+0000 7f333a59c640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f333c160c70 0x7f333c0cfbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40380/0 (socket says 172.21.15.92:40380) 2023-11-24T06:11:43.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:43.256+0000 7f333a59c640 1 -- 172.21.15.92:0/1916255824 learned_addr learned my addr 172.21.15.92:0/1916255824 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:11:43.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:43.256+0000 7f333a59c640 1 -- 172.21.15.92:0/1916255824 >> v1:172.21.15.118:6789/0 conn(0x7f333c0d0100 legacy=0x7f333c0cedb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:43.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:43.256+0000 7f333a59c640 1 -- 172.21.15.92:0/1916255824 >> v1:172.21.15.92:6789/0 conn(0x7f333c0d1550 legacy=0x7f333c0b3150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:43.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:43.256+0000 7f333a59c640 1 -- 172.21.15.92:0/1916255824 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f333c161050 con 0x7f333c160c70 2023-11-24T06:11:43.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:43.257+0000 7f333a59c640 1 --2- 172.21.15.92:0/1916255824 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f333c160c70 0x7f333c0cfbc0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f332c009e50 tx=0x7f332c02bf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2169abcc3b75b28a server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:43.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:43.258+0000 7f333959a640 1 -- 172.21.15.92:0/1916255824 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f332c02cb10 con 0x7f333c160c70 2023-11-24T06:11:43.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:43.258+0000 7f333959a640 1 -- 172.21.15.92:0/1916255824 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f332c02cc70 con 0x7f333c160c70 2023-11-24T06:11:43.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:43.258+0000 7f333959a640 1 -- 172.21.15.92:0/1916255824 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f332c0343f0 con 0x7f333c160c70 2023-11-24T06:11:43.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:43.258+0000 7f3340f53640 1 -- 172.21.15.92:0/1916255824 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f333c160c70 msgr2=0x7f333c0cfbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:43.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:43.258+0000 7f3340f53640 1 --2- 172.21.15.92:0/1916255824 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f333c160c70 0x7f333c0cfbc0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f332c009e50 tx=0x7f332c02bf70 comp rx=0 tx=0).stop 2023-11-24T06:11:43.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:43.258+0000 7f3340f53640 1 -- 172.21.15.92:0/1916255824 shutdown_connections 2023-11-24T06:11:43.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:43.258+0000 7f3340f53640 1 --2- 172.21.15.92:0/1916255824 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f333c160c70 0x7f333c0cfbc0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:43.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:43.258+0000 7f3340f53640 1 -- 172.21.15.92:0/1916255824 >> 172.21.15.92:0/1916255824 conn(0x7f333c09c500 msgr2=0x7f333c0d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:43.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:43.258+0000 7f3340f53640 1 -- 172.21.15.92:0/1916255824 shutdown_connections 2023-11-24T06:11:43.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:43.258+0000 7f3340f53640 1 -- 172.21.15.92:0/1916255824 wait complete. 2023-11-24T06:11:43.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:43.260+0000 7f3340f53640 1 Processor -- start 2023-11-24T06:11:43.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:43.260+0000 7f3340f53640 1 -- start start 2023-11-24T06:11:43.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:43.261+0000 7f3340f53640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f333c0d0100 0x7f333c15bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:43.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:43.261+0000 7f3340f53640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f333c0d1550 0x7f333c0c7760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:43.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:43.261+0000 7f3340f53640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f333c160c70 0x7f333c0b6ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:43.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:43.261+0000 7f3340f53640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f333c0ca620 con 0x7f333c0d0100 2023-11-24T06:11:43.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:43.261+0000 7f3340f53640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f333c0ca900 con 0x7f333c160c70 2023-11-24T06:11:43.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:43.261+0000 7f3340f53640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f333c0cabe0 con 0x7f333c0d1550 2023-11-24T06:11:43.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:43.261+0000 7f333a59c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f333c0d0100 0x7f333c15bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:43.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:43.261+0000 7f333a59c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f333c0d0100 0x7f333c15bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40396/0 (socket says 172.21.15.92:40396) 2023-11-24T06:11:43.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:43.261+0000 7f333a59c640 1 -- 172.21.15.92:0/3748595208 learned_addr learned my addr 172.21.15.92:0/3748595208 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:11:43.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:43.262+0000 7f333a59c640 1 -- 172.21.15.92:0/3748595208 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f333c0d1550 msgr2=0x7f333c0c7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:43.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:43.262+0000 7f333a59c640 1 --2- 172.21.15.92:0/3748595208 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f333c0d1550 0x7f333c0c7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:43.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:43.262+0000 7f333a59c640 1 -- 172.21.15.92:0/3748595208 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f333c160c70 msgr2=0x7f333c0b6ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:43.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:43.262+0000 7f333ad9d640 1 --2- 172.21.15.92:0/3748595208 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f333c160c70 0x7f333c0b6ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:43.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:43.262+0000 7f333a59c640 1 --2- 172.21.15.92:0/3748595208 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f333c160c70 0x7f333c0b6ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:43.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:43.262+0000 7f333a59c640 1 -- 172.21.15.92:0/3748595208 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f332c009b30 con 0x7f333c0d0100 2023-11-24T06:11:43.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:43.262+0000 7f333a59c640 1 --2- 172.21.15.92:0/3748595208 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f333c0d0100 0x7f333c15bc90 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f332c009e50 tx=0x7f332c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:43.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:43.263+0000 7f3332ffd640 1 -- 172.21.15.92:0/3748595208 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f332c03e450 con 0x7f333c0d0100 2023-11-24T06:11:43.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:43.263+0000 7f3332ffd640 1 -- 172.21.15.92:0/3748595208 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f332c03e5f0 con 0x7f333c0d0100 2023-11-24T06:11:43.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:43.263+0000 7f3340f53640 1 -- 172.21.15.92:0/3748595208 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f333c0caec0 con 0x7f333c0d0100 2023-11-24T06:11:43.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:43.263+0000 7f3332ffd640 1 -- 172.21.15.92:0/3748595208 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f332c03e9a0 con 0x7f333c0d0100 2023-11-24T06:11:43.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:43.263+0000 7f3340f53640 1 -- 172.21.15.92:0/3748595208 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f333c0ca280 con 0x7f333c0d0100 2023-11-24T06:11:43.269 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:43.264+0000 7f3340f53640 1 -- 172.21.15.92:0/3748595208 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f333c0675d0 con 0x7f333c0d0100 2023-11-24T06:11:43.269 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:43.265+0000 7f3332ffd640 1 -- 172.21.15.92:0/3748595208 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f332c03eb00 con 0x7f333c0d0100 2023-11-24T06:11:43.269 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:43.266+0000 7f3332ffd640 1 --2- 172.21.15.92:0/3748595208 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f331c06ae40 0x7f331c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:43.269 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:43.266+0000 7f3332ffd640 1 -- 172.21.15.92:0/3748595208 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f332c0ac420 con 0x7f333c0d0100 2023-11-24T06:11:43.269 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:43.266+0000 7f3339d9b640 1 --2- 172.21.15.92:0/3748595208 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f331c06ae40 0x7f331c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:43.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:43.267+0000 7f3339d9b640 1 --2- 172.21.15.92:0/3748595208 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f331c06ae40 0x7f331c06d300 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f3324000f60 tx=0x7f33240079b0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:43.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:43.271+0000 7f3332ffd640 1 -- 172.21.15.92:0/3748595208 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f332c077860 con 0x7f333c0d0100 2023-11-24T06:11:43.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:43.467+0000 7f3340f53640 1 -- 172.21.15.92:0/3748595208 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f333c0b5750 con 0x7f333c0d0100 2023-11-24T06:11:44.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:44.032+0000 7f3332ffd640 1 -- 172.21.15.92:0/3748595208 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v31) v1 ==== 170+0+0 (secure 0 0 0) 0x7f332c03edb0 con 0x7f333c0d0100 2023-11-24T06:11:44.036 INFO:teuthology.orchestra.run.smithi092.stderr:pool 'cephfs_data' created 2023-11-24T06:11:44.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:44.036+0000 7f3340f53640 1 -- 172.21.15.92:0/3748595208 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f331c06ae40 msgr2=0x7f331c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:44.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:44.036+0000 7f3340f53640 1 --2- 172.21.15.92:0/3748595208 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f331c06ae40 0x7f331c06d300 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f3324000f60 tx=0x7f33240079b0 comp rx=0 tx=0).stop 2023-11-24T06:11:44.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:44.036+0000 7f3340f53640 1 -- 172.21.15.92:0/3748595208 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f333c0d0100 msgr2=0x7f333c15bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:44.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:44.036+0000 7f3340f53640 1 --2- 172.21.15.92:0/3748595208 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f333c0d0100 0x7f333c15bc90 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f332c009e50 tx=0x7f332c004a40 comp rx=0 tx=0).stop 2023-11-24T06:11:44.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:44.036+0000 7f3340f53640 1 -- 172.21.15.92:0/3748595208 shutdown_connections 2023-11-24T06:11:44.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:44.036+0000 7f3340f53640 1 --2- 172.21.15.92:0/3748595208 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f331c06ae40 0x7f331c06d300 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:44.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:44.036+0000 7f3340f53640 1 --2- 172.21.15.92:0/3748595208 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f333c160c70 0x7f333c0b6ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:44.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:44.036+0000 7f3340f53640 1 --2- 172.21.15.92:0/3748595208 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f333c0d1550 0x7f333c0c7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:44.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:44.036+0000 7f3340f53640 1 --2- 172.21.15.92:0/3748595208 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f333c0d0100 0x7f333c15bc90 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:44.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:44.036+0000 7f3340f53640 1 -- 172.21.15.92:0/3748595208 >> 172.21.15.92:0/3748595208 conn(0x7f333c09c500 msgr2=0x7f333c0ce5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:44.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:44.036+0000 7f3340f53640 1 -- 172.21.15.92:0/3748595208 shutdown_connections 2023-11-24T06:11:44.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:44.036+0000 7f3340f53640 1 -- 172.21.15.92:0/3748595208 wait complete. 2023-11-24T06:11:44.073 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-11-24T06:11:44.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:44.255+0000 7f3cd23a9640 1 Processor -- start 2023-11-24T06:11:44.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:44.255+0000 7f3cd23a9640 1 -- start start 2023-11-24T06:11:44.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:44.256+0000 7f3cd23a9640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3ccc150760 0x7f3ccc15d300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:44.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:44.257+0000 7f3cd23a9640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f3ccc15d840 0x7f3ccc0b2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:44.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:44.257+0000 7f3cd23a9640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ccc150ba0 con 0x7f3ccc072230 2023-11-24T06:11:44.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:44.257+0000 7f3cd23a9640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ccc072610 con 0x7f3ccc15d840 2023-11-24T06:11:44.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:44.257+0000 7f3cd23a9640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ccc15dcd0 con 0x7f3ccc150760 2023-11-24T06:11:44.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:44.257+0000 7f3ccbfff640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f3ccc072230 0x7f3ccc0cf290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:59706/0 (socket says 172.21.15.92:59706) 2023-11-24T06:11:44.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:44.257+0000 7f3ccbfff640 1 -- 172.21.15.92:0/3845245182 learned_addr learned my addr 172.21.15.92:0/3845245182 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-24T06:11:44.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:44.257+0000 7f3cd0946640 1 --2- 172.21.15.92:0/3845245182 >> v2:172.21.15.92:3300/0 conn(0x7f3ccc15d840 0x7f3ccc0b2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:44.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:44.257+0000 7f3ccb7fe640 1 --2- 172.21.15.92:0/3845245182 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3ccc150760 0x7f3ccc15d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:44.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:44.257+0000 7f3cd0946640 1 -- 172.21.15.92:0/3845245182 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3ccc150760 msgr2=0x7f3ccc15d300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:44.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:44.257+0000 7f3cd0946640 1 --2- 172.21.15.92:0/3845245182 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3ccc150760 0x7f3ccc15d300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:44.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:44.257+0000 7f3cd0946640 1 -- 172.21.15.92:0/3845245182 >> v1:172.21.15.92:6789/0 conn(0x7f3ccc072230 legacy=0x7f3ccc0cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:44.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:44.257+0000 7f3cd0946640 1 -- 172.21.15.92:0/3845245182 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ccc071190 con 0x7f3ccc15d840 2023-11-24T06:11:44.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:44.257+0000 7f3ccaffd640 1 -- 172.21.15.92:0/3845245182 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 183674719 0 0) 0x7f3ccc150ba0 con 0x7f3ccc072230 2023-11-24T06:11:44.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:44.257+0000 7f3ccb7fe640 1 --2- 172.21.15.92:0/3845245182 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3ccc150760 0x7f3ccc15d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:11:44.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:44.259+0000 7f3cd0946640 1 --2- 172.21.15.92:0/3845245182 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3ccc15d840 0x7f3ccc0b2bd0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f3cc4009d90 tx=0x7f3cc402beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46937972ddc80196 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:44.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:44.259+0000 7f3ccaffd640 1 -- 172.21.15.92:0/3845245182 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cc402cc60 con 0x7f3ccc15d840 2023-11-24T06:11:44.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:44.259+0000 7f3ccaffd640 1 -- 172.21.15.92:0/3845245182 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cc402cdc0 con 0x7f3ccc15d840 2023-11-24T06:11:44.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:44.259+0000 7f3ccaffd640 1 -- 172.21.15.92:0/3845245182 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cc4034660 con 0x7f3ccc15d840 2023-11-24T06:11:44.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:44.259+0000 7f3cd23a9640 1 -- 172.21.15.92:0/3845245182 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3ccc15d840 msgr2=0x7f3ccc0b2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:44.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:44.259+0000 7f3cd23a9640 1 --2- 172.21.15.92:0/3845245182 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3ccc15d840 0x7f3ccc0b2bd0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f3cc4009d90 tx=0x7f3cc402beb0 comp rx=0 tx=0).stop 2023-11-24T06:11:44.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:44.259+0000 7f3cd23a9640 1 -- 172.21.15.92:0/3845245182 shutdown_connections 2023-11-24T06:11:44.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:44.259+0000 7f3cd23a9640 1 --2- 172.21.15.92:0/3845245182 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3ccc15d840 0x7f3ccc0b2bd0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:44.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:44.259+0000 7f3cd23a9640 1 --2- 172.21.15.92:0/3845245182 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3ccc150760 0x7f3ccc15d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:44.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:44.259+0000 7f3cd23a9640 1 -- 172.21.15.92:0/3845245182 >> 172.21.15.92:0/3845245182 conn(0x7f3ccc06a800 msgr2=0x7f3ccc06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:44.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:44.260+0000 7f3cd23a9640 1 -- 172.21.15.92:0/3845245182 shutdown_connections 2023-11-24T06:11:44.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:44.260+0000 7f3cd23a9640 1 -- 172.21.15.92:0/3845245182 wait complete. 2023-11-24T06:11:44.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:44.261+0000 7f3cd23a9640 1 Processor -- start 2023-11-24T06:11:44.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:44.261+0000 7f3cd23a9640 1 -- start start 2023-11-24T06:11:44.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:44.262+0000 7f3cd23a9640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3ccc072230 0x7f3ccc159a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:44.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:44.262+0000 7f3cd23a9640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3ccc150760 0x7f3ccc158290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:44.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:44.262+0000 7f3cd23a9640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3ccc15d840 0x7f3ccc1425a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:44.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:44.262+0000 7f3cd23a9640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ccc156350 con 0x7f3ccc072230 2023-11-24T06:11:44.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:44.262+0000 7f3cd23a9640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ccc158990 con 0x7f3ccc150760 2023-11-24T06:11:44.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:44.262+0000 7f3cd23a9640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ccc158c70 con 0x7f3ccc15d840 2023-11-24T06:11:44.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:44.263+0000 7f3ccbfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3ccc072230 0x7f3ccc159a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:44.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:44.263+0000 7f3ccbfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3ccc072230 0x7f3ccc159a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40410/0 (socket says 172.21.15.92:40410) 2023-11-24T06:11:44.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:44.263+0000 7f3ccbfff640 1 -- 172.21.15.92:0/708517881 learned_addr learned my addr 172.21.15.92:0/708517881 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:11:44.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:44.263+0000 7f3ccbfff640 1 -- 172.21.15.92:0/708517881 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3ccc15d840 msgr2=0x7f3ccc1425a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:44.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:44.263+0000 7f3ccbfff640 1 --2- 172.21.15.92:0/708517881 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3ccc15d840 0x7f3ccc1425a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:44.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:44.263+0000 7f3ccbfff640 1 -- 172.21.15.92:0/708517881 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3ccc150760 msgr2=0x7f3ccc158290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:44.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:44.263+0000 7f3ccb7fe640 1 --2- 172.21.15.92:0/708517881 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3ccc150760 0x7f3ccc158290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:44.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:44.263+0000 7f3ccbfff640 1 --2- 172.21.15.92:0/708517881 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3ccc150760 0x7f3ccc158290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:44.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:44.263+0000 7f3ccbfff640 1 -- 172.21.15.92:0/708517881 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cc4009a70 con 0x7f3ccc072230 2023-11-24T06:11:44.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:44.263+0000 7f3ccbfff640 1 --2- 172.21.15.92:0/708517881 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3ccc072230 0x7f3ccc159a10 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f3cbc00c4d0 tx=0x7f3cbc00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:44.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:44.264+0000 7f3cc8ff9640 1 -- 172.21.15.92:0/708517881 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cbc0074e0 con 0x7f3ccc072230 2023-11-24T06:11:44.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:44.264+0000 7f3cc8ff9640 1 -- 172.21.15.92:0/708517881 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cbc007660 con 0x7f3ccc072230 2023-11-24T06:11:44.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:44.264+0000 7f3cc8ff9640 1 -- 172.21.15.92:0/708517881 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cbc004a20 con 0x7f3ccc072230 2023-11-24T06:11:44.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:44.264+0000 7f3cd23a9640 1 -- 172.21.15.92:0/708517881 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ccc158f50 con 0x7f3ccc072230 2023-11-24T06:11:44.269 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:44.264+0000 7f3cd23a9640 1 -- 172.21.15.92:0/708517881 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ccc159510 con 0x7f3ccc072230 2023-11-24T06:11:44.269 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:44.265+0000 7f3cd23a9640 1 -- 172.21.15.92:0/708517881 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ccc0675d0 con 0x7f3ccc072230 2023-11-24T06:11:44.269 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:44.266+0000 7f3cc8ff9640 1 -- 172.21.15.92:0/708517881 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f3cbc004b80 con 0x7f3ccc072230 2023-11-24T06:11:44.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:44.267+0000 7f3cc8ff9640 1 --2- 172.21.15.92:0/708517881 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f3cb806ae40 0x7f3cb806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:44.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:44.267+0000 7f3cc8ff9640 1 -- 172.21.15.92:0/708517881 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f3cbc088750 con 0x7f3ccc072230 2023-11-24T06:11:44.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:44.267+0000 7f3ccb7fe640 1 --2- 172.21.15.92:0/708517881 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f3cb806ae40 0x7f3cb806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:44.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:44.268+0000 7f3ccb7fe640 1 --2- 172.21.15.92:0/708517881 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f3cb806ae40 0x7f3cb806d300 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f3ccc0b35d0 tx=0x7f3cc0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:44.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:44.272+0000 7f3cc8ff9640 1 -- 172.21.15.92:0/708517881 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3cbc053860 con 0x7f3ccc072230 2023-11-24T06:11:44.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:44.500+0000 7f3cd23a9640 1 -- 172.21.15.92:0/708517881 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f3ccc06d570 con 0x7f3ccc072230 2023-11-24T06:11:45.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:45.485+0000 7f3cc8ff9640 1 -- 172.21.15.92:0/708517881 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 4 and data pool 5 v13) v1 ==== 172+0+0 (secure 0 0 0) 0x7f3cbc0180c0 con 0x7f3ccc072230 2023-11-24T06:11:45.489 INFO:teuthology.orchestra.run.smithi092.stderr:new fs with metadata pool 4 and data pool 5 2023-11-24T06:11:45.492 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:45.489+0000 7f3cd23a9640 1 -- 172.21.15.92:0/708517881 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f3cb806ae40 msgr2=0x7f3cb806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:45.492 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:45.489+0000 7f3cd23a9640 1 --2- 172.21.15.92:0/708517881 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f3cb806ae40 0x7f3cb806d300 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f3ccc0b35d0 tx=0x7f3cc0008040 comp rx=0 tx=0).stop 2023-11-24T06:11:45.492 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:45.489+0000 7f3cd23a9640 1 -- 172.21.15.92:0/708517881 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3ccc072230 msgr2=0x7f3ccc159a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:45.492 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:45.489+0000 7f3cd23a9640 1 --2- 172.21.15.92:0/708517881 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3ccc072230 0x7f3ccc159a10 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f3cbc00c4d0 tx=0x7f3cbc00c990 comp rx=0 tx=0).stop 2023-11-24T06:11:45.493 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:45.489+0000 7f3cd23a9640 1 -- 172.21.15.92:0/708517881 shutdown_connections 2023-11-24T06:11:45.493 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:45.489+0000 7f3cd23a9640 1 --2- 172.21.15.92:0/708517881 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f3cb806ae40 0x7f3cb806d300 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:45.493 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:45.489+0000 7f3cd23a9640 1 --2- 172.21.15.92:0/708517881 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3ccc15d840 0x7f3ccc1425a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:45.493 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:45.489+0000 7f3cd23a9640 1 --2- 172.21.15.92:0/708517881 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3ccc150760 0x7f3ccc158290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:45.493 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:45.489+0000 7f3cd23a9640 1 --2- 172.21.15.92:0/708517881 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3ccc072230 0x7f3ccc159a10 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:45.493 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:45.489+0000 7f3cd23a9640 1 -- 172.21.15.92:0/708517881 >> 172.21.15.92:0/708517881 conn(0x7f3ccc06a800 msgr2=0x7f3ccc0ce340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:45.493 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:45.489+0000 7f3cd23a9640 1 -- 172.21.15.92:0/708517881 shutdown_connections 2023-11-24T06:11:45.493 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:45.489+0000 7f3cd23a9640 1 -- 172.21.15.92:0/708517881 wait complete. 2023-11-24T06:11:45.517 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:11:45.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:45.697+0000 7fab4359e640 1 Processor -- start 2023-11-24T06:11:45.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:45.697+0000 7fab4359e640 1 -- start start 2023-11-24T06:11:45.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:45.698+0000 7fab4359e640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fab440cc500 0x7fab440cc990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:45.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:45.698+0000 7fab4359e640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab44070e50 con 0x7fab4406d160 2023-11-24T06:11:45.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:45.698+0000 7fab4359e640 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab44144020 con 0x7fab44143c40 2023-11-24T06:11:45.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:45.698+0000 7fab4359e640 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab440723c0 con 0x7fab440cc500 2023-11-24T06:11:45.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:45.698+0000 7fab42d9d640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fab4406d160 0x7fab4406d560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:59710/0 (socket says 172.21.15.92:59710) 2023-11-24T06:11:45.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:45.698+0000 7fab42d9d640 1 -- 172.21.15.92:0/4268252057 learned_addr learned my addr 172.21.15.92:0/4268252057 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-24T06:11:45.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:45.699+0000 7fab39d9b640 1 --2- 172.21.15.92:0/4268252057 >> v2:172.21.15.118:3300/0 conn(0x7fab440cc500 0x7fab440cc990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:45.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:45.699+0000 7fab41d9b640 1 -- 172.21.15.92:0/4268252057 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1841717922 0 0) 0x7fab44070e50 con 0x7fab4406d160 2023-11-24T06:11:45.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:45.699+0000 7fab41d9b640 1 -- 172.21.15.92:0/4268252057 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fab30003520 con 0x7fab4406d160 2023-11-24T06:11:45.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:45.699+0000 7fab41d9b640 1 -- 172.21.15.92:0/4268252057 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3360718923 0 0) 0x7fab30003520 con 0x7fab4406d160 2023-11-24T06:11:45.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:45.699+0000 7fab41d9b640 1 -- 172.21.15.92:0/4268252057 >> v2:172.21.15.118:3300/0 conn(0x7fab440cc500 msgr2=0x7fab440cc990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:45.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:45.699+0000 7fab41d9b640 1 --2- 172.21.15.92:0/4268252057 >> v2:172.21.15.118:3300/0 conn(0x7fab440cc500 0x7fab440cc990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:45.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:45.699+0000 7fab41d9b640 1 -- 172.21.15.92:0/4268252057 >> v1:172.21.15.118:6789/0 conn(0x7fab44143c40 legacy=0x7fab44071290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:45.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:45.699+0000 7fab41d9b640 1 -- 172.21.15.92:0/4268252057 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab44071e00 con 0x7fab4406d160 2023-11-24T06:11:45.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:45.699+0000 7fab41d9b640 1 -- 172.21.15.92:0/4268252057 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2607661350 0 0) 0x7fab3c001670 con 0x7fab4406d160 2023-11-24T06:11:45.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:45.699+0000 7fab41d9b640 1 -- 172.21.15.92:0/4268252057 >> v1:172.21.15.92:6789/0 conn(0x7fab4406d160 legacy=0x7fab4406d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:45.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:45.699+0000 7fab41d9b640 1 --2- 172.21.15.92:0/4268252057 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fab30003970 0x7fab30003d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:45.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:45.699+0000 7fab41d9b640 1 --2- 172.21.15.92:0/4268252057 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fab300047c0 0x7fab30004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:45.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:45.699+0000 7fab41d9b640 1 --2- 172.21.15.92:0/4268252057 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fab300052a0 0x7fab3000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:45.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:45.699+0000 7fab41d9b640 1 -- 172.21.15.92:0/4268252057 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab440723c0 con 0x7fab300047c0 2023-11-24T06:11:45.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:45.699+0000 7fab41d9b640 1 -- 172.21.15.92:0/4268252057 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab3000c150 con 0x7fab30003970 2023-11-24T06:11:45.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:45.699+0000 7fab41d9b640 1 -- 172.21.15.92:0/4268252057 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab3000c310 con 0x7fab300052a0 2023-11-24T06:11:45.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:45.700+0000 7fab39d9b640 1 --2- 172.21.15.92:0/4268252057 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fab300047c0 0x7fab30004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:45.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:45.700+0000 7fab4259c640 1 --2- 172.21.15.92:0/4268252057 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fab30003970 0x7fab30003d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:45.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:45.700+0000 7fab39d9b640 1 -- 172.21.15.92:0/4268252057 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fab300052a0 msgr2=0x7fab3000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:45.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:45.700+0000 7fab42d9d640 1 --2- 172.21.15.92:0/4268252057 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fab300052a0 0x7fab3000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:45.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:45.700+0000 7fab39d9b640 1 --2- 172.21.15.92:0/4268252057 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fab300052a0 0x7fab3000b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:45.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:45.700+0000 7fab39d9b640 1 -- 172.21.15.92:0/4268252057 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fab30003970 msgr2=0x7fab30003d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:45.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:45.700+0000 7fab39d9b640 1 --2- 172.21.15.92:0/4268252057 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fab30003970 0x7fab30003d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:45.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:45.700+0000 7fab39d9b640 1 -- 172.21.15.92:0/4268252057 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab30003520 con 0x7fab300047c0 2023-11-24T06:11:45.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:45.700+0000 7fab42d9d640 1 --2- 172.21.15.92:0/4268252057 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fab300052a0 0x7fab3000b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:11:45.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:45.700+0000 7fab42d9d640 1 -- 172.21.15.92:0/4268252057 reap_dead start 2023-11-24T06:11:45.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:45.701+0000 7fab39d9b640 1 --2- 172.21.15.92:0/4268252057 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fab300047c0 0x7fab30004c40 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fab2c00ba00 tx=0x7fab300057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fb0f4079608bf9a server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:45.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:45.701+0000 7fab41d9b640 1 -- 172.21.15.92:0/4268252057 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab2c0325f0 con 0x7fab300047c0 2023-11-24T06:11:45.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:45.701+0000 7fab41d9b640 1 -- 172.21.15.92:0/4268252057 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab2c032750 con 0x7fab300047c0 2023-11-24T06:11:45.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:45.701+0000 7fab41d9b640 1 -- 172.21.15.92:0/4268252057 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab2c032ae0 con 0x7fab300047c0 2023-11-24T06:11:45.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:45.702+0000 7fab4359e640 1 -- 172.21.15.92:0/4268252057 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fab300047c0 msgr2=0x7fab30004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:45.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:45.702+0000 7fab4359e640 1 --2- 172.21.15.92:0/4268252057 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fab300047c0 0x7fab30004c40 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fab2c00ba00 tx=0x7fab300057f0 comp rx=0 tx=0).stop 2023-11-24T06:11:45.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:45.702+0000 7fab4359e640 1 -- 172.21.15.92:0/4268252057 shutdown_connections 2023-11-24T06:11:45.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:45.702+0000 7fab4359e640 1 --2- 172.21.15.92:0/4268252057 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fab300047c0 0x7fab30004c40 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:45.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:45.702+0000 7fab4359e640 1 -- 172.21.15.92:0/4268252057 >> 172.21.15.92:0/4268252057 conn(0x7fab4406a800 msgr2=0x7fab4406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:45.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:45.702+0000 7fab4359e640 1 -- 172.21.15.92:0/4268252057 shutdown_connections 2023-11-24T06:11:45.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:45.702+0000 7fab4359e640 1 -- 172.21.15.92:0/4268252057 wait complete. 2023-11-24T06:11:45.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:45.704+0000 7fab4359e640 1 Processor -- start 2023-11-24T06:11:45.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:45.704+0000 7fab4359e640 1 -- start start 2023-11-24T06:11:45.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:45.705+0000 7fab4359e640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fab300047c0 0x7fab4415c270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:45.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:45.705+0000 7fab4359e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fab4415d750 0x7fab4415a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:45.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:45.705+0000 7fab4359e640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fab44142280 0x7fab44142700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:45.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:45.705+0000 7fab4359e640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab4415c7b0 con 0x7fab4415d750 2023-11-24T06:11:45.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:45.705+0000 7fab4359e640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab44142c40 con 0x7fab44142280 2023-11-24T06:11:45.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:45.705+0000 7fab4359e640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab4415c9c0 con 0x7fab300047c0 2023-11-24T06:11:45.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:45.705+0000 7fab39d9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fab4415d750 0x7fab4415a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:45.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:45.705+0000 7fab39d9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fab4415d750 0x7fab4415a6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40416/0 (socket says 172.21.15.92:40416) 2023-11-24T06:11:45.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:45.705+0000 7fab39d9b640 1 -- 172.21.15.92:0/1394604173 learned_addr learned my addr 172.21.15.92:0/1394604173 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:11:45.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:45.706+0000 7fab39d9b640 1 -- 172.21.15.92:0/1394604173 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fab300047c0 msgr2=0x7fab4415c270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:45.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:45.706+0000 7fab4259c640 1 --2- 172.21.15.92:0/1394604173 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fab300047c0 0x7fab4415c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:45.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:45.706+0000 7fab39d9b640 1 --2- 172.21.15.92:0/1394604173 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fab300047c0 0x7fab4415c270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:45.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:45.706+0000 7fab39d9b640 1 -- 172.21.15.92:0/1394604173 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fab44142280 msgr2=0x7fab44142700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:45.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:45.706+0000 7fab39d9b640 1 --2- 172.21.15.92:0/1394604173 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fab44142280 0x7fab44142700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:45.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:45.706+0000 7fab39d9b640 1 -- 172.21.15.92:0/1394604173 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab4415cca0 con 0x7fab4415d750 2023-11-24T06:11:45.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:45.706+0000 7fab39d9b640 1 --2- 172.21.15.92:0/1394604173 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fab4415d750 0x7fab4415a6a0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fab2c00bb20 tx=0x7fab2c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:45.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:45.706+0000 7fab3b7fe640 1 -- 172.21.15.92:0/1394604173 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab2c03ca80 con 0x7fab4415d750 2023-11-24T06:11:45.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:45.706+0000 7fab3b7fe640 1 -- 172.21.15.92:0/1394604173 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab2c03cbe0 con 0x7fab4415d750 2023-11-24T06:11:45.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:45.706+0000 7fab3b7fe640 1 -- 172.21.15.92:0/1394604173 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab2c0463f0 con 0x7fab4415d750 2023-11-24T06:11:45.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:45.706+0000 7fab4359e640 1 -- 172.21.15.92:0/1394604173 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab440b1d90 con 0x7fab4415d750 2023-11-24T06:11:45.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:45.707+0000 7fab4359e640 1 -- 172.21.15.92:0/1394604173 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab4415dbd0 con 0x7fab4415d750 2023-11-24T06:11:45.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:45.708+0000 7fab4359e640 1 -- 172.21.15.92:0/1394604173 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab4415cee0 con 0x7fab4415d750 2023-11-24T06:11:45.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:45.708+0000 7fab3b7fe640 1 -- 172.21.15.92:0/1394604173 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fab2c04f450 con 0x7fab4415d750 2023-11-24T06:11:45.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:45.709+0000 7fab3b7fe640 1 --2- 172.21.15.92:0/1394604173 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fab2406ae40 0x7fab2406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:45.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:45.709+0000 7fab3b7fe640 1 -- 172.21.15.92:0/1394604173 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fab2c0abca0 con 0x7fab4415d750 2023-11-24T06:11:45.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:45.710+0000 7fab4259c640 1 --2- 172.21.15.92:0/1394604173 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fab2406ae40 0x7fab2406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:45.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:45.710+0000 7fab4259c640 1 --2- 172.21.15.92:0/1394604173 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fab2406ae40 0x7fab2406d300 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fab34007fd0 tx=0x7fab34009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:45.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:45.714+0000 7fab3b7fe640 1 -- 172.21.15.92:0/1394604173 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fab4415cee0 con 0x7fab4415d750 2023-11-24T06:11:45.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:45.898+0000 7fab4359e640 1 -- 172.21.15.92:0/1394604173 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fab440b1bf0 con 0x7fab4415d750 2023-11-24T06:11:45.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:45.900+0000 7fab3b7fe640 1 -- 172.21.15.92:0/1394604173 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) v1 ==== 74+0+16534 (secure 0 0 0) 0x7fab440b1bf0 con 0x7fab4415d750 2023-11-24T06:11:45.904 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-24T06:11:45.904 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":32,"fsid":"2d6dee5f-8aa3-470f-931a-50ade963b1cb","created":"2023-11-24T06:10:04.707204+0000","modified":"2023-11-24T06:11:44.502503+0000","last_up_change":"2023-11-24T06:10:17.598352+0000","last_in_change":"2023-11-24T06:10:09.040793+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:10:17.733098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:11:42.241681+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-11-24T06:11:43.522720+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ff37a9fb-135f-471f-b4b0-f7b5772006c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6803","nonce":20560714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6805","nonce":20560714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6809","nonce":20560714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6807","nonce":20560714}]},"public_addr":"172.21.15.92:6803/20560714","cluster_addr":"172.21.15.92:6805/20560714","heartbeat_back_addr":"172.21.15.92:6809/20560714","heartbeat_front_addr":"172.21.15.92:6807/20560714","state":["exists","up"]},{"osd":1,"uuid":"03cdd353-dcf1-4f38-94fe-a795137c06b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6819","nonce":2451239906}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6821","nonce":2451239906}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6825","nonce":2451239906}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6823","nonce":2451239906}]},"public_addr":"172.21.15.92:6819/2451239906","cluster_addr":"172.21.15.92:6821/2451239906","heartbeat_back_addr":"172.21.15.92:6825/2451239906","heartbeat_front_addr":"172.21.15.92:6823/2451239906","state":["exists","up"]},{"osd":2,"uuid":"a4a20d80-5d57-484b-becd-9aa8070dd3aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6811","nonce":1124979277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6813","nonce":1124979277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6817","nonce":1124979277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6815","nonce":1124979277}]},"public_addr":"172.21.15.92:6811/1124979277","cluster_addr":"172.21.15.92:6813/1124979277","heartbeat_back_addr":"172.21.15.92:6817/1124979277","heartbeat_front_addr":"172.21.15.92:6815/1124979277","state":["exists","up"]},{"osd":3,"uuid":"4648d7ce-9b29-4583-9425-4ba8fd2b1903","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6827","nonce":1734115782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6829","nonce":1734115782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6833","nonce":1734115782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6831","nonce":1734115782}]},"public_addr":"172.21.15.92:6827/1734115782","cluster_addr":"172.21.15.92:6829/1734115782","heartbeat_back_addr":"172.21.15.92:6833/1734115782","heartbeat_front_addr":"172.21.15.92:6831/1734115782","state":["exists","up"]},{"osd":4,"uuid":"3bc9d7eb-c6fa-4fcf-a3ec-ade604e05aac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6819","nonce":3425937229}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6822","nonce":3425937229}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6830","nonce":3425937229}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6826","nonce":3425937229}]},"public_addr":"172.21.15.118:6819/3425937229","cluster_addr":"172.21.15.118:6822/3425937229","heartbeat_back_addr":"172.21.15.118:6830/3425937229","heartbeat_front_addr":"172.21.15.118:6826/3425937229","state":["exists","up"]},{"osd":5,"uuid":"dce9ba6d-59ed-46d1-84fc-a84b13de10c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6801","nonce":827455984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6803","nonce":827455984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6807","nonce":827455984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6805","nonce":827455984}]},"public_addr":"172.21.15.118:6801/827455984","cluster_addr":"172.21.15.118:6803/827455984","heartbeat_back_addr":"172.21.15.118:6807/827455984","heartbeat_front_addr":"172.21.15.118:6805/827455984","state":["exists","up"]},{"osd":6,"uuid":"82aec504-ae58-4709-8d30-46f2b12081b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6809","nonce":2946986141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6811","nonce":2946986141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6815","nonce":2946986141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6813","nonce":2946986141}]},"public_addr":"172.21.15.118:6809/2946986141","cluster_addr":"172.21.15.118:6811/2946986141","heartbeat_back_addr":"172.21.15.118:6815/2946986141","heartbeat_front_addr":"172.21.15.118:6813/2946986141","state":["exists","up"]},{"osd":7,"uuid":"4ed4c624-c88f-4d35-bdb1-bf209b5dffb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6817","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6818","nonce":48122059}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6821","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6823","nonce":48122059}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6829","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6831","nonce":48122059}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6825","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6827","nonce":48122059}]},"public_addr":"172.21.15.118:6818/48122059","cluster_addr":"172.21.15.118:6823/48122059","heartbeat_back_addr":"172.21.15.118:6831/48122059","heartbeat_front_addr":"172.21.15.118:6827/48122059","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.536664+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.368324+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.271201+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.342971+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.621832+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.387110+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.632946+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.416032+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:11:45.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:45.903+0000 7fab4359e640 1 -- 172.21.15.92:0/1394604173 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fab2406ae40 msgr2=0x7fab2406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:45.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:45.903+0000 7fab4359e640 1 --2- 172.21.15.92:0/1394604173 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fab2406ae40 0x7fab2406d300 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fab34007fd0 tx=0x7fab34009040 comp rx=0 tx=0).stop 2023-11-24T06:11:45.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:45.903+0000 7fab4359e640 1 -- 172.21.15.92:0/1394604173 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fab4415d750 msgr2=0x7fab4415a6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:45.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:45.903+0000 7fab4359e640 1 --2- 172.21.15.92:0/1394604173 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fab4415d750 0x7fab4415a6a0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fab2c00bb20 tx=0x7fab2c004730 comp rx=0 tx=0).stop 2023-11-24T06:11:45.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:45.903+0000 7fab4359e640 1 -- 172.21.15.92:0/1394604173 shutdown_connections 2023-11-24T06:11:45.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:45.903+0000 7fab4359e640 1 --2- 172.21.15.92:0/1394604173 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fab2406ae40 0x7fab2406d300 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:45.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:45.903+0000 7fab4359e640 1 --2- 172.21.15.92:0/1394604173 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fab44142280 0x7fab44142700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:45.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:45.903+0000 7fab4359e640 1 --2- 172.21.15.92:0/1394604173 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fab4415d750 0x7fab4415a6a0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:45.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:45.903+0000 7fab4359e640 1 --2- 172.21.15.92:0/1394604173 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fab300047c0 0x7fab4415c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:45.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:45.903+0000 7fab4359e640 1 -- 172.21.15.92:0/1394604173 >> 172.21.15.92:0/1394604173 conn(0x7fab4406a800 msgr2=0x7fab4406abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:45.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:45.903+0000 7fab4359e640 1 -- 172.21.15.92:0/1394604173 shutdown_connections 2023-11-24T06:11:45.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:45.903+0000 7fab4359e640 1 -- 172.21.15.92:0/1394604173 wait complete. 2023-11-24T06:11:45.924 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:11:46.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:46.125+0000 7fc934864640 1 Processor -- start 2023-11-24T06:11:46.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:46.125+0000 7fc934864640 1 -- start start 2023-11-24T06:11:46.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:46.126+0000 7fc934864640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fc93013e360 0x7fc9300cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:46.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:46.126+0000 7fc934864640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fc93015f9c0 0x7fc9300cf7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:46.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:46.126+0000 7fc934864640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc93015c5f0 con 0x7fc93013df80 2023-11-24T06:11:46.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:46.126+0000 7fc934864640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc93006c470 con 0x7fc93013e360 2023-11-24T06:11:46.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:46.126+0000 7fc934864640 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc93006c750 con 0x7fc93015f9c0 2023-11-24T06:11:46.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:46.127+0000 7fc92dd9b640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fc93013e360 0x7fc9300cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:46.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:46.127+0000 7fc92dd9b640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fc93013e360 0x7fc9300cf290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40422/0 (socket says 172.21.15.92:40422) 2023-11-24T06:11:46.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:46.127+0000 7fc92dd9b640 1 -- 172.21.15.92:0/2427528966 learned_addr learned my addr 172.21.15.92:0/2427528966 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:11:46.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:46.127+0000 7fc92ed9d640 1 --2- 172.21.15.92:0/2427528966 >> v2:172.21.15.118:3300/0 conn(0x7fc93015f9c0 0x7fc9300cf7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:46.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:46.127+0000 7fc92dd9b640 1 -- 172.21.15.92:0/2427528966 >> v2:172.21.15.118:3300/0 conn(0x7fc93015f9c0 msgr2=0x7fc9300cf7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:46.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:46.127+0000 7fc92dd9b640 1 --2- 172.21.15.92:0/2427528966 >> v2:172.21.15.118:3300/0 conn(0x7fc93015f9c0 0x7fc9300cf7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:46.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:46.127+0000 7fc92dd9b640 1 -- 172.21.15.92:0/2427528966 >> v1:172.21.15.92:6789/0 conn(0x7fc93013df80 legacy=0x7fc93015bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:46.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:46.127+0000 7fc92dd9b640 1 -- 172.21.15.92:0/2427528966 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc93006ca30 con 0x7fc93013e360 2023-11-24T06:11:46.132 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:46.128+0000 7fc92dd9b640 1 --2- 172.21.15.92:0/2427528966 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc93013e360 0x7fc9300cf290 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fc9300ccae0 tx=0x7fc91802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=617ece3b9231bbf3 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:46.132 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:46.129+0000 7fc92d59a640 1 -- 172.21.15.92:0/2427528966 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc918030860 con 0x7fc93013e360 2023-11-24T06:11:46.133 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:46.129+0000 7fc92d59a640 1 -- 172.21.15.92:0/2427528966 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9180309c0 con 0x7fc93013e360 2023-11-24T06:11:46.133 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:46.129+0000 7fc92d59a640 1 -- 172.21.15.92:0/2427528966 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc918030d50 con 0x7fc93013e360 2023-11-24T06:11:46.133 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:46.129+0000 7fc934864640 1 -- 172.21.15.92:0/2427528966 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc93013e360 msgr2=0x7fc9300cf290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:46.133 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:46.129+0000 7fc934864640 1 --2- 172.21.15.92:0/2427528966 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc93013e360 0x7fc9300cf290 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fc9300ccae0 tx=0x7fc91802b9a0 comp rx=0 tx=0).stop 2023-11-24T06:11:46.133 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:46.129+0000 7fc934864640 1 -- 172.21.15.92:0/2427528966 shutdown_connections 2023-11-24T06:11:46.133 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:46.129+0000 7fc934864640 1 --2- 172.21.15.92:0/2427528966 >> v2:172.21.15.118:3300/0 conn(0x7fc93015f9c0 0x7fc9300cf7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:46.133 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:46.129+0000 7fc934864640 1 --2- 172.21.15.92:0/2427528966 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc93013e360 0x7fc9300cf290 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:46.133 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:46.129+0000 7fc934864640 1 -- 172.21.15.92:0/2427528966 >> 172.21.15.92:0/2427528966 conn(0x7fc93006a800 msgr2=0x7fc93006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:46.133 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:46.129+0000 7fc934864640 1 -- 172.21.15.92:0/2427528966 shutdown_connections 2023-11-24T06:11:46.133 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:46.129+0000 7fc934864640 1 -- 172.21.15.92:0/2427528966 wait complete. 2023-11-24T06:11:46.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:46.131+0000 7fc934864640 1 Processor -- start 2023-11-24T06:11:46.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:46.131+0000 7fc934864640 1 -- start start 2023-11-24T06:11:46.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:46.132+0000 7fc934864640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc93013df80 0x7fc9301f4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:46.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:46.132+0000 7fc934864640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc93013e360 0x7fc9300a8eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:46.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:46.132+0000 7fc934864640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc93015f9c0 0x7fc9300b2f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:46.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:46.132+0000 7fc934864640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9301f5330 con 0x7fc93013df80 2023-11-24T06:11:46.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:46.132+0000 7fc934864640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9301f5610 con 0x7fc93013e360 2023-11-24T06:11:46.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:46.132+0000 7fc934864640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9301f58f0 con 0x7fc93015f9c0 2023-11-24T06:11:46.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:46.132+0000 7fc92e59c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc93013df80 0x7fc9301f4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:46.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:46.132+0000 7fc92e59c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc93013df80 0x7fc9301f4d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40434/0 (socket says 172.21.15.92:40434) 2023-11-24T06:11:46.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:46.132+0000 7fc92e59c640 1 -- 172.21.15.92:0/2535206064 learned_addr learned my addr 172.21.15.92:0/2535206064 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:11:46.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:46.133+0000 7fc92dd9b640 1 --2- 172.21.15.92:0/2535206064 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc93013e360 0x7fc9300a8eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:46.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:46.133+0000 7fc92e59c640 1 -- 172.21.15.92:0/2535206064 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc93015f9c0 msgr2=0x7fc9300b2f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:46.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:46.133+0000 7fc92e59c640 1 --2- 172.21.15.92:0/2535206064 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc93015f9c0 0x7fc9300b2f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:46.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:46.133+0000 7fc92e59c640 1 -- 172.21.15.92:0/2535206064 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc93013e360 msgr2=0x7fc9300a8eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:46.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:46.133+0000 7fc92e59c640 1 --2- 172.21.15.92:0/2535206064 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc93013e360 0x7fc9300a8eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:46.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:46.133+0000 7fc92e59c640 1 -- 172.21.15.92:0/2535206064 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9180094d0 con 0x7fc93013df80 2023-11-24T06:11:46.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:46.133+0000 7fc92e59c640 1 --2- 172.21.15.92:0/2535206064 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc93013df80 0x7fc9301f4d50 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fc92000c710 tx=0x7fc92000cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:46.138 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:46.133+0000 7fc926ffd640 1 -- 172.21.15.92:0/2535206064 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc920003b90 con 0x7fc93013df80 2023-11-24T06:11:46.138 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:46.133+0000 7fc926ffd640 1 -- 172.21.15.92:0/2535206064 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc920003cf0 con 0x7fc93013df80 2023-11-24T06:11:46.138 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:46.133+0000 7fc934864640 1 -- 172.21.15.92:0/2535206064 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9301f5bd0 con 0x7fc93013df80 2023-11-24T06:11:46.138 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:46.133+0000 7fc926ffd640 1 -- 172.21.15.92:0/2535206064 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc92000aa90 con 0x7fc93013df80 2023-11-24T06:11:46.138 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:46.133+0000 7fc934864640 1 -- 172.21.15.92:0/2535206064 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9300a93f0 con 0x7fc93013df80 2023-11-24T06:11:46.138 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:46.135+0000 7fc934864640 1 -- 172.21.15.92:0/2535206064 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9300cf970 con 0x7fc93013df80 2023-11-24T06:11:46.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:46.135+0000 7fc926ffd640 1 -- 172.21.15.92:0/2535206064 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc92000abf0 con 0x7fc93013df80 2023-11-24T06:11:46.140 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:46.137+0000 7fc926ffd640 1 --2- 172.21.15.92:0/2535206064 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fc91006ae40 0x7fc91006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:46.140 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:46.137+0000 7fc926ffd640 1 -- 172.21.15.92:0/2535206064 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc920089450 con 0x7fc93013df80 2023-11-24T06:11:46.140 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:46.137+0000 7fc92dd9b640 1 --2- 172.21.15.92:0/2535206064 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fc91006ae40 0x7fc91006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:46.140 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:46.138+0000 7fc92dd9b640 1 --2- 172.21.15.92:0/2535206064 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fc91006ae40 0x7fc91006d300 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fc918036000 tx=0x7fc91803b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:46.144 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:46.141+0000 7fc926ffd640 1 -- 172.21.15.92:0/2535206064 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc9200544f0 con 0x7fc93013df80 2023-11-24T06:11:46.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:46.336+0000 7fc934864640 1 -- 172.21.15.92:0/2535206064 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc9300675d0 con 0x7fc93013df80 2023-11-24T06:11:46.342 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:46.339+0000 7fc926ffd640 1 -- 172.21.15.92:0/2535206064 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) v1 ==== 74+0+16521 (secure 0 0 0) 0x7fc920057b10 con 0x7fc93013df80 2023-11-24T06:11:46.342 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-24T06:11:46.342 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":33,"fsid":"2d6dee5f-8aa3-470f-931a-50ade963b1cb","created":"2023-11-24T06:10:04.707204+0000","modified":"2023-11-24T06:11:46.026094+0000","last_up_change":"2023-11-24T06:10:17.598352+0000","last_in_change":"2023-11-24T06:10:09.040793+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:10:17.733098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:11:42.241681+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-11-24T06:11:43.522720+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ff37a9fb-135f-471f-b4b0-f7b5772006c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6803","nonce":20560714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6805","nonce":20560714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6809","nonce":20560714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6807","nonce":20560714}]},"public_addr":"172.21.15.92:6803/20560714","cluster_addr":"172.21.15.92:6805/20560714","heartbeat_back_addr":"172.21.15.92:6809/20560714","heartbeat_front_addr":"172.21.15.92:6807/20560714","state":["exists","up"]},{"osd":1,"uuid":"03cdd353-dcf1-4f38-94fe-a795137c06b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6819","nonce":2451239906}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6821","nonce":2451239906}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6825","nonce":2451239906}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6823","nonce":2451239906}]},"public_addr":"172.21.15.92:6819/2451239906","cluster_addr":"172.21.15.92:6821/2451239906","heartbeat_back_addr":"172.21.15.92:6825/2451239906","heartbeat_front_addr":"172.21.15.92:6823/2451239906","state":["exists","up"]},{"osd":2,"uuid":"a4a20d80-5d57-484b-becd-9aa8070dd3aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6811","nonce":1124979277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6813","nonce":1124979277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6817","nonce":1124979277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6815","nonce":1124979277}]},"public_addr":"172.21.15.92:6811/1124979277","cluster_addr":"172.21.15.92:6813/1124979277","heartbeat_back_addr":"172.21.15.92:6817/1124979277","heartbeat_front_addr":"172.21.15.92:6815/1124979277","state":["exists","up"]},{"osd":3,"uuid":"4648d7ce-9b29-4583-9425-4ba8fd2b1903","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6827","nonce":1734115782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6829","nonce":1734115782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6833","nonce":1734115782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6831","nonce":1734115782}]},"public_addr":"172.21.15.92:6827/1734115782","cluster_addr":"172.21.15.92:6829/1734115782","heartbeat_back_addr":"172.21.15.92:6833/1734115782","heartbeat_front_addr":"172.21.15.92:6831/1734115782","state":["exists","up"]},{"osd":4,"uuid":"3bc9d7eb-c6fa-4fcf-a3ec-ade604e05aac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6819","nonce":3425937229}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6822","nonce":3425937229}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6830","nonce":3425937229}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6826","nonce":3425937229}]},"public_addr":"172.21.15.118:6819/3425937229","cluster_addr":"172.21.15.118:6822/3425937229","heartbeat_back_addr":"172.21.15.118:6830/3425937229","heartbeat_front_addr":"172.21.15.118:6826/3425937229","state":["exists","up"]},{"osd":5,"uuid":"dce9ba6d-59ed-46d1-84fc-a84b13de10c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6801","nonce":827455984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6803","nonce":827455984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6807","nonce":827455984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6805","nonce":827455984}]},"public_addr":"172.21.15.118:6801/827455984","cluster_addr":"172.21.15.118:6803/827455984","heartbeat_back_addr":"172.21.15.118:6807/827455984","heartbeat_front_addr":"172.21.15.118:6805/827455984","state":["exists","up"]},{"osd":6,"uuid":"82aec504-ae58-4709-8d30-46f2b12081b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6809","nonce":2946986141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6811","nonce":2946986141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6815","nonce":2946986141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6813","nonce":2946986141}]},"public_addr":"172.21.15.118:6809/2946986141","cluster_addr":"172.21.15.118:6811/2946986141","heartbeat_back_addr":"172.21.15.118:6815/2946986141","heartbeat_front_addr":"172.21.15.118:6813/2946986141","state":["exists","up"]},{"osd":7,"uuid":"4ed4c624-c88f-4d35-bdb1-bf209b5dffb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6817","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6818","nonce":48122059}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6821","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6823","nonce":48122059}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6829","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6831","nonce":48122059}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6825","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6827","nonce":48122059}]},"public_addr":"172.21.15.118:6818/48122059","cluster_addr":"172.21.15.118:6823/48122059","heartbeat_back_addr":"172.21.15.118:6831/48122059","heartbeat_front_addr":"172.21.15.118:6827/48122059","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.536664+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.368324+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.271201+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.342971+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.621832+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.387110+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.632946+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.416032+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:11:46.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:46.342+0000 7fc934864640 1 -- 172.21.15.92:0/2535206064 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fc91006ae40 msgr2=0x7fc91006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:46.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:46.342+0000 7fc934864640 1 --2- 172.21.15.92:0/2535206064 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fc91006ae40 0x7fc91006d300 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fc918036000 tx=0x7fc91803b040 comp rx=0 tx=0).stop 2023-11-24T06:11:46.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:46.342+0000 7fc934864640 1 -- 172.21.15.92:0/2535206064 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc93013df80 msgr2=0x7fc9301f4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:46.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:46.342+0000 7fc934864640 1 --2- 172.21.15.92:0/2535206064 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc93013df80 0x7fc9301f4d50 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fc92000c710 tx=0x7fc92000cbd0 comp rx=0 tx=0).stop 2023-11-24T06:11:46.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:46.342+0000 7fc934864640 1 -- 172.21.15.92:0/2535206064 shutdown_connections 2023-11-24T06:11:46.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:46.342+0000 7fc934864640 1 --2- 172.21.15.92:0/2535206064 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fc91006ae40 0x7fc91006d300 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:46.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:46.342+0000 7fc934864640 1 --2- 172.21.15.92:0/2535206064 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc93015f9c0 0x7fc9300b2f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:46.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:46.342+0000 7fc934864640 1 --2- 172.21.15.92:0/2535206064 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc93013e360 0x7fc9300a8eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:46.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:46.342+0000 7fc934864640 1 --2- 172.21.15.92:0/2535206064 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc93013df80 0x7fc9301f4d50 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:46.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:46.342+0000 7fc934864640 1 -- 172.21.15.92:0/2535206064 >> 172.21.15.92:0/2535206064 conn(0x7fc93006a800 msgr2=0x7fc9300cf260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:46.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:46.342+0000 7fc934864640 1 -- 172.21.15.92:0/2535206064 shutdown_connections 2023-11-24T06:11:46.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:46.342+0000 7fc934864640 1 -- 172.21.15.92:0/2535206064 wait complete. 2023-11-24T06:11:46.364 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-11-24T06:11:46.572 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:46.569+0000 7fea3f0e7640 1 Processor -- start 2023-11-24T06:11:46.572 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:46.569+0000 7fea3f0e7640 1 -- start start 2023-11-24T06:11:46.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:46.570+0000 7fea3f0e7640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fea3806a3e0 0x7fea380ac8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:46.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:46.570+0000 7fea3f0e7640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea38071de0 con 0x7fea380aede0 2023-11-24T06:11:46.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:46.570+0000 7fea3f0e7640 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea380af270 con 0x7fea380d0b60 2023-11-24T06:11:46.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:46.570+0000 7fea3f0e7640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea38072030 con 0x7fea3806a3e0 2023-11-24T06:11:46.574 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:46.570+0000 7fea3ce83640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fea3806a3e0 0x7fea380ac8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:46.574 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:46.570+0000 7fea3ce83640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fea3806a3e0 0x7fea380ac8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40438/0 (socket says 172.21.15.92:40438) 2023-11-24T06:11:46.574 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:46.570+0000 7fea3ce83640 1 -- 172.21.15.92:0/1523113818 learned_addr learned my addr 172.21.15.92:0/1523113818 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:11:46.574 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:46.571+0000 7fea3ce83640 1 -- 172.21.15.92:0/1523113818 >> v1:172.21.15.118:6789/0 conn(0x7fea380d0b60 legacy=0x7fea3806a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:46.575 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:46.571+0000 7fea3ce83640 1 -- 172.21.15.92:0/1523113818 >> v1:172.21.15.92:6789/0 conn(0x7fea380aede0 legacy=0x7fea3806f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:46.575 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:46.571+0000 7fea3ce83640 1 -- 172.21.15.92:0/1523113818 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea38072310 con 0x7fea3806a3e0 2023-11-24T06:11:46.575 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:46.572+0000 7fea3ce83640 1 --2- 172.21.15.92:0/1523113818 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fea3806a3e0 0x7fea380ac8d0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fea34009e50 tx=0x7fea3402bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f14f9ac77d4355e server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:46.575 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:46.572+0000 7fea2f7fe640 1 -- 172.21.15.92:0/1523113818 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea3402cb20 con 0x7fea3806a3e0 2023-11-24T06:11:46.575 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:46.572+0000 7fea2f7fe640 1 -- 172.21.15.92:0/1523113818 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea3402cc80 con 0x7fea3806a3e0 2023-11-24T06:11:46.575 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:46.572+0000 7fea2f7fe640 1 -- 172.21.15.92:0/1523113818 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea340343f0 con 0x7fea3806a3e0 2023-11-24T06:11:46.576 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:46.573+0000 7fea3f0e7640 1 -- 172.21.15.92:0/1523113818 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fea3806a3e0 msgr2=0x7fea380ac8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:46.576 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:46.573+0000 7fea3f0e7640 1 --2- 172.21.15.92:0/1523113818 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fea3806a3e0 0x7fea380ac8d0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fea34009e50 tx=0x7fea3402bf80 comp rx=0 tx=0).stop 2023-11-24T06:11:46.576 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:46.573+0000 7fea3f0e7640 1 -- 172.21.15.92:0/1523113818 shutdown_connections 2023-11-24T06:11:46.576 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:46.573+0000 7fea3f0e7640 1 --2- 172.21.15.92:0/1523113818 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fea3806a3e0 0x7fea380ac8d0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:46.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:46.573+0000 7fea3f0e7640 1 -- 172.21.15.92:0/1523113818 >> 172.21.15.92:0/1523113818 conn(0x7fea380cfef0 msgr2=0x7fea380d0300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:46.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:46.573+0000 7fea3f0e7640 1 -- 172.21.15.92:0/1523113818 shutdown_connections 2023-11-24T06:11:46.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:46.573+0000 7fea3f0e7640 1 -- 172.21.15.92:0/1523113818 wait complete. 2023-11-24T06:11:46.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:46.574+0000 7fea3f0e7640 1 Processor -- start 2023-11-24T06:11:46.578 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:46.575+0000 7fea3f0e7640 1 -- start start 2023-11-24T06:11:46.579 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:46.575+0000 7fea3f0e7640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fea3806a3e0 0x7fea3814e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:46.579 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:46.575+0000 7fea3f0e7640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fea380aede0 0x7fea38156bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:46.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:46.576+0000 7fea3f0e7640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fea380d0b60 0x7fea38157130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:46.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:46.576+0000 7fea3f0e7640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea3814ee90 con 0x7fea380aede0 2023-11-24T06:11:46.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:46.576+0000 7fea3f0e7640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea3815c6d0 con 0x7fea3806a3e0 2023-11-24T06:11:46.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:46.576+0000 7fea3f0e7640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea3815bb50 con 0x7fea380d0b60 2023-11-24T06:11:46.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:46.576+0000 7fea2ffff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fea380aede0 0x7fea38156bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:46.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:46.576+0000 7fea2ffff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fea380aede0 0x7fea38156bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40448/0 (socket says 172.21.15.92:40448) 2023-11-24T06:11:46.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:46.576+0000 7fea2ffff640 1 -- 172.21.15.92:0/3508008984 learned_addr learned my addr 172.21.15.92:0/3508008984 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:11:46.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:46.576+0000 7fea2ffff640 1 -- 172.21.15.92:0/3508008984 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fea380d0b60 msgr2=0x7fea38157130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:46.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:46.576+0000 7fea3ce83640 1 --2- 172.21.15.92:0/3508008984 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fea3806a3e0 0x7fea3814e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:46.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:46.576+0000 7fea2ffff640 1 --2- 172.21.15.92:0/3508008984 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fea380d0b60 0x7fea38157130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:46.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:46.576+0000 7fea2ffff640 1 -- 172.21.15.92:0/3508008984 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fea3806a3e0 msgr2=0x7fea3814e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:46.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:46.576+0000 7fea2ffff640 1 --2- 172.21.15.92:0/3508008984 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fea3806a3e0 0x7fea3814e950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:46.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:46.576+0000 7fea2ffff640 1 -- 172.21.15.92:0/3508008984 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea34009b30 con 0x7fea380aede0 2023-11-24T06:11:46.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:46.576+0000 7fea2ffff640 1 --2- 172.21.15.92:0/3508008984 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fea380aede0 0x7fea38156bf0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fea2800b3c0 tx=0x7fea2800b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:46.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:46.577+0000 7fea2d7fa640 1 -- 172.21.15.92:0/3508008984 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea28011830 con 0x7fea380aede0 2023-11-24T06:11:46.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:46.577+0000 7fea2d7fa640 1 -- 172.21.15.92:0/3508008984 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea28011990 con 0x7fea380aede0 2023-11-24T06:11:46.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:46.577+0000 7fea2d7fa640 1 -- 172.21.15.92:0/3508008984 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea28011d60 con 0x7fea380aede0 2023-11-24T06:11:46.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:46.577+0000 7fea3f0e7640 1 -- 172.21.15.92:0/3508008984 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea3815be30 con 0x7fea380aede0 2023-11-24T06:11:46.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:46.577+0000 7fea3f0e7640 1 -- 172.21.15.92:0/3508008984 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea3815c3f0 con 0x7fea380aede0 2023-11-24T06:11:46.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:46.578+0000 7fea3f0e7640 1 -- 172.21.15.92:0/3508008984 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea380ad0d0 con 0x7fea380aede0 2023-11-24T06:11:46.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:46.579+0000 7fea2d7fa640 1 -- 172.21.15.92:0/3508008984 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fea2801b460 con 0x7fea380aede0 2023-11-24T06:11:46.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:46.580+0000 7fea2d7fa640 1 --2- 172.21.15.92:0/3508008984 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fea2406ae40 0x7fea2406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:46.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:46.580+0000 7fea2d7fa640 1 -- 172.21.15.92:0/3508008984 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fea28089390 con 0x7fea380aede0 2023-11-24T06:11:46.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:46.580+0000 7fea3ce83640 1 --2- 172.21.15.92:0/3508008984 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fea2406ae40 0x7fea2406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:46.584 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:46.581+0000 7fea3ce83640 1 --2- 172.21.15.92:0/3508008984 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fea2406ae40 0x7fea2406d300 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fea34035000 tx=0x7fea34030000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:46.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:46.584+0000 7fea2d7fa640 1 -- 172.21.15.92:0/3508008984 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fea280544d0 con 0x7fea380aede0 2023-11-24T06:11:46.604 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T06:11:46.610 DEBUG:teuthology.orchestra.run.smithi118:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T06:11:46.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:46.812+0000 7fea3f0e7640 1 -- 172.21.15.92:0/3508008984 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fea380cc9d0 con 0x7fea380aede0 2023-11-24T06:11:46.996 INFO:tasks.ceph.mds.c.smithi092.stderr:2023-11-24T06:11:46.993+0000 7f16ce26b640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-24T06:11:47.510 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:47.507+0000 7fea2d7fa640 1 -- 172.21.15.92:0/3508008984 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v16) v1 ==== 118+0+0 (secure 0 0 0) 0x7fea380cc9d0 con 0x7fea380aede0 2023-11-24T06:11:47.514 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:47.510+0000 7fea3f0e7640 1 -- 172.21.15.92:0/3508008984 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fea2406ae40 msgr2=0x7fea2406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:47.514 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:47.510+0000 7fea3f0e7640 1 --2- 172.21.15.92:0/3508008984 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fea2406ae40 0x7fea2406d300 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fea34035000 tx=0x7fea34030000 comp rx=0 tx=0).stop 2023-11-24T06:11:47.514 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:47.510+0000 7fea3f0e7640 1 -- 172.21.15.92:0/3508008984 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fea380aede0 msgr2=0x7fea38156bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:47.514 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:47.510+0000 7fea3f0e7640 1 --2- 172.21.15.92:0/3508008984 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fea380aede0 0x7fea38156bf0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fea2800b3c0 tx=0x7fea2800b880 comp rx=0 tx=0).stop 2023-11-24T06:11:47.514 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:47.510+0000 7fea3f0e7640 1 -- 172.21.15.92:0/3508008984 shutdown_connections 2023-11-24T06:11:47.514 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:47.510+0000 7fea3f0e7640 1 --2- 172.21.15.92:0/3508008984 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fea2406ae40 0x7fea2406d300 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:47.515 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:47.510+0000 7fea3f0e7640 1 --2- 172.21.15.92:0/3508008984 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fea380d0b60 0x7fea38157130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:47.515 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:47.510+0000 7fea3f0e7640 1 --2- 172.21.15.92:0/3508008984 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fea380aede0 0x7fea38156bf0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:47.515 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:47.510+0000 7fea3f0e7640 1 --2- 172.21.15.92:0/3508008984 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fea3806a3e0 0x7fea3814e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:47.515 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:47.510+0000 7fea3f0e7640 1 -- 172.21.15.92:0/3508008984 >> 172.21.15.92:0/3508008984 conn(0x7fea380cfef0 msgr2=0x7fea3806d190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:47.515 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:47.511+0000 7fea3f0e7640 1 -- 172.21.15.92:0/3508008984 shutdown_connections 2023-11-24T06:11:47.515 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:47.511+0000 7fea3f0e7640 1 -- 172.21.15.92:0/3508008984 wait complete. 2023-11-24T06:11:47.534 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-11-24T06:11:47.534 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-11-24T06:11:47.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:47.750+0000 7f327dcbb640 1 Processor -- start 2023-11-24T06:11:47.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:47.751+0000 7f327dcbb640 1 -- start start 2023-11-24T06:11:47.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:47.752+0000 7f327dcbb640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f3278074520 0x7f32780ac050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:47.756 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:47.752+0000 7f327dcbb640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f32780ae750 0x7f32780ac590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:47.756 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:47.752+0000 7f327dcbb640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3278162100 con 0x7f32780cc1e0 2023-11-24T06:11:47.756 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:47.752+0000 7f327dcbb640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32780ccf20 con 0x7f3278074520 2023-11-24T06:11:47.756 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:47.752+0000 7f327dcbb640 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32780cd200 con 0x7f32780ae750 2023-11-24T06:11:47.756 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:47.752+0000 7f32777fe640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f32780cc1e0 0x7f32780cea50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:59748/0 (socket says 172.21.15.92:59748) 2023-11-24T06:11:47.756 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:47.752+0000 7f32777fe640 1 -- 172.21.15.92:0/3717134404 learned_addr learned my addr 172.21.15.92:0/3717134404 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-24T06:11:47.756 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:47.753+0000 7f3276ffd640 1 --2- 172.21.15.92:0/3717134404 >> v2:172.21.15.92:3300/0 conn(0x7f3278074520 0x7f32780ac050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:47.756 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:47.753+0000 7f3277fff640 1 --2- 172.21.15.92:0/3717134404 >> v2:172.21.15.118:3300/0 conn(0x7f32780ae750 0x7f32780ac590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:47.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:47.753+0000 7f32767fc640 1 -- 172.21.15.92:0/3717134404 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3892395080 0 0) 0x7f3278162100 con 0x7f32780cc1e0 2023-11-24T06:11:47.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:47.753+0000 7f32767fc640 1 -- 172.21.15.92:0/3717134404 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3264003520 con 0x7f32780cc1e0 2023-11-24T06:11:47.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:47.753+0000 7f3276ffd640 1 -- 172.21.15.92:0/3717134404 >> v2:172.21.15.118:3300/0 conn(0x7f32780ae750 msgr2=0x7f32780ac590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:47.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:47.753+0000 7f3276ffd640 1 --2- 172.21.15.92:0/3717134404 >> v2:172.21.15.118:3300/0 conn(0x7f32780ae750 0x7f32780ac590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:47.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:47.753+0000 7f3276ffd640 1 -- 172.21.15.92:0/3717134404 >> v1:172.21.15.92:6789/0 conn(0x7f32780cc1e0 legacy=0x7f32780cea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:47.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:47.753+0000 7f3276ffd640 1 -- 172.21.15.92:0/3717134404 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f327806daa0 con 0x7f3278074520 2023-11-24T06:11:47.758 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:47.754+0000 7f3276ffd640 1 --2- 172.21.15.92:0/3717134404 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3278074520 0x7f32780ac050 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f32780716e0 tx=0x7f326002b7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a3e50f2f9712268 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:47.758 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:47.755+0000 7f32767fc640 1 -- 172.21.15.92:0/3717134404 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3260031710 con 0x7f3278074520 2023-11-24T06:11:47.758 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:47.755+0000 7f32767fc640 1 -- 172.21.15.92:0/3717134404 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f326002f050 con 0x7f3278074520 2023-11-24T06:11:47.758 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:47.755+0000 7f32767fc640 1 -- 172.21.15.92:0/3717134404 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3260031a70 con 0x7f3278074520 2023-11-24T06:11:47.758 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:47.755+0000 7f327dcbb640 1 -- 172.21.15.92:0/3717134404 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3278074520 msgr2=0x7f32780ac050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:47.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:47.755+0000 7f327dcbb640 1 --2- 172.21.15.92:0/3717134404 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3278074520 0x7f32780ac050 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f32780716e0 tx=0x7f326002b7b0 comp rx=0 tx=0).stop 2023-11-24T06:11:47.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:47.755+0000 7f327dcbb640 1 -- 172.21.15.92:0/3717134404 shutdown_connections 2023-11-24T06:11:47.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:47.755+0000 7f327dcbb640 1 --2- 172.21.15.92:0/3717134404 >> v2:172.21.15.118:3300/0 conn(0x7f32780ae750 0x7f32780ac590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:47.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:47.755+0000 7f327dcbb640 1 --2- 172.21.15.92:0/3717134404 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3278074520 0x7f32780ac050 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:47.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:47.755+0000 7f327dcbb640 1 -- 172.21.15.92:0/3717134404 >> 172.21.15.92:0/3717134404 conn(0x7f32780b0d70 msgr2=0x7f32780b1150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:47.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:47.756+0000 7f327dcbb640 1 -- 172.21.15.92:0/3717134404 shutdown_connections 2023-11-24T06:11:47.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:47.756+0000 7f327dcbb640 1 -- 172.21.15.92:0/3717134404 wait complete. 2023-11-24T06:11:47.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:47.757+0000 7f327dcbb640 1 Processor -- start 2023-11-24T06:11:47.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:47.758+0000 7f327dcbb640 1 -- start start 2023-11-24T06:11:47.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:47.759+0000 7f327dcbb640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3278074520 0x7f3278159390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:47.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:47.759+0000 7f327dcbb640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f32780ae750 0x7f32781f3af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:47.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:47.759+0000 7f327dcbb640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f32780cc1e0 0x7f32781f4230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:47.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:47.759+0000 7f327dcbb640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32780c73c0 con 0x7f32780ae750 2023-11-24T06:11:47.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:47.759+0000 7f327dcbb640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32781f4030 con 0x7f32780cc1e0 2023-11-24T06:11:47.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:47.759+0000 7f327dcbb640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32780cac80 con 0x7f3278074520 2023-11-24T06:11:47.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:47.759+0000 7f3276ffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f32780ae750 0x7f32781f3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:47.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:47.759+0000 7f3276ffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f32780ae750 0x7f32781f3af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40466/0 (socket says 172.21.15.92:40466) 2023-11-24T06:11:47.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:47.759+0000 7f3276ffd640 1 -- 172.21.15.92:0/1392365106 learned_addr learned my addr 172.21.15.92:0/1392365106 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:11:47.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:47.759+0000 7f3276ffd640 1 -- 172.21.15.92:0/1392365106 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3278074520 msgr2=0x7f3278159390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:47.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:47.759+0000 7f3277fff640 1 --2- 172.21.15.92:0/1392365106 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f32780cc1e0 0x7f32781f4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:47.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:47.759+0000 7f32777fe640 1 --2- 172.21.15.92:0/1392365106 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3278074520 0x7f3278159390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:47.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:47.759+0000 7f3276ffd640 1 --2- 172.21.15.92:0/1392365106 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3278074520 0x7f3278159390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:47.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:47.759+0000 7f3276ffd640 1 -- 172.21.15.92:0/1392365106 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f32780cc1e0 msgr2=0x7f32781f4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:47.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:47.759+0000 7f3276ffd640 1 --2- 172.21.15.92:0/1392365106 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f32780cc1e0 0x7f32781f4230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:47.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:47.759+0000 7f3276ffd640 1 -- 172.21.15.92:0/1392365106 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3260009400 con 0x7f32780ae750 2023-11-24T06:11:47.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:47.760+0000 7f3276ffd640 1 --2- 172.21.15.92:0/1392365106 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f32780ae750 0x7f32781f3af0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f3260000c00 tx=0x7f32600046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:47.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:47.760+0000 7f326ffff640 1 -- 172.21.15.92:0/1392365106 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f326003c810 con 0x7f32780ae750 2023-11-24T06:11:47.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:47.760+0000 7f326ffff640 1 -- 172.21.15.92:0/1392365106 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f326003c970 con 0x7f32780ae750 2023-11-24T06:11:47.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:47.760+0000 7f327dcbb640 1 -- 172.21.15.92:0/1392365106 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32780c6740 con 0x7f32780ae750 2023-11-24T06:11:47.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:47.760+0000 7f326ffff640 1 -- 172.21.15.92:0/1392365106 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f326003cd20 con 0x7f32780ae750 2023-11-24T06:11:47.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:47.760+0000 7f327dcbb640 1 -- 172.21.15.92:0/1392365106 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32780c6d00 con 0x7f32780ae750 2023-11-24T06:11:47.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:47.761+0000 7f327dcbb640 1 -- 172.21.15.92:0/1392365106 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32780c7500 con 0x7f32780ae750 2023-11-24T06:11:47.766 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:47.762+0000 7f326ffff640 1 -- 172.21.15.92:0/1392365106 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f326003a040 con 0x7f32780ae750 2023-11-24T06:11:47.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:47.764+0000 7f326ffff640 1 --2- 172.21.15.92:0/1392365106 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f325806ae70 0x7f325806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:47.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:47.764+0000 7f326ffff640 1 -- 172.21.15.92:0/1392365106 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f32600aab10 con 0x7f32780ae750 2023-11-24T06:11:47.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:47.764+0000 7f32777fe640 1 --2- 172.21.15.92:0/1392365106 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f325806ae70 0x7f325806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:47.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:47.764+0000 7f32777fe640 1 --2- 172.21.15.92:0/1392365106 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f325806ae70 0x7f325806d330 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f32781f0c60 tx=0x7f3268009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:47.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:47.768+0000 7f326ffff640 1 -- 172.21.15.92:0/1392365106 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f32780c7500 con 0x7f32780ae750 2023-11-24T06:11:47.999 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:47.996+0000 7f327dcbb640 1 -- 172.21.15.92:0/1392365106 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f32780c7500 con 0x7f32780ae750 2023-11-24T06:11:48.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:48.517+0000 7f326ffff640 1 -- 172.21.15.92:0/1392365106 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v17) v1 ==== 122+0+0 (secure 0 0 0) 0x7f32780c7500 con 0x7f32780ae750 2023-11-24T06:11:48.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:48.521+0000 7f327dcbb640 1 -- 172.21.15.92:0/1392365106 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f325806ae70 msgr2=0x7f325806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:48.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:48.521+0000 7f327dcbb640 1 --2- 172.21.15.92:0/1392365106 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f325806ae70 0x7f325806d330 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f32781f0c60 tx=0x7f3268009040 comp rx=0 tx=0).stop 2023-11-24T06:11:48.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:48.521+0000 7f327dcbb640 1 -- 172.21.15.92:0/1392365106 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f32780ae750 msgr2=0x7f32781f3af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:48.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:48.521+0000 7f327dcbb640 1 --2- 172.21.15.92:0/1392365106 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f32780ae750 0x7f32781f3af0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f3260000c00 tx=0x7f32600046c0 comp rx=0 tx=0).stop 2023-11-24T06:11:48.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:48.522+0000 7f327dcbb640 1 -- 172.21.15.92:0/1392365106 shutdown_connections 2023-11-24T06:11:48.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:48.522+0000 7f327dcbb640 1 --2- 172.21.15.92:0/1392365106 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f325806ae70 0x7f325806d330 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:48.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:48.522+0000 7f327dcbb640 1 --2- 172.21.15.92:0/1392365106 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f32780cc1e0 0x7f32781f4230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:48.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:48.522+0000 7f327dcbb640 1 --2- 172.21.15.92:0/1392365106 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f32780ae750 0x7f32781f3af0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:48.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:48.522+0000 7f327dcbb640 1 --2- 172.21.15.92:0/1392365106 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3278074520 0x7f3278159390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:48.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:48.522+0000 7f327dcbb640 1 -- 172.21.15.92:0/1392365106 >> 172.21.15.92:0/1392365106 conn(0x7f32780b0d70 msgr2=0x7f32780aece0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:48.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:48.522+0000 7f327dcbb640 1 -- 172.21.15.92:0/1392365106 shutdown_connections 2023-11-24T06:11:48.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:48.522+0000 7f327dcbb640 1 -- 172.21.15.92:0/1392365106 wait complete. 2023-11-24T06:11:48.544 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-11-24T06:11:48.544 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T06:11:48.758 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:48.755+0000 7ff4e8f4c640 1 Processor -- start 2023-11-24T06:11:48.758 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:48.755+0000 7ff4e8f4c640 1 -- start start 2023-11-24T06:11:48.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:48.757+0000 7ff4e8f4c640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ff4e4143ae0 0x7ff4e40cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:48.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:48.757+0000 7ff4e8f4c640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff4e40cf7d0 0x7ff4e415dca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:48.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:48.757+0000 7ff4e8f4c640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7ff4e415e1e0 0x7ff4e415a670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:48.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:48.757+0000 7ff4e8f4c640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4e40cfc30 con 0x7ff4e4143ae0 2023-11-24T06:11:48.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:48.757+0000 7ff4e8f4c640 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4e40b1ee0 con 0x7ff4e415e1e0 2023-11-24T06:11:48.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:48.757+0000 7ff4e8f4c640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4e40b2780 con 0x7ff4e40cf7d0 2023-11-24T06:11:48.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:48.757+0000 7ff4e259c640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ff4e4143ae0 0x7ff4e40cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:48.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:48.757+0000 7ff4e259c640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ff4e4143ae0 0x7ff4e40cf290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40478/0 (socket says 172.21.15.92:40478) 2023-11-24T06:11:48.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:48.757+0000 7ff4e259c640 1 -- 172.21.15.92:0/3503966789 learned_addr learned my addr 172.21.15.92:0/3503966789 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:11:48.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:48.758+0000 7ff4e2d9d640 1 --2- 172.21.15.92:0/3503966789 >> v2:172.21.15.118:3300/0 conn(0x7ff4e415e1e0 0x7ff4e415a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:48.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:48.758+0000 7ff4e1d9b640 1 --2- 172.21.15.92:0/3503966789 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff4e40cf7d0 0x7ff4e415dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:48.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:48.758+0000 7ff4e259c640 1 -- 172.21.15.92:0/3503966789 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff4e40cf7d0 msgr2=0x7ff4e415dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:48.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:48.758+0000 7ff4e259c640 1 --2- 172.21.15.92:0/3503966789 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff4e40cf7d0 0x7ff4e415dca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:48.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:48.758+0000 7ff4e259c640 1 -- 172.21.15.92:0/3503966789 >> v2:172.21.15.118:3300/0 conn(0x7ff4e415e1e0 msgr2=0x7ff4e415a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:48.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:48.758+0000 7ff4e259c640 1 --2- 172.21.15.92:0/3503966789 >> v2:172.21.15.118:3300/0 conn(0x7ff4e415e1e0 0x7ff4e415a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:48.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:48.758+0000 7ff4e259c640 1 -- 172.21.15.92:0/3503966789 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4e415e990 con 0x7ff4e4143ae0 2023-11-24T06:11:48.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:48.758+0000 7ff4e2d9d640 1 --2- 172.21.15.92:0/3503966789 >> v2:172.21.15.118:3300/0 conn(0x7ff4e415e1e0 0x7ff4e415a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:11:48.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:48.759+0000 7ff4e259c640 1 --2- 172.21.15.92:0/3503966789 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff4e4143ae0 0x7ff4e40cf290 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7ff4e4072160 tx=0x7ff4d402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fa7e1476ed11411 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:48.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:48.759+0000 7ff4e159a640 1 -- 172.21.15.92:0/3503966789 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4d4031870 con 0x7ff4e4143ae0 2023-11-24T06:11:48.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:48.759+0000 7ff4e159a640 1 -- 172.21.15.92:0/3503966789 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4d40319d0 con 0x7ff4e4143ae0 2023-11-24T06:11:48.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:48.759+0000 7ff4e159a640 1 -- 172.21.15.92:0/3503966789 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4d4031d60 con 0x7ff4e4143ae0 2023-11-24T06:11:48.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:48.760+0000 7ff4e8f4c640 1 -- 172.21.15.92:0/3503966789 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff4e4143ae0 msgr2=0x7ff4e40cf290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:48.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:48.760+0000 7ff4e8f4c640 1 --2- 172.21.15.92:0/3503966789 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff4e4143ae0 0x7ff4e40cf290 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7ff4e4072160 tx=0x7ff4d402b910 comp rx=0 tx=0).stop 2023-11-24T06:11:48.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:48.760+0000 7ff4e8f4c640 1 -- 172.21.15.92:0/3503966789 shutdown_connections 2023-11-24T06:11:48.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:48.760+0000 7ff4e8f4c640 1 --2- 172.21.15.92:0/3503966789 >> v2:172.21.15.118:3300/0 conn(0x7ff4e415e1e0 0x7ff4e415a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:48.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:48.760+0000 7ff4e8f4c640 1 --2- 172.21.15.92:0/3503966789 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff4e40cf7d0 0x7ff4e415dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:48.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:48.760+0000 7ff4e8f4c640 1 --2- 172.21.15.92:0/3503966789 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff4e4143ae0 0x7ff4e40cf290 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:48.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:48.760+0000 7ff4e8f4c640 1 -- 172.21.15.92:0/3503966789 >> 172.21.15.92:0/3503966789 conn(0x7ff4e406a800 msgr2=0x7ff4e406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:48.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:48.760+0000 7ff4e8f4c640 1 -- 172.21.15.92:0/3503966789 shutdown_connections 2023-11-24T06:11:48.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:48.760+0000 7ff4e8f4c640 1 -- 172.21.15.92:0/3503966789 wait complete. 2023-11-24T06:11:48.766 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:48.762+0000 7ff4e8f4c640 1 Processor -- start 2023-11-24T06:11:48.766 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:48.762+0000 7ff4e8f4c640 1 -- start start 2023-11-24T06:11:48.766 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:48.763+0000 7ff4e8f4c640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff4e40cf7d0 0x7ff4e4158ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:48.766 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:48.763+0000 7ff4e8f4c640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff4e4143ae0 0x7ff4e413c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:48.766 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:48.763+0000 7ff4e8f4c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff4e415e1e0 0x7ff4e415cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:48.766 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:48.763+0000 7ff4e8f4c640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4e4159410 con 0x7ff4e415e1e0 2023-11-24T06:11:48.766 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:48.763+0000 7ff4e8f4c640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4e413cd80 con 0x7ff4e40cf7d0 2023-11-24T06:11:48.766 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:48.763+0000 7ff4e8f4c640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4e413d060 con 0x7ff4e4143ae0 2023-11-24T06:11:48.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:48.763+0000 7ff4e2d9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff4e415e1e0 0x7ff4e415cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:48.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:48.763+0000 7ff4e2d9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff4e415e1e0 0x7ff4e415cbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40494/0 (socket says 172.21.15.92:40494) 2023-11-24T06:11:48.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:48.763+0000 7ff4e2d9d640 1 -- 172.21.15.92:0/1064673311 learned_addr learned my addr 172.21.15.92:0/1064673311 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:11:48.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:48.764+0000 7ff4e259c640 1 --2- 172.21.15.92:0/1064673311 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff4e40cf7d0 0x7ff4e4158ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:48.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:48.764+0000 7ff4e1d9b640 1 --2- 172.21.15.92:0/1064673311 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff4e4143ae0 0x7ff4e413c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:48.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:48.764+0000 7ff4e2d9d640 1 -- 172.21.15.92:0/1064673311 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff4e4143ae0 msgr2=0x7ff4e413c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:48.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:48.764+0000 7ff4e2d9d640 1 --2- 172.21.15.92:0/1064673311 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff4e4143ae0 0x7ff4e413c0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:48.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:48.764+0000 7ff4e2d9d640 1 -- 172.21.15.92:0/1064673311 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff4e40cf7d0 msgr2=0x7ff4e4158ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:48.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:48.764+0000 7ff4e2d9d640 1 --2- 172.21.15.92:0/1064673311 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff4e40cf7d0 0x7ff4e4158ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:48.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:48.764+0000 7ff4e2d9d640 1 -- 172.21.15.92:0/1064673311 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4d40094d0 con 0x7ff4e415e1e0 2023-11-24T06:11:48.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:48.764+0000 7ff4e1d9b640 1 --2- 172.21.15.92:0/1064673311 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff4e4143ae0 0x7ff4e413c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:11:48.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:48.764+0000 7ff4e2d9d640 1 --2- 172.21.15.92:0/1064673311 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff4e415e1e0 0x7ff4e415cbc0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7ff4dc00cd50 tx=0x7ff4dc00a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:48.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:48.764+0000 7ff4d2ffd640 1 -- 172.21.15.92:0/1064673311 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4dc00f680 con 0x7ff4e415e1e0 2023-11-24T06:11:48.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:48.764+0000 7ff4d2ffd640 1 -- 172.21.15.92:0/1064673311 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4dc00f800 con 0x7ff4e415e1e0 2023-11-24T06:11:48.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:48.764+0000 7ff4d2ffd640 1 -- 172.21.15.92:0/1064673311 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4dc00fbb0 con 0x7ff4e415e1e0 2023-11-24T06:11:48.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:48.764+0000 7ff4e8f4c640 1 -- 172.21.15.92:0/1064673311 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4e413d340 con 0x7ff4e415e1e0 2023-11-24T06:11:48.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:48.765+0000 7ff4e8f4c640 1 -- 172.21.15.92:0/1064673311 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4e4159790 con 0x7ff4e415e1e0 2023-11-24T06:11:48.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:48.766+0000 7ff4e8f4c640 1 -- 172.21.15.92:0/1064673311 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4e40b2bf0 con 0x7ff4e415e1e0 2023-11-24T06:11:48.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:48.767+0000 7ff4d2ffd640 1 -- 172.21.15.92:0/1064673311 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ff4dc00b040 con 0x7ff4e415e1e0 2023-11-24T06:11:48.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:48.768+0000 7ff4d2ffd640 1 --2- 172.21.15.92:0/1064673311 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7ff4c406ae40 0x7ff4c406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:48.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:48.768+0000 7ff4d2ffd640 1 -- 172.21.15.92:0/1064673311 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff4dc08a010 con 0x7ff4e415e1e0 2023-11-24T06:11:48.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:48.768+0000 7ff4e259c640 1 --2- 172.21.15.92:0/1064673311 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7ff4c406ae40 0x7ff4c406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:48.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:48.769+0000 7ff4e259c640 1 --2- 172.21.15.92:0/1064673311 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7ff4c406ae40 0x7ff4c406d300 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ff4d4036000 tx=0x7ff4d403c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:48.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:48.772+0000 7ff4d2ffd640 1 -- 172.21.15.92:0/1064673311 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff4e40b2bf0 con 0x7ff4e415e1e0 2023-11-24T06:11:48.999 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:48.996+0000 7ff4e8f4c640 1 -- 172.21.15.92:0/1064673311 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff4e413d860 con 0x7ff4e415e1e0 2023-11-24T06:11:49.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:48.997+0000 7ff4d2ffd640 1 -- 172.21.15.92:0/1064673311 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4919 (secure 0 0 0) 0x7ff4e413d860 con 0x7ff4e415e1e0 2023-11-24T06:11:49.000 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-24T06:11:49.001 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4637,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6834/2665242481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":2665242481},{"type":"v1","addr":"172.21.15.118:6834","nonce":2665242481}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":4640,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6835/2822230615","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6833","nonce":2822230615},{"type":"v1","addr":"172.21.15.118:6835","nonce":2822230615}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":4741,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6836/3889219158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":3889219158},{"type":"v1","addr":"172.21.15.92:6836","nonce":3889219158}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T06:11:44.503003+0000","modified":"2023-11-24T06:11:47.998422+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5235},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5235":{"gid":5235,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.92:6837/1413573590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":1413573590},{"type":"v1","addr":"172.21.15.92:6837","nonce":1413573590}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-24T06:11:49.001 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 17 2023-11-24T06:11:49.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:48.999+0000 7ff4e8f4c640 1 -- 172.21.15.92:0/1064673311 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7ff4c406ae40 msgr2=0x7ff4c406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:49.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:48.999+0000 7ff4e8f4c640 1 --2- 172.21.15.92:0/1064673311 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7ff4c406ae40 0x7ff4c406d300 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ff4d4036000 tx=0x7ff4d403c040 comp rx=0 tx=0).stop 2023-11-24T06:11:49.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:48.999+0000 7ff4e8f4c640 1 -- 172.21.15.92:0/1064673311 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff4e415e1e0 msgr2=0x7ff4e415cbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:49.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:48.999+0000 7ff4e8f4c640 1 --2- 172.21.15.92:0/1064673311 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff4e415e1e0 0x7ff4e415cbc0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7ff4dc00cd50 tx=0x7ff4dc00a650 comp rx=0 tx=0).stop 2023-11-24T06:11:49.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:49.000+0000 7ff4e8f4c640 1 -- 172.21.15.92:0/1064673311 shutdown_connections 2023-11-24T06:11:49.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:49.000+0000 7ff4e8f4c640 1 --2- 172.21.15.92:0/1064673311 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7ff4c406ae40 0x7ff4c406d300 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:49.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:49.000+0000 7ff4e8f4c640 1 --2- 172.21.15.92:0/1064673311 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff4e415e1e0 0x7ff4e415cbc0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:49.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:49.000+0000 7ff4e8f4c640 1 --2- 172.21.15.92:0/1064673311 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff4e4143ae0 0x7ff4e413c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:49.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:49.000+0000 7ff4e8f4c640 1 --2- 172.21.15.92:0/1064673311 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff4e40cf7d0 0x7ff4e4158ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:49.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:49.000+0000 7ff4e8f4c640 1 -- 172.21.15.92:0/1064673311 >> 172.21.15.92:0/1064673311 conn(0x7ff4e406a800 msgr2=0x7ff4e40aea20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:49.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:49.000+0000 7ff4e8f4c640 1 -- 172.21.15.92:0/1064673311 shutdown_connections 2023-11-24T06:11:49.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:49.000+0000 7ff4e8f4c640 1 -- 172.21.15.92:0/1064673311 wait complete. 2023-11-24T06:11:49.021 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:11:49.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:49.223+0000 7eff3e87c640 1 Processor -- start 2023-11-24T06:11:49.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:49.223+0000 7eff3e87c640 1 -- start start 2023-11-24T06:11:49.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:49.224+0000 7eff3e87c640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7eff3813df80 0x7eff3815bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:49.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:49.224+0000 7eff3e87c640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7eff380cf9a0 0x7eff380cd840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:49.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:49.224+0000 7eff3e87c640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff3806c470 con 0x7eff3813e360 2023-11-24T06:11:49.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:49.224+0000 7eff3e87c640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff3806c750 con 0x7eff3813df80 2023-11-24T06:11:49.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:49.225+0000 7eff3e87c640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff3806ca30 con 0x7eff380cf9a0 2023-11-24T06:11:49.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:49.225+0000 7eff37fff640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7eff3813df80 0x7eff3815bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:49.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:49.225+0000 7eff37fff640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7eff3813df80 0x7eff3815bee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58594/0 (socket says 172.21.15.92:58594) 2023-11-24T06:11:49.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:49.225+0000 7eff37fff640 1 -- 172.21.15.92:0/1701236290 learned_addr learned my addr 172.21.15.92:0/1701236290 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:11:49.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:49.225+0000 7eff3ce19640 1 --2- 172.21.15.92:0/1701236290 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7eff380cf9a0 0x7eff380cd840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:49.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:49.225+0000 7eff37fff640 1 -- 172.21.15.92:0/1701236290 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7eff380cf9a0 msgr2=0x7eff380cd840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:49.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:49.225+0000 7eff37fff640 1 --2- 172.21.15.92:0/1701236290 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7eff380cf9a0 0x7eff380cd840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:49.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:49.225+0000 7eff37fff640 1 -- 172.21.15.92:0/1701236290 >> v1:172.21.15.92:6789/0 conn(0x7eff3813e360 legacy=0x7eff380cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:49.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:49.225+0000 7eff37fff640 1 -- 172.21.15.92:0/1701236290 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff3806cd10 con 0x7eff3813df80 2023-11-24T06:11:49.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:49.225+0000 7eff3ce19640 1 --2- 172.21.15.92:0/1701236290 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7eff380cf9a0 0x7eff380cd840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:11:49.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:49.226+0000 7eff37fff640 1 --2- 172.21.15.92:0/1701236290 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7eff3813df80 0x7eff3815bee0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7eff20009870 tx=0x7eff2002b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=476ce0f85c993f8c server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:49.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:49.227+0000 7eff36ffd640 1 -- 172.21.15.92:0/1701236290 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff20030840 con 0x7eff3813df80 2023-11-24T06:11:49.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:49.227+0000 7eff36ffd640 1 -- 172.21.15.92:0/1701236290 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff200309a0 con 0x7eff3813df80 2023-11-24T06:11:49.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:49.227+0000 7eff36ffd640 1 -- 172.21.15.92:0/1701236290 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff20030d30 con 0x7eff3813df80 2023-11-24T06:11:49.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:49.227+0000 7eff3e87c640 1 -- 172.21.15.92:0/1701236290 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7eff3813df80 msgr2=0x7eff3815bee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:49.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:49.227+0000 7eff3e87c640 1 --2- 172.21.15.92:0/1701236290 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7eff3813df80 0x7eff3815bee0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7eff20009870 tx=0x7eff2002b980 comp rx=0 tx=0).stop 2023-11-24T06:11:49.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:49.227+0000 7eff3e87c640 1 -- 172.21.15.92:0/1701236290 shutdown_connections 2023-11-24T06:11:49.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:49.227+0000 7eff3e87c640 1 --2- 172.21.15.92:0/1701236290 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7eff380cf9a0 0x7eff380cd840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:49.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:49.227+0000 7eff3e87c640 1 --2- 172.21.15.92:0/1701236290 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7eff3813df80 0x7eff3815bee0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:49.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:49.227+0000 7eff3e87c640 1 -- 172.21.15.92:0/1701236290 >> 172.21.15.92:0/1701236290 conn(0x7eff3806a800 msgr2=0x7eff3806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:49.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:49.228+0000 7eff3e87c640 1 -- 172.21.15.92:0/1701236290 shutdown_connections 2023-11-24T06:11:49.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:49.228+0000 7eff3e87c640 1 -- 172.21.15.92:0/1701236290 wait complete. 2023-11-24T06:11:49.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:49.229+0000 7eff3e87c640 1 Processor -- start 2023-11-24T06:11:49.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:49.230+0000 7eff3e87c640 1 -- start start 2023-11-24T06:11:49.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:49.231+0000 7eff3e87c640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7eff380cf9a0 0x7eff38152fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:49.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:49.231+0000 7eff3e87c640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7eff3813df80 0x7eff380a8fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:49.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:49.231+0000 7eff3e87c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7eff3813e360 0x7eff3815a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:49.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:49.231+0000 7eff3e87c640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff381f66c0 con 0x7eff3813e360 2023-11-24T06:11:49.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:49.231+0000 7eff3e87c640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff381f69a0 con 0x7eff3813df80 2023-11-24T06:11:49.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:49.231+0000 7eff3e87c640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff381f6c80 con 0x7eff380cf9a0 2023-11-24T06:11:49.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:49.231+0000 7eff3ce19640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7eff3813e360 0x7eff3815a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:49.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:49.231+0000 7eff3ce19640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7eff3813e360 0x7eff3815a180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58596/0 (socket says 172.21.15.92:58596) 2023-11-24T06:11:49.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:49.231+0000 7eff3ce19640 1 -- 172.21.15.92:0/3128579754 learned_addr learned my addr 172.21.15.92:0/3128579754 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:11:49.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:49.231+0000 7eff37fff640 1 --2- 172.21.15.92:0/3128579754 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7eff380cf9a0 0x7eff38152fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:49.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:49.231+0000 7eff377fe640 1 --2- 172.21.15.92:0/3128579754 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7eff3813df80 0x7eff380a8fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:49.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:49.232+0000 7eff3ce19640 1 -- 172.21.15.92:0/3128579754 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7eff380cf9a0 msgr2=0x7eff38152fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:49.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:49.232+0000 7eff3ce19640 1 --2- 172.21.15.92:0/3128579754 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7eff380cf9a0 0x7eff38152fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:49.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:49.232+0000 7eff3ce19640 1 -- 172.21.15.92:0/3128579754 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7eff3813df80 msgr2=0x7eff380a8fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:49.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:49.232+0000 7eff3ce19640 1 --2- 172.21.15.92:0/3128579754 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7eff3813df80 0x7eff380a8fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:49.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:49.232+0000 7eff3ce19640 1 -- 172.21.15.92:0/3128579754 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff200094d0 con 0x7eff3813e360 2023-11-24T06:11:49.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:49.232+0000 7eff37fff640 1 --2- 172.21.15.92:0/3128579754 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7eff380cf9a0 0x7eff38152fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:11:49.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:49.232+0000 7eff3ce19640 1 --2- 172.21.15.92:0/3128579754 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7eff3813e360 0x7eff3815a180 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7eff3000cd80 tx=0x7eff3000a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:49.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:49.232+0000 7eff34ff9640 1 -- 172.21.15.92:0/3128579754 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff3000f680 con 0x7eff3813e360 2023-11-24T06:11:49.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:49.232+0000 7eff34ff9640 1 -- 172.21.15.92:0/3128579754 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff3000f800 con 0x7eff3813e360 2023-11-24T06:11:49.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:49.232+0000 7eff34ff9640 1 -- 172.21.15.92:0/3128579754 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff3000fbb0 con 0x7eff3813e360 2023-11-24T06:11:49.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:49.232+0000 7eff3e87c640 1 -- 172.21.15.92:0/3128579754 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff381f6f60 con 0x7eff3813e360 2023-11-24T06:11:49.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:49.233+0000 7eff3e87c640 1 -- 172.21.15.92:0/3128579754 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff38153510 con 0x7eff3813e360 2023-11-24T06:11:49.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:49.234+0000 7eff3e87c640 1 -- 172.21.15.92:0/3128579754 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff380703e0 con 0x7eff3813e360 2023-11-24T06:11:49.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:49.234+0000 7eff34ff9640 1 -- 172.21.15.92:0/3128579754 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7eff3000b040 con 0x7eff3813e360 2023-11-24T06:11:49.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:49.235+0000 7eff34ff9640 1 --2- 172.21.15.92:0/3128579754 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7eff1806ae40 0x7eff1806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:49.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:49.235+0000 7eff34ff9640 1 -- 172.21.15.92:0/3128579754 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7eff30089ff0 con 0x7eff3813e360 2023-11-24T06:11:49.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:49.236+0000 7eff37fff640 1 --2- 172.21.15.92:0/3128579754 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7eff1806ae40 0x7eff1806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:49.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:49.236+0000 7eff37fff640 1 --2- 172.21.15.92:0/3128579754 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7eff1806ae40 0x7eff1806d300 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7eff20036000 tx=0x7eff20008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:49.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:49.240+0000 7eff34ff9640 1 -- 172.21.15.92:0/3128579754 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7eff30055060 con 0x7eff3813e360 2023-11-24T06:11:49.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:49.430+0000 7eff3e87c640 1 -- 172.21.15.92:0/3128579754 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7eff380675d0 con 0x7eff3813e360 2023-11-24T06:11:49.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:49.433+0000 7eff34ff9640 1 -- 172.21.15.92:0/3128579754 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) v1 ==== 74+0+16521 (secure 0 0 0) 0x7eff30058680 con 0x7eff3813e360 2023-11-24T06:11:49.437 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-24T06:11:49.437 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":33,"fsid":"2d6dee5f-8aa3-470f-931a-50ade963b1cb","created":"2023-11-24T06:10:04.707204+0000","modified":"2023-11-24T06:11:46.026094+0000","last_up_change":"2023-11-24T06:10:17.598352+0000","last_in_change":"2023-11-24T06:10:09.040793+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:10:17.733098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:11:42.241681+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-11-24T06:11:43.522720+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ff37a9fb-135f-471f-b4b0-f7b5772006c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6803","nonce":20560714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6805","nonce":20560714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6809","nonce":20560714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6807","nonce":20560714}]},"public_addr":"172.21.15.92:6803/20560714","cluster_addr":"172.21.15.92:6805/20560714","heartbeat_back_addr":"172.21.15.92:6809/20560714","heartbeat_front_addr":"172.21.15.92:6807/20560714","state":["exists","up"]},{"osd":1,"uuid":"03cdd353-dcf1-4f38-94fe-a795137c06b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6819","nonce":2451239906}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6821","nonce":2451239906}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6825","nonce":2451239906}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6823","nonce":2451239906}]},"public_addr":"172.21.15.92:6819/2451239906","cluster_addr":"172.21.15.92:6821/2451239906","heartbeat_back_addr":"172.21.15.92:6825/2451239906","heartbeat_front_addr":"172.21.15.92:6823/2451239906","state":["exists","up"]},{"osd":2,"uuid":"a4a20d80-5d57-484b-becd-9aa8070dd3aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6811","nonce":1124979277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6813","nonce":1124979277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6817","nonce":1124979277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6815","nonce":1124979277}]},"public_addr":"172.21.15.92:6811/1124979277","cluster_addr":"172.21.15.92:6813/1124979277","heartbeat_back_addr":"172.21.15.92:6817/1124979277","heartbeat_front_addr":"172.21.15.92:6815/1124979277","state":["exists","up"]},{"osd":3,"uuid":"4648d7ce-9b29-4583-9425-4ba8fd2b1903","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6827","nonce":1734115782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6829","nonce":1734115782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6833","nonce":1734115782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6831","nonce":1734115782}]},"public_addr":"172.21.15.92:6827/1734115782","cluster_addr":"172.21.15.92:6829/1734115782","heartbeat_back_addr":"172.21.15.92:6833/1734115782","heartbeat_front_addr":"172.21.15.92:6831/1734115782","state":["exists","up"]},{"osd":4,"uuid":"3bc9d7eb-c6fa-4fcf-a3ec-ade604e05aac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6819","nonce":3425937229}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6822","nonce":3425937229}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6830","nonce":3425937229}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6826","nonce":3425937229}]},"public_addr":"172.21.15.118:6819/3425937229","cluster_addr":"172.21.15.118:6822/3425937229","heartbeat_back_addr":"172.21.15.118:6830/3425937229","heartbeat_front_addr":"172.21.15.118:6826/3425937229","state":["exists","up"]},{"osd":5,"uuid":"dce9ba6d-59ed-46d1-84fc-a84b13de10c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6801","nonce":827455984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6803","nonce":827455984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6807","nonce":827455984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6805","nonce":827455984}]},"public_addr":"172.21.15.118:6801/827455984","cluster_addr":"172.21.15.118:6803/827455984","heartbeat_back_addr":"172.21.15.118:6807/827455984","heartbeat_front_addr":"172.21.15.118:6805/827455984","state":["exists","up"]},{"osd":6,"uuid":"82aec504-ae58-4709-8d30-46f2b12081b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6809","nonce":2946986141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6811","nonce":2946986141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6815","nonce":2946986141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6813","nonce":2946986141}]},"public_addr":"172.21.15.118:6809/2946986141","cluster_addr":"172.21.15.118:6811/2946986141","heartbeat_back_addr":"172.21.15.118:6815/2946986141","heartbeat_front_addr":"172.21.15.118:6813/2946986141","state":["exists","up"]},{"osd":7,"uuid":"4ed4c624-c88f-4d35-bdb1-bf209b5dffb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6817","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6818","nonce":48122059}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6821","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6823","nonce":48122059}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6829","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6831","nonce":48122059}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6825","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6827","nonce":48122059}]},"public_addr":"172.21.15.118:6818/48122059","cluster_addr":"172.21.15.118:6823/48122059","heartbeat_back_addr":"172.21.15.118:6831/48122059","heartbeat_front_addr":"172.21.15.118:6827/48122059","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.536664+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.368324+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.271201+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.342971+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.621832+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.387110+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.632946+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.416032+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:11:49.439 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:49.436+0000 7eff3e87c640 1 -- 172.21.15.92:0/3128579754 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7eff1806ae40 msgr2=0x7eff1806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:49.439 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:49.436+0000 7eff3e87c640 1 --2- 172.21.15.92:0/3128579754 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7eff1806ae40 0x7eff1806d300 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7eff20036000 tx=0x7eff20008030 comp rx=0 tx=0).stop 2023-11-24T06:11:49.439 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:49.436+0000 7eff3e87c640 1 -- 172.21.15.92:0/3128579754 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7eff3813e360 msgr2=0x7eff3815a180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:49.439 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:49.436+0000 7eff3e87c640 1 --2- 172.21.15.92:0/3128579754 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7eff3813e360 0x7eff3815a180 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7eff3000cd80 tx=0x7eff3000a650 comp rx=0 tx=0).stop 2023-11-24T06:11:49.440 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:49.436+0000 7eff3e87c640 1 -- 172.21.15.92:0/3128579754 shutdown_connections 2023-11-24T06:11:49.440 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:49.436+0000 7eff3e87c640 1 --2- 172.21.15.92:0/3128579754 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7eff1806ae40 0x7eff1806d300 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:49.440 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:49.436+0000 7eff3e87c640 1 --2- 172.21.15.92:0/3128579754 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7eff3813e360 0x7eff3815a180 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:49.440 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:49.436+0000 7eff3e87c640 1 --2- 172.21.15.92:0/3128579754 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7eff3813df80 0x7eff380a8fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:49.440 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:49.436+0000 7eff3e87c640 1 --2- 172.21.15.92:0/3128579754 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7eff380cf9a0 0x7eff38152fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:49.440 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:49.436+0000 7eff3e87c640 1 -- 172.21.15.92:0/3128579754 >> 172.21.15.92:0/3128579754 conn(0x7eff3806a800 msgr2=0x7eff3806abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:49.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:49.437+0000 7eff3e87c640 1 -- 172.21.15.92:0/3128579754 shutdown_connections 2023-11-24T06:11:49.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:49.437+0000 7eff3e87c640 1 -- 172.21.15.92:0/3128579754 wait complete. 2023-11-24T06:11:49.459 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-11-24T06:11:49.459 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-24T06:11:49.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:49.668+0000 7fb0ff59e640 1 Processor -- start 2023-11-24T06:11:49.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:49.669+0000 7fb0ff59e640 1 -- start start 2023-11-24T06:11:49.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:49.670+0000 7fb0ff59e640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb100071f00 0x7fb100146770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:49.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:49.670+0000 7fb0ff59e640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1000b2e10 con 0x7fb100071410 2023-11-24T06:11:49.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:49.670+0000 7fb0ff59e640 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb10006cec0 con 0x7fb10006c7c0 2023-11-24T06:11:49.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:49.670+0000 7fb0ff59e640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb10006d1a0 con 0x7fb100071f00 2023-11-24T06:11:49.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:49.671+0000 7fb0fe59c640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fb100071410 0x7fb1000717f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:41226/0 (socket says 172.21.15.92:41226) 2023-11-24T06:11:49.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:49.671+0000 7fb0fe59c640 1 -- 172.21.15.92:0/579006569 learned_addr learned my addr 172.21.15.92:0/579006569 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-24T06:11:49.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:49.671+0000 7fb0fdd9b640 1 -- 172.21.15.92:0/579006569 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2371625903 0 0) 0x7fb1000b2e10 con 0x7fb100071410 2023-11-24T06:11:49.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:49.671+0000 7fb0fdd9b640 1 -- 172.21.15.92:0/579006569 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb0ec003520 con 0x7fb100071410 2023-11-24T06:11:49.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:49.671+0000 7fb0f7fff640 1 --2- 172.21.15.92:0/579006569 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb100071f00 0x7fb100146770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:49.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:49.671+0000 7fb0fdd9b640 1 -- 172.21.15.92:0/579006569 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2034395685 0 0) 0x7fb0ec003520 con 0x7fb100071410 2023-11-24T06:11:49.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:49.671+0000 7fb0fdd9b640 1 -- 172.21.15.92:0/579006569 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb100071f00 msgr2=0x7fb100146770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:49.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:49.671+0000 7fb0fdd9b640 1 --2- 172.21.15.92:0/579006569 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb100071f00 0x7fb100146770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:49.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:49.671+0000 7fb0fdd9b640 1 -- 172.21.15.92:0/579006569 >> v1:172.21.15.118:6789/0 conn(0x7fb10006c7c0 legacy=0x7fb100166cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:49.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:49.671+0000 7fb0fdd9b640 1 -- 172.21.15.92:0/579006569 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb10006d480 con 0x7fb100071410 2023-11-24T06:11:49.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:49.671+0000 7fb0fdd9b640 1 -- 172.21.15.92:0/579006569 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2607661350 0 0) 0x7fb0f0001670 con 0x7fb100071410 2023-11-24T06:11:49.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:49.672+0000 7fb0fdd9b640 1 -- 172.21.15.92:0/579006569 >> v1:172.21.15.92:6789/0 conn(0x7fb100071410 legacy=0x7fb1000717f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:49.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:49.672+0000 7fb0fdd9b640 1 --2- 172.21.15.92:0/579006569 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb0ec0039a0 0x7fb0ec003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:49.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:49.672+0000 7fb0fdd9b640 1 --2- 172.21.15.92:0/579006569 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb0ec0047c0 0x7fb0ec004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:49.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:49.672+0000 7fb0fdd9b640 1 --2- 172.21.15.92:0/579006569 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb100071f00 0x7fb0ec0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:49.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:49.672+0000 7fb0fdd9b640 1 -- 172.21.15.92:0/579006569 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb10006d1a0 con 0x7fb0ec0047c0 2023-11-24T06:11:49.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:49.672+0000 7fb0fdd9b640 1 -- 172.21.15.92:0/579006569 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0ec00bae0 con 0x7fb0ec0039a0 2023-11-24T06:11:49.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:49.672+0000 7fb0fdd9b640 1 -- 172.21.15.92:0/579006569 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0ec00bca0 con 0x7fb100071f00 2023-11-24T06:11:49.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:49.672+0000 7fb0f7fff640 1 --2- 172.21.15.92:0/579006569 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb0ec0047c0 0x7fb0ec004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:49.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:49.672+0000 7fb0fe59c640 1 --2- 172.21.15.92:0/579006569 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb0ec0039a0 0x7fb0ec003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:49.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:49.672+0000 7fb0f7fff640 1 -- 172.21.15.92:0/579006569 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb100071f00 msgr2=0x7fb0ec0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:49.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:49.672+0000 7fb0fed9d640 1 --2- 172.21.15.92:0/579006569 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb100071f00 0x7fb0ec0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:49.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:49.672+0000 7fb0f7fff640 1 --2- 172.21.15.92:0/579006569 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb100071f00 0x7fb0ec0051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:49.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:49.672+0000 7fb0f7fff640 1 -- 172.21.15.92:0/579006569 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb0ec0039a0 msgr2=0x7fb0ec003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:49.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:49.672+0000 7fb0f7fff640 1 --2- 172.21.15.92:0/579006569 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb0ec0039a0 0x7fb0ec003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:49.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:49.672+0000 7fb0f7fff640 1 -- 172.21.15.92:0/579006569 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0ec003520 con 0x7fb0ec0047c0 2023-11-24T06:11:49.680 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:49.674+0000 7fb0f7fff640 1 --2- 172.21.15.92:0/579006569 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb0ec0047c0 0x7fb0ec004c40 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fb0e800ba30 tx=0x7fb0ec0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e687a2bd8b7fc1e1 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:49.680 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:49.674+0000 7fb0fdd9b640 1 -- 172.21.15.92:0/579006569 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0e8032640 con 0x7fb0ec0047c0 2023-11-24T06:11:49.680 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:49.674+0000 7fb0fdd9b640 1 -- 172.21.15.92:0/579006569 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0e80327a0 con 0x7fb0ec0047c0 2023-11-24T06:11:49.680 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:49.674+0000 7fb0fdd9b640 1 -- 172.21.15.92:0/579006569 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0e8032b30 con 0x7fb0ec0047c0 2023-11-24T06:11:49.680 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:49.674+0000 7fb0ff59e640 1 -- 172.21.15.92:0/579006569 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb0ec0047c0 msgr2=0x7fb0ec004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:49.680 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:49.674+0000 7fb0ff59e640 1 --2- 172.21.15.92:0/579006569 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb0ec0047c0 0x7fb0ec004c40 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fb0e800ba30 tx=0x7fb0ec0057f0 comp rx=0 tx=0).stop 2023-11-24T06:11:49.680 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:49.674+0000 7fb0fed9d640 1 -- 172.21.15.92:0/579006569 reap_dead start 2023-11-24T06:11:49.680 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:49.675+0000 7fb0ff59e640 1 -- 172.21.15.92:0/579006569 shutdown_connections 2023-11-24T06:11:49.680 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:49.675+0000 7fb0ff59e640 1 -- 172.21.15.92:0/579006569 >> 172.21.15.92:0/579006569 conn(0x7fb10006a800 msgr2=0x7fb10006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:49.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:49.675+0000 7fb0ff59e640 1 -- 172.21.15.92:0/579006569 shutdown_connections 2023-11-24T06:11:49.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:49.675+0000 7fb0ff59e640 1 -- 172.21.15.92:0/579006569 wait complete. 2023-11-24T06:11:49.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:49.676+0000 7fb0ff59e640 1 Processor -- start 2023-11-24T06:11:49.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:49.677+0000 7fb0ff59e640 1 -- start start 2023-11-24T06:11:49.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:49.677+0000 7fb0ff59e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb1000ce460 0x7fb10014f200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:49.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:49.677+0000 7fb0ff59e640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb1000ce840 0x7fb10014d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:49.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:49.677+0000 7fb0ff59e640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb10014f880 0x7fb10014fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:49.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:49.677+0000 7fb0ff59e640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb100150270 con 0x7fb1000ce460 2023-11-24T06:11:49.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:49.677+0000 7fb0ff59e640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb10014e160 con 0x7fb10014f880 2023-11-24T06:11:49.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:49.677+0000 7fb0ff59e640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb10014e440 con 0x7fb1000ce840 2023-11-24T06:11:49.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:49.678+0000 7fb0fe59c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb1000ce460 0x7fb10014f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:49.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:49.678+0000 7fb0fe59c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb1000ce460 0x7fb10014f200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58618/0 (socket says 172.21.15.92:58618) 2023-11-24T06:11:49.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:49.678+0000 7fb0fe59c640 1 -- 172.21.15.92:0/1215975082 learned_addr learned my addr 172.21.15.92:0/1215975082 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:11:49.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:49.678+0000 7fb0fe59c640 1 -- 172.21.15.92:0/1215975082 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb1000ce840 msgr2=0x7fb10014d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:49.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:49.678+0000 7fb0fe59c640 1 --2- 172.21.15.92:0/1215975082 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb1000ce840 0x7fb10014d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:49.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:49.678+0000 7fb0fe59c640 1 -- 172.21.15.92:0/1215975082 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb10014f880 msgr2=0x7fb10014fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:49.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:49.678+0000 7fb0fe59c640 1 --2- 172.21.15.92:0/1215975082 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb10014f880 0x7fb10014fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:49.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:49.678+0000 7fb0fe59c640 1 -- 172.21.15.92:0/1215975082 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb10014e720 con 0x7fb1000ce460 2023-11-24T06:11:49.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:49.678+0000 7fb0fe59c640 1 --2- 172.21.15.92:0/1215975082 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb1000ce460 0x7fb10014f200 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fb0f000e7f0 tx=0x7fb0f000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:49.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:49.679+0000 7fb0f6ffd640 1 -- 172.21.15.92:0/1215975082 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0f000c8d0 con 0x7fb1000ce460 2023-11-24T06:11:49.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:49.679+0000 7fb0f6ffd640 1 -- 172.21.15.92:0/1215975082 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0f000ca30 con 0x7fb1000ce460 2023-11-24T06:11:49.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:49.679+0000 7fb0ff59e640 1 -- 172.21.15.92:0/1215975082 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb10014ea00 con 0x7fb1000ce460 2023-11-24T06:11:49.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:49.679+0000 7fb0f6ffd640 1 -- 172.21.15.92:0/1215975082 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0f000cdc0 con 0x7fb1000ce460 2023-11-24T06:11:49.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:49.679+0000 7fb0ff59e640 1 -- 172.21.15.92:0/1215975082 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb10014dd30 con 0x7fb1000ce460 2023-11-24T06:11:49.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:49.680+0000 7fb0ff59e640 1 -- 172.21.15.92:0/1215975082 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1000b1980 con 0x7fb1000ce460 2023-11-24T06:11:49.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:49.681+0000 7fb0f6ffd640 1 -- 172.21.15.92:0/1215975082 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb0f001d460 con 0x7fb1000ce460 2023-11-24T06:11:49.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:49.682+0000 7fb0f6ffd640 1 --2- 172.21.15.92:0/1215975082 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fb0e006ae40 0x7fb0e006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:49.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:49.682+0000 7fb0f6ffd640 1 -- 172.21.15.92:0/1215975082 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb0f0089710 con 0x7fb1000ce460 2023-11-24T06:11:49.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:49.682+0000 7fb0f7fff640 1 --2- 172.21.15.92:0/1215975082 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fb0e006ae40 0x7fb0e006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:49.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:49.683+0000 7fb0f7fff640 1 --2- 172.21.15.92:0/1215975082 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fb0e006ae40 0x7fb0e006d300 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fb0e800ba00 tx=0x7fb0e8032000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:49.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:49.686+0000 7fb0f6ffd640 1 -- 172.21.15.92:0/1215975082 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb0f0054730 con 0x7fb1000ce460 2023-11-24T06:11:49.876 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:49.873+0000 7fb0ff59e640 1 -- 172.21.15.92:0/1215975082 --> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb10006cbe0 con 0x7fb0e006ae40 2023-11-24T06:11:49.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:49.886+0000 7fb0f6ffd640 1 -- 172.21.15.92:0/1215975082 <== mgr.4101 v2:172.21.15.92:6800/784027401 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+287415 (secure 0 0 0) 0x7fb0e8032910 con 0x7fb0e006ae40 2023-11-24T06:11:49.892 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-24T06:11:49.895 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2023-11-24T06:11:49.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:49.893+0000 7fb0ff59e640 1 -- 172.21.15.92:0/1215975082 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fb0e006ae40 msgr2=0x7fb0e006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:49.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:49.893+0000 7fb0ff59e640 1 --2- 172.21.15.92:0/1215975082 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fb0e006ae40 0x7fb0e006d300 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fb0e800ba00 tx=0x7fb0e8032000 comp rx=0 tx=0).stop 2023-11-24T06:11:49.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:49.893+0000 7fb0ff59e640 1 -- 172.21.15.92:0/1215975082 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb1000ce460 msgr2=0x7fb10014f200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:49.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:49.893+0000 7fb0ff59e640 1 --2- 172.21.15.92:0/1215975082 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb1000ce460 0x7fb10014f200 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fb0f000e7f0 tx=0x7fb0f000ecb0 comp rx=0 tx=0).stop 2023-11-24T06:11:49.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:49.893+0000 7fb0ff59e640 1 -- 172.21.15.92:0/1215975082 shutdown_connections 2023-11-24T06:11:49.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:49.893+0000 7fb0ff59e640 1 --2- 172.21.15.92:0/1215975082 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fb0e006ae40 0x7fb0e006d300 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:49.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:49.893+0000 7fb0ff59e640 1 --2- 172.21.15.92:0/1215975082 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb10014f880 0x7fb10014fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:49.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:49.893+0000 7fb0ff59e640 1 --2- 172.21.15.92:0/1215975082 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb1000ce840 0x7fb10014d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:49.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:49.893+0000 7fb0ff59e640 1 --2- 172.21.15.92:0/1215975082 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb1000ce460 0x7fb10014f200 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:49.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:49.893+0000 7fb0ff59e640 1 -- 172.21.15.92:0/1215975082 >> 172.21.15.92:0/1215975082 conn(0x7fb10006a800 msgr2=0x7fb1000af800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:49.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:49.893+0000 7fb0ff59e640 1 -- 172.21.15.92:0/1215975082 shutdown_connections 2023-11-24T06:11:49.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:49.893+0000 7fb0ff59e640 1 -- 172.21.15.92:0/1215975082 wait complete. 2023-11-24T06:11:49.916 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":72,"stamp":"2023-11-24T06:11:49.692027+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":170,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236944,"kb_used_data":4128,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754737776,"statfs":{"total":773094113280,"available":772851482624,"internally_reserved":0,"allocated":4227072,"data_stored":2359720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":33,"apply_latency_ms":33,"commit_latency_ns":33000000,"apply_latency_ns":33000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001647"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:11:46.043761+0000","last_change":"2023-11-24T06:11:45.058797+0000","last_active":"2023-11-24T06:11:46.043761+0000","last_peered":"2023-11-24T06:11:46.043761+0000","last_clean":"2023-11-24T06:11:46.043761+0000","last_became_active":"2023-11-24T06:11:45.057069+0000","last_became_peered":"2023-11-24T06:11:45.057069+0000","last_unstale":"2023-11-24T06:11:46.043761+0000","last_undegraded":"2023-11-24T06:11:46.043761+0000","last_fullsized":"2023-11-24T06:11:46.043761+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_clean_scrub_stamp":"2023-11-24T06:11:44.018686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:13:39.376575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:11:46.163519+0000","last_change":"2023-11-24T06:11:45.064449+0000","last_active":"2023-11-24T06:11:46.163519+0000","last_peered":"2023-11-24T06:11:46.163519+0000","last_clean":"2023-11-24T06:11:46.163519+0000","last_became_active":"2023-11-24T06:11:45.063848+0000","last_became_peered":"2023-11-24T06:11:45.063848+0000","last_unstale":"2023-11-24T06:11:46.163519+0000","last_undegraded":"2023-11-24T06:11:46.163519+0000","last_fullsized":"2023-11-24T06:11:46.163519+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_clean_scrub_stamp":"2023-11-24T06:11:44.018686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:16:44.582224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:11:46.163206+0000","last_change":"2023-11-24T06:11:45.067600+0000","last_active":"2023-11-24T06:11:46.163206+0000","last_peered":"2023-11-24T06:11:46.163206+0000","last_clean":"2023-11-24T06:11:46.163206+0000","last_became_active":"2023-11-24T06:11:45.067162+0000","last_became_peered":"2023-11-24T06:11:45.067162+0000","last_unstale":"2023-11-24T06:11:46.163206+0000","last_undegraded":"2023-11-24T06:11:46.163206+0000","last_fullsized":"2023-11-24T06:11:46.163206+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_clean_scrub_stamp":"2023-11-24T06:11:44.018686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:09:22.132463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:11:46.163962+0000","last_change":"2023-11-24T06:11:45.070948+0000","last_active":"2023-11-24T06:11:46.163962+0000","last_peered":"2023-11-24T06:11:46.163962+0000","last_clean":"2023-11-24T06:11:46.163962+0000","last_became_active":"2023-11-24T06:11:45.070666+0000","last_became_peered":"2023-11-24T06:11:45.070666+0000","last_unstale":"2023-11-24T06:11:46.163962+0000","last_undegraded":"2023-11-24T06:11:46.163962+0000","last_fullsized":"2023-11-24T06:11:46.163962+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_clean_scrub_stamp":"2023-11-24T06:11:44.018686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:55:17.339395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:11:46.042077+0000","last_change":"2023-11-24T06:11:45.062816+0000","last_active":"2023-11-24T06:11:46.042077+0000","last_peered":"2023-11-24T06:11:46.042077+0000","last_clean":"2023-11-24T06:11:46.042077+0000","last_became_active":"2023-11-24T06:11:45.062515+0000","last_became_peered":"2023-11-24T06:11:45.062515+0000","last_unstale":"2023-11-24T06:11:46.042077+0000","last_undegraded":"2023-11-24T06:11:46.042077+0000","last_fullsized":"2023-11-24T06:11:46.042077+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_clean_scrub_stamp":"2023-11-24T06:11:44.018686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:11:22.756651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:11:46.162595+0000","last_change":"2023-11-24T06:11:45.057850+0000","last_active":"2023-11-24T06:11:46.162595+0000","last_peered":"2023-11-24T06:11:46.162595+0000","last_clean":"2023-11-24T06:11:46.162595+0000","last_became_active":"2023-11-24T06:11:45.057542+0000","last_became_peered":"2023-11-24T06:11:45.057542+0000","last_unstale":"2023-11-24T06:11:46.162595+0000","last_undegraded":"2023-11-24T06:11:46.162595+0000","last_fullsized":"2023-11-24T06:11:46.162595+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_clean_scrub_stamp":"2023-11-24T06:11:44.018686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:40:45.042808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:11:46.163420+0000","last_change":"2023-11-24T06:11:45.061096+0000","last_active":"2023-11-24T06:11:46.163420+0000","last_peered":"2023-11-24T06:11:46.163420+0000","last_clean":"2023-11-24T06:11:46.163420+0000","last_became_active":"2023-11-24T06:11:45.059143+0000","last_became_peered":"2023-11-24T06:11:45.059143+0000","last_unstale":"2023-11-24T06:11:46.163420+0000","last_undegraded":"2023-11-24T06:11:46.163420+0000","last_fullsized":"2023-11-24T06:11:46.163420+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_clean_scrub_stamp":"2023-11-24T06:11:44.018686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:09:00.870428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:11:46.162583+0000","last_change":"2023-11-24T06:11:45.059120+0000","last_active":"2023-11-24T06:11:46.162583+0000","last_peered":"2023-11-24T06:11:46.162583+0000","last_clean":"2023-11-24T06:11:46.162583+0000","last_became_active":"2023-11-24T06:11:45.058237+0000","last_became_peered":"2023-11-24T06:11:45.058237+0000","last_unstale":"2023-11-24T06:11:46.162583+0000","last_undegraded":"2023-11-24T06:11:46.162583+0000","last_fullsized":"2023-11-24T06:11:46.162583+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_clean_scrub_stamp":"2023-11-24T06:11:44.018686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:15:02.243714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:11:46.044897+0000","last_change":"2023-11-24T06:11:45.070261+0000","last_active":"2023-11-24T06:11:46.044897+0000","last_peered":"2023-11-24T06:11:46.044897+0000","last_clean":"2023-11-24T06:11:46.044897+0000","last_became_active":"2023-11-24T06:11:45.069920+0000","last_became_peered":"2023-11-24T06:11:45.069920+0000","last_unstale":"2023-11-24T06:11:46.044897+0000","last_undegraded":"2023-11-24T06:11:46.044897+0000","last_fullsized":"2023-11-24T06:11:46.044897+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_clean_scrub_stamp":"2023-11-24T06:11:44.018686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:42:12.263089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:11:46.163614+0000","last_change":"2023-11-24T06:11:45.063665+0000","last_active":"2023-11-24T06:11:46.163614+0000","last_peered":"2023-11-24T06:11:46.163614+0000","last_clean":"2023-11-24T06:11:46.163614+0000","last_became_active":"2023-11-24T06:11:45.063365+0000","last_became_peered":"2023-11-24T06:11:45.063365+0000","last_unstale":"2023-11-24T06:11:46.163614+0000","last_undegraded":"2023-11-24T06:11:46.163614+0000","last_fullsized":"2023-11-24T06:11:46.163614+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_clean_scrub_stamp":"2023-11-24T06:11:44.018686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:31:16.199313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:11:46.164428+0000","last_change":"2023-11-24T06:11:45.067860+0000","last_active":"2023-11-24T06:11:46.164428+0000","last_peered":"2023-11-24T06:11:46.164428+0000","last_clean":"2023-11-24T06:11:46.164428+0000","last_became_active":"2023-11-24T06:11:45.067588+0000","last_became_peered":"2023-11-24T06:11:45.067588+0000","last_unstale":"2023-11-24T06:11:46.164428+0000","last_undegraded":"2023-11-24T06:11:46.164428+0000","last_fullsized":"2023-11-24T06:11:46.164428+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_clean_scrub_stamp":"2023-11-24T06:11:44.018686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:26:22.270872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:11:46.162513+0000","last_change":"2023-11-24T06:11:45.064149+0000","last_active":"2023-11-24T06:11:46.162513+0000","last_peered":"2023-11-24T06:11:46.162513+0000","last_clean":"2023-11-24T06:11:46.162513+0000","last_became_active":"2023-11-24T06:11:45.063599+0000","last_became_peered":"2023-11-24T06:11:45.063599+0000","last_unstale":"2023-11-24T06:11:46.162513+0000","last_undegraded":"2023-11-24T06:11:46.162513+0000","last_fullsized":"2023-11-24T06:11:46.162513+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_clean_scrub_stamp":"2023-11-24T06:11:44.018686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:19:03.083820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:11:46.042337+0000","last_change":"2023-11-24T06:11:45.055852+0000","last_active":"2023-11-24T06:11:46.042337+0000","last_peered":"2023-11-24T06:11:46.042337+0000","last_clean":"2023-11-24T06:11:46.042337+0000","last_became_active":"2023-11-24T06:11:45.055554+0000","last_became_peered":"2023-11-24T06:11:45.055554+0000","last_unstale":"2023-11-24T06:11:46.042337+0000","last_undegraded":"2023-11-24T06:11:46.042337+0000","last_fullsized":"2023-11-24T06:11:46.042337+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_clean_scrub_stamp":"2023-11-24T06:11:44.018686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:06:55.244746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:11:46.162778+0000","last_change":"2023-11-24T06:11:45.056844+0000","last_active":"2023-11-24T06:11:46.162778+0000","last_peered":"2023-11-24T06:11:46.162778+0000","last_clean":"2023-11-24T06:11:46.162778+0000","last_became_active":"2023-11-24T06:11:45.056431+0000","last_became_peered":"2023-11-24T06:11:45.056431+0000","last_unstale":"2023-11-24T06:11:46.162778+0000","last_undegraded":"2023-11-24T06:11:46.162778+0000","last_fullsized":"2023-11-24T06:11:46.162778+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_clean_scrub_stamp":"2023-11-24T06:11:44.018686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:34:37.780831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:11:46.162483+0000","last_change":"2023-11-24T06:11:45.054090+0000","last_active":"2023-11-24T06:11:46.162483+0000","last_peered":"2023-11-24T06:11:46.162483+0000","last_clean":"2023-11-24T06:11:46.162483+0000","last_became_active":"2023-11-24T06:11:45.053254+0000","last_became_peered":"2023-11-24T06:11:45.053254+0000","last_unstale":"2023-11-24T06:11:46.162483+0000","last_undegraded":"2023-11-24T06:11:46.162483+0000","last_fullsized":"2023-11-24T06:11:46.162483+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_clean_scrub_stamp":"2023-11-24T06:11:44.018686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:51:51.030233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:11:46.163168+0000","last_change":"2023-11-24T06:11:45.062778+0000","last_active":"2023-11-24T06:11:46.163168+0000","last_peered":"2023-11-24T06:11:46.163168+0000","last_clean":"2023-11-24T06:11:46.163168+0000","last_became_active":"2023-11-24T06:11:45.062448+0000","last_became_peered":"2023-11-24T06:11:45.062448+0000","last_unstale":"2023-11-24T06:11:46.163168+0000","last_undegraded":"2023-11-24T06:11:46.163168+0000","last_fullsized":"2023-11-24T06:11:46.163168+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_clean_scrub_stamp":"2023-11-24T06:11:44.018686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:34:37.780831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:11:46.165151+0000","last_change":"2023-11-24T06:11:45.062099+0000","last_active":"2023-11-24T06:11:46.165151+0000","last_peered":"2023-11-24T06:11:46.165151+0000","last_clean":"2023-11-24T06:11:46.165151+0000","last_became_active":"2023-11-24T06:11:45.061742+0000","last_became_peered":"2023-11-24T06:11:45.061742+0000","last_unstale":"2023-11-24T06:11:46.165151+0000","last_undegraded":"2023-11-24T06:11:46.165151+0000","last_fullsized":"2023-11-24T06:11:46.165151+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_clean_scrub_stamp":"2023-11-24T06:11:44.018686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:54:36.987896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:11:46.163642+0000","last_change":"2023-11-24T06:11:45.064668+0000","last_active":"2023-11-24T06:11:46.163642+0000","last_peered":"2023-11-24T06:11:46.163642+0000","last_clean":"2023-11-24T06:11:46.163642+0000","last_became_active":"2023-11-24T06:11:45.064170+0000","last_became_peered":"2023-11-24T06:11:45.064170+0000","last_unstale":"2023-11-24T06:11:46.163642+0000","last_undegraded":"2023-11-24T06:11:46.163642+0000","last_fullsized":"2023-11-24T06:11:46.163642+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_clean_scrub_stamp":"2023-11-24T06:11:44.018686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:26:22.270872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:11:46.163138+0000","last_change":"2023-11-24T06:11:45.066500+0000","last_active":"2023-11-24T06:11:46.163138+0000","last_peered":"2023-11-24T06:11:46.163138+0000","last_clean":"2023-11-24T06:11:46.163138+0000","last_became_active":"2023-11-24T06:11:45.065691+0000","last_became_peered":"2023-11-24T06:11:45.065691+0000","last_unstale":"2023-11-24T06:11:46.163138+0000","last_undegraded":"2023-11-24T06:11:46.163138+0000","last_fullsized":"2023-11-24T06:11:46.163138+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_clean_scrub_stamp":"2023-11-24T06:11:44.018686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:46:05.186588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:11:46.042709+0000","last_change":"2023-11-24T06:11:45.059123+0000","last_active":"2023-11-24T06:11:46.042709+0000","last_peered":"2023-11-24T06:11:46.042709+0000","last_clean":"2023-11-24T06:11:46.042709+0000","last_became_active":"2023-11-24T06:11:45.057405+0000","last_became_peered":"2023-11-24T06:11:45.057405+0000","last_unstale":"2023-11-24T06:11:46.042709+0000","last_undegraded":"2023-11-24T06:11:46.042709+0000","last_fullsized":"2023-11-24T06:11:46.042709+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_clean_scrub_stamp":"2023-11-24T06:11:44.018686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:19:27.730483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:11:46.043997+0000","last_change":"2023-11-24T06:11:45.064510+0000","last_active":"2023-11-24T06:11:46.043997+0000","last_peered":"2023-11-24T06:11:46.043997+0000","last_clean":"2023-11-24T06:11:46.043997+0000","last_became_active":"2023-11-24T06:11:45.063697+0000","last_became_peered":"2023-11-24T06:11:45.063697+0000","last_unstale":"2023-11-24T06:11:46.043997+0000","last_undegraded":"2023-11-24T06:11:46.043997+0000","last_fullsized":"2023-11-24T06:11:46.043997+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_clean_scrub_stamp":"2023-11-24T06:11:44.018686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:50:54.991874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:11:46.043537+0000","last_change":"2023-11-24T06:11:45.059816+0000","last_active":"2023-11-24T06:11:46.043537+0000","last_peered":"2023-11-24T06:11:46.043537+0000","last_clean":"2023-11-24T06:11:46.043537+0000","last_became_active":"2023-11-24T06:11:45.059557+0000","last_became_peered":"2023-11-24T06:11:45.059557+0000","last_unstale":"2023-11-24T06:11:46.043537+0000","last_undegraded":"2023-11-24T06:11:46.043537+0000","last_fullsized":"2023-11-24T06:11:46.043537+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_clean_scrub_stamp":"2023-11-24T06:11:44.018686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:32:48.890199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:11:46.159882+0000","last_change":"2023-11-24T06:11:45.061186+0000","last_active":"2023-11-24T06:11:46.159882+0000","last_peered":"2023-11-24T06:11:46.159882+0000","last_clean":"2023-11-24T06:11:46.159882+0000","last_became_active":"2023-11-24T06:11:45.059551+0000","last_became_peered":"2023-11-24T06:11:45.059551+0000","last_unstale":"2023-11-24T06:11:46.159882+0000","last_undegraded":"2023-11-24T06:11:46.159882+0000","last_fullsized":"2023-11-24T06:11:46.159882+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_clean_scrub_stamp":"2023-11-24T06:11:44.018686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:32:48.890199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:11:46.162187+0000","last_change":"2023-11-24T06:11:45.051196+0000","last_active":"2023-11-24T06:11:46.162187+0000","last_peered":"2023-11-24T06:11:46.162187+0000","last_clean":"2023-11-24T06:11:46.162187+0000","last_became_active":"2023-11-24T06:11:45.050781+0000","last_became_peered":"2023-11-24T06:11:45.050781+0000","last_unstale":"2023-11-24T06:11:46.162187+0000","last_undegraded":"2023-11-24T06:11:46.162187+0000","last_fullsized":"2023-11-24T06:11:46.162187+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_clean_scrub_stamp":"2023-11-24T06:11:44.018686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:55:17.339395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:11:46.163685+0000","last_change":"2023-11-24T06:11:45.062098+0000","last_active":"2023-11-24T06:11:46.163685+0000","last_peered":"2023-11-24T06:11:46.163685+0000","last_clean":"2023-11-24T06:11:46.163685+0000","last_became_active":"2023-11-24T06:11:45.061798+0000","last_became_peered":"2023-11-24T06:11:45.061798+0000","last_unstale":"2023-11-24T06:11:46.163685+0000","last_undegraded":"2023-11-24T06:11:46.163685+0000","last_fullsized":"2023-11-24T06:11:46.163685+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_clean_scrub_stamp":"2023-11-24T06:11:44.018686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:01:16.849532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:11:46.163766+0000","last_change":"2023-11-24T06:11:45.060932+0000","last_active":"2023-11-24T06:11:46.163766+0000","last_peered":"2023-11-24T06:11:46.163766+0000","last_clean":"2023-11-24T06:11:46.163766+0000","last_became_active":"2023-11-24T06:11:45.060427+0000","last_became_peered":"2023-11-24T06:11:45.060427+0000","last_unstale":"2023-11-24T06:11:46.163766+0000","last_undegraded":"2023-11-24T06:11:46.163766+0000","last_fullsized":"2023-11-24T06:11:46.163766+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_clean_scrub_stamp":"2023-11-24T06:11:44.018686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:54:36.987896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:11:46.163174+0000","last_change":"2023-11-24T06:11:45.071131+0000","last_active":"2023-11-24T06:11:46.163174+0000","last_peered":"2023-11-24T06:11:46.163174+0000","last_clean":"2023-11-24T06:11:46.163174+0000","last_became_active":"2023-11-24T06:11:45.070910+0000","last_became_peered":"2023-11-24T06:11:45.070910+0000","last_unstale":"2023-11-24T06:11:46.163174+0000","last_undegraded":"2023-11-24T06:11:46.163174+0000","last_fullsized":"2023-11-24T06:11:46.163174+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_clean_scrub_stamp":"2023-11-24T06:11:44.018686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:42:12.263089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:11:46.163263+0000","last_change":"2023-11-24T06:11:45.063976+0000","last_active":"2023-11-24T06:11:46.163263+0000","last_peered":"2023-11-24T06:11:46.163263+0000","last_clean":"2023-11-24T06:11:46.163263+0000","last_became_active":"2023-11-24T06:11:45.063251+0000","last_became_peered":"2023-11-24T06:11:45.063251+0000","last_unstale":"2023-11-24T06:11:46.163263+0000","last_undegraded":"2023-11-24T06:11:46.163263+0000","last_fullsized":"2023-11-24T06:11:46.163263+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_clean_scrub_stamp":"2023-11-24T06:11:44.018686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:40:45.042808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:11:46.044351+0000","last_change":"2023-11-24T06:11:45.068044+0000","last_active":"2023-11-24T06:11:46.044351+0000","last_peered":"2023-11-24T06:11:46.044351+0000","last_clean":"2023-11-24T06:11:46.044351+0000","last_became_active":"2023-11-24T06:11:45.061965+0000","last_became_peered":"2023-11-24T06:11:45.061965+0000","last_unstale":"2023-11-24T06:11:46.044351+0000","last_undegraded":"2023-11-24T06:11:46.044351+0000","last_fullsized":"2023-11-24T06:11:46.044351+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_clean_scrub_stamp":"2023-11-24T06:11:44.018686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:17:19.814894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:11:46.164000+0000","last_change":"2023-11-24T06:11:45.058528+0000","last_active":"2023-11-24T06:11:46.164000+0000","last_peered":"2023-11-24T06:11:46.164000+0000","last_clean":"2023-11-24T06:11:46.164000+0000","last_became_active":"2023-11-24T06:11:45.057303+0000","last_became_peered":"2023-11-24T06:11:45.057303+0000","last_unstale":"2023-11-24T06:11:46.164000+0000","last_undegraded":"2023-11-24T06:11:46.164000+0000","last_fullsized":"2023-11-24T06:11:46.164000+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_clean_scrub_stamp":"2023-11-24T06:11:44.018686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:46:05.186588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:11:46.043376+0000","last_change":"2023-11-24T06:11:45.052716+0000","last_active":"2023-11-24T06:11:46.043376+0000","last_peered":"2023-11-24T06:11:46.043376+0000","last_clean":"2023-11-24T06:11:46.043376+0000","last_became_active":"2023-11-24T06:11:45.052417+0000","last_became_peered":"2023-11-24T06:11:45.052417+0000","last_unstale":"2023-11-24T06:11:46.043376+0000","last_undegraded":"2023-11-24T06:11:46.043376+0000","last_fullsized":"2023-11-24T06:11:46.043376+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_clean_scrub_stamp":"2023-11-24T06:11:44.018686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:24:27.017978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:11:46.162563+0000","last_change":"2023-11-24T06:11:45.071582+0000","last_active":"2023-11-24T06:11:46.162563+0000","last_peered":"2023-11-24T06:11:46.162563+0000","last_clean":"2023-11-24T06:11:46.162563+0000","last_became_active":"2023-11-24T06:11:45.071034+0000","last_became_peered":"2023-11-24T06:11:45.071034+0000","last_unstale":"2023-11-24T06:11:46.162563+0000","last_undegraded":"2023-11-24T06:11:46.162563+0000","last_fullsized":"2023-11-24T06:11:46.162563+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_clean_scrub_stamp":"2023-11-24T06:11:44.018686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:19:03.083820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:11:46.160046+0000","last_change":"2023-11-24T06:11:45.059450+0000","last_active":"2023-11-24T06:11:46.160046+0000","last_peered":"2023-11-24T06:11:46.160046+0000","last_clean":"2023-11-24T06:11:46.160046+0000","last_became_active":"2023-11-24T06:11:45.058794+0000","last_became_peered":"2023-11-24T06:11:45.058794+0000","last_unstale":"2023-11-24T06:11:46.160046+0000","last_undegraded":"2023-11-24T06:11:46.160046+0000","last_fullsized":"2023-11-24T06:11:46.160046+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_clean_scrub_stamp":"2023-11-24T06:11:44.018686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:13:39.376575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"32'4","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:11:46.044233+0000","last_change":"2023-11-24T06:11:44.064796+0000","last_active":"2023-11-24T06:11:46.044233+0000","last_peered":"2023-11-24T06:11:46.044233+0000","last_clean":"2023-11-24T06:11:46.044233+0000","last_became_active":"2023-11-24T06:11:44.064532+0000","last_became_peered":"2023-11-24T06:11:44.064532+0000","last_unstale":"2023-11-24T06:11:46.044233+0000","last_undegraded":"2023-11-24T06:11:46.044233+0000","last_fullsized":"2023-11-24T06:11:46.044233+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:43.003933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:43.003933+0000","last_clean_scrub_stamp":"2023-11-24T06:11:43.003933+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:10:24.858886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:11:46.044083+0000","last_change":"2023-11-24T06:11:45.062393+0000","last_active":"2023-11-24T06:11:46.044083+0000","last_peered":"2023-11-24T06:11:46.044083+0000","last_clean":"2023-11-24T06:11:46.044083+0000","last_became_active":"2023-11-24T06:11:45.060876+0000","last_became_peered":"2023-11-24T06:11:45.060876+0000","last_unstale":"2023-11-24T06:11:46.044083+0000","last_undegraded":"2023-11-24T06:11:46.044083+0000","last_fullsized":"2023-11-24T06:11:46.044083+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_clean_scrub_stamp":"2023-11-24T06:11:44.018686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:50:54.991874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"32'1","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:11:46.160502+0000","last_change":"2023-11-24T06:11:44.069817+0000","last_active":"2023-11-24T06:11:46.160502+0000","last_peered":"2023-11-24T06:11:46.160502+0000","last_clean":"2023-11-24T06:11:46.160502+0000","last_became_active":"2023-11-24T06:11:44.068458+0000","last_became_peered":"2023-11-24T06:11:44.068458+0000","last_unstale":"2023-11-24T06:11:46.160502+0000","last_undegraded":"2023-11-24T06:11:46.160502+0000","last_fullsized":"2023-11-24T06:11:46.160502+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:43.003933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:43.003933+0000","last_clean_scrub_stamp":"2023-11-24T06:11:43.003933+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:10:24.858886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:11:46.043594+0000","last_change":"2023-11-24T06:11:45.061725+0000","last_active":"2023-11-24T06:11:46.043594+0000","last_peered":"2023-11-24T06:11:46.043594+0000","last_clean":"2023-11-24T06:11:46.043594+0000","last_became_active":"2023-11-24T06:11:45.061352+0000","last_became_peered":"2023-11-24T06:11:45.061352+0000","last_unstale":"2023-11-24T06:11:46.043594+0000","last_undegraded":"2023-11-24T06:11:46.043594+0000","last_fullsized":"2023-11-24T06:11:46.043594+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_clean_scrub_stamp":"2023-11-24T06:11:44.018686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:13:39.376575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"32'2","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:11:46.163402+0000","last_change":"2023-11-24T06:11:44.067226+0000","last_active":"2023-11-24T06:11:46.163402+0000","last_peered":"2023-11-24T06:11:46.163402+0000","last_clean":"2023-11-24T06:11:46.163402+0000","last_became_active":"2023-11-24T06:11:44.066968+0000","last_became_peered":"2023-11-24T06:11:44.066968+0000","last_unstale":"2023-11-24T06:11:46.163402+0000","last_undegraded":"2023-11-24T06:11:46.163402+0000","last_fullsized":"2023-11-24T06:11:46.163402+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:43.003933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:43.003933+0000","last_clean_scrub_stamp":"2023-11-24T06:11:43.003933+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:51:50.015479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:11:46.163325+0000","last_change":"2023-11-24T06:11:45.071343+0000","last_active":"2023-11-24T06:11:46.163325+0000","last_peered":"2023-11-24T06:11:46.163325+0000","last_clean":"2023-11-24T06:11:46.163325+0000","last_became_active":"2023-11-24T06:11:45.071017+0000","last_became_peered":"2023-11-24T06:11:45.071017+0000","last_unstale":"2023-11-24T06:11:46.163325+0000","last_undegraded":"2023-11-24T06:11:46.163325+0000","last_fullsized":"2023-11-24T06:11:46.163325+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_clean_scrub_stamp":"2023-11-24T06:11:44.018686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:44:01.946330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"32'2","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:11:46.042584+0000","last_change":"2023-11-24T06:11:44.060544+0000","last_active":"2023-11-24T06:11:46.042584+0000","last_peered":"2023-11-24T06:11:46.042584+0000","last_clean":"2023-11-24T06:11:46.042584+0000","last_became_active":"2023-11-24T06:11:44.060231+0000","last_became_peered":"2023-11-24T06:11:44.060231+0000","last_unstale":"2023-11-24T06:11:46.042584+0000","last_undegraded":"2023-11-24T06:11:46.042584+0000","last_fullsized":"2023-11-24T06:11:46.042584+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:43.003933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:43.003933+0000","last_clean_scrub_stamp":"2023-11-24T06:11:43.003933+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:31:27.908252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:11:46.164305+0000","last_change":"2023-11-24T06:11:45.066633+0000","last_active":"2023-11-24T06:11:46.164305+0000","last_peered":"2023-11-24T06:11:46.164305+0000","last_clean":"2023-11-24T06:11:46.164305+0000","last_became_active":"2023-11-24T06:11:45.066306+0000","last_became_peered":"2023-11-24T06:11:45.066306+0000","last_unstale":"2023-11-24T06:11:46.164305+0000","last_undegraded":"2023-11-24T06:11:46.164305+0000","last_fullsized":"2023-11-24T06:11:46.164305+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_clean_scrub_stamp":"2023-11-24T06:11:44.018686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:16:44.582224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"32'1","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:11:46.044662+0000","last_change":"2023-11-24T06:11:44.067069+0000","last_active":"2023-11-24T06:11:46.044662+0000","last_peered":"2023-11-24T06:11:46.044662+0000","last_clean":"2023-11-24T06:11:46.044662+0000","last_became_active":"2023-11-24T06:11:44.066806+0000","last_became_peered":"2023-11-24T06:11:44.066806+0000","last_unstale":"2023-11-24T06:11:46.044662+0000","last_undegraded":"2023-11-24T06:11:46.044662+0000","last_fullsized":"2023-11-24T06:11:46.044662+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:43.003933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:43.003933+0000","last_clean_scrub_stamp":"2023-11-24T06:11:43.003933+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:18:36.729170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:11:46.162590+0000","last_change":"2023-11-24T06:11:45.065318+0000","last_active":"2023-11-24T06:11:46.162590+0000","last_peered":"2023-11-24T06:11:46.162590+0000","last_clean":"2023-11-24T06:11:46.162590+0000","last_became_active":"2023-11-24T06:11:45.064999+0000","last_became_peered":"2023-11-24T06:11:45.064999+0000","last_unstale":"2023-11-24T06:11:46.162590+0000","last_undegraded":"2023-11-24T06:11:46.162590+0000","last_fullsized":"2023-11-24T06:11:46.162590+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_clean_scrub_stamp":"2023-11-24T06:11:44.018686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:15:02.243714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"32'1","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:11:46.043768+0000","last_change":"2023-11-24T06:11:44.067797+0000","last_active":"2023-11-24T06:11:46.043768+0000","last_peered":"2023-11-24T06:11:46.043768+0000","last_clean":"2023-11-24T06:11:46.043768+0000","last_became_active":"2023-11-24T06:11:44.067418+0000","last_became_peered":"2023-11-24T06:11:44.067418+0000","last_unstale":"2023-11-24T06:11:46.043768+0000","last_undegraded":"2023-11-24T06:11:46.043768+0000","last_fullsized":"2023-11-24T06:11:46.043768+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:43.003933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:43.003933+0000","last_clean_scrub_stamp":"2023-11-24T06:11:43.003933+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:52:00.088217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:11:46.042963+0000","last_change":"2023-11-24T06:11:45.068363+0000","last_active":"2023-11-24T06:11:46.042963+0000","last_peered":"2023-11-24T06:11:46.042963+0000","last_clean":"2023-11-24T06:11:46.042963+0000","last_became_active":"2023-11-24T06:11:45.058507+0000","last_became_peered":"2023-11-24T06:11:45.058507+0000","last_unstale":"2023-11-24T06:11:46.042963+0000","last_undegraded":"2023-11-24T06:11:46.042963+0000","last_fullsized":"2023-11-24T06:11:46.042963+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_clean_scrub_stamp":"2023-11-24T06:11:44.018686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:56:05.022334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:11:46.163969+0000","last_change":"2023-11-24T06:11:44.067156+0000","last_active":"2023-11-24T06:11:46.163969+0000","last_peered":"2023-11-24T06:11:46.163969+0000","last_clean":"2023-11-24T06:11:46.163969+0000","last_became_active":"2023-11-24T06:11:44.066595+0000","last_became_peered":"2023-11-24T06:11:44.066595+0000","last_unstale":"2023-11-24T06:11:46.163969+0000","last_undegraded":"2023-11-24T06:11:46.163969+0000","last_fullsized":"2023-11-24T06:11:46.163969+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:43.003933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:43.003933+0000","last_clean_scrub_stamp":"2023-11-24T06:11:43.003933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:44:00.931577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:11:46.164811+0000","last_change":"2023-11-24T06:11:45.059416+0000","last_active":"2023-11-24T06:11:46.164811+0000","last_peered":"2023-11-24T06:11:46.164811+0000","last_clean":"2023-11-24T06:11:46.164811+0000","last_became_active":"2023-11-24T06:11:45.059083+0000","last_became_peered":"2023-11-24T06:11:45.059083+0000","last_unstale":"2023-11-24T06:11:46.164811+0000","last_undegraded":"2023-11-24T06:11:46.164811+0000","last_fullsized":"2023-11-24T06:11:46.164811+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_clean_scrub_stamp":"2023-11-24T06:11:44.018686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:17:19.814894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:11:46.043172+0000","last_change":"2023-11-24T06:11:44.068295+0000","last_active":"2023-11-24T06:11:46.043172+0000","last_peered":"2023-11-24T06:11:46.043172+0000","last_clean":"2023-11-24T06:11:46.043172+0000","last_became_active":"2023-11-24T06:11:44.066015+0000","last_became_peered":"2023-11-24T06:11:44.066015+0000","last_unstale":"2023-11-24T06:11:46.043172+0000","last_undegraded":"2023-11-24T06:11:46.043172+0000","last_fullsized":"2023-11-24T06:11:46.043172+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:43.003933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:43.003933+0000","last_clean_scrub_stamp":"2023-11-24T06:11:43.003933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:19:26.715729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:11:46.164371+0000","last_change":"2023-11-24T06:11:45.065842+0000","last_active":"2023-11-24T06:11:46.164371+0000","last_peered":"2023-11-24T06:11:46.164371+0000","last_clean":"2023-11-24T06:11:46.164371+0000","last_became_active":"2023-11-24T06:11:45.065524+0000","last_became_peered":"2023-11-24T06:11:45.065524+0000","last_unstale":"2023-11-24T06:11:46.164371+0000","last_undegraded":"2023-11-24T06:11:46.164371+0000","last_fullsized":"2023-11-24T06:11:46.164371+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_clean_scrub_stamp":"2023-11-24T06:11:44.018686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:34:37.780831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:11:46.164450+0000","last_change":"2023-11-24T06:11:44.066427+0000","last_active":"2023-11-24T06:11:46.164450+0000","last_peered":"2023-11-24T06:11:46.164450+0000","last_clean":"2023-11-24T06:11:46.164450+0000","last_became_active":"2023-11-24T06:11:44.066107+0000","last_became_peered":"2023-11-24T06:11:44.066107+0000","last_unstale":"2023-11-24T06:11:46.164450+0000","last_undegraded":"2023-11-24T06:11:46.164450+0000","last_fullsized":"2023-11-24T06:11:46.164450+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:43.003933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:43.003933+0000","last_clean_scrub_stamp":"2023-11-24T06:11:43.003933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:15:01.228961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:11:46.164747+0000","last_change":"2023-11-24T06:11:45.069665+0000","last_active":"2023-11-24T06:11:46.164747+0000","last_peered":"2023-11-24T06:11:46.164747+0000","last_clean":"2023-11-24T06:11:46.164747+0000","last_became_active":"2023-11-24T06:11:45.069330+0000","last_became_peered":"2023-11-24T06:11:45.069330+0000","last_unstale":"2023-11-24T06:11:46.164747+0000","last_undegraded":"2023-11-24T06:11:46.164747+0000","last_fullsized":"2023-11-24T06:11:46.164747+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_clean_scrub_stamp":"2023-11-24T06:11:44.018686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:57:54.566169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:11:46.160547+0000","last_change":"2023-11-24T06:11:44.058412+0000","last_active":"2023-11-24T06:11:46.160547+0000","last_peered":"2023-11-24T06:11:46.160547+0000","last_clean":"2023-11-24T06:11:46.160547+0000","last_became_active":"2023-11-24T06:11:44.058065+0000","last_became_peered":"2023-11-24T06:11:44.058065+0000","last_unstale":"2023-11-24T06:11:46.160547+0000","last_undegraded":"2023-11-24T06:11:46.160547+0000","last_fullsized":"2023-11-24T06:11:46.160547+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:43.003933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:43.003933+0000","last_clean_scrub_stamp":"2023-11-24T06:11:43.003933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:02:25.402413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:11:46.163091+0000","last_change":"2023-11-24T06:11:45.058531+0000","last_active":"2023-11-24T06:11:46.163091+0000","last_peered":"2023-11-24T06:11:46.163091+0000","last_clean":"2023-11-24T06:11:46.163091+0000","last_became_active":"2023-11-24T06:11:45.058201+0000","last_became_peered":"2023-11-24T06:11:45.058201+0000","last_unstale":"2023-11-24T06:11:46.163091+0000","last_undegraded":"2023-11-24T06:11:46.163091+0000","last_fullsized":"2023-11-24T06:11:46.163091+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_clean_scrub_stamp":"2023-11-24T06:11:44.018686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:09:00.870428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"32'3","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:11:46.163810+0000","last_change":"2023-11-24T06:11:44.066151+0000","last_active":"2023-11-24T06:11:46.163810+0000","last_peered":"2023-11-24T06:11:46.163810+0000","last_clean":"2023-11-24T06:11:46.163810+0000","last_became_active":"2023-11-24T06:11:44.065797+0000","last_became_peered":"2023-11-24T06:11:44.065797+0000","last_unstale":"2023-11-24T06:11:46.163810+0000","last_undegraded":"2023-11-24T06:11:46.163810+0000","last_fullsized":"2023-11-24T06:11:46.163810+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:43.003933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:43.003933+0000","last_clean_scrub_stamp":"2023-11-24T06:11:43.003933+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:44:00.931577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:11:46.164528+0000","last_change":"2023-11-24T06:11:45.067459+0000","last_active":"2023-11-24T06:11:46.164528+0000","last_peered":"2023-11-24T06:11:46.164528+0000","last_clean":"2023-11-24T06:11:46.164528+0000","last_became_active":"2023-11-24T06:11:45.066804+0000","last_became_peered":"2023-11-24T06:11:45.066804+0000","last_unstale":"2023-11-24T06:11:46.164528+0000","last_undegraded":"2023-11-24T06:11:46.164528+0000","last_fullsized":"2023-11-24T06:11:46.164528+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_clean_scrub_stamp":"2023-11-24T06:11:44.018686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:09:00.870428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"32'6","reported_seq":18,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:11:46.163186+0000","last_change":"2023-11-24T06:11:44.068681+0000","last_active":"2023-11-24T06:11:46.163186+0000","last_peered":"2023-11-24T06:11:46.163186+0000","last_clean":"2023-11-24T06:11:46.163186+0000","last_became_active":"2023-11-24T06:11:44.068403+0000","last_became_peered":"2023-11-24T06:11:44.068403+0000","last_unstale":"2023-11-24T06:11:46.163186+0000","last_undegraded":"2023-11-24T06:11:46.163186+0000","last_fullsized":"2023-11-24T06:11:46.163186+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:43.003933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:43.003933+0000","last_clean_scrub_stamp":"2023-11-24T06:11:43.003933+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:19:02.069067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:11:46.164901+0000","last_change":"2023-11-24T06:11:45.066579+0000","last_active":"2023-11-24T06:11:46.164901+0000","last_peered":"2023-11-24T06:11:46.164901+0000","last_clean":"2023-11-24T06:11:46.164901+0000","last_became_active":"2023-11-24T06:11:45.066011+0000","last_became_peered":"2023-11-24T06:11:45.066011+0000","last_unstale":"2023-11-24T06:11:46.164901+0000","last_undegraded":"2023-11-24T06:11:46.164901+0000","last_fullsized":"2023-11-24T06:11:46.164901+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_clean_scrub_stamp":"2023-11-24T06:11:44.018686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:19:03.083820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"32'2","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:11:46.044719+0000","last_change":"2023-11-24T06:11:44.065844+0000","last_active":"2023-11-24T06:11:46.044719+0000","last_peered":"2023-11-24T06:11:46.044719+0000","last_clean":"2023-11-24T06:11:46.044719+0000","last_became_active":"2023-11-24T06:11:44.065561+0000","last_became_peered":"2023-11-24T06:11:44.065561+0000","last_unstale":"2023-11-24T06:11:46.044719+0000","last_undegraded":"2023-11-24T06:11:46.044719+0000","last_fullsized":"2023-11-24T06:11:46.044719+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:43.003933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:43.003933+0000","last_clean_scrub_stamp":"2023-11-24T06:11:43.003933+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:19:53.395242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:11:46.163370+0000","last_change":"2023-11-24T06:11:45.060959+0000","last_active":"2023-11-24T06:11:46.163370+0000","last_peered":"2023-11-24T06:11:46.163370+0000","last_clean":"2023-11-24T06:11:46.163370+0000","last_became_active":"2023-11-24T06:11:45.059011+0000","last_became_peered":"2023-11-24T06:11:45.059011+0000","last_unstale":"2023-11-24T06:11:46.163370+0000","last_undegraded":"2023-11-24T06:11:46.163370+0000","last_fullsized":"2023-11-24T06:11:46.163370+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_clean_scrub_stamp":"2023-11-24T06:11:44.018686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:29:49.006871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:11:46.164048+0000","last_change":"2023-11-24T06:11:44.064612+0000","last_active":"2023-11-24T06:11:46.164048+0000","last_peered":"2023-11-24T06:11:46.164048+0000","last_clean":"2023-11-24T06:11:46.164048+0000","last_became_active":"2023-11-24T06:11:44.064309+0000","last_became_peered":"2023-11-24T06:11:44.064309+0000","last_unstale":"2023-11-24T06:11:46.164048+0000","last_undegraded":"2023-11-24T06:11:46.164048+0000","last_fullsized":"2023-11-24T06:11:46.164048+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:43.003933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:43.003933+0000","last_clean_scrub_stamp":"2023-11-24T06:11:43.003933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:52:24.803556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:11:46.162799+0000","last_change":"2023-11-24T06:11:45.071836+0000","last_active":"2023-11-24T06:11:46.162799+0000","last_peered":"2023-11-24T06:11:46.162799+0000","last_clean":"2023-11-24T06:11:46.162799+0000","last_became_active":"2023-11-24T06:11:45.071012+0000","last_became_peered":"2023-11-24T06:11:45.071012+0000","last_unstale":"2023-11-24T06:11:46.162799+0000","last_undegraded":"2023-11-24T06:11:46.162799+0000","last_fullsized":"2023-11-24T06:11:46.162799+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_clean_scrub_stamp":"2023-11-24T06:11:44.018686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:09:22.132463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:11:46.163170+0000","last_change":"2023-11-24T06:11:44.067439+0000","last_active":"2023-11-24T06:11:46.163170+0000","last_peered":"2023-11-24T06:11:46.163170+0000","last_clean":"2023-11-24T06:11:46.163170+0000","last_became_active":"2023-11-24T06:11:44.065500+0000","last_became_peered":"2023-11-24T06:11:44.065500+0000","last_unstale":"2023-11-24T06:11:46.163170+0000","last_undegraded":"2023-11-24T06:11:46.163170+0000","last_fullsized":"2023-11-24T06:11:46.163170+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:43.003933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:43.003933+0000","last_clean_scrub_stamp":"2023-11-24T06:11:43.003933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:01:15.834779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:11:46.043578+0000","last_change":"2023-11-24T06:11:45.069211+0000","last_active":"2023-11-24T06:11:46.043578+0000","last_peered":"2023-11-24T06:11:46.043578+0000","last_clean":"2023-11-24T06:11:46.043578+0000","last_became_active":"2023-11-24T06:11:45.068912+0000","last_became_peered":"2023-11-24T06:11:45.068912+0000","last_unstale":"2023-11-24T06:11:46.043578+0000","last_undegraded":"2023-11-24T06:11:46.043578+0000","last_fullsized":"2023-11-24T06:11:46.043578+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_clean_scrub_stamp":"2023-11-24T06:11:44.018686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:56:05.022334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"32'8","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:11:46.162044+0000","last_change":"2023-11-24T06:11:44.069838+0000","last_active":"2023-11-24T06:11:46.162044+0000","last_peered":"2023-11-24T06:11:46.162044+0000","last_clean":"2023-11-24T06:11:46.162044+0000","last_became_active":"2023-11-24T06:11:44.069111+0000","last_became_peered":"2023-11-24T06:11:44.069111+0000","last_unstale":"2023-11-24T06:11:46.162044+0000","last_undegraded":"2023-11-24T06:11:46.162044+0000","last_fullsized":"2023-11-24T06:11:46.162044+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:43.003933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:43.003933+0000","last_clean_scrub_stamp":"2023-11-24T06:11:43.003933+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:31:15.184560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:11:46.044156+0000","last_change":"2023-11-24T06:11:45.064079+0000","last_active":"2023-11-24T06:11:46.044156+0000","last_peered":"2023-11-24T06:11:46.044156+0000","last_clean":"2023-11-24T06:11:46.044156+0000","last_became_active":"2023-11-24T06:11:45.063771+0000","last_became_peered":"2023-11-24T06:11:45.063771+0000","last_unstale":"2023-11-24T06:11:46.044156+0000","last_undegraded":"2023-11-24T06:11:46.044156+0000","last_fullsized":"2023-11-24T06:11:46.044156+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_clean_scrub_stamp":"2023-11-24T06:11:44.018686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:44:42.836131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"32'1","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:11:46.163815+0000","last_change":"2023-11-24T06:11:44.071090+0000","last_active":"2023-11-24T06:11:46.163815+0000","last_peered":"2023-11-24T06:11:46.163815+0000","last_clean":"2023-11-24T06:11:46.163815+0000","last_became_active":"2023-11-24T06:11:44.070760+0000","last_became_peered":"2023-11-24T06:11:44.070760+0000","last_unstale":"2023-11-24T06:11:46.163815+0000","last_undegraded":"2023-11-24T06:11:46.163815+0000","last_fullsized":"2023-11-24T06:11:46.163815+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:43.003933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:43.003933+0000","last_clean_scrub_stamp":"2023-11-24T06:11:43.003933+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:13:38.361822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:11:46.164890+0000","last_change":"2023-11-24T06:11:45.054126+0000","last_active":"2023-11-24T06:11:46.164890+0000","last_peered":"2023-11-24T06:11:46.164890+0000","last_clean":"2023-11-24T06:11:46.164890+0000","last_became_active":"2023-11-24T06:11:45.053804+0000","last_became_peered":"2023-11-24T06:11:45.053804+0000","last_unstale":"2023-11-24T06:11:46.164890+0000","last_undegraded":"2023-11-24T06:11:46.164890+0000","last_fullsized":"2023-11-24T06:11:46.164890+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_clean_scrub_stamp":"2023-11-24T06:11:44.018686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:46:05.186588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"32'2","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:11:46.044379+0000","last_change":"2023-11-24T06:11:44.068028+0000","last_active":"2023-11-24T06:11:46.044379+0000","last_peered":"2023-11-24T06:11:46.044379+0000","last_clean":"2023-11-24T06:11:46.044379+0000","last_became_active":"2023-11-24T06:11:44.067730+0000","last_became_peered":"2023-11-24T06:11:44.067730+0000","last_unstale":"2023-11-24T06:11:46.044379+0000","last_undegraded":"2023-11-24T06:11:46.044379+0000","last_fullsized":"2023-11-24T06:11:46.044379+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:43.003933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:43.003933+0000","last_clean_scrub_stamp":"2023-11-24T06:11:43.003933+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:09:08.543385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:11:46.164308+0000","last_change":"2023-11-24T06:11:45.070909+0000","last_active":"2023-11-24T06:11:46.164308+0000","last_peered":"2023-11-24T06:11:46.164308+0000","last_clean":"2023-11-24T06:11:46.164308+0000","last_became_active":"2023-11-24T06:11:45.070566+0000","last_became_peered":"2023-11-24T06:11:45.070566+0000","last_unstale":"2023-11-24T06:11:46.164308+0000","last_undegraded":"2023-11-24T06:11:46.164308+0000","last_fullsized":"2023-11-24T06:11:46.164308+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_clean_scrub_stamp":"2023-11-24T06:11:44.018686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:51:51.030233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"32'1","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:11:46.044581+0000","last_change":"2023-11-24T06:11:44.058740+0000","last_active":"2023-11-24T06:11:46.044581+0000","last_peered":"2023-11-24T06:11:46.044581+0000","last_clean":"2023-11-24T06:11:46.044581+0000","last_became_active":"2023-11-24T06:11:44.058415+0000","last_became_peered":"2023-11-24T06:11:44.058415+0000","last_unstale":"2023-11-24T06:11:46.044581+0000","last_undegraded":"2023-11-24T06:11:46.044581+0000","last_fullsized":"2023-11-24T06:11:46.044581+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:43.003933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:43.003933+0000","last_clean_scrub_stamp":"2023-11-24T06:11:43.003933+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:52:24.803556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:11:46.042883+0000","last_change":"2023-11-24T06:11:45.060673+0000","last_active":"2023-11-24T06:11:46.042883+0000","last_peered":"2023-11-24T06:11:46.042883+0000","last_clean":"2023-11-24T06:11:46.042883+0000","last_became_active":"2023-11-24T06:11:45.060391+0000","last_became_peered":"2023-11-24T06:11:45.060391+0000","last_unstale":"2023-11-24T06:11:46.042883+0000","last_undegraded":"2023-11-24T06:11:46.042883+0000","last_fullsized":"2023-11-24T06:11:46.042883+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_clean_scrub_stamp":"2023-11-24T06:11:44.018686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:36:31.999711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:11:46.043790+0000","last_change":"2023-11-24T06:11:44.068577+0000","last_active":"2023-11-24T06:11:46.043790+0000","last_peered":"2023-11-24T06:11:46.043790+0000","last_clean":"2023-11-24T06:11:46.043790+0000","last_became_active":"2023-11-24T06:11:44.067760+0000","last_became_peered":"2023-11-24T06:11:44.067760+0000","last_unstale":"2023-11-24T06:11:46.043790+0000","last_undegraded":"2023-11-24T06:11:46.043790+0000","last_fullsized":"2023-11-24T06:11:46.043790+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:43.003933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:43.003933+0000","last_clean_scrub_stamp":"2023-11-24T06:11:43.003933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:10:24.858886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:11:46.042046+0000","last_change":"2023-11-24T06:11:45.063538+0000","last_active":"2023-11-24T06:11:46.042046+0000","last_peered":"2023-11-24T06:11:46.042046+0000","last_clean":"2023-11-24T06:11:46.042046+0000","last_became_active":"2023-11-24T06:11:45.063262+0000","last_became_peered":"2023-11-24T06:11:45.063262+0000","last_unstale":"2023-11-24T06:11:46.042046+0000","last_undegraded":"2023-11-24T06:11:46.042046+0000","last_fullsized":"2023-11-24T06:11:46.042046+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_clean_scrub_stamp":"2023-11-24T06:11:44.018686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:31:19.555803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"32'2","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:11:46.160245+0000","last_change":"2023-11-24T06:11:44.062187+0000","last_active":"2023-11-24T06:11:46.160245+0000","last_peered":"2023-11-24T06:11:46.160245+0000","last_clean":"2023-11-24T06:11:46.160245+0000","last_became_active":"2023-11-24T06:11:44.061478+0000","last_became_peered":"2023-11-24T06:11:44.061478+0000","last_unstale":"2023-11-24T06:11:46.160245+0000","last_undegraded":"2023-11-24T06:11:46.160245+0000","last_fullsized":"2023-11-24T06:11:46.160245+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:43.003933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:43.003933+0000","last_clean_scrub_stamp":"2023-11-24T06:11:43.003933+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:24:26.003225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:11:46.042211+0000","last_change":"2023-11-24T06:11:45.065311+0000","last_active":"2023-11-24T06:11:46.042211+0000","last_peered":"2023-11-24T06:11:46.042211+0000","last_clean":"2023-11-24T06:11:46.042211+0000","last_became_active":"2023-11-24T06:11:45.064227+0000","last_became_peered":"2023-11-24T06:11:45.064227+0000","last_unstale":"2023-11-24T06:11:46.042211+0000","last_undegraded":"2023-11-24T06:11:46.042211+0000","last_fullsized":"2023-11-24T06:11:46.042211+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_clean_scrub_stamp":"2023-11-24T06:11:44.018686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:17:04.162486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:11:46.043816+0000","last_change":"2023-11-24T06:11:44.063640+0000","last_active":"2023-11-24T06:11:46.043816+0000","last_peered":"2023-11-24T06:11:46.043816+0000","last_clean":"2023-11-24T06:11:46.043816+0000","last_became_active":"2023-11-24T06:11:44.063346+0000","last_became_peered":"2023-11-24T06:11:44.063346+0000","last_unstale":"2023-11-24T06:11:46.043816+0000","last_undegraded":"2023-11-24T06:11:46.043816+0000","last_fullsized":"2023-11-24T06:11:46.043816+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:43.003933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:43.003933+0000","last_clean_scrub_stamp":"2023-11-24T06:11:43.003933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:09:08.543385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:11:46.041827+0000","last_change":"2023-11-24T06:11:45.061275+0000","last_active":"2023-11-24T06:11:46.041827+0000","last_peered":"2023-11-24T06:11:46.041827+0000","last_clean":"2023-11-24T06:11:46.041827+0000","last_became_active":"2023-11-24T06:11:45.060990+0000","last_became_peered":"2023-11-24T06:11:45.060990+0000","last_unstale":"2023-11-24T06:11:46.041827+0000","last_undegraded":"2023-11-24T06:11:46.041827+0000","last_fullsized":"2023-11-24T06:11:46.041827+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_clean_scrub_stamp":"2023-11-24T06:11:44.018686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:33:25.977675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:11:46.042860+0000","last_change":"2023-11-24T06:11:44.062996+0000","last_active":"2023-11-24T06:11:46.042860+0000","last_peered":"2023-11-24T06:11:46.042860+0000","last_clean":"2023-11-24T06:11:46.042860+0000","last_became_active":"2023-11-24T06:11:44.062606+0000","last_became_peered":"2023-11-24T06:11:44.062606+0000","last_unstale":"2023-11-24T06:11:46.042860+0000","last_undegraded":"2023-11-24T06:11:46.042860+0000","last_fullsized":"2023-11-24T06:11:46.042860+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:43.003933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:43.003933+0000","last_clean_scrub_stamp":"2023-11-24T06:11:43.003933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:36:30.984958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:11:46.163074+0000","last_change":"2023-11-24T06:11:45.060500+0000","last_active":"2023-11-24T06:11:46.163074+0000","last_peered":"2023-11-24T06:11:46.163074+0000","last_clean":"2023-11-24T06:11:46.163074+0000","last_became_active":"2023-11-24T06:11:45.060206+0000","last_became_peered":"2023-11-24T06:11:45.060206+0000","last_unstale":"2023-11-24T06:11:46.163074+0000","last_undegraded":"2023-11-24T06:11:46.163074+0000","last_fullsized":"2023-11-24T06:11:46.163074+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_clean_scrub_stamp":"2023-11-24T06:11:44.018686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:40:45.042808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:11:46.162035+0000","last_change":"2023-11-24T06:11:44.066907+0000","last_active":"2023-11-24T06:11:46.162035+0000","last_peered":"2023-11-24T06:11:46.162035+0000","last_clean":"2023-11-24T06:11:46.162035+0000","last_became_active":"2023-11-24T06:11:44.066171+0000","last_became_peered":"2023-11-24T06:11:44.066171+0000","last_unstale":"2023-11-24T06:11:46.162035+0000","last_undegraded":"2023-11-24T06:11:46.162035+0000","last_fullsized":"2023-11-24T06:11:46.162035+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:43.003933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:43.003933+0000","last_clean_scrub_stamp":"2023-11-24T06:11:43.003933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:02:25.402413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:11:46.042829+0000","last_change":"2023-11-24T06:11:45.058858+0000","last_active":"2023-11-24T06:11:46.042829+0000","last_peered":"2023-11-24T06:11:46.042829+0000","last_clean":"2023-11-24T06:11:46.042829+0000","last_became_active":"2023-11-24T06:11:45.057504+0000","last_became_peered":"2023-11-24T06:11:45.057504+0000","last_unstale":"2023-11-24T06:11:46.042829+0000","last_undegraded":"2023-11-24T06:11:46.042829+0000","last_fullsized":"2023-11-24T06:11:46.042829+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_clean_scrub_stamp":"2023-11-24T06:11:44.018686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:52:01.102970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"32'2","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:11:46.163909+0000","last_change":"2023-11-24T06:11:44.068959+0000","last_active":"2023-11-24T06:11:46.163909+0000","last_peered":"2023-11-24T06:11:46.163909+0000","last_clean":"2023-11-24T06:11:46.163909+0000","last_became_active":"2023-11-24T06:11:44.068639+0000","last_became_peered":"2023-11-24T06:11:44.068639+0000","last_unstale":"2023-11-24T06:11:46.163909+0000","last_undegraded":"2023-11-24T06:11:46.163909+0000","last_fullsized":"2023-11-24T06:11:46.163909+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:43.003933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:43.003933+0000","last_clean_scrub_stamp":"2023-11-24T06:11:43.003933+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:50:53.977121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:11:46.164938+0000","last_change":"2023-11-24T06:11:45.067368+0000","last_active":"2023-11-24T06:11:46.164938+0000","last_peered":"2023-11-24T06:11:46.164938+0000","last_clean":"2023-11-24T06:11:46.164938+0000","last_became_active":"2023-11-24T06:11:45.067027+0000","last_became_peered":"2023-11-24T06:11:45.067027+0000","last_unstale":"2023-11-24T06:11:46.164938+0000","last_undegraded":"2023-11-24T06:11:46.164938+0000","last_fullsized":"2023-11-24T06:11:46.164938+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_clean_scrub_stamp":"2023-11-24T06:11:44.018686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:29:49.006871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:11:46.160498+0000","last_change":"2023-11-24T06:11:44.060007+0000","last_active":"2023-11-24T06:11:46.160498+0000","last_peered":"2023-11-24T06:11:46.160498+0000","last_clean":"2023-11-24T06:11:46.160498+0000","last_became_active":"2023-11-24T06:11:44.059721+0000","last_became_peered":"2023-11-24T06:11:44.059721+0000","last_unstale":"2023-11-24T06:11:46.160498+0000","last_undegraded":"2023-11-24T06:11:46.160498+0000","last_fullsized":"2023-11-24T06:11:46.160498+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:43.003933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:43.003933+0000","last_clean_scrub_stamp":"2023-11-24T06:11:43.003933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:52:24.803556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:11:46.160164+0000","last_change":"2023-11-24T06:11:45.066908+0000","last_active":"2023-11-24T06:11:46.160164+0000","last_peered":"2023-11-24T06:11:46.160164+0000","last_clean":"2023-11-24T06:11:46.160164+0000","last_became_active":"2023-11-24T06:11:45.066554+0000","last_became_peered":"2023-11-24T06:11:45.066554+0000","last_unstale":"2023-11-24T06:11:46.160164+0000","last_undegraded":"2023-11-24T06:11:46.160164+0000","last_fullsized":"2023-11-24T06:11:46.160164+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_clean_scrub_stamp":"2023-11-24T06:11:44.018686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:50:54.991874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:11:46.164459+0000","last_change":"2023-11-24T06:11:44.063030+0000","last_active":"2023-11-24T06:11:46.164459+0000","last_peered":"2023-11-24T06:11:46.164459+0000","last_clean":"2023-11-24T06:11:46.164459+0000","last_became_active":"2023-11-24T06:11:44.061414+0000","last_became_peered":"2023-11-24T06:11:44.061414+0000","last_unstale":"2023-11-24T06:11:46.164459+0000","last_undegraded":"2023-11-24T06:11:46.164459+0000","last_fullsized":"2023-11-24T06:11:46.164459+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:43.003933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:43.003933+0000","last_clean_scrub_stamp":"2023-11-24T06:11:43.003933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:32:47.875446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:11:46.043578+0000","last_change":"2023-11-24T06:11:45.058989+0000","last_active":"2023-11-24T06:11:46.043578+0000","last_peered":"2023-11-24T06:11:46.043578+0000","last_clean":"2023-11-24T06:11:46.043578+0000","last_became_active":"2023-11-24T06:11:45.058674+0000","last_became_peered":"2023-11-24T06:11:45.058674+0000","last_unstale":"2023-11-24T06:11:46.043578+0000","last_undegraded":"2023-11-24T06:11:46.043578+0000","last_fullsized":"2023-11-24T06:11:46.043578+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_clean_scrub_stamp":"2023-11-24T06:11:44.018686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:32:48.890199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:11:46.160346+0000","last_change":"2023-11-24T06:11:44.062364+0000","last_active":"2023-11-24T06:11:46.160346+0000","last_peered":"2023-11-24T06:11:46.160346+0000","last_clean":"2023-11-24T06:11:46.160346+0000","last_became_active":"2023-11-24T06:11:44.061926+0000","last_became_peered":"2023-11-24T06:11:44.061926+0000","last_unstale":"2023-11-24T06:11:46.160346+0000","last_undegraded":"2023-11-24T06:11:46.160346+0000","last_fullsized":"2023-11-24T06:11:46.160346+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:43.003933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:43.003933+0000","last_clean_scrub_stamp":"2023-11-24T06:11:43.003933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:56:04.007581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:11:46.160149+0000","last_change":"2023-11-24T06:11:45.060034+0000","last_active":"2023-11-24T06:11:46.160149+0000","last_peered":"2023-11-24T06:11:46.160149+0000","last_clean":"2023-11-24T06:11:46.160149+0000","last_became_active":"2023-11-24T06:11:45.058485+0000","last_became_peered":"2023-11-24T06:11:45.058485+0000","last_unstale":"2023-11-24T06:11:46.160149+0000","last_undegraded":"2023-11-24T06:11:46.160149+0000","last_fullsized":"2023-11-24T06:11:46.160149+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_clean_scrub_stamp":"2023-11-24T06:11:44.018686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:44:42.836131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:11:46.164282+0000","last_change":"2023-11-24T06:11:44.070989+0000","last_active":"2023-11-24T06:11:46.164282+0000","last_peered":"2023-11-24T06:11:46.164282+0000","last_clean":"2023-11-24T06:11:46.164282+0000","last_became_active":"2023-11-24T06:11:44.070701+0000","last_became_peered":"2023-11-24T06:11:44.070701+0000","last_unstale":"2023-11-24T06:11:46.164282+0000","last_undegraded":"2023-11-24T06:11:46.164282+0000","last_fullsized":"2023-11-24T06:11:46.164282+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:43.003933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:43.003933+0000","last_clean_scrub_stamp":"2023-11-24T06:11:43.003933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:10:24.858886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:11:46.162057+0000","last_change":"2023-11-24T06:11:45.062075+0000","last_active":"2023-11-24T06:11:46.162057+0000","last_peered":"2023-11-24T06:11:46.162057+0000","last_clean":"2023-11-24T06:11:46.162057+0000","last_became_active":"2023-11-24T06:11:45.061638+0000","last_became_peered":"2023-11-24T06:11:45.061638+0000","last_unstale":"2023-11-24T06:11:46.162057+0000","last_undegraded":"2023-11-24T06:11:46.162057+0000","last_fullsized":"2023-11-24T06:11:46.162057+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_clean_scrub_stamp":"2023-11-24T06:11:44.018686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:09:22.132463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:11:46.043119+0000","last_change":"2023-11-24T06:11:44.063839+0000","last_active":"2023-11-24T06:11:46.043119+0000","last_peered":"2023-11-24T06:11:46.043119+0000","last_clean":"2023-11-24T06:11:46.043119+0000","last_became_active":"2023-11-24T06:11:44.063573+0000","last_became_peered":"2023-11-24T06:11:44.063573+0000","last_unstale":"2023-11-24T06:11:46.043119+0000","last_undegraded":"2023-11-24T06:11:46.043119+0000","last_fullsized":"2023-11-24T06:11:46.043119+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:43.003933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:43.003933+0000","last_clean_scrub_stamp":"2023-11-24T06:11:43.003933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:52:52.987858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:11:46.042867+0000","last_change":"2023-11-24T06:11:45.065154+0000","last_active":"2023-11-24T06:11:46.042867+0000","last_peered":"2023-11-24T06:11:46.042867+0000","last_clean":"2023-11-24T06:11:46.042867+0000","last_became_active":"2023-11-24T06:11:45.064868+0000","last_became_peered":"2023-11-24T06:11:45.064868+0000","last_unstale":"2023-11-24T06:11:46.042867+0000","last_undegraded":"2023-11-24T06:11:46.042867+0000","last_fullsized":"2023-11-24T06:11:46.042867+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_clean_scrub_stamp":"2023-11-24T06:11:44.018686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:24:27.017978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:11:46.162337+0000","last_change":"2023-11-24T06:11:44.071760+0000","last_active":"2023-11-24T06:11:46.162337+0000","last_peered":"2023-11-24T06:11:46.162337+0000","last_clean":"2023-11-24T06:11:46.162337+0000","last_became_active":"2023-11-24T06:11:44.071383+0000","last_became_peered":"2023-11-24T06:11:44.071383+0000","last_unstale":"2023-11-24T06:11:46.162337+0000","last_undegraded":"2023-11-24T06:11:46.162337+0000","last_fullsized":"2023-11-24T06:11:46.162337+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:43.003933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:43.003933+0000","last_clean_scrub_stamp":"2023-11-24T06:11:43.003933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:55:16.324642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:11:46.164257+0000","last_change":"2023-11-24T06:11:45.057810+0000","last_active":"2023-11-24T06:11:46.164257+0000","last_peered":"2023-11-24T06:11:46.164257+0000","last_clean":"2023-11-24T06:11:46.164257+0000","last_became_active":"2023-11-24T06:11:45.057459+0000","last_became_peered":"2023-11-24T06:11:45.057459+0000","last_unstale":"2023-11-24T06:11:46.164257+0000","last_undegraded":"2023-11-24T06:11:46.164257+0000","last_fullsized":"2023-11-24T06:11:46.164257+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_clean_scrub_stamp":"2023-11-24T06:11:44.018686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:29:49.006871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:11:46.044329+0000","last_change":"2023-11-24T06:11:44.066074+0000","last_active":"2023-11-24T06:11:46.044329+0000","last_peered":"2023-11-24T06:11:46.044329+0000","last_clean":"2023-11-24T06:11:46.044329+0000","last_became_active":"2023-11-24T06:11:44.065792+0000","last_became_peered":"2023-11-24T06:11:44.065792+0000","last_unstale":"2023-11-24T06:11:46.044329+0000","last_undegraded":"2023-11-24T06:11:46.044329+0000","last_fullsized":"2023-11-24T06:11:46.044329+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:43.003933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:43.003933+0000","last_clean_scrub_stamp":"2023-11-24T06:11:43.003933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:36:30.984958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":176,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T06:11:45.049256+0000","last_change":"2023-11-24T06:10:20.082677+0000","last_active":"2023-11-24T06:11:45.049256+0000","last_peered":"2023-11-24T06:11:45.049256+0000","last_clean":"2023-11-24T06:11:45.049256+0000","last_became_active":"2023-11-24T06:10:20.080551+0000","last_became_peered":"2023-11-24T06:10:20.080551+0000","last_unstale":"2023-11-24T06:11:45.049256+0000","last_undegraded":"2023-11-24T06:11:45.049256+0000","last_fullsized":"2023-11-24T06:11:45.049256+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:18.602209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:18.602209+0000","last_clean_scrub_stamp":"2023-11-24T06:10:18.602209+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:47:24.216235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574868,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32132,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339708,"statfs":{"total":96636764160,"available":96603860992,"internally_reserved":0,"allocated":970752,"data_stored":737382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:11:18 2023","interfaces":[{"interface":"back","average":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"min":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":1.9630000000000001,"5min":1.9630000000000001,"15min":1.9630000000000001},"last":1.103},{"interface":"front","average":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"min":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"max":{"1min":1.621,"5min":1.621,"15min":1.621},"last":1.4059999999999999}]},{"osd":1,"last update":"Fri Nov 24 06:11:18 2023","interfaces":[{"interface":"back","average":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"min":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"max":{"1min":2.0259999999999998,"5min":2.0259999999999998,"15min":2.0259999999999998},"last":0.89900000000000002},{"interface":"front","average":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":2.1480000000000001,"5min":2.1480000000000001,"15min":2.1480000000000001},"last":0.99099999999999999}]},{"osd":2,"last update":"Fri Nov 24 06:11:18 2023","interfaces":[{"interface":"back","average":{"1min":1.143,"5min":1.143,"15min":1.143},"min":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":1.7809999999999999,"5min":1.7809999999999999,"15min":1.7809999999999999},"last":1.054},{"interface":"front","average":{"1min":1.1719999999999999,"5min":1.1719999999999999,"15min":1.1719999999999999},"min":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"max":{"1min":1.4810000000000001,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":1.2130000000000001}]},{"osd":3,"last update":"Fri Nov 24 06:11:18 2023","interfaces":[{"interface":"back","average":{"1min":1.206,"5min":1.206,"15min":1.206},"min":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"max":{"1min":1.593,"5min":1.593,"15min":1.593},"last":1.462},{"interface":"front","average":{"1min":1.3109999999999999,"5min":1.3109999999999999,"15min":1.3109999999999999},"min":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"max":{"1min":1.6539999999999999,"5min":1.6539999999999999,"15min":1.6539999999999999},"last":1.359}]},{"osd":4,"last update":"Fri Nov 24 06:11:18 2023","interfaces":[{"interface":"back","average":{"1min":1.2889999999999999,"5min":1.2889999999999999,"15min":1.2889999999999999},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.76,"5min":1.76,"15min":1.76},"last":1.5069999999999999},{"interface":"front","average":{"1min":1.329,"5min":1.329,"15min":1.329},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":2.1680000000000001,"5min":2.1680000000000001,"15min":2.1680000000000001},"last":1.5720000000000001}]},{"osd":5,"last update":"Fri Nov 24 06:11:18 2023","interfaces":[{"interface":"back","average":{"1min":1.446,"5min":1.446,"15min":1.446},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.8819999999999999,"5min":1.8819999999999999,"15min":1.8819999999999999},"last":1.621},{"interface":"front","average":{"1min":1.454,"5min":1.454,"15min":1.454},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":2.0539999999999998,"5min":2.0539999999999998,"15min":2.0539999999999998},"last":1.7529999999999999}]},{"osd":6,"last update":"Fri Nov 24 06:11:18 2023","interfaces":[{"interface":"back","average":{"1min":1.5209999999999999,"5min":1.5209999999999999,"15min":1.5209999999999999},"min":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.9750000000000001,"5min":1.9750000000000001,"15min":1.9750000000000001},"last":1.71},{"interface":"front","average":{"1min":1.53,"5min":1.53,"15min":1.53},"min":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":2.0680000000000001,"5min":2.0680000000000001,"15min":2.0680000000000001},"last":1.8640000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574869,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31552,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340288,"statfs":{"total":96636764160,"available":96604454912,"internally_reserved":0,"allocated":380928,"data_stored":147540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:11:18 2023","interfaces":[{"interface":"back","average":{"1min":1.2190000000000001,"5min":1.2190000000000001,"15min":1.2190000000000001},"min":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":1.921,"5min":1.921,"15min":1.921},"last":0.72899999999999998},{"interface":"front","average":{"1min":1.2250000000000001,"5min":1.2250000000000001,"15min":1.2250000000000001},"min":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":2.0230000000000001,"5min":2.0230000000000001,"15min":2.0230000000000001},"last":0.89800000000000002}]},{"osd":1,"last update":"Fri Nov 24 06:11:18 2023","interfaces":[{"interface":"back","average":{"1min":1.167,"5min":1.167,"15min":1.167},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":2.0609999999999999,"5min":2.0609999999999999,"15min":2.0609999999999999},"last":1.135},{"interface":"front","average":{"1min":1.2350000000000001,"5min":1.2350000000000001,"15min":1.2350000000000001},"min":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"max":{"1min":1.8899999999999999,"5min":1.8899999999999999,"15min":1.8899999999999999},"last":0.79600000000000004}]},{"osd":2,"last update":"Fri Nov 24 06:11:18 2023","interfaces":[{"interface":"back","average":{"1min":1.1519999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"min":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"max":{"1min":1.655,"5min":1.655,"15min":1.655},"last":0.93400000000000005},{"interface":"front","average":{"1min":1.4390000000000001,"5min":1.4390000000000001,"15min":1.4390000000000001},"min":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"max":{"1min":2.1640000000000001,"5min":2.1640000000000001,"15min":2.1640000000000001},"last":1.323}]},{"osd":3,"last update":"Fri Nov 24 06:11:18 2023","interfaces":[{"interface":"back","average":{"1min":1.3140000000000001,"5min":1.3140000000000001,"15min":1.3140000000000001},"min":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"max":{"1min":1.919,"5min":1.919,"15min":1.919},"last":1.012},{"interface":"front","average":{"1min":1.296,"5min":1.296,"15min":1.296},"min":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.6870000000000001,"5min":1.6870000000000001,"15min":1.6870000000000001},"last":1.0840000000000001}]},{"osd":4,"last update":"Fri Nov 24 06:11:18 2023","interfaces":[{"interface":"back","average":{"1min":1.458,"5min":1.458,"15min":1.458},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":2.2410000000000001,"5min":2.2410000000000001,"15min":2.2410000000000001},"last":1.4450000000000001},{"interface":"front","average":{"1min":1.48,"5min":1.48,"15min":1.48},"min":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":2.21,"5min":2.21,"15min":2.21},"last":1.2090000000000001}]},{"osd":5,"last update":"Fri Nov 24 06:11:18 2023","interfaces":[{"interface":"back","average":{"1min":1.4770000000000001,"5min":1.4770000000000001,"15min":1.4770000000000001},"min":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"max":{"1min":2.0659999999999998,"5min":2.0659999999999998,"15min":2.0659999999999998},"last":1.2669999999999999},{"interface":"front","average":{"1min":1.657,"5min":1.657,"15min":1.657},"min":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"max":{"1min":2.4089999999999998,"5min":2.4089999999999998,"15min":2.4089999999999998},"last":1.5349999999999999}]},{"osd":7,"last update":"Fri Nov 24 06:11:18 2023","interfaces":[{"interface":"back","average":{"1min":1.591,"5min":1.591,"15min":1.591},"min":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":2.5139999999999998,"5min":2.5139999999999998,"15min":2.5139999999999998},"last":1.585},{"interface":"front","average":{"1min":1.548,"5min":1.548,"15min":1.548},"min":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":2.3370000000000002,"5min":2.3370000000000002,"15min":2.3370000000000002},"last":1.379}]}]},{"osd":5,"up_from":13,"seq":55834574869,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31548,"kb_used_data":368,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340292,"statfs":{"total":96636764160,"available":96604459008,"internally_reserved":0,"allocated":376832,"data_stored":146982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:11:18 2023","interfaces":[{"interface":"back","average":{"1min":1.2509999999999999,"5min":1.2509999999999999,"15min":1.2509999999999999},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":3.0030000000000001,"5min":3.0030000000000001,"15min":3.0030000000000001},"last":0.49099999999999999},{"interface":"front","average":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":1.2270000000000001},"min":{"1min":0.749,"5min":0.749,"15min":0.749},"max":{"1min":2.0979999999999999,"5min":2.0979999999999999,"15min":2.0979999999999999},"last":0.39100000000000001}]},{"osd":1,"last update":"Fri Nov 24 06:11:18 2023","interfaces":[{"interface":"back","average":{"1min":1.276,"5min":1.276,"15min":1.276},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":5.0499999999999998,"5min":5.0499999999999998,"15min":5.0499999999999998},"last":0.53300000000000003},{"interface":"front","average":{"1min":1.48,"5min":1.48,"15min":1.48},"min":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":3.6930000000000001,"5min":3.6930000000000001,"15min":3.6930000000000001},"last":0.435}]},{"osd":2,"last update":"Fri Nov 24 06:11:18 2023","interfaces":[{"interface":"back","average":{"1min":1.345,"5min":1.345,"15min":1.345},"min":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"max":{"1min":3.085,"5min":3.085,"15min":3.085},"last":0.93899999999999995},{"interface":"front","average":{"1min":1.6160000000000001,"5min":1.6160000000000001,"15min":1.6160000000000001},"min":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"max":{"1min":4.7359999999999998,"5min":4.7359999999999998,"15min":4.7359999999999998},"last":0.46400000000000002}]},{"osd":3,"last update":"Fri Nov 24 06:11:18 2023","interfaces":[{"interface":"back","average":{"1min":1.421,"5min":1.421,"15min":1.421},"min":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"max":{"1min":3.2229999999999999,"5min":3.2229999999999999,"15min":3.2229999999999999},"last":0.77900000000000003},{"interface":"front","average":{"1min":1.4450000000000001,"5min":1.4450000000000001,"15min":1.4450000000000001},"min":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"max":{"1min":2.3260000000000001,"5min":2.3260000000000001,"15min":2.3260000000000001},"last":0.76100000000000001}]},{"osd":4,"last update":"Fri Nov 24 06:11:18 2023","interfaces":[{"interface":"back","average":{"1min":1.5669999999999999,"5min":1.5669999999999999,"15min":1.5669999999999999},"min":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"max":{"1min":3.1230000000000002,"5min":3.1230000000000002,"15min":3.1230000000000002},"last":0.84299999999999997},{"interface":"front","average":{"1min":1.5589999999999999,"5min":1.5589999999999999,"15min":1.5589999999999999},"min":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"max":{"1min":2.4809999999999999,"5min":2.4809999999999999,"15min":2.4809999999999999},"last":0.64700000000000002}]},{"osd":6,"last update":"Fri Nov 24 06:11:18 2023","interfaces":[{"interface":"back","average":{"1min":1.7569999999999999,"5min":1.7569999999999999,"15min":1.7569999999999999},"min":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"max":{"1min":4.6529999999999996,"5min":4.6529999999999996,"15min":4.6529999999999996},"last":0.58599999999999997},{"interface":"front","average":{"1min":1.75,"5min":1.75,"15min":1.75},"min":{"1min":1.125,"5min":1.125,"15min":1.125},"max":{"1min":4.3529999999999998,"5min":4.3529999999999998,"15min":4.3529999999999998},"last":0.90400000000000003}]},{"osd":7,"last update":"Fri Nov 24 06:11:18 2023","interfaces":[{"interface":"back","average":{"1min":1.7110000000000001,"5min":1.7110000000000001,"15min":1.7110000000000001},"min":{"1min":1.2450000000000001,"5min":1.2450000000000001,"15min":1.2450000000000001},"max":{"1min":2.8839999999999999,"5min":2.8839999999999999,"15min":2.8839999999999999},"last":0.874},{"interface":"front","average":{"1min":1.867,"5min":1.867,"15min":1.867},"min":{"1min":1.204,"5min":1.204,"15min":1.204},"max":{"1min":4.8739999999999997,"5min":4.8739999999999997,"15min":4.8739999999999997},"last":0.69599999999999995}]}]},{"osd":4,"up_from":13,"seq":55834574869,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31552,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340288,"statfs":{"total":96636764160,"available":96604454912,"internally_reserved":0,"allocated":380928,"data_stored":147028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:11:21 2023","interfaces":[{"interface":"back","average":{"1min":1.042,"5min":1.042,"15min":1.042},"min":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":1.498,"5min":1.498,"15min":1.498},"last":1.3149999999999999},{"interface":"front","average":{"1min":1.2170000000000001,"5min":1.2170000000000001,"15min":1.2170000000000001},"min":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"max":{"1min":1.8220000000000001,"5min":1.8220000000000001,"15min":1.8220000000000001},"last":1.085}]},{"osd":1,"last update":"Fri Nov 24 06:11:21 2023","interfaces":[{"interface":"back","average":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"min":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":1.6859999999999999,"5min":1.6859999999999999,"15min":1.6859999999999999},"last":0.874},{"interface":"front","average":{"1min":1.1779999999999999,"5min":1.1779999999999999,"15min":1.1779999999999999},"min":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":1.875,"5min":1.875,"15min":1.875},"last":1.139}]},{"osd":2,"last update":"Fri Nov 24 06:11:21 2023","interfaces":[{"interface":"back","average":{"1min":1.2509999999999999,"5min":1.2509999999999999,"15min":1.2509999999999999},"min":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"max":{"1min":2.319,"5min":2.319,"15min":2.319},"last":1.21},{"interface":"front","average":{"1min":1.3089999999999999,"5min":1.3089999999999999,"15min":1.3089999999999999},"min":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"max":{"1min":2.0990000000000002,"5min":2.0990000000000002,"15min":2.0990000000000002},"last":1.3620000000000001}]},{"osd":3,"last update":"Fri Nov 24 06:11:21 2023","interfaces":[{"interface":"back","average":{"1min":1.4119999999999999,"5min":1.4119999999999999,"15min":1.4119999999999999},"min":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"max":{"1min":2.0179999999999998,"5min":2.0179999999999998,"15min":2.0179999999999998},"last":1.272},{"interface":"front","average":{"1min":1.4419999999999999,"5min":1.4419999999999999,"15min":1.4419999999999999},"min":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"max":{"1min":2.1720000000000002,"5min":2.1720000000000002,"15min":2.1720000000000002},"last":1.569}]},{"osd":5,"last update":"Fri Nov 24 06:11:21 2023","interfaces":[{"interface":"back","average":{"1min":1.3380000000000001,"5min":1.3380000000000001,"15min":1.3380000000000001},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.895,"5min":1.895,"15min":1.895},"last":1.419},{"interface":"front","average":{"1min":1.552,"5min":1.552,"15min":1.552},"min":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"max":{"1min":2.5059999999999998,"5min":2.5059999999999998,"15min":2.5059999999999998},"last":1.4359999999999999}]},{"osd":6,"last update":"Fri Nov 24 06:11:21 2023","interfaces":[{"interface":"back","average":{"1min":1.5469999999999999,"5min":1.5469999999999999,"15min":1.5469999999999999},"min":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"max":{"1min":2.2069999999999999,"5min":2.2069999999999999,"15min":2.2069999999999999},"last":1.3919999999999999},{"interface":"front","average":{"1min":1.6850000000000001,"5min":1.6850000000000001,"15min":1.6850000000000001},"min":{"1min":1.119,"5min":1.119,"15min":1.119},"max":{"1min":2.4510000000000001,"5min":2.4510000000000001,"15min":2.4510000000000001},"last":1.633}]},{"osd":7,"last update":"Fri Nov 24 06:11:21 2023","interfaces":[{"interface":"back","average":{"1min":1.6439999999999999,"5min":1.6439999999999999,"15min":1.6439999999999999},"min":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"max":{"1min":2.4350000000000001,"5min":2.4350000000000001,"15min":2.4350000000000001},"last":1.5249999999999999},{"interface":"front","average":{"1min":1.633,"5min":1.633,"15min":1.633},"min":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"max":{"1min":3.1120000000000001,"5min":3.1120000000000001,"15min":3.1120000000000001},"last":1.8200000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574869,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27400,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344440,"statfs":{"total":96636764160,"available":96608706560,"internally_reserved":0,"allocated":389120,"data_stored":148558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:11:22 2023","interfaces":[{"interface":"back","average":{"1min":1.008,"5min":1.008,"15min":1.008},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":2.766,"5min":2.766,"15min":2.766},"last":1.165},{"interface":"front","average":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.73,"5min":1.73,"15min":1.73},"last":1.4370000000000001}]},{"osd":1,"last update":"Fri Nov 24 06:11:22 2023","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":2.5779999999999998,"5min":2.5779999999999998,"15min":2.5779999999999998},"last":0.97699999999999998},{"interface":"front","average":{"1min":1.1839999999999999,"5min":1.1839999999999999,"15min":1.1839999999999999},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":3.6899999999999999,"5min":3.6899999999999999,"15min":3.6899999999999999},"last":1.4670000000000001}]},{"osd":2,"last update":"Fri Nov 24 06:11:22 2023","interfaces":[{"interface":"back","average":{"1min":1.177,"5min":1.177,"15min":1.177},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":4.0620000000000003,"5min":4.0620000000000003,"15min":4.0620000000000003},"last":1.081},{"interface":"front","average":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":1.2270000000000001},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":2.9529999999999998,"5min":2.9529999999999998,"15min":2.9529999999999998},"last":1.3700000000000001}]},{"osd":4,"last update":"Fri Nov 24 06:11:22 2023","interfaces":[{"interface":"back","average":{"1min":1.4279999999999999,"5min":1.4279999999999999,"15min":1.4279999999999999},"min":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":4.125,"5min":4.125,"15min":4.125},"last":1.518},{"interface":"front","average":{"1min":1.4470000000000001,"5min":1.4470000000000001,"15min":1.4470000000000001},"min":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"max":{"1min":3.9409999999999998,"5min":3.9409999999999998,"15min":3.9409999999999998},"last":1.2}]},{"osd":5,"last update":"Fri Nov 24 06:11:22 2023","interfaces":[{"interface":"back","average":{"1min":1.4470000000000001,"5min":1.4470000000000001,"15min":1.4470000000000001},"min":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"max":{"1min":3.3359999999999999,"5min":3.3359999999999999,"15min":3.3359999999999999},"last":1.544},{"interface":"front","average":{"1min":1.46,"5min":1.46,"15min":1.46},"min":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":4.1669999999999998,"5min":4.1669999999999998,"15min":4.1669999999999998},"last":1.0349999999999999}]},{"osd":6,"last update":"Fri Nov 24 06:11:22 2023","interfaces":[{"interface":"back","average":{"1min":1.4370000000000001,"5min":1.4370000000000001,"15min":1.4370000000000001},"min":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":4.0190000000000001,"5min":4.0190000000000001,"15min":4.0190000000000001},"last":1.1419999999999999},{"interface":"front","average":{"1min":1.5329999999999999,"5min":1.5329999999999999,"15min":1.5329999999999999},"min":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"max":{"1min":3.2480000000000002,"5min":3.2480000000000002,"15min":3.2480000000000002},"last":1.627}]},{"osd":7,"last update":"Fri Nov 24 06:11:22 2023","interfaces":[{"interface":"back","average":{"1min":1.613,"5min":1.613,"15min":1.613},"min":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"max":{"1min":4.202,"5min":4.202,"15min":4.202},"last":1.2889999999999999},{"interface":"front","average":{"1min":1.6220000000000001,"5min":1.6220000000000001,"15min":1.6220000000000001},"min":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"max":{"1min":3.9849999999999999,"5min":3.9849999999999999,"15min":3.9849999999999999},"last":1.2549999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574869,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27384,"kb_used_data":364,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344456,"statfs":{"total":96636764160,"available":96608722944,"internally_reserved":0,"allocated":372736,"data_stored":146380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:11:21 2023","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.6259999999999999,"5min":1.6259999999999999,"15min":1.6259999999999999},"last":0.67400000000000004},{"interface":"front","average":{"1min":1.101,"5min":1.101,"15min":1.101},"min":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.7669999999999999,"5min":1.7669999999999999,"15min":1.7669999999999999},"last":0.63}]},{"osd":1,"last update":"Fri Nov 24 06:11:21 2023","interfaces":[{"interface":"back","average":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.506,"5min":1.506,"15min":1.506},"last":1.028},{"interface":"front","average":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.8140000000000001,"5min":1.8140000000000001,"15min":1.8140000000000001},"last":1.0600000000000001}]},{"osd":3,"last update":"Fri Nov 24 06:11:21 2023","interfaces":[{"interface":"back","average":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":2.3559999999999999,"5min":2.3559999999999999,"15min":2.3559999999999999},"last":0.75600000000000001},{"interface":"front","average":{"1min":1.123,"5min":1.123,"15min":1.123},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":2.1840000000000002,"5min":2.1840000000000002,"15min":2.1840000000000002},"last":1.1120000000000001}]},{"osd":4,"last update":"Fri Nov 24 06:11:21 2023","interfaces":[{"interface":"back","average":{"1min":1.419,"5min":1.419,"15min":1.419},"min":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":3.411,"5min":3.411,"15min":3.411},"last":1.2769999999999999},{"interface":"front","average":{"1min":1.506,"5min":1.506,"15min":1.506},"min":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"max":{"1min":3.7109999999999999,"5min":3.7109999999999999,"15min":3.7109999999999999},"last":1.395}]},{"osd":5,"last update":"Fri Nov 24 06:11:21 2023","interfaces":[{"interface":"back","average":{"1min":1.325,"5min":1.325,"15min":1.325},"min":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"max":{"1min":2.5259999999999998,"5min":2.5259999999999998,"15min":2.5259999999999998},"last":1.252},{"interface":"front","average":{"1min":1.302,"5min":1.302,"15min":1.302},"min":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"max":{"1min":2.1640000000000001,"5min":2.1640000000000001,"15min":2.1640000000000001},"last":1.5029999999999999}]},{"osd":6,"last update":"Fri Nov 24 06:11:21 2023","interfaces":[{"interface":"back","average":{"1min":1.4570000000000001,"5min":1.4570000000000001,"15min":1.4570000000000001},"min":{"1min":1.073,"5min":1.073,"15min":1.073},"max":{"1min":2.3039999999999998,"5min":2.3039999999999998,"15min":2.3039999999999998},"last":1.198},{"interface":"front","average":{"1min":1.4690000000000001,"5min":1.4690000000000001,"15min":1.4690000000000001},"min":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"max":{"1min":2.605,"5min":2.605,"15min":2.605},"last":1.169}]},{"osd":7,"last update":"Fri Nov 24 06:11:21 2023","interfaces":[{"interface":"back","average":{"1min":1.595,"5min":1.595,"15min":1.595},"min":{"1min":1.1910000000000001,"5min":1.1910000000000001,"15min":1.1910000000000001},"max":{"1min":2.3740000000000001,"5min":2.3740000000000001,"15min":2.3740000000000001},"last":1.331},{"interface":"front","average":{"1min":1.4670000000000001,"5min":1.4670000000000001,"15min":1.4670000000000001},"min":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"max":{"1min":2.5699999999999998,"5min":2.5699999999999998,"15min":2.5699999999999998},"last":1.222}]}]},{"osd":1,"up_from":13,"seq":55834574868,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27408,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344432,"statfs":{"total":96636764160,"available":96608698368,"internally_reserved":0,"allocated":389120,"data_stored":148558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:11:19 2023","interfaces":[{"interface":"back","average":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.9299999999999999,"5min":1.9299999999999999,"15min":1.9299999999999999},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.7669999999999999,"5min":1.7669999999999999,"15min":1.7669999999999999},"last":0.71899999999999997}]},{"osd":2,"last update":"Fri Nov 24 06:11:19 2023","interfaces":[{"interface":"back","average":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":2.5329999999999999,"5min":2.5329999999999999,"15min":2.5329999999999999},"last":0.74099999999999999},{"interface":"front","average":{"1min":1.03,"5min":1.03,"15min":1.03},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.657,"5min":1.657,"15min":1.657},"last":0.57899999999999996}]},{"osd":3,"last update":"Fri Nov 24 06:11:19 2023","interfaces":[{"interface":"back","average":{"1min":1.038,"5min":1.038,"15min":1.038},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.841,"5min":1.841,"15min":1.841},"last":0.69299999999999995},{"interface":"front","average":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":2.8940000000000001,"5min":2.8940000000000001,"15min":2.8940000000000001},"last":0.95899999999999996}]},{"osd":4,"last update":"Fri Nov 24 06:11:19 2023","interfaces":[{"interface":"back","average":{"1min":1.282,"5min":1.282,"15min":1.282},"min":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"max":{"1min":2.1930000000000001,"5min":2.1930000000000001,"15min":2.1930000000000001},"last":1.0900000000000001},{"interface":"front","average":{"1min":1.2,"5min":1.2,"15min":1.2},"min":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":1.784,"5min":1.784,"15min":1.784},"last":1.169}]},{"osd":5,"last update":"Fri Nov 24 06:11:19 2023","interfaces":[{"interface":"back","average":{"1min":1.53,"5min":1.53,"15min":1.53},"min":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"max":{"1min":5.5449999999999999,"5min":5.5449999999999999,"15min":5.5449999999999999},"last":1.0269999999999999},{"interface":"front","average":{"1min":1.5640000000000001,"5min":1.5640000000000001,"15min":1.5640000000000001},"min":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"max":{"1min":6.0030000000000001,"5min":6.0030000000000001,"15min":6.0030000000000001},"last":1.2370000000000001}]},{"osd":6,"last update":"Fri Nov 24 06:11:19 2023","interfaces":[{"interface":"back","average":{"1min":1.569,"5min":1.569,"15min":1.569},"min":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"max":{"1min":5.2809999999999997,"5min":5.2809999999999997,"15min":5.2809999999999997},"last":0.98699999999999999},{"interface":"front","average":{"1min":1.4750000000000001,"5min":1.4750000000000001,"15min":1.4750000000000001},"min":{"1min":0.749,"5min":0.749,"15min":0.749},"max":{"1min":4.327,"5min":4.327,"15min":4.327},"last":1.1499999999999999}]},{"osd":7,"last update":"Fri Nov 24 06:11:19 2023","interfaces":[{"interface":"back","average":{"1min":1.359,"5min":1.359,"15min":1.359},"min":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":2.3069999999999999,"5min":2.3069999999999999,"15min":2.3069999999999999},"last":1.01},{"interface":"front","average":{"1min":1.5980000000000001,"5min":1.5980000000000001,"15min":1.5980000000000001},"min":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"max":{"1min":5.4500000000000002,"5min":5.4500000000000002,"15min":5.4500000000000002},"last":1.0509999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574868,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27968,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343872,"statfs":{"total":96636764160,"available":96608124928,"internally_reserved":0,"allocated":966656,"data_stored":737292,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Nov 24 06:11:18 2023","interfaces":[{"interface":"back","average":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":2.3999999999999999,"5min":2.3999999999999999,"15min":2.3999999999999999},"last":0.97799999999999998},{"interface":"front","average":{"1min":1.1879999999999999,"5min":1.1879999999999999,"15min":1.1879999999999999},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":4.8289999999999997,"5min":4.8289999999999997,"15min":4.8289999999999997},"last":0.71299999999999997}]},{"osd":2,"last update":"Fri Nov 24 06:11:18 2023","interfaces":[{"interface":"back","average":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":2.5139999999999998,"5min":2.5139999999999998,"15min":2.5139999999999998},"last":0.61299999999999999},{"interface":"front","average":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":2.3599999999999999,"5min":2.3599999999999999,"15min":2.3599999999999999},"last":0.86699999999999999}]},{"osd":3,"last update":"Fri Nov 24 06:11:18 2023","interfaces":[{"interface":"back","average":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":2.5840000000000001,"5min":2.5840000000000001,"15min":2.5840000000000001},"last":0.749},{"interface":"front","average":{"1min":1.415,"5min":1.415,"15min":1.415},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":5.1230000000000002,"5min":5.1230000000000002,"15min":5.1230000000000002},"last":1.1359999999999999}]},{"osd":4,"last update":"Fri Nov 24 06:11:18 2023","interfaces":[{"interface":"back","average":{"1min":1.2829999999999999,"5min":1.2829999999999999,"15min":1.2829999999999999},"min":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"max":{"1min":2.887,"5min":2.887,"15min":2.887},"last":1.085},{"interface":"front","average":{"1min":1.3360000000000001,"5min":1.3360000000000001,"15min":1.3360000000000001},"min":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":2.8530000000000002,"5min":2.8530000000000002,"15min":2.8530000000000002},"last":0.92800000000000005}]},{"osd":5,"last update":"Fri Nov 24 06:11:18 2023","interfaces":[{"interface":"back","average":{"1min":1.575,"5min":1.575,"15min":1.575},"min":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"max":{"1min":4.6299999999999999,"5min":4.6299999999999999,"15min":4.6299999999999999},"last":1.218},{"interface":"front","average":{"1min":1.333,"5min":1.333,"15min":1.333},"min":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":2.794,"5min":2.794,"15min":2.794},"last":1.2509999999999999}]},{"osd":6,"last update":"Fri Nov 24 06:11:18 2023","interfaces":[{"interface":"back","average":{"1min":1.6719999999999999,"5min":1.6719999999999999,"15min":1.6719999999999999},"min":{"1min":0.747,"5min":0.747,"15min":0.747},"max":{"1min":4.798,"5min":4.798,"15min":4.798},"last":1.169},{"interface":"front","average":{"1min":1.637,"5min":1.637,"15min":1.637},"min":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"max":{"1min":4.7149999999999999,"5min":4.7149999999999999,"15min":4.7149999999999999},"last":1.351}]},{"osd":7,"last update":"Fri Nov 24 06:11:18 2023","interfaces":[{"interface":"back","average":{"1min":1.716,"5min":1.716,"15min":1.716},"min":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":4.7649999999999997,"5min":4.7649999999999997,"15min":4.7649999999999997},"last":1.0569999999999999},{"interface":"front","average":{"1min":1.6950000000000001,"5min":1.6950000000000001,"15min":1.6950000000000001},"min":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"max":{"1min":4.6820000000000004,"5min":4.6820000000000004,"15min":4.6820000000000004},"last":1.3}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-24T06:11:49.922 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-24T06:11:50.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:50.168+0000 7f4521831640 1 Processor -- start 2023-11-24T06:11:50.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:50.168+0000 7f4521831640 1 -- start start 2023-11-24T06:11:50.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:50.170+0000 7f4521831640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f451c13df80 0x7f451c15bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:50.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:50.170+0000 7f4521831640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f451c13e360 0x7f451c0cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:50.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:50.170+0000 7f4521831640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f451c06c470 con 0x7f451c15f9c0 2023-11-24T06:11:50.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:50.170+0000 7f4521831640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f451c06c750 con 0x7f451c13e360 2023-11-24T06:11:50.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:50.170+0000 7f4521831640 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f451c06ca30 con 0x7f451c13df80 2023-11-24T06:11:50.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:50.170+0000 7f451b7fe640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f451c15f9c0 0x7f451c1408d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:41232/0 (socket says 172.21.15.92:41232) 2023-11-24T06:11:50.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:50.170+0000 7f451b7fe640 1 -- 172.21.15.92:0/2467342504 learned_addr learned my addr 172.21.15.92:0/2467342504 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-24T06:11:50.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:50.170+0000 7f451a7fc640 1 --2- 172.21.15.92:0/2467342504 >> v2:172.21.15.92:3300/0 conn(0x7f451c13e360 0x7f451c0cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:50.175 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:50.170+0000 7f451affd640 1 --2- 172.21.15.92:0/2467342504 >> v2:172.21.15.118:3300/0 conn(0x7f451c13df80 0x7f451c15bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:50.175 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:50.171+0000 7f451a7fc640 1 -- 172.21.15.92:0/2467342504 >> v2:172.21.15.118:3300/0 conn(0x7f451c13df80 msgr2=0x7f451c15bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:50.175 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:50.171+0000 7f451a7fc640 1 --2- 172.21.15.92:0/2467342504 >> v2:172.21.15.118:3300/0 conn(0x7f451c13df80 0x7f451c15bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:50.175 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:50.171+0000 7f451a7fc640 1 -- 172.21.15.92:0/2467342504 >> v1:172.21.15.92:6789/0 conn(0x7f451c15f9c0 legacy=0x7f451c1408d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:50.175 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:50.171+0000 7f451a7fc640 1 -- 172.21.15.92:0/2467342504 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f451c06cd10 con 0x7f451c13e360 2023-11-24T06:11:50.175 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:50.172+0000 7f451a7fc640 1 --2- 172.21.15.92:0/2467342504 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f451c13e360 0x7f451c0cf290 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f451c072770 tx=0x7f451002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2e2fd599d45712d server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:50.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:50.172+0000 7f4519ffb640 1 -- 172.21.15.92:0/2467342504 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4510031830 con 0x7f451c13e360 2023-11-24T06:11:50.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:50.172+0000 7f4519ffb640 1 -- 172.21.15.92:0/2467342504 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4510031990 con 0x7f451c13e360 2023-11-24T06:11:50.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:50.172+0000 7f4519ffb640 1 -- 172.21.15.92:0/2467342504 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4510031d20 con 0x7f451c13e360 2023-11-24T06:11:50.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:50.173+0000 7f4521831640 1 -- 172.21.15.92:0/2467342504 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f451c13e360 msgr2=0x7f451c0cf290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:50.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:50.173+0000 7f4521831640 1 --2- 172.21.15.92:0/2467342504 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f451c13e360 0x7f451c0cf290 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f451c072770 tx=0x7f451002b8d0 comp rx=0 tx=0).stop 2023-11-24T06:11:50.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:50.173+0000 7f4521831640 1 -- 172.21.15.92:0/2467342504 shutdown_connections 2023-11-24T06:11:50.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:50.173+0000 7f4521831640 1 --2- 172.21.15.92:0/2467342504 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f451c13e360 0x7f451c0cf290 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:50.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:50.173+0000 7f4521831640 1 --2- 172.21.15.92:0/2467342504 >> v2:172.21.15.118:3300/0 conn(0x7f451c13df80 0x7f451c15bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:50.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:50.173+0000 7f4521831640 1 -- 172.21.15.92:0/2467342504 >> 172.21.15.92:0/2467342504 conn(0x7f451c06a800 msgr2=0x7f451c06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:50.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:50.173+0000 7f4521831640 1 -- 172.21.15.92:0/2467342504 shutdown_connections 2023-11-24T06:11:50.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:50.173+0000 7f4521831640 1 -- 172.21.15.92:0/2467342504 wait complete. 2023-11-24T06:11:50.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:50.174+0000 7f4521831640 1 Processor -- start 2023-11-24T06:11:50.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:50.175+0000 7f4521831640 1 -- start start 2023-11-24T06:11:50.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:50.175+0000 7f4521831640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f451c13df80 0x7f451c14dae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:50.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:50.175+0000 7f4521831640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f451c13e360 0x7f451c1f2490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:50.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:50.176+0000 7f4521831640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f451c15f9c0 0x7f451c1f6ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:50.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:50.176+0000 7f4521831640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f451c1f29d0 con 0x7f451c13df80 2023-11-24T06:11:50.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:50.176+0000 7f4521831640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f451c151570 con 0x7f451c15f9c0 2023-11-24T06:11:50.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:50.176+0000 7f4521831640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f451c1f7020 con 0x7f451c13e360 2023-11-24T06:11:50.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:50.176+0000 7f451affd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f451c13df80 0x7f451c14dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:50.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:50.176+0000 7f451affd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f451c13df80 0x7f451c14dae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58638/0 (socket says 172.21.15.92:58638) 2023-11-24T06:11:50.180 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:50.176+0000 7f451affd640 1 -- 172.21.15.92:0/2553807774 learned_addr learned my addr 172.21.15.92:0/2553807774 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:11:50.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:50.176+0000 7f451affd640 1 -- 172.21.15.92:0/2553807774 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f451c13e360 msgr2=0x7f451c1f2490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:50.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:50.176+0000 7f451affd640 1 --2- 172.21.15.92:0/2553807774 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f451c13e360 0x7f451c1f2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:50.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:50.176+0000 7f451affd640 1 -- 172.21.15.92:0/2553807774 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f451c15f9c0 msgr2=0x7f451c1f6ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:50.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:50.176+0000 7f451b7fe640 1 --2- 172.21.15.92:0/2553807774 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f451c15f9c0 0x7f451c1f6ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:50.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:50.176+0000 7f451affd640 1 --2- 172.21.15.92:0/2553807774 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f451c15f9c0 0x7f451c1f6ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:50.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:50.176+0000 7f451affd640 1 -- 172.21.15.92:0/2553807774 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4510009400 con 0x7f451c13df80 2023-11-24T06:11:50.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:50.177+0000 7f451affd640 1 --2- 172.21.15.92:0/2553807774 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f451c13df80 0x7f451c14dae0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f450c00d620 tx=0x7f450c00dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:50.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:50.177+0000 7f4503fff640 1 -- 172.21.15.92:0/2553807774 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f450c00b620 con 0x7f451c13df80 2023-11-24T06:11:50.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:50.177+0000 7f4503fff640 1 -- 172.21.15.92:0/2553807774 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f450c00b780 con 0x7f451c13df80 2023-11-24T06:11:50.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:50.177+0000 7f4503fff640 1 -- 172.21.15.92:0/2553807774 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f450c00bb30 con 0x7f451c13df80 2023-11-24T06:11:50.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:50.177+0000 7f4521831640 1 -- 172.21.15.92:0/2553807774 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f451c14eab0 con 0x7f451c13df80 2023-11-24T06:11:50.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:50.177+0000 7f4521831640 1 -- 172.21.15.92:0/2553807774 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f451c14e7d0 con 0x7f451c13df80 2023-11-24T06:11:50.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:50.178+0000 7f4521831640 1 -- 172.21.15.92:0/2553807774 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f451c0675d0 con 0x7f451c13df80 2023-11-24T06:11:50.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:50.179+0000 7f4503fff640 1 -- 172.21.15.92:0/2553807774 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f450c00bc90 con 0x7f451c13df80 2023-11-24T06:11:50.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:50.180+0000 7f4503fff640 1 --2- 172.21.15.92:0/2553807774 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f450806ae40 0x7f450806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:50.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:50.180+0000 7f4503fff640 1 -- 172.21.15.92:0/2553807774 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f450c089b80 con 0x7f451c13df80 2023-11-24T06:11:50.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:50.180+0000 7f451a7fc640 1 --2- 172.21.15.92:0/2553807774 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f450806ae40 0x7f450806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:50.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:50.181+0000 7f451a7fc640 1 --2- 172.21.15.92:0/2553807774 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f450806ae40 0x7f450806d300 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f4510036000 tx=0x7f4510030000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:50.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:50.184+0000 7f4503fff640 1 -- 172.21.15.92:0/2553807774 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f450c054b90 con 0x7f451c13df80 2023-11-24T06:11:50.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:50.369+0000 7f4521831640 1 -- 172.21.15.92:0/2553807774 --> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f451c0ce850 con 0x7f450806ae40 2023-11-24T06:11:50.395 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:50.392+0000 7f4503fff640 1 -- 172.21.15.92:0/2553807774 <== mgr.4101 v2:172.21.15.92:6800/784027401 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+287415 (secure 0 0 0) 0x7f451003b430 con 0x7f450806ae40 2023-11-24T06:11:50.398 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-24T06:11:50.401 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2023-11-24T06:11:50.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:50.399+0000 7f4521831640 1 -- 172.21.15.92:0/2553807774 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f450806ae40 msgr2=0x7f450806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:50.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:50.399+0000 7f4521831640 1 --2- 172.21.15.92:0/2553807774 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f450806ae40 0x7f450806d300 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f4510036000 tx=0x7f4510030000 comp rx=0 tx=0).stop 2023-11-24T06:11:50.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:50.399+0000 7f4521831640 1 -- 172.21.15.92:0/2553807774 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f451c13df80 msgr2=0x7f451c14dae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:50.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:50.399+0000 7f4521831640 1 --2- 172.21.15.92:0/2553807774 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f451c13df80 0x7f451c14dae0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f450c00d620 tx=0x7f450c00dae0 comp rx=0 tx=0).stop 2023-11-24T06:11:50.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:50.399+0000 7f4521831640 1 -- 172.21.15.92:0/2553807774 shutdown_connections 2023-11-24T06:11:50.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:50.399+0000 7f4521831640 1 --2- 172.21.15.92:0/2553807774 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f450806ae40 0x7f450806d300 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:50.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:50.399+0000 7f4521831640 1 --2- 172.21.15.92:0/2553807774 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f451c15f9c0 0x7f451c1f6ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:50.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:50.399+0000 7f4521831640 1 --2- 172.21.15.92:0/2553807774 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f451c13e360 0x7f451c1f2490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:50.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:50.399+0000 7f4521831640 1 --2- 172.21.15.92:0/2553807774 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f451c13df80 0x7f451c14dae0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:50.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:50.399+0000 7f4521831640 1 -- 172.21.15.92:0/2553807774 >> 172.21.15.92:0/2553807774 conn(0x7f451c06a800 msgr2=0x7f451c0affe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:50.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:50.399+0000 7f4521831640 1 -- 172.21.15.92:0/2553807774 shutdown_connections 2023-11-24T06:11:50.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:50.399+0000 7f4521831640 1 -- 172.21.15.92:0/2553807774 wait complete. 2023-11-24T06:11:50.422 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":72,"stamp":"2023-11-24T06:11:49.692027+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":170,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236944,"kb_used_data":4128,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754737776,"statfs":{"total":773094113280,"available":772851482624,"internally_reserved":0,"allocated":4227072,"data_stored":2359720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":33,"apply_latency_ms":33,"commit_latency_ns":33000000,"apply_latency_ns":33000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001647"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:11:46.043761+0000","last_change":"2023-11-24T06:11:45.058797+0000","last_active":"2023-11-24T06:11:46.043761+0000","last_peered":"2023-11-24T06:11:46.043761+0000","last_clean":"2023-11-24T06:11:46.043761+0000","last_became_active":"2023-11-24T06:11:45.057069+0000","last_became_peered":"2023-11-24T06:11:45.057069+0000","last_unstale":"2023-11-24T06:11:46.043761+0000","last_undegraded":"2023-11-24T06:11:46.043761+0000","last_fullsized":"2023-11-24T06:11:46.043761+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_clean_scrub_stamp":"2023-11-24T06:11:44.018686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:13:39.376575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:11:46.163519+0000","last_change":"2023-11-24T06:11:45.064449+0000","last_active":"2023-11-24T06:11:46.163519+0000","last_peered":"2023-11-24T06:11:46.163519+0000","last_clean":"2023-11-24T06:11:46.163519+0000","last_became_active":"2023-11-24T06:11:45.063848+0000","last_became_peered":"2023-11-24T06:11:45.063848+0000","last_unstale":"2023-11-24T06:11:46.163519+0000","last_undegraded":"2023-11-24T06:11:46.163519+0000","last_fullsized":"2023-11-24T06:11:46.163519+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_clean_scrub_stamp":"2023-11-24T06:11:44.018686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:16:44.582224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:11:46.163206+0000","last_change":"2023-11-24T06:11:45.067600+0000","last_active":"2023-11-24T06:11:46.163206+0000","last_peered":"2023-11-24T06:11:46.163206+0000","last_clean":"2023-11-24T06:11:46.163206+0000","last_became_active":"2023-11-24T06:11:45.067162+0000","last_became_peered":"2023-11-24T06:11:45.067162+0000","last_unstale":"2023-11-24T06:11:46.163206+0000","last_undegraded":"2023-11-24T06:11:46.163206+0000","last_fullsized":"2023-11-24T06:11:46.163206+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_clean_scrub_stamp":"2023-11-24T06:11:44.018686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:09:22.132463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:11:46.163962+0000","last_change":"2023-11-24T06:11:45.070948+0000","last_active":"2023-11-24T06:11:46.163962+0000","last_peered":"2023-11-24T06:11:46.163962+0000","last_clean":"2023-11-24T06:11:46.163962+0000","last_became_active":"2023-11-24T06:11:45.070666+0000","last_became_peered":"2023-11-24T06:11:45.070666+0000","last_unstale":"2023-11-24T06:11:46.163962+0000","last_undegraded":"2023-11-24T06:11:46.163962+0000","last_fullsized":"2023-11-24T06:11:46.163962+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_clean_scrub_stamp":"2023-11-24T06:11:44.018686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:55:17.339395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:11:46.042077+0000","last_change":"2023-11-24T06:11:45.062816+0000","last_active":"2023-11-24T06:11:46.042077+0000","last_peered":"2023-11-24T06:11:46.042077+0000","last_clean":"2023-11-24T06:11:46.042077+0000","last_became_active":"2023-11-24T06:11:45.062515+0000","last_became_peered":"2023-11-24T06:11:45.062515+0000","last_unstale":"2023-11-24T06:11:46.042077+0000","last_undegraded":"2023-11-24T06:11:46.042077+0000","last_fullsized":"2023-11-24T06:11:46.042077+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_clean_scrub_stamp":"2023-11-24T06:11:44.018686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:11:22.756651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:11:46.162595+0000","last_change":"2023-11-24T06:11:45.057850+0000","last_active":"2023-11-24T06:11:46.162595+0000","last_peered":"2023-11-24T06:11:46.162595+0000","last_clean":"2023-11-24T06:11:46.162595+0000","last_became_active":"2023-11-24T06:11:45.057542+0000","last_became_peered":"2023-11-24T06:11:45.057542+0000","last_unstale":"2023-11-24T06:11:46.162595+0000","last_undegraded":"2023-11-24T06:11:46.162595+0000","last_fullsized":"2023-11-24T06:11:46.162595+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_clean_scrub_stamp":"2023-11-24T06:11:44.018686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:40:45.042808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:11:46.163420+0000","last_change":"2023-11-24T06:11:45.061096+0000","last_active":"2023-11-24T06:11:46.163420+0000","last_peered":"2023-11-24T06:11:46.163420+0000","last_clean":"2023-11-24T06:11:46.163420+0000","last_became_active":"2023-11-24T06:11:45.059143+0000","last_became_peered":"2023-11-24T06:11:45.059143+0000","last_unstale":"2023-11-24T06:11:46.163420+0000","last_undegraded":"2023-11-24T06:11:46.163420+0000","last_fullsized":"2023-11-24T06:11:46.163420+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_clean_scrub_stamp":"2023-11-24T06:11:44.018686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:09:00.870428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:11:46.162583+0000","last_change":"2023-11-24T06:11:45.059120+0000","last_active":"2023-11-24T06:11:46.162583+0000","last_peered":"2023-11-24T06:11:46.162583+0000","last_clean":"2023-11-24T06:11:46.162583+0000","last_became_active":"2023-11-24T06:11:45.058237+0000","last_became_peered":"2023-11-24T06:11:45.058237+0000","last_unstale":"2023-11-24T06:11:46.162583+0000","last_undegraded":"2023-11-24T06:11:46.162583+0000","last_fullsized":"2023-11-24T06:11:46.162583+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_clean_scrub_stamp":"2023-11-24T06:11:44.018686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:15:02.243714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:11:46.044897+0000","last_change":"2023-11-24T06:11:45.070261+0000","last_active":"2023-11-24T06:11:46.044897+0000","last_peered":"2023-11-24T06:11:46.044897+0000","last_clean":"2023-11-24T06:11:46.044897+0000","last_became_active":"2023-11-24T06:11:45.069920+0000","last_became_peered":"2023-11-24T06:11:45.069920+0000","last_unstale":"2023-11-24T06:11:46.044897+0000","last_undegraded":"2023-11-24T06:11:46.044897+0000","last_fullsized":"2023-11-24T06:11:46.044897+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_clean_scrub_stamp":"2023-11-24T06:11:44.018686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:42:12.263089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:11:46.163614+0000","last_change":"2023-11-24T06:11:45.063665+0000","last_active":"2023-11-24T06:11:46.163614+0000","last_peered":"2023-11-24T06:11:46.163614+0000","last_clean":"2023-11-24T06:11:46.163614+0000","last_became_active":"2023-11-24T06:11:45.063365+0000","last_became_peered":"2023-11-24T06:11:45.063365+0000","last_unstale":"2023-11-24T06:11:46.163614+0000","last_undegraded":"2023-11-24T06:11:46.163614+0000","last_fullsized":"2023-11-24T06:11:46.163614+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_clean_scrub_stamp":"2023-11-24T06:11:44.018686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:31:16.199313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:11:46.164428+0000","last_change":"2023-11-24T06:11:45.067860+0000","last_active":"2023-11-24T06:11:46.164428+0000","last_peered":"2023-11-24T06:11:46.164428+0000","last_clean":"2023-11-24T06:11:46.164428+0000","last_became_active":"2023-11-24T06:11:45.067588+0000","last_became_peered":"2023-11-24T06:11:45.067588+0000","last_unstale":"2023-11-24T06:11:46.164428+0000","last_undegraded":"2023-11-24T06:11:46.164428+0000","last_fullsized":"2023-11-24T06:11:46.164428+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_clean_scrub_stamp":"2023-11-24T06:11:44.018686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:26:22.270872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:11:46.162513+0000","last_change":"2023-11-24T06:11:45.064149+0000","last_active":"2023-11-24T06:11:46.162513+0000","last_peered":"2023-11-24T06:11:46.162513+0000","last_clean":"2023-11-24T06:11:46.162513+0000","last_became_active":"2023-11-24T06:11:45.063599+0000","last_became_peered":"2023-11-24T06:11:45.063599+0000","last_unstale":"2023-11-24T06:11:46.162513+0000","last_undegraded":"2023-11-24T06:11:46.162513+0000","last_fullsized":"2023-11-24T06:11:46.162513+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_clean_scrub_stamp":"2023-11-24T06:11:44.018686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:19:03.083820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:11:46.042337+0000","last_change":"2023-11-24T06:11:45.055852+0000","last_active":"2023-11-24T06:11:46.042337+0000","last_peered":"2023-11-24T06:11:46.042337+0000","last_clean":"2023-11-24T06:11:46.042337+0000","last_became_active":"2023-11-24T06:11:45.055554+0000","last_became_peered":"2023-11-24T06:11:45.055554+0000","last_unstale":"2023-11-24T06:11:46.042337+0000","last_undegraded":"2023-11-24T06:11:46.042337+0000","last_fullsized":"2023-11-24T06:11:46.042337+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_clean_scrub_stamp":"2023-11-24T06:11:44.018686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:06:55.244746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:11:46.162778+0000","last_change":"2023-11-24T06:11:45.056844+0000","last_active":"2023-11-24T06:11:46.162778+0000","last_peered":"2023-11-24T06:11:46.162778+0000","last_clean":"2023-11-24T06:11:46.162778+0000","last_became_active":"2023-11-24T06:11:45.056431+0000","last_became_peered":"2023-11-24T06:11:45.056431+0000","last_unstale":"2023-11-24T06:11:46.162778+0000","last_undegraded":"2023-11-24T06:11:46.162778+0000","last_fullsized":"2023-11-24T06:11:46.162778+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_clean_scrub_stamp":"2023-11-24T06:11:44.018686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:34:37.780831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:11:46.162483+0000","last_change":"2023-11-24T06:11:45.054090+0000","last_active":"2023-11-24T06:11:46.162483+0000","last_peered":"2023-11-24T06:11:46.162483+0000","last_clean":"2023-11-24T06:11:46.162483+0000","last_became_active":"2023-11-24T06:11:45.053254+0000","last_became_peered":"2023-11-24T06:11:45.053254+0000","last_unstale":"2023-11-24T06:11:46.162483+0000","last_undegraded":"2023-11-24T06:11:46.162483+0000","last_fullsized":"2023-11-24T06:11:46.162483+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_clean_scrub_stamp":"2023-11-24T06:11:44.018686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:51:51.030233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:11:46.163168+0000","last_change":"2023-11-24T06:11:45.062778+0000","last_active":"2023-11-24T06:11:46.163168+0000","last_peered":"2023-11-24T06:11:46.163168+0000","last_clean":"2023-11-24T06:11:46.163168+0000","last_became_active":"2023-11-24T06:11:45.062448+0000","last_became_peered":"2023-11-24T06:11:45.062448+0000","last_unstale":"2023-11-24T06:11:46.163168+0000","last_undegraded":"2023-11-24T06:11:46.163168+0000","last_fullsized":"2023-11-24T06:11:46.163168+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_clean_scrub_stamp":"2023-11-24T06:11:44.018686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:34:37.780831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:11:46.165151+0000","last_change":"2023-11-24T06:11:45.062099+0000","last_active":"2023-11-24T06:11:46.165151+0000","last_peered":"2023-11-24T06:11:46.165151+0000","last_clean":"2023-11-24T06:11:46.165151+0000","last_became_active":"2023-11-24T06:11:45.061742+0000","last_became_peered":"2023-11-24T06:11:45.061742+0000","last_unstale":"2023-11-24T06:11:46.165151+0000","last_undegraded":"2023-11-24T06:11:46.165151+0000","last_fullsized":"2023-11-24T06:11:46.165151+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_clean_scrub_stamp":"2023-11-24T06:11:44.018686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:54:36.987896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:11:46.163642+0000","last_change":"2023-11-24T06:11:45.064668+0000","last_active":"2023-11-24T06:11:46.163642+0000","last_peered":"2023-11-24T06:11:46.163642+0000","last_clean":"2023-11-24T06:11:46.163642+0000","last_became_active":"2023-11-24T06:11:45.064170+0000","last_became_peered":"2023-11-24T06:11:45.064170+0000","last_unstale":"2023-11-24T06:11:46.163642+0000","last_undegraded":"2023-11-24T06:11:46.163642+0000","last_fullsized":"2023-11-24T06:11:46.163642+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_clean_scrub_stamp":"2023-11-24T06:11:44.018686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:26:22.270872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:11:46.163138+0000","last_change":"2023-11-24T06:11:45.066500+0000","last_active":"2023-11-24T06:11:46.163138+0000","last_peered":"2023-11-24T06:11:46.163138+0000","last_clean":"2023-11-24T06:11:46.163138+0000","last_became_active":"2023-11-24T06:11:45.065691+0000","last_became_peered":"2023-11-24T06:11:45.065691+0000","last_unstale":"2023-11-24T06:11:46.163138+0000","last_undegraded":"2023-11-24T06:11:46.163138+0000","last_fullsized":"2023-11-24T06:11:46.163138+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_clean_scrub_stamp":"2023-11-24T06:11:44.018686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:46:05.186588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:11:46.042709+0000","last_change":"2023-11-24T06:11:45.059123+0000","last_active":"2023-11-24T06:11:46.042709+0000","last_peered":"2023-11-24T06:11:46.042709+0000","last_clean":"2023-11-24T06:11:46.042709+0000","last_became_active":"2023-11-24T06:11:45.057405+0000","last_became_peered":"2023-11-24T06:11:45.057405+0000","last_unstale":"2023-11-24T06:11:46.042709+0000","last_undegraded":"2023-11-24T06:11:46.042709+0000","last_fullsized":"2023-11-24T06:11:46.042709+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_clean_scrub_stamp":"2023-11-24T06:11:44.018686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:19:27.730483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:11:46.043997+0000","last_change":"2023-11-24T06:11:45.064510+0000","last_active":"2023-11-24T06:11:46.043997+0000","last_peered":"2023-11-24T06:11:46.043997+0000","last_clean":"2023-11-24T06:11:46.043997+0000","last_became_active":"2023-11-24T06:11:45.063697+0000","last_became_peered":"2023-11-24T06:11:45.063697+0000","last_unstale":"2023-11-24T06:11:46.043997+0000","last_undegraded":"2023-11-24T06:11:46.043997+0000","last_fullsized":"2023-11-24T06:11:46.043997+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_clean_scrub_stamp":"2023-11-24T06:11:44.018686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:50:54.991874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:11:46.043537+0000","last_change":"2023-11-24T06:11:45.059816+0000","last_active":"2023-11-24T06:11:46.043537+0000","last_peered":"2023-11-24T06:11:46.043537+0000","last_clean":"2023-11-24T06:11:46.043537+0000","last_became_active":"2023-11-24T06:11:45.059557+0000","last_became_peered":"2023-11-24T06:11:45.059557+0000","last_unstale":"2023-11-24T06:11:46.043537+0000","last_undegraded":"2023-11-24T06:11:46.043537+0000","last_fullsized":"2023-11-24T06:11:46.043537+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_clean_scrub_stamp":"2023-11-24T06:11:44.018686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:32:48.890199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:11:46.159882+0000","last_change":"2023-11-24T06:11:45.061186+0000","last_active":"2023-11-24T06:11:46.159882+0000","last_peered":"2023-11-24T06:11:46.159882+0000","last_clean":"2023-11-24T06:11:46.159882+0000","last_became_active":"2023-11-24T06:11:45.059551+0000","last_became_peered":"2023-11-24T06:11:45.059551+0000","last_unstale":"2023-11-24T06:11:46.159882+0000","last_undegraded":"2023-11-24T06:11:46.159882+0000","last_fullsized":"2023-11-24T06:11:46.159882+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_clean_scrub_stamp":"2023-11-24T06:11:44.018686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:32:48.890199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:11:46.162187+0000","last_change":"2023-11-24T06:11:45.051196+0000","last_active":"2023-11-24T06:11:46.162187+0000","last_peered":"2023-11-24T06:11:46.162187+0000","last_clean":"2023-11-24T06:11:46.162187+0000","last_became_active":"2023-11-24T06:11:45.050781+0000","last_became_peered":"2023-11-24T06:11:45.050781+0000","last_unstale":"2023-11-24T06:11:46.162187+0000","last_undegraded":"2023-11-24T06:11:46.162187+0000","last_fullsized":"2023-11-24T06:11:46.162187+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_clean_scrub_stamp":"2023-11-24T06:11:44.018686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:55:17.339395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:11:46.163685+0000","last_change":"2023-11-24T06:11:45.062098+0000","last_active":"2023-11-24T06:11:46.163685+0000","last_peered":"2023-11-24T06:11:46.163685+0000","last_clean":"2023-11-24T06:11:46.163685+0000","last_became_active":"2023-11-24T06:11:45.061798+0000","last_became_peered":"2023-11-24T06:11:45.061798+0000","last_unstale":"2023-11-24T06:11:46.163685+0000","last_undegraded":"2023-11-24T06:11:46.163685+0000","last_fullsized":"2023-11-24T06:11:46.163685+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_clean_scrub_stamp":"2023-11-24T06:11:44.018686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:01:16.849532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:11:46.163766+0000","last_change":"2023-11-24T06:11:45.060932+0000","last_active":"2023-11-24T06:11:46.163766+0000","last_peered":"2023-11-24T06:11:46.163766+0000","last_clean":"2023-11-24T06:11:46.163766+0000","last_became_active":"2023-11-24T06:11:45.060427+0000","last_became_peered":"2023-11-24T06:11:45.060427+0000","last_unstale":"2023-11-24T06:11:46.163766+0000","last_undegraded":"2023-11-24T06:11:46.163766+0000","last_fullsized":"2023-11-24T06:11:46.163766+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_clean_scrub_stamp":"2023-11-24T06:11:44.018686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:54:36.987896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:11:46.163174+0000","last_change":"2023-11-24T06:11:45.071131+0000","last_active":"2023-11-24T06:11:46.163174+0000","last_peered":"2023-11-24T06:11:46.163174+0000","last_clean":"2023-11-24T06:11:46.163174+0000","last_became_active":"2023-11-24T06:11:45.070910+0000","last_became_peered":"2023-11-24T06:11:45.070910+0000","last_unstale":"2023-11-24T06:11:46.163174+0000","last_undegraded":"2023-11-24T06:11:46.163174+0000","last_fullsized":"2023-11-24T06:11:46.163174+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_clean_scrub_stamp":"2023-11-24T06:11:44.018686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:42:12.263089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:11:46.163263+0000","last_change":"2023-11-24T06:11:45.063976+0000","last_active":"2023-11-24T06:11:46.163263+0000","last_peered":"2023-11-24T06:11:46.163263+0000","last_clean":"2023-11-24T06:11:46.163263+0000","last_became_active":"2023-11-24T06:11:45.063251+0000","last_became_peered":"2023-11-24T06:11:45.063251+0000","last_unstale":"2023-11-24T06:11:46.163263+0000","last_undegraded":"2023-11-24T06:11:46.163263+0000","last_fullsized":"2023-11-24T06:11:46.163263+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_clean_scrub_stamp":"2023-11-24T06:11:44.018686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:40:45.042808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:11:46.044351+0000","last_change":"2023-11-24T06:11:45.068044+0000","last_active":"2023-11-24T06:11:46.044351+0000","last_peered":"2023-11-24T06:11:46.044351+0000","last_clean":"2023-11-24T06:11:46.044351+0000","last_became_active":"2023-11-24T06:11:45.061965+0000","last_became_peered":"2023-11-24T06:11:45.061965+0000","last_unstale":"2023-11-24T06:11:46.044351+0000","last_undegraded":"2023-11-24T06:11:46.044351+0000","last_fullsized":"2023-11-24T06:11:46.044351+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_clean_scrub_stamp":"2023-11-24T06:11:44.018686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:17:19.814894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:11:46.164000+0000","last_change":"2023-11-24T06:11:45.058528+0000","last_active":"2023-11-24T06:11:46.164000+0000","last_peered":"2023-11-24T06:11:46.164000+0000","last_clean":"2023-11-24T06:11:46.164000+0000","last_became_active":"2023-11-24T06:11:45.057303+0000","last_became_peered":"2023-11-24T06:11:45.057303+0000","last_unstale":"2023-11-24T06:11:46.164000+0000","last_undegraded":"2023-11-24T06:11:46.164000+0000","last_fullsized":"2023-11-24T06:11:46.164000+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_clean_scrub_stamp":"2023-11-24T06:11:44.018686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:46:05.186588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:11:46.043376+0000","last_change":"2023-11-24T06:11:45.052716+0000","last_active":"2023-11-24T06:11:46.043376+0000","last_peered":"2023-11-24T06:11:46.043376+0000","last_clean":"2023-11-24T06:11:46.043376+0000","last_became_active":"2023-11-24T06:11:45.052417+0000","last_became_peered":"2023-11-24T06:11:45.052417+0000","last_unstale":"2023-11-24T06:11:46.043376+0000","last_undegraded":"2023-11-24T06:11:46.043376+0000","last_fullsized":"2023-11-24T06:11:46.043376+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_clean_scrub_stamp":"2023-11-24T06:11:44.018686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:24:27.017978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:11:46.162563+0000","last_change":"2023-11-24T06:11:45.071582+0000","last_active":"2023-11-24T06:11:46.162563+0000","last_peered":"2023-11-24T06:11:46.162563+0000","last_clean":"2023-11-24T06:11:46.162563+0000","last_became_active":"2023-11-24T06:11:45.071034+0000","last_became_peered":"2023-11-24T06:11:45.071034+0000","last_unstale":"2023-11-24T06:11:46.162563+0000","last_undegraded":"2023-11-24T06:11:46.162563+0000","last_fullsized":"2023-11-24T06:11:46.162563+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_clean_scrub_stamp":"2023-11-24T06:11:44.018686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:19:03.083820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:11:46.160046+0000","last_change":"2023-11-24T06:11:45.059450+0000","last_active":"2023-11-24T06:11:46.160046+0000","last_peered":"2023-11-24T06:11:46.160046+0000","last_clean":"2023-11-24T06:11:46.160046+0000","last_became_active":"2023-11-24T06:11:45.058794+0000","last_became_peered":"2023-11-24T06:11:45.058794+0000","last_unstale":"2023-11-24T06:11:46.160046+0000","last_undegraded":"2023-11-24T06:11:46.160046+0000","last_fullsized":"2023-11-24T06:11:46.160046+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_clean_scrub_stamp":"2023-11-24T06:11:44.018686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:13:39.376575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"32'4","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:11:46.044233+0000","last_change":"2023-11-24T06:11:44.064796+0000","last_active":"2023-11-24T06:11:46.044233+0000","last_peered":"2023-11-24T06:11:46.044233+0000","last_clean":"2023-11-24T06:11:46.044233+0000","last_became_active":"2023-11-24T06:11:44.064532+0000","last_became_peered":"2023-11-24T06:11:44.064532+0000","last_unstale":"2023-11-24T06:11:46.044233+0000","last_undegraded":"2023-11-24T06:11:46.044233+0000","last_fullsized":"2023-11-24T06:11:46.044233+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:43.003933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:43.003933+0000","last_clean_scrub_stamp":"2023-11-24T06:11:43.003933+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:10:24.858886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:11:46.044083+0000","last_change":"2023-11-24T06:11:45.062393+0000","last_active":"2023-11-24T06:11:46.044083+0000","last_peered":"2023-11-24T06:11:46.044083+0000","last_clean":"2023-11-24T06:11:46.044083+0000","last_became_active":"2023-11-24T06:11:45.060876+0000","last_became_peered":"2023-11-24T06:11:45.060876+0000","last_unstale":"2023-11-24T06:11:46.044083+0000","last_undegraded":"2023-11-24T06:11:46.044083+0000","last_fullsized":"2023-11-24T06:11:46.044083+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_clean_scrub_stamp":"2023-11-24T06:11:44.018686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:50:54.991874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"32'1","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:11:46.160502+0000","last_change":"2023-11-24T06:11:44.069817+0000","last_active":"2023-11-24T06:11:46.160502+0000","last_peered":"2023-11-24T06:11:46.160502+0000","last_clean":"2023-11-24T06:11:46.160502+0000","last_became_active":"2023-11-24T06:11:44.068458+0000","last_became_peered":"2023-11-24T06:11:44.068458+0000","last_unstale":"2023-11-24T06:11:46.160502+0000","last_undegraded":"2023-11-24T06:11:46.160502+0000","last_fullsized":"2023-11-24T06:11:46.160502+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:43.003933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:43.003933+0000","last_clean_scrub_stamp":"2023-11-24T06:11:43.003933+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:10:24.858886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:11:46.043594+0000","last_change":"2023-11-24T06:11:45.061725+0000","last_active":"2023-11-24T06:11:46.043594+0000","last_peered":"2023-11-24T06:11:46.043594+0000","last_clean":"2023-11-24T06:11:46.043594+0000","last_became_active":"2023-11-24T06:11:45.061352+0000","last_became_peered":"2023-11-24T06:11:45.061352+0000","last_unstale":"2023-11-24T06:11:46.043594+0000","last_undegraded":"2023-11-24T06:11:46.043594+0000","last_fullsized":"2023-11-24T06:11:46.043594+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_clean_scrub_stamp":"2023-11-24T06:11:44.018686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:13:39.376575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"32'2","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:11:46.163402+0000","last_change":"2023-11-24T06:11:44.067226+0000","last_active":"2023-11-24T06:11:46.163402+0000","last_peered":"2023-11-24T06:11:46.163402+0000","last_clean":"2023-11-24T06:11:46.163402+0000","last_became_active":"2023-11-24T06:11:44.066968+0000","last_became_peered":"2023-11-24T06:11:44.066968+0000","last_unstale":"2023-11-24T06:11:46.163402+0000","last_undegraded":"2023-11-24T06:11:46.163402+0000","last_fullsized":"2023-11-24T06:11:46.163402+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:43.003933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:43.003933+0000","last_clean_scrub_stamp":"2023-11-24T06:11:43.003933+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:51:50.015479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:11:46.163325+0000","last_change":"2023-11-24T06:11:45.071343+0000","last_active":"2023-11-24T06:11:46.163325+0000","last_peered":"2023-11-24T06:11:46.163325+0000","last_clean":"2023-11-24T06:11:46.163325+0000","last_became_active":"2023-11-24T06:11:45.071017+0000","last_became_peered":"2023-11-24T06:11:45.071017+0000","last_unstale":"2023-11-24T06:11:46.163325+0000","last_undegraded":"2023-11-24T06:11:46.163325+0000","last_fullsized":"2023-11-24T06:11:46.163325+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_clean_scrub_stamp":"2023-11-24T06:11:44.018686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:44:01.946330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"32'2","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:11:46.042584+0000","last_change":"2023-11-24T06:11:44.060544+0000","last_active":"2023-11-24T06:11:46.042584+0000","last_peered":"2023-11-24T06:11:46.042584+0000","last_clean":"2023-11-24T06:11:46.042584+0000","last_became_active":"2023-11-24T06:11:44.060231+0000","last_became_peered":"2023-11-24T06:11:44.060231+0000","last_unstale":"2023-11-24T06:11:46.042584+0000","last_undegraded":"2023-11-24T06:11:46.042584+0000","last_fullsized":"2023-11-24T06:11:46.042584+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:43.003933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:43.003933+0000","last_clean_scrub_stamp":"2023-11-24T06:11:43.003933+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:31:27.908252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:11:46.164305+0000","last_change":"2023-11-24T06:11:45.066633+0000","last_active":"2023-11-24T06:11:46.164305+0000","last_peered":"2023-11-24T06:11:46.164305+0000","last_clean":"2023-11-24T06:11:46.164305+0000","last_became_active":"2023-11-24T06:11:45.066306+0000","last_became_peered":"2023-11-24T06:11:45.066306+0000","last_unstale":"2023-11-24T06:11:46.164305+0000","last_undegraded":"2023-11-24T06:11:46.164305+0000","last_fullsized":"2023-11-24T06:11:46.164305+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_clean_scrub_stamp":"2023-11-24T06:11:44.018686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:16:44.582224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"32'1","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:11:46.044662+0000","last_change":"2023-11-24T06:11:44.067069+0000","last_active":"2023-11-24T06:11:46.044662+0000","last_peered":"2023-11-24T06:11:46.044662+0000","last_clean":"2023-11-24T06:11:46.044662+0000","last_became_active":"2023-11-24T06:11:44.066806+0000","last_became_peered":"2023-11-24T06:11:44.066806+0000","last_unstale":"2023-11-24T06:11:46.044662+0000","last_undegraded":"2023-11-24T06:11:46.044662+0000","last_fullsized":"2023-11-24T06:11:46.044662+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:43.003933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:43.003933+0000","last_clean_scrub_stamp":"2023-11-24T06:11:43.003933+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:18:36.729170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:11:46.162590+0000","last_change":"2023-11-24T06:11:45.065318+0000","last_active":"2023-11-24T06:11:46.162590+0000","last_peered":"2023-11-24T06:11:46.162590+0000","last_clean":"2023-11-24T06:11:46.162590+0000","last_became_active":"2023-11-24T06:11:45.064999+0000","last_became_peered":"2023-11-24T06:11:45.064999+0000","last_unstale":"2023-11-24T06:11:46.162590+0000","last_undegraded":"2023-11-24T06:11:46.162590+0000","last_fullsized":"2023-11-24T06:11:46.162590+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_clean_scrub_stamp":"2023-11-24T06:11:44.018686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:15:02.243714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"32'1","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:11:46.043768+0000","last_change":"2023-11-24T06:11:44.067797+0000","last_active":"2023-11-24T06:11:46.043768+0000","last_peered":"2023-11-24T06:11:46.043768+0000","last_clean":"2023-11-24T06:11:46.043768+0000","last_became_active":"2023-11-24T06:11:44.067418+0000","last_became_peered":"2023-11-24T06:11:44.067418+0000","last_unstale":"2023-11-24T06:11:46.043768+0000","last_undegraded":"2023-11-24T06:11:46.043768+0000","last_fullsized":"2023-11-24T06:11:46.043768+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:43.003933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:43.003933+0000","last_clean_scrub_stamp":"2023-11-24T06:11:43.003933+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:52:00.088217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:11:46.042963+0000","last_change":"2023-11-24T06:11:45.068363+0000","last_active":"2023-11-24T06:11:46.042963+0000","last_peered":"2023-11-24T06:11:46.042963+0000","last_clean":"2023-11-24T06:11:46.042963+0000","last_became_active":"2023-11-24T06:11:45.058507+0000","last_became_peered":"2023-11-24T06:11:45.058507+0000","last_unstale":"2023-11-24T06:11:46.042963+0000","last_undegraded":"2023-11-24T06:11:46.042963+0000","last_fullsized":"2023-11-24T06:11:46.042963+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_clean_scrub_stamp":"2023-11-24T06:11:44.018686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:56:05.022334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:11:46.163969+0000","last_change":"2023-11-24T06:11:44.067156+0000","last_active":"2023-11-24T06:11:46.163969+0000","last_peered":"2023-11-24T06:11:46.163969+0000","last_clean":"2023-11-24T06:11:46.163969+0000","last_became_active":"2023-11-24T06:11:44.066595+0000","last_became_peered":"2023-11-24T06:11:44.066595+0000","last_unstale":"2023-11-24T06:11:46.163969+0000","last_undegraded":"2023-11-24T06:11:46.163969+0000","last_fullsized":"2023-11-24T06:11:46.163969+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:43.003933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:43.003933+0000","last_clean_scrub_stamp":"2023-11-24T06:11:43.003933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:44:00.931577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:11:46.164811+0000","last_change":"2023-11-24T06:11:45.059416+0000","last_active":"2023-11-24T06:11:46.164811+0000","last_peered":"2023-11-24T06:11:46.164811+0000","last_clean":"2023-11-24T06:11:46.164811+0000","last_became_active":"2023-11-24T06:11:45.059083+0000","last_became_peered":"2023-11-24T06:11:45.059083+0000","last_unstale":"2023-11-24T06:11:46.164811+0000","last_undegraded":"2023-11-24T06:11:46.164811+0000","last_fullsized":"2023-11-24T06:11:46.164811+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_clean_scrub_stamp":"2023-11-24T06:11:44.018686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:17:19.814894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:11:46.043172+0000","last_change":"2023-11-24T06:11:44.068295+0000","last_active":"2023-11-24T06:11:46.043172+0000","last_peered":"2023-11-24T06:11:46.043172+0000","last_clean":"2023-11-24T06:11:46.043172+0000","last_became_active":"2023-11-24T06:11:44.066015+0000","last_became_peered":"2023-11-24T06:11:44.066015+0000","last_unstale":"2023-11-24T06:11:46.043172+0000","last_undegraded":"2023-11-24T06:11:46.043172+0000","last_fullsized":"2023-11-24T06:11:46.043172+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:43.003933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:43.003933+0000","last_clean_scrub_stamp":"2023-11-24T06:11:43.003933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:19:26.715729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:11:46.164371+0000","last_change":"2023-11-24T06:11:45.065842+0000","last_active":"2023-11-24T06:11:46.164371+0000","last_peered":"2023-11-24T06:11:46.164371+0000","last_clean":"2023-11-24T06:11:46.164371+0000","last_became_active":"2023-11-24T06:11:45.065524+0000","last_became_peered":"2023-11-24T06:11:45.065524+0000","last_unstale":"2023-11-24T06:11:46.164371+0000","last_undegraded":"2023-11-24T06:11:46.164371+0000","last_fullsized":"2023-11-24T06:11:46.164371+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_clean_scrub_stamp":"2023-11-24T06:11:44.018686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:34:37.780831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:11:46.164450+0000","last_change":"2023-11-24T06:11:44.066427+0000","last_active":"2023-11-24T06:11:46.164450+0000","last_peered":"2023-11-24T06:11:46.164450+0000","last_clean":"2023-11-24T06:11:46.164450+0000","last_became_active":"2023-11-24T06:11:44.066107+0000","last_became_peered":"2023-11-24T06:11:44.066107+0000","last_unstale":"2023-11-24T06:11:46.164450+0000","last_undegraded":"2023-11-24T06:11:46.164450+0000","last_fullsized":"2023-11-24T06:11:46.164450+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:43.003933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:43.003933+0000","last_clean_scrub_stamp":"2023-11-24T06:11:43.003933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:15:01.228961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:11:46.164747+0000","last_change":"2023-11-24T06:11:45.069665+0000","last_active":"2023-11-24T06:11:46.164747+0000","last_peered":"2023-11-24T06:11:46.164747+0000","last_clean":"2023-11-24T06:11:46.164747+0000","last_became_active":"2023-11-24T06:11:45.069330+0000","last_became_peered":"2023-11-24T06:11:45.069330+0000","last_unstale":"2023-11-24T06:11:46.164747+0000","last_undegraded":"2023-11-24T06:11:46.164747+0000","last_fullsized":"2023-11-24T06:11:46.164747+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_clean_scrub_stamp":"2023-11-24T06:11:44.018686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:57:54.566169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:11:46.160547+0000","last_change":"2023-11-24T06:11:44.058412+0000","last_active":"2023-11-24T06:11:46.160547+0000","last_peered":"2023-11-24T06:11:46.160547+0000","last_clean":"2023-11-24T06:11:46.160547+0000","last_became_active":"2023-11-24T06:11:44.058065+0000","last_became_peered":"2023-11-24T06:11:44.058065+0000","last_unstale":"2023-11-24T06:11:46.160547+0000","last_undegraded":"2023-11-24T06:11:46.160547+0000","last_fullsized":"2023-11-24T06:11:46.160547+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:43.003933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:43.003933+0000","last_clean_scrub_stamp":"2023-11-24T06:11:43.003933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:02:25.402413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:11:46.163091+0000","last_change":"2023-11-24T06:11:45.058531+0000","last_active":"2023-11-24T06:11:46.163091+0000","last_peered":"2023-11-24T06:11:46.163091+0000","last_clean":"2023-11-24T06:11:46.163091+0000","last_became_active":"2023-11-24T06:11:45.058201+0000","last_became_peered":"2023-11-24T06:11:45.058201+0000","last_unstale":"2023-11-24T06:11:46.163091+0000","last_undegraded":"2023-11-24T06:11:46.163091+0000","last_fullsized":"2023-11-24T06:11:46.163091+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_clean_scrub_stamp":"2023-11-24T06:11:44.018686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:09:00.870428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"32'3","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:11:46.163810+0000","last_change":"2023-11-24T06:11:44.066151+0000","last_active":"2023-11-24T06:11:46.163810+0000","last_peered":"2023-11-24T06:11:46.163810+0000","last_clean":"2023-11-24T06:11:46.163810+0000","last_became_active":"2023-11-24T06:11:44.065797+0000","last_became_peered":"2023-11-24T06:11:44.065797+0000","last_unstale":"2023-11-24T06:11:46.163810+0000","last_undegraded":"2023-11-24T06:11:46.163810+0000","last_fullsized":"2023-11-24T06:11:46.163810+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:43.003933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:43.003933+0000","last_clean_scrub_stamp":"2023-11-24T06:11:43.003933+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:44:00.931577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:11:46.164528+0000","last_change":"2023-11-24T06:11:45.067459+0000","last_active":"2023-11-24T06:11:46.164528+0000","last_peered":"2023-11-24T06:11:46.164528+0000","last_clean":"2023-11-24T06:11:46.164528+0000","last_became_active":"2023-11-24T06:11:45.066804+0000","last_became_peered":"2023-11-24T06:11:45.066804+0000","last_unstale":"2023-11-24T06:11:46.164528+0000","last_undegraded":"2023-11-24T06:11:46.164528+0000","last_fullsized":"2023-11-24T06:11:46.164528+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_clean_scrub_stamp":"2023-11-24T06:11:44.018686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:09:00.870428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"32'6","reported_seq":18,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:11:46.163186+0000","last_change":"2023-11-24T06:11:44.068681+0000","last_active":"2023-11-24T06:11:46.163186+0000","last_peered":"2023-11-24T06:11:46.163186+0000","last_clean":"2023-11-24T06:11:46.163186+0000","last_became_active":"2023-11-24T06:11:44.068403+0000","last_became_peered":"2023-11-24T06:11:44.068403+0000","last_unstale":"2023-11-24T06:11:46.163186+0000","last_undegraded":"2023-11-24T06:11:46.163186+0000","last_fullsized":"2023-11-24T06:11:46.163186+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:43.003933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:43.003933+0000","last_clean_scrub_stamp":"2023-11-24T06:11:43.003933+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:19:02.069067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:11:46.164901+0000","last_change":"2023-11-24T06:11:45.066579+0000","last_active":"2023-11-24T06:11:46.164901+0000","last_peered":"2023-11-24T06:11:46.164901+0000","last_clean":"2023-11-24T06:11:46.164901+0000","last_became_active":"2023-11-24T06:11:45.066011+0000","last_became_peered":"2023-11-24T06:11:45.066011+0000","last_unstale":"2023-11-24T06:11:46.164901+0000","last_undegraded":"2023-11-24T06:11:46.164901+0000","last_fullsized":"2023-11-24T06:11:46.164901+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_clean_scrub_stamp":"2023-11-24T06:11:44.018686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:19:03.083820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"32'2","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:11:46.044719+0000","last_change":"2023-11-24T06:11:44.065844+0000","last_active":"2023-11-24T06:11:46.044719+0000","last_peered":"2023-11-24T06:11:46.044719+0000","last_clean":"2023-11-24T06:11:46.044719+0000","last_became_active":"2023-11-24T06:11:44.065561+0000","last_became_peered":"2023-11-24T06:11:44.065561+0000","last_unstale":"2023-11-24T06:11:46.044719+0000","last_undegraded":"2023-11-24T06:11:46.044719+0000","last_fullsized":"2023-11-24T06:11:46.044719+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:43.003933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:43.003933+0000","last_clean_scrub_stamp":"2023-11-24T06:11:43.003933+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:19:53.395242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:11:46.163370+0000","last_change":"2023-11-24T06:11:45.060959+0000","last_active":"2023-11-24T06:11:46.163370+0000","last_peered":"2023-11-24T06:11:46.163370+0000","last_clean":"2023-11-24T06:11:46.163370+0000","last_became_active":"2023-11-24T06:11:45.059011+0000","last_became_peered":"2023-11-24T06:11:45.059011+0000","last_unstale":"2023-11-24T06:11:46.163370+0000","last_undegraded":"2023-11-24T06:11:46.163370+0000","last_fullsized":"2023-11-24T06:11:46.163370+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_clean_scrub_stamp":"2023-11-24T06:11:44.018686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:29:49.006871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:11:46.164048+0000","last_change":"2023-11-24T06:11:44.064612+0000","last_active":"2023-11-24T06:11:46.164048+0000","last_peered":"2023-11-24T06:11:46.164048+0000","last_clean":"2023-11-24T06:11:46.164048+0000","last_became_active":"2023-11-24T06:11:44.064309+0000","last_became_peered":"2023-11-24T06:11:44.064309+0000","last_unstale":"2023-11-24T06:11:46.164048+0000","last_undegraded":"2023-11-24T06:11:46.164048+0000","last_fullsized":"2023-11-24T06:11:46.164048+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:43.003933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:43.003933+0000","last_clean_scrub_stamp":"2023-11-24T06:11:43.003933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:52:24.803556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:11:46.162799+0000","last_change":"2023-11-24T06:11:45.071836+0000","last_active":"2023-11-24T06:11:46.162799+0000","last_peered":"2023-11-24T06:11:46.162799+0000","last_clean":"2023-11-24T06:11:46.162799+0000","last_became_active":"2023-11-24T06:11:45.071012+0000","last_became_peered":"2023-11-24T06:11:45.071012+0000","last_unstale":"2023-11-24T06:11:46.162799+0000","last_undegraded":"2023-11-24T06:11:46.162799+0000","last_fullsized":"2023-11-24T06:11:46.162799+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_clean_scrub_stamp":"2023-11-24T06:11:44.018686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:09:22.132463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:11:46.163170+0000","last_change":"2023-11-24T06:11:44.067439+0000","last_active":"2023-11-24T06:11:46.163170+0000","last_peered":"2023-11-24T06:11:46.163170+0000","last_clean":"2023-11-24T06:11:46.163170+0000","last_became_active":"2023-11-24T06:11:44.065500+0000","last_became_peered":"2023-11-24T06:11:44.065500+0000","last_unstale":"2023-11-24T06:11:46.163170+0000","last_undegraded":"2023-11-24T06:11:46.163170+0000","last_fullsized":"2023-11-24T06:11:46.163170+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:43.003933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:43.003933+0000","last_clean_scrub_stamp":"2023-11-24T06:11:43.003933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:01:15.834779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:11:46.043578+0000","last_change":"2023-11-24T06:11:45.069211+0000","last_active":"2023-11-24T06:11:46.043578+0000","last_peered":"2023-11-24T06:11:46.043578+0000","last_clean":"2023-11-24T06:11:46.043578+0000","last_became_active":"2023-11-24T06:11:45.068912+0000","last_became_peered":"2023-11-24T06:11:45.068912+0000","last_unstale":"2023-11-24T06:11:46.043578+0000","last_undegraded":"2023-11-24T06:11:46.043578+0000","last_fullsized":"2023-11-24T06:11:46.043578+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_clean_scrub_stamp":"2023-11-24T06:11:44.018686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:56:05.022334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"32'8","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:11:46.162044+0000","last_change":"2023-11-24T06:11:44.069838+0000","last_active":"2023-11-24T06:11:46.162044+0000","last_peered":"2023-11-24T06:11:46.162044+0000","last_clean":"2023-11-24T06:11:46.162044+0000","last_became_active":"2023-11-24T06:11:44.069111+0000","last_became_peered":"2023-11-24T06:11:44.069111+0000","last_unstale":"2023-11-24T06:11:46.162044+0000","last_undegraded":"2023-11-24T06:11:46.162044+0000","last_fullsized":"2023-11-24T06:11:46.162044+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:43.003933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:43.003933+0000","last_clean_scrub_stamp":"2023-11-24T06:11:43.003933+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:31:15.184560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:11:46.044156+0000","last_change":"2023-11-24T06:11:45.064079+0000","last_active":"2023-11-24T06:11:46.044156+0000","last_peered":"2023-11-24T06:11:46.044156+0000","last_clean":"2023-11-24T06:11:46.044156+0000","last_became_active":"2023-11-24T06:11:45.063771+0000","last_became_peered":"2023-11-24T06:11:45.063771+0000","last_unstale":"2023-11-24T06:11:46.044156+0000","last_undegraded":"2023-11-24T06:11:46.044156+0000","last_fullsized":"2023-11-24T06:11:46.044156+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_clean_scrub_stamp":"2023-11-24T06:11:44.018686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:44:42.836131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"32'1","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:11:46.163815+0000","last_change":"2023-11-24T06:11:44.071090+0000","last_active":"2023-11-24T06:11:46.163815+0000","last_peered":"2023-11-24T06:11:46.163815+0000","last_clean":"2023-11-24T06:11:46.163815+0000","last_became_active":"2023-11-24T06:11:44.070760+0000","last_became_peered":"2023-11-24T06:11:44.070760+0000","last_unstale":"2023-11-24T06:11:46.163815+0000","last_undegraded":"2023-11-24T06:11:46.163815+0000","last_fullsized":"2023-11-24T06:11:46.163815+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:43.003933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:43.003933+0000","last_clean_scrub_stamp":"2023-11-24T06:11:43.003933+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:13:38.361822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:11:46.164890+0000","last_change":"2023-11-24T06:11:45.054126+0000","last_active":"2023-11-24T06:11:46.164890+0000","last_peered":"2023-11-24T06:11:46.164890+0000","last_clean":"2023-11-24T06:11:46.164890+0000","last_became_active":"2023-11-24T06:11:45.053804+0000","last_became_peered":"2023-11-24T06:11:45.053804+0000","last_unstale":"2023-11-24T06:11:46.164890+0000","last_undegraded":"2023-11-24T06:11:46.164890+0000","last_fullsized":"2023-11-24T06:11:46.164890+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_clean_scrub_stamp":"2023-11-24T06:11:44.018686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:46:05.186588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"32'2","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:11:46.044379+0000","last_change":"2023-11-24T06:11:44.068028+0000","last_active":"2023-11-24T06:11:46.044379+0000","last_peered":"2023-11-24T06:11:46.044379+0000","last_clean":"2023-11-24T06:11:46.044379+0000","last_became_active":"2023-11-24T06:11:44.067730+0000","last_became_peered":"2023-11-24T06:11:44.067730+0000","last_unstale":"2023-11-24T06:11:46.044379+0000","last_undegraded":"2023-11-24T06:11:46.044379+0000","last_fullsized":"2023-11-24T06:11:46.044379+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:43.003933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:43.003933+0000","last_clean_scrub_stamp":"2023-11-24T06:11:43.003933+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:09:08.543385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:11:46.164308+0000","last_change":"2023-11-24T06:11:45.070909+0000","last_active":"2023-11-24T06:11:46.164308+0000","last_peered":"2023-11-24T06:11:46.164308+0000","last_clean":"2023-11-24T06:11:46.164308+0000","last_became_active":"2023-11-24T06:11:45.070566+0000","last_became_peered":"2023-11-24T06:11:45.070566+0000","last_unstale":"2023-11-24T06:11:46.164308+0000","last_undegraded":"2023-11-24T06:11:46.164308+0000","last_fullsized":"2023-11-24T06:11:46.164308+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_clean_scrub_stamp":"2023-11-24T06:11:44.018686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:51:51.030233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"32'1","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:11:46.044581+0000","last_change":"2023-11-24T06:11:44.058740+0000","last_active":"2023-11-24T06:11:46.044581+0000","last_peered":"2023-11-24T06:11:46.044581+0000","last_clean":"2023-11-24T06:11:46.044581+0000","last_became_active":"2023-11-24T06:11:44.058415+0000","last_became_peered":"2023-11-24T06:11:44.058415+0000","last_unstale":"2023-11-24T06:11:46.044581+0000","last_undegraded":"2023-11-24T06:11:46.044581+0000","last_fullsized":"2023-11-24T06:11:46.044581+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:43.003933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:43.003933+0000","last_clean_scrub_stamp":"2023-11-24T06:11:43.003933+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:52:24.803556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:11:46.042883+0000","last_change":"2023-11-24T06:11:45.060673+0000","last_active":"2023-11-24T06:11:46.042883+0000","last_peered":"2023-11-24T06:11:46.042883+0000","last_clean":"2023-11-24T06:11:46.042883+0000","last_became_active":"2023-11-24T06:11:45.060391+0000","last_became_peered":"2023-11-24T06:11:45.060391+0000","last_unstale":"2023-11-24T06:11:46.042883+0000","last_undegraded":"2023-11-24T06:11:46.042883+0000","last_fullsized":"2023-11-24T06:11:46.042883+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_clean_scrub_stamp":"2023-11-24T06:11:44.018686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:36:31.999711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:11:46.043790+0000","last_change":"2023-11-24T06:11:44.068577+0000","last_active":"2023-11-24T06:11:46.043790+0000","last_peered":"2023-11-24T06:11:46.043790+0000","last_clean":"2023-11-24T06:11:46.043790+0000","last_became_active":"2023-11-24T06:11:44.067760+0000","last_became_peered":"2023-11-24T06:11:44.067760+0000","last_unstale":"2023-11-24T06:11:46.043790+0000","last_undegraded":"2023-11-24T06:11:46.043790+0000","last_fullsized":"2023-11-24T06:11:46.043790+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:43.003933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:43.003933+0000","last_clean_scrub_stamp":"2023-11-24T06:11:43.003933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:10:24.858886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:11:46.042046+0000","last_change":"2023-11-24T06:11:45.063538+0000","last_active":"2023-11-24T06:11:46.042046+0000","last_peered":"2023-11-24T06:11:46.042046+0000","last_clean":"2023-11-24T06:11:46.042046+0000","last_became_active":"2023-11-24T06:11:45.063262+0000","last_became_peered":"2023-11-24T06:11:45.063262+0000","last_unstale":"2023-11-24T06:11:46.042046+0000","last_undegraded":"2023-11-24T06:11:46.042046+0000","last_fullsized":"2023-11-24T06:11:46.042046+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_clean_scrub_stamp":"2023-11-24T06:11:44.018686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:31:19.555803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"32'2","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:11:46.160245+0000","last_change":"2023-11-24T06:11:44.062187+0000","last_active":"2023-11-24T06:11:46.160245+0000","last_peered":"2023-11-24T06:11:46.160245+0000","last_clean":"2023-11-24T06:11:46.160245+0000","last_became_active":"2023-11-24T06:11:44.061478+0000","last_became_peered":"2023-11-24T06:11:44.061478+0000","last_unstale":"2023-11-24T06:11:46.160245+0000","last_undegraded":"2023-11-24T06:11:46.160245+0000","last_fullsized":"2023-11-24T06:11:46.160245+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:43.003933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:43.003933+0000","last_clean_scrub_stamp":"2023-11-24T06:11:43.003933+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:24:26.003225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:11:46.042211+0000","last_change":"2023-11-24T06:11:45.065311+0000","last_active":"2023-11-24T06:11:46.042211+0000","last_peered":"2023-11-24T06:11:46.042211+0000","last_clean":"2023-11-24T06:11:46.042211+0000","last_became_active":"2023-11-24T06:11:45.064227+0000","last_became_peered":"2023-11-24T06:11:45.064227+0000","last_unstale":"2023-11-24T06:11:46.042211+0000","last_undegraded":"2023-11-24T06:11:46.042211+0000","last_fullsized":"2023-11-24T06:11:46.042211+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_clean_scrub_stamp":"2023-11-24T06:11:44.018686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:17:04.162486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:11:46.043816+0000","last_change":"2023-11-24T06:11:44.063640+0000","last_active":"2023-11-24T06:11:46.043816+0000","last_peered":"2023-11-24T06:11:46.043816+0000","last_clean":"2023-11-24T06:11:46.043816+0000","last_became_active":"2023-11-24T06:11:44.063346+0000","last_became_peered":"2023-11-24T06:11:44.063346+0000","last_unstale":"2023-11-24T06:11:46.043816+0000","last_undegraded":"2023-11-24T06:11:46.043816+0000","last_fullsized":"2023-11-24T06:11:46.043816+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:43.003933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:43.003933+0000","last_clean_scrub_stamp":"2023-11-24T06:11:43.003933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:09:08.543385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:11:46.041827+0000","last_change":"2023-11-24T06:11:45.061275+0000","last_active":"2023-11-24T06:11:46.041827+0000","last_peered":"2023-11-24T06:11:46.041827+0000","last_clean":"2023-11-24T06:11:46.041827+0000","last_became_active":"2023-11-24T06:11:45.060990+0000","last_became_peered":"2023-11-24T06:11:45.060990+0000","last_unstale":"2023-11-24T06:11:46.041827+0000","last_undegraded":"2023-11-24T06:11:46.041827+0000","last_fullsized":"2023-11-24T06:11:46.041827+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_clean_scrub_stamp":"2023-11-24T06:11:44.018686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:33:25.977675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:11:46.042860+0000","last_change":"2023-11-24T06:11:44.062996+0000","last_active":"2023-11-24T06:11:46.042860+0000","last_peered":"2023-11-24T06:11:46.042860+0000","last_clean":"2023-11-24T06:11:46.042860+0000","last_became_active":"2023-11-24T06:11:44.062606+0000","last_became_peered":"2023-11-24T06:11:44.062606+0000","last_unstale":"2023-11-24T06:11:46.042860+0000","last_undegraded":"2023-11-24T06:11:46.042860+0000","last_fullsized":"2023-11-24T06:11:46.042860+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:43.003933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:43.003933+0000","last_clean_scrub_stamp":"2023-11-24T06:11:43.003933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:36:30.984958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:11:46.163074+0000","last_change":"2023-11-24T06:11:45.060500+0000","last_active":"2023-11-24T06:11:46.163074+0000","last_peered":"2023-11-24T06:11:46.163074+0000","last_clean":"2023-11-24T06:11:46.163074+0000","last_became_active":"2023-11-24T06:11:45.060206+0000","last_became_peered":"2023-11-24T06:11:45.060206+0000","last_unstale":"2023-11-24T06:11:46.163074+0000","last_undegraded":"2023-11-24T06:11:46.163074+0000","last_fullsized":"2023-11-24T06:11:46.163074+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_clean_scrub_stamp":"2023-11-24T06:11:44.018686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:40:45.042808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:11:46.162035+0000","last_change":"2023-11-24T06:11:44.066907+0000","last_active":"2023-11-24T06:11:46.162035+0000","last_peered":"2023-11-24T06:11:46.162035+0000","last_clean":"2023-11-24T06:11:46.162035+0000","last_became_active":"2023-11-24T06:11:44.066171+0000","last_became_peered":"2023-11-24T06:11:44.066171+0000","last_unstale":"2023-11-24T06:11:46.162035+0000","last_undegraded":"2023-11-24T06:11:46.162035+0000","last_fullsized":"2023-11-24T06:11:46.162035+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:43.003933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:43.003933+0000","last_clean_scrub_stamp":"2023-11-24T06:11:43.003933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:02:25.402413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:11:46.042829+0000","last_change":"2023-11-24T06:11:45.058858+0000","last_active":"2023-11-24T06:11:46.042829+0000","last_peered":"2023-11-24T06:11:46.042829+0000","last_clean":"2023-11-24T06:11:46.042829+0000","last_became_active":"2023-11-24T06:11:45.057504+0000","last_became_peered":"2023-11-24T06:11:45.057504+0000","last_unstale":"2023-11-24T06:11:46.042829+0000","last_undegraded":"2023-11-24T06:11:46.042829+0000","last_fullsized":"2023-11-24T06:11:46.042829+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_clean_scrub_stamp":"2023-11-24T06:11:44.018686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:52:01.102970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"32'2","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:11:46.163909+0000","last_change":"2023-11-24T06:11:44.068959+0000","last_active":"2023-11-24T06:11:46.163909+0000","last_peered":"2023-11-24T06:11:46.163909+0000","last_clean":"2023-11-24T06:11:46.163909+0000","last_became_active":"2023-11-24T06:11:44.068639+0000","last_became_peered":"2023-11-24T06:11:44.068639+0000","last_unstale":"2023-11-24T06:11:46.163909+0000","last_undegraded":"2023-11-24T06:11:46.163909+0000","last_fullsized":"2023-11-24T06:11:46.163909+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:43.003933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:43.003933+0000","last_clean_scrub_stamp":"2023-11-24T06:11:43.003933+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:50:53.977121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:11:46.164938+0000","last_change":"2023-11-24T06:11:45.067368+0000","last_active":"2023-11-24T06:11:46.164938+0000","last_peered":"2023-11-24T06:11:46.164938+0000","last_clean":"2023-11-24T06:11:46.164938+0000","last_became_active":"2023-11-24T06:11:45.067027+0000","last_became_peered":"2023-11-24T06:11:45.067027+0000","last_unstale":"2023-11-24T06:11:46.164938+0000","last_undegraded":"2023-11-24T06:11:46.164938+0000","last_fullsized":"2023-11-24T06:11:46.164938+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_clean_scrub_stamp":"2023-11-24T06:11:44.018686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:29:49.006871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:11:46.160498+0000","last_change":"2023-11-24T06:11:44.060007+0000","last_active":"2023-11-24T06:11:46.160498+0000","last_peered":"2023-11-24T06:11:46.160498+0000","last_clean":"2023-11-24T06:11:46.160498+0000","last_became_active":"2023-11-24T06:11:44.059721+0000","last_became_peered":"2023-11-24T06:11:44.059721+0000","last_unstale":"2023-11-24T06:11:46.160498+0000","last_undegraded":"2023-11-24T06:11:46.160498+0000","last_fullsized":"2023-11-24T06:11:46.160498+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:43.003933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:43.003933+0000","last_clean_scrub_stamp":"2023-11-24T06:11:43.003933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:52:24.803556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:11:46.160164+0000","last_change":"2023-11-24T06:11:45.066908+0000","last_active":"2023-11-24T06:11:46.160164+0000","last_peered":"2023-11-24T06:11:46.160164+0000","last_clean":"2023-11-24T06:11:46.160164+0000","last_became_active":"2023-11-24T06:11:45.066554+0000","last_became_peered":"2023-11-24T06:11:45.066554+0000","last_unstale":"2023-11-24T06:11:46.160164+0000","last_undegraded":"2023-11-24T06:11:46.160164+0000","last_fullsized":"2023-11-24T06:11:46.160164+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_clean_scrub_stamp":"2023-11-24T06:11:44.018686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:50:54.991874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:11:46.164459+0000","last_change":"2023-11-24T06:11:44.063030+0000","last_active":"2023-11-24T06:11:46.164459+0000","last_peered":"2023-11-24T06:11:46.164459+0000","last_clean":"2023-11-24T06:11:46.164459+0000","last_became_active":"2023-11-24T06:11:44.061414+0000","last_became_peered":"2023-11-24T06:11:44.061414+0000","last_unstale":"2023-11-24T06:11:46.164459+0000","last_undegraded":"2023-11-24T06:11:46.164459+0000","last_fullsized":"2023-11-24T06:11:46.164459+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:43.003933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:43.003933+0000","last_clean_scrub_stamp":"2023-11-24T06:11:43.003933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:32:47.875446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:11:46.043578+0000","last_change":"2023-11-24T06:11:45.058989+0000","last_active":"2023-11-24T06:11:46.043578+0000","last_peered":"2023-11-24T06:11:46.043578+0000","last_clean":"2023-11-24T06:11:46.043578+0000","last_became_active":"2023-11-24T06:11:45.058674+0000","last_became_peered":"2023-11-24T06:11:45.058674+0000","last_unstale":"2023-11-24T06:11:46.043578+0000","last_undegraded":"2023-11-24T06:11:46.043578+0000","last_fullsized":"2023-11-24T06:11:46.043578+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_clean_scrub_stamp":"2023-11-24T06:11:44.018686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:32:48.890199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:11:46.160346+0000","last_change":"2023-11-24T06:11:44.062364+0000","last_active":"2023-11-24T06:11:46.160346+0000","last_peered":"2023-11-24T06:11:46.160346+0000","last_clean":"2023-11-24T06:11:46.160346+0000","last_became_active":"2023-11-24T06:11:44.061926+0000","last_became_peered":"2023-11-24T06:11:44.061926+0000","last_unstale":"2023-11-24T06:11:46.160346+0000","last_undegraded":"2023-11-24T06:11:46.160346+0000","last_fullsized":"2023-11-24T06:11:46.160346+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:43.003933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:43.003933+0000","last_clean_scrub_stamp":"2023-11-24T06:11:43.003933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:56:04.007581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:11:46.160149+0000","last_change":"2023-11-24T06:11:45.060034+0000","last_active":"2023-11-24T06:11:46.160149+0000","last_peered":"2023-11-24T06:11:46.160149+0000","last_clean":"2023-11-24T06:11:46.160149+0000","last_became_active":"2023-11-24T06:11:45.058485+0000","last_became_peered":"2023-11-24T06:11:45.058485+0000","last_unstale":"2023-11-24T06:11:46.160149+0000","last_undegraded":"2023-11-24T06:11:46.160149+0000","last_fullsized":"2023-11-24T06:11:46.160149+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_clean_scrub_stamp":"2023-11-24T06:11:44.018686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:44:42.836131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:11:46.164282+0000","last_change":"2023-11-24T06:11:44.070989+0000","last_active":"2023-11-24T06:11:46.164282+0000","last_peered":"2023-11-24T06:11:46.164282+0000","last_clean":"2023-11-24T06:11:46.164282+0000","last_became_active":"2023-11-24T06:11:44.070701+0000","last_became_peered":"2023-11-24T06:11:44.070701+0000","last_unstale":"2023-11-24T06:11:46.164282+0000","last_undegraded":"2023-11-24T06:11:46.164282+0000","last_fullsized":"2023-11-24T06:11:46.164282+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:43.003933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:43.003933+0000","last_clean_scrub_stamp":"2023-11-24T06:11:43.003933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:10:24.858886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:11:46.162057+0000","last_change":"2023-11-24T06:11:45.062075+0000","last_active":"2023-11-24T06:11:46.162057+0000","last_peered":"2023-11-24T06:11:46.162057+0000","last_clean":"2023-11-24T06:11:46.162057+0000","last_became_active":"2023-11-24T06:11:45.061638+0000","last_became_peered":"2023-11-24T06:11:45.061638+0000","last_unstale":"2023-11-24T06:11:46.162057+0000","last_undegraded":"2023-11-24T06:11:46.162057+0000","last_fullsized":"2023-11-24T06:11:46.162057+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_clean_scrub_stamp":"2023-11-24T06:11:44.018686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:09:22.132463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:11:46.043119+0000","last_change":"2023-11-24T06:11:44.063839+0000","last_active":"2023-11-24T06:11:46.043119+0000","last_peered":"2023-11-24T06:11:46.043119+0000","last_clean":"2023-11-24T06:11:46.043119+0000","last_became_active":"2023-11-24T06:11:44.063573+0000","last_became_peered":"2023-11-24T06:11:44.063573+0000","last_unstale":"2023-11-24T06:11:46.043119+0000","last_undegraded":"2023-11-24T06:11:46.043119+0000","last_fullsized":"2023-11-24T06:11:46.043119+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:43.003933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:43.003933+0000","last_clean_scrub_stamp":"2023-11-24T06:11:43.003933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:52:52.987858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:11:46.042867+0000","last_change":"2023-11-24T06:11:45.065154+0000","last_active":"2023-11-24T06:11:46.042867+0000","last_peered":"2023-11-24T06:11:46.042867+0000","last_clean":"2023-11-24T06:11:46.042867+0000","last_became_active":"2023-11-24T06:11:45.064868+0000","last_became_peered":"2023-11-24T06:11:45.064868+0000","last_unstale":"2023-11-24T06:11:46.042867+0000","last_undegraded":"2023-11-24T06:11:46.042867+0000","last_fullsized":"2023-11-24T06:11:46.042867+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_clean_scrub_stamp":"2023-11-24T06:11:44.018686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:24:27.017978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:11:46.162337+0000","last_change":"2023-11-24T06:11:44.071760+0000","last_active":"2023-11-24T06:11:46.162337+0000","last_peered":"2023-11-24T06:11:46.162337+0000","last_clean":"2023-11-24T06:11:46.162337+0000","last_became_active":"2023-11-24T06:11:44.071383+0000","last_became_peered":"2023-11-24T06:11:44.071383+0000","last_unstale":"2023-11-24T06:11:46.162337+0000","last_undegraded":"2023-11-24T06:11:46.162337+0000","last_fullsized":"2023-11-24T06:11:46.162337+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:43.003933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:43.003933+0000","last_clean_scrub_stamp":"2023-11-24T06:11:43.003933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:55:16.324642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:11:46.164257+0000","last_change":"2023-11-24T06:11:45.057810+0000","last_active":"2023-11-24T06:11:46.164257+0000","last_peered":"2023-11-24T06:11:46.164257+0000","last_clean":"2023-11-24T06:11:46.164257+0000","last_became_active":"2023-11-24T06:11:45.057459+0000","last_became_peered":"2023-11-24T06:11:45.057459+0000","last_unstale":"2023-11-24T06:11:46.164257+0000","last_undegraded":"2023-11-24T06:11:46.164257+0000","last_fullsized":"2023-11-24T06:11:46.164257+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:44.018686+0000","last_clean_scrub_stamp":"2023-11-24T06:11:44.018686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:29:49.006871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:11:46.044329+0000","last_change":"2023-11-24T06:11:44.066074+0000","last_active":"2023-11-24T06:11:46.044329+0000","last_peered":"2023-11-24T06:11:46.044329+0000","last_clean":"2023-11-24T06:11:46.044329+0000","last_became_active":"2023-11-24T06:11:44.065792+0000","last_became_peered":"2023-11-24T06:11:44.065792+0000","last_unstale":"2023-11-24T06:11:46.044329+0000","last_undegraded":"2023-11-24T06:11:46.044329+0000","last_fullsized":"2023-11-24T06:11:46.044329+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:43.003933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:43.003933+0000","last_clean_scrub_stamp":"2023-11-24T06:11:43.003933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:36:30.984958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":176,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T06:11:45.049256+0000","last_change":"2023-11-24T06:10:20.082677+0000","last_active":"2023-11-24T06:11:45.049256+0000","last_peered":"2023-11-24T06:11:45.049256+0000","last_clean":"2023-11-24T06:11:45.049256+0000","last_became_active":"2023-11-24T06:10:20.080551+0000","last_became_peered":"2023-11-24T06:10:20.080551+0000","last_unstale":"2023-11-24T06:11:45.049256+0000","last_undegraded":"2023-11-24T06:11:45.049256+0000","last_fullsized":"2023-11-24T06:11:45.049256+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:18.602209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:18.602209+0000","last_clean_scrub_stamp":"2023-11-24T06:10:18.602209+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:47:24.216235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574868,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32132,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339708,"statfs":{"total":96636764160,"available":96603860992,"internally_reserved":0,"allocated":970752,"data_stored":737382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:11:18 2023","interfaces":[{"interface":"back","average":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"min":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":1.9630000000000001,"5min":1.9630000000000001,"15min":1.9630000000000001},"last":1.103},{"interface":"front","average":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"min":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"max":{"1min":1.621,"5min":1.621,"15min":1.621},"last":1.4059999999999999}]},{"osd":1,"last update":"Fri Nov 24 06:11:18 2023","interfaces":[{"interface":"back","average":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"min":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"max":{"1min":2.0259999999999998,"5min":2.0259999999999998,"15min":2.0259999999999998},"last":0.89900000000000002},{"interface":"front","average":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":2.1480000000000001,"5min":2.1480000000000001,"15min":2.1480000000000001},"last":0.99099999999999999}]},{"osd":2,"last update":"Fri Nov 24 06:11:18 2023","interfaces":[{"interface":"back","average":{"1min":1.143,"5min":1.143,"15min":1.143},"min":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":1.7809999999999999,"5min":1.7809999999999999,"15min":1.7809999999999999},"last":1.054},{"interface":"front","average":{"1min":1.1719999999999999,"5min":1.1719999999999999,"15min":1.1719999999999999},"min":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"max":{"1min":1.4810000000000001,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":1.2130000000000001}]},{"osd":3,"last update":"Fri Nov 24 06:11:18 2023","interfaces":[{"interface":"back","average":{"1min":1.206,"5min":1.206,"15min":1.206},"min":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"max":{"1min":1.593,"5min":1.593,"15min":1.593},"last":1.462},{"interface":"front","average":{"1min":1.3109999999999999,"5min":1.3109999999999999,"15min":1.3109999999999999},"min":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"max":{"1min":1.6539999999999999,"5min":1.6539999999999999,"15min":1.6539999999999999},"last":1.359}]},{"osd":4,"last update":"Fri Nov 24 06:11:18 2023","interfaces":[{"interface":"back","average":{"1min":1.2889999999999999,"5min":1.2889999999999999,"15min":1.2889999999999999},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.76,"5min":1.76,"15min":1.76},"last":1.5069999999999999},{"interface":"front","average":{"1min":1.329,"5min":1.329,"15min":1.329},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":2.1680000000000001,"5min":2.1680000000000001,"15min":2.1680000000000001},"last":1.5720000000000001}]},{"osd":5,"last update":"Fri Nov 24 06:11:18 2023","interfaces":[{"interface":"back","average":{"1min":1.446,"5min":1.446,"15min":1.446},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.8819999999999999,"5min":1.8819999999999999,"15min":1.8819999999999999},"last":1.621},{"interface":"front","average":{"1min":1.454,"5min":1.454,"15min":1.454},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":2.0539999999999998,"5min":2.0539999999999998,"15min":2.0539999999999998},"last":1.7529999999999999}]},{"osd":6,"last update":"Fri Nov 24 06:11:18 2023","interfaces":[{"interface":"back","average":{"1min":1.5209999999999999,"5min":1.5209999999999999,"15min":1.5209999999999999},"min":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.9750000000000001,"5min":1.9750000000000001,"15min":1.9750000000000001},"last":1.71},{"interface":"front","average":{"1min":1.53,"5min":1.53,"15min":1.53},"min":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":2.0680000000000001,"5min":2.0680000000000001,"15min":2.0680000000000001},"last":1.8640000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574869,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31552,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340288,"statfs":{"total":96636764160,"available":96604454912,"internally_reserved":0,"allocated":380928,"data_stored":147540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:11:18 2023","interfaces":[{"interface":"back","average":{"1min":1.2190000000000001,"5min":1.2190000000000001,"15min":1.2190000000000001},"min":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":1.921,"5min":1.921,"15min":1.921},"last":0.72899999999999998},{"interface":"front","average":{"1min":1.2250000000000001,"5min":1.2250000000000001,"15min":1.2250000000000001},"min":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":2.0230000000000001,"5min":2.0230000000000001,"15min":2.0230000000000001},"last":0.89800000000000002}]},{"osd":1,"last update":"Fri Nov 24 06:11:18 2023","interfaces":[{"interface":"back","average":{"1min":1.167,"5min":1.167,"15min":1.167},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":2.0609999999999999,"5min":2.0609999999999999,"15min":2.0609999999999999},"last":1.135},{"interface":"front","average":{"1min":1.2350000000000001,"5min":1.2350000000000001,"15min":1.2350000000000001},"min":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"max":{"1min":1.8899999999999999,"5min":1.8899999999999999,"15min":1.8899999999999999},"last":0.79600000000000004}]},{"osd":2,"last update":"Fri Nov 24 06:11:18 2023","interfaces":[{"interface":"back","average":{"1min":1.1519999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"min":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"max":{"1min":1.655,"5min":1.655,"15min":1.655},"last":0.93400000000000005},{"interface":"front","average":{"1min":1.4390000000000001,"5min":1.4390000000000001,"15min":1.4390000000000001},"min":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"max":{"1min":2.1640000000000001,"5min":2.1640000000000001,"15min":2.1640000000000001},"last":1.323}]},{"osd":3,"last update":"Fri Nov 24 06:11:18 2023","interfaces":[{"interface":"back","average":{"1min":1.3140000000000001,"5min":1.3140000000000001,"15min":1.3140000000000001},"min":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"max":{"1min":1.919,"5min":1.919,"15min":1.919},"last":1.012},{"interface":"front","average":{"1min":1.296,"5min":1.296,"15min":1.296},"min":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.6870000000000001,"5min":1.6870000000000001,"15min":1.6870000000000001},"last":1.0840000000000001}]},{"osd":4,"last update":"Fri Nov 24 06:11:18 2023","interfaces":[{"interface":"back","average":{"1min":1.458,"5min":1.458,"15min":1.458},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":2.2410000000000001,"5min":2.2410000000000001,"15min":2.2410000000000001},"last":1.4450000000000001},{"interface":"front","average":{"1min":1.48,"5min":1.48,"15min":1.48},"min":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":2.21,"5min":2.21,"15min":2.21},"last":1.2090000000000001}]},{"osd":5,"last update":"Fri Nov 24 06:11:18 2023","interfaces":[{"interface":"back","average":{"1min":1.4770000000000001,"5min":1.4770000000000001,"15min":1.4770000000000001},"min":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"max":{"1min":2.0659999999999998,"5min":2.0659999999999998,"15min":2.0659999999999998},"last":1.2669999999999999},{"interface":"front","average":{"1min":1.657,"5min":1.657,"15min":1.657},"min":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"max":{"1min":2.4089999999999998,"5min":2.4089999999999998,"15min":2.4089999999999998},"last":1.5349999999999999}]},{"osd":7,"last update":"Fri Nov 24 06:11:18 2023","interfaces":[{"interface":"back","average":{"1min":1.591,"5min":1.591,"15min":1.591},"min":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":2.5139999999999998,"5min":2.5139999999999998,"15min":2.5139999999999998},"last":1.585},{"interface":"front","average":{"1min":1.548,"5min":1.548,"15min":1.548},"min":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":2.3370000000000002,"5min":2.3370000000000002,"15min":2.3370000000000002},"last":1.379}]}]},{"osd":5,"up_from":13,"seq":55834574869,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31548,"kb_used_data":368,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340292,"statfs":{"total":96636764160,"available":96604459008,"internally_reserved":0,"allocated":376832,"data_stored":146982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:11:18 2023","interfaces":[{"interface":"back","average":{"1min":1.2509999999999999,"5min":1.2509999999999999,"15min":1.2509999999999999},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":3.0030000000000001,"5min":3.0030000000000001,"15min":3.0030000000000001},"last":0.49099999999999999},{"interface":"front","average":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":1.2270000000000001},"min":{"1min":0.749,"5min":0.749,"15min":0.749},"max":{"1min":2.0979999999999999,"5min":2.0979999999999999,"15min":2.0979999999999999},"last":0.39100000000000001}]},{"osd":1,"last update":"Fri Nov 24 06:11:18 2023","interfaces":[{"interface":"back","average":{"1min":1.276,"5min":1.276,"15min":1.276},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":5.0499999999999998,"5min":5.0499999999999998,"15min":5.0499999999999998},"last":0.53300000000000003},{"interface":"front","average":{"1min":1.48,"5min":1.48,"15min":1.48},"min":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":3.6930000000000001,"5min":3.6930000000000001,"15min":3.6930000000000001},"last":0.435}]},{"osd":2,"last update":"Fri Nov 24 06:11:18 2023","interfaces":[{"interface":"back","average":{"1min":1.345,"5min":1.345,"15min":1.345},"min":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"max":{"1min":3.085,"5min":3.085,"15min":3.085},"last":0.93899999999999995},{"interface":"front","average":{"1min":1.6160000000000001,"5min":1.6160000000000001,"15min":1.6160000000000001},"min":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"max":{"1min":4.7359999999999998,"5min":4.7359999999999998,"15min":4.7359999999999998},"last":0.46400000000000002}]},{"osd":3,"last update":"Fri Nov 24 06:11:18 2023","interfaces":[{"interface":"back","average":{"1min":1.421,"5min":1.421,"15min":1.421},"min":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"max":{"1min":3.2229999999999999,"5min":3.2229999999999999,"15min":3.2229999999999999},"last":0.77900000000000003},{"interface":"front","average":{"1min":1.4450000000000001,"5min":1.4450000000000001,"15min":1.4450000000000001},"min":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"max":{"1min":2.3260000000000001,"5min":2.3260000000000001,"15min":2.3260000000000001},"last":0.76100000000000001}]},{"osd":4,"last update":"Fri Nov 24 06:11:18 2023","interfaces":[{"interface":"back","average":{"1min":1.5669999999999999,"5min":1.5669999999999999,"15min":1.5669999999999999},"min":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"max":{"1min":3.1230000000000002,"5min":3.1230000000000002,"15min":3.1230000000000002},"last":0.84299999999999997},{"interface":"front","average":{"1min":1.5589999999999999,"5min":1.5589999999999999,"15min":1.5589999999999999},"min":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"max":{"1min":2.4809999999999999,"5min":2.4809999999999999,"15min":2.4809999999999999},"last":0.64700000000000002}]},{"osd":6,"last update":"Fri Nov 24 06:11:18 2023","interfaces":[{"interface":"back","average":{"1min":1.7569999999999999,"5min":1.7569999999999999,"15min":1.7569999999999999},"min":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"max":{"1min":4.6529999999999996,"5min":4.6529999999999996,"15min":4.6529999999999996},"last":0.58599999999999997},{"interface":"front","average":{"1min":1.75,"5min":1.75,"15min":1.75},"min":{"1min":1.125,"5min":1.125,"15min":1.125},"max":{"1min":4.3529999999999998,"5min":4.3529999999999998,"15min":4.3529999999999998},"last":0.90400000000000003}]},{"osd":7,"last update":"Fri Nov 24 06:11:18 2023","interfaces":[{"interface":"back","average":{"1min":1.7110000000000001,"5min":1.7110000000000001,"15min":1.7110000000000001},"min":{"1min":1.2450000000000001,"5min":1.2450000000000001,"15min":1.2450000000000001},"max":{"1min":2.8839999999999999,"5min":2.8839999999999999,"15min":2.8839999999999999},"last":0.874},{"interface":"front","average":{"1min":1.867,"5min":1.867,"15min":1.867},"min":{"1min":1.204,"5min":1.204,"15min":1.204},"max":{"1min":4.8739999999999997,"5min":4.8739999999999997,"15min":4.8739999999999997},"last":0.69599999999999995}]}]},{"osd":4,"up_from":13,"seq":55834574869,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31552,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340288,"statfs":{"total":96636764160,"available":96604454912,"internally_reserved":0,"allocated":380928,"data_stored":147028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:11:21 2023","interfaces":[{"interface":"back","average":{"1min":1.042,"5min":1.042,"15min":1.042},"min":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":1.498,"5min":1.498,"15min":1.498},"last":1.3149999999999999},{"interface":"front","average":{"1min":1.2170000000000001,"5min":1.2170000000000001,"15min":1.2170000000000001},"min":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"max":{"1min":1.8220000000000001,"5min":1.8220000000000001,"15min":1.8220000000000001},"last":1.085}]},{"osd":1,"last update":"Fri Nov 24 06:11:21 2023","interfaces":[{"interface":"back","average":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"min":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":1.6859999999999999,"5min":1.6859999999999999,"15min":1.6859999999999999},"last":0.874},{"interface":"front","average":{"1min":1.1779999999999999,"5min":1.1779999999999999,"15min":1.1779999999999999},"min":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":1.875,"5min":1.875,"15min":1.875},"last":1.139}]},{"osd":2,"last update":"Fri Nov 24 06:11:21 2023","interfaces":[{"interface":"back","average":{"1min":1.2509999999999999,"5min":1.2509999999999999,"15min":1.2509999999999999},"min":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"max":{"1min":2.319,"5min":2.319,"15min":2.319},"last":1.21},{"interface":"front","average":{"1min":1.3089999999999999,"5min":1.3089999999999999,"15min":1.3089999999999999},"min":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"max":{"1min":2.0990000000000002,"5min":2.0990000000000002,"15min":2.0990000000000002},"last":1.3620000000000001}]},{"osd":3,"last update":"Fri Nov 24 06:11:21 2023","interfaces":[{"interface":"back","average":{"1min":1.4119999999999999,"5min":1.4119999999999999,"15min":1.4119999999999999},"min":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"max":{"1min":2.0179999999999998,"5min":2.0179999999999998,"15min":2.0179999999999998},"last":1.272},{"interface":"front","average":{"1min":1.4419999999999999,"5min":1.4419999999999999,"15min":1.4419999999999999},"min":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"max":{"1min":2.1720000000000002,"5min":2.1720000000000002,"15min":2.1720000000000002},"last":1.569}]},{"osd":5,"last update":"Fri Nov 24 06:11:21 2023","interfaces":[{"interface":"back","average":{"1min":1.3380000000000001,"5min":1.3380000000000001,"15min":1.3380000000000001},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.895,"5min":1.895,"15min":1.895},"last":1.419},{"interface":"front","average":{"1min":1.552,"5min":1.552,"15min":1.552},"min":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"max":{"1min":2.5059999999999998,"5min":2.5059999999999998,"15min":2.5059999999999998},"last":1.4359999999999999}]},{"osd":6,"last update":"Fri Nov 24 06:11:21 2023","interfaces":[{"interface":"back","average":{"1min":1.5469999999999999,"5min":1.5469999999999999,"15min":1.5469999999999999},"min":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"max":{"1min":2.2069999999999999,"5min":2.2069999999999999,"15min":2.2069999999999999},"last":1.3919999999999999},{"interface":"front","average":{"1min":1.6850000000000001,"5min":1.6850000000000001,"15min":1.6850000000000001},"min":{"1min":1.119,"5min":1.119,"15min":1.119},"max":{"1min":2.4510000000000001,"5min":2.4510000000000001,"15min":2.4510000000000001},"last":1.633}]},{"osd":7,"last update":"Fri Nov 24 06:11:21 2023","interfaces":[{"interface":"back","average":{"1min":1.6439999999999999,"5min":1.6439999999999999,"15min":1.6439999999999999},"min":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"max":{"1min":2.4350000000000001,"5min":2.4350000000000001,"15min":2.4350000000000001},"last":1.5249999999999999},{"interface":"front","average":{"1min":1.633,"5min":1.633,"15min":1.633},"min":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"max":{"1min":3.1120000000000001,"5min":3.1120000000000001,"15min":3.1120000000000001},"last":1.8200000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574869,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27400,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344440,"statfs":{"total":96636764160,"available":96608706560,"internally_reserved":0,"allocated":389120,"data_stored":148558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:11:22 2023","interfaces":[{"interface":"back","average":{"1min":1.008,"5min":1.008,"15min":1.008},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":2.766,"5min":2.766,"15min":2.766},"last":1.165},{"interface":"front","average":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.73,"5min":1.73,"15min":1.73},"last":1.4370000000000001}]},{"osd":1,"last update":"Fri Nov 24 06:11:22 2023","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":2.5779999999999998,"5min":2.5779999999999998,"15min":2.5779999999999998},"last":0.97699999999999998},{"interface":"front","average":{"1min":1.1839999999999999,"5min":1.1839999999999999,"15min":1.1839999999999999},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":3.6899999999999999,"5min":3.6899999999999999,"15min":3.6899999999999999},"last":1.4670000000000001}]},{"osd":2,"last update":"Fri Nov 24 06:11:22 2023","interfaces":[{"interface":"back","average":{"1min":1.177,"5min":1.177,"15min":1.177},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":4.0620000000000003,"5min":4.0620000000000003,"15min":4.0620000000000003},"last":1.081},{"interface":"front","average":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":1.2270000000000001},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":2.9529999999999998,"5min":2.9529999999999998,"15min":2.9529999999999998},"last":1.3700000000000001}]},{"osd":4,"last update":"Fri Nov 24 06:11:22 2023","interfaces":[{"interface":"back","average":{"1min":1.4279999999999999,"5min":1.4279999999999999,"15min":1.4279999999999999},"min":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":4.125,"5min":4.125,"15min":4.125},"last":1.518},{"interface":"front","average":{"1min":1.4470000000000001,"5min":1.4470000000000001,"15min":1.4470000000000001},"min":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"max":{"1min":3.9409999999999998,"5min":3.9409999999999998,"15min":3.9409999999999998},"last":1.2}]},{"osd":5,"last update":"Fri Nov 24 06:11:22 2023","interfaces":[{"interface":"back","average":{"1min":1.4470000000000001,"5min":1.4470000000000001,"15min":1.4470000000000001},"min":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"max":{"1min":3.3359999999999999,"5min":3.3359999999999999,"15min":3.3359999999999999},"last":1.544},{"interface":"front","average":{"1min":1.46,"5min":1.46,"15min":1.46},"min":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":4.1669999999999998,"5min":4.1669999999999998,"15min":4.1669999999999998},"last":1.0349999999999999}]},{"osd":6,"last update":"Fri Nov 24 06:11:22 2023","interfaces":[{"interface":"back","average":{"1min":1.4370000000000001,"5min":1.4370000000000001,"15min":1.4370000000000001},"min":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":4.0190000000000001,"5min":4.0190000000000001,"15min":4.0190000000000001},"last":1.1419999999999999},{"interface":"front","average":{"1min":1.5329999999999999,"5min":1.5329999999999999,"15min":1.5329999999999999},"min":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"max":{"1min":3.2480000000000002,"5min":3.2480000000000002,"15min":3.2480000000000002},"last":1.627}]},{"osd":7,"last update":"Fri Nov 24 06:11:22 2023","interfaces":[{"interface":"back","average":{"1min":1.613,"5min":1.613,"15min":1.613},"min":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"max":{"1min":4.202,"5min":4.202,"15min":4.202},"last":1.2889999999999999},{"interface":"front","average":{"1min":1.6220000000000001,"5min":1.6220000000000001,"15min":1.6220000000000001},"min":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"max":{"1min":3.9849999999999999,"5min":3.9849999999999999,"15min":3.9849999999999999},"last":1.2549999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574869,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27384,"kb_used_data":364,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344456,"statfs":{"total":96636764160,"available":96608722944,"internally_reserved":0,"allocated":372736,"data_stored":146380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:11:21 2023","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.6259999999999999,"5min":1.6259999999999999,"15min":1.6259999999999999},"last":0.67400000000000004},{"interface":"front","average":{"1min":1.101,"5min":1.101,"15min":1.101},"min":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.7669999999999999,"5min":1.7669999999999999,"15min":1.7669999999999999},"last":0.63}]},{"osd":1,"last update":"Fri Nov 24 06:11:21 2023","interfaces":[{"interface":"back","average":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.506,"5min":1.506,"15min":1.506},"last":1.028},{"interface":"front","average":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.8140000000000001,"5min":1.8140000000000001,"15min":1.8140000000000001},"last":1.0600000000000001}]},{"osd":3,"last update":"Fri Nov 24 06:11:21 2023","interfaces":[{"interface":"back","average":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":2.3559999999999999,"5min":2.3559999999999999,"15min":2.3559999999999999},"last":0.75600000000000001},{"interface":"front","average":{"1min":1.123,"5min":1.123,"15min":1.123},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":2.1840000000000002,"5min":2.1840000000000002,"15min":2.1840000000000002},"last":1.1120000000000001}]},{"osd":4,"last update":"Fri Nov 24 06:11:21 2023","interfaces":[{"interface":"back","average":{"1min":1.419,"5min":1.419,"15min":1.419},"min":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":3.411,"5min":3.411,"15min":3.411},"last":1.2769999999999999},{"interface":"front","average":{"1min":1.506,"5min":1.506,"15min":1.506},"min":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"max":{"1min":3.7109999999999999,"5min":3.7109999999999999,"15min":3.7109999999999999},"last":1.395}]},{"osd":5,"last update":"Fri Nov 24 06:11:21 2023","interfaces":[{"interface":"back","average":{"1min":1.325,"5min":1.325,"15min":1.325},"min":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"max":{"1min":2.5259999999999998,"5min":2.5259999999999998,"15min":2.5259999999999998},"last":1.252},{"interface":"front","average":{"1min":1.302,"5min":1.302,"15min":1.302},"min":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"max":{"1min":2.1640000000000001,"5min":2.1640000000000001,"15min":2.1640000000000001},"last":1.5029999999999999}]},{"osd":6,"last update":"Fri Nov 24 06:11:21 2023","interfaces":[{"interface":"back","average":{"1min":1.4570000000000001,"5min":1.4570000000000001,"15min":1.4570000000000001},"min":{"1min":1.073,"5min":1.073,"15min":1.073},"max":{"1min":2.3039999999999998,"5min":2.3039999999999998,"15min":2.3039999999999998},"last":1.198},{"interface":"front","average":{"1min":1.4690000000000001,"5min":1.4690000000000001,"15min":1.4690000000000001},"min":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"max":{"1min":2.605,"5min":2.605,"15min":2.605},"last":1.169}]},{"osd":7,"last update":"Fri Nov 24 06:11:21 2023","interfaces":[{"interface":"back","average":{"1min":1.595,"5min":1.595,"15min":1.595},"min":{"1min":1.1910000000000001,"5min":1.1910000000000001,"15min":1.1910000000000001},"max":{"1min":2.3740000000000001,"5min":2.3740000000000001,"15min":2.3740000000000001},"last":1.331},{"interface":"front","average":{"1min":1.4670000000000001,"5min":1.4670000000000001,"15min":1.4670000000000001},"min":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"max":{"1min":2.5699999999999998,"5min":2.5699999999999998,"15min":2.5699999999999998},"last":1.222}]}]},{"osd":1,"up_from":13,"seq":55834574868,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27408,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344432,"statfs":{"total":96636764160,"available":96608698368,"internally_reserved":0,"allocated":389120,"data_stored":148558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:11:19 2023","interfaces":[{"interface":"back","average":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.9299999999999999,"5min":1.9299999999999999,"15min":1.9299999999999999},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.7669999999999999,"5min":1.7669999999999999,"15min":1.7669999999999999},"last":0.71899999999999997}]},{"osd":2,"last update":"Fri Nov 24 06:11:19 2023","interfaces":[{"interface":"back","average":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":2.5329999999999999,"5min":2.5329999999999999,"15min":2.5329999999999999},"last":0.74099999999999999},{"interface":"front","average":{"1min":1.03,"5min":1.03,"15min":1.03},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.657,"5min":1.657,"15min":1.657},"last":0.57899999999999996}]},{"osd":3,"last update":"Fri Nov 24 06:11:19 2023","interfaces":[{"interface":"back","average":{"1min":1.038,"5min":1.038,"15min":1.038},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.841,"5min":1.841,"15min":1.841},"last":0.69299999999999995},{"interface":"front","average":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":2.8940000000000001,"5min":2.8940000000000001,"15min":2.8940000000000001},"last":0.95899999999999996}]},{"osd":4,"last update":"Fri Nov 24 06:11:19 2023","interfaces":[{"interface":"back","average":{"1min":1.282,"5min":1.282,"15min":1.282},"min":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"max":{"1min":2.1930000000000001,"5min":2.1930000000000001,"15min":2.1930000000000001},"last":1.0900000000000001},{"interface":"front","average":{"1min":1.2,"5min":1.2,"15min":1.2},"min":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":1.784,"5min":1.784,"15min":1.784},"last":1.169}]},{"osd":5,"last update":"Fri Nov 24 06:11:19 2023","interfaces":[{"interface":"back","average":{"1min":1.53,"5min":1.53,"15min":1.53},"min":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"max":{"1min":5.5449999999999999,"5min":5.5449999999999999,"15min":5.5449999999999999},"last":1.0269999999999999},{"interface":"front","average":{"1min":1.5640000000000001,"5min":1.5640000000000001,"15min":1.5640000000000001},"min":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"max":{"1min":6.0030000000000001,"5min":6.0030000000000001,"15min":6.0030000000000001},"last":1.2370000000000001}]},{"osd":6,"last update":"Fri Nov 24 06:11:19 2023","interfaces":[{"interface":"back","average":{"1min":1.569,"5min":1.569,"15min":1.569},"min":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"max":{"1min":5.2809999999999997,"5min":5.2809999999999997,"15min":5.2809999999999997},"last":0.98699999999999999},{"interface":"front","average":{"1min":1.4750000000000001,"5min":1.4750000000000001,"15min":1.4750000000000001},"min":{"1min":0.749,"5min":0.749,"15min":0.749},"max":{"1min":4.327,"5min":4.327,"15min":4.327},"last":1.1499999999999999}]},{"osd":7,"last update":"Fri Nov 24 06:11:19 2023","interfaces":[{"interface":"back","average":{"1min":1.359,"5min":1.359,"15min":1.359},"min":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":2.3069999999999999,"5min":2.3069999999999999,"15min":2.3069999999999999},"last":1.01},{"interface":"front","average":{"1min":1.5980000000000001,"5min":1.5980000000000001,"15min":1.5980000000000001},"min":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"max":{"1min":5.4500000000000002,"5min":5.4500000000000002,"15min":5.4500000000000002},"last":1.0509999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574868,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27968,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343872,"statfs":{"total":96636764160,"available":96608124928,"internally_reserved":0,"allocated":966656,"data_stored":737292,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Nov 24 06:11:18 2023","interfaces":[{"interface":"back","average":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":2.3999999999999999,"5min":2.3999999999999999,"15min":2.3999999999999999},"last":0.97799999999999998},{"interface":"front","average":{"1min":1.1879999999999999,"5min":1.1879999999999999,"15min":1.1879999999999999},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":4.8289999999999997,"5min":4.8289999999999997,"15min":4.8289999999999997},"last":0.71299999999999997}]},{"osd":2,"last update":"Fri Nov 24 06:11:18 2023","interfaces":[{"interface":"back","average":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":2.5139999999999998,"5min":2.5139999999999998,"15min":2.5139999999999998},"last":0.61299999999999999},{"interface":"front","average":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":2.3599999999999999,"5min":2.3599999999999999,"15min":2.3599999999999999},"last":0.86699999999999999}]},{"osd":3,"last update":"Fri Nov 24 06:11:18 2023","interfaces":[{"interface":"back","average":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":2.5840000000000001,"5min":2.5840000000000001,"15min":2.5840000000000001},"last":0.749},{"interface":"front","average":{"1min":1.415,"5min":1.415,"15min":1.415},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":5.1230000000000002,"5min":5.1230000000000002,"15min":5.1230000000000002},"last":1.1359999999999999}]},{"osd":4,"last update":"Fri Nov 24 06:11:18 2023","interfaces":[{"interface":"back","average":{"1min":1.2829999999999999,"5min":1.2829999999999999,"15min":1.2829999999999999},"min":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"max":{"1min":2.887,"5min":2.887,"15min":2.887},"last":1.085},{"interface":"front","average":{"1min":1.3360000000000001,"5min":1.3360000000000001,"15min":1.3360000000000001},"min":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":2.8530000000000002,"5min":2.8530000000000002,"15min":2.8530000000000002},"last":0.92800000000000005}]},{"osd":5,"last update":"Fri Nov 24 06:11:18 2023","interfaces":[{"interface":"back","average":{"1min":1.575,"5min":1.575,"15min":1.575},"min":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"max":{"1min":4.6299999999999999,"5min":4.6299999999999999,"15min":4.6299999999999999},"last":1.218},{"interface":"front","average":{"1min":1.333,"5min":1.333,"15min":1.333},"min":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":2.794,"5min":2.794,"15min":2.794},"last":1.2509999999999999}]},{"osd":6,"last update":"Fri Nov 24 06:11:18 2023","interfaces":[{"interface":"back","average":{"1min":1.6719999999999999,"5min":1.6719999999999999,"15min":1.6719999999999999},"min":{"1min":0.747,"5min":0.747,"15min":0.747},"max":{"1min":4.798,"5min":4.798,"15min":4.798},"last":1.169},{"interface":"front","average":{"1min":1.637,"5min":1.637,"15min":1.637},"min":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"max":{"1min":4.7149999999999999,"5min":4.7149999999999999,"15min":4.7149999999999999},"last":1.351}]},{"osd":7,"last update":"Fri Nov 24 06:11:18 2023","interfaces":[{"interface":"back","average":{"1min":1.716,"5min":1.716,"15min":1.716},"min":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":4.7649999999999997,"5min":4.7649999999999997,"15min":4.7649999999999997},"last":1.0569999999999999},{"interface":"front","average":{"1min":1.6950000000000001,"5min":1.6950000000000001,"15min":1.6950000000000001},"min":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"max":{"1min":4.6820000000000004,"5min":4.6820000000000004,"15min":4.6820000000000004},"last":1.3}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-24T06:11:50.427 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-11-24T06:11:50.427 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-11-24T06:11:50.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:50.691+0000 7f5a623c1640 1 Processor -- start 2023-11-24T06:11:50.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:50.692+0000 7f5a623c1640 1 -- start start 2023-11-24T06:11:50.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:50.693+0000 7f5a623c1640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5a5c0ced50 0x7f5a5c0b3da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:50.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:50.693+0000 7f5a623c1640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a5c160cc0 con 0x7f5a5c1608e0 2023-11-24T06:11:50.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:50.693+0000 7f5a623c1640 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a5c0d1640 con 0x7f5a5c0cff80 2023-11-24T06:11:50.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:50.693+0000 7f5a623c1640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a5c0d1fe0 con 0x7f5a5c0ced50 2023-11-24T06:11:50.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:50.693+0000 7f5a613bf640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f5a5c1608e0 0x7f5a5c0cf870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:41246/0 (socket says 172.21.15.92:41246) 2023-11-24T06:11:50.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:50.693+0000 7f5a613bf640 1 -- 172.21.15.92:0/1907893715 learned_addr learned my addr 172.21.15.92:0/1907893715 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-24T06:11:50.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:50.694+0000 7f5a53fff640 1 -- 172.21.15.92:0/1907893715 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1528721483 0 0) 0x7f5a5c160cc0 con 0x7f5a5c1608e0 2023-11-24T06:11:50.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:50.694+0000 7f5a53fff640 1 -- 172.21.15.92:0/1907893715 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5a48003520 con 0x7f5a5c1608e0 2023-11-24T06:11:50.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:50.694+0000 7f5a60bbe640 1 --2- 172.21.15.92:0/1907893715 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5a5c0ced50 0x7f5a5c0b3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:50.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:50.694+0000 7f5a53fff640 1 -- 172.21.15.92:0/1907893715 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 95504847 0 0) 0x7f5a48003520 con 0x7f5a5c1608e0 2023-11-24T06:11:50.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:50.694+0000 7f5a53fff640 1 -- 172.21.15.92:0/1907893715 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5a5c0ced50 msgr2=0x7f5a5c0b3da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:50.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:50.694+0000 7f5a53fff640 1 --2- 172.21.15.92:0/1907893715 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5a5c0ced50 0x7f5a5c0b3da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:50.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:50.694+0000 7f5a53fff640 1 -- 172.21.15.92:0/1907893715 >> v1:172.21.15.118:6789/0 conn(0x7f5a5c0cff80 legacy=0x7f5a5c0b5fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:50.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:50.694+0000 7f5a53fff640 1 -- 172.21.15.92:0/1907893715 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a5c0d1920 con 0x7f5a5c1608e0 2023-11-24T06:11:50.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:50.694+0000 7f5a53fff640 1 -- 172.21.15.92:0/1907893715 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2607661350 0 0) 0x7f5a58001670 con 0x7f5a5c1608e0 2023-11-24T06:11:50.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:50.694+0000 7f5a53fff640 1 -- 172.21.15.92:0/1907893715 >> v1:172.21.15.92:6789/0 conn(0x7f5a5c1608e0 legacy=0x7f5a5c0cf870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:50.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:50.694+0000 7f5a53fff640 1 --2- 172.21.15.92:0/1907893715 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5a480039a0 0x7f5a48003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:50.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:50.694+0000 7f5a53fff640 1 --2- 172.21.15.92:0/1907893715 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5a480047c0 0x7f5a48004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:50.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:50.694+0000 7f5a53fff640 1 --2- 172.21.15.92:0/1907893715 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5a5c0ced50 0x7f5a480051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:50.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:50.694+0000 7f5a53fff640 1 -- 172.21.15.92:0/1907893715 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a5c0d1fe0 con 0x7f5a480047c0 2023-11-24T06:11:50.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:50.694+0000 7f5a53fff640 1 -- 172.21.15.92:0/1907893715 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a4800bab0 con 0x7f5a480039a0 2023-11-24T06:11:50.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:50.694+0000 7f5a53fff640 1 -- 172.21.15.92:0/1907893715 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a4800bc70 con 0x7f5a5c0ced50 2023-11-24T06:11:50.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:50.694+0000 7f5a60bbe640 1 --2- 172.21.15.92:0/1907893715 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5a480047c0 0x7f5a48004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:50.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:50.695+0000 7f5a613bf640 1 --2- 172.21.15.92:0/1907893715 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5a480039a0 0x7f5a48003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:50.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:50.695+0000 7f5a61bc0640 1 --2- 172.21.15.92:0/1907893715 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5a5c0ced50 0x7f5a480051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:50.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:50.695+0000 7f5a60bbe640 1 -- 172.21.15.92:0/1907893715 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5a5c0ced50 msgr2=0x7f5a480051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:50.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:50.695+0000 7f5a60bbe640 1 --2- 172.21.15.92:0/1907893715 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5a5c0ced50 0x7f5a480051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:50.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:50.695+0000 7f5a60bbe640 1 -- 172.21.15.92:0/1907893715 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5a480039a0 msgr2=0x7f5a48003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:50.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:50.695+0000 7f5a60bbe640 1 --2- 172.21.15.92:0/1907893715 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5a480039a0 0x7f5a48003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:50.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:50.695+0000 7f5a60bbe640 1 -- 172.21.15.92:0/1907893715 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a48003520 con 0x7f5a480047c0 2023-11-24T06:11:50.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:50.695+0000 7f5a613bf640 1 --2- 172.21.15.92:0/1907893715 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5a480039a0 0x7f5a48003dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:11:50.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:50.695+0000 7f5a61bc0640 1 --2- 172.21.15.92:0/1907893715 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5a5c0ced50 0x7f5a480051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:11:50.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:50.696+0000 7f5a60bbe640 1 --2- 172.21.15.92:0/1907893715 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5a480047c0 0x7f5a48004c40 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f5a4c00ba00 tx=0x7f5a480057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d24016cf923e21c server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:50.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:50.696+0000 7f5a53fff640 1 -- 172.21.15.92:0/1907893715 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a4c032620 con 0x7f5a480047c0 2023-11-24T06:11:50.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:50.696+0000 7f5a53fff640 1 -- 172.21.15.92:0/1907893715 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a4c032780 con 0x7f5a480047c0 2023-11-24T06:11:50.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:50.696+0000 7f5a53fff640 1 -- 172.21.15.92:0/1907893715 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a4c032b10 con 0x7f5a480047c0 2023-11-24T06:11:50.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:50.697+0000 7f5a623c1640 1 -- 172.21.15.92:0/1907893715 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5a480047c0 msgr2=0x7f5a48004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:50.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:50.697+0000 7f5a623c1640 1 --2- 172.21.15.92:0/1907893715 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5a480047c0 0x7f5a48004c40 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f5a4c00ba00 tx=0x7f5a480057f0 comp rx=0 tx=0).stop 2023-11-24T06:11:50.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:50.697+0000 7f5a61bc0640 1 -- 172.21.15.92:0/1907893715 reap_dead start 2023-11-24T06:11:50.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:50.697+0000 7f5a623c1640 1 -- 172.21.15.92:0/1907893715 shutdown_connections 2023-11-24T06:11:50.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:50.697+0000 7f5a623c1640 1 -- 172.21.15.92:0/1907893715 >> 172.21.15.92:0/1907893715 conn(0x7f5a5c0d2fc0 msgr2=0x7f5a5c0d33d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:50.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:50.697+0000 7f5a623c1640 1 -- 172.21.15.92:0/1907893715 shutdown_connections 2023-11-24T06:11:50.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:50.697+0000 7f5a623c1640 1 -- 172.21.15.92:0/1907893715 wait complete. 2023-11-24T06:11:50.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:50.699+0000 7f5a623c1640 1 Processor -- start 2023-11-24T06:11:50.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:50.699+0000 7f5a623c1640 1 -- start start 2023-11-24T06:11:50.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:50.700+0000 7f5a623c1640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5a480047c0 0x7f5a5c0b8340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:50.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:50.700+0000 7f5a623c1640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5a5c14a430 0x7f5a5c1f68e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:50.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:50.700+0000 7f5a623c1640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5a5c06b600 0x7f5a5c06ba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:50.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:50.700+0000 7f5a623c1640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a5c14a8b0 con 0x7f5a5c14a430 2023-11-24T06:11:50.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:50.700+0000 7f5a623c1640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a5c1542d0 con 0x7f5a5c06b600 2023-11-24T06:11:50.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:50.700+0000 7f5a623c1640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a5c06bfc0 con 0x7f5a480047c0 2023-11-24T06:11:50.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:50.700+0000 7f5a60bbe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5a5c14a430 0x7f5a5c1f68e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:50.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:50.700+0000 7f5a60bbe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5a5c14a430 0x7f5a5c1f68e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58656/0 (socket says 172.21.15.92:58656) 2023-11-24T06:11:50.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:50.700+0000 7f5a60bbe640 1 -- 172.21.15.92:0/1049943731 learned_addr learned my addr 172.21.15.92:0/1049943731 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:11:50.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:50.701+0000 7f5a61bc0640 1 --2- 172.21.15.92:0/1049943731 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5a5c06b600 0x7f5a5c06ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:50.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:50.701+0000 7f5a613bf640 1 --2- 172.21.15.92:0/1049943731 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5a480047c0 0x7f5a5c0b8340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:50.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:50.701+0000 7f5a60bbe640 1 -- 172.21.15.92:0/1049943731 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5a480047c0 msgr2=0x7f5a5c0b8340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:50.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:50.701+0000 7f5a60bbe640 1 --2- 172.21.15.92:0/1049943731 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5a480047c0 0x7f5a5c0b8340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:50.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:50.701+0000 7f5a60bbe640 1 -- 172.21.15.92:0/1049943731 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5a5c06b600 msgr2=0x7f5a5c06ba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:50.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:50.701+0000 7f5a60bbe640 1 --2- 172.21.15.92:0/1049943731 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5a5c06b600 0x7f5a5c06ba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:50.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:50.701+0000 7f5a60bbe640 1 -- 172.21.15.92:0/1049943731 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a5c1f6e20 con 0x7f5a5c14a430 2023-11-24T06:11:50.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:50.701+0000 7f5a61bc0640 1 --2- 172.21.15.92:0/1049943731 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5a5c06b600 0x7f5a5c06ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:11:50.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:50.701+0000 7f5a60bbe640 1 --2- 172.21.15.92:0/1049943731 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5a5c14a430 0x7f5a5c1f68e0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f5a4c02bea0 tx=0x7f5a4c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:50.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:50.701+0000 7f5a51ffb640 1 -- 172.21.15.92:0/1049943731 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a4c03b620 con 0x7f5a5c14a430 2023-11-24T06:11:50.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:50.701+0000 7f5a51ffb640 1 -- 172.21.15.92:0/1049943731 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a4c03b780 con 0x7f5a5c14a430 2023-11-24T06:11:50.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:50.701+0000 7f5a51ffb640 1 -- 172.21.15.92:0/1049943731 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a4c03bb30 con 0x7f5a5c14a430 2023-11-24T06:11:50.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:50.701+0000 7f5a623c1640 1 -- 172.21.15.92:0/1049943731 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a5c1f70c0 con 0x7f5a5c14a430 2023-11-24T06:11:50.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:50.702+0000 7f5a623c1640 1 -- 172.21.15.92:0/1049943731 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a5c1f7680 con 0x7f5a5c14a430 2023-11-24T06:11:50.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:50.703+0000 7f5a623c1640 1 -- 172.21.15.92:0/1049943731 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a5c1f7870 con 0x7f5a5c14a430 2023-11-24T06:11:50.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:50.703+0000 7f5a51ffb640 1 -- 172.21.15.92:0/1049943731 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5a4c03bc90 con 0x7f5a5c14a430 2023-11-24T06:11:50.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:50.705+0000 7f5a51ffb640 1 --2- 172.21.15.92:0/1049943731 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f5a4006ae40 0x7f5a4006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:50.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:50.705+0000 7f5a51ffb640 1 -- 172.21.15.92:0/1049943731 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5a4c0ab770 con 0x7f5a5c14a430 2023-11-24T06:11:50.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:50.705+0000 7f5a613bf640 1 --2- 172.21.15.92:0/1049943731 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f5a4006ae40 0x7f5a4006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:50.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:50.706+0000 7f5a613bf640 1 --2- 172.21.15.92:0/1049943731 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f5a4006ae40 0x7f5a4006d300 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f5a58001eb0 tx=0x7f5a58009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:50.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:50.709+0000 7f5a51ffb640 1 -- 172.21.15.92:0/1049943731 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5a5c1f7870 con 0x7f5a5c14a430 2023-11-24T06:11:50.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:50.961+0000 7f5a623c1640 1 -- 172.21.15.92:0/1049943731 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f5a5c1f7870 con 0x7f5a5c14a430 2023-11-24T06:11:50.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:50.965+0000 7f5a51ffb640 1 -- 172.21.15.92:0/1049943731 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v9) v1 ==== 190+0+147 (secure 0 0 0) 0x7f5a4c0360b0 con 0x7f5a5c14a430 2023-11-24T06:11:50.969 INFO:teuthology.orchestra.run.smithi092.stdout:[client.0] 2023-11-24T06:11:50.969 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQARPmBlGMUsDhAAlgrJvA/TT5hpUYd5o9OzIQ== 2023-11-24T06:11:50.969 INFO:teuthology.orchestra.run.smithi092.stdout: caps mds = "allow" 2023-11-24T06:11:50.969 INFO:teuthology.orchestra.run.smithi092.stdout: caps mon = "allow r" 2023-11-24T06:11:50.969 INFO:teuthology.orchestra.run.smithi092.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-11-24T06:11:50.969 INFO:teuthology.orchestra.run.smithi092.stderr:updated caps for client.0 2023-11-24T06:11:50.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:50.968+0000 7f5a623c1640 1 -- 172.21.15.92:0/1049943731 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f5a4006ae40 msgr2=0x7f5a4006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:50.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:50.968+0000 7f5a623c1640 1 --2- 172.21.15.92:0/1049943731 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f5a4006ae40 0x7f5a4006d300 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f5a58001eb0 tx=0x7f5a58009040 comp rx=0 tx=0).stop 2023-11-24T06:11:50.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:50.968+0000 7f5a623c1640 1 -- 172.21.15.92:0/1049943731 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5a5c14a430 msgr2=0x7f5a5c1f68e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:50.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:50.968+0000 7f5a623c1640 1 --2- 172.21.15.92:0/1049943731 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5a5c14a430 0x7f5a5c1f68e0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f5a4c02bea0 tx=0x7f5a4c004730 comp rx=0 tx=0).stop 2023-11-24T06:11:50.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:50.968+0000 7f5a623c1640 1 -- 172.21.15.92:0/1049943731 shutdown_connections 2023-11-24T06:11:50.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:50.968+0000 7f5a623c1640 1 --2- 172.21.15.92:0/1049943731 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f5a4006ae40 0x7f5a4006d300 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:50.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:50.968+0000 7f5a623c1640 1 --2- 172.21.15.92:0/1049943731 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5a5c06b600 0x7f5a5c06ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:50.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:50.968+0000 7f5a623c1640 1 --2- 172.21.15.92:0/1049943731 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5a5c14a430 0x7f5a5c1f68e0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:50.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:50.968+0000 7f5a623c1640 1 --2- 172.21.15.92:0/1049943731 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5a480047c0 0x7f5a5c0b8340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:50.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:50.968+0000 7f5a623c1640 1 -- 172.21.15.92:0/1049943731 >> 172.21.15.92:0/1049943731 conn(0x7f5a5c0d2fc0 msgr2=0x7f5a5c15d890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:50.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:50.968+0000 7f5a623c1640 1 -- 172.21.15.92:0/1049943731 shutdown_connections 2023-11-24T06:11:50.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:50.968+0000 7f5a623c1640 1 -- 172.21.15.92:0/1049943731 wait complete. 2023-11-24T06:11:50.989 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T06:11:51.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:51.185+0000 7f8500c97640 1 Processor -- start 2023-11-24T06:11:51.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:51.185+0000 7f8500c97640 1 -- start start 2023-11-24T06:11:51.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:51.187+0000 7f8500c97640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f84fc13df80 0x7f84fc15bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:51.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:51.187+0000 7f8500c97640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f84fc0cf9a0 0x7f84fc0cd840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:51.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:51.187+0000 7f8500c97640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84fc06c470 con 0x7f84fc13e360 2023-11-24T06:11:51.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:51.187+0000 7f8500c97640 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84fc06c750 con 0x7f84fc13df80 2023-11-24T06:11:51.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:51.187+0000 7f8500c97640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84fc06ca30 con 0x7f84fc0cf9a0 2023-11-24T06:11:51.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:51.187+0000 7f84f1d9b640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f84fc13e360 0x7f84fc0cf290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:41260/0 (socket says 172.21.15.92:41260) 2023-11-24T06:11:51.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:51.187+0000 7f84f1d9b640 1 -- 172.21.15.92:0/3768300176 learned_addr learned my addr 172.21.15.92:0/3768300176 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-24T06:11:51.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:51.187+0000 7f84fad9d640 1 --2- 172.21.15.92:0/3768300176 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f84fc0cf9a0 0x7f84fc0cd840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:51.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:51.187+0000 7f84fa59c640 1 --2- 172.21.15.92:0/3768300176 >> v2:172.21.15.118:3300/0 conn(0x7f84fc13df80 0x7f84fc15bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:51.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:51.187+0000 7f84f9d9b640 1 -- 172.21.15.92:0/3768300176 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2985975751 0 0) 0x7f84fc06c470 con 0x7f84fc13e360 2023-11-24T06:11:51.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:51.188+0000 7f84f9d9b640 1 -- 172.21.15.92:0/3768300176 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f84e8003520 con 0x7f84fc13e360 2023-11-24T06:11:51.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:51.188+0000 7f84f9d9b640 1 -- 172.21.15.92:0/3768300176 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 539192781 0 0) 0x7f84e8003520 con 0x7f84fc13e360 2023-11-24T06:11:51.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:51.188+0000 7f84f9d9b640 1 -- 172.21.15.92:0/3768300176 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f84fc0cf9a0 msgr2=0x7f84fc0cd840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:51.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:51.188+0000 7f84f9d9b640 1 --2- 172.21.15.92:0/3768300176 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f84fc0cf9a0 0x7f84fc0cd840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:51.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:51.188+0000 7f84f9d9b640 1 -- 172.21.15.92:0/3768300176 >> v2:172.21.15.118:3300/0 conn(0x7f84fc13df80 msgr2=0x7f84fc15bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:51.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:51.188+0000 7f84f9d9b640 1 --2- 172.21.15.92:0/3768300176 >> v2:172.21.15.118:3300/0 conn(0x7f84fc13df80 0x7f84fc15bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:51.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:51.188+0000 7f84f9d9b640 1 -- 172.21.15.92:0/3768300176 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84fc06cd10 con 0x7f84fc13e360 2023-11-24T06:11:51.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:51.188+0000 7f84f9d9b640 1 -- 172.21.15.92:0/3768300176 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2607661350 0 0) 0x7f84e4001670 con 0x7f84fc13e360 2023-11-24T06:11:51.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:51.188+0000 7f84f9d9b640 1 -- 172.21.15.92:0/3768300176 >> v1:172.21.15.92:6789/0 conn(0x7f84fc13e360 legacy=0x7f84fc0cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:51.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:51.188+0000 7f84f9d9b640 1 --2- 172.21.15.92:0/3768300176 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f84e8003970 0x7f84e8003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:51.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:51.188+0000 7f84f9d9b640 1 --2- 172.21.15.92:0/3768300176 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f84fc0cf9a0 0x7f84e80047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:51.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:51.188+0000 7f84f9d9b640 1 --2- 172.21.15.92:0/3768300176 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f84e8004d00 0x7f84e8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:51.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:51.188+0000 7f84f9d9b640 1 -- 172.21.15.92:0/3768300176 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84fc06c750 con 0x7f84e8003970 2023-11-24T06:11:51.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:51.188+0000 7f84f9d9b640 1 -- 172.21.15.92:0/3768300176 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84fc06ca30 con 0x7f84e8004d00 2023-11-24T06:11:51.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:51.188+0000 7f84f9d9b640 1 -- 172.21.15.92:0/3768300176 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84e800b980 con 0x7f84fc0cf9a0 2023-11-24T06:11:51.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:51.188+0000 7f84fa59c640 1 --2- 172.21.15.92:0/3768300176 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f84e8003970 0x7f84e8003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:51.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:51.189+0000 7f84fa59c640 1 -- 172.21.15.92:0/3768300176 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f84fc0cf9a0 msgr2=0x7f84e80047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:51.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:51.189+0000 7f84fa59c640 1 --2- 172.21.15.92:0/3768300176 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f84fc0cf9a0 0x7f84e80047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:51.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:51.189+0000 7f84fa59c640 1 -- 172.21.15.92:0/3768300176 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f84e8004d00 msgr2=0x7f84e8005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:51.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:51.189+0000 7f84fa59c640 1 --2- 172.21.15.92:0/3768300176 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f84e8004d00 0x7f84e8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:51.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:51.189+0000 7f84fa59c640 1 -- 172.21.15.92:0/3768300176 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84e8003520 con 0x7f84e8003970 2023-11-24T06:11:51.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:51.190+0000 7f84fa59c640 1 --2- 172.21.15.92:0/3768300176 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f84e8003970 0x7f84e8003d90 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f84ec009af0 tx=0x7f84e80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44f4af01a449a843 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:51.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:51.190+0000 7f84f9d9b640 1 -- 172.21.15.92:0/3768300176 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84ec02c710 con 0x7f84e8003970 2023-11-24T06:11:51.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:51.190+0000 7f84f9d9b640 1 -- 172.21.15.92:0/3768300176 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84ec02c870 con 0x7f84e8003970 2023-11-24T06:11:51.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:51.190+0000 7f84f9d9b640 1 -- 172.21.15.92:0/3768300176 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84ec02cc00 con 0x7f84e8003970 2023-11-24T06:11:51.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:51.190+0000 7f8500c97640 1 -- 172.21.15.92:0/3768300176 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f84e8003970 msgr2=0x7f84e8003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:51.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:51.190+0000 7f8500c97640 1 --2- 172.21.15.92:0/3768300176 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f84e8003970 0x7f84e8003d90 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f84ec009af0 tx=0x7f84e80057f0 comp rx=0 tx=0).stop 2023-11-24T06:11:51.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:51.190+0000 7f84fad9d640 1 -- 172.21.15.92:0/3768300176 reap_dead start 2023-11-24T06:11:51.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:51.191+0000 7f8500c97640 1 -- 172.21.15.92:0/3768300176 shutdown_connections 2023-11-24T06:11:51.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:51.191+0000 7f8500c97640 1 -- 172.21.15.92:0/3768300176 >> 172.21.15.92:0/3768300176 conn(0x7f84fc06a800 msgr2=0x7f84fc06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:51.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:51.191+0000 7f8500c97640 1 -- 172.21.15.92:0/3768300176 shutdown_connections 2023-11-24T06:11:51.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:51.191+0000 7f8500c97640 1 -- 172.21.15.92:0/3768300176 wait complete. 2023-11-24T06:11:51.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:51.192+0000 7f8500c97640 1 Processor -- start 2023-11-24T06:11:51.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:51.193+0000 7f8500c97640 1 -- start start 2023-11-24T06:11:51.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:51.194+0000 7f8500c97640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f84e8003970 0x7f84fc0ab990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:51.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:51.194+0000 7f8500c97640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f84fc135750 0x7f84fc0711e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:51.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:51.194+0000 7f8500c97640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f84fc071720 0x7f84fc15d060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:51.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:51.194+0000 7f8500c97640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84fc070670 con 0x7f84fc071720 2023-11-24T06:11:51.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:51.194+0000 7f8500c97640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84fc071ba0 con 0x7f84fc135750 2023-11-24T06:11:51.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:51.194+0000 7f8500c97640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84fc0a91c0 con 0x7f84e8003970 2023-11-24T06:11:51.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:51.194+0000 7f84fad9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f84fc071720 0x7f84fc15d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:51.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:51.194+0000 7f84fad9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f84fc071720 0x7f84fc15d060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58678/0 (socket says 172.21.15.92:58678) 2023-11-24T06:11:51.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:51.194+0000 7f84fad9d640 1 -- 172.21.15.92:0/3369912607 learned_addr learned my addr 172.21.15.92:0/3369912607 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:11:51.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:51.194+0000 7f84fa59c640 1 --2- 172.21.15.92:0/3369912607 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f84e8003970 0x7f84fc0ab990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:51.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:51.194+0000 7f84fad9d640 1 -- 172.21.15.92:0/3369912607 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f84e8003970 msgr2=0x7f84fc0ab990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:51.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:51.194+0000 7f84f1d9b640 1 --2- 172.21.15.92:0/3369912607 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f84fc135750 0x7f84fc0711e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:51.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:51.194+0000 7f84fad9d640 1 --2- 172.21.15.92:0/3369912607 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f84e8003970 0x7f84fc0ab990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:51.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:51.194+0000 7f84fad9d640 1 -- 172.21.15.92:0/3369912607 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f84fc135750 msgr2=0x7f84fc0711e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:51.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:51.194+0000 7f84fad9d640 1 --2- 172.21.15.92:0/3369912607 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f84fc135750 0x7f84fc0711e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:51.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:51.194+0000 7f84fad9d640 1 -- 172.21.15.92:0/3369912607 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84fc0a85f0 con 0x7f84fc071720 2023-11-24T06:11:51.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:51.195+0000 7f84fad9d640 1 --2- 172.21.15.92:0/3369912607 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f84fc071720 0x7f84fc15d060 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f84fc0cfea0 tx=0x7f84f400b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:51.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:51.195+0000 7f84f37fe640 1 -- 172.21.15.92:0/3369912607 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84f4010040 con 0x7f84fc071720 2023-11-24T06:11:51.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:51.195+0000 7f84f37fe640 1 -- 172.21.15.92:0/3369912607 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84f400ba80 con 0x7f84fc071720 2023-11-24T06:11:51.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:51.195+0000 7f84f37fe640 1 -- 172.21.15.92:0/3369912607 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84f400be50 con 0x7f84fc071720 2023-11-24T06:11:51.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:51.195+0000 7f8500c97640 1 -- 172.21.15.92:0/3369912607 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84fc0a8890 con 0x7f84fc071720 2023-11-24T06:11:51.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:51.195+0000 7f8500c97640 1 -- 172.21.15.92:0/3369912607 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84fc0a8e50 con 0x7f84fc071720 2023-11-24T06:11:51.201 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:51.196+0000 7f8500c97640 1 -- 172.21.15.92:0/3369912607 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84fc0675d0 con 0x7f84fc071720 2023-11-24T06:11:51.201 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:51.197+0000 7f84f37fe640 1 -- 172.21.15.92:0/3369912607 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f84f401d400 con 0x7f84fc071720 2023-11-24T06:11:51.201 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:51.198+0000 7f84f37fe640 1 --2- 172.21.15.92:0/3369912607 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f84dc06ae40 0x7f84dc06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:51.201 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:51.198+0000 7f84f37fe640 1 -- 172.21.15.92:0/3369912607 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f84f4089d40 con 0x7f84fc071720 2023-11-24T06:11:51.201 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:51.198+0000 7f84fa59c640 1 --2- 172.21.15.92:0/3369912607 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f84dc06ae40 0x7f84dc06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:51.202 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:51.199+0000 7f84fa59c640 1 --2- 172.21.15.92:0/3369912607 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f84dc06ae40 0x7f84dc06d300 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f84ec036000 tx=0x7f84ec030000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:51.206 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:51.203+0000 7f84f37fe640 1 -- 172.21.15.92:0/3369912607 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f84f4058870 con 0x7f84fc071720 2023-11-24T06:11:51.417 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:51.414+0000 7f8500c97640 1 -- 172.21.15.92:0/3369912607 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f84fc06df00 con 0x7f84fc071720 2023-11-24T06:11:51.418 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:51.415+0000 7f84f37fe640 1 -- 172.21.15.92:0/3369912607 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4919 (secure 0 0 0) 0x7f84f4058400 con 0x7f84fc071720 2023-11-24T06:11:51.418 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-24T06:11:51.418 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4637,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6834/2665242481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":2665242481},{"type":"v1","addr":"172.21.15.118:6834","nonce":2665242481}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":4640,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6835/2822230615","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6833","nonce":2822230615},{"type":"v1","addr":"172.21.15.118:6835","nonce":2822230615}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":4741,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6836/3889219158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":3889219158},{"type":"v1","addr":"172.21.15.92:6836","nonce":3889219158}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T06:11:44.503003+0000","modified":"2023-11-24T06:11:47.998422+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5235},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5235":{"gid":5235,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.92:6837/1413573590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":1413573590},{"type":"v1","addr":"172.21.15.92:6837","nonce":1413573590}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-24T06:11:51.418 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 17 2023-11-24T06:11:51.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:51.417+0000 7f8500c97640 1 -- 172.21.15.92:0/3369912607 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f84dc06ae40 msgr2=0x7f84dc06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:51.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:51.417+0000 7f8500c97640 1 --2- 172.21.15.92:0/3369912607 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f84dc06ae40 0x7f84dc06d300 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f84ec036000 tx=0x7f84ec030000 comp rx=0 tx=0).stop 2023-11-24T06:11:51.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:51.417+0000 7f8500c97640 1 -- 172.21.15.92:0/3369912607 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f84fc071720 msgr2=0x7f84fc15d060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:51.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:51.417+0000 7f8500c97640 1 --2- 172.21.15.92:0/3369912607 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f84fc071720 0x7f84fc15d060 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f84fc0cfea0 tx=0x7f84f400b480 comp rx=0 tx=0).stop 2023-11-24T06:11:51.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:51.417+0000 7f8500c97640 1 -- 172.21.15.92:0/3369912607 shutdown_connections 2023-11-24T06:11:51.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:51.417+0000 7f8500c97640 1 --2- 172.21.15.92:0/3369912607 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f84dc06ae40 0x7f84dc06d300 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:51.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:51.417+0000 7f8500c97640 1 --2- 172.21.15.92:0/3369912607 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f84fc071720 0x7f84fc15d060 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:51.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:51.417+0000 7f8500c97640 1 --2- 172.21.15.92:0/3369912607 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f84fc135750 0x7f84fc0711e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:51.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:51.417+0000 7f8500c97640 1 --2- 172.21.15.92:0/3369912607 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f84e8003970 0x7f84fc0ab990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:51.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:51.417+0000 7f8500c97640 1 -- 172.21.15.92:0/3369912607 >> 172.21.15.92:0/3369912607 conn(0x7f84fc06a800 msgr2=0x7f84fc06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:51.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:51.417+0000 7f8500c97640 1 -- 172.21.15.92:0/3369912607 shutdown_connections 2023-11-24T06:11:51.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:51.417+0000 7f8500c97640 1 -- 172.21.15.92:0/3369912607 wait complete. 2023-11-24T06:11:51.841 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-24T06:11:44.503003+0000', 'modified': '2023-11-24T06:11:47.998422+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5235}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5235': {'gid': 5235, 'name': 'c', 'rank': 0, 'incarnation': 13, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.92:6837/1413573590', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6835', 'nonce': 1413573590}, {'type': 'v1', 'addr': '172.21.15.92:6837', 'nonce': 1413573590}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [5], 'metadata_pool': 4, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-24T06:11:51.842 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-24T06:11:51.842 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-11-24T06:11:52.071 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.068+0000 7feedb16c640 1 Processor -- start 2023-11-24T06:11:52.071 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.068+0000 7feedb16c640 1 -- start start 2023-11-24T06:11:52.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.070+0000 7feedb16c640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7feed413cfa0 0x7feed415afb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:52.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.070+0000 7feedb16c640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7feed40cd530 0x7feed413f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:52.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.070+0000 7feedb16c640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feed40cdec0 con 0x7feed40cd9b0 2023-11-24T06:11:52.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.070+0000 7feedb16c640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feed40ce1a0 con 0x7feed413cfa0 2023-11-24T06:11:52.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.070+0000 7feedb16c640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feed40ce480 con 0x7feed40cd530 2023-11-24T06:11:52.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.070+0000 7feeda16a640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7feed413cfa0 0x7feed415afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:52.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.070+0000 7feeda16a640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7feed413cfa0 0x7feed415afb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58694/0 (socket says 172.21.15.92:58694) 2023-11-24T06:11:52.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.070+0000 7feeda16a640 1 -- 172.21.15.92:0/2580956408 learned_addr learned my addr 172.21.15.92:0/2580956408 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:11:52.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.071+0000 7feed9969640 1 --2- 172.21.15.92:0/2580956408 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7feed40cd530 0x7feed413f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:52.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.071+0000 7feeda16a640 1 -- 172.21.15.92:0/2580956408 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7feed40cd530 msgr2=0x7feed413f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:52.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.071+0000 7feeda16a640 1 --2- 172.21.15.92:0/2580956408 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7feed40cd530 0x7feed413f5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:52.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.071+0000 7feeda16a640 1 -- 172.21.15.92:0/2580956408 >> v1:172.21.15.92:6789/0 conn(0x7feed40cd9b0 legacy=0x7feed415d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:52.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.071+0000 7feeda16a640 1 -- 172.21.15.92:0/2580956408 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feed40ce760 con 0x7feed413cfa0 2023-11-24T06:11:52.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.072+0000 7feeda16a640 1 --2- 172.21.15.92:0/2580956408 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feed413cfa0 0x7feed415afb0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7feec4009870 tx=0x7feec402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6544029192fae2ad server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:52.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.072+0000 7feed9168640 1 -- 172.21.15.92:0/2580956408 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feec4031640 con 0x7feed413cfa0 2023-11-24T06:11:52.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.072+0000 7feed9168640 1 -- 172.21.15.92:0/2580956408 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feec40317a0 con 0x7feed413cfa0 2023-11-24T06:11:52.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.072+0000 7feed9168640 1 -- 172.21.15.92:0/2580956408 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feec4031b30 con 0x7feed413cfa0 2023-11-24T06:11:52.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.073+0000 7feedb16c640 1 -- 172.21.15.92:0/2580956408 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feed413cfa0 msgr2=0x7feed415afb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:52.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.073+0000 7feedb16c640 1 --2- 172.21.15.92:0/2580956408 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feed413cfa0 0x7feed415afb0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7feec4009870 tx=0x7feec402b8f0 comp rx=0 tx=0).stop 2023-11-24T06:11:52.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.073+0000 7feedb16c640 1 -- 172.21.15.92:0/2580956408 shutdown_connections 2023-11-24T06:11:52.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.073+0000 7feedb16c640 1 --2- 172.21.15.92:0/2580956408 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7feed40cd530 0x7feed413f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:52.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.073+0000 7feedb16c640 1 --2- 172.21.15.92:0/2580956408 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feed413cfa0 0x7feed415afb0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:52.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.073+0000 7feedb16c640 1 -- 172.21.15.92:0/2580956408 >> 172.21.15.92:0/2580956408 conn(0x7feed406a800 msgr2=0x7feed406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:52.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.073+0000 7feedb16c640 1 -- 172.21.15.92:0/2580956408 shutdown_connections 2023-11-24T06:11:52.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.073+0000 7feedb16c640 1 -- 172.21.15.92:0/2580956408 wait complete. 2023-11-24T06:11:52.081 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.075+0000 7feedb16c640 1 Processor -- start 2023-11-24T06:11:52.081 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.075+0000 7feedb16c640 1 -- start start 2023-11-24T06:11:52.081 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.076+0000 7feedb16c640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7feed40cd530 0x7feed40ca080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:52.081 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.076+0000 7feedb16c640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7feed40cd9b0 0x7feed41f7620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:52.082 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.076+0000 7feedb16c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feed413cfa0 0x7feed413c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:52.082 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.076+0000 7feedb16c640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feed41f7b60 con 0x7feed413cfa0 2023-11-24T06:11:52.082 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.076+0000 7feedb16c640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feed41e53e0 con 0x7feed40cd530 2023-11-24T06:11:52.082 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.076+0000 7feedb16c640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feed41e59a0 con 0x7feed40cd9b0 2023-11-24T06:11:52.082 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.077+0000 7feeda96b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feed413cfa0 0x7feed413c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:52.082 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.077+0000 7feeda96b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feed413cfa0 0x7feed413c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58704/0 (socket says 172.21.15.92:58704) 2023-11-24T06:11:52.083 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.077+0000 7feeda96b640 1 -- 172.21.15.92:0/1512996077 learned_addr learned my addr 172.21.15.92:0/1512996077 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:11:52.083 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.077+0000 7feeda16a640 1 --2- 172.21.15.92:0/1512996077 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7feed40cd530 0x7feed40ca080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:52.083 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.077+0000 7feed9969640 1 --2- 172.21.15.92:0/1512996077 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7feed40cd9b0 0x7feed41f7620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:52.083 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.077+0000 7feeda96b640 1 -- 172.21.15.92:0/1512996077 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7feed40cd9b0 msgr2=0x7feed41f7620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:52.083 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.077+0000 7feeda96b640 1 --2- 172.21.15.92:0/1512996077 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7feed40cd9b0 0x7feed41f7620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:52.083 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.077+0000 7feeda96b640 1 -- 172.21.15.92:0/1512996077 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7feed40cd530 msgr2=0x7feed40ca080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:52.084 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.077+0000 7feeda96b640 1 --2- 172.21.15.92:0/1512996077 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7feed40cd530 0x7feed40ca080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:52.084 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.077+0000 7feeda96b640 1 -- 172.21.15.92:0/1512996077 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feec40094d0 con 0x7feed413cfa0 2023-11-24T06:11:52.084 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.077+0000 7feeda16a640 1 --2- 172.21.15.92:0/1512996077 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7feed40cd530 0x7feed40ca080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:11:52.084 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.077+0000 7feeda96b640 1 --2- 172.21.15.92:0/1512996077 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feed413cfa0 0x7feed413c580 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7feecc00c7d0 tx=0x7feecc00cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:52.084 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.078+0000 7feed2ffd640 1 -- 172.21.15.92:0/1512996077 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feecc003b70 con 0x7feed413cfa0 2023-11-24T06:11:52.084 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.078+0000 7feed2ffd640 1 -- 172.21.15.92:0/1512996077 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feecc003cf0 con 0x7feed413cfa0 2023-11-24T06:11:52.084 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.078+0000 7feed2ffd640 1 -- 172.21.15.92:0/1512996077 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feecc00aa30 con 0x7feed413cfa0 2023-11-24T06:11:52.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.078+0000 7feedb16c640 1 -- 172.21.15.92:0/1512996077 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feed40ca6d0 con 0x7feed413cfa0 2023-11-24T06:11:52.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.078+0000 7feedb16c640 1 -- 172.21.15.92:0/1512996077 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feed41e5630 con 0x7feed413cfa0 2023-11-24T06:11:52.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.080+0000 7feed2ffd640 1 -- 172.21.15.92:0/1512996077 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7feecc00ab90 con 0x7feed413cfa0 2023-11-24T06:11:52.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.081+0000 7feed2ffd640 1 --2- 172.21.15.92:0/1512996077 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7feec006ae40 0x7feec006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:52.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.081+0000 7feed2ffd640 1 -- 172.21.15.92:0/1512996077 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7feecc089450 con 0x7feed413cfa0 2023-11-24T06:11:52.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.081+0000 7feeda16a640 1 --2- 172.21.15.92:0/1512996077 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7feec006ae40 0x7feec006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:52.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.082+0000 7feedef64740 1 Processor -- start 2023-11-24T06:11:52.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.082+0000 7feeda16a640 1 --2- 172.21.15.92:0/1512996077 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7feec006ae40 0x7feec006d300 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7feec4036000 tx=0x7feec403c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:52.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.082+0000 7feedef64740 1 -- start start 2023-11-24T06:11:52.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.083+0000 7feedef64740 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x55a67c845070 0x55a67c845450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:52.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.083+0000 7feedef64740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a67c83fd70 0x55a67c844270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:52.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.083+0000 7feedef64740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a67c8447b0 0x55a67c888730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:52.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.083+0000 7feedef64740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a67c7a3880 con 0x55a67c83fd70 2023-11-24T06:11:52.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.083+0000 7feedef64740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a67c7b2ae0 con 0x55a67c8447b0 2023-11-24T06:11:52.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.083+0000 7feedef64740 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a67c7b01d0 con 0x55a67c845070 2023-11-24T06:11:52.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.083+0000 7feed9969640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a67c83fd70 0x55a67c844270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:52.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.083+0000 7feed9969640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a67c83fd70 0x55a67c844270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58712/0 (socket says 172.21.15.92:58712) 2023-11-24T06:11:52.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.083+0000 7feed9969640 1 -- 172.21.15.92:0/9123569 learned_addr learned my addr 172.21.15.92:0/9123569 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:11:52.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.084+0000 7feeda16a640 1 --2- 172.21.15.92:0/9123569 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x55a67c845070 0x55a67c845450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:52.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.084+0000 7feed9969640 1 -- 172.21.15.92:0/9123569 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x55a67c845070 msgr2=0x55a67c845450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:52.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.084+0000 7feed9969640 1 --2- 172.21.15.92:0/9123569 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x55a67c845070 0x55a67c845450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:52.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.084+0000 7feed9969640 1 -- 172.21.15.92:0/9123569 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a67c8447b0 msgr2=0x55a67c888730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:52.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.084+0000 7feed9969640 1 --2- 172.21.15.92:0/9123569 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a67c8447b0 0x55a67c888730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:52.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.084+0000 7feed9969640 1 -- 172.21.15.92:0/9123569 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a67c773860 con 0x55a67c83fd70 2023-11-24T06:11:52.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.084+0000 7feed9969640 1 --2- 172.21.15.92:0/9123569 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a67c83fd70 0x55a67c844270 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7feec800b850 tx=0x7feec800bd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68c30539140d612e server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:52.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.084+0000 7feebbfff640 1 -- 172.21.15.92:0/9123569 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feec800dbb0 con 0x55a67c83fd70 2023-11-24T06:11:52.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.084+0000 7feebbfff640 1 -- 172.21.15.92:0/9123569 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feec800c040 con 0x55a67c83fd70 2023-11-24T06:11:52.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.084+0000 7feebbfff640 1 -- 172.21.15.92:0/9123569 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feec8012460 con 0x55a67c83fd70 2023-11-24T06:11:52.095 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.085+0000 7feedef64740 1 -- 172.21.15.92:0/9123569 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a67c83fd70 msgr2=0x55a67c844270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:52.095 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.085+0000 7feedef64740 1 --2- 172.21.15.92:0/9123569 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a67c83fd70 0x55a67c844270 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7feec800b850 tx=0x7feec800bd10 comp rx=0 tx=0).stop 2023-11-24T06:11:52.095 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.085+0000 7feedef64740 1 -- 172.21.15.92:0/9123569 shutdown_connections 2023-11-24T06:11:52.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.085+0000 7feedef64740 1 --2- 172.21.15.92:0/9123569 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a67c8447b0 0x55a67c888730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:52.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.085+0000 7feedef64740 1 --2- 172.21.15.92:0/9123569 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a67c83fd70 0x55a67c844270 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:52.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.085+0000 7feedef64740 1 --2- 172.21.15.92:0/9123569 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x55a67c845070 0x55a67c845450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:52.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.085+0000 7feedef64740 1 -- 172.21.15.92:0/9123569 >> 172.21.15.92:0/9123569 conn(0x55a67c806c70 msgr2=0x55a67c88ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:52.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.085+0000 7feedef64740 1 -- 172.21.15.92:0/9123569 shutdown_connections 2023-11-24T06:11:52.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.085+0000 7feedef64740 1 -- 172.21.15.92:0/9123569 wait complete. 2023-11-24T06:11:52.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.086+0000 7feedef64740 1 -- start start 2023-11-24T06:11:52.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.086+0000 7feedef64740 1 Processor -- start 2023-11-24T06:11:52.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.087+0000 7feedef64740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a67c83fd70 0x55a67c86aea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:52.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.087+0000 7feedef64740 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x55a67c8447b0 0x55a67c86ef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:52.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.087+0000 7feedef64740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a67c845070 0x55a67c86f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:52.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.087+0000 7feedef64740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a67c7abac0 con 0x55a67c83fd70 2023-11-24T06:11:52.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.087+0000 7feedef64740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a67c7482f0 con 0x55a67c845070 2023-11-24T06:11:52.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.087+0000 7feedef64740 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a67c765c70 con 0x55a67c8447b0 2023-11-24T06:11:52.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.087+0000 7feedef64740 10 client.0 _ll_register_callbacks cb 0x55a67c76a3b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-24T06:11:52.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.087+0000 7feeda16a640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a67c83fd70 0x55a67c86aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:52.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.088+0000 7feeda16a640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a67c83fd70 0x55a67c86aea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58716/0 (socket says 172.21.15.92:58716) 2023-11-24T06:11:52.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.088+0000 7feeda16a640 1 -- 172.21.15.92:0/3405769121 learned_addr learned my addr 172.21.15.92:0/3405769121 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:11:52.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.088+0000 7feeda96b640 1 --2- 172.21.15.92:0/3405769121 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a67c845070 0x55a67c86f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:52.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.088+0000 7feed9969640 1 --2- 172.21.15.92:0/3405769121 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x55a67c8447b0 0x55a67c86ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:52.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.088+0000 7feeda16a640 1 -- 172.21.15.92:0/3405769121 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x55a67c8447b0 msgr2=0x55a67c86ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:52.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.088+0000 7feeda16a640 1 --2- 172.21.15.92:0/3405769121 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x55a67c8447b0 0x55a67c86ef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:52.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.088+0000 7feeda16a640 1 -- 172.21.15.92:0/3405769121 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a67c845070 msgr2=0x55a67c86f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:52.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.088+0000 7feeda16a640 1 --2- 172.21.15.92:0/3405769121 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a67c845070 0x55a67c86f4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:52.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.088+0000 7feeda16a640 1 -- 172.21.15.92:0/3405769121 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feec800b450 con 0x55a67c83fd70 2023-11-24T06:11:52.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.088+0000 7feeda16a640 1 --2- 172.21.15.92:0/3405769121 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a67c83fd70 0x55a67c86aea0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7feec4009990 tx=0x7feec4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cf85873f50e537e server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:52.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.088+0000 7feeb97fa640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-11-24T06:11:52.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.089+0000 7feeb97fa640 1 -- 172.21.15.92:0/3405769121 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feec403ae30 con 0x55a67c83fd70 2023-11-24T06:11:52.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.089+0000 7feeb97fa640 1 -- 172.21.15.92:0/3405769121 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7feec4050af0 con 0x55a67c83fd70 2023-11-24T06:11:52.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.089+0000 7feedef64740 1 -- 172.21.15.92:0/3405769121 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a67c7a4aa0 con 0x55a67c83fd70 2023-11-24T06:11:52.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.089+0000 7feeb97fa640 1 -- 172.21.15.92:0/3405769121 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7feec4050cb0 con 0x55a67c83fd70 2023-11-24T06:11:52.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.089+0000 7feedef64740 10 client.5418 fetch_fsmap learned FSMap version 17 2023-11-24T06:11:52.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.089+0000 7feedef64740 1 -- 172.21.15.92:0/3405769121 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) v3 -- 0x55a67c7aa140 con 0x55a67c83fd70 2023-11-24T06:11:52.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.089+0000 7feeb97fa640 1 -- 172.21.15.92:0/3405769121 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a67c7a4aa0 con 0x55a67c83fd70 2023-11-24T06:11:52.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.089+0000 7feeb97fa640 1 -- 172.21.15.92:0/3405769121 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 17) v1 ==== 3294+0+0 (secure 0 0 0) 0x7feec4045610 con 0x55a67c83fd70 2023-11-24T06:11:52.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.089+0000 7feedef64740 10 client.5418 fetch_fsmap finished waiting for FSMap version 17 2023-11-24T06:11:52.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.089+0000 7feedef64740 10 client.5418 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-24T06:11:52.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.089+0000 7feedef64740 20 client.5418 populate_metadata read hostname 'smithi092' 2023-11-24T06:11:52.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.089+0000 7feedef64740 1 --2- 172.21.15.92:0/3405769121 >> [v2:172.21.15.92:6835/1413573590,v1:172.21.15.92:6837/1413573590] conn(0x55a67c8717b0 0x55a67c86d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:52.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.089+0000 7feedef64740 4 client.5418 mds_command: new command op to 5235 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-24T06:11:52.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.089+0000 7feedef64740 1 -- 172.21.15.92:0/3405769121 --> [v2:172.21.15.92:6835/1413573590,v1:172.21.15.92:6837/1413573590] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a67c7a47c0 con 0x55a67c8717b0 2023-11-24T06:11:52.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.090+0000 7feed9969640 1 --2- 172.21.15.92:0/3405769121 >> [v2:172.21.15.92:6835/1413573590,v1:172.21.15.92:6837/1413573590] conn(0x55a67c8717b0 0x55a67c86d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:52.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.090+0000 7feed9969640 1 --2- 172.21.15.92:0/3405769121 >> [v2:172.21.15.92:6835/1413573590,v1:172.21.15.92:6837/1413573590] conn(0x55a67c8717b0 0x55a67c86d4a0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e1f9fc513e641c58 server_cookie=cdd90ccc124d7a9b in_seq=0 out_seq=0 2023-11-24T06:11:52.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.090+0000 7feeb97fa640 10 client.5418 ms_handle_connect on v2:172.21.15.92:6835/1413573590 2023-11-24T06:11:52.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.091+0000 7feeb97fa640 1 -- 172.21.15.92:0/3405769121 <== mds.0 v2:172.21.15.92:6835/1413573590 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55a67c7a47c0 con 0x55a67c8717b0 2023-11-24T06:11:52.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.091+0000 7feeb97fa640 10 client.5418 handle_command_reply: tid=0 2023-11-24T06:11:52.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.091+0000 7feedef64740 1 client.5418 shutdown 2023-11-24T06:11:52.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.091+0000 7feedef64740 1 -- 172.21.15.92:0/3405769121 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a67c83fd70 msgr2=0x55a67c86aea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:52.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.091+0000 7feedef64740 1 --2- 172.21.15.92:0/3405769121 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a67c83fd70 0x55a67c86aea0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7feec4009990 tx=0x7feec4004730 comp rx=0 tx=0).stop 2023-11-24T06:11:52.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.091+0000 7feedef64740 1 -- 172.21.15.92:0/3405769121 shutdown_connections 2023-11-24T06:11:52.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.091+0000 7feedef64740 1 --2- 172.21.15.92:0/3405769121 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a67c845070 0x55a67c86f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:52.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.091+0000 7feedef64740 1 --2- 172.21.15.92:0/3405769121 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x55a67c8447b0 0x55a67c86ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:52.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.091+0000 7feedef64740 1 --2- 172.21.15.92:0/3405769121 >> [v2:172.21.15.92:6835/1413573590,v1:172.21.15.92:6837/1413573590] conn(0x55a67c8717b0 0x55a67c86d4a0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:52.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.091+0000 7feedef64740 1 --2- 172.21.15.92:0/3405769121 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a67c83fd70 0x55a67c86aea0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:52.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.091+0000 7feedef64740 1 -- 172.21.15.92:0/3405769121 >> 172.21.15.92:0/3405769121 conn(0x55a67c806c70 msgr2=0x55a67c83edd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:52.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.091+0000 7feeb97fa640 0 client.5418 ms_handle_reset on v2:172.21.15.92:6835/1413573590 2023-11-24T06:11:52.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.091+0000 7feedef64740 1 -- 172.21.15.92:0/3405769121 shutdown_connections 2023-11-24T06:11:52.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.091+0000 7feedef64740 1 -- 172.21.15.92:0/3405769121 wait complete. 2023-11-24T06:11:52.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.092+0000 7feedef64740 20 client.5418 trim_cache size 0 max 16384 2023-11-24T06:11:52.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.108+0000 7feedef64740 1 Processor -- start 2023-11-24T06:11:52.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.108+0000 7feedef64740 1 -- start start 2023-11-24T06:11:52.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.109+0000 7feedef64740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a67c83fd70 0x55a67c887950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:52.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.109+0000 7feedef64740 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x55a67c8447b0 0x55a67c8924d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:52.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.109+0000 7feedef64740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a67c845070 0x55a67c892a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:52.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.109+0000 7feedef64740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a67c50ef20 con 0x55a67c845070 2023-11-24T06:11:52.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.109+0000 7feedef64740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a67c83ab00 con 0x55a67c83fd70 2023-11-24T06:11:52.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.109+0000 7feedef64740 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a67c86d4e0 con 0x55a67c8447b0 2023-11-24T06:11:52.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.109+0000 7feeda96b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a67c845070 0x55a67c892a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:52.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.109+0000 7feeda96b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a67c845070 0x55a67c892a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58722/0 (socket says 172.21.15.92:58722) 2023-11-24T06:11:52.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.109+0000 7feeda96b640 1 -- 172.21.15.92:0/559505759 learned_addr learned my addr 172.21.15.92:0/559505759 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:11:52.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.109+0000 7feeda16a640 1 --2- 172.21.15.92:0/559505759 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a67c83fd70 0x55a67c887950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:52.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.110+0000 7feed9969640 1 --2- 172.21.15.92:0/559505759 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x55a67c8447b0 0x55a67c8924d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:52.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.110+0000 7feeda96b640 1 -- 172.21.15.92:0/559505759 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x55a67c8447b0 msgr2=0x55a67c8924d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:52.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.110+0000 7feeda96b640 1 --2- 172.21.15.92:0/559505759 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x55a67c8447b0 0x55a67c8924d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:52.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.110+0000 7feeda96b640 1 -- 172.21.15.92:0/559505759 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a67c83fd70 msgr2=0x55a67c887950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:52.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.110+0000 7feeda96b640 1 --2- 172.21.15.92:0/559505759 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a67c83fd70 0x55a67c887950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:52.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.110+0000 7feeda96b640 1 -- 172.21.15.92:0/559505759 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a67c747b80 con 0x55a67c845070 2023-11-24T06:11:52.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.110+0000 7feeda96b640 1 --2- 172.21.15.92:0/559505759 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a67c845070 0x55a67c892a10 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7feecc05ff80 tx=0x7feecc011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbce4d92db577353 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:52.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.110+0000 7feed0dff640 1 -- 172.21.15.92:0/559505759 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feecc0588b0 con 0x55a67c845070 2023-11-24T06:11:52.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.110+0000 7feed0dff640 1 -- 172.21.15.92:0/559505759 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feecc0581b0 con 0x55a67c845070 2023-11-24T06:11:52.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.110+0000 7feed0dff640 1 -- 172.21.15.92:0/559505759 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feecc05c300 con 0x55a67c845070 2023-11-24T06:11:52.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.111+0000 7feedef64740 1 -- 172.21.15.92:0/559505759 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a67c845070 msgr2=0x55a67c892a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:52.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.111+0000 7feedef64740 1 --2- 172.21.15.92:0/559505759 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a67c845070 0x55a67c892a10 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7feecc05ff80 tx=0x7feecc011000 comp rx=0 tx=0).stop 2023-11-24T06:11:52.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.111+0000 7feedef64740 1 -- 172.21.15.92:0/559505759 shutdown_connections 2023-11-24T06:11:52.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.111+0000 7feedef64740 1 --2- 172.21.15.92:0/559505759 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a67c845070 0x55a67c892a10 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:52.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.111+0000 7feedef64740 1 --2- 172.21.15.92:0/559505759 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x55a67c8447b0 0x55a67c8924d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:52.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.111+0000 7feedef64740 1 --2- 172.21.15.92:0/559505759 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a67c83fd70 0x55a67c887950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:52.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.111+0000 7feedef64740 1 -- 172.21.15.92:0/559505759 >> 172.21.15.92:0/559505759 conn(0x55a67c806c70 msgr2=0x55a67c845450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:52.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.111+0000 7feedef64740 1 -- 172.21.15.92:0/559505759 shutdown_connections 2023-11-24T06:11:52.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.111+0000 7feedef64740 1 -- 172.21.15.92:0/559505759 wait complete. 2023-11-24T06:11:52.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.111+0000 7feedef64740 1 -- start start 2023-11-24T06:11:52.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.112+0000 7feedef64740 1 Processor -- start 2023-11-24T06:11:52.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.113+0000 7feedef64740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a67c83fd70 0x55a67c8fe340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:52.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.113+0000 7feedef64740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a67c8447b0 0x55a67c86f100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:52.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.113+0000 7feedef64740 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x55a67c845070 0x55a67c8fad80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:52.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.113+0000 7feedef64740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a67c899c30 con 0x55a67c8447b0 2023-11-24T06:11:52.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.113+0000 7feedef64740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a67c88ea10 con 0x55a67c83fd70 2023-11-24T06:11:52.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.113+0000 7feedef64740 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a67c899700 con 0x55a67c845070 2023-11-24T06:11:52.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.113+0000 7feedef64740 10 client.0 _ll_register_callbacks cb 0x7feec40096f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-24T06:11:52.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.113+0000 7feed9969640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a67c8447b0 0x55a67c86f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:52.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.113+0000 7feed9969640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a67c8447b0 0x55a67c86f100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58732/0 (socket says 172.21.15.92:58732) 2023-11-24T06:11:52.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.113+0000 7feed9969640 1 -- 172.21.15.92:0/2168817526 learned_addr learned my addr 172.21.15.92:0/2168817526 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:11:52.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.113+0000 7feed9969640 1 -- 172.21.15.92:0/2168817526 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x55a67c845070 msgr2=0x55a67c8fad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:52.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.113+0000 7feed9969640 1 --2- 172.21.15.92:0/2168817526 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x55a67c845070 0x55a67c8fad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:52.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.113+0000 7feed9969640 1 -- 172.21.15.92:0/2168817526 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a67c83fd70 msgr2=0x55a67c8fe340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:52.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.113+0000 7feed9969640 1 --2- 172.21.15.92:0/2168817526 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a67c83fd70 0x55a67c8fe340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:52.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.113+0000 7feed9969640 1 -- 172.21.15.92:0/2168817526 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feed41e5630 con 0x55a67c8447b0 2023-11-24T06:11:52.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.113+0000 7feed9969640 1 --2- 172.21.15.92:0/2168817526 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a67c8447b0 0x55a67c86f100 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7feec8000c00 tx=0x7feec8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61b038bce9fa886f server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:52.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.113+0000 7feeb97fa640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-11-24T06:11:52.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.114+0000 7feeb97fa640 1 -- 172.21.15.92:0/2168817526 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feec800dee0 con 0x55a67c8447b0 2023-11-24T06:11:52.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.114+0000 7feeb97fa640 1 -- 172.21.15.92:0/2168817526 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7feec8020040 con 0x55a67c8447b0 2023-11-24T06:11:52.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.114+0000 7feedef64740 1 -- 172.21.15.92:0/2168817526 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7feec800c040 con 0x55a67c8447b0 2023-11-24T06:11:52.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.114+0000 7feeb97fa640 1 -- 172.21.15.92:0/2168817526 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7feec8004240 con 0x55a67c8447b0 2023-11-24T06:11:52.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.114+0000 7feedef64740 10 client.5424 fetch_fsmap learned FSMap version 17 2023-11-24T06:11:52.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.114+0000 7feedef64740 1 -- 172.21.15.92:0/2168817526 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) v3 -- 0x55a67c76d170 con 0x55a67c8447b0 2023-11-24T06:11:52.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.114+0000 7feeb97fa640 1 -- 172.21.15.92:0/2168817526 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feec800c040 con 0x55a67c8447b0 2023-11-24T06:11:52.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.114+0000 7feeb97fa640 1 -- 172.21.15.92:0/2168817526 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 17) v1 ==== 3294+0+0 (secure 0 0 0) 0x7feec80043a0 con 0x55a67c8447b0 2023-11-24T06:11:52.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.114+0000 7feedef64740 10 client.5424 fetch_fsmap finished waiting for FSMap version 17 2023-11-24T06:11:52.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.114+0000 7feedef64740 10 client.5424 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-24T06:11:52.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.114+0000 7feedef64740 20 client.5424 populate_metadata read hostname 'smithi092' 2023-11-24T06:11:52.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.114+0000 7feedef64740 1 --2- 172.21.15.92:0/2168817526 >> [v2:172.21.15.92:6835/1413573590,v1:172.21.15.92:6837/1413573590] conn(0x55a67c88dfe0 0x55a67c88e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:52.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.114+0000 7feedef64740 4 client.5424 mds_command: new command op to 5235 tid=0 multi_id=0 [{"prefix": "status"}] 2023-11-24T06:11:52.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.114+0000 7feedef64740 1 -- 172.21.15.92:0/2168817526 --> [v2:172.21.15.92:6835/1413573590,v1:172.21.15.92:6837/1413573590] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55a67c4ac6c0 con 0x55a67c88dfe0 2023-11-24T06:11:52.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.115+0000 7feeda16a640 1 --2- 172.21.15.92:0/2168817526 >> [v2:172.21.15.92:6835/1413573590,v1:172.21.15.92:6837/1413573590] conn(0x55a67c88dfe0 0x55a67c88e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:52.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.115+0000 7feeda16a640 1 --2- 172.21.15.92:0/2168817526 >> [v2:172.21.15.92:6835/1413573590,v1:172.21.15.92:6837/1413573590] conn(0x55a67c88dfe0 0x55a67c88e3c0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ec8beef17bbc6b15 server_cookie=7689750c9aa2fcf2 in_seq=0 out_seq=0 2023-11-24T06:11:52.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.115+0000 7feeb97fa640 10 client.5424 ms_handle_connect on v2:172.21.15.92:6835/1413573590 2023-11-24T06:11:52.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.115+0000 7feeb97fa640 1 -- 172.21.15.92:0/2168817526 <== mds.0 v2:172.21.15.92:6835/1413573590 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x55a67c4ac6c0 con 0x55a67c88dfe0 2023-11-24T06:11:52.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.115+0000 7feeb97fa640 10 client.5424 handle_command_reply: tid=0 2023-11-24T06:11:52.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.115+0000 7feedef64740 1 client.5424 shutdown 2023-11-24T06:11:52.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.115+0000 7feedef64740 1 -- 172.21.15.92:0/2168817526 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a67c8447b0 msgr2=0x55a67c86f100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:52.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.115+0000 7feedef64740 1 --2- 172.21.15.92:0/2168817526 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a67c8447b0 0x55a67c86f100 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7feec8000c00 tx=0x7feec8004730 comp rx=0 tx=0).stop 2023-11-24T06:11:52.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.116+0000 7feedef64740 1 -- 172.21.15.92:0/2168817526 shutdown_connections 2023-11-24T06:11:52.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.116+0000 7feedef64740 1 --2- 172.21.15.92:0/2168817526 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x55a67c845070 0x55a67c8fad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:52.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.116+0000 7feedef64740 1 --2- 172.21.15.92:0/2168817526 >> [v2:172.21.15.92:6835/1413573590,v1:172.21.15.92:6837/1413573590] conn(0x55a67c88dfe0 0x55a67c88e3c0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:52.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.116+0000 7feedef64740 1 --2- 172.21.15.92:0/2168817526 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a67c8447b0 0x55a67c86f100 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:52.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.116+0000 7feedef64740 1 --2- 172.21.15.92:0/2168817526 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a67c83fd70 0x55a67c8fe340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:52.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.116+0000 7feeb97fa640 0 client.5424 ms_handle_reset on v2:172.21.15.92:6835/1413573590 2023-11-24T06:11:52.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.116+0000 7feedef64740 1 -- 172.21.15.92:0/2168817526 >> 172.21.15.92:0/2168817526 conn(0x55a67c806c70 msgr2=0x55a67c83b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:52.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.116+0000 7feedef64740 1 -- 172.21.15.92:0/2168817526 shutdown_connections 2023-11-24T06:11:52.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.116+0000 7feedef64740 1 -- 172.21.15.92:0/2168817526 wait complete. 2023-11-24T06:11:52.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.116+0000 7feedef64740 20 client.5424 trim_cache size 0 max 16384 2023-11-24T06:11:52.125 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-24T06:11:52.125 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_fsid": "2d6dee5f-8aa3-470f-931a-50ade963b1cb", 2023-11-24T06:11:52.125 INFO:teuthology.orchestra.run.smithi092.stdout: "whoami": 0, 2023-11-24T06:11:52.125 INFO:teuthology.orchestra.run.smithi092.stdout: "id": 5235, 2023-11-24T06:11:52.125 INFO:teuthology.orchestra.run.smithi092.stdout: "want_state": "up:active", 2023-11-24T06:11:52.125 INFO:teuthology.orchestra.run.smithi092.stdout: "state": "up:active", 2023-11-24T06:11:52.125 INFO:teuthology.orchestra.run.smithi092.stdout: "fs_name": "cephfs", 2023-11-24T06:11:52.126 INFO:teuthology.orchestra.run.smithi092.stdout: "rank_uptime": 6.6283883320000001, 2023-11-24T06:11:52.126 INFO:teuthology.orchestra.run.smithi092.stdout: "mdsmap_epoch": 17, 2023-11-24T06:11:52.126 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_epoch": 33, 2023-11-24T06:11:52.126 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_epoch_barrier": 33, 2023-11-24T06:11:52.126 INFO:teuthology.orchestra.run.smithi092.stdout: "uptime": 15.886436002 2023-11-24T06:11:52.126 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-24T06:11:52.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.116+0000 7feedb16c640 1 -- 172.21.15.92:0/1512996077 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7feec006ae40 msgr2=0x7feec006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:52.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.116+0000 7feedb16c640 1 --2- 172.21.15.92:0/1512996077 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7feec006ae40 0x7feec006d300 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7feec4036000 tx=0x7feec403c040 comp rx=0 tx=0).stop 2023-11-24T06:11:52.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.116+0000 7feedb16c640 1 -- 172.21.15.92:0/1512996077 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feed413cfa0 msgr2=0x7feed413c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:52.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.116+0000 7feedb16c640 1 --2- 172.21.15.92:0/1512996077 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feed413cfa0 0x7feed413c580 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7feecc00c7d0 tx=0x7feecc00cc90 comp rx=0 tx=0).stop 2023-11-24T06:11:52.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.116+0000 7feedb16c640 1 -- 172.21.15.92:0/1512996077 shutdown_connections 2023-11-24T06:11:52.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.116+0000 7feedb16c640 1 --2- 172.21.15.92:0/1512996077 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7feec006ae40 0x7feec006d300 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:52.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.117+0000 7feedb16c640 1 --2- 172.21.15.92:0/1512996077 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feed413cfa0 0x7feed413c580 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:52.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.117+0000 7feedb16c640 1 --2- 172.21.15.92:0/1512996077 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7feed40cd9b0 0x7feed41f7620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:52.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.117+0000 7feedb16c640 1 --2- 172.21.15.92:0/1512996077 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7feed40cd530 0x7feed40ca080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:52.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.117+0000 7feedb16c640 1 -- 172.21.15.92:0/1512996077 >> 172.21.15.92:0/1512996077 conn(0x7feed406a800 msgr2=0x7feed41589e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:52.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.117+0000 7feedb16c640 1 -- 172.21.15.92:0/1512996077 shutdown_connections 2023-11-24T06:11:52.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.117+0000 7feedb16c640 1 -- 172.21.15.92:0/1512996077 wait complete. 2023-11-24T06:11:52.136 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-24T06:11:52.335 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.332+0000 7fec3541e640 1 Processor -- start 2023-11-24T06:11:52.335 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.332+0000 7fec3541e640 1 -- start start 2023-11-24T06:11:52.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.334+0000 7fec3541e640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fec300cf9a0 0x7fec3015e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:52.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.334+0000 7fec3541e640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec30070c40 con 0x7fec300b3ae0 2023-11-24T06:11:52.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.334+0000 7fec3541e640 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec3006c470 con 0x7fec300ae5a0 2023-11-24T06:11:52.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.334+0000 7fec3541e640 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec30071450 con 0x7fec300cf9a0 2023-11-24T06:11:52.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.334+0000 7fec2e7fc640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fec300b3ae0 0x7fec300cf290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:41276/0 (socket says 172.21.15.92:41276) 2023-11-24T06:11:52.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.334+0000 7fec2e7fc640 1 -- 172.21.15.92:0/714520227 learned_addr learned my addr 172.21.15.92:0/714520227 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-24T06:11:52.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.334+0000 7fec2f7fe640 1 --2- 172.21.15.92:0/714520227 >> v2:172.21.15.118:3300/0 conn(0x7fec300cf9a0 0x7fec3015e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:52.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.335+0000 7fec2dffb640 1 -- 172.21.15.92:0/714520227 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1552109938 0 0) 0x7fec30070c40 con 0x7fec300b3ae0 2023-11-24T06:11:52.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.335+0000 7fec2dffb640 1 -- 172.21.15.92:0/714520227 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fec18003520 con 0x7fec300b3ae0 2023-11-24T06:11:52.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.335+0000 7fec2dffb640 1 -- 172.21.15.92:0/714520227 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1225368592 0 0) 0x7fec18003520 con 0x7fec300b3ae0 2023-11-24T06:11:52.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.335+0000 7fec2dffb640 1 -- 172.21.15.92:0/714520227 >> v2:172.21.15.118:3300/0 conn(0x7fec300cf9a0 msgr2=0x7fec3015e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:52.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.335+0000 7fec2dffb640 1 --2- 172.21.15.92:0/714520227 >> v2:172.21.15.118:3300/0 conn(0x7fec300cf9a0 0x7fec3015e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:52.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.335+0000 7fec2dffb640 1 -- 172.21.15.92:0/714520227 >> v1:172.21.15.118:6789/0 conn(0x7fec300ae5a0 legacy=0x7fec3006cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:52.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.335+0000 7fec2dffb640 1 -- 172.21.15.92:0/714520227 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec3006c750 con 0x7fec300b3ae0 2023-11-24T06:11:52.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.335+0000 7fec2dffb640 1 -- 172.21.15.92:0/714520227 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2607661350 0 0) 0x7fec24001670 con 0x7fec300b3ae0 2023-11-24T06:11:52.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.335+0000 7fec2dffb640 1 -- 172.21.15.92:0/714520227 >> v1:172.21.15.92:6789/0 conn(0x7fec300b3ae0 legacy=0x7fec300cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:52.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.335+0000 7fec2dffb640 1 --2- 172.21.15.92:0/714520227 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fec18003970 0x7fec18003d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:52.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.335+0000 7fec2dffb640 1 --2- 172.21.15.92:0/714520227 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fec180047c0 0x7fec18004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:52.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.335+0000 7fec2dffb640 1 --2- 172.21.15.92:0/714520227 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fec18005180 0x7fec1800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:52.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.335+0000 7fec2dffb640 1 -- 172.21.15.92:0/714520227 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec30071450 con 0x7fec180047c0 2023-11-24T06:11:52.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.335+0000 7fec2dffb640 1 -- 172.21.15.92:0/714520227 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec1800c000 con 0x7fec18003970 2023-11-24T06:11:52.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.335+0000 7fec2dffb640 1 -- 172.21.15.92:0/714520227 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec1800c1c0 con 0x7fec18005180 2023-11-24T06:11:52.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.335+0000 7fec2e7fc640 1 --2- 172.21.15.92:0/714520227 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fec180047c0 0x7fec18004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:52.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.336+0000 7fec2effd640 1 --2- 172.21.15.92:0/714520227 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fec18003970 0x7fec18003d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:52.341 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.336+0000 7fec2e7fc640 1 -- 172.21.15.92:0/714520227 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fec18005180 msgr2=0x7fec1800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:52.341 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.336+0000 7fec2e7fc640 1 --2- 172.21.15.92:0/714520227 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fec18005180 0x7fec1800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:52.341 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.336+0000 7fec2e7fc640 1 -- 172.21.15.92:0/714520227 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fec18003970 msgr2=0x7fec18003d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:52.341 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.336+0000 7fec2e7fc640 1 --2- 172.21.15.92:0/714520227 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fec18003970 0x7fec18003d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:52.341 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.336+0000 7fec2e7fc640 1 -- 172.21.15.92:0/714520227 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec18003520 con 0x7fec180047c0 2023-11-24T06:11:52.341 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.336+0000 7fec2f7fe640 1 -- 172.21.15.92:0/714520227 reap_dead start 2023-11-24T06:11:52.342 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.337+0000 7fec2e7fc640 1 --2- 172.21.15.92:0/714520227 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fec180047c0 0x7fec18004c40 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fec2400c210 tx=0x7fec180057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55e78731cf1d092b server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:52.342 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.337+0000 7fec2dffb640 1 -- 172.21.15.92:0/714520227 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec24005ad0 con 0x7fec180047c0 2023-11-24T06:11:52.342 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.337+0000 7fec2dffb640 1 -- 172.21.15.92:0/714520227 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec24005c30 con 0x7fec180047c0 2023-11-24T06:11:52.342 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.337+0000 7fec2dffb640 1 -- 172.21.15.92:0/714520227 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec2402c7c0 con 0x7fec180047c0 2023-11-24T06:11:52.342 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.337+0000 7fec3541e640 1 -- 172.21.15.92:0/714520227 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fec180047c0 msgr2=0x7fec18004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:52.342 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.337+0000 7fec3541e640 1 --2- 172.21.15.92:0/714520227 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fec180047c0 0x7fec18004c40 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fec2400c210 tx=0x7fec180057f0 comp rx=0 tx=0).stop 2023-11-24T06:11:52.342 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.338+0000 7fec3541e640 1 -- 172.21.15.92:0/714520227 shutdown_connections 2023-11-24T06:11:52.343 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.338+0000 7fec3541e640 1 --2- 172.21.15.92:0/714520227 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fec180047c0 0x7fec18004c40 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:52.343 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.338+0000 7fec3541e640 1 -- 172.21.15.92:0/714520227 >> 172.21.15.92:0/714520227 conn(0x7fec3006a800 msgr2=0x7fec3006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:52.343 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.338+0000 7fec3541e640 1 -- 172.21.15.92:0/714520227 shutdown_connections 2023-11-24T06:11:52.343 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.338+0000 7fec3541e640 1 -- 172.21.15.92:0/714520227 wait complete. 2023-11-24T06:11:52.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.339+0000 7fec3541e640 1 Processor -- start 2023-11-24T06:11:52.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.340+0000 7fec3541e640 1 -- start start 2023-11-24T06:11:52.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.340+0000 7fec3541e640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fec180047c0 0x7fec3015f530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:52.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.340+0000 7fec3541e640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fec30157d40 0x7fec30158270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:52.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.340+0000 7fec3541e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fec301587b0 0x7fec30158c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:52.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.340+0000 7fec3541e640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec30160600 con 0x7fec301587b0 2023-11-24T06:11:52.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.340+0000 7fec3541e640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec300cba60 con 0x7fec30157d40 2023-11-24T06:11:52.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.340+0000 7fec3541e640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec3015fd60 con 0x7fec180047c0 2023-11-24T06:11:52.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.341+0000 7fec2f7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fec301587b0 0x7fec30158c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:52.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.341+0000 7fec2f7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fec301587b0 0x7fec30158c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58742/0 (socket says 172.21.15.92:58742) 2023-11-24T06:11:52.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.341+0000 7fec2f7fe640 1 -- 172.21.15.92:0/3873139980 learned_addr learned my addr 172.21.15.92:0/3873139980 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:11:52.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.341+0000 7fec2effd640 1 --2- 172.21.15.92:0/3873139980 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fec180047c0 0x7fec3015f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:52.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.341+0000 7fec2e7fc640 1 --2- 172.21.15.92:0/3873139980 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fec30157d40 0x7fec30158270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:52.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.341+0000 7fec2f7fe640 1 -- 172.21.15.92:0/3873139980 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fec180047c0 msgr2=0x7fec3015f530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:52.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.341+0000 7fec2f7fe640 1 --2- 172.21.15.92:0/3873139980 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fec180047c0 0x7fec3015f530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:52.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.341+0000 7fec2f7fe640 1 -- 172.21.15.92:0/3873139980 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fec30157d40 msgr2=0x7fec30158270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:52.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.341+0000 7fec2f7fe640 1 --2- 172.21.15.92:0/3873139980 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fec30157d40 0x7fec30158270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:52.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.341+0000 7fec2f7fe640 1 -- 172.21.15.92:0/3873139980 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec30160040 con 0x7fec301587b0 2023-11-24T06:11:52.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.341+0000 7fec2e7fc640 1 --2- 172.21.15.92:0/3873139980 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fec30157d40 0x7fec30158270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:11:52.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.341+0000 7fec2f7fe640 1 --2- 172.21.15.92:0/3873139980 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fec301587b0 0x7fec30158c30 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fec3006dfa0 tx=0x7fec2800b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:52.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.342+0000 7fec177fe640 1 -- 172.21.15.92:0/3873139980 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec28010040 con 0x7fec301587b0 2023-11-24T06:11:52.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.342+0000 7fec177fe640 1 -- 172.21.15.92:0/3873139980 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec2800bc80 con 0x7fec301587b0 2023-11-24T06:11:52.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.342+0000 7fec3541e640 1 -- 172.21.15.92:0/3873139980 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec30160320 con 0x7fec301587b0 2023-11-24T06:11:52.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.342+0000 7fec177fe640 1 -- 172.21.15.92:0/3873139980 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec28014400 con 0x7fec301587b0 2023-11-24T06:11:52.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.342+0000 7fec3541e640 1 -- 172.21.15.92:0/3873139980 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec30150280 con 0x7fec301587b0 2023-11-24T06:11:52.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.343+0000 7fec3541e640 1 -- 172.21.15.92:0/3873139980 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec300675d0 con 0x7fec301587b0 2023-11-24T06:11:52.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.344+0000 7fec177fe640 1 -- 172.21.15.92:0/3873139980 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fec28014560 con 0x7fec301587b0 2023-11-24T06:11:52.348 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.345+0000 7fec177fe640 1 --2- 172.21.15.92:0/3873139980 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fec1c06ae40 0x7fec1c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:52.348 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.345+0000 7fec177fe640 1 -- 172.21.15.92:0/3873139980 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fec28089c20 con 0x7fec301587b0 2023-11-24T06:11:52.348 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.345+0000 7fec2effd640 1 --2- 172.21.15.92:0/3873139980 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fec1c06ae40 0x7fec1c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:52.348 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.346+0000 7fec2effd640 1 --2- 172.21.15.92:0/3873139980 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fec1c06ae40 0x7fec1c06d300 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fec20003c90 tx=0x7fec20009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:52.352 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.349+0000 7fec177fe640 1 -- 172.21.15.92:0/3873139980 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fec28054cd0 con 0x7fec301587b0 2023-11-24T06:11:52.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.578+0000 7fec3541e640 1 -- 172.21.15.92:0/3873139980 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fec3015e400 con 0x7fec301587b0 2023-11-24T06:11:52.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.579+0000 7fec177fe640 1 -- 172.21.15.92:0/3873139980 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) v1 ==== 53+0+73 (secure 0 0 0) 0x7fec280582f0 con 0x7fec301587b0 2023-11-24T06:11:52.583 INFO:teuthology.orchestra.run.smithi092.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-24T06:11:52.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.582+0000 7fec3541e640 1 -- 172.21.15.92:0/3873139980 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fec1c06ae40 msgr2=0x7fec1c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:52.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.582+0000 7fec3541e640 1 --2- 172.21.15.92:0/3873139980 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fec1c06ae40 0x7fec1c06d300 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fec20003c90 tx=0x7fec20009040 comp rx=0 tx=0).stop 2023-11-24T06:11:52.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.582+0000 7fec3541e640 1 -- 172.21.15.92:0/3873139980 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fec301587b0 msgr2=0x7fec30158c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:52.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.582+0000 7fec3541e640 1 --2- 172.21.15.92:0/3873139980 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fec301587b0 0x7fec30158c30 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fec3006dfa0 tx=0x7fec2800b480 comp rx=0 tx=0).stop 2023-11-24T06:11:52.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.583+0000 7fec3541e640 1 -- 172.21.15.92:0/3873139980 shutdown_connections 2023-11-24T06:11:52.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.583+0000 7fec3541e640 1 --2- 172.21.15.92:0/3873139980 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fec1c06ae40 0x7fec1c06d300 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:52.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.583+0000 7fec3541e640 1 --2- 172.21.15.92:0/3873139980 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fec301587b0 0x7fec30158c30 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:52.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.583+0000 7fec3541e640 1 --2- 172.21.15.92:0/3873139980 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fec30157d40 0x7fec30158270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:52.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.583+0000 7fec3541e640 1 --2- 172.21.15.92:0/3873139980 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fec180047c0 0x7fec3015f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:52.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.583+0000 7fec3541e640 1 -- 172.21.15.92:0/3873139980 >> 172.21.15.92:0/3873139980 conn(0x7fec3006a800 msgr2=0x7fec3006cbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:52.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.583+0000 7fec3541e640 1 -- 172.21.15.92:0/3873139980 shutdown_connections 2023-11-24T06:11:52.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:52.583+0000 7fec3541e640 1 -- 172.21.15.92:0/3873139980 wait complete. 2023-11-24T06:11:52.604 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-24T06:11:52.604 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-24T06:11:52.604 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi092.front.sepia.ceph.com 2023-11-24T06:11:52.604 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-24T06:11:52.604 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-24T06:11:52.605 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-24T06:11:52.605 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-24T06:11:52.605 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-24T06:11:52.605 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-24T06:11:52.605 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:11:52.605 DEBUG:teuthology.orchestra.run.smithi092:> ip addr 2023-11-24T06:11:52.655 INFO:teuthology.orchestra.run.smithi092.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-24T06:11:52.655 INFO:teuthology.orchestra.run.smithi092.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-24T06:11:52.655 INFO:teuthology.orchestra.run.smithi092.stdout: inet 127.0.0.1/8 scope host lo 2023-11-24T06:11:52.655 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:11:52.655 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 ::1/128 scope host 2023-11-24T06:11:52.656 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:11:52.656 INFO:teuthology.orchestra.run.smithi092.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T06:11:52.656 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:d8:a1:b2 brd ff:ff:ff:ff:ff:ff 2023-11-24T06:11:52.656 INFO:teuthology.orchestra.run.smithi092.stdout: altname enp7s0f0 2023-11-24T06:11:52.656 INFO:teuthology.orchestra.run.smithi092.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T06:11:52.656 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:d8:a1:b3 brd ff:ff:ff:ff:ff:ff 2023-11-24T06:11:52.656 INFO:teuthology.orchestra.run.smithi092.stdout: altname enp7s0f1 2023-11-24T06:11:52.656 INFO:teuthology.orchestra.run.smithi092.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T06:11:52.656 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:8f:cd:44 brd ff:ff:ff:ff:ff:ff 2023-11-24T06:11:52.656 INFO:teuthology.orchestra.run.smithi092.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-24T06:11:52.656 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:8f:cd:45 brd ff:ff:ff:ff:ff:ff 2023-11-24T06:11:52.656 INFO:teuthology.orchestra.run.smithi092.stdout: inet 172.21.15.92/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-24T06:11:52.657 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft 42129sec preferred_lft 42129sec 2023-11-24T06:11:52.657 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 fe80::ec4:7aff:fe8f:cd45/64 scope link 2023-11-24T06:11:52.657 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:11:52.657 INFO:teuthology.orchestra.run.smithi092.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-24T06:11:52.657 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 7a:4d:f0:06:a7:4e brd ff:ff:ff:ff:ff:ff 2023-11-24T06:11:52.657 INFO:teuthology.orchestra.run.smithi092.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-24T06:11:52.657 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:11:52.657 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 fe80::9055:2fff:fec5:ff24/64 scope link 2023-11-24T06:11:52.657 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:11:52.657 INFO:teuthology.orchestra.run.smithi092.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-24T06:11:52.657 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether fa:d3:03:06:91:4f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-24T06:11:52.657 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 fe80::f8d3:3ff:fe06:914f/64 scope link 2023-11-24T06:11:52.658 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:11:52.658 INFO:teuthology.orchestra.run.smithi092.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-24T06:11:52.658 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 7a:4d:f0:06:a7:4e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-24T06:11:52.658 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 fe80::784d:f0ff:fe06:a74e/64 scope link 2023-11-24T06:11:52.658 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:11:52.659 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:11:52.659 DEBUG:teuthology.orchestra.run.smithi092:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-24T06:11:52.750 INFO:teuthology.orchestra.run.smithi092.stdout:1 2023-11-24T06:11:52.753 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:11:52.754 DEBUG:teuthology.orchestra.run.smithi092:> route 2023-11-24T06:11:52.808 INFO:teuthology.orchestra.run.smithi092.stdout:Kernel IP routing table 2023-11-24T06:11:52.808 INFO:teuthology.orchestra.run.smithi092.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-24T06:11:52.808 INFO:teuthology.orchestra.run.smithi092.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-24T06:11:52.808 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-24T06:11:52.808 INFO:teuthology.orchestra.run.smithi092.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-24T06:11:52.809 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:11:52.809 DEBUG:teuthology.orchestra.run.smithi092:> (cd / && exec bash -c ' 2023-11-24T06:11:52.809 DEBUG:teuthology.orchestra.run.smithi092:> set -e 2023-11-24T06:11:52.809 DEBUG:teuthology.orchestra.run.smithi092:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-24T06:11:52.809 DEBUG:teuthology.orchestra.run.smithi092:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-24T06:11:52.809 DEBUG:teuthology.orchestra.run.smithi092:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-24T06:11:52.809 DEBUG:teuthology.orchestra.run.smithi092:> ') 2023-11-24T06:11:52.996 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:11:52.996 DEBUG:teuthology.orchestra.run.smithi092:> ip netns list 2023-11-24T06:11:53.049 INFO:teuthology.orchestra.run.smithi092.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-24T06:11:53.050 INFO:teuthology.orchestra.run.smithi092.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-24T06:11:53.051 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-24T06:11:53.051 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:11:53.051 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip link set brx.0 up 2023-11-24T06:11:53.142 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-11-24T06:11:53.142 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T06:11:53.142 DEBUG:teuthology.orchestra.run.smithi092:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-11-24T06:11:53.193 INFO:teuthology.orchestra.run.smithi092.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-11-24T06:11:53.193 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T06:11:53.193 DEBUG:teuthology.orchestra.run.smithi092:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-11-24T06:11:53.248 DEBUG:teuthology.orchestra.run.smithi092:> sudo modprobe fuse 2023-11-24T06:11:53.343 DEBUG:teuthology.orchestra.run.smithi092:> cat /proc/self/mounts | awk '{print $2}' 2023-11-24T06:11:53.399 INFO:teuthology.orchestra.run.smithi092.stdout:/proc 2023-11-24T06:11:53.399 INFO:teuthology.orchestra.run.smithi092.stdout:/sys 2023-11-24T06:11:53.399 INFO:teuthology.orchestra.run.smithi092.stdout:/dev 2023-11-24T06:11:53.399 INFO:teuthology.orchestra.run.smithi092.stdout:/sys/kernel/security 2023-11-24T06:11:53.399 INFO:teuthology.orchestra.run.smithi092.stdout:/dev/shm 2023-11-24T06:11:53.399 INFO:teuthology.orchestra.run.smithi092.stdout:/dev/pts 2023-11-24T06:11:53.399 INFO:teuthology.orchestra.run.smithi092.stdout:/run 2023-11-24T06:11:53.400 INFO:teuthology.orchestra.run.smithi092.stdout:/sys/fs/cgroup 2023-11-24T06:11:53.400 INFO:teuthology.orchestra.run.smithi092.stdout:/sys/fs/pstore 2023-11-24T06:11:53.400 INFO:teuthology.orchestra.run.smithi092.stdout:/ 2023-11-24T06:11:53.400 INFO:teuthology.orchestra.run.smithi092.stdout:/sys/fs/selinux 2023-11-24T06:11:53.400 INFO:teuthology.orchestra.run.smithi092.stdout:/proc/sys/fs/binfmt_misc 2023-11-24T06:11:53.400 INFO:teuthology.orchestra.run.smithi092.stdout:/dev/hugepages 2023-11-24T06:11:53.401 INFO:teuthology.orchestra.run.smithi092.stdout:/dev/mqueue 2023-11-24T06:11:53.401 INFO:teuthology.orchestra.run.smithi092.stdout:/sys/kernel/debug 2023-11-24T06:11:53.401 INFO:teuthology.orchestra.run.smithi092.stdout:/sys/kernel/tracing 2023-11-24T06:11:53.401 INFO:teuthology.orchestra.run.smithi092.stdout:/sys/fs/fuse/connections 2023-11-24T06:11:53.401 INFO:teuthology.orchestra.run.smithi092.stdout:/sys/kernel/config 2023-11-24T06:11:53.401 INFO:teuthology.orchestra.run.smithi092.stdout:/run/credentials/systemd-sysctl.service 2023-11-24T06:11:53.402 INFO:teuthology.orchestra.run.smithi092.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-24T06:11:53.402 INFO:teuthology.orchestra.run.smithi092.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-24T06:11:53.402 INFO:teuthology.orchestra.run.smithi092.stdout:/var/lib/nfs/rpc_pipefs 2023-11-24T06:11:53.402 INFO:teuthology.orchestra.run.smithi092.stdout:/run/user/1000 2023-11-24T06:11:53.402 INFO:teuthology.orchestra.run.smithi092.stdout:/var/lib/ceph 2023-11-24T06:11:53.403 INFO:teuthology.orchestra.run.smithi092.stdout:/var/lib/ceph/osd/ceph-0 2023-11-24T06:11:53.403 INFO:teuthology.orchestra.run.smithi092.stdout:/var/lib/ceph/osd/ceph-1 2023-11-24T06:11:53.403 INFO:teuthology.orchestra.run.smithi092.stdout:/var/lib/ceph/osd/ceph-2 2023-11-24T06:11:53.403 INFO:teuthology.orchestra.run.smithi092.stdout:/var/lib/ceph/osd/ceph-3 2023-11-24T06:11:53.403 INFO:teuthology.orchestra.run.smithi092.stdout:/run/netns 2023-11-24T06:11:53.403 INFO:teuthology.orchestra.run.smithi092.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-24T06:11:53.404 INFO:teuthology.orchestra.run.smithi092.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-24T06:11:53.404 INFO:teuthology.orchestra.run.smithi092.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-24T06:11:53.404 INFO:teuthology.orchestra.run.smithi092.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-24T06:11:53.405 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:11:53.406 DEBUG:teuthology.orchestra.run.smithi092:> ls /sys/fs/fuse/connections 2023-11-24T06:11:53.463 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-11-24T06:11:53.463 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-11-24T06:11:53.469 DEBUG:teuthology.orchestra.run.smithi092:> sudo modprobe fuse 2023-11-24T06:11:53.549 DEBUG:teuthology.orchestra.run.smithi092:> cat /proc/self/mounts | awk '{print $2}' 2023-11-24T06:11:53.590 INFO:teuthology.orchestra.run.smithi092.stdout:/proc 2023-11-24T06:11:53.590 INFO:teuthology.orchestra.run.smithi092.stdout:/sys 2023-11-24T06:11:53.591 INFO:teuthology.orchestra.run.smithi092.stdout:/dev 2023-11-24T06:11:53.591 INFO:teuthology.orchestra.run.smithi092.stdout:/sys/kernel/security 2023-11-24T06:11:53.591 INFO:teuthology.orchestra.run.smithi092.stdout:/dev/shm 2023-11-24T06:11:53.591 INFO:teuthology.orchestra.run.smithi092.stdout:/dev/pts 2023-11-24T06:11:53.591 INFO:teuthology.orchestra.run.smithi092.stdout:/run 2023-11-24T06:11:53.591 INFO:teuthology.orchestra.run.smithi092.stdout:/sys/fs/cgroup 2023-11-24T06:11:53.591 INFO:teuthology.orchestra.run.smithi092.stdout:/sys/fs/pstore 2023-11-24T06:11:53.591 INFO:teuthology.orchestra.run.smithi092.stdout:/ 2023-11-24T06:11:53.591 INFO:teuthology.orchestra.run.smithi092.stdout:/sys/fs/selinux 2023-11-24T06:11:53.591 INFO:teuthology.orchestra.run.smithi092.stdout:/proc/sys/fs/binfmt_misc 2023-11-24T06:11:53.591 INFO:teuthology.orchestra.run.smithi092.stdout:/dev/hugepages 2023-11-24T06:11:53.591 INFO:teuthology.orchestra.run.smithi092.stdout:/dev/mqueue 2023-11-24T06:11:53.592 INFO:teuthology.orchestra.run.smithi092.stdout:/sys/kernel/debug 2023-11-24T06:11:53.592 INFO:teuthology.orchestra.run.smithi092.stdout:/sys/kernel/tracing 2023-11-24T06:11:53.592 INFO:teuthology.orchestra.run.smithi092.stdout:/sys/fs/fuse/connections 2023-11-24T06:11:53.592 INFO:teuthology.orchestra.run.smithi092.stdout:/sys/kernel/config 2023-11-24T06:11:53.592 INFO:teuthology.orchestra.run.smithi092.stdout:/run/credentials/systemd-sysctl.service 2023-11-24T06:11:53.592 INFO:teuthology.orchestra.run.smithi092.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-24T06:11:53.592 INFO:teuthology.orchestra.run.smithi092.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-24T06:11:53.592 INFO:teuthology.orchestra.run.smithi092.stdout:/var/lib/nfs/rpc_pipefs 2023-11-24T06:11:53.592 INFO:teuthology.orchestra.run.smithi092.stdout:/run/user/1000 2023-11-24T06:11:53.592 INFO:teuthology.orchestra.run.smithi092.stdout:/var/lib/ceph 2023-11-24T06:11:53.592 INFO:teuthology.orchestra.run.smithi092.stdout:/var/lib/ceph/osd/ceph-0 2023-11-24T06:11:53.592 INFO:teuthology.orchestra.run.smithi092.stdout:/var/lib/ceph/osd/ceph-1 2023-11-24T06:11:53.593 INFO:teuthology.orchestra.run.smithi092.stdout:/var/lib/ceph/osd/ceph-2 2023-11-24T06:11:53.593 INFO:teuthology.orchestra.run.smithi092.stdout:/var/lib/ceph/osd/ceph-3 2023-11-24T06:11:53.593 INFO:teuthology.orchestra.run.smithi092.stdout:/run/netns 2023-11-24T06:11:53.593 INFO:teuthology.orchestra.run.smithi092.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-24T06:11:53.593 INFO:teuthology.orchestra.run.smithi092.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-24T06:11:53.593 INFO:teuthology.orchestra.run.smithi092.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-24T06:11:53.593 INFO:teuthology.orchestra.run.smithi092.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-24T06:11:53.594 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:11:53.594 DEBUG:teuthology.orchestra.run.smithi092:> ls /sys/fs/fuse/connections 2023-11-24T06:11:53.655 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi092.stderr:2023-11-24T06:11:53.652+0000 7fbd0820b480 -1 init, newargv = 0x5601c8d0c9e0 newargc=15 2023-11-24T06:11:53.655 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi092.stderr:ceph-fuse[102972]: starting ceph client 2023-11-24T06:11:53.662 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi092.stderr:ceph-fuse[102972]: starting fuse 2023-11-24T06:11:53.682 INFO:teuthology.orchestra.run.smithi092.stdout:40 2023-11-24T06:11:53.682 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [40] 2023-11-24T06:11:53.682 DEBUG:teuthology.orchestra.run.smithi092:> sudo stdin-killer -- python3 -c ' 2023-11-24T06:11:53.682 DEBUG:teuthology.orchestra.run.smithi092:> import glob 2023-11-24T06:11:53.682 DEBUG:teuthology.orchestra.run.smithi092:> import re 2023-11-24T06:11:53.683 DEBUG:teuthology.orchestra.run.smithi092:> import os 2023-11-24T06:11:53.683 DEBUG:teuthology.orchestra.run.smithi092:> import subprocess 2023-11-24T06:11:53.683 DEBUG:teuthology.orchestra.run.smithi092:> 2023-11-24T06:11:53.683 DEBUG:teuthology.orchestra.run.smithi092:> def _find_admin_socket(client_name): 2023-11-24T06:11:53.683 DEBUG:teuthology.orchestra.run.smithi092:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-11-24T06:11:53.683 DEBUG:teuthology.orchestra.run.smithi092:> files = glob.glob(asok_path) 2023-11-24T06:11:53.683 DEBUG:teuthology.orchestra.run.smithi092:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-11-24T06:11:53.683 DEBUG:teuthology.orchestra.run.smithi092:> 2023-11-24T06:11:53.683 DEBUG:teuthology.orchestra.run.smithi092:> # Given a non-glob path, it better be there 2023-11-24T06:11:53.683 DEBUG:teuthology.orchestra.run.smithi092:> if "*" not in asok_path: 2023-11-24T06:11:53.683 DEBUG:teuthology.orchestra.run.smithi092:> assert(len(files) == 1) 2023-11-24T06:11:53.684 DEBUG:teuthology.orchestra.run.smithi092:> return files[0] 2023-11-24T06:11:53.684 DEBUG:teuthology.orchestra.run.smithi092:> 2023-11-24T06:11:53.684 DEBUG:teuthology.orchestra.run.smithi092:> for f in files: 2023-11-24T06:11:53.684 DEBUG:teuthology.orchestra.run.smithi092:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-11-24T06:11:53.684 DEBUG:teuthology.orchestra.run.smithi092:> if os.path.exists("/proc/{0}".format(pid)): 2023-11-24T06:11:53.684 DEBUG:teuthology.orchestra.run.smithi092:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-11-24T06:11:53.684 DEBUG:teuthology.orchestra.run.smithi092:> contents = proc_f.read() 2023-11-24T06:11:53.684 DEBUG:teuthology.orchestra.run.smithi092:> if mountpoint in contents: 2023-11-24T06:11:53.684 DEBUG:teuthology.orchestra.run.smithi092:> return f 2023-11-24T06:11:53.684 DEBUG:teuthology.orchestra.run.smithi092:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-11-24T06:11:53.684 DEBUG:teuthology.orchestra.run.smithi092:> 2023-11-24T06:11:53.684 DEBUG:teuthology.orchestra.run.smithi092:> print(_find_admin_socket("client.0")) 2023-11-24T06:11:53.685 DEBUG:teuthology.orchestra.run.smithi092:> ' 2023-11-24T06:11:53.884 INFO:teuthology.orchestra.run.smithi092.stdout:/var/run/ceph/ceph-client.0.102972.asok 2023-11-24T06:11:53.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-24T06:11:53.900 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.102972.asok 2023-11-24T06:11:53.900 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:11:53.900 DEBUG:teuthology.orchestra.run.smithi092:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.102972.asok status 2023-11-24T06:11:54.141 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-24T06:11:54.141 INFO:teuthology.orchestra.run.smithi092.stdout: "metadata": { 2023-11-24T06:11:54.142 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph_sha1": "a13ea5ff00cb4df7a9cc941166ddeb358881144c", 2023-11-24T06:11:54.142 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph_version": "ceph version 18.0.0-7484-ga13ea5ff (a13ea5ff00cb4df7a9cc941166ddeb358881144c) reef (dev)", 2023-11-24T06:11:54.142 INFO:teuthology.orchestra.run.smithi092.stdout: "entity_id": "0", 2023-11-24T06:11:54.142 INFO:teuthology.orchestra.run.smithi092.stdout: "hostname": "smithi092", 2023-11-24T06:11:54.142 INFO:teuthology.orchestra.run.smithi092.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-11-24T06:11:54.142 INFO:teuthology.orchestra.run.smithi092.stdout: "pid": "102972", 2023-11-24T06:11:54.142 INFO:teuthology.orchestra.run.smithi092.stdout: "root": "/" 2023-11-24T06:11:54.142 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-24T06:11:54.142 INFO:teuthology.orchestra.run.smithi092.stdout: "dentry_count": 0, 2023-11-24T06:11:54.142 INFO:teuthology.orchestra.run.smithi092.stdout: "dentry_pinned_count": 0, 2023-11-24T06:11:54.142 INFO:teuthology.orchestra.run.smithi092.stdout: "id": 5436, 2023-11-24T06:11:54.142 INFO:teuthology.orchestra.run.smithi092.stdout: "inst": { 2023-11-24T06:11:54.143 INFO:teuthology.orchestra.run.smithi092.stdout: "name": { 2023-11-24T06:11:54.143 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "client", 2023-11-24T06:11:54.143 INFO:teuthology.orchestra.run.smithi092.stdout: "num": 5436 2023-11-24T06:11:54.143 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-24T06:11:54.143 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": { 2023-11-24T06:11:54.143 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2023-11-24T06:11:54.143 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "192.168.0.1:0", 2023-11-24T06:11:54.143 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 1167031755 2023-11-24T06:11:54.143 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-24T06:11:54.143 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-24T06:11:54.144 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": { 2023-11-24T06:11:54.144 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2023-11-24T06:11:54.144 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "192.168.0.1:0", 2023-11-24T06:11:54.144 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 1167031755 2023-11-24T06:11:54.144 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-24T06:11:54.144 INFO:teuthology.orchestra.run.smithi092.stdout: "inst_str": "client.5436 192.168.0.1:0/1167031755", 2023-11-24T06:11:54.144 INFO:teuthology.orchestra.run.smithi092.stdout: "addr_str": "192.168.0.1:0/1167031755", 2023-11-24T06:11:54.144 INFO:teuthology.orchestra.run.smithi092.stdout: "inode_count": 1, 2023-11-24T06:11:54.144 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_epoch": 17, 2023-11-24T06:11:54.144 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_epoch": 33, 2023-11-24T06:11:54.144 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_epoch_barrier": 0, 2023-11-24T06:11:54.144 INFO:teuthology.orchestra.run.smithi092.stdout: "blocklisted": false, 2023-11-24T06:11:54.145 INFO:teuthology.orchestra.run.smithi092.stdout: "fs_name": "cephfs" 2023-11-24T06:11:54.145 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-24T06:11:54.156 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:11:54.156 DEBUG:teuthology.orchestra.run.smithi092:> stat --file-system '--printf=%T 2023-11-24T06:11:54.156 DEBUG:teuthology.orchestra.run.smithi092:> ' -- /home/ubuntu/cephtest/mnt.0 2023-11-24T06:11:54.214 INFO:teuthology.orchestra.run.smithi092.stdout:fuseblk 2023-11-24T06:11:54.214 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-11-24T06:11:54.214 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:11:54.214 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-24T06:11:54.312 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-24T06:11:54.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:54.523+0000 7f4dc95ec640 1 Processor -- start 2023-11-24T06:11:54.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:54.523+0000 7f4dc95ec640 1 -- start start 2023-11-24T06:11:54.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:54.525+0000 7f4dc95ec640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f4dc413e360 0x7f4dc40cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:54.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:54.525+0000 7f4dc95ec640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4dc415f9c0 0x7f4dc40cf7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:54.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:54.525+0000 7f4dc95ec640 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4dc415c5f0 con 0x7f4dc413df80 2023-11-24T06:11:54.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:54.525+0000 7f4dc95ec640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4dc406c470 con 0x7f4dc413e360 2023-11-24T06:11:54.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:54.525+0000 7f4dc95ec640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4dc406c750 con 0x7f4dc415f9c0 2023-11-24T06:11:54.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:54.525+0000 7f4dc27fc640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f4dc413e360 0x7f4dc40cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:54.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:54.525+0000 7f4dc27fc640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f4dc413e360 0x7f4dc40cf290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58756/0 (socket says 172.21.15.92:58756) 2023-11-24T06:11:54.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:54.525+0000 7f4dc27fc640 1 -- 172.21.15.92:0/3104913957 learned_addr learned my addr 172.21.15.92:0/3104913957 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:11:54.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:54.525+0000 7f4dc37fe640 1 --2- 172.21.15.92:0/3104913957 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4dc415f9c0 0x7f4dc40cf7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:54.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:54.526+0000 7f4dc27fc640 1 -- 172.21.15.92:0/3104913957 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4dc415f9c0 msgr2=0x7f4dc40cf7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:54.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:54.526+0000 7f4dc27fc640 1 --2- 172.21.15.92:0/3104913957 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4dc415f9c0 0x7f4dc40cf7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:54.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:54.526+0000 7f4dc27fc640 1 -- 172.21.15.92:0/3104913957 >> v1:172.21.15.118:6789/0 conn(0x7f4dc413df80 legacy=0x7f4dc415bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:54.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:54.526+0000 7f4dc27fc640 1 -- 172.21.15.92:0/3104913957 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4dc406ca30 con 0x7f4dc413e360 2023-11-24T06:11:54.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:54.527+0000 7f4dc27fc640 1 --2- 172.21.15.92:0/3104913957 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4dc413e360 0x7f4dc40cf290 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f4dc406ded0 tx=0x7f4db802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eeb7e9331d8248be server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:54.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:54.527+0000 7f4dc1ffb640 1 -- 172.21.15.92:0/3104913957 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4db8031830 con 0x7f4dc413e360 2023-11-24T06:11:54.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:54.527+0000 7f4dc1ffb640 1 -- 172.21.15.92:0/3104913957 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4db8031990 con 0x7f4dc413e360 2023-11-24T06:11:54.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:54.527+0000 7f4dc1ffb640 1 -- 172.21.15.92:0/3104913957 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4db8031d20 con 0x7f4dc413e360 2023-11-24T06:11:54.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:54.528+0000 7f4dc95ec640 1 -- 172.21.15.92:0/3104913957 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4dc413e360 msgr2=0x7f4dc40cf290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:54.532 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:54.528+0000 7f4dc95ec640 1 --2- 172.21.15.92:0/3104913957 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4dc413e360 0x7f4dc40cf290 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f4dc406ded0 tx=0x7f4db802b8d0 comp rx=0 tx=0).stop 2023-11-24T06:11:54.532 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:54.528+0000 7f4dc95ec640 1 -- 172.21.15.92:0/3104913957 shutdown_connections 2023-11-24T06:11:54.532 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:54.528+0000 7f4dc95ec640 1 --2- 172.21.15.92:0/3104913957 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4dc415f9c0 0x7f4dc40cf7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:54.532 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:54.528+0000 7f4dc95ec640 1 --2- 172.21.15.92:0/3104913957 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4dc413e360 0x7f4dc40cf290 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:54.532 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:54.528+0000 7f4dc95ec640 1 -- 172.21.15.92:0/3104913957 >> 172.21.15.92:0/3104913957 conn(0x7f4dc406a800 msgr2=0x7f4dc406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:54.532 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:54.528+0000 7f4dc95ec640 1 -- 172.21.15.92:0/3104913957 shutdown_connections 2023-11-24T06:11:54.532 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:54.528+0000 7f4dc95ec640 1 -- 172.21.15.92:0/3104913957 wait complete. 2023-11-24T06:11:54.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:54.530+0000 7f4dc95ec640 1 Processor -- start 2023-11-24T06:11:54.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:54.530+0000 7f4dc95ec640 1 -- start start 2023-11-24T06:11:54.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:54.531+0000 7f4dc95ec640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4dc413df80 0x7f4dc413d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:54.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:54.531+0000 7f4dc95ec640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4dc413e360 0x7f4dc414b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:54.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:54.531+0000 7f4dc95ec640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4dc415f9c0 0x7f4dc41f0d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:54.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:54.531+0000 7f4dc95ec640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dc41f1db0 con 0x7f4dc413df80 2023-11-24T06:11:54.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:54.531+0000 7f4dc95ec640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dc413de10 con 0x7f4dc413e360 2023-11-24T06:11:54.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:54.531+0000 7f4dc95ec640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4dc41f17f0 con 0x7f4dc415f9c0 2023-11-24T06:11:54.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:54.531+0000 7f4dc2ffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4dc413df80 0x7f4dc413d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:54.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:54.531+0000 7f4dc2ffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4dc413df80 0x7f4dc413d8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58758/0 (socket says 172.21.15.92:58758) 2023-11-24T06:11:54.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:54.531+0000 7f4dc2ffd640 1 -- 172.21.15.92:0/763709406 learned_addr learned my addr 172.21.15.92:0/763709406 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:11:54.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:54.531+0000 7f4dc27fc640 1 --2- 172.21.15.92:0/763709406 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4dc413e360 0x7f4dc414b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:54.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:54.531+0000 7f4dc2ffd640 1 -- 172.21.15.92:0/763709406 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4dc415f9c0 msgr2=0x7f4dc41f0d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:54.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:54.531+0000 7f4dc37fe640 1 --2- 172.21.15.92:0/763709406 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4dc415f9c0 0x7f4dc41f0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:54.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:54.531+0000 7f4dc2ffd640 1 --2- 172.21.15.92:0/763709406 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4dc415f9c0 0x7f4dc41f0d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:54.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:54.531+0000 7f4dc2ffd640 1 -- 172.21.15.92:0/763709406 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4dc413e360 msgr2=0x7f4dc414b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:54.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:54.531+0000 7f4dc2ffd640 1 --2- 172.21.15.92:0/763709406 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4dc413e360 0x7f4dc414b3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:54.537 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:54.531+0000 7f4dc2ffd640 1 -- 172.21.15.92:0/763709406 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4db80094d0 con 0x7f4dc413df80 2023-11-24T06:11:54.537 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:54.532+0000 7f4dc27fc640 1 --2- 172.21.15.92:0/763709406 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4dc413e360 0x7f4dc414b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:11:54.537 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:54.532+0000 7f4dc2ffd640 1 --2- 172.21.15.92:0/763709406 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4dc413df80 0x7f4dc413d8d0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f4db400b3f0 tx=0x7f4db400b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:54.537 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:54.532+0000 7f4dab7fe640 1 -- 172.21.15.92:0/763709406 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4db4011830 con 0x7f4dc413df80 2023-11-24T06:11:54.537 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:54.532+0000 7f4dab7fe640 1 -- 172.21.15.92:0/763709406 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4db4011990 con 0x7f4dc413df80 2023-11-24T06:11:54.537 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:54.532+0000 7f4dab7fe640 1 -- 172.21.15.92:0/763709406 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4db4011d40 con 0x7f4dc413df80 2023-11-24T06:11:54.537 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:54.532+0000 7f4dc95ec640 1 -- 172.21.15.92:0/763709406 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4dc41f1ad0 con 0x7f4dc413df80 2023-11-24T06:11:54.537 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:54.532+0000 7f4dc95ec640 1 -- 172.21.15.92:0/763709406 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4dc414aea0 con 0x7f4dc413df80 2023-11-24T06:11:54.538 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:54.533+0000 7f4dc95ec640 1 -- 172.21.15.92:0/763709406 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4dc414aa20 con 0x7f4dc413df80 2023-11-24T06:11:54.538 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:54.534+0000 7f4dab7fe640 1 -- 172.21.15.92:0/763709406 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f4db401a400 con 0x7f4dc413df80 2023-11-24T06:11:54.538 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:54.535+0000 7f4dab7fe640 1 --2- 172.21.15.92:0/763709406 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f4db006ae40 0x7f4db006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:54.539 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:54.535+0000 7f4dab7fe640 1 -- 172.21.15.92:0/763709406 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4db4089560 con 0x7f4dc413df80 2023-11-24T06:11:54.539 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:54.536+0000 7f4dc27fc640 1 --2- 172.21.15.92:0/763709406 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f4db006ae40 0x7f4db006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:54.539 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:54.536+0000 7f4dc27fc640 1 --2- 172.21.15.92:0/763709406 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f4db006ae40 0x7f4db006d300 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f4db8036000 tx=0x7f4db803c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:54.543 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:54.540+0000 7f4dab7fe640 1 -- 172.21.15.92:0/763709406 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4dc414aa20 con 0x7f4dc413df80 2023-11-24T06:11:54.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:54.769+0000 7f4dc95ec640 1 -- 172.21.15.92:0/763709406 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f4dc414aa20 con 0x7f4dc413df80 2023-11-24T06:11:54.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:54.770+0000 7f4dab7fe640 1 -- 172.21.15.92:0/763709406 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) v1 ==== 53+0+73 (secure 0 0 0) 0x7f4dc414aa20 con 0x7f4dc413df80 2023-11-24T06:11:54.773 INFO:teuthology.orchestra.run.smithi092.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-24T06:11:54.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:54.773+0000 7f4dc95ec640 1 -- 172.21.15.92:0/763709406 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f4db006ae40 msgr2=0x7f4db006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:54.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:54.773+0000 7f4dc95ec640 1 --2- 172.21.15.92:0/763709406 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f4db006ae40 0x7f4db006d300 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f4db8036000 tx=0x7f4db803c040 comp rx=0 tx=0).stop 2023-11-24T06:11:54.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:54.773+0000 7f4dc95ec640 1 -- 172.21.15.92:0/763709406 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4dc413df80 msgr2=0x7f4dc413d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:54.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:54.773+0000 7f4dc95ec640 1 --2- 172.21.15.92:0/763709406 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4dc413df80 0x7f4dc413d8d0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f4db400b3f0 tx=0x7f4db400b8b0 comp rx=0 tx=0).stop 2023-11-24T06:11:54.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:54.773+0000 7f4dc95ec640 1 -- 172.21.15.92:0/763709406 shutdown_connections 2023-11-24T06:11:54.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:54.773+0000 7f4dc95ec640 1 --2- 172.21.15.92:0/763709406 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f4db006ae40 0x7f4db006d300 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:54.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:54.773+0000 7f4dc95ec640 1 --2- 172.21.15.92:0/763709406 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4dc415f9c0 0x7f4dc41f0d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:54.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:54.773+0000 7f4dc95ec640 1 --2- 172.21.15.92:0/763709406 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4dc413e360 0x7f4dc414b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:54.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:54.773+0000 7f4dc95ec640 1 --2- 172.21.15.92:0/763709406 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4dc413df80 0x7f4dc413d8d0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:54.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:54.773+0000 7f4dc95ec640 1 -- 172.21.15.92:0/763709406 >> 172.21.15.92:0/763709406 conn(0x7f4dc406a800 msgr2=0x7f4dc415de70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:54.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:54.773+0000 7f4dc95ec640 1 -- 172.21.15.92:0/763709406 shutdown_connections 2023-11-24T06:11:54.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:54.773+0000 7f4dc95ec640 1 -- 172.21.15.92:0/763709406 wait complete. 2023-11-24T06:11:54.793 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-24T06:11:54.793 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-24T06:11:54.793 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi118.front.sepia.ceph.com 2023-11-24T06:11:54.793 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-11-24T06:11:54.793 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-11-24T06:11:54.794 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-24T06:11:54.794 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-24T06:11:54.794 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-24T06:11:54.794 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-11-24T06:11:54.794 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:11:54.794 DEBUG:teuthology.orchestra.run.smithi118:> ip addr 2023-11-24T06:11:54.856 INFO:teuthology.orchestra.run.smithi118.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-24T06:11:54.856 INFO:teuthology.orchestra.run.smithi118.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-24T06:11:54.856 INFO:teuthology.orchestra.run.smithi118.stdout: inet 127.0.0.1/8 scope host lo 2023-11-24T06:11:54.856 INFO:teuthology.orchestra.run.smithi118.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:11:54.856 INFO:teuthology.orchestra.run.smithi118.stdout: inet6 ::1/128 scope host 2023-11-24T06:11:54.857 INFO:teuthology.orchestra.run.smithi118.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:11:54.857 INFO:teuthology.orchestra.run.smithi118.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T06:11:54.857 INFO:teuthology.orchestra.run.smithi118.stdout: link/ether 0c:c4:7a:d8:a4:cc brd ff:ff:ff:ff:ff:ff 2023-11-24T06:11:54.857 INFO:teuthology.orchestra.run.smithi118.stdout: altname enp7s0f0 2023-11-24T06:11:54.857 INFO:teuthology.orchestra.run.smithi118.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T06:11:54.857 INFO:teuthology.orchestra.run.smithi118.stdout: link/ether 0c:c4:7a:d8:a4:cd brd ff:ff:ff:ff:ff:ff 2023-11-24T06:11:54.857 INFO:teuthology.orchestra.run.smithi118.stdout: altname enp7s0f1 2023-11-24T06:11:54.857 INFO:teuthology.orchestra.run.smithi118.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T06:11:54.857 INFO:teuthology.orchestra.run.smithi118.stdout: link/ether 0c:c4:7a:8f:d7:24 brd ff:ff:ff:ff:ff:ff 2023-11-24T06:11:54.857 INFO:teuthology.orchestra.run.smithi118.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-24T06:11:54.857 INFO:teuthology.orchestra.run.smithi118.stdout: link/ether 0c:c4:7a:8f:d7:25 brd ff:ff:ff:ff:ff:ff 2023-11-24T06:11:54.857 INFO:teuthology.orchestra.run.smithi118.stdout: inet 172.21.15.118/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-24T06:11:54.858 INFO:teuthology.orchestra.run.smithi118.stdout: valid_lft 42133sec preferred_lft 42133sec 2023-11-24T06:11:54.858 INFO:teuthology.orchestra.run.smithi118.stdout: inet6 fe80::ec4:7aff:fe8f:d725/64 scope link 2023-11-24T06:11:54.858 INFO:teuthology.orchestra.run.smithi118.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:11:54.858 INFO:teuthology.orchestra.run.smithi118.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-24T06:11:54.858 INFO:teuthology.orchestra.run.smithi118.stdout: link/ether 16:ef:1e:16:88:fb brd ff:ff:ff:ff:ff:ff 2023-11-24T06:11:54.858 INFO:teuthology.orchestra.run.smithi118.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-24T06:11:54.858 INFO:teuthology.orchestra.run.smithi118.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:11:54.858 INFO:teuthology.orchestra.run.smithi118.stdout: inet6 fe80::50a3:24ff:fea4:c41e/64 scope link 2023-11-24T06:11:54.858 INFO:teuthology.orchestra.run.smithi118.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:11:54.858 INFO:teuthology.orchestra.run.smithi118.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-24T06:11:54.859 INFO:teuthology.orchestra.run.smithi118.stdout: link/ether 16:ef:1e:16:88:fb brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-24T06:11:54.859 INFO:teuthology.orchestra.run.smithi118.stdout: inet6 fe80::14ef:1eff:fe16:88fb/64 scope link 2023-11-24T06:11:54.859 INFO:teuthology.orchestra.run.smithi118.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:11:54.859 INFO:teuthology.orchestra.run.smithi118.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-24T06:11:54.859 INFO:teuthology.orchestra.run.smithi118.stdout: link/ether 6a:b2:50:ae:7a:72 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-24T06:11:54.859 INFO:teuthology.orchestra.run.smithi118.stdout: inet6 fe80::68b2:50ff:feae:7a72/64 scope link 2023-11-24T06:11:54.859 INFO:teuthology.orchestra.run.smithi118.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:11:54.860 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:11:54.860 DEBUG:teuthology.orchestra.run.smithi118:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-24T06:11:54.952 INFO:teuthology.orchestra.run.smithi118.stdout:1 2023-11-24T06:11:54.957 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:11:54.957 DEBUG:teuthology.orchestra.run.smithi118:> route 2023-11-24T06:11:55.013 INFO:teuthology.orchestra.run.smithi118.stdout:Kernel IP routing table 2023-11-24T06:11:55.014 INFO:teuthology.orchestra.run.smithi118.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-24T06:11:55.014 INFO:teuthology.orchestra.run.smithi118.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-24T06:11:55.014 INFO:teuthology.orchestra.run.smithi118.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-24T06:11:55.014 INFO:teuthology.orchestra.run.smithi118.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-24T06:11:55.014 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:11:55.015 DEBUG:teuthology.orchestra.run.smithi118:> (cd / && exec bash -c ' 2023-11-24T06:11:55.015 DEBUG:teuthology.orchestra.run.smithi118:> set -e 2023-11-24T06:11:55.015 DEBUG:teuthology.orchestra.run.smithi118:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-24T06:11:55.015 DEBUG:teuthology.orchestra.run.smithi118:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-24T06:11:55.015 DEBUG:teuthology.orchestra.run.smithi118:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-24T06:11:55.015 DEBUG:teuthology.orchestra.run.smithi118:> ') 2023-11-24T06:11:55.200 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:11:55.200 DEBUG:teuthology.orchestra.run.smithi118:> ip netns list 2023-11-24T06:11:55.250 INFO:teuthology.orchestra.run.smithi118.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-11-24T06:11:55.250 INFO:teuthology.orchestra.run.smithi118.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-11-24T06:11:55.251 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-11-24T06:11:55.252 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:11:55.252 DEBUG:teuthology.orchestra.run.smithi118:> sudo ip link set brx.0 up 2023-11-24T06:11:55.342 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2023-11-24T06:11:55.342 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T06:11:55.342 DEBUG:teuthology.orchestra.run.smithi118:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-11-24T06:11:55.393 INFO:teuthology.orchestra.run.smithi118.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-11-24T06:11:55.393 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T06:11:55.393 DEBUG:teuthology.orchestra.run.smithi118:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-11-24T06:11:55.441 DEBUG:teuthology.orchestra.run.smithi118:> sudo modprobe fuse 2023-11-24T06:11:55.523 DEBUG:teuthology.orchestra.run.smithi118:> cat /proc/self/mounts | awk '{print $2}' 2023-11-24T06:11:55.572 INFO:teuthology.orchestra.run.smithi118.stdout:/proc 2023-11-24T06:11:55.572 INFO:teuthology.orchestra.run.smithi118.stdout:/sys 2023-11-24T06:11:55.572 INFO:teuthology.orchestra.run.smithi118.stdout:/dev 2023-11-24T06:11:55.573 INFO:teuthology.orchestra.run.smithi118.stdout:/sys/kernel/security 2023-11-24T06:11:55.573 INFO:teuthology.orchestra.run.smithi118.stdout:/dev/shm 2023-11-24T06:11:55.573 INFO:teuthology.orchestra.run.smithi118.stdout:/dev/pts 2023-11-24T06:11:55.573 INFO:teuthology.orchestra.run.smithi118.stdout:/run 2023-11-24T06:11:55.573 INFO:teuthology.orchestra.run.smithi118.stdout:/sys/fs/cgroup 2023-11-24T06:11:55.573 INFO:teuthology.orchestra.run.smithi118.stdout:/sys/fs/pstore 2023-11-24T06:11:55.573 INFO:teuthology.orchestra.run.smithi118.stdout:/ 2023-11-24T06:11:55.573 INFO:teuthology.orchestra.run.smithi118.stdout:/sys/fs/selinux 2023-11-24T06:11:55.573 INFO:teuthology.orchestra.run.smithi118.stdout:/proc/sys/fs/binfmt_misc 2023-11-24T06:11:55.573 INFO:teuthology.orchestra.run.smithi118.stdout:/dev/mqueue 2023-11-24T06:11:55.573 INFO:teuthology.orchestra.run.smithi118.stdout:/dev/hugepages 2023-11-24T06:11:55.574 INFO:teuthology.orchestra.run.smithi118.stdout:/sys/kernel/debug 2023-11-24T06:11:55.574 INFO:teuthology.orchestra.run.smithi118.stdout:/sys/kernel/tracing 2023-11-24T06:11:55.574 INFO:teuthology.orchestra.run.smithi118.stdout:/sys/fs/fuse/connections 2023-11-24T06:11:55.574 INFO:teuthology.orchestra.run.smithi118.stdout:/sys/kernel/config 2023-11-24T06:11:55.574 INFO:teuthology.orchestra.run.smithi118.stdout:/run/credentials/systemd-sysctl.service 2023-11-24T06:11:55.574 INFO:teuthology.orchestra.run.smithi118.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-24T06:11:55.574 INFO:teuthology.orchestra.run.smithi118.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-24T06:11:55.574 INFO:teuthology.orchestra.run.smithi118.stdout:/var/lib/nfs/rpc_pipefs 2023-11-24T06:11:55.574 INFO:teuthology.orchestra.run.smithi118.stdout:/run/user/1000 2023-11-24T06:11:55.574 INFO:teuthology.orchestra.run.smithi118.stdout:/var/lib/ceph 2023-11-24T06:11:55.574 INFO:teuthology.orchestra.run.smithi118.stdout:/var/lib/ceph/osd/ceph-4 2023-11-24T06:11:55.575 INFO:teuthology.orchestra.run.smithi118.stdout:/var/lib/ceph/osd/ceph-5 2023-11-24T06:11:55.575 INFO:teuthology.orchestra.run.smithi118.stdout:/var/lib/ceph/osd/ceph-6 2023-11-24T06:11:55.575 INFO:teuthology.orchestra.run.smithi118.stdout:/var/lib/ceph/osd/ceph-7 2023-11-24T06:11:55.575 INFO:teuthology.orchestra.run.smithi118.stdout:/run/netns 2023-11-24T06:11:55.575 INFO:teuthology.orchestra.run.smithi118.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-24T06:11:55.575 INFO:teuthology.orchestra.run.smithi118.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-24T06:11:55.575 INFO:teuthology.orchestra.run.smithi118.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-24T06:11:55.575 INFO:teuthology.orchestra.run.smithi118.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-24T06:11:55.576 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:11:55.576 DEBUG:teuthology.orchestra.run.smithi118:> ls /sys/fs/fuse/connections 2023-11-24T06:11:55.623 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-11-24T06:11:55.623 DEBUG:teuthology.orchestra.run.smithi118:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2023-11-24T06:11:55.627 DEBUG:teuthology.orchestra.run.smithi118:> sudo modprobe fuse 2023-11-24T06:11:55.700 DEBUG:teuthology.orchestra.run.smithi118:> cat /proc/self/mounts | awk '{print $2}' 2023-11-24T06:11:55.741 INFO:teuthology.orchestra.run.smithi118.stdout:/proc 2023-11-24T06:11:55.741 INFO:teuthology.orchestra.run.smithi118.stdout:/sys 2023-11-24T06:11:55.741 INFO:teuthology.orchestra.run.smithi118.stdout:/dev 2023-11-24T06:11:55.741 INFO:teuthology.orchestra.run.smithi118.stdout:/sys/kernel/security 2023-11-24T06:11:55.741 INFO:teuthology.orchestra.run.smithi118.stdout:/dev/shm 2023-11-24T06:11:55.741 INFO:teuthology.orchestra.run.smithi118.stdout:/dev/pts 2023-11-24T06:11:55.741 INFO:teuthology.orchestra.run.smithi118.stdout:/run 2023-11-24T06:11:55.741 INFO:teuthology.orchestra.run.smithi118.stdout:/sys/fs/cgroup 2023-11-24T06:11:55.741 INFO:teuthology.orchestra.run.smithi118.stdout:/sys/fs/pstore 2023-11-24T06:11:55.741 INFO:teuthology.orchestra.run.smithi118.stdout:/ 2023-11-24T06:11:55.742 INFO:teuthology.orchestra.run.smithi118.stdout:/sys/fs/selinux 2023-11-24T06:11:55.742 INFO:teuthology.orchestra.run.smithi118.stdout:/proc/sys/fs/binfmt_misc 2023-11-24T06:11:55.742 INFO:teuthology.orchestra.run.smithi118.stdout:/dev/mqueue 2023-11-24T06:11:55.742 INFO:teuthology.orchestra.run.smithi118.stdout:/dev/hugepages 2023-11-24T06:11:55.742 INFO:teuthology.orchestra.run.smithi118.stdout:/sys/kernel/debug 2023-11-24T06:11:55.742 INFO:teuthology.orchestra.run.smithi118.stdout:/sys/kernel/tracing 2023-11-24T06:11:55.742 INFO:teuthology.orchestra.run.smithi118.stdout:/sys/fs/fuse/connections 2023-11-24T06:11:55.742 INFO:teuthology.orchestra.run.smithi118.stdout:/sys/kernel/config 2023-11-24T06:11:55.742 INFO:teuthology.orchestra.run.smithi118.stdout:/run/credentials/systemd-sysctl.service 2023-11-24T06:11:55.742 INFO:teuthology.orchestra.run.smithi118.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-24T06:11:55.742 INFO:teuthology.orchestra.run.smithi118.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-24T06:11:55.742 INFO:teuthology.orchestra.run.smithi118.stdout:/var/lib/nfs/rpc_pipefs 2023-11-24T06:11:55.743 INFO:teuthology.orchestra.run.smithi118.stdout:/run/user/1000 2023-11-24T06:11:55.743 INFO:teuthology.orchestra.run.smithi118.stdout:/var/lib/ceph 2023-11-24T06:11:55.743 INFO:teuthology.orchestra.run.smithi118.stdout:/var/lib/ceph/osd/ceph-4 2023-11-24T06:11:55.743 INFO:teuthology.orchestra.run.smithi118.stdout:/var/lib/ceph/osd/ceph-5 2023-11-24T06:11:55.743 INFO:teuthology.orchestra.run.smithi118.stdout:/var/lib/ceph/osd/ceph-6 2023-11-24T06:11:55.743 INFO:teuthology.orchestra.run.smithi118.stdout:/var/lib/ceph/osd/ceph-7 2023-11-24T06:11:55.743 INFO:teuthology.orchestra.run.smithi118.stdout:/run/netns 2023-11-24T06:11:55.743 INFO:teuthology.orchestra.run.smithi118.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-24T06:11:55.743 INFO:teuthology.orchestra.run.smithi118.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-24T06:11:55.743 INFO:teuthology.orchestra.run.smithi118.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-24T06:11:55.743 INFO:teuthology.orchestra.run.smithi118.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-24T06:11:55.744 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:11:55.744 DEBUG:teuthology.orchestra.run.smithi118:> ls /sys/fs/fuse/connections 2023-11-24T06:11:55.804 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi118.stderr:2023-11-24T06:11:55.801+0000 7fc457045480 -1 init, newargv = 0x558fe8be0970 newargc=15 2023-11-24T06:11:55.805 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi118.stderr:ceph-fuse[94335]: starting ceph client 2023-11-24T06:11:55.817 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi118.stderr:ceph-fuse[94335]: starting fuse 2023-11-24T06:11:55.830 INFO:teuthology.orchestra.run.smithi118.stdout:41 2023-11-24T06:11:55.830 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2023-11-24T06:11:55.830 DEBUG:teuthology.orchestra.run.smithi118:> sudo stdin-killer -- python3 -c ' 2023-11-24T06:11:55.830 DEBUG:teuthology.orchestra.run.smithi118:> import glob 2023-11-24T06:11:55.831 DEBUG:teuthology.orchestra.run.smithi118:> import re 2023-11-24T06:11:55.831 DEBUG:teuthology.orchestra.run.smithi118:> import os 2023-11-24T06:11:55.831 DEBUG:teuthology.orchestra.run.smithi118:> import subprocess 2023-11-24T06:11:55.831 DEBUG:teuthology.orchestra.run.smithi118:> 2023-11-24T06:11:55.831 DEBUG:teuthology.orchestra.run.smithi118:> def _find_admin_socket(client_name): 2023-11-24T06:11:55.831 DEBUG:teuthology.orchestra.run.smithi118:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2023-11-24T06:11:55.831 DEBUG:teuthology.orchestra.run.smithi118:> files = glob.glob(asok_path) 2023-11-24T06:11:55.831 DEBUG:teuthology.orchestra.run.smithi118:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2023-11-24T06:11:55.831 DEBUG:teuthology.orchestra.run.smithi118:> 2023-11-24T06:11:55.831 DEBUG:teuthology.orchestra.run.smithi118:> # Given a non-glob path, it better be there 2023-11-24T06:11:55.831 DEBUG:teuthology.orchestra.run.smithi118:> if "*" not in asok_path: 2023-11-24T06:11:55.831 DEBUG:teuthology.orchestra.run.smithi118:> assert(len(files) == 1) 2023-11-24T06:11:55.831 DEBUG:teuthology.orchestra.run.smithi118:> return files[0] 2023-11-24T06:11:55.832 DEBUG:teuthology.orchestra.run.smithi118:> 2023-11-24T06:11:55.832 DEBUG:teuthology.orchestra.run.smithi118:> for f in files: 2023-11-24T06:11:55.832 DEBUG:teuthology.orchestra.run.smithi118:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-11-24T06:11:55.832 DEBUG:teuthology.orchestra.run.smithi118:> if os.path.exists("/proc/{0}".format(pid)): 2023-11-24T06:11:55.832 DEBUG:teuthology.orchestra.run.smithi118:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-11-24T06:11:55.832 DEBUG:teuthology.orchestra.run.smithi118:> contents = proc_f.read() 2023-11-24T06:11:55.832 DEBUG:teuthology.orchestra.run.smithi118:> if mountpoint in contents: 2023-11-24T06:11:55.832 DEBUG:teuthology.orchestra.run.smithi118:> return f 2023-11-24T06:11:55.832 DEBUG:teuthology.orchestra.run.smithi118:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-11-24T06:11:55.832 DEBUG:teuthology.orchestra.run.smithi118:> 2023-11-24T06:11:55.832 DEBUG:teuthology.orchestra.run.smithi118:> print(_find_admin_socket("client.1")) 2023-11-24T06:11:55.832 DEBUG:teuthology.orchestra.run.smithi118:> ' 2023-11-24T06:11:56.033 INFO:teuthology.orchestra.run.smithi118.stdout:/var/run/ceph/ceph-client.1.94335.asok 2023-11-24T06:11:56.038 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:11:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-24T06:11:56.050 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.94335.asok 2023-11-24T06:11:56.050 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:11:56.051 DEBUG:teuthology.orchestra.run.smithi118:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.94335.asok status 2023-11-24T06:11:56.287 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2023-11-24T06:11:56.287 INFO:teuthology.orchestra.run.smithi118.stdout: "metadata": { 2023-11-24T06:11:56.288 INFO:teuthology.orchestra.run.smithi118.stdout: "ceph_sha1": "a13ea5ff00cb4df7a9cc941166ddeb358881144c", 2023-11-24T06:11:56.288 INFO:teuthology.orchestra.run.smithi118.stdout: "ceph_version": "ceph version 18.0.0-7484-ga13ea5ff (a13ea5ff00cb4df7a9cc941166ddeb358881144c) reef (dev)", 2023-11-24T06:11:56.288 INFO:teuthology.orchestra.run.smithi118.stdout: "entity_id": "1", 2023-11-24T06:11:56.288 INFO:teuthology.orchestra.run.smithi118.stdout: "hostname": "smithi118", 2023-11-24T06:11:56.288 INFO:teuthology.orchestra.run.smithi118.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2023-11-24T06:11:56.288 INFO:teuthology.orchestra.run.smithi118.stdout: "pid": "94335", 2023-11-24T06:11:56.288 INFO:teuthology.orchestra.run.smithi118.stdout: "root": "/" 2023-11-24T06:11:56.288 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-11-24T06:11:56.288 INFO:teuthology.orchestra.run.smithi118.stdout: "dentry_count": 0, 2023-11-24T06:11:56.288 INFO:teuthology.orchestra.run.smithi118.stdout: "dentry_pinned_count": 0, 2023-11-24T06:11:56.288 INFO:teuthology.orchestra.run.smithi118.stdout: "id": 4813, 2023-11-24T06:11:56.288 INFO:teuthology.orchestra.run.smithi118.stdout: "inst": { 2023-11-24T06:11:56.288 INFO:teuthology.orchestra.run.smithi118.stdout: "name": { 2023-11-24T06:11:56.289 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "client", 2023-11-24T06:11:56.289 INFO:teuthology.orchestra.run.smithi118.stdout: "num": 4813 2023-11-24T06:11:56.289 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-11-24T06:11:56.289 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": { 2023-11-24T06:11:56.289 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v1", 2023-11-24T06:11:56.289 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "192.168.0.1:0", 2023-11-24T06:11:56.289 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 2380074623 2023-11-24T06:11:56.289 INFO:teuthology.orchestra.run.smithi118.stdout: } 2023-11-24T06:11:56.289 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-11-24T06:11:56.289 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": { 2023-11-24T06:11:56.289 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v1", 2023-11-24T06:11:56.289 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "192.168.0.1:0", 2023-11-24T06:11:56.289 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 2380074623 2023-11-24T06:11:56.290 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-11-24T06:11:56.290 INFO:teuthology.orchestra.run.smithi118.stdout: "inst_str": "client.4813 192.168.0.1:0/2380074623", 2023-11-24T06:11:56.290 INFO:teuthology.orchestra.run.smithi118.stdout: "addr_str": "192.168.0.1:0/2380074623", 2023-11-24T06:11:56.290 INFO:teuthology.orchestra.run.smithi118.stdout: "inode_count": 1, 2023-11-24T06:11:56.290 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_epoch": 17, 2023-11-24T06:11:56.290 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_epoch": 33, 2023-11-24T06:11:56.290 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_epoch_barrier": 0, 2023-11-24T06:11:56.290 INFO:teuthology.orchestra.run.smithi118.stdout: "blocklisted": false, 2023-11-24T06:11:56.290 INFO:teuthology.orchestra.run.smithi118.stdout: "fs_name": "cephfs" 2023-11-24T06:11:56.290 INFO:teuthology.orchestra.run.smithi118.stdout:} 2023-11-24T06:11:56.301 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:11:56.302 DEBUG:teuthology.orchestra.run.smithi118:> stat --file-system '--printf=%T 2023-11-24T06:11:56.302 DEBUG:teuthology.orchestra.run.smithi118:> ' -- /home/ubuntu/cephtest/mnt.1 2023-11-24T06:11:56.356 INFO:teuthology.orchestra.run.smithi118.stdout:fuseblk 2023-11-24T06:11:56.356 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2023-11-24T06:11:56.356 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:11:56.356 DEBUG:teuthology.orchestra.run.smithi118:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-11-24T06:11:56.448 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2023-11-24T06:11:56.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:56.674+0000 7fd993fa9640 1 Processor -- start 2023-11-24T06:11:56.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:56.674+0000 7fd993fa9640 1 -- start start 2023-11-24T06:11:56.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:56.676+0000 7fd993fa9640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd98c14b2c0 0x7fd98c0cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:56.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:56.676+0000 7fd993fa9640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd98c0cfbf0 con 0x7fd98c071070 2023-11-24T06:11:56.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:56.676+0000 7fd993fa9640 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd98c071690 con 0x7fd98c0cf7d0 2023-11-24T06:11:56.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:56.676+0000 7fd993fa9640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd98c14b6a0 con 0x7fd98c14b2c0 2023-11-24T06:11:56.679 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:56.676+0000 7fd992546640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fd98c071070 0x7fd98c0cc160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:41288/0 (socket says 172.21.15.92:41288) 2023-11-24T06:11:56.679 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:56.676+0000 7fd992546640 1 -- 172.21.15.92:0/2514397695 learned_addr learned my addr 172.21.15.92:0/2514397695 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-24T06:11:56.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:56.677+0000 7fd991d45640 1 --2- 172.21.15.92:0/2514397695 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd98c14b2c0 0x7fd98c0cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:56.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:56.677+0000 7fd990d43640 1 -- 172.21.15.92:0/2514397695 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4044729618 0 0) 0x7fd98c0cfbf0 con 0x7fd98c071070 2023-11-24T06:11:56.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:56.677+0000 7fd990d43640 1 -- 172.21.15.92:0/2514397695 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd978003520 con 0x7fd98c071070 2023-11-24T06:11:56.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:56.677+0000 7fd990d43640 1 -- 172.21.15.92:0/2514397695 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2293518692 0 0) 0x7fd978003520 con 0x7fd98c071070 2023-11-24T06:11:56.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:56.677+0000 7fd990d43640 1 -- 172.21.15.92:0/2514397695 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd98c14b2c0 msgr2=0x7fd98c0cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:56.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:56.677+0000 7fd990d43640 1 --2- 172.21.15.92:0/2514397695 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd98c14b2c0 0x7fd98c0cf290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:56.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:56.677+0000 7fd990d43640 1 -- 172.21.15.92:0/2514397695 >> v1:172.21.15.118:6789/0 conn(0x7fd98c0cf7d0 legacy=0x7fd98c070960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:56.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:56.677+0000 7fd990d43640 1 -- 172.21.15.92:0/2514397695 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd98c071970 con 0x7fd98c071070 2023-11-24T06:11:56.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:56.677+0000 7fd990d43640 1 -- 172.21.15.92:0/2514397695 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2607661350 0 0) 0x7fd9840019f0 con 0x7fd98c071070 2023-11-24T06:11:56.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:56.677+0000 7fd990d43640 1 -- 172.21.15.92:0/2514397695 >> v1:172.21.15.92:6789/0 conn(0x7fd98c071070 legacy=0x7fd98c0cc160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:56.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:56.677+0000 7fd990d43640 1 --2- 172.21.15.92:0/2514397695 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd98c14b2c0 0x7fd9780039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:56.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:56.677+0000 7fd990d43640 1 --2- 172.21.15.92:0/2514397695 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd978003fd0 0x7fd9780047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:56.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:56.677+0000 7fd990d43640 1 --2- 172.21.15.92:0/2514397695 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd978004d00 0x7fd978005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:56.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:56.677+0000 7fd990d43640 1 -- 172.21.15.92:0/2514397695 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd98c14b6a0 con 0x7fd978003fd0 2023-11-24T06:11:56.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:56.677+0000 7fd990d43640 1 -- 172.21.15.92:0/2514397695 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd97800bb40 con 0x7fd978004d00 2023-11-24T06:11:56.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:56.677+0000 7fd990d43640 1 -- 172.21.15.92:0/2514397695 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd97800bd00 con 0x7fd98c14b2c0 2023-11-24T06:11:56.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:56.678+0000 7fd991544640 1 --2- 172.21.15.92:0/2514397695 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd978003fd0 0x7fd9780047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:56.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:56.678+0000 7fd991d45640 1 --2- 172.21.15.92:0/2514397695 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd98c14b2c0 0x7fd9780039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:56.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:56.678+0000 7fd992546640 1 --2- 172.21.15.92:0/2514397695 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd978004d00 0x7fd978005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:56.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:56.678+0000 7fd991544640 1 -- 172.21.15.92:0/2514397695 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd98c14b2c0 msgr2=0x7fd9780039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:56.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:56.678+0000 7fd991544640 1 --2- 172.21.15.92:0/2514397695 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd98c14b2c0 0x7fd9780039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:56.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:56.678+0000 7fd991544640 1 -- 172.21.15.92:0/2514397695 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd978004d00 msgr2=0x7fd978005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:56.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:56.678+0000 7fd991544640 1 --2- 172.21.15.92:0/2514397695 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd978004d00 0x7fd978005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:56.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:56.678+0000 7fd991544640 1 -- 172.21.15.92:0/2514397695 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd978003520 con 0x7fd978003fd0 2023-11-24T06:11:56.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:56.678+0000 7fd991d45640 1 --2- 172.21.15.92:0/2514397695 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd98c14b2c0 0x7fd9780039a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:11:56.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:56.678+0000 7fd992546640 1 --2- 172.21.15.92:0/2514397695 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd978004d00 0x7fd978005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:11:56.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:56.679+0000 7fd991544640 1 --2- 172.21.15.92:0/2514397695 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd978003fd0 0x7fd9780047c0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fd97c001750 tx=0x7fd9780057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a5831d67f00f13e server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:56.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:56.679+0000 7fd990d43640 1 -- 172.21.15.92:0/2514397695 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd97c0052d0 con 0x7fd978003fd0 2023-11-24T06:11:56.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:56.679+0000 7fd990d43640 1 -- 172.21.15.92:0/2514397695 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd97c02e050 con 0x7fd978003fd0 2023-11-24T06:11:56.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:56.679+0000 7fd990d43640 1 -- 172.21.15.92:0/2514397695 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd97c003830 con 0x7fd978003fd0 2023-11-24T06:11:56.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:56.680+0000 7fd993fa9640 1 -- 172.21.15.92:0/2514397695 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd978003fd0 msgr2=0x7fd9780047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:56.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:56.680+0000 7fd993fa9640 1 --2- 172.21.15.92:0/2514397695 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd978003fd0 0x7fd9780047c0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fd97c001750 tx=0x7fd9780057f0 comp rx=0 tx=0).stop 2023-11-24T06:11:56.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:56.680+0000 7fd992546640 1 -- 172.21.15.92:0/2514397695 reap_dead start 2023-11-24T06:11:56.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:56.680+0000 7fd993fa9640 1 -- 172.21.15.92:0/2514397695 shutdown_connections 2023-11-24T06:11:56.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:56.680+0000 7fd993fa9640 1 -- 172.21.15.92:0/2514397695 >> 172.21.15.92:0/2514397695 conn(0x7fd98c06a800 msgr2=0x7fd98c06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:56.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:56.680+0000 7fd993fa9640 1 -- 172.21.15.92:0/2514397695 shutdown_connections 2023-11-24T06:11:56.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:56.680+0000 7fd993fa9640 1 -- 172.21.15.92:0/2514397695 wait complete. 2023-11-24T06:11:56.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:56.682+0000 7fd993fa9640 1 Processor -- start 2023-11-24T06:11:56.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:56.682+0000 7fd993fa9640 1 -- start start 2023-11-24T06:11:56.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:56.683+0000 7fd993fa9640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd978003fd0 0x7fd98c1f8440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:56.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:56.683+0000 7fd993fa9640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd98c1610f0 0x7fd98c1488e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:56.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:56.683+0000 7fd993fa9640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd98c161550 0x7fd98c148e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:56.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:56.683+0000 7fd993fa9640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd98c1f8980 con 0x7fd98c1610f0 2023-11-24T06:11:56.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:56.683+0000 7fd993fa9640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd98c13ec00 con 0x7fd98c161550 2023-11-24T06:11:56.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:56.683+0000 7fd993fa9640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd98c13eee0 con 0x7fd978003fd0 2023-11-24T06:11:56.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:56.683+0000 7fd991544640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd98c1610f0 0x7fd98c1488e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:56.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:56.683+0000 7fd991544640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd98c1610f0 0x7fd98c1488e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58764/0 (socket says 172.21.15.92:58764) 2023-11-24T06:11:56.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:56.683+0000 7fd991544640 1 -- 172.21.15.92:0/2555737307 learned_addr learned my addr 172.21.15.92:0/2555737307 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:11:56.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:56.684+0000 7fd992546640 1 --2- 172.21.15.92:0/2555737307 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd98c161550 0x7fd98c148e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:56.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:56.684+0000 7fd991544640 1 -- 172.21.15.92:0/2555737307 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd978003fd0 msgr2=0x7fd98c1f8440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:56.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:56.684+0000 7fd991d45640 1 --2- 172.21.15.92:0/2555737307 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd978003fd0 0x7fd98c1f8440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:56.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:56.684+0000 7fd991544640 1 --2- 172.21.15.92:0/2555737307 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd978003fd0 0x7fd98c1f8440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:56.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:56.684+0000 7fd991544640 1 -- 172.21.15.92:0/2555737307 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd98c161550 msgr2=0x7fd98c148e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:56.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:56.684+0000 7fd991544640 1 --2- 172.21.15.92:0/2555737307 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd98c161550 0x7fd98c148e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:56.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:56.684+0000 7fd991544640 1 -- 172.21.15.92:0/2555737307 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd98c13f1c0 con 0x7fd98c1610f0 2023-11-24T06:11:56.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:56.684+0000 7fd991544640 1 --2- 172.21.15.92:0/2555737307 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd98c1610f0 0x7fd98c1488e0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fd97c008fd0 tx=0x7fd97c001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:56.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:56.684+0000 7fd9827fc640 1 -- 172.21.15.92:0/2555737307 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd97c0076c0 con 0x7fd98c1610f0 2023-11-24T06:11:56.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:56.684+0000 7fd9827fc640 1 -- 172.21.15.92:0/2555737307 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd97c007a30 con 0x7fd98c1610f0 2023-11-24T06:11:56.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:56.684+0000 7fd9827fc640 1 -- 172.21.15.92:0/2555737307 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd97c006be0 con 0x7fd98c1610f0 2023-11-24T06:11:56.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:56.684+0000 7fd993fa9640 1 -- 172.21.15.92:0/2555737307 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd98c13f4a0 con 0x7fd98c1610f0 2023-11-24T06:11:56.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:56.685+0000 7fd993fa9640 1 -- 172.21.15.92:0/2555737307 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd98c1637d0 con 0x7fd98c1610f0 2023-11-24T06:11:56.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:56.686+0000 7fd993fa9640 1 -- 172.21.15.92:0/2555737307 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd98c067000 con 0x7fd98c1610f0 2023-11-24T06:11:56.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:56.687+0000 7fd9827fc640 1 -- 172.21.15.92:0/2555737307 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd97c006d40 con 0x7fd98c1610f0 2023-11-24T06:11:56.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:56.688+0000 7fd9827fc640 1 --2- 172.21.15.92:0/2555737307 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fd97006ae40 0x7fd97006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:56.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:56.688+0000 7fd9827fc640 1 -- 172.21.15.92:0/2555737307 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd97c0ab770 con 0x7fd98c1610f0 2023-11-24T06:11:56.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:56.688+0000 7fd991d45640 1 --2- 172.21.15.92:0/2555737307 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fd97006ae40 0x7fd97006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:56.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:56.689+0000 7fd991d45640 1 --2- 172.21.15.92:0/2555737307 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fd97006ae40 0x7fd97006d300 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fd988006fd0 tx=0x7fd988008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:56.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:56.692+0000 7fd9827fc640 1 -- 172.21.15.92:0/2555737307 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd97c076780 con 0x7fd98c1610f0 2023-11-24T06:11:56.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:56.923+0000 7fd993fa9640 1 -- 172.21.15.92:0/2555737307 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fd98c0675d0 con 0x7fd98c1610f0 2023-11-24T06:11:57.575 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:57.573+0000 7fd9827fc640 1 -- 172.21.15.92:0/2555737307 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v18) v1 ==== 136+0+0 (secure 0 0 0) 0x7fd97c079da0 con 0x7fd98c1610f0 2023-11-24T06:11:57.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:57.577+0000 7fd993fa9640 1 -- 172.21.15.92:0/2555737307 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fd97006ae40 msgr2=0x7fd97006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:57.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:57.577+0000 7fd993fa9640 1 --2- 172.21.15.92:0/2555737307 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fd97006ae40 0x7fd97006d300 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fd988006fd0 tx=0x7fd988008040 comp rx=0 tx=0).stop 2023-11-24T06:11:57.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:57.577+0000 7fd993fa9640 1 -- 172.21.15.92:0/2555737307 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd98c1610f0 msgr2=0x7fd98c1488e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:57.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:57.577+0000 7fd993fa9640 1 --2- 172.21.15.92:0/2555737307 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd98c1610f0 0x7fd98c1488e0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fd97c008fd0 tx=0x7fd97c001120 comp rx=0 tx=0).stop 2023-11-24T06:11:57.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:57.577+0000 7fd993fa9640 1 -- 172.21.15.92:0/2555737307 shutdown_connections 2023-11-24T06:11:57.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:57.577+0000 7fd993fa9640 1 --2- 172.21.15.92:0/2555737307 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fd97006ae40 0x7fd97006d300 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:57.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:57.577+0000 7fd993fa9640 1 --2- 172.21.15.92:0/2555737307 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd98c161550 0x7fd98c148e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:57.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:57.577+0000 7fd993fa9640 1 --2- 172.21.15.92:0/2555737307 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd98c1610f0 0x7fd98c1488e0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:57.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:57.577+0000 7fd993fa9640 1 --2- 172.21.15.92:0/2555737307 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd978003fd0 0x7fd98c1f8440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:57.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:57.577+0000 7fd993fa9640 1 -- 172.21.15.92:0/2555737307 >> 172.21.15.92:0/2555737307 conn(0x7fd98c06a800 msgr2=0x7fd98c0afde0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:57.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:57.578+0000 7fd993fa9640 1 -- 172.21.15.92:0/2555737307 shutdown_connections 2023-11-24T06:11:57.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:57.578+0000 7fd993fa9640 1 -- 172.21.15.92:0/2555737307 wait complete. 2023-11-24T06:11:57.601 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-11-24T06:11:57.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:57.820+0000 7fa31746b640 1 Processor -- start 2023-11-24T06:11:57.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:57.820+0000 7fa31746b640 1 -- start start 2023-11-24T06:11:57.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:57.822+0000 7fa31746b640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fa31013e360 0x7fa3100cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:57.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:57.822+0000 7fa31746b640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fa31015f9c0 0x7fa3100cf7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:57.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:57.822+0000 7fa31746b640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa31015c5f0 con 0x7fa31013df80 2023-11-24T06:11:57.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:57.822+0000 7fa31746b640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa31006c470 con 0x7fa31013e360 2023-11-24T06:11:57.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:57.822+0000 7fa31746b640 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa31006c750 con 0x7fa31015f9c0 2023-11-24T06:11:57.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:57.822+0000 7fa314a06640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fa31013e360 0x7fa3100cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:57.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:57.822+0000 7fa314a06640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fa31013e360 0x7fa3100cf290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58776/0 (socket says 172.21.15.92:58776) 2023-11-24T06:11:57.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:57.822+0000 7fa314a06640 1 -- 172.21.15.92:0/3688426904 learned_addr learned my addr 172.21.15.92:0/3688426904 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:11:57.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:57.823+0000 7fa314a06640 1 -- 172.21.15.92:0/3688426904 >> v2:172.21.15.118:3300/0 conn(0x7fa31015f9c0 msgr2=0x7fa3100cf7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:57.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:57.823+0000 7fa314a06640 1 --2- 172.21.15.92:0/3688426904 >> v2:172.21.15.118:3300/0 conn(0x7fa31015f9c0 0x7fa3100cf7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:57.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:57.823+0000 7fa314a06640 1 -- 172.21.15.92:0/3688426904 >> v1:172.21.15.92:6789/0 conn(0x7fa31013df80 legacy=0x7fa31015bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:57.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:57.823+0000 7fa314a06640 1 -- 172.21.15.92:0/3688426904 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa31006ca30 con 0x7fa31013e360 2023-11-24T06:11:57.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:57.824+0000 7fa314a06640 1 --2- 172.21.15.92:0/3688426904 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa31013e360 0x7fa3100cf290 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fa3100ccae0 tx=0x7fa30002b9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19a8241ff5172d38 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:57.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:57.824+0000 7fa307fff640 1 -- 172.21.15.92:0/3688426904 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa300030870 con 0x7fa31013e360 2023-11-24T06:11:57.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:57.824+0000 7fa307fff640 1 -- 172.21.15.92:0/3688426904 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3000309d0 con 0x7fa31013e360 2023-11-24T06:11:57.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:57.824+0000 7fa307fff640 1 -- 172.21.15.92:0/3688426904 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa300030d60 con 0x7fa31013e360 2023-11-24T06:11:57.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:57.825+0000 7fa31746b640 1 -- 172.21.15.92:0/3688426904 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa31013e360 msgr2=0x7fa3100cf290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:57.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:57.825+0000 7fa31746b640 1 --2- 172.21.15.92:0/3688426904 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa31013e360 0x7fa3100cf290 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fa3100ccae0 tx=0x7fa30002b9b0 comp rx=0 tx=0).stop 2023-11-24T06:11:57.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:57.825+0000 7fa31746b640 1 -- 172.21.15.92:0/3688426904 shutdown_connections 2023-11-24T06:11:57.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:57.825+0000 7fa31746b640 1 --2- 172.21.15.92:0/3688426904 >> v2:172.21.15.118:3300/0 conn(0x7fa31015f9c0 0x7fa3100cf7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:57.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:57.825+0000 7fa31746b640 1 --2- 172.21.15.92:0/3688426904 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa31013e360 0x7fa3100cf290 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:57.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:57.825+0000 7fa31746b640 1 -- 172.21.15.92:0/3688426904 >> 172.21.15.92:0/3688426904 conn(0x7fa31006a800 msgr2=0x7fa31006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:57.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:57.825+0000 7fa31746b640 1 -- 172.21.15.92:0/3688426904 shutdown_connections 2023-11-24T06:11:57.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:57.825+0000 7fa31746b640 1 -- 172.21.15.92:0/3688426904 wait complete. 2023-11-24T06:11:57.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:57.827+0000 7fa31746b640 1 Processor -- start 2023-11-24T06:11:57.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:57.827+0000 7fa31746b640 1 -- start start 2023-11-24T06:11:57.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:57.828+0000 7fa31746b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa31013df80 0x7fa3100a8730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:57.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:57.828+0000 7fa31746b640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa31013e360 0x7fa3100ad9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:57.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:57.828+0000 7fa31746b640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa31015f9c0 0x7fa3100adf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:57.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:57.828+0000 7fa31746b640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa310130f10 con 0x7fa31013df80 2023-11-24T06:11:57.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:57.828+0000 7fa31746b640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa310131330 con 0x7fa31015f9c0 2023-11-24T06:11:57.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:57.828+0000 7fa31746b640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa310130290 con 0x7fa31013e360 2023-11-24T06:11:57.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:57.828+0000 7fa315207640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa31013df80 0x7fa3100a8730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:57.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:57.828+0000 7fa315207640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa31013df80 0x7fa3100a8730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58778/0 (socket says 172.21.15.92:58778) 2023-11-24T06:11:57.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:57.828+0000 7fa315207640 1 -- 172.21.15.92:0/3556484095 learned_addr learned my addr 172.21.15.92:0/3556484095 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:11:57.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:57.828+0000 7fa315207640 1 -- 172.21.15.92:0/3556484095 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa31013e360 msgr2=0x7fa3100ad9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:57.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:57.829+0000 7fa314a06640 1 --2- 172.21.15.92:0/3556484095 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa31013e360 0x7fa3100ad9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:57.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:57.829+0000 7fa315a08640 1 --2- 172.21.15.92:0/3556484095 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa31015f9c0 0x7fa3100adf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:57.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:57.829+0000 7fa315207640 1 --2- 172.21.15.92:0/3556484095 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa31013e360 0x7fa3100ad9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:57.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:57.829+0000 7fa315207640 1 -- 172.21.15.92:0/3556484095 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa31015f9c0 msgr2=0x7fa3100adf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:57.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:57.829+0000 7fa315207640 1 --2- 172.21.15.92:0/3556484095 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa31015f9c0 0x7fa3100adf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:57.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:57.829+0000 7fa315207640 1 -- 172.21.15.92:0/3556484095 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3000094d0 con 0x7fa31013df80 2023-11-24T06:11:57.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:57.829+0000 7fa315a08640 1 --2- 172.21.15.92:0/3556484095 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa31015f9c0 0x7fa3100adf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:11:57.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:57.829+0000 7fa315207640 1 --2- 172.21.15.92:0/3556484095 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa31013df80 0x7fa3100a8730 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fa30c00c390 tx=0x7fa30c00c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:57.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:57.829+0000 7fa305ffb640 1 -- 172.21.15.92:0/3556484095 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa30c0052d0 con 0x7fa31013df80 2023-11-24T06:11:57.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:57.829+0000 7fa305ffb640 1 -- 172.21.15.92:0/3556484095 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa30c005430 con 0x7fa31013df80 2023-11-24T06:11:57.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:57.829+0000 7fa305ffb640 1 -- 172.21.15.92:0/3556484095 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa30c003830 con 0x7fa31013df80 2023-11-24T06:11:57.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:57.829+0000 7fa31746b640 1 -- 172.21.15.92:0/3556484095 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa310130570 con 0x7fa31013df80 2023-11-24T06:11:57.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:57.829+0000 7fa31746b640 1 -- 172.21.15.92:0/3556484095 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa310130b30 con 0x7fa31013df80 2023-11-24T06:11:57.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:57.830+0000 7fa31746b640 1 -- 172.21.15.92:0/3556484095 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa310131180 con 0x7fa31013df80 2023-11-24T06:11:57.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:57.831+0000 7fa305ffb640 1 -- 172.21.15.92:0/3556484095 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa30c003990 con 0x7fa31013df80 2023-11-24T06:11:57.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:57.832+0000 7fa305ffb640 1 --2- 172.21.15.92:0/3556484095 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fa2f406ae40 0x7fa2f406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:57.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:57.832+0000 7fa305ffb640 1 -- 172.21.15.92:0/3556484095 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa30c088b10 con 0x7fa31013df80 2023-11-24T06:11:57.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:57.832+0000 7fa314a06640 1 --2- 172.21.15.92:0/3556484095 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fa2f406ae40 0x7fa2f406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:57.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:57.833+0000 7fa314a06640 1 --2- 172.21.15.92:0/3556484095 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fa2f406ae40 0x7fa2f406d300 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fa300036000 tx=0x7fa30003b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:57.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:57.836+0000 7fa305ffb640 1 -- 172.21.15.92:0/3556484095 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa30c053c50 con 0x7fa31013df80 2023-11-24T06:11:58.051 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:58.049+0000 7fa31746b640 1 -- 172.21.15.92:0/3556484095 --> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fa31006cff0 con 0x7fa2f406ae40 2023-11-24T06:11:58.054 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:58.052+0000 7fa305ffb640 1 -- 172.21.15.92:0/3556484095 <== mgr.4101 v2:172.21.15.92:6800/784027401 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7fa300030c40 con 0x7fa2f406ae40 2023-11-24T06:11:58.054 INFO:teuthology.orchestra.run.smithi092.stdout:[ 2023-11-24T06:11:58.055 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-24T06:11:58.055 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "cephfs" 2023-11-24T06:11:58.055 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-24T06:11:58.055 INFO:teuthology.orchestra.run.smithi092.stdout:] 2023-11-24T06:11:58.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:58.054+0000 7fa31746b640 1 -- 172.21.15.92:0/3556484095 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fa2f406ae40 msgr2=0x7fa2f406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:58.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:58.054+0000 7fa31746b640 1 --2- 172.21.15.92:0/3556484095 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fa2f406ae40 0x7fa2f406d300 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fa300036000 tx=0x7fa30003b040 comp rx=0 tx=0).stop 2023-11-24T06:11:58.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:58.054+0000 7fa31746b640 1 -- 172.21.15.92:0/3556484095 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa31013df80 msgr2=0x7fa3100a8730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:58.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:58.054+0000 7fa31746b640 1 --2- 172.21.15.92:0/3556484095 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa31013df80 0x7fa3100a8730 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fa30c00c390 tx=0x7fa30c00c850 comp rx=0 tx=0).stop 2023-11-24T06:11:58.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:58.055+0000 7fa31746b640 1 -- 172.21.15.92:0/3556484095 shutdown_connections 2023-11-24T06:11:58.058 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:58.055+0000 7fa31746b640 1 --2- 172.21.15.92:0/3556484095 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fa2f406ae40 0x7fa2f406d300 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:58.058 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:58.055+0000 7fa31746b640 1 --2- 172.21.15.92:0/3556484095 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa31015f9c0 0x7fa3100adf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:58.058 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:58.055+0000 7fa31746b640 1 --2- 172.21.15.92:0/3556484095 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa31013e360 0x7fa3100ad9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:58.058 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:58.055+0000 7fa31746b640 1 --2- 172.21.15.92:0/3556484095 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa31013df80 0x7fa3100a8730 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:58.058 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:58.055+0000 7fa31746b640 1 -- 172.21.15.92:0/3556484095 >> 172.21.15.92:0/3556484095 conn(0x7fa31006a800 msgr2=0x7fa3100afb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:58.058 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:58.055+0000 7fa31746b640 1 -- 172.21.15.92:0/3556484095 shutdown_connections 2023-11-24T06:11:58.058 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:58.055+0000 7fa31746b640 1 -- 172.21.15.92:0/3556484095 wait complete. 2023-11-24T06:11:58.073 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-24T06:11:58.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:58.284+0000 7f00b491d640 1 Processor -- start 2023-11-24T06:11:58.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:58.284+0000 7f00b491d640 1 -- start start 2023-11-24T06:11:58.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:58.286+0000 7f00b491d640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f00b0144d00 0x7f00b00ce340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:58.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:58.286+0000 7f00b491d640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f00b00d0a90 0x7f00b00b2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:58.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:58.286+0000 7f00b491d640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00b00cd180 con 0x7f00b00d0610 2023-11-24T06:11:58.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:58.286+0000 7f00b491d640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00b00cd420 con 0x7f00b00d0a90 2023-11-24T06:11:58.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:58.286+0000 7f00b491d640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00b00cf020 con 0x7f00b0144d00 2023-11-24T06:11:58.289 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:58.286+0000 7f00add9b640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f00b00d0610 0x7f00b00cf7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:41318/0 (socket says 172.21.15.92:41318) 2023-11-24T06:11:58.289 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:58.286+0000 7f00aed9d640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f00b00d0a90 0x7f00b00b2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:58.289 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:58.286+0000 7f00add9b640 1 -- 172.21.15.92:0/3584955962 learned_addr learned my addr 172.21.15.92:0/3584955962 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-24T06:11:58.289 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:58.286+0000 7f00ae59c640 1 --2- 172.21.15.92:0/3584955962 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f00b0144d00 0x7f00b00ce340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:58.289 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:58.287+0000 7f00aed9d640 1 -- 172.21.15.92:0/3584955962 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f00b0144d00 msgr2=0x7f00b00ce340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:58.289 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:58.287+0000 7f00aed9d640 1 --2- 172.21.15.92:0/3584955962 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f00b0144d00 0x7f00b00ce340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:58.289 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:58.287+0000 7f00aed9d640 1 -- 172.21.15.92:0/3584955962 >> v1:172.21.15.92:6789/0 conn(0x7f00b00d0610 legacy=0x7f00b00cf7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:58.290 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:58.287+0000 7f00ad59a640 1 -- 172.21.15.92:0/3584955962 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3616941922 0 0) 0x7f00b00cd180 con 0x7f00b00d0610 2023-11-24T06:11:58.290 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:58.287+0000 7f00aed9d640 1 -- 172.21.15.92:0/3584955962 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00b00cd6c0 con 0x7f00b00d0a90 2023-11-24T06:11:58.290 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:58.288+0000 7f00aed9d640 1 --2- 172.21.15.92:0/3584955962 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f00b00d0a90 0x7f00b00b2ce0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f00a80097a0 tx=0x7f00a802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6478c16e55e4e319 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:58.290 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:58.288+0000 7f00ad59a640 1 -- 172.21.15.92:0/3584955962 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00a8031830 con 0x7f00b00d0a90 2023-11-24T06:11:58.290 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:58.288+0000 7f00ad59a640 1 -- 172.21.15.92:0/3584955962 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00a8031990 con 0x7f00b00d0a90 2023-11-24T06:11:58.291 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:58.288+0000 7f00ad59a640 1 -- 172.21.15.92:0/3584955962 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00a8031d20 con 0x7f00b00d0a90 2023-11-24T06:11:58.291 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:58.289+0000 7f00b491d640 1 -- 172.21.15.92:0/3584955962 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f00b00d0a90 msgr2=0x7f00b00b2ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:58.291 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:58.289+0000 7f00b491d640 1 --2- 172.21.15.92:0/3584955962 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f00b00d0a90 0x7f00b00b2ce0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f00a80097a0 tx=0x7f00a802b8d0 comp rx=0 tx=0).stop 2023-11-24T06:11:58.292 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:58.289+0000 7f00b491d640 1 -- 172.21.15.92:0/3584955962 shutdown_connections 2023-11-24T06:11:58.292 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:58.289+0000 7f00b491d640 1 --2- 172.21.15.92:0/3584955962 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f00b00d0a90 0x7f00b00b2ce0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:58.292 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:58.289+0000 7f00b491d640 1 --2- 172.21.15.92:0/3584955962 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f00b0144d00 0x7f00b00ce340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:58.292 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:58.289+0000 7f00b491d640 1 -- 172.21.15.92:0/3584955962 >> 172.21.15.92:0/3584955962 conn(0x7f00b0066690 msgr2=0x7f00b006a3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:58.292 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:58.289+0000 7f00b491d640 1 -- 172.21.15.92:0/3584955962 shutdown_connections 2023-11-24T06:11:58.292 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:58.289+0000 7f00b491d640 1 -- 172.21.15.92:0/3584955962 wait complete. 2023-11-24T06:11:58.292 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:58.291+0000 7f00b491d640 1 Processor -- start 2023-11-24T06:11:58.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:58.291+0000 7f00b491d640 1 -- start start 2023-11-24T06:11:58.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:58.292+0000 7f00b491d640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f00b00d0610 0x7f00b01f31e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:58.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:58.292+0000 7f00b491d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f00b00d0a90 0x7f00b0142e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:58.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:58.292+0000 7f00b491d640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f00b0144d00 0x7f00b015e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:58.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:58.292+0000 7f00b491d640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00b0140de0 con 0x7f00b00d0a90 2023-11-24T06:11:58.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:58.292+0000 7f00b491d640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00b006b940 con 0x7f00b00d0610 2023-11-24T06:11:58.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:58.292+0000 7f00b491d640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00b006bc20 con 0x7f00b0144d00 2023-11-24T06:11:58.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:58.292+0000 7f00add9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f00b00d0a90 0x7f00b0142e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:58.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:58.292+0000 7f00add9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f00b00d0a90 0x7f00b0142e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58806/0 (socket says 172.21.15.92:58806) 2023-11-24T06:11:58.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:58.292+0000 7f00add9b640 1 -- 172.21.15.92:0/273840551 learned_addr learned my addr 172.21.15.92:0/273840551 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:11:58.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:58.292+0000 7f00ae59c640 1 --2- 172.21.15.92:0/273840551 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f00b00d0610 0x7f00b01f31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:58.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:58.292+0000 7f00aed9d640 1 --2- 172.21.15.92:0/273840551 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f00b0144d00 0x7f00b015e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:58.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:58.292+0000 7f00add9b640 1 -- 172.21.15.92:0/273840551 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f00b0144d00 msgr2=0x7f00b015e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:58.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:58.292+0000 7f00add9b640 1 --2- 172.21.15.92:0/273840551 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f00b0144d00 0x7f00b015e1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:58.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:58.292+0000 7f00add9b640 1 -- 172.21.15.92:0/273840551 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f00b00d0610 msgr2=0x7f00b01f31e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:58.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:58.292+0000 7f00add9b640 1 --2- 172.21.15.92:0/273840551 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f00b00d0610 0x7f00b01f31e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:58.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:58.292+0000 7f00add9b640 1 -- 172.21.15.92:0/273840551 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00a8009400 con 0x7f00b00d0a90 2023-11-24T06:11:58.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:58.292+0000 7f00ae59c640 1 --2- 172.21.15.92:0/273840551 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f00b00d0610 0x7f00b01f31e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:11:58.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:58.293+0000 7f00add9b640 1 --2- 172.21.15.92:0/273840551 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f00b00d0a90 0x7f00b0142e90 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f009800c4d0 tx=0x7f009800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:58.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:58.293+0000 7f00a6ffd640 1 -- 172.21.15.92:0/273840551 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00980074e0 con 0x7f00b00d0a90 2023-11-24T06:11:58.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:58.293+0000 7f00a6ffd640 1 -- 172.21.15.92:0/273840551 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0098007660 con 0x7f00b00d0a90 2023-11-24T06:11:58.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:58.293+0000 7f00b491d640 1 -- 172.21.15.92:0/273840551 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00b006bf00 con 0x7f00b00d0a90 2023-11-24T06:11:58.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:58.293+0000 7f00a6ffd640 1 -- 172.21.15.92:0/273840551 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0098004a20 con 0x7f00b00d0a90 2023-11-24T06:11:58.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:58.293+0000 7f00b491d640 1 -- 172.21.15.92:0/273840551 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00b006c4c0 con 0x7f00b00d0a90 2023-11-24T06:11:58.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:58.294+0000 7f00b491d640 1 -- 172.21.15.92:0/273840551 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00b006c6e0 con 0x7f00b00d0a90 2023-11-24T06:11:58.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:58.295+0000 7f00a6ffd640 1 -- 172.21.15.92:0/273840551 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f0098004b80 con 0x7f00b00d0a90 2023-11-24T06:11:58.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:58.296+0000 7f00a6ffd640 1 --2- 172.21.15.92:0/273840551 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f009006ae40 0x7f009006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:58.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:58.296+0000 7f00a6ffd640 1 -- 172.21.15.92:0/273840551 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0098088c90 con 0x7f00b00d0a90 2023-11-24T06:11:58.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:58.296+0000 7f00ae59c640 1 --2- 172.21.15.92:0/273840551 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f009006ae40 0x7f009006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:58.299 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:58.297+0000 7f00ae59c640 1 --2- 172.21.15.92:0/273840551 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f009006ae40 0x7f009006d300 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f00a0009f50 tx=0x7f00a000b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:58.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:58.300+0000 7f00a6ffd640 1 -- 172.21.15.92:0/273840551 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0098053cb0 con 0x7f00b00d0a90 2023-11-24T06:11:58.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:58.493+0000 7f00b491d640 1 -- 172.21.15.92:0/273840551 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f00b00675d0 con 0x7f00b00d0a90 2023-11-24T06:11:58.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:58.494+0000 7f00a6ffd640 1 -- 172.21.15.92:0/273840551 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7f0098018070 con 0x7f00b00d0a90 2023-11-24T06:11:58.499 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:58.496+0000 7f00b491d640 1 -- 172.21.15.92:0/273840551 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f009006ae40 msgr2=0x7f009006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:58.499 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:58.497+0000 7f00b491d640 1 --2- 172.21.15.92:0/273840551 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f009006ae40 0x7f009006d300 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f00a0009f50 tx=0x7f00a000b040 comp rx=0 tx=0).stop 2023-11-24T06:11:58.499 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:58.497+0000 7f00b491d640 1 -- 172.21.15.92:0/273840551 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f00b00d0a90 msgr2=0x7f00b0142e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:58.499 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:58.497+0000 7f00b491d640 1 --2- 172.21.15.92:0/273840551 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f00b00d0a90 0x7f00b0142e90 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f009800c4d0 tx=0x7f009800c990 comp rx=0 tx=0).stop 2023-11-24T06:11:58.499 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:58.497+0000 7f00b491d640 1 -- 172.21.15.92:0/273840551 shutdown_connections 2023-11-24T06:11:58.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:58.497+0000 7f00b491d640 1 --2- 172.21.15.92:0/273840551 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f009006ae40 0x7f009006d300 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:58.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:58.497+0000 7f00b491d640 1 --2- 172.21.15.92:0/273840551 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f00b0144d00 0x7f00b015e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:58.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:58.497+0000 7f00b491d640 1 --2- 172.21.15.92:0/273840551 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f00b00d0a90 0x7f00b0142e90 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:58.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:58.497+0000 7f00b491d640 1 --2- 172.21.15.92:0/273840551 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f00b00d0610 0x7f00b01f31e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:58.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:58.497+0000 7f00b491d640 1 -- 172.21.15.92:0/273840551 >> 172.21.15.92:0/273840551 conn(0x7f00b0066690 msgr2=0x7f00b0143d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:58.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:58.497+0000 7f00b491d640 1 -- 172.21.15.92:0/273840551 shutdown_connections 2023-11-24T06:11:58.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:58.497+0000 7f00b491d640 1 -- 172.21.15.92:0/273840551 wait complete. 2023-11-24T06:11:58.518 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume create vol9070 2023-11-24T06:11:58.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:58.729+0000 7f707a5b0640 1 Processor -- start 2023-11-24T06:11:58.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:58.730+0000 7f707a5b0640 1 -- start start 2023-11-24T06:11:58.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:58.731+0000 7f707a5b0640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f707413df80 0x7f707415bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:58.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:58.731+0000 7f707a5b0640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f70740cf9a0 0x7f70740cd840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:58.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:58.731+0000 7f707a5b0640 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f707406c470 con 0x7f707413e360 2023-11-24T06:11:58.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:58.731+0000 7f707a5b0640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f707406c750 con 0x7f707413df80 2023-11-24T06:11:58.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:58.731+0000 7f707a5b0640 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f707406ca30 con 0x7f70740cf9a0 2023-11-24T06:11:58.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:58.732+0000 7f7073fff640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f707413df80 0x7f707415bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:58.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:58.732+0000 7f7073fff640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f707413df80 0x7f707415bee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58808/0 (socket says 172.21.15.92:58808) 2023-11-24T06:11:58.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:58.732+0000 7f7073fff640 1 -- 172.21.15.92:0/911971768 learned_addr learned my addr 172.21.15.92:0/911971768 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:11:58.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:58.732+0000 7f7073fff640 1 -- 172.21.15.92:0/911971768 >> v2:172.21.15.118:3300/0 conn(0x7f70740cf9a0 msgr2=0x7f70740cd840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:58.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:58.732+0000 7f7073fff640 1 --2- 172.21.15.92:0/911971768 >> v2:172.21.15.118:3300/0 conn(0x7f70740cf9a0 0x7f70740cd840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:58.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:58.732+0000 7f7073fff640 1 -- 172.21.15.92:0/911971768 >> v1:172.21.15.118:6789/0 conn(0x7f707413e360 legacy=0x7f70740cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:58.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:58.732+0000 7f7073fff640 1 -- 172.21.15.92:0/911971768 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f707406cd10 con 0x7f707413df80 2023-11-24T06:11:58.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:58.733+0000 7f7073fff640 1 --2- 172.21.15.92:0/911971768 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f707413df80 0x7f707415bee0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f70740ccb10 tx=0x7f706402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3a986330eaf98cf server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:58.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:58.733+0000 7f70737fe640 1 -- 172.21.15.92:0/911971768 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7064031870 con 0x7f707413df80 2023-11-24T06:11:58.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:58.734+0000 7f70737fe640 1 -- 172.21.15.92:0/911971768 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70640319d0 con 0x7f707413df80 2023-11-24T06:11:58.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:58.734+0000 7f70737fe640 1 -- 172.21.15.92:0/911971768 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7064031d60 con 0x7f707413df80 2023-11-24T06:11:58.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:58.734+0000 7f707a5b0640 1 -- 172.21.15.92:0/911971768 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f707413df80 msgr2=0x7f707415bee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:58.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:58.734+0000 7f707a5b0640 1 --2- 172.21.15.92:0/911971768 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f707413df80 0x7f707415bee0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f70740ccb10 tx=0x7f706402b910 comp rx=0 tx=0).stop 2023-11-24T06:11:58.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:58.734+0000 7f707a5b0640 1 -- 172.21.15.92:0/911971768 shutdown_connections 2023-11-24T06:11:58.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:58.734+0000 7f707a5b0640 1 --2- 172.21.15.92:0/911971768 >> v2:172.21.15.118:3300/0 conn(0x7f70740cf9a0 0x7f70740cd840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:58.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:58.734+0000 7f707a5b0640 1 --2- 172.21.15.92:0/911971768 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f707413df80 0x7f707415bee0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:58.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:58.734+0000 7f707a5b0640 1 -- 172.21.15.92:0/911971768 >> 172.21.15.92:0/911971768 conn(0x7f707406a800 msgr2=0x7f707406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:58.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:58.734+0000 7f707a5b0640 1 -- 172.21.15.92:0/911971768 shutdown_connections 2023-11-24T06:11:58.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:58.734+0000 7f707a5b0640 1 -- 172.21.15.92:0/911971768 wait complete. 2023-11-24T06:11:58.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:58.736+0000 7f707a5b0640 1 Processor -- start 2023-11-24T06:11:58.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:58.736+0000 7f707a5b0640 1 -- start start 2023-11-24T06:11:58.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:58.737+0000 7f707a5b0640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f70740cf9a0 0x7f707413d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:58.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:58.737+0000 7f707a5b0640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f707413df80 0x7f7074152ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:58.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:58.737+0000 7f707a5b0640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f707413e360 0x7f7074153020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:58.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:58.737+0000 7f707a5b0640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70741f3170 con 0x7f707413e360 2023-11-24T06:11:58.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:58.737+0000 7f707a5b0640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7074153610 con 0x7f707413df80 2023-11-24T06:11:58.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:58.737+0000 7f707a5b0640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70741538f0 con 0x7f70740cf9a0 2023-11-24T06:11:58.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:58.738+0000 7f7078b4d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f707413e360 0x7f7074153020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:58.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:58.738+0000 7f7078b4d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f707413e360 0x7f7074153020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58820/0 (socket says 172.21.15.92:58820) 2023-11-24T06:11:58.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:58.738+0000 7f7078b4d640 1 -- 172.21.15.92:0/651067315 learned_addr learned my addr 172.21.15.92:0/651067315 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:11:58.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:58.738+0000 7f7078b4d640 1 -- 172.21.15.92:0/651067315 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f70740cf9a0 msgr2=0x7f707413d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:58.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:58.738+0000 7f7078b4d640 1 --2- 172.21.15.92:0/651067315 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f70740cf9a0 0x7f707413d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:58.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:58.738+0000 7f7078b4d640 1 -- 172.21.15.92:0/651067315 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f707413df80 msgr2=0x7f7074152ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:58.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:58.738+0000 7f7078b4d640 1 --2- 172.21.15.92:0/651067315 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f707413df80 0x7f7074152ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:58.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:58.738+0000 7f7078b4d640 1 -- 172.21.15.92:0/651067315 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70640094d0 con 0x7f707413e360 2023-11-24T06:11:58.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:58.738+0000 7f7078b4d640 1 --2- 172.21.15.92:0/651067315 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f707413e360 0x7f7074153020 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f706c00b9e0 tx=0x7f706c00bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:58.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:58.739+0000 7f70717fa640 1 -- 172.21.15.92:0/651067315 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f706c00cb20 con 0x7f707413e360 2023-11-24T06:11:58.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:58.739+0000 7f70717fa640 1 -- 172.21.15.92:0/651067315 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f706c00cca0 con 0x7f707413e360 2023-11-24T06:11:58.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:58.739+0000 7f707a5b0640 1 -- 172.21.15.92:0/651067315 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7074153bd0 con 0x7f707413e360 2023-11-24T06:11:58.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:58.739+0000 7f70717fa640 1 -- 172.21.15.92:0/651067315 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f706c014680 con 0x7f707413e360 2023-11-24T06:11:58.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:58.739+0000 7f707a5b0640 1 -- 172.21.15.92:0/651067315 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70741e5400 con 0x7f707413e360 2023-11-24T06:11:58.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:58.740+0000 7f707a5b0640 1 -- 172.21.15.92:0/651067315 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70740675d0 con 0x7f707413e360 2023-11-24T06:11:58.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:58.741+0000 7f70717fa640 1 -- 172.21.15.92:0/651067315 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f706c01d3b0 con 0x7f707413e360 2023-11-24T06:11:58.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:58.742+0000 7f70717fa640 1 --2- 172.21.15.92:0/651067315 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f705406ae40 0x7f705406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:58.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:58.742+0000 7f70717fa640 1 -- 172.21.15.92:0/651067315 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f706c089610 con 0x7f707413e360 2023-11-24T06:11:58.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:58.742+0000 7f7073fff640 1 --2- 172.21.15.92:0/651067315 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f705406ae40 0x7f705406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:58.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:58.743+0000 7f7073fff640 1 --2- 172.21.15.92:0/651067315 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f705406ae40 0x7f705406d300 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f7064036000 tx=0x7f7064030000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:58.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:58.746+0000 7f70717fa640 1 -- 172.21.15.92:0/651067315 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f706c054620 con 0x7f707413e360 2023-11-24T06:11:58.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:11:58.961+0000 7f707a5b0640 1 -- 172.21.15.92:0/651067315 --> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] -- mgr_command(tid 0: {"prefix": "fs volume create", "name": "vol9070", "target": ["mon-mgr", ""]}) v1 -- 0x7f707415f9c0 con 0x7f705406ae40 2023-11-24T06:12:01.632 INFO:teuthology.orchestra.run.smithi092.stderr:Volume created successfully (no MDS daemons created) 2023-11-24T06:12:01.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:01.625+0000 7f70717fa640 1 -- 172.21.15.92:0/651067315 <== mgr.4101 v2:172.21.15.92:6800/784027401 1 ==== mgr_command_reply(tid 0: 0 Volume created successfully (no MDS daemons created)) v1 ==== 60+0+0 (secure 0 0 0) 0x7f7064031c40 con 0x7f705406ae40 2023-11-24T06:12:01.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:01.631+0000 7f707a5b0640 1 -- 172.21.15.92:0/651067315 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f705406ae40 msgr2=0x7f705406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:01.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:01.631+0000 7f707a5b0640 1 --2- 172.21.15.92:0/651067315 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f705406ae40 0x7f705406d300 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f7064036000 tx=0x7f7064030000 comp rx=0 tx=0).stop 2023-11-24T06:12:01.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:01.631+0000 7f707a5b0640 1 -- 172.21.15.92:0/651067315 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f707413e360 msgr2=0x7f7074153020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:01.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:01.631+0000 7f707a5b0640 1 --2- 172.21.15.92:0/651067315 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f707413e360 0x7f7074153020 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f706c00b9e0 tx=0x7f706c00bea0 comp rx=0 tx=0).stop 2023-11-24T06:12:01.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:01.632+0000 7f707a5b0640 1 -- 172.21.15.92:0/651067315 shutdown_connections 2023-11-24T06:12:01.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:01.632+0000 7f707a5b0640 1 --2- 172.21.15.92:0/651067315 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f705406ae40 0x7f705406d300 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:01.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:01.632+0000 7f707a5b0640 1 --2- 172.21.15.92:0/651067315 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f707413e360 0x7f7074153020 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:01.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:01.632+0000 7f707a5b0640 1 --2- 172.21.15.92:0/651067315 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f707413df80 0x7f7074152ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:01.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:01.632+0000 7f707a5b0640 1 --2- 172.21.15.92:0/651067315 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f70740cf9a0 0x7f707413d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:01.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:01.632+0000 7f707a5b0640 1 -- 172.21.15.92:0/651067315 >> 172.21.15.92:0/651067315 conn(0x7f707406a800 msgr2=0x7f7074070ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:01.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:01.633+0000 7f707a5b0640 1 -- 172.21.15.92:0/651067315 shutdown_connections 2023-11-24T06:12:01.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:01.633+0000 7f707a5b0640 1 -- 172.21.15.92:0/651067315 wait complete. 2023-11-24T06:12:01.665 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-11-24T06:12:01.851 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:01.849+0000 7f025853c640 1 Processor -- start 2023-11-24T06:12:01.851 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:01.849+0000 7f025853c640 1 -- start start 2023-11-24T06:12:01.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:01.851+0000 7f025853c640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f025013e360 0x7f02500cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:01.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:01.851+0000 7f025853c640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f025015f9c0 0x7f02500cf7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:01.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:01.851+0000 7f025853c640 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f025015c5f0 con 0x7f025013df80 2023-11-24T06:12:01.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:01.851+0000 7f025853c640 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f025006c470 con 0x7f025013e360 2023-11-24T06:12:01.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:01.851+0000 7f025853c640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f025006c750 con 0x7f025015f9c0 2023-11-24T06:12:01.854 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:01.851+0000 7f0256ad9640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f025015f9c0 0x7f02500cf7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:01.854 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:01.851+0000 7f0256ad9640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f025015f9c0 0x7f02500cf7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3301/0 says I am v2:172.21.15.92:38656/0 (socket says 172.21.15.92:38656) 2023-11-24T06:12:01.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:01.851+0000 7f0256ad9640 1 -- 172.21.15.92:0/179144234 learned_addr learned my addr 172.21.15.92:0/179144234 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:12:01.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:01.851+0000 7f0255ad7640 1 --2- 172.21.15.92:0/179144234 >> v2:172.21.15.118:3300/0 conn(0x7f025013e360 0x7f02500cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:01.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:01.852+0000 7f0256ad9640 1 -- 172.21.15.92:0/179144234 >> v2:172.21.15.118:3300/0 conn(0x7f025013e360 msgr2=0x7f02500cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:01.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:01.852+0000 7f0256ad9640 1 --2- 172.21.15.92:0/179144234 >> v2:172.21.15.118:3300/0 conn(0x7f025013e360 0x7f02500cf290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:01.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:01.852+0000 7f0256ad9640 1 -- 172.21.15.92:0/179144234 >> v1:172.21.15.118:6789/0 conn(0x7f025013df80 legacy=0x7f025015bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:01.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:01.852+0000 7f0256ad9640 1 -- 172.21.15.92:0/179144234 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f025006ca30 con 0x7f025015f9c0 2023-11-24T06:12:01.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:01.853+0000 7f0256ad9640 1 --2- 172.21.15.92:0/179144234 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f025015f9c0 0x7f02500cf7d0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f0248009e50 tx=0x7f024802bf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=20869dc0c9a3bfce server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:01.856 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:01.854+0000 7f02552d6640 1 -- 172.21.15.92:0/179144234 <== mon.2 v2:172.21.15.118:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f024802cb00 con 0x7f025015f9c0 2023-11-24T06:12:01.856 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:01.854+0000 7f02552d6640 1 -- 172.21.15.92:0/179144234 <== mon.2 v2:172.21.15.118:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f024802cc60 con 0x7f025015f9c0 2023-11-24T06:12:01.856 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:01.854+0000 7f02552d6640 1 -- 172.21.15.92:0/179144234 <== mon.2 v2:172.21.15.118:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02480343f0 con 0x7f025015f9c0 2023-11-24T06:12:01.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:01.855+0000 7f025853c640 1 -- 172.21.15.92:0/179144234 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f025015f9c0 msgr2=0x7f02500cf7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:01.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:01.855+0000 7f025853c640 1 --2- 172.21.15.92:0/179144234 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f025015f9c0 0x7f02500cf7d0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f0248009e50 tx=0x7f024802bf60 comp rx=0 tx=0).stop 2023-11-24T06:12:01.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:01.855+0000 7f025853c640 1 -- 172.21.15.92:0/179144234 shutdown_connections 2023-11-24T06:12:01.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:01.855+0000 7f025853c640 1 --2- 172.21.15.92:0/179144234 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f025015f9c0 0x7f02500cf7d0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:01.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:01.855+0000 7f025853c640 1 --2- 172.21.15.92:0/179144234 >> v2:172.21.15.118:3300/0 conn(0x7f025013e360 0x7f02500cf290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:01.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:01.855+0000 7f025853c640 1 -- 172.21.15.92:0/179144234 >> 172.21.15.92:0/179144234 conn(0x7f025006a800 msgr2=0x7f025006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:01.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:01.855+0000 7f025853c640 1 -- 172.21.15.92:0/179144234 shutdown_connections 2023-11-24T06:12:01.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:01.855+0000 7f025853c640 1 -- 172.21.15.92:0/179144234 wait complete. 2023-11-24T06:12:01.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:01.857+0000 7f025853c640 1 Processor -- start 2023-11-24T06:12:01.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:01.857+0000 7f025853c640 1 -- start start 2023-11-24T06:12:01.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:01.858+0000 7f025853c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f025013df80 0x7f0250152fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:01.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:01.858+0000 7f025853c640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f025013e360 0x7f02500a95e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:01.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:01.858+0000 7f025853c640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f025015f9c0 0x7f0250159f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:01.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:01.858+0000 7f025853c640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02500722b0 con 0x7f025013df80 2023-11-24T06:12:01.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:01.858+0000 7f025853c640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0250071730 con 0x7f025015f9c0 2023-11-24T06:12:01.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:01.858+0000 7f025853c640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0250071a10 con 0x7f025013e360 2023-11-24T06:12:01.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:01.859+0000 7f02562d8640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f025013df80 0x7f0250152fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:01.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:01.859+0000 7f02562d8640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f025013df80 0x7f0250152fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46814/0 (socket says 172.21.15.92:46814) 2023-11-24T06:12:01.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:01.859+0000 7f02562d8640 1 -- 172.21.15.92:0/251943847 learned_addr learned my addr 172.21.15.92:0/251943847 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:12:01.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:01.859+0000 7f0256ad9640 1 --2- 172.21.15.92:0/251943847 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f025015f9c0 0x7f0250159f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:01.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:01.859+0000 7f0255ad7640 1 --2- 172.21.15.92:0/251943847 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f025013e360 0x7f02500a95e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:01.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:01.859+0000 7f02562d8640 1 -- 172.21.15.92:0/251943847 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f025013e360 msgr2=0x7f02500a95e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:01.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:01.859+0000 7f02562d8640 1 --2- 172.21.15.92:0/251943847 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f025013e360 0x7f02500a95e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:01.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:01.859+0000 7f02562d8640 1 -- 172.21.15.92:0/251943847 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f025015f9c0 msgr2=0x7f0250159f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:01.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:01.859+0000 7f02562d8640 1 --2- 172.21.15.92:0/251943847 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f025015f9c0 0x7f0250159f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:01.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:01.859+0000 7f02562d8640 1 -- 172.21.15.92:0/251943847 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0248009b30 con 0x7f025013df80 2023-11-24T06:12:01.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:01.860+0000 7f02562d8640 1 --2- 172.21.15.92:0/251943847 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f025013df80 0x7f0250152fb0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f024c00c390 tx=0x7f024c00c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:01.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:01.860+0000 7f0246ffd640 1 -- 172.21.15.92:0/251943847 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f024c0052d0 con 0x7f025013df80 2023-11-24T06:12:01.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:01.860+0000 7f0246ffd640 1 -- 172.21.15.92:0/251943847 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f024c005430 con 0x7f025013df80 2023-11-24T06:12:01.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:01.860+0000 7f0246ffd640 1 -- 172.21.15.92:0/251943847 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f024c003830 con 0x7f025013df80 2023-11-24T06:12:01.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:01.860+0000 7f025853c640 1 -- 172.21.15.92:0/251943847 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0250071cf0 con 0x7f025013df80 2023-11-24T06:12:01.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:01.860+0000 7f025853c640 1 -- 172.21.15.92:0/251943847 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02501534f0 con 0x7f025013df80 2023-11-24T06:12:01.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:01.861+0000 7f025853c640 1 -- 172.21.15.92:0/251943847 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02500675d0 con 0x7f025013df80 2023-11-24T06:12:01.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:01.862+0000 7f0246ffd640 1 -- 172.21.15.92:0/251943847 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f024c003990 con 0x7f025013df80 2023-11-24T06:12:01.866 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:01.863+0000 7f0246ffd640 1 --2- 172.21.15.92:0/251943847 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f023406ae40 0x7f023406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:01.866 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:01.864+0000 7f0246ffd640 1 -- 172.21.15.92:0/251943847 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6543+0+0 (secure 0 0 0) 0x7f024c088ef0 con 0x7f025013df80 2023-11-24T06:12:01.866 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:01.864+0000 7f0255ad7640 1 --2- 172.21.15.92:0/251943847 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f023406ae40 0x7f023406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:01.866 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:01.864+0000 7f0255ad7640 1 --2- 172.21.15.92:0/251943847 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f023406ae40 0x7f023406d300 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f02501f6770 tx=0x7f0240008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:01.871 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:01.869+0000 7f0246ffd640 1 -- 172.21.15.92:0/251943847 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f024c053c60 con 0x7f025013df80 2023-11-24T06:12:02.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:02.074+0000 7f025853c640 1 -- 172.21.15.92:0/251943847 --> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f025006cff0 con 0x7f023406ae40 2023-11-24T06:12:02.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:02.077+0000 7f0246ffd640 1 -- 172.21.15.92:0/251943847 <== mgr.4101 v2:172.21.15.92:6800/784027401 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+80 (secure 0 0 0) 0x7f0240009800 con 0x7f023406ae40 2023-11-24T06:12:02.079 INFO:teuthology.orchestra.run.smithi092.stdout:[ 2023-11-24T06:12:02.079 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-24T06:12:02.079 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "cephfs" 2023-11-24T06:12:02.079 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-24T06:12:02.080 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-24T06:12:02.080 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "vol9070" 2023-11-24T06:12:02.080 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-24T06:12:02.080 INFO:teuthology.orchestra.run.smithi092.stdout:] 2023-11-24T06:12:02.081 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:02.079+0000 7f025853c640 1 -- 172.21.15.92:0/251943847 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f023406ae40 msgr2=0x7f023406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:02.082 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:02.079+0000 7f025853c640 1 --2- 172.21.15.92:0/251943847 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f023406ae40 0x7f023406d300 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f02501f6770 tx=0x7f0240008040 comp rx=0 tx=0).stop 2023-11-24T06:12:02.082 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:02.079+0000 7f025853c640 1 -- 172.21.15.92:0/251943847 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f025013df80 msgr2=0x7f0250152fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:02.082 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:02.079+0000 7f025853c640 1 --2- 172.21.15.92:0/251943847 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f025013df80 0x7f0250152fb0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f024c00c390 tx=0x7f024c00c850 comp rx=0 tx=0).stop 2023-11-24T06:12:02.082 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:02.079+0000 7f025853c640 1 -- 172.21.15.92:0/251943847 shutdown_connections 2023-11-24T06:12:02.082 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:02.079+0000 7f025853c640 1 --2- 172.21.15.92:0/251943847 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f023406ae40 0x7f023406d300 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:02.082 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:02.079+0000 7f025853c640 1 --2- 172.21.15.92:0/251943847 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f025015f9c0 0x7f0250159f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:02.082 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:02.079+0000 7f025853c640 1 --2- 172.21.15.92:0/251943847 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f025013e360 0x7f02500a95e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:02.082 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:02.079+0000 7f025853c640 1 --2- 172.21.15.92:0/251943847 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f025013df80 0x7f0250152fb0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:02.082 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:02.079+0000 7f025853c640 1 -- 172.21.15.92:0/251943847 >> 172.21.15.92:0/251943847 conn(0x7f025006a800 msgr2=0x7f02500b0010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:02.082 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:02.079+0000 7f025853c640 1 -- 172.21.15.92:0/251943847 shutdown_connections 2023-11-24T06:12:02.082 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:02.079+0000 7f025853c640 1 -- 172.21.15.92:0/251943847 wait complete. 2023-11-24T06:12:02.099 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool ls detail --format=json 2023-11-24T06:12:02.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:02.308+0000 7f0e78ffc640 1 Processor -- start 2023-11-24T06:12:02.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:02.308+0000 7f0e78ffc640 1 -- start start 2023-11-24T06:12:02.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:02.310+0000 7f0e78ffc640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0e7413e360 0x7f0e740cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:02.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:02.310+0000 7f0e78ffc640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e7415c5f0 con 0x7f0e7413df80 2023-11-24T06:12:02.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:02.310+0000 7f0e78ffc640 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e7406c470 con 0x7f0e7415f9c0 2023-11-24T06:12:02.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:02.310+0000 7f0e78ffc640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e7406c750 con 0x7f0e7413e360 2023-11-24T06:12:02.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:02.311+0000 7f0e7259c640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f0e7413df80 0x7f0e7415bee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:46790/0 (socket says 172.21.15.92:46790) 2023-11-24T06:12:02.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:02.311+0000 7f0e7259c640 1 -- 172.21.15.92:0/900792213 learned_addr learned my addr 172.21.15.92:0/900792213 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-24T06:12:02.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:02.311+0000 7f0e69d9b640 1 --2- 172.21.15.92:0/900792213 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0e7413e360 0x7f0e740cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:02.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:02.311+0000 7f0e71d9b640 1 -- 172.21.15.92:0/900792213 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1930567940 0 0) 0x7f0e7415c5f0 con 0x7f0e7413df80 2023-11-24T06:12:02.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:02.311+0000 7f0e71d9b640 1 -- 172.21.15.92:0/900792213 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0e60003520 con 0x7f0e7413df80 2023-11-24T06:12:02.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:02.312+0000 7f0e69d9b640 1 -- 172.21.15.92:0/900792213 >> v1:172.21.15.118:6789/0 conn(0x7f0e7415f9c0 legacy=0x7f0e740cd840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:02.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:02.312+0000 7f0e69d9b640 1 -- 172.21.15.92:0/900792213 >> v1:172.21.15.92:6789/0 conn(0x7f0e7413df80 legacy=0x7f0e7415bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:02.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:02.312+0000 7f0e69d9b640 1 -- 172.21.15.92:0/900792213 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e7406ca30 con 0x7f0e7413e360 2023-11-24T06:12:02.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:02.313+0000 7f0e69d9b640 1 --2- 172.21.15.92:0/900792213 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0e7413e360 0x7f0e740cf290 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f0e5c0097d0 tx=0x7f0e5c02b940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5b65fd569c202480 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:02.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:02.313+0000 7f0e71d9b640 1 -- 172.21.15.92:0/900792213 <== mon.2 v2:172.21.15.118:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e5c030800 con 0x7f0e7413e360 2023-11-24T06:12:02.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:02.313+0000 7f0e71d9b640 1 -- 172.21.15.92:0/900792213 <== mon.2 v2:172.21.15.118:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e5c030960 con 0x7f0e7413e360 2023-11-24T06:12:02.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:02.313+0000 7f0e71d9b640 1 -- 172.21.15.92:0/900792213 <== mon.2 v2:172.21.15.118:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e5c030cf0 con 0x7f0e7413e360 2023-11-24T06:12:02.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:02.314+0000 7f0e78ffc640 1 -- 172.21.15.92:0/900792213 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0e7413e360 msgr2=0x7f0e740cf290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:02.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:02.314+0000 7f0e78ffc640 1 --2- 172.21.15.92:0/900792213 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0e7413e360 0x7f0e740cf290 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f0e5c0097d0 tx=0x7f0e5c02b940 comp rx=0 tx=0).stop 2023-11-24T06:12:02.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:02.314+0000 7f0e78ffc640 1 -- 172.21.15.92:0/900792213 shutdown_connections 2023-11-24T06:12:02.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:02.314+0000 7f0e78ffc640 1 --2- 172.21.15.92:0/900792213 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0e7413e360 0x7f0e740cf290 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:02.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:02.314+0000 7f0e78ffc640 1 -- 172.21.15.92:0/900792213 >> 172.21.15.92:0/900792213 conn(0x7f0e7406a800 msgr2=0x7f0e7406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:02.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:02.314+0000 7f0e78ffc640 1 -- 172.21.15.92:0/900792213 shutdown_connections 2023-11-24T06:12:02.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:02.314+0000 7f0e78ffc640 1 -- 172.21.15.92:0/900792213 wait complete. 2023-11-24T06:12:02.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:02.316+0000 7f0e78ffc640 1 Processor -- start 2023-11-24T06:12:02.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:02.316+0000 7f0e78ffc640 1 -- start start 2023-11-24T06:12:02.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:02.317+0000 7f0e78ffc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0e7413df80 0x7f0e74150fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:02.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:02.317+0000 7f0e78ffc640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0e7413e360 0x7f0e740702c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:02.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:02.317+0000 7f0e78ffc640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0e7415f9c0 0x7f0e7413c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:02.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:02.317+0000 7f0e78ffc640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e7413cf20 con 0x7f0e7413df80 2023-11-24T06:12:02.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:02.317+0000 7f0e78ffc640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e7414af40 con 0x7f0e7413e360 2023-11-24T06:12:02.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:02.317+0000 7f0e78ffc640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e7414d3a0 con 0x7f0e7415f9c0 2023-11-24T06:12:02.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:02.317+0000 7f0e7259c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0e7413df80 0x7f0e74150fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:02.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:02.317+0000 7f0e7259c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0e7413df80 0x7f0e74150fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46830/0 (socket says 172.21.15.92:46830) 2023-11-24T06:12:02.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:02.317+0000 7f0e7259c640 1 -- 172.21.15.92:0/3166962717 learned_addr learned my addr 172.21.15.92:0/3166962717 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:12:02.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:02.318+0000 7f0e69d9b640 1 --2- 172.21.15.92:0/3166962717 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0e7413e360 0x7f0e740702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:02.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:02.318+0000 7f0e72d9d640 1 --2- 172.21.15.92:0/3166962717 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0e7415f9c0 0x7f0e7413c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:02.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:02.318+0000 7f0e7259c640 1 -- 172.21.15.92:0/3166962717 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0e7415f9c0 msgr2=0x7f0e7413c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:02.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:02.318+0000 7f0e7259c640 1 --2- 172.21.15.92:0/3166962717 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0e7415f9c0 0x7f0e7413c9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:02.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:02.318+0000 7f0e7259c640 1 -- 172.21.15.92:0/3166962717 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0e7413e360 msgr2=0x7f0e740702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:02.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:02.318+0000 7f0e7259c640 1 --2- 172.21.15.92:0/3166962717 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0e7413e360 0x7f0e740702c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:02.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:02.318+0000 7f0e7259c640 1 -- 172.21.15.92:0/3166962717 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e5c009400 con 0x7f0e7413df80 2023-11-24T06:12:02.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:02.318+0000 7f0e69d9b640 1 --2- 172.21.15.92:0/3166962717 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0e7413e360 0x7f0e740702c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:12:02.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:02.318+0000 7f0e72d9d640 1 --2- 172.21.15.92:0/3166962717 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0e7415f9c0 0x7f0e7413c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:12:02.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:02.318+0000 7f0e7259c640 1 --2- 172.21.15.92:0/3166962717 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0e7413df80 0x7f0e74150fa0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f0e6400c410 tx=0x7f0e6400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:02.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:02.318+0000 7f0e6b7fe640 1 -- 172.21.15.92:0/3166962717 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e64005ce0 con 0x7f0e7413df80 2023-11-24T06:12:02.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:02.318+0000 7f0e6b7fe640 1 -- 172.21.15.92:0/3166962717 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e64005e40 con 0x7f0e7413df80 2023-11-24T06:12:02.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:02.318+0000 7f0e78ffc640 1 -- 172.21.15.92:0/3166962717 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e7414d680 con 0x7f0e7413df80 2023-11-24T06:12:02.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:02.318+0000 7f0e6b7fe640 1 -- 172.21.15.92:0/3166962717 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e640072f0 con 0x7f0e7413df80 2023-11-24T06:12:02.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:02.318+0000 7f0e78ffc640 1 -- 172.21.15.92:0/3166962717 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e7414dc40 con 0x7f0e7413df80 2023-11-24T06:12:02.323 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:02.320+0000 7f0e78ffc640 1 -- 172.21.15.92:0/3166962717 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e7414ddd0 con 0x7f0e7413df80 2023-11-24T06:12:02.323 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:02.321+0000 7f0e6b7fe640 1 -- 172.21.15.92:0/3166962717 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f0e64007450 con 0x7f0e7413df80 2023-11-24T06:12:02.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:02.321+0000 7f0e6b7fe640 1 --2- 172.21.15.92:0/3166962717 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f0e5406ae70 0x7f0e5406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:02.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:02.321+0000 7f0e6b7fe640 1 -- 172.21.15.92:0/3166962717 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6543+0+0 (secure 0 0 0) 0x7f0e64088f90 con 0x7f0e7413df80 2023-11-24T06:12:02.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:02.322+0000 7f0e69d9b640 1 --2- 172.21.15.92:0/3166962717 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f0e5406ae70 0x7f0e5406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:02.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:02.322+0000 7f0e69d9b640 1 --2- 172.21.15.92:0/3166962717 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f0e5406ae70 0x7f0e5406d330 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f0e5c036000 tx=0x7f0e5c03b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:02.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:02.326+0000 7f0e6b7fe640 1 -- 172.21.15.92:0/3166962717 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0e640bbe90 con 0x7f0e7413df80 2023-11-24T06:12:02.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:02.514+0000 7f0e78ffc640 1 -- 172.21.15.92:0/3166962717 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool ls", "detail": "detail", "format": "json"} v 0) v1 -- 0x7f0e740675d0 con 0x7f0e7413df80 2023-11-24T06:12:02.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:02.516+0000 7f0e6b7fe640 1 -- 172.21.15.92:0/3166962717 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool ls", "detail": "detail", "format": "json"}]=0 v36) v1 ==== 97+0+9133 (secure 0 0 0) 0x7f0e640170a0 con 0x7f0e7413df80 2023-11-24T06:12:02.518 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-24T06:12:02.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:02.518+0000 7f0e78ffc640 1 -- 172.21.15.92:0/3166962717 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f0e5406ae70 msgr2=0x7f0e5406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:02.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:02.518+0000 7f0e78ffc640 1 --2- 172.21.15.92:0/3166962717 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f0e5406ae70 0x7f0e5406d330 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f0e5c036000 tx=0x7f0e5c03b040 comp rx=0 tx=0).stop 2023-11-24T06:12:02.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:02.519+0000 7f0e78ffc640 1 -- 172.21.15.92:0/3166962717 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0e7413df80 msgr2=0x7f0e74150fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:02.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:02.519+0000 7f0e78ffc640 1 --2- 172.21.15.92:0/3166962717 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0e7413df80 0x7f0e74150fa0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f0e6400c410 tx=0x7f0e6400c8d0 comp rx=0 tx=0).stop 2023-11-24T06:12:02.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:02.519+0000 7f0e78ffc640 1 -- 172.21.15.92:0/3166962717 shutdown_connections 2023-11-24T06:12:02.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:02.519+0000 7f0e78ffc640 1 --2- 172.21.15.92:0/3166962717 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f0e5406ae70 0x7f0e5406d330 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:02.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:02.519+0000 7f0e78ffc640 1 --2- 172.21.15.92:0/3166962717 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0e7415f9c0 0x7f0e7413c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:02.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:02.519+0000 7f0e78ffc640 1 --2- 172.21.15.92:0/3166962717 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0e7413e360 0x7f0e740702c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:02.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:02.519+0000 7f0e78ffc640 1 --2- 172.21.15.92:0/3166962717 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0e7413df80 0x7f0e74150fa0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:02.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:02.519+0000 7f0e78ffc640 1 -- 172.21.15.92:0/3166962717 >> 172.21.15.92:0/3166962717 conn(0x7f0e7406a800 msgr2=0x7f0e74153960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:02.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:02.519+0000 7f0e78ffc640 1 -- 172.21.15.92:0/3166962717 shutdown_connections 2023-11-24T06:12:02.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:02.519+0000 7f0e78ffc640 1 -- 172.21.15.92:0/3166962717 wait complete. 2023-11-24T06:12:02.538 INFO:teuthology.orchestra.run.smithi092.stdout:[{"pool_id":1,"pool_name":".mgr","create_time":"2023-11-24T06:10:17.733098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool_id":4,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:11:42.241681+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool_id":5,"pool_name":"cephfs_data","create_time":"2023-11-24T06:11:43.522720+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool_id":6,"pool_name":"cephfs.vol9070.meta","create_time":"2023-11-24T06:11:59.018248+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"vol9070"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool_id":7,"pool_name":"cephfs.vol9070.data","create_time":"2023-11-24T06:11:59.662550+0000","flags":163841,"flags_names":"hashpspool,creating,bulk","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"vol9070"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}] 2023-11-24T06:12:02.539 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs get vol9070 --format=json 2023-11-24T06:12:02.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:02.747+0000 7f264f541640 1 Processor -- start 2023-11-24T06:12:02.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:02.747+0000 7f264f541640 1 -- start start 2023-11-24T06:12:02.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:02.748+0000 7f264f541640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f264813e360 0x7f26480cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:02.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:02.748+0000 7f264f541640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f264815f9c0 0x7f26480cf7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:02.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:02.749+0000 7f264f541640 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f264815c5f0 con 0x7f264813df80 2023-11-24T06:12:02.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:02.749+0000 7f264f541640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f264806c470 con 0x7f264815f9c0 2023-11-24T06:12:02.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:02.749+0000 7f264f541640 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f264806c750 con 0x7f264813e360 2023-11-24T06:12:02.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:02.749+0000 7f264dade640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f264815f9c0 0x7f26480cf7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:02.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:02.749+0000 7f264dade640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f264815f9c0 0x7f26480cf7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46834/0 (socket says 172.21.15.92:46834) 2023-11-24T06:12:02.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:02.749+0000 7f264dade640 1 -- 172.21.15.92:0/3896286214 learned_addr learned my addr 172.21.15.92:0/3896286214 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:12:02.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:02.749+0000 7f264dade640 1 -- 172.21.15.92:0/3896286214 >> v2:172.21.15.118:3300/0 conn(0x7f264813e360 msgr2=0x7f26480cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:02.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:02.749+0000 7f264dade640 1 --2- 172.21.15.92:0/3896286214 >> v2:172.21.15.118:3300/0 conn(0x7f264813e360 0x7f26480cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:02.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:02.749+0000 7f264dade640 1 -- 172.21.15.92:0/3896286214 >> v1:172.21.15.118:6789/0 conn(0x7f264813df80 legacy=0x7f264815bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:02.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:02.749+0000 7f264dade640 1 -- 172.21.15.92:0/3896286214 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f264806ca30 con 0x7f264815f9c0 2023-11-24T06:12:02.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:02.750+0000 7f264dade640 1 --2- 172.21.15.92:0/3896286214 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f264815f9c0 0x7f26480cf7d0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f26480ccae0 tx=0x7f264002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7fcfce2c48c5e99 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:02.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:02.751+0000 7f263ffff640 1 -- 172.21.15.92:0/3896286214 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f264002ca70 con 0x7f264815f9c0 2023-11-24T06:12:02.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:02.751+0000 7f263ffff640 1 -- 172.21.15.92:0/3896286214 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f264002cbd0 con 0x7f264815f9c0 2023-11-24T06:12:02.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:02.751+0000 7f263ffff640 1 -- 172.21.15.92:0/3896286214 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26400343f0 con 0x7f264815f9c0 2023-11-24T06:12:02.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:02.751+0000 7f264f541640 1 -- 172.21.15.92:0/3896286214 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f264815f9c0 msgr2=0x7f26480cf7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:02.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:02.751+0000 7f264f541640 1 --2- 172.21.15.92:0/3896286214 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f264815f9c0 0x7f26480cf7d0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f26480ccae0 tx=0x7f264002bed0 comp rx=0 tx=0).stop 2023-11-24T06:12:02.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:02.751+0000 7f264f541640 1 -- 172.21.15.92:0/3896286214 shutdown_connections 2023-11-24T06:12:02.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:02.751+0000 7f264f541640 1 --2- 172.21.15.92:0/3896286214 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f264815f9c0 0x7f26480cf7d0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:02.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:02.751+0000 7f264f541640 1 --2- 172.21.15.92:0/3896286214 >> v2:172.21.15.118:3300/0 conn(0x7f264813e360 0x7f26480cf290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:02.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:02.751+0000 7f264f541640 1 -- 172.21.15.92:0/3896286214 >> 172.21.15.92:0/3896286214 conn(0x7f264806a800 msgr2=0x7f264806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:02.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:02.751+0000 7f264f541640 1 -- 172.21.15.92:0/3896286214 shutdown_connections 2023-11-24T06:12:02.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:02.751+0000 7f264f541640 1 -- 172.21.15.92:0/3896286214 wait complete. 2023-11-24T06:12:02.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:02.753+0000 7f264f541640 1 Processor -- start 2023-11-24T06:12:02.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:02.753+0000 7f264f541640 1 -- start start 2023-11-24T06:12:02.756 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:02.754+0000 7f264f541640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f264813df80 0x7f264813ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:02.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:02.754+0000 7f264f541640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f264813e360 0x7f264814a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:02.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:02.754+0000 7f264f541640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f264815f9c0 0x7f26480b3290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:02.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:02.754+0000 7f264f541640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2648070770 con 0x7f264815f9c0 2023-11-24T06:12:02.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:02.754+0000 7f264f541640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f264814bee0 con 0x7f264813df80 2023-11-24T06:12:02.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:02.754+0000 7f264f541640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f264813d350 con 0x7f264813e360 2023-11-24T06:12:02.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:02.755+0000 7f264dade640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f264815f9c0 0x7f26480b3290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:02.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:02.755+0000 7f264dade640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f264815f9c0 0x7f26480b3290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46842/0 (socket says 172.21.15.92:46842) 2023-11-24T06:12:02.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:02.755+0000 7f264dade640 1 -- 172.21.15.92:0/3874702444 learned_addr learned my addr 172.21.15.92:0/3874702444 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:12:02.758 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:02.755+0000 7f264dade640 1 -- 172.21.15.92:0/3874702444 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f264813e360 msgr2=0x7f264814a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:02.758 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:02.755+0000 7f264dade640 1 --2- 172.21.15.92:0/3874702444 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f264813e360 0x7f264814a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:02.758 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:02.755+0000 7f264dade640 1 -- 172.21.15.92:0/3874702444 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f264813df80 msgr2=0x7f264813ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:02.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:02.755+0000 7f264dade640 1 --2- 172.21.15.92:0/3874702444 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f264813df80 0x7f264813ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:02.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:02.755+0000 7f264dade640 1 -- 172.21.15.92:0/3874702444 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2640009b30 con 0x7f264815f9c0 2023-11-24T06:12:02.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:02.755+0000 7f264dade640 1 --2- 172.21.15.92:0/3874702444 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f264815f9c0 0x7f26480b3290 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f2640036040 tx=0x7f2640004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:02.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:02.755+0000 7f263dffb640 1 -- 172.21.15.92:0/3874702444 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f264003e450 con 0x7f264815f9c0 2023-11-24T06:12:02.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:02.756+0000 7f263dffb640 1 -- 172.21.15.92:0/3874702444 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f264003e5b0 con 0x7f264815f9c0 2023-11-24T06:12:02.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:02.756+0000 7f264f541640 1 -- 172.21.15.92:0/3874702444 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f264814b640 con 0x7f264815f9c0 2023-11-24T06:12:02.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:02.756+0000 7f263dffb640 1 -- 172.21.15.92:0/3874702444 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f264003e960 con 0x7f264815f9c0 2023-11-24T06:12:02.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:02.756+0000 7f264f541640 1 -- 172.21.15.92:0/3874702444 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f264814bc00 con 0x7f264815f9c0 2023-11-24T06:12:02.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:02.757+0000 7f264f541640 1 -- 172.21.15.92:0/3874702444 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2648070540 con 0x7f264815f9c0 2023-11-24T06:12:02.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:02.758+0000 7f263dffb640 1 -- 172.21.15.92:0/3874702444 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f264003eac0 con 0x7f264815f9c0 2023-11-24T06:12:02.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:02.759+0000 7f263dffb640 1 --2- 172.21.15.92:0/3874702444 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f262c06ae40 0x7f262c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:02.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:02.759+0000 7f263dffb640 1 -- 172.21.15.92:0/3874702444 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6543+0+0 (secure 0 0 0) 0x7f26400ac8f0 con 0x7f264815f9c0 2023-11-24T06:12:02.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:02.759+0000 7f264d2dd640 1 --2- 172.21.15.92:0/3874702444 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f262c06ae40 0x7f262c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:02.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:02.760+0000 7f264d2dd640 1 --2- 172.21.15.92:0/3874702444 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f262c06ae40 0x7f262c06d300 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f2644009650 tx=0x7f2644006ba0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:02.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:02.764+0000 7f263dffb640 1 -- 172.21.15.92:0/3874702444 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2648070540 con 0x7f264815f9c0 2023-11-24T06:12:02.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:02.977+0000 7f264f541640 1 -- 172.21.15.92:0/3874702444 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs get", "fs_name": "vol9070", "format": "json"} v 0) v1 -- 0x7f2648070540 con 0x7f264815f9c0 2023-11-24T06:12:02.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:02.978+0000 7f263dffb640 1 -- 172.21.15.92:0/3874702444 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs get", "fs_name": "vol9070", "format": "json"}]=0 v21) v1 ==== 94+0+2008 (secure 0 0 0) 0x7f2648070540 con 0x7f264815f9c0 2023-11-24T06:12:02.980 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-24T06:12:02.980 INFO:teuthology.orchestra.run.smithi092.stdout:{"mdsmap":{"epoch":20,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T06:12:00.631858+0000","modified":"2023-11-24T06:12:01.654139+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4741},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4741":{"gid":4741,"name":"a","rank":0,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.92:6836/3889219158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":3889219158},{"type":"v1","addr":"172.21.15.92:6836","nonce":3889219158}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"vol9070","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":3} 2023-11-24T06:12:02.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:02.981+0000 7f264f541640 1 -- 172.21.15.92:0/3874702444 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f262c06ae40 msgr2=0x7f262c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:02.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:02.981+0000 7f264f541640 1 --2- 172.21.15.92:0/3874702444 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f262c06ae40 0x7f262c06d300 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f2644009650 tx=0x7f2644006ba0 comp rx=0 tx=0).stop 2023-11-24T06:12:02.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:02.981+0000 7f264f541640 1 -- 172.21.15.92:0/3874702444 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f264815f9c0 msgr2=0x7f26480b3290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:02.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:02.981+0000 7f264f541640 1 --2- 172.21.15.92:0/3874702444 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f264815f9c0 0x7f26480b3290 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f2640036040 tx=0x7f2640004a40 comp rx=0 tx=0).stop 2023-11-24T06:12:02.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:02.981+0000 7f264f541640 1 -- 172.21.15.92:0/3874702444 shutdown_connections 2023-11-24T06:12:02.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:02.981+0000 7f264f541640 1 --2- 172.21.15.92:0/3874702444 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f262c06ae40 0x7f262c06d300 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:02.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:02.981+0000 7f264f541640 1 --2- 172.21.15.92:0/3874702444 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f264815f9c0 0x7f26480b3290 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:02.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:02.981+0000 7f264f541640 1 --2- 172.21.15.92:0/3874702444 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f264813e360 0x7f264814a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:02.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:02.981+0000 7f264f541640 1 --2- 172.21.15.92:0/3874702444 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f264813df80 0x7f264813ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:02.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:02.981+0000 7f264f541640 1 -- 172.21.15.92:0/3874702444 >> 172.21.15.92:0/3874702444 conn(0x7f264806a800 msgr2=0x7f264815dd70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:02.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:02.981+0000 7f264f541640 1 -- 172.21.15.92:0/3874702444 shutdown_connections 2023-11-24T06:12:02.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:02.981+0000 7f264f541640 1 -- 172.21.15.92:0/3874702444 wait complete. 2023-11-24T06:12:02.998 INFO:tasks.ceph.mds.a.smithi092.stderr:2023-11-24T06:12:02.996+0000 7ff2fe697640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-24T06:12:02.999 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume rm vol9070 --yes-i-really-mean-it 2023-11-24T06:12:03.202 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:03.200+0000 7fce5d54c640 1 Processor -- start 2023-11-24T06:12:03.202 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:03.200+0000 7fce5d54c640 1 -- start start 2023-11-24T06:12:03.204 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:03.202+0000 7fce5d54c640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fce5813df80 0x7fce5815bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:03.204 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:03.202+0000 7fce5d54c640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce5806c470 con 0x7fce580cf9a0 2023-11-24T06:12:03.204 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:03.202+0000 7fce5d54c640 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce5806c750 con 0x7fce5813e360 2023-11-24T06:12:03.204 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:03.202+0000 7fce5d54c640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce5806ca30 con 0x7fce5813df80 2023-11-24T06:12:03.206 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:03.202+0000 7fce577fe640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fce580cf9a0 0x7fce580cd840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:46794/0 (socket says 172.21.15.92:46794) 2023-11-24T06:12:03.206 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:03.202+0000 7fce577fe640 1 -- 172.21.15.92:0/2518438561 learned_addr learned my addr 172.21.15.92:0/2518438561 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-24T06:12:03.206 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:03.203+0000 7fce56ffd640 1 --2- 172.21.15.92:0/2518438561 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fce5813df80 0x7fce5815bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:03.206 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:03.203+0000 7fce567fc640 1 -- 172.21.15.92:0/2518438561 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 635019545 0 0) 0x7fce5806c470 con 0x7fce580cf9a0 2023-11-24T06:12:03.206 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:03.203+0000 7fce567fc640 1 -- 172.21.15.92:0/2518438561 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fce44003520 con 0x7fce580cf9a0 2023-11-24T06:12:03.206 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:03.203+0000 7fce567fc640 1 -- 172.21.15.92:0/2518438561 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1027872047 0 0) 0x7fce44003520 con 0x7fce580cf9a0 2023-11-24T06:12:03.206 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:03.203+0000 7fce567fc640 1 -- 172.21.15.92:0/2518438561 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fce5813df80 msgr2=0x7fce5815bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:03.206 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:03.203+0000 7fce567fc640 1 --2- 172.21.15.92:0/2518438561 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fce5813df80 0x7fce5815bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:03.206 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:03.203+0000 7fce567fc640 1 -- 172.21.15.92:0/2518438561 >> v1:172.21.15.118:6789/0 conn(0x7fce5813e360 legacy=0x7fce580cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:03.206 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:03.203+0000 7fce567fc640 1 -- 172.21.15.92:0/2518438561 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce5806cd10 con 0x7fce580cf9a0 2023-11-24T06:12:03.207 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:03.203+0000 7fce567fc640 1 -- 172.21.15.92:0/2518438561 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2607661350 0 0) 0x7fce500019f0 con 0x7fce580cf9a0 2023-11-24T06:12:03.207 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:03.203+0000 7fce567fc640 1 -- 172.21.15.92:0/2518438561 >> v1:172.21.15.92:6789/0 conn(0x7fce580cf9a0 legacy=0x7fce580cd840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:03.207 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:03.203+0000 7fce567fc640 1 --2- 172.21.15.92:0/2518438561 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fce440039a0 0x7fce44003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:03.207 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:03.203+0000 7fce567fc640 1 --2- 172.21.15.92:0/2518438561 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fce5813df80 0x7fce440047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:03.207 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:03.203+0000 7fce567fc640 1 --2- 172.21.15.92:0/2518438561 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fce44004d00 0x7fce44005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:03.207 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:03.203+0000 7fce567fc640 1 -- 172.21.15.92:0/2518438561 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce5806ca30 con 0x7fce44004d00 2023-11-24T06:12:03.207 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:03.203+0000 7fce567fc640 1 -- 172.21.15.92:0/2518438561 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce4400ba30 con 0x7fce440039a0 2023-11-24T06:12:03.207 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:03.203+0000 7fce567fc640 1 -- 172.21.15.92:0/2518438561 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce4400bbf0 con 0x7fce5813df80 2023-11-24T06:12:03.207 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:03.204+0000 7fce577fe640 1 --2- 172.21.15.92:0/2518438561 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fce44004d00 0x7fce44005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:03.209 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:03.204+0000 7fce4ffff640 1 --2- 172.21.15.92:0/2518438561 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fce5813df80 0x7fce440047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:03.209 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:03.204+0000 7fce56ffd640 1 --2- 172.21.15.92:0/2518438561 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fce440039a0 0x7fce44003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:03.209 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:03.204+0000 7fce577fe640 1 -- 172.21.15.92:0/2518438561 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fce5813df80 msgr2=0x7fce440047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:03.209 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:03.204+0000 7fce577fe640 1 --2- 172.21.15.92:0/2518438561 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fce5813df80 0x7fce440047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:03.209 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:03.204+0000 7fce577fe640 1 -- 172.21.15.92:0/2518438561 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fce440039a0 msgr2=0x7fce44003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:03.209 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:03.204+0000 7fce577fe640 1 --2- 172.21.15.92:0/2518438561 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fce440039a0 0x7fce44003dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:03.209 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:03.204+0000 7fce577fe640 1 -- 172.21.15.92:0/2518438561 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce44003520 con 0x7fce44004d00 2023-11-24T06:12:03.209 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:03.205+0000 7fce577fe640 1 --2- 172.21.15.92:0/2518438561 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fce44004d00 0x7fce44005180 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fce5000a800 tx=0x7fce440057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=217e7907e67e7852 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:03.209 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:03.205+0000 7fce567fc640 1 -- 172.21.15.92:0/2518438561 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce500072d0 con 0x7fce44004d00 2023-11-24T06:12:03.210 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:03.205+0000 7fce567fc640 1 -- 172.21.15.92:0/2518438561 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce5002f070 con 0x7fce44004d00 2023-11-24T06:12:03.210 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:03.205+0000 7fce567fc640 1 -- 172.21.15.92:0/2518438561 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce500076b0 con 0x7fce44004d00 2023-11-24T06:12:03.210 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:03.206+0000 7fce5d54c640 1 -- 172.21.15.92:0/2518438561 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fce44004d00 msgr2=0x7fce44005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:03.210 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:03.206+0000 7fce5d54c640 1 --2- 172.21.15.92:0/2518438561 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fce44004d00 0x7fce44005180 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fce5000a800 tx=0x7fce440057f0 comp rx=0 tx=0).stop 2023-11-24T06:12:03.210 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:03.206+0000 7fce577fe640 1 -- 172.21.15.92:0/2518438561 reap_dead start 2023-11-24T06:12:03.210 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:03.206+0000 7fce5d54c640 1 -- 172.21.15.92:0/2518438561 shutdown_connections 2023-11-24T06:12:03.210 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:03.206+0000 7fce5d54c640 1 -- 172.21.15.92:0/2518438561 >> 172.21.15.92:0/2518438561 conn(0x7fce5806a800 msgr2=0x7fce5806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:03.210 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:03.206+0000 7fce5d54c640 1 -- 172.21.15.92:0/2518438561 shutdown_connections 2023-11-24T06:12:03.210 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:03.206+0000 7fce5d54c640 1 -- 172.21.15.92:0/2518438561 wait complete. 2023-11-24T06:12:03.213 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:03.208+0000 7fce5d54c640 1 Processor -- start 2023-11-24T06:12:03.213 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:03.208+0000 7fce5d54c640 1 -- start start 2023-11-24T06:12:03.213 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:03.209+0000 7fce5d54c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fce44004d00 0x7fce5815db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:03.213 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:03.209+0000 7fce5d54c640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fce5813d210 0x7fce5813d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:03.213 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:03.209+0000 7fce5d54c640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fce5814afa0 0x7fce5814b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:03.213 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:03.209+0000 7fce5d54c640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce5813dbd0 con 0x7fce44004d00 2023-11-24T06:12:03.213 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:03.209+0000 7fce5d54c640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce5814bd30 con 0x7fce5813d210 2023-11-24T06:12:03.213 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:03.209+0000 7fce5d54c640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce5814c010 con 0x7fce5814afa0 2023-11-24T06:12:03.213 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:03.209+0000 7fce56ffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fce44004d00 0x7fce5815db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:03.213 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:03.209+0000 7fce56ffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fce44004d00 0x7fce5815db10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46866/0 (socket says 172.21.15.92:46866) 2023-11-24T06:12:03.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:03.209+0000 7fce56ffd640 1 -- 172.21.15.92:0/2149936728 learned_addr learned my addr 172.21.15.92:0/2149936728 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:12:03.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:03.210+0000 7fce577fe640 1 --2- 172.21.15.92:0/2149936728 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fce5814afa0 0x7fce5814b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:03.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:03.210+0000 7fce56ffd640 1 -- 172.21.15.92:0/2149936728 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fce5814afa0 msgr2=0x7fce5814b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:03.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:03.210+0000 7fce56ffd640 1 --2- 172.21.15.92:0/2149936728 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fce5814afa0 0x7fce5814b450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:03.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:03.210+0000 7fce4ffff640 1 --2- 172.21.15.92:0/2149936728 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fce5813d210 0x7fce5813d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:03.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:03.210+0000 7fce56ffd640 1 -- 172.21.15.92:0/2149936728 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fce5813d210 msgr2=0x7fce5813d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:03.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:03.210+0000 7fce56ffd640 1 --2- 172.21.15.92:0/2149936728 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fce5813d210 0x7fce5813d690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:03.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:03.210+0000 7fce56ffd640 1 -- 172.21.15.92:0/2149936728 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce5814c2f0 con 0x7fce44004d00 2023-11-24T06:12:03.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:03.210+0000 7fce56ffd640 1 --2- 172.21.15.92:0/2149936728 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fce44004d00 0x7fce5815db10 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fce48010860 tx=0x7fce48010d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:03.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:03.210+0000 7fce4f7fe640 1 -- 172.21.15.92:0/2149936728 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce48009610 con 0x7fce44004d00 2023-11-24T06:12:03.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:03.210+0000 7fce4f7fe640 1 -- 172.21.15.92:0/2149936728 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce4800eb60 con 0x7fce44004d00 2023-11-24T06:12:03.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:03.210+0000 7fce5d54c640 1 -- 172.21.15.92:0/2149936728 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce5814c5d0 con 0x7fce44004d00 2023-11-24T06:12:03.215 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:03.210+0000 7fce4f7fe640 1 -- 172.21.15.92:0/2149936728 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce4801c400 con 0x7fce44004d00 2023-11-24T06:12:03.215 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:03.211+0000 7fce5d54c640 1 -- 172.21.15.92:0/2149936728 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce5814cdd0 con 0x7fce44004d00 2023-11-24T06:12:03.215 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:03.212+0000 7fce5d54c640 1 -- 172.21.15.92:0/2149936728 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce580675d0 con 0x7fce44004d00 2023-11-24T06:12:03.215 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:03.213+0000 7fce4f7fe640 1 -- 172.21.15.92:0/2149936728 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fce4801c560 con 0x7fce44004d00 2023-11-24T06:12:03.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:03.214+0000 7fce4f7fe640 1 --2- 172.21.15.92:0/2149936728 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fce3806ae70 0x7fce3806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:03.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:03.214+0000 7fce4f7fe640 1 -- 172.21.15.92:0/2149936728 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6543+0+0 (secure 0 0 0) 0x7fce48089fe0 con 0x7fce44004d00 2023-11-24T06:12:03.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:03.214+0000 7fce4ffff640 1 --2- 172.21.15.92:0/2149936728 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fce3806ae70 0x7fce3806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:03.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:03.215+0000 7fce4ffff640 1 --2- 172.21.15.92:0/2149936728 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fce3806ae70 0x7fce3806d330 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fce40007fd0 tx=0x7fce40009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:03.220 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:03.218+0000 7fce4f7fe640 1 -- 172.21.15.92:0/2149936728 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fce48054d10 con 0x7fce44004d00 2023-11-24T06:12:03.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:03.434+0000 7fce5d54c640 1 -- 172.21.15.92:0/2149936728 --> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] -- mgr_command(tid 0: {"prefix": "fs volume rm", "vol_name": "vol9070", "yes-i-really-mean-it": "--yes-i-really-mean-it", "target": ["mon-mgr", ""]}) v1 -- 0x7fce5815bee0 con 0x7fce3806ae70 2023-11-24T06:12:03.446 INFO:tasks.ceph.mon.a.smithi092.stderr:2023-11-24T06:12:03.444+0000 7f83f3ce4640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-24T06:12:03.466 INFO:tasks.ceph.mds.a.smithi092.stderr: -13> 2023-11-24T06:12:02.996+0000 7ff2fe697640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-24T06:12:03.517 INFO:tasks.ceph.mds.a.smithi092.stdout:starting mds.a at 2023-11-24T06:12:06.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:06.475+0000 7fce4f7fe640 1 -- 172.21.15.92:0/2149936728 <== mgr.4101 v2:172.21.15.92:6800/784027401 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+78 (secure 0 0 0) 0x7fce4000d050 con 0x7fce3806ae70 2023-11-24T06:12:06.479 INFO:teuthology.orchestra.run.smithi092.stdout:metadata pool: cephfs.vol9070.meta data pool: ['cephfs.vol9070.data'] removed 2023-11-24T06:12:06.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:06.480+0000 7fce5d54c640 1 -- 172.21.15.92:0/2149936728 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fce3806ae70 msgr2=0x7fce3806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:06.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:06.480+0000 7fce5d54c640 1 --2- 172.21.15.92:0/2149936728 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fce3806ae70 0x7fce3806d330 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fce40007fd0 tx=0x7fce40009040 comp rx=0 tx=0).stop 2023-11-24T06:12:06.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:06.480+0000 7fce5d54c640 1 -- 172.21.15.92:0/2149936728 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fce44004d00 msgr2=0x7fce5815db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:06.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:06.480+0000 7fce5d54c640 1 --2- 172.21.15.92:0/2149936728 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fce44004d00 0x7fce5815db10 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fce48010860 tx=0x7fce48010d20 comp rx=0 tx=0).stop 2023-11-24T06:12:06.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:06.480+0000 7fce5d54c640 1 -- 172.21.15.92:0/2149936728 shutdown_connections 2023-11-24T06:12:06.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:06.480+0000 7fce5d54c640 1 --2- 172.21.15.92:0/2149936728 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fce3806ae70 0x7fce3806d330 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:06.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:06.480+0000 7fce5d54c640 1 --2- 172.21.15.92:0/2149936728 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fce5814afa0 0x7fce5814b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:06.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:06.480+0000 7fce5d54c640 1 --2- 172.21.15.92:0/2149936728 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fce5813d210 0x7fce5813d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:06.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:06.480+0000 7fce5d54c640 1 --2- 172.21.15.92:0/2149936728 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fce44004d00 0x7fce5815db10 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:06.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:06.480+0000 7fce5d54c640 1 -- 172.21.15.92:0/2149936728 >> 172.21.15.92:0/2149936728 conn(0x7fce5806a800 msgr2=0x7fce580cd840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:06.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:06.480+0000 7fce5d54c640 1 -- 172.21.15.92:0/2149936728 shutdown_connections 2023-11-24T06:12:06.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:06.481+0000 7fce5d54c640 1 -- 172.21.15.92:0/2149936728 wait complete. 2023-11-24T06:12:06.506 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-24T06:12:06.506 DEBUG:teuthology.orchestra.run.smithi092:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-24T06:12:06.623 DEBUG:teuthology.orchestra.run.smithi118:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-24T06:12:06.745 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-24T06:12:06.745 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-11-24T06:12:06.745 DEBUG:teuthology.orchestra.run.smithi092:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:12:06.804 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:12:07.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:07.011+0000 7fd88dac2640 1 Processor -- start 2023-11-24T06:12:07.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:07.011+0000 7fd88dac2640 1 -- start start 2023-11-24T06:12:07.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:07.013+0000 7fd88dac2640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fd888143ae0 0x7fd8880cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:07.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:07.013+0000 7fd88dac2640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd8880cf7d0 0x7fd88815dca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:07.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:07.013+0000 7fd88dac2640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fd88815e1e0 0x7fd88815a670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:07.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:07.013+0000 7fd88dac2640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8880cfc30 con 0x7fd888143ae0 2023-11-24T06:12:07.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:07.013+0000 7fd88dac2640 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8880b1ee0 con 0x7fd88815e1e0 2023-11-24T06:12:07.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:07.013+0000 7fd88dac2640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8880b2780 con 0x7fd8880cf7d0 2023-11-24T06:12:07.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:07.013+0000 7fd8877fe640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fd888143ae0 0x7fd8880cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:07.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:07.013+0000 7fd8877fe640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fd888143ae0 0x7fd8880cf290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46904/0 (socket says 172.21.15.92:46904) 2023-11-24T06:12:07.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:07.013+0000 7fd8877fe640 1 -- 172.21.15.92:0/70278093 learned_addr learned my addr 172.21.15.92:0/70278093 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:12:07.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:07.013+0000 7fd8877fe640 1 -- 172.21.15.92:0/70278093 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd8880cf7d0 msgr2=0x7fd88815dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:07.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:07.013+0000 7fd8877fe640 1 --2- 172.21.15.92:0/70278093 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd8880cf7d0 0x7fd88815dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:07.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:07.013+0000 7fd8877fe640 1 -- 172.21.15.92:0/70278093 >> v2:172.21.15.118:3300/0 conn(0x7fd88815e1e0 msgr2=0x7fd88815a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:07.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:07.013+0000 7fd8877fe640 1 --2- 172.21.15.92:0/70278093 >> v2:172.21.15.118:3300/0 conn(0x7fd88815e1e0 0x7fd88815a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:07.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:07.013+0000 7fd8877fe640 1 -- 172.21.15.92:0/70278093 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd88815e990 con 0x7fd888143ae0 2023-11-24T06:12:07.019 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:07.014+0000 7fd8877fe640 1 --2- 172.21.15.92:0/70278093 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd888143ae0 0x7fd8880cf290 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fd888072160 tx=0x7fd87802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b57d2a0382c1621 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:07.019 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:07.015+0000 7fd8867fc640 1 -- 172.21.15.92:0/70278093 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd878031870 con 0x7fd888143ae0 2023-11-24T06:12:07.019 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:07.015+0000 7fd8867fc640 1 -- 172.21.15.92:0/70278093 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8780319d0 con 0x7fd888143ae0 2023-11-24T06:12:07.019 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:07.015+0000 7fd8867fc640 1 -- 172.21.15.92:0/70278093 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd878031d60 con 0x7fd888143ae0 2023-11-24T06:12:07.019 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:07.015+0000 7fd88dac2640 1 -- 172.21.15.92:0/70278093 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd888143ae0 msgr2=0x7fd8880cf290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:07.019 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:07.015+0000 7fd88dac2640 1 --2- 172.21.15.92:0/70278093 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd888143ae0 0x7fd8880cf290 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fd888072160 tx=0x7fd87802b910 comp rx=0 tx=0).stop 2023-11-24T06:12:07.019 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:07.015+0000 7fd88dac2640 1 -- 172.21.15.92:0/70278093 shutdown_connections 2023-11-24T06:12:07.019 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:07.015+0000 7fd88dac2640 1 --2- 172.21.15.92:0/70278093 >> v2:172.21.15.118:3300/0 conn(0x7fd88815e1e0 0x7fd88815a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:07.019 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:07.015+0000 7fd88dac2640 1 --2- 172.21.15.92:0/70278093 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd8880cf7d0 0x7fd88815dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:07.020 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:07.015+0000 7fd88dac2640 1 --2- 172.21.15.92:0/70278093 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd888143ae0 0x7fd8880cf290 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:07.020 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:07.015+0000 7fd88dac2640 1 -- 172.21.15.92:0/70278093 >> 172.21.15.92:0/70278093 conn(0x7fd88806a800 msgr2=0x7fd88806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:07.020 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:07.015+0000 7fd88dac2640 1 -- 172.21.15.92:0/70278093 shutdown_connections 2023-11-24T06:12:07.020 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:07.016+0000 7fd88dac2640 1 -- 172.21.15.92:0/70278093 wait complete. 2023-11-24T06:12:07.020 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:07.017+0000 7fd88dac2640 1 Processor -- start 2023-11-24T06:12:07.020 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:07.017+0000 7fd88dac2640 1 -- start start 2023-11-24T06:12:07.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:07.018+0000 7fd88dac2640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd8880cf7d0 0x7fd88815ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:07.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:07.018+0000 7fd88dac2640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd888143ae0 0x7fd8880af4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:07.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:07.018+0000 7fd88dac2640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd88815e1e0 0x7fd888164b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:07.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:07.018+0000 7fd88dac2640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd888142bf0 con 0x7fd88815e1e0 2023-11-24T06:12:07.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:07.018+0000 7fd88dac2640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8881422e0 con 0x7fd888143ae0 2023-11-24T06:12:07.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:07.018+0000 7fd88dac2640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8880affb0 con 0x7fd8880cf7d0 2023-11-24T06:12:07.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:07.018+0000 7fd887fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd88815e1e0 0x7fd888164b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:07.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:07.019+0000 7fd887fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd88815e1e0 0x7fd888164b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46920/0 (socket says 172.21.15.92:46920) 2023-11-24T06:12:07.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:07.019+0000 7fd887fff640 1 -- 172.21.15.92:0/2709269855 learned_addr learned my addr 172.21.15.92:0/2709269855 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:12:07.023 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:07.019+0000 7fd8877fe640 1 --2- 172.21.15.92:0/2709269855 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd8880cf7d0 0x7fd88815ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:07.024 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:07.019+0000 7fd887fff640 1 -- 172.21.15.92:0/2709269855 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd8880cf7d0 msgr2=0x7fd88815ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:07.024 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:07.019+0000 7fd887fff640 1 --2- 172.21.15.92:0/2709269855 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd8880cf7d0 0x7fd88815ca60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:07.024 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:07.019+0000 7fd887fff640 1 -- 172.21.15.92:0/2709269855 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd888143ae0 msgr2=0x7fd8880af4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:07.024 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:07.019+0000 7fd886ffd640 1 --2- 172.21.15.92:0/2709269855 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd888143ae0 0x7fd8880af4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:07.024 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:07.019+0000 7fd887fff640 1 --2- 172.21.15.92:0/2709269855 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd888143ae0 0x7fd8880af4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:07.024 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:07.019+0000 7fd887fff640 1 -- 172.21.15.92:0/2709269855 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8780094d0 con 0x7fd88815e1e0 2023-11-24T06:12:07.024 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:07.019+0000 7fd887fff640 1 --2- 172.21.15.92:0/2709269855 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd88815e1e0 0x7fd888164b50 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fd88000b9a0 tx=0x7fd88000be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:07.024 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:07.019+0000 7fd86ffff640 1 -- 172.21.15.92:0/2709269855 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd88000cba0 con 0x7fd88815e1e0 2023-11-24T06:12:07.024 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:07.019+0000 7fd86ffff640 1 -- 172.21.15.92:0/2709269855 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd88000cd20 con 0x7fd88815e1e0 2023-11-24T06:12:07.024 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:07.019+0000 7fd86ffff640 1 -- 172.21.15.92:0/2709269855 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd880014680 con 0x7fd88815e1e0 2023-11-24T06:12:07.024 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:07.020+0000 7fd88dac2640 1 -- 172.21.15.92:0/2709269855 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd888142530 con 0x7fd88815e1e0 2023-11-24T06:12:07.024 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:07.020+0000 7fd88dac2640 1 -- 172.21.15.92:0/2709269855 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd888140bf0 con 0x7fd88815e1e0 2023-11-24T06:12:07.025 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:07.021+0000 7fd88dac2640 1 -- 172.21.15.92:0/2709269855 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8880b2bf0 con 0x7fd88815e1e0 2023-11-24T06:12:07.025 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:07.022+0000 7fd86ffff640 1 -- 172.21.15.92:0/2709269855 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd88001d3b0 con 0x7fd88815e1e0 2023-11-24T06:12:07.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:07.023+0000 7fd86ffff640 1 --2- 172.21.15.92:0/2709269855 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fd87406ae40 0x7fd87406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:07.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:07.023+0000 7fd86ffff640 1 -- 172.21.15.92:0/2709269855 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fd8800895f0 con 0x7fd88815e1e0 2023-11-24T06:12:07.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:07.023+0000 7fd8877fe640 1 --2- 172.21.15.92:0/2709269855 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fd87406ae40 0x7fd87406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:07.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:07.024+0000 7fd8877fe640 1 --2- 172.21.15.92:0/2709269855 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fd87406ae40 0x7fd87406d300 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fd878036000 tx=0x7fd87803c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:07.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:07.028+0000 7fd86ffff640 1 -- 172.21.15.92:0/2709269855 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd8880b2bf0 con 0x7fd88815e1e0 2023-11-24T06:12:07.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:07.228+0000 7fd88dac2640 1 -- 172.21.15.92:0/2709269855 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd8880b2bf0 con 0x7fd88815e1e0 2023-11-24T06:12:07.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:07.231+0000 7fd86ffff640 1 -- 172.21.15.92:0/2709269855 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+16650 (secure 0 0 0) 0x7fd8880b2bf0 con 0x7fd88815e1e0 2023-11-24T06:12:07.235 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-24T06:12:07.235 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":40,"fsid":"2d6dee5f-8aa3-470f-931a-50ade963b1cb","created":"2023-11-24T06:10:04.707204+0000","modified":"2023-11-24T06:12:06.456223+0000","last_up_change":"2023-11-24T06:10:17.598352+0000","last_in_change":"2023-11-24T06:10:09.040793+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:10:17.733098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:11:42.241681+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-11-24T06:11:43.522720+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ff37a9fb-135f-471f-b4b0-f7b5772006c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6803","nonce":20560714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6805","nonce":20560714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6809","nonce":20560714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6807","nonce":20560714}]},"public_addr":"172.21.15.92:6803/20560714","cluster_addr":"172.21.15.92:6805/20560714","heartbeat_back_addr":"172.21.15.92:6809/20560714","heartbeat_front_addr":"172.21.15.92:6807/20560714","state":["exists","up"]},{"osd":1,"uuid":"03cdd353-dcf1-4f38-94fe-a795137c06b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6819","nonce":2451239906}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6821","nonce":2451239906}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6825","nonce":2451239906}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6823","nonce":2451239906}]},"public_addr":"172.21.15.92:6819/2451239906","cluster_addr":"172.21.15.92:6821/2451239906","heartbeat_back_addr":"172.21.15.92:6825/2451239906","heartbeat_front_addr":"172.21.15.92:6823/2451239906","state":["exists","up"]},{"osd":2,"uuid":"a4a20d80-5d57-484b-becd-9aa8070dd3aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6811","nonce":1124979277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6813","nonce":1124979277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6817","nonce":1124979277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6815","nonce":1124979277}]},"public_addr":"172.21.15.92:6811/1124979277","cluster_addr":"172.21.15.92:6813/1124979277","heartbeat_back_addr":"172.21.15.92:6817/1124979277","heartbeat_front_addr":"172.21.15.92:6815/1124979277","state":["exists","up"]},{"osd":3,"uuid":"4648d7ce-9b29-4583-9425-4ba8fd2b1903","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6827","nonce":1734115782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6829","nonce":1734115782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6833","nonce":1734115782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6831","nonce":1734115782}]},"public_addr":"172.21.15.92:6827/1734115782","cluster_addr":"172.21.15.92:6829/1734115782","heartbeat_back_addr":"172.21.15.92:6833/1734115782","heartbeat_front_addr":"172.21.15.92:6831/1734115782","state":["exists","up"]},{"osd":4,"uuid":"3bc9d7eb-c6fa-4fcf-a3ec-ade604e05aac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6819","nonce":3425937229}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6822","nonce":3425937229}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6830","nonce":3425937229}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6826","nonce":3425937229}]},"public_addr":"172.21.15.118:6819/3425937229","cluster_addr":"172.21.15.118:6822/3425937229","heartbeat_back_addr":"172.21.15.118:6830/3425937229","heartbeat_front_addr":"172.21.15.118:6826/3425937229","state":["exists","up"]},{"osd":5,"uuid":"dce9ba6d-59ed-46d1-84fc-a84b13de10c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6801","nonce":827455984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6803","nonce":827455984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6807","nonce":827455984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6805","nonce":827455984}]},"public_addr":"172.21.15.118:6801/827455984","cluster_addr":"172.21.15.118:6803/827455984","heartbeat_back_addr":"172.21.15.118:6807/827455984","heartbeat_front_addr":"172.21.15.118:6805/827455984","state":["exists","up"]},{"osd":6,"uuid":"82aec504-ae58-4709-8d30-46f2b12081b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6809","nonce":2946986141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6811","nonce":2946986141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6815","nonce":2946986141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6813","nonce":2946986141}]},"public_addr":"172.21.15.118:6809/2946986141","cluster_addr":"172.21.15.118:6811/2946986141","heartbeat_back_addr":"172.21.15.118:6815/2946986141","heartbeat_front_addr":"172.21.15.118:6813/2946986141","state":["exists","up"]},{"osd":7,"uuid":"4ed4c624-c88f-4d35-bdb1-bf209b5dffb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6817","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6818","nonce":48122059}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6821","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6823","nonce":48122059}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6829","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6831","nonce":48122059}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6825","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6827","nonce":48122059}]},"public_addr":"172.21.15.118:6818/48122059","cluster_addr":"172.21.15.118:6823/48122059","heartbeat_back_addr":"172.21.15.118:6831/48122059","heartbeat_front_addr":"172.21.15.118:6827/48122059","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.536664+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.368324+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.271201+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.342971+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.621832+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.387110+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.632946+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.416032+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.92:6836/3889219158":"2023-11-25T06:12:03.443855+0000","172.21.15.92:6834/3889219158":"2023-11-25T06:12:03.443855+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:12:07.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:07.234+0000 7fd88dac2640 1 -- 172.21.15.92:0/2709269855 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fd87406ae40 msgr2=0x7fd87406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:07.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:07.234+0000 7fd88dac2640 1 --2- 172.21.15.92:0/2709269855 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fd87406ae40 0x7fd87406d300 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fd878036000 tx=0x7fd87803c040 comp rx=0 tx=0).stop 2023-11-24T06:12:07.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:07.234+0000 7fd88dac2640 1 -- 172.21.15.92:0/2709269855 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd88815e1e0 msgr2=0x7fd888164b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:07.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:07.234+0000 7fd88dac2640 1 --2- 172.21.15.92:0/2709269855 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd88815e1e0 0x7fd888164b50 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fd88000b9a0 tx=0x7fd88000be60 comp rx=0 tx=0).stop 2023-11-24T06:12:07.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:07.234+0000 7fd88dac2640 1 -- 172.21.15.92:0/2709269855 shutdown_connections 2023-11-24T06:12:07.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:07.234+0000 7fd88dac2640 1 --2- 172.21.15.92:0/2709269855 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fd87406ae40 0x7fd87406d300 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:07.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:07.234+0000 7fd88dac2640 1 --2- 172.21.15.92:0/2709269855 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd88815e1e0 0x7fd888164b50 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:07.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:07.234+0000 7fd88dac2640 1 --2- 172.21.15.92:0/2709269855 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd888143ae0 0x7fd8880af4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:07.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:07.234+0000 7fd88dac2640 1 --2- 172.21.15.92:0/2709269855 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd8880cf7d0 0x7fd88815ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:07.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:07.234+0000 7fd88dac2640 1 -- 172.21.15.92:0/2709269855 >> 172.21.15.92:0/2709269855 conn(0x7fd88806a800 msgr2=0x7fd88815bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:07.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:07.234+0000 7fd88dac2640 1 -- 172.21.15.92:0/2709269855 shutdown_connections 2023-11-24T06:12:07.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:07.234+0000 7fd88dac2640 1 -- 172.21.15.92:0/2709269855 wait complete. 2023-11-24T06:12:07.255 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T06:10:17.733098+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T06:11:42.241681+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '32', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T06:11:43.522720+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '33', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T06:12:07.255 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T06:12:07.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:07.469+0000 7fded25e5640 1 Processor -- start 2023-11-24T06:12:07.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:07.469+0000 7fded25e5640 1 -- start start 2023-11-24T06:12:07.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:07.471+0000 7fded25e5640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fdecc0ae5a0 0x7fdecc15e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:07.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:07.471+0000 7fded25e5640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fdecc0b3ae0 0x7fdecc06cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:07.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:07.471+0000 7fded25e5640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fdecc06d120 0x7fdecc0cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:07.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:07.471+0000 7fded25e5640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdecc071450 con 0x7fdecc0ae5a0 2023-11-24T06:12:07.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:07.471+0000 7fded25e5640 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdecc070c40 con 0x7fdecc06d120 2023-11-24T06:12:07.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:07.471+0000 7fded25e5640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdecc06c470 con 0x7fdecc0b3ae0 2023-11-24T06:12:07.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:07.471+0000 7fdecbfff640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fdecc0ae5a0 0x7fdecc15e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:07.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:07.471+0000 7fdecbfff640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fdecc0ae5a0 0x7fdecc15e430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46934/0 (socket says 172.21.15.92:46934) 2023-11-24T06:12:07.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:07.471+0000 7fdecbfff640 1 -- 172.21.15.92:0/3635066435 learned_addr learned my addr 172.21.15.92:0/3635066435 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:12:07.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:07.472+0000 7fdecbfff640 1 -- 172.21.15.92:0/3635066435 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fdecc0b3ae0 msgr2=0x7fdecc06cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:07.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:07.472+0000 7fdecbfff640 1 --2- 172.21.15.92:0/3635066435 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fdecc0b3ae0 0x7fdecc06cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:07.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:07.472+0000 7fdecbfff640 1 -- 172.21.15.92:0/3635066435 >> v2:172.21.15.118:3300/0 conn(0x7fdecc06d120 msgr2=0x7fdecc0cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:07.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:07.472+0000 7fded0b82640 1 --2- 172.21.15.92:0/3635066435 >> v2:172.21.15.118:3300/0 conn(0x7fdecc06d120 0x7fdecc0cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:07.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:07.472+0000 7fdecbfff640 1 --2- 172.21.15.92:0/3635066435 >> v2:172.21.15.118:3300/0 conn(0x7fdecc06d120 0x7fdecc0cf290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:07.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:07.472+0000 7fdecbfff640 1 -- 172.21.15.92:0/3635066435 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdecc06c750 con 0x7fdecc0ae5a0 2023-11-24T06:12:07.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:07.473+0000 7fdecbfff640 1 --2- 172.21.15.92:0/3635066435 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdecc0ae5a0 0x7fdecc15e430 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fdecc06e230 tx=0x7fdebc02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7aec971c99b94bb8 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:07.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:07.473+0000 7fdecaffd640 1 -- 172.21.15.92:0/3635066435 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdebc031870 con 0x7fdecc0ae5a0 2023-11-24T06:12:07.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:07.473+0000 7fdecaffd640 1 -- 172.21.15.92:0/3635066435 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdebc0319d0 con 0x7fdecc0ae5a0 2023-11-24T06:12:07.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:07.473+0000 7fdecaffd640 1 -- 172.21.15.92:0/3635066435 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdebc031d60 con 0x7fdecc0ae5a0 2023-11-24T06:12:07.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:07.474+0000 7fded25e5640 1 -- 172.21.15.92:0/3635066435 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdecc0ae5a0 msgr2=0x7fdecc15e430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:07.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:07.474+0000 7fded25e5640 1 --2- 172.21.15.92:0/3635066435 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdecc0ae5a0 0x7fdecc15e430 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fdecc06e230 tx=0x7fdebc02b910 comp rx=0 tx=0).stop 2023-11-24T06:12:07.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:07.474+0000 7fded25e5640 1 -- 172.21.15.92:0/3635066435 shutdown_connections 2023-11-24T06:12:07.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:07.474+0000 7fded25e5640 1 --2- 172.21.15.92:0/3635066435 >> v2:172.21.15.118:3300/0 conn(0x7fdecc06d120 0x7fdecc0cf290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:07.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:07.474+0000 7fded25e5640 1 --2- 172.21.15.92:0/3635066435 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fdecc0b3ae0 0x7fdecc06cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:07.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:07.474+0000 7fded25e5640 1 --2- 172.21.15.92:0/3635066435 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdecc0ae5a0 0x7fdecc15e430 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:07.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:07.474+0000 7fded25e5640 1 -- 172.21.15.92:0/3635066435 >> 172.21.15.92:0/3635066435 conn(0x7fdecc06a800 msgr2=0x7fdecc06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:07.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:07.474+0000 7fded25e5640 1 -- 172.21.15.92:0/3635066435 shutdown_connections 2023-11-24T06:12:07.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:07.474+0000 7fded25e5640 1 -- 172.21.15.92:0/3635066435 wait complete. 2023-11-24T06:12:07.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:07.476+0000 7fded25e5640 1 Processor -- start 2023-11-24T06:12:07.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:07.476+0000 7fded25e5640 1 -- start start 2023-11-24T06:12:07.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:07.477+0000 7fded25e5640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fdecc06d120 0x7fdecc1f59e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:07.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:07.477+0000 7fded25e5640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdecc0ae5a0 0x7fdecc1596b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:07.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:07.477+0000 7fded25e5640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fdecc0b3ae0 0x7fdecc159bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:07.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:07.477+0000 7fded25e5640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdecc1f3a00 con 0x7fdecc0ae5a0 2023-11-24T06:12:07.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:07.477+0000 7fded25e5640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdecc1f2b30 con 0x7fdecc0b3ae0 2023-11-24T06:12:07.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:07.477+0000 7fded25e5640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdecc1f5f20 con 0x7fdecc06d120 2023-11-24T06:12:07.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:07.477+0000 7fdecb7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdecc0ae5a0 0x7fdecc1596b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:07.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:07.477+0000 7fdecb7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdecc0ae5a0 0x7fdecc1596b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46944/0 (socket says 172.21.15.92:46944) 2023-11-24T06:12:07.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:07.477+0000 7fdecb7fe640 1 -- 172.21.15.92:0/3808955736 learned_addr learned my addr 172.21.15.92:0/3808955736 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:12:07.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:07.477+0000 7fded0b82640 1 --2- 172.21.15.92:0/3808955736 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fdecc0b3ae0 0x7fdecc159bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:07.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:07.477+0000 7fdecbfff640 1 --2- 172.21.15.92:0/3808955736 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fdecc06d120 0x7fdecc1f59e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:07.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:07.477+0000 7fdecb7fe640 1 -- 172.21.15.92:0/3808955736 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fdecc06d120 msgr2=0x7fdecc1f59e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:07.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:07.477+0000 7fdecb7fe640 1 --2- 172.21.15.92:0/3808955736 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fdecc06d120 0x7fdecc1f59e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:07.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:07.477+0000 7fdecb7fe640 1 -- 172.21.15.92:0/3808955736 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fdecc0b3ae0 msgr2=0x7fdecc159bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:07.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:07.477+0000 7fdecb7fe640 1 --2- 172.21.15.92:0/3808955736 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fdecc0b3ae0 0x7fdecc159bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:07.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:07.477+0000 7fdecb7fe640 1 -- 172.21.15.92:0/3808955736 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdebc0094d0 con 0x7fdecc0ae5a0 2023-11-24T06:12:07.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:07.477+0000 7fded0b82640 1 --2- 172.21.15.92:0/3808955736 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fdecc0b3ae0 0x7fdecc159bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:12:07.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:07.478+0000 7fdecb7fe640 1 --2- 172.21.15.92:0/3808955736 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdecc0ae5a0 0x7fdecc1596b0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fdec000b440 tx=0x7fdec000b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:07.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:07.478+0000 7fdec8ff9640 1 -- 172.21.15.92:0/3808955736 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdec0011890 con 0x7fdecc0ae5a0 2023-11-24T06:12:07.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:07.478+0000 7fdec8ff9640 1 -- 172.21.15.92:0/3808955736 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdec00119f0 con 0x7fdecc0ae5a0 2023-11-24T06:12:07.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:07.478+0000 7fded25e5640 1 -- 172.21.15.92:0/3808955736 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdecc1f2d80 con 0x7fdecc0ae5a0 2023-11-24T06:12:07.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:07.478+0000 7fdec8ff9640 1 -- 172.21.15.92:0/3808955736 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdec0011da0 con 0x7fdecc0ae5a0 2023-11-24T06:12:07.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:07.478+0000 7fded25e5640 1 -- 172.21.15.92:0/3808955736 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdecc1f3340 con 0x7fdecc0ae5a0 2023-11-24T06:12:07.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:07.479+0000 7fded25e5640 1 -- 172.21.15.92:0/3808955736 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdecc0675d0 con 0x7fdecc0ae5a0 2023-11-24T06:12:07.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:07.480+0000 7fdec8ff9640 1 -- 172.21.15.92:0/3808955736 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fdec001a400 con 0x7fdecc0ae5a0 2023-11-24T06:12:07.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:07.481+0000 7fdec8ff9640 1 --2- 172.21.15.92:0/3808955736 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fdeb806ae40 0x7fdeb806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:07.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:07.481+0000 7fdec8ff9640 1 -- 172.21.15.92:0/3808955736 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fdec0089a30 con 0x7fdecc0ae5a0 2023-11-24T06:12:07.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:07.481+0000 7fdecbfff640 1 --2- 172.21.15.92:0/3808955736 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fdeb806ae40 0x7fdeb806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:07.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:07.482+0000 7fdecbfff640 1 --2- 172.21.15.92:0/3808955736 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fdeb806ae40 0x7fdeb806d300 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fdebc036000 tx=0x7fdebc03c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:07.488 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:07.486+0000 7fdec8ff9640 1 -- 172.21.15.92:0/3808955736 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fdec0054a90 con 0x7fdecc0ae5a0 2023-11-24T06:12:07.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:07.674+0000 7fded25e5640 1 -- 172.21.15.92:0/3808955736 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fdecc1f3b40 con 0x7fdecc0ae5a0 2023-11-24T06:12:07.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:07.675+0000 7fdec8ff9640 1 -- 172.21.15.92:0/3808955736 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v40) v1 ==== 93+0+10 (secure 0 0 0) 0x7fdec0015020 con 0x7fdecc0ae5a0 2023-11-24T06:12:07.678 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 1 2023-11-24T06:12:07.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:07.677+0000 7fded25e5640 1 -- 172.21.15.92:0/3808955736 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fdeb806ae40 msgr2=0x7fdeb806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:07.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:07.677+0000 7fded25e5640 1 --2- 172.21.15.92:0/3808955736 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fdeb806ae40 0x7fdeb806d300 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fdebc036000 tx=0x7fdebc03c040 comp rx=0 tx=0).stop 2023-11-24T06:12:07.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:07.677+0000 7fded25e5640 1 -- 172.21.15.92:0/3808955736 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdecc0ae5a0 msgr2=0x7fdecc1596b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:07.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:07.677+0000 7fded25e5640 1 --2- 172.21.15.92:0/3808955736 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdecc0ae5a0 0x7fdecc1596b0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fdec000b440 tx=0x7fdec000b900 comp rx=0 tx=0).stop 2023-11-24T06:12:07.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:07.678+0000 7fded25e5640 1 -- 172.21.15.92:0/3808955736 shutdown_connections 2023-11-24T06:12:07.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:07.678+0000 7fded25e5640 1 --2- 172.21.15.92:0/3808955736 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fdeb806ae40 0x7fdeb806d300 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:07.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:07.678+0000 7fded25e5640 1 --2- 172.21.15.92:0/3808955736 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fdecc0b3ae0 0x7fdecc159bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:07.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:07.678+0000 7fded25e5640 1 --2- 172.21.15.92:0/3808955736 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdecc0ae5a0 0x7fdecc1596b0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:07.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:07.678+0000 7fded25e5640 1 --2- 172.21.15.92:0/3808955736 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fdecc06d120 0x7fdecc1f59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:07.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:07.678+0000 7fded25e5640 1 -- 172.21.15.92:0/3808955736 >> 172.21.15.92:0/3808955736 conn(0x7fdecc06a800 msgr2=0x7fdecc06abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:07.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:07.678+0000 7fded25e5640 1 -- 172.21.15.92:0/3808955736 shutdown_connections 2023-11-24T06:12:07.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:07.678+0000 7fded25e5640 1 -- 172.21.15.92:0/3808955736 wait complete. 2023-11-24T06:12:07.699 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T06:12:07.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:07.903+0000 7f5f9359e640 1 Processor -- start 2023-11-24T06:12:07.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:07.904+0000 7f5f9359e640 1 -- start start 2023-11-24T06:12:07.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:07.905+0000 7f5f9359e640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f5f940cf9a0 0x7f5f940cfde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:07.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:07.905+0000 7f5f9359e640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f940d0320 con 0x7f5f940b31d0 2023-11-24T06:12:07.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:07.905+0000 7f5f9359e640 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f940d1c60 con 0x7f5f9415d4b0 2023-11-24T06:12:07.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:07.905+0000 7f5f9359e640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f940b2730 con 0x7f5f940cf9a0 2023-11-24T06:12:07.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:07.905+0000 7f5f91d9b640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f5f940cf9a0 0x7f5f940cfde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:07.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:07.905+0000 7f5f91d9b640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f5f940cf9a0 0x7f5f940cfde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46946/0 (socket says 172.21.15.92:46946) 2023-11-24T06:12:07.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:07.906+0000 7f5f91d9b640 1 -- 172.21.15.92:0/949374515 learned_addr learned my addr 172.21.15.92:0/949374515 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:12:07.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:07.906+0000 7f5f91d9b640 1 -- 172.21.15.92:0/949374515 >> v1:172.21.15.118:6789/0 conn(0x7f5f9415d4b0 legacy=0x7f5f940d1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:07.910 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:07.906+0000 7f5f91d9b640 1 -- 172.21.15.92:0/949374515 >> v1:172.21.15.92:6789/0 conn(0x7f5f940b31d0 legacy=0x7f5f940b3660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:07.910 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:07.906+0000 7f5f91d9b640 1 -- 172.21.15.92:0/949374515 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f940d1ec0 con 0x7f5f940cf9a0 2023-11-24T06:12:07.910 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:07.906+0000 7f5f9159a640 1 -- 172.21.15.92:0/949374515 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2654252806 0 0) 0x7f5f940d0320 con 0x7f5f940b31d0 2023-11-24T06:12:07.910 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:07.907+0000 7f5f91d9b640 1 --2- 172.21.15.92:0/949374515 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5f940cf9a0 0x7f5f940cfde0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f5f7c009870 tx=0x7f5f7c02ba30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a35e454e76b9e8f5 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:07.911 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:07.907+0000 7f5f9159a640 1 -- 172.21.15.92:0/949374515 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f7c02f850 con 0x7f5f940cf9a0 2023-11-24T06:12:07.911 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:07.908+0000 7f5f9159a640 1 -- 172.21.15.92:0/949374515 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f7c02f9b0 con 0x7f5f940cf9a0 2023-11-24T06:12:07.911 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:07.908+0000 7f5f9159a640 1 -- 172.21.15.92:0/949374515 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f7c02fd40 con 0x7f5f940cf9a0 2023-11-24T06:12:07.911 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:07.908+0000 7f5f9359e640 1 -- 172.21.15.92:0/949374515 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5f940cf9a0 msgr2=0x7f5f940cfde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:07.912 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:07.908+0000 7f5f9359e640 1 --2- 172.21.15.92:0/949374515 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5f940cf9a0 0x7f5f940cfde0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f5f7c009870 tx=0x7f5f7c02ba30 comp rx=0 tx=0).stop 2023-11-24T06:12:07.912 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:07.908+0000 7f5f9359e640 1 -- 172.21.15.92:0/949374515 shutdown_connections 2023-11-24T06:12:07.912 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:07.908+0000 7f5f9359e640 1 --2- 172.21.15.92:0/949374515 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5f940cf9a0 0x7f5f940cfde0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:07.912 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:07.908+0000 7f5f9359e640 1 -- 172.21.15.92:0/949374515 >> 172.21.15.92:0/949374515 conn(0x7f5f940d3000 msgr2=0x7f5f940d3410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:07.912 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:07.908+0000 7f5f9359e640 1 -- 172.21.15.92:0/949374515 shutdown_connections 2023-11-24T06:12:07.912 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:07.908+0000 7f5f9359e640 1 -- 172.21.15.92:0/949374515 wait complete. 2023-11-24T06:12:07.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:07.910+0000 7f5f9359e640 1 Processor -- start 2023-11-24T06:12:07.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:07.910+0000 7f5f9359e640 1 -- start start 2023-11-24T06:12:07.916 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:07.911+0000 7f5f9359e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5f940b31d0 0x7f5f941f5d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:07.916 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:07.911+0000 7f5f9359e640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5f940cf9a0 0x7f5f9409ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:07.916 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:07.911+0000 7f5f9359e640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5f9415d4b0 0x7f5f9409f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:07.916 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:07.911+0000 7f5f9359e640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f940cb870 con 0x7f5f940b31d0 2023-11-24T06:12:07.916 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:07.911+0000 7f5f9359e640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f941e28d0 con 0x7f5f940cf9a0 2023-11-24T06:12:07.916 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:07.911+0000 7f5f9359e640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f940cabf0 con 0x7f5f9415d4b0 2023-11-24T06:12:07.916 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:07.911+0000 7f5f9259c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5f940b31d0 0x7f5f941f5d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:07.916 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:07.912+0000 7f5f9259c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5f940b31d0 0x7f5f941f5d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46962/0 (socket says 172.21.15.92:46962) 2023-11-24T06:12:07.916 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:07.912+0000 7f5f9259c640 1 -- 172.21.15.92:0/1052010377 learned_addr learned my addr 172.21.15.92:0/1052010377 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:12:07.916 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:07.912+0000 7f5f91d9b640 1 --2- 172.21.15.92:0/1052010377 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5f940cf9a0 0x7f5f9409ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:07.917 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:07.912+0000 7f5f92d9d640 1 --2- 172.21.15.92:0/1052010377 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5f9415d4b0 0x7f5f9409f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:07.917 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:07.912+0000 7f5f9259c640 1 -- 172.21.15.92:0/1052010377 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5f9415d4b0 msgr2=0x7f5f9409f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:07.917 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:07.912+0000 7f5f9259c640 1 --2- 172.21.15.92:0/1052010377 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5f9415d4b0 0x7f5f9409f4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:07.917 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:07.912+0000 7f5f9259c640 1 -- 172.21.15.92:0/1052010377 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5f940cf9a0 msgr2=0x7f5f9409ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:07.917 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:07.912+0000 7f5f9259c640 1 --2- 172.21.15.92:0/1052010377 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5f940cf9a0 0x7f5f9409ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:07.917 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:07.912+0000 7f5f9259c640 1 -- 172.21.15.92:0/1052010377 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f7c0094d0 con 0x7f5f940b31d0 2023-11-24T06:12:07.917 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:07.912+0000 7f5f9259c640 1 --2- 172.21.15.92:0/1052010377 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5f940b31d0 0x7f5f941f5d80 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f5f8400c730 tx=0x7f5f8400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:07.917 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:07.913+0000 7f5f8affd640 1 -- 172.21.15.92:0/1052010377 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f84003b70 con 0x7f5f940b31d0 2023-11-24T06:12:07.917 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:07.913+0000 7f5f8affd640 1 -- 172.21.15.92:0/1052010377 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f84003cf0 con 0x7f5f940b31d0 2023-11-24T06:12:07.917 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:07.913+0000 7f5f8affd640 1 -- 172.21.15.92:0/1052010377 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f84003ea0 con 0x7f5f940b31d0 2023-11-24T06:12:07.917 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:07.913+0000 7f5f9359e640 1 -- 172.21.15.92:0/1052010377 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f940caed0 con 0x7f5f940b31d0 2023-11-24T06:12:07.917 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:07.913+0000 7f5f9359e640 1 -- 172.21.15.92:0/1052010377 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f940cb490 con 0x7f5f940b31d0 2023-11-24T06:12:07.918 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:07.914+0000 7f5f9359e640 1 -- 172.21.15.92:0/1052010377 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f9408c830 con 0x7f5f940b31d0 2023-11-24T06:12:07.918 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:07.915+0000 7f5f8affd640 1 -- 172.21.15.92:0/1052010377 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5f8400aa90 con 0x7f5f940b31d0 2023-11-24T06:12:07.920 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:07.916+0000 7f5f8affd640 1 --2- 172.21.15.92:0/1052010377 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f5f7406ae40 0x7f5f7406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:07.920 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:07.917+0000 7f5f8affd640 1 -- 172.21.15.92:0/1052010377 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f5f840894a0 con 0x7f5f940b31d0 2023-11-24T06:12:07.920 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:07.917+0000 7f5f91d9b640 1 --2- 172.21.15.92:0/1052010377 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f5f7406ae40 0x7f5f7406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:07.920 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:07.917+0000 7f5f91d9b640 1 --2- 172.21.15.92:0/1052010377 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f5f7406ae40 0x7f5f7406d300 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f5f7c036000 tx=0x7f5f7c008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:07.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:07.921+0000 7f5f8affd640 1 -- 172.21.15.92:0/1052010377 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5f840544f0 con 0x7f5f940b31d0 2023-11-24T06:12:08.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:08.112+0000 7f5f9359e640 1 -- 172.21.15.92:0/1052010377 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f5f94067f00 con 0x7f5f940b31d0 2023-11-24T06:12:08.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:08.113+0000 7f5f8affd640 1 -- 172.21.15.92:0/1052010377 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v40) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5f84057b10 con 0x7f5f940b31d0 2023-11-24T06:12:08.117 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 32 2023-11-24T06:12:08.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:08.116+0000 7f5f9359e640 1 -- 172.21.15.92:0/1052010377 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f5f7406ae40 msgr2=0x7f5f7406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:08.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:08.116+0000 7f5f9359e640 1 --2- 172.21.15.92:0/1052010377 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f5f7406ae40 0x7f5f7406d300 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f5f7c036000 tx=0x7f5f7c008030 comp rx=0 tx=0).stop 2023-11-24T06:12:08.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:08.116+0000 7f5f9359e640 1 -- 172.21.15.92:0/1052010377 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5f940b31d0 msgr2=0x7f5f941f5d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:08.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:08.116+0000 7f5f9359e640 1 --2- 172.21.15.92:0/1052010377 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5f940b31d0 0x7f5f941f5d80 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f5f8400c730 tx=0x7f5f8400cbf0 comp rx=0 tx=0).stop 2023-11-24T06:12:08.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:08.116+0000 7f5f9359e640 1 -- 172.21.15.92:0/1052010377 shutdown_connections 2023-11-24T06:12:08.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:08.116+0000 7f5f9359e640 1 --2- 172.21.15.92:0/1052010377 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f5f7406ae40 0x7f5f7406d300 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:08.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:08.116+0000 7f5f9359e640 1 --2- 172.21.15.92:0/1052010377 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5f9415d4b0 0x7f5f9409f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:08.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:08.116+0000 7f5f9359e640 1 --2- 172.21.15.92:0/1052010377 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5f940cf9a0 0x7f5f9409ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:08.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:08.116+0000 7f5f9359e640 1 --2- 172.21.15.92:0/1052010377 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5f940b31d0 0x7f5f941f5d80 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:08.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:08.116+0000 7f5f9359e640 1 -- 172.21.15.92:0/1052010377 >> 172.21.15.92:0/1052010377 conn(0x7f5f940d3000 msgr2=0x7f5f940d1530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:08.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:08.116+0000 7f5f9359e640 1 -- 172.21.15.92:0/1052010377 shutdown_connections 2023-11-24T06:12:08.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:08.117+0000 7f5f9359e640 1 -- 172.21.15.92:0/1052010377 wait complete. 2023-11-24T06:12:08.138 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T06:12:08.355 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:08.351+0000 7ff675232640 1 Processor -- start 2023-11-24T06:12:08.355 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:08.352+0000 7ff675232640 1 -- start start 2023-11-24T06:12:08.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:08.353+0000 7ff675232640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7ff67013df80 0x7ff67015bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:08.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:08.353+0000 7ff675232640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ff67013e360 0x7ff6700cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:08.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:08.353+0000 7ff675232640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff67015f9c0 0x7ff6700cf7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:08.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:08.353+0000 7ff675232640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff67006c470 con 0x7ff67013e360 2023-11-24T06:12:08.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:08.353+0000 7ff675232640 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff67006c750 con 0x7ff67013df80 2023-11-24T06:12:08.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:08.353+0000 7ff675232640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff67006ca30 con 0x7ff67015f9c0 2023-11-24T06:12:08.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:08.354+0000 7ff667fff640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ff67013e360 0x7ff6700cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:08.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:08.354+0000 7ff667fff640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ff67013e360 0x7ff6700cf290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46966/0 (socket says 172.21.15.92:46966) 2023-11-24T06:12:08.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:08.354+0000 7ff667fff640 1 -- 172.21.15.92:0/3571835746 learned_addr learned my addr 172.21.15.92:0/3571835746 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:12:08.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:08.354+0000 7ff66ed9d640 1 --2- 172.21.15.92:0/3571835746 >> v2:172.21.15.118:3300/0 conn(0x7ff67013df80 0x7ff67015bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:08.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:08.354+0000 7ff66f59e640 1 --2- 172.21.15.92:0/3571835746 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff67015f9c0 0x7ff6700cf7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:08.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:08.354+0000 7ff667fff640 1 -- 172.21.15.92:0/3571835746 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff67015f9c0 msgr2=0x7ff6700cf7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:08.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:08.354+0000 7ff667fff640 1 --2- 172.21.15.92:0/3571835746 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff67015f9c0 0x7ff6700cf7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:08.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:08.354+0000 7ff667fff640 1 -- 172.21.15.92:0/3571835746 >> v2:172.21.15.118:3300/0 conn(0x7ff67013df80 msgr2=0x7ff67015bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:08.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:08.354+0000 7ff667fff640 1 --2- 172.21.15.92:0/3571835746 >> v2:172.21.15.118:3300/0 conn(0x7ff67013df80 0x7ff67015bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:08.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:08.354+0000 7ff667fff640 1 -- 172.21.15.92:0/3571835746 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff67006cd10 con 0x7ff67013e360 2023-11-24T06:12:08.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:08.354+0000 7ff66f59e640 1 --2- 172.21.15.92:0/3571835746 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff67015f9c0 0x7ff6700cf7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:12:08.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:08.355+0000 7ff667fff640 1 --2- 172.21.15.92:0/3571835746 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff67013e360 0x7ff6700cf290 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7ff670072770 tx=0x7ff65802b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2209f128ed72657 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:08.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:08.356+0000 7ff66e59c640 1 -- 172.21.15.92:0/3571835746 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff658031860 con 0x7ff67013e360 2023-11-24T06:12:08.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:08.356+0000 7ff66e59c640 1 -- 172.21.15.92:0/3571835746 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6580319c0 con 0x7ff67013e360 2023-11-24T06:12:08.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:08.356+0000 7ff66e59c640 1 -- 172.21.15.92:0/3571835746 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff658031d50 con 0x7ff67013e360 2023-11-24T06:12:08.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:08.356+0000 7ff675232640 1 -- 172.21.15.92:0/3571835746 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff67013e360 msgr2=0x7ff6700cf290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:08.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:08.356+0000 7ff675232640 1 --2- 172.21.15.92:0/3571835746 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff67013e360 0x7ff6700cf290 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7ff670072770 tx=0x7ff65802b900 comp rx=0 tx=0).stop 2023-11-24T06:12:08.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:08.356+0000 7ff675232640 1 -- 172.21.15.92:0/3571835746 shutdown_connections 2023-11-24T06:12:08.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:08.356+0000 7ff675232640 1 --2- 172.21.15.92:0/3571835746 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff67015f9c0 0x7ff6700cf7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:08.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:08.356+0000 7ff675232640 1 --2- 172.21.15.92:0/3571835746 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff67013e360 0x7ff6700cf290 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:08.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:08.356+0000 7ff675232640 1 --2- 172.21.15.92:0/3571835746 >> v2:172.21.15.118:3300/0 conn(0x7ff67013df80 0x7ff67015bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:08.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:08.356+0000 7ff675232640 1 -- 172.21.15.92:0/3571835746 >> 172.21.15.92:0/3571835746 conn(0x7ff67006a800 msgr2=0x7ff67006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:08.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:08.357+0000 7ff675232640 1 -- 172.21.15.92:0/3571835746 shutdown_connections 2023-11-24T06:12:08.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:08.357+0000 7ff675232640 1 -- 172.21.15.92:0/3571835746 wait complete. 2023-11-24T06:12:08.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:08.358+0000 7ff675232640 1 Processor -- start 2023-11-24T06:12:08.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:08.358+0000 7ff675232640 1 -- start start 2023-11-24T06:12:08.363 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:08.359+0000 7ff675232640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff67013df80 0x7ff6700adda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:08.363 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:08.359+0000 7ff675232640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff67013e360 0x7ff67015d860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:08.363 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:08.359+0000 7ff675232640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff67015f9c0 0x7ff67015dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:08.363 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:08.359+0000 7ff675232640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6700b06d0 con 0x7ff67013df80 2023-11-24T06:12:08.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:08.359+0000 7ff675232640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6701f1590 con 0x7ff67015f9c0 2023-11-24T06:12:08.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:08.359+0000 7ff675232640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6701f1bd0 con 0x7ff67013e360 2023-11-24T06:12:08.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:08.360+0000 7ff66ed9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff67013df80 0x7ff6700adda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:08.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:08.360+0000 7ff66ed9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff67013df80 0x7ff6700adda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46974/0 (socket says 172.21.15.92:46974) 2023-11-24T06:12:08.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:08.360+0000 7ff66ed9d640 1 -- 172.21.15.92:0/1372723714 learned_addr learned my addr 172.21.15.92:0/1372723714 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:12:08.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:08.360+0000 7ff66ed9d640 1 -- 172.21.15.92:0/1372723714 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff67013e360 msgr2=0x7ff67015d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:08.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:08.360+0000 7ff66ed9d640 1 --2- 172.21.15.92:0/1372723714 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff67013e360 0x7ff67015d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:08.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:08.360+0000 7ff66f59e640 1 --2- 172.21.15.92:0/1372723714 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff67015f9c0 0x7ff67015dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:08.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:08.360+0000 7ff66ed9d640 1 -- 172.21.15.92:0/1372723714 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff67015f9c0 msgr2=0x7ff67015dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:08.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:08.360+0000 7ff66ed9d640 1 --2- 172.21.15.92:0/1372723714 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff67015f9c0 0x7ff67015dda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:08.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:08.360+0000 7ff66ed9d640 1 -- 172.21.15.92:0/1372723714 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6580094d0 con 0x7ff67013df80 2023-11-24T06:12:08.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:08.360+0000 7ff66f59e640 1 --2- 172.21.15.92:0/1372723714 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff67015f9c0 0x7ff67015dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:12:08.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:08.360+0000 7ff66ed9d640 1 --2- 172.21.15.92:0/1372723714 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff67013df80 0x7ff6700adda0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7ff66000d5f0 tx=0x7ff66000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:08.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:08.361+0000 7ff6677fe640 1 -- 172.21.15.92:0/1372723714 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6600125f0 con 0x7ff67013df80 2023-11-24T06:12:08.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:08.361+0000 7ff6677fe640 1 -- 172.21.15.92:0/1372723714 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff660012750 con 0x7ff67013df80 2023-11-24T06:12:08.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:08.361+0000 7ff675232640 1 -- 172.21.15.92:0/1372723714 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6701f1eb0 con 0x7ff67013df80 2023-11-24T06:12:08.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:08.361+0000 7ff6677fe640 1 -- 172.21.15.92:0/1372723714 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff660012b00 con 0x7ff67013df80 2023-11-24T06:12:08.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:08.361+0000 7ff675232640 1 -- 172.21.15.92:0/1372723714 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6700ae2e0 con 0x7ff67013df80 2023-11-24T06:12:08.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:08.362+0000 7ff675232640 1 -- 172.21.15.92:0/1372723714 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6700675d0 con 0x7ff67013df80 2023-11-24T06:12:08.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:08.363+0000 7ff6677fe640 1 -- 172.21.15.92:0/1372723714 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ff660012c60 con 0x7ff67013df80 2023-11-24T06:12:08.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:08.364+0000 7ff6677fe640 1 --2- 172.21.15.92:0/1372723714 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7ff65006ae40 0x7ff65006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:08.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:08.364+0000 7ff6677fe640 1 -- 172.21.15.92:0/1372723714 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5795+0+0 (secure 0 0 0) 0x7ff660089b50 con 0x7ff67013df80 2023-11-24T06:12:08.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:08.364+0000 7ff667fff640 1 --2- 172.21.15.92:0/1372723714 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7ff65006ae40 0x7ff65006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:08.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:08.365+0000 7ff667fff640 1 --2- 172.21.15.92:0/1372723714 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7ff65006ae40 0x7ff65006d300 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7ff658036000 tx=0x7ff658030000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:08.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:08.368+0000 7ff6677fe640 1 -- 172.21.15.92:0/1372723714 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff660054b90 con 0x7ff67013df80 2023-11-24T06:12:08.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:08.558+0000 7ff675232640 1 -- 172.21.15.92:0/1372723714 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff67006df00 con 0x7ff67013df80 2023-11-24T06:12:08.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:08.559+0000 7ff6677fe640 1 -- 172.21.15.92:0/1372723714 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v40) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff6600581b0 con 0x7ff67013df80 2023-11-24T06:12:08.562 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 64 2023-11-24T06:12:08.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:08.562+0000 7ff675232640 1 -- 172.21.15.92:0/1372723714 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7ff65006ae40 msgr2=0x7ff65006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:08.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:08.562+0000 7ff675232640 1 --2- 172.21.15.92:0/1372723714 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7ff65006ae40 0x7ff65006d300 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7ff658036000 tx=0x7ff658030000 comp rx=0 tx=0).stop 2023-11-24T06:12:08.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:08.562+0000 7ff675232640 1 -- 172.21.15.92:0/1372723714 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff67013df80 msgr2=0x7ff6700adda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:08.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:08.562+0000 7ff675232640 1 --2- 172.21.15.92:0/1372723714 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff67013df80 0x7ff6700adda0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7ff66000d5f0 tx=0x7ff66000dab0 comp rx=0 tx=0).stop 2023-11-24T06:12:08.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:08.562+0000 7ff675232640 1 -- 172.21.15.92:0/1372723714 shutdown_connections 2023-11-24T06:12:08.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:08.562+0000 7ff675232640 1 --2- 172.21.15.92:0/1372723714 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7ff65006ae40 0x7ff65006d300 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:08.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:08.562+0000 7ff675232640 1 --2- 172.21.15.92:0/1372723714 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff67015f9c0 0x7ff67015dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:08.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:08.562+0000 7ff675232640 1 --2- 172.21.15.92:0/1372723714 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff67013e360 0x7ff67015d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:08.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:08.562+0000 7ff675232640 1 --2- 172.21.15.92:0/1372723714 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff67013df80 0x7ff6700adda0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:08.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:08.562+0000 7ff675232640 1 -- 172.21.15.92:0/1372723714 >> 172.21.15.92:0/1372723714 conn(0x7ff67006a800 msgr2=0x7ff670155400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:08.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:08.562+0000 7ff675232640 1 -- 172.21.15.92:0/1372723714 shutdown_connections 2023-11-24T06:12:08.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:08.562+0000 7ff675232640 1 -- 172.21.15.92:0/1372723714 wait complete. 2023-11-24T06:12:08.584 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-24T06:12:08.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:08.787+0000 7ff42f036640 1 Processor -- start 2023-11-24T06:12:08.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:08.787+0000 7ff42f036640 1 -- start start 2023-11-24T06:12:08.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:08.789+0000 7ff42f036640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7ff42813df80 0x7ff42815bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:08.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:08.789+0000 7ff42f036640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff42806c470 con 0x7ff4280cf9a0 2023-11-24T06:12:08.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:08.789+0000 7ff42f036640 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff42806c750 con 0x7ff42813e360 2023-11-24T06:12:08.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:08.789+0000 7ff42f036640 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff42806ca30 con 0x7ff42813df80 2023-11-24T06:12:08.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:08.789+0000 7ff42d5d3640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7ff4280cf9a0 0x7ff4280cd840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:46808/0 (socket says 172.21.15.92:46808) 2023-11-24T06:12:08.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:08.789+0000 7ff42d5d3640 1 -- 172.21.15.92:0/4061688882 learned_addr learned my addr 172.21.15.92:0/4061688882 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-24T06:12:08.794 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:08.790+0000 7ff42cdd2640 1 --2- 172.21.15.92:0/4061688882 >> v2:172.21.15.118:3300/0 conn(0x7ff42813df80 0x7ff42815bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:08.794 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:08.790+0000 7ff4277fe640 1 -- 172.21.15.92:0/4061688882 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1044158430 0 0) 0x7ff42806c470 con 0x7ff4280cf9a0 2023-11-24T06:12:08.794 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:08.790+0000 7ff4277fe640 1 -- 172.21.15.92:0/4061688882 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff418003520 con 0x7ff4280cf9a0 2023-11-24T06:12:08.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:08.790+0000 7ff4277fe640 1 -- 172.21.15.92:0/4061688882 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3046252287 0 0) 0x7ff418003520 con 0x7ff4280cf9a0 2023-11-24T06:12:08.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:08.790+0000 7ff4277fe640 1 -- 172.21.15.92:0/4061688882 >> v2:172.21.15.118:3300/0 conn(0x7ff42813df80 msgr2=0x7ff42815bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:08.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:08.790+0000 7ff4277fe640 1 --2- 172.21.15.92:0/4061688882 >> v2:172.21.15.118:3300/0 conn(0x7ff42813df80 0x7ff42815bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:08.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:08.790+0000 7ff4277fe640 1 -- 172.21.15.92:0/4061688882 >> v1:172.21.15.118:6789/0 conn(0x7ff42813e360 legacy=0x7ff4280cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:08.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:08.790+0000 7ff4277fe640 1 -- 172.21.15.92:0/4061688882 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff42806cd10 con 0x7ff4280cf9a0 2023-11-24T06:12:08.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:08.790+0000 7ff4277fe640 1 -- 172.21.15.92:0/4061688882 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2607661350 0 0) 0x7ff4200019f0 con 0x7ff4280cf9a0 2023-11-24T06:12:08.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:08.790+0000 7ff4277fe640 1 -- 172.21.15.92:0/4061688882 >> v1:172.21.15.92:6789/0 conn(0x7ff4280cf9a0 legacy=0x7ff4280cd840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:08.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:08.790+0000 7ff4277fe640 1 --2- 172.21.15.92:0/4061688882 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff418003970 0x7ff418003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:08.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:08.790+0000 7ff4277fe640 1 --2- 172.21.15.92:0/4061688882 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff4180047c0 0x7ff418004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:08.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:08.790+0000 7ff4277fe640 1 --2- 172.21.15.92:0/4061688882 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff418005180 0x7ff41800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:08.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:08.790+0000 7ff4277fe640 1 -- 172.21.15.92:0/4061688882 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff42806ca30 con 0x7ff4180047c0 2023-11-24T06:12:08.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:08.790+0000 7ff4277fe640 1 -- 172.21.15.92:0/4061688882 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff41800c020 con 0x7ff418005180 2023-11-24T06:12:08.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:08.790+0000 7ff4277fe640 1 -- 172.21.15.92:0/4061688882 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff41800c1e0 con 0x7ff418003970 2023-11-24T06:12:08.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:08.791+0000 7ff427fff640 1 --2- 172.21.15.92:0/4061688882 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff4180047c0 0x7ff418004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:08.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:08.791+0000 7ff427fff640 1 -- 172.21.15.92:0/4061688882 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff418003970 msgr2=0x7ff418003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:08.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:08.791+0000 7ff427fff640 1 --2- 172.21.15.92:0/4061688882 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff418003970 0x7ff418003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:08.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:08.791+0000 7ff427fff640 1 -- 172.21.15.92:0/4061688882 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff418005180 msgr2=0x7ff41800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:08.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:08.791+0000 7ff427fff640 1 --2- 172.21.15.92:0/4061688882 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff418005180 0x7ff41800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:08.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:08.791+0000 7ff427fff640 1 -- 172.21.15.92:0/4061688882 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff418003520 con 0x7ff4180047c0 2023-11-24T06:12:08.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:08.791+0000 7ff42d5d3640 1 -- 172.21.15.92:0/4061688882 reap_dead start 2023-11-24T06:12:08.797 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:08.792+0000 7ff427fff640 1 --2- 172.21.15.92:0/4061688882 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff4180047c0 0x7ff418004c40 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7ff414001720 tx=0x7ff4180057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fca876bd3d9d8ab server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:08.797 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:08.792+0000 7ff4277fe640 1 -- 172.21.15.92:0/4061688882 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4140052d0 con 0x7ff4180047c0 2023-11-24T06:12:08.797 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:08.792+0000 7ff4277fe640 1 -- 172.21.15.92:0/4061688882 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff41402e050 con 0x7ff4180047c0 2023-11-24T06:12:08.797 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:08.792+0000 7ff4277fe640 1 -- 172.21.15.92:0/4061688882 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff414003830 con 0x7ff4180047c0 2023-11-24T06:12:08.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:08.793+0000 7ff42f036640 1 -- 172.21.15.92:0/4061688882 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff4180047c0 msgr2=0x7ff418004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:08.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:08.793+0000 7ff42f036640 1 --2- 172.21.15.92:0/4061688882 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff4180047c0 0x7ff418004c40 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7ff414001720 tx=0x7ff4180057f0 comp rx=0 tx=0).stop 2023-11-24T06:12:08.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:08.793+0000 7ff42f036640 1 -- 172.21.15.92:0/4061688882 shutdown_connections 2023-11-24T06:12:08.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:08.793+0000 7ff42f036640 1 --2- 172.21.15.92:0/4061688882 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff4180047c0 0x7ff418004c40 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:08.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:08.793+0000 7ff42f036640 1 -- 172.21.15.92:0/4061688882 >> 172.21.15.92:0/4061688882 conn(0x7ff42806a800 msgr2=0x7ff42806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:08.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:08.793+0000 7ff42f036640 1 -- 172.21.15.92:0/4061688882 shutdown_connections 2023-11-24T06:12:08.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:08.793+0000 7ff42f036640 1 -- 172.21.15.92:0/4061688882 wait complete. 2023-11-24T06:12:08.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:08.795+0000 7ff42f036640 1 Processor -- start 2023-11-24T06:12:08.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:08.795+0000 7ff42f036640 1 -- start start 2023-11-24T06:12:08.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:08.796+0000 7ff42f036640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff4180047c0 0x7ff428156c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:08.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:08.796+0000 7ff42f036640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff428153cc0 0x7ff42813d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:08.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:08.796+0000 7ff42f036640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff4280ad750 0x7ff4280adbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:08.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:08.796+0000 7ff42f036640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff42813de10 con 0x7ff4180047c0 2023-11-24T06:12:08.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:08.796+0000 7ff42f036640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4280ae110 con 0x7ff428153cc0 2023-11-24T06:12:08.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:08.796+0000 7ff42f036640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4281553a0 con 0x7ff4280ad750 2023-11-24T06:12:08.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:08.796+0000 7ff42cdd2640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff4180047c0 0x7ff428156c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:08.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:08.796+0000 7ff42cdd2640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff4180047c0 0x7ff428156c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46990/0 (socket says 172.21.15.92:46990) 2023-11-24T06:12:08.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:08.796+0000 7ff42cdd2640 1 -- 172.21.15.92:0/2303773918 learned_addr learned my addr 172.21.15.92:0/2303773918 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:12:08.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:08.797+0000 7ff42cdd2640 1 -- 172.21.15.92:0/2303773918 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff4280ad750 msgr2=0x7ff4280adbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:08.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:08.797+0000 7ff42cdd2640 1 --2- 172.21.15.92:0/2303773918 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff4280ad750 0x7ff4280adbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:08.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:08.797+0000 7ff42cdd2640 1 -- 172.21.15.92:0/2303773918 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff428153cc0 msgr2=0x7ff42813d8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:08.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:08.797+0000 7ff42cdd2640 1 --2- 172.21.15.92:0/2303773918 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff428153cc0 0x7ff42813d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:08.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:08.797+0000 7ff42cdd2640 1 -- 172.21.15.92:0/2303773918 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4281507e0 con 0x7ff4180047c0 2023-11-24T06:12:08.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:08.797+0000 7ff42cdd2640 1 --2- 172.21.15.92:0/2303773918 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff4180047c0 0x7ff428156c40 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7ff41c00e770 tx=0x7ff41c00ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:08.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:08.797+0000 7ff4257fa640 1 -- 172.21.15.92:0/2303773918 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff41c00c730 con 0x7ff4180047c0 2023-11-24T06:12:08.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:08.797+0000 7ff4257fa640 1 -- 172.21.15.92:0/2303773918 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff41c00c890 con 0x7ff4180047c0 2023-11-24T06:12:08.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:08.797+0000 7ff4257fa640 1 -- 172.21.15.92:0/2303773918 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff41c00cc40 con 0x7ff4180047c0 2023-11-24T06:12:08.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:08.797+0000 7ff42f036640 1 -- 172.21.15.92:0/2303773918 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff428150ac0 con 0x7ff4180047c0 2023-11-24T06:12:08.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:08.797+0000 7ff42f036640 1 -- 172.21.15.92:0/2303773918 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff428151080 con 0x7ff4180047c0 2023-11-24T06:12:08.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:08.799+0000 7ff42f036640 1 -- 172.21.15.92:0/2303773918 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4280675d0 con 0x7ff4180047c0 2023-11-24T06:12:08.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:08.799+0000 7ff4257fa640 1 -- 172.21.15.92:0/2303773918 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ff41c013470 con 0x7ff4180047c0 2023-11-24T06:12:08.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:08.800+0000 7ff4257fa640 1 --2- 172.21.15.92:0/2303773918 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7ff40c06ae70 0x7ff40c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:08.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:08.801+0000 7ff4257fa640 1 -- 172.21.15.92:0/2303773918 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5795+0+0 (secure 0 0 0) 0x7ff41c0896a0 con 0x7ff4180047c0 2023-11-24T06:12:08.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:08.801+0000 7ff427fff640 1 --2- 172.21.15.92:0/2303773918 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7ff40c06ae70 0x7ff40c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:08.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:08.801+0000 7ff427fff640 1 --2- 172.21.15.92:0/2303773918 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7ff40c06ae70 0x7ff40c06d330 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7ff414033000 tx=0x7ff41402f000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:08.808 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:08.805+0000 7ff4257fa640 1 -- 172.21.15.92:0/2303773918 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff41c0546f0 con 0x7ff4180047c0 2023-11-24T06:12:09.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:09.004+0000 7ff42f036640 1 -- 172.21.15.92:0/2303773918 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7ff42806df00 con 0x7ff4180047c0 2023-11-24T06:12:09.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:09.005+0000 7ff4257fa640 1 -- 172.21.15.92:0/2303773918 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v40) v1 ==== 80+0+122 (secure 0 0 0) 0x7ff41c017090 con 0x7ff4180047c0 2023-11-24T06:12:09.009 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.15.92:6836/3889219158 2023-11-25T06:12:03.443855+0000 2023-11-24T06:12:09.009 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.15.92:6834/3889219158 2023-11-25T06:12:03.443855+0000 2023-11-24T06:12:09.009 INFO:teuthology.orchestra.run.smithi092.stderr:listed 2 entries 2023-11-24T06:12:09.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:09.007+0000 7ff42f036640 1 -- 172.21.15.92:0/2303773918 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7ff40c06ae70 msgr2=0x7ff40c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:09.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:09.007+0000 7ff42f036640 1 --2- 172.21.15.92:0/2303773918 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7ff40c06ae70 0x7ff40c06d330 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7ff414033000 tx=0x7ff41402f000 comp rx=0 tx=0).stop 2023-11-24T06:12:09.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:09.007+0000 7ff42f036640 1 -- 172.21.15.92:0/2303773918 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff4180047c0 msgr2=0x7ff428156c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:09.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:09.007+0000 7ff42f036640 1 --2- 172.21.15.92:0/2303773918 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff4180047c0 0x7ff428156c40 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7ff41c00e770 tx=0x7ff41c00ec30 comp rx=0 tx=0).stop 2023-11-24T06:12:09.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:09.008+0000 7ff42f036640 1 -- 172.21.15.92:0/2303773918 shutdown_connections 2023-11-24T06:12:09.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:09.008+0000 7ff42f036640 1 --2- 172.21.15.92:0/2303773918 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7ff40c06ae70 0x7ff40c06d330 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:09.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:09.008+0000 7ff42f036640 1 --2- 172.21.15.92:0/2303773918 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff4280ad750 0x7ff4280adbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:09.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:09.008+0000 7ff42f036640 1 --2- 172.21.15.92:0/2303773918 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff428153cc0 0x7ff42813d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:09.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:09.008+0000 7ff42f036640 1 --2- 172.21.15.92:0/2303773918 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff4180047c0 0x7ff428156c40 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:09.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:09.008+0000 7ff42f036640 1 -- 172.21.15.92:0/2303773918 >> 172.21.15.92:0/2303773918 conn(0x7ff42806a800 msgr2=0x7ff42815dbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:09.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:09.008+0000 7ff42f036640 1 -- 172.21.15.92:0/2303773918 shutdown_connections 2023-11-24T06:12:09.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:09.008+0000 7ff42f036640 1 -- 172.21.15.92:0/2303773918 wait complete. 2023-11-24T06:12:09.028 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi092.front.sepia.ceph.com... 2023-11-24T06:12:09.028 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:12:09.029 DEBUG:teuthology.orchestra.run.smithi092:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-11-24T06:12:09.117 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi092.stderr:ceph-fuse[102972]: fuse finished with error 0 and tester_r 0 2023-11-24T06:12:09.121 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi092.front.sepia.ceph.com 2023-11-24T06:12:09.122 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:12:09.122 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-24T06:12:09.270 DEBUG:teuthology.orchestra.run.smithi118:> set -ex 2023-11-24T06:12:09.270 DEBUG:teuthology.orchestra.run.smithi118:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:12:09.330 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:12:09.560 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:09.557+0000 7f2308ee7640 1 Processor -- start 2023-11-24T06:12:09.560 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:09.557+0000 7f2308ee7640 1 -- start start 2023-11-24T06:12:09.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:09.559+0000 7f2308ee7640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f23040ae5a0 0x7f230415e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:09.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:09.559+0000 7f2308ee7640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f23040b3ae0 0x7f230406cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:09.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:09.559+0000 7f2308ee7640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f230406d120 0x7f23040cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:09.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:09.559+0000 7f2308ee7640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2304071450 con 0x7f23040b3ae0 2023-11-24T06:12:09.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:09.559+0000 7f2308ee7640 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2304070c40 con 0x7f230406d120 2023-11-24T06:12:09.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:09.559+0000 7f2308ee7640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f230406c470 con 0x7f23040ae5a0 2023-11-24T06:12:09.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:09.559+0000 7f22fbfff640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f23040b3ae0 0x7f230406cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:09.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:09.559+0000 7f22fbfff640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f23040b3ae0 0x7f230406cbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:60390/0 (socket says 172.21.15.92:60390) 2023-11-24T06:12:09.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:09.559+0000 7f22fbfff640 1 -- 172.21.15.92:0/475773733 learned_addr learned my addr 172.21.15.92:0/475773733 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:12:09.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:09.560+0000 7f2302d9d640 1 --2- 172.21.15.92:0/475773733 >> v2:172.21.15.118:3300/0 conn(0x7f230406d120 0x7f23040cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:09.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:09.560+0000 7f230259c640 1 --2- 172.21.15.92:0/475773733 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f23040ae5a0 0x7f230415e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:09.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:09.560+0000 7f22fbfff640 1 -- 172.21.15.92:0/475773733 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f23040ae5a0 msgr2=0x7f230415e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:09.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:09.560+0000 7f22fbfff640 1 --2- 172.21.15.92:0/475773733 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f23040ae5a0 0x7f230415e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:09.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:09.560+0000 7f22fbfff640 1 -- 172.21.15.92:0/475773733 >> v2:172.21.15.118:3300/0 conn(0x7f230406d120 msgr2=0x7f23040cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:09.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:09.560+0000 7f22fbfff640 1 --2- 172.21.15.92:0/475773733 >> v2:172.21.15.118:3300/0 conn(0x7f230406d120 0x7f23040cf290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:09.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:09.560+0000 7f22fbfff640 1 -- 172.21.15.92:0/475773733 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f230406c750 con 0x7f23040b3ae0 2023-11-24T06:12:09.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:09.560+0000 7f2302d9d640 1 --2- 172.21.15.92:0/475773733 >> v2:172.21.15.118:3300/0 conn(0x7f230406d120 0x7f23040cf290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:12:09.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:09.560+0000 7f230259c640 1 --2- 172.21.15.92:0/475773733 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f23040ae5a0 0x7f230415e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:12:09.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:09.561+0000 7f22fbfff640 1 --2- 172.21.15.92:0/475773733 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f23040b3ae0 0x7f230406cbe0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f230406e230 tx=0x7f22ec02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7ff1c1fbb254b2d server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:09.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:09.561+0000 7f2301d9b640 1 -- 172.21.15.92:0/475773733 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22ec031870 con 0x7f23040b3ae0 2023-11-24T06:12:09.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:09.561+0000 7f2301d9b640 1 -- 172.21.15.92:0/475773733 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22ec0319d0 con 0x7f23040b3ae0 2023-11-24T06:12:09.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:09.561+0000 7f2301d9b640 1 -- 172.21.15.92:0/475773733 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22ec031d60 con 0x7f23040b3ae0 2023-11-24T06:12:09.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:09.562+0000 7f2308ee7640 1 -- 172.21.15.92:0/475773733 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f23040b3ae0 msgr2=0x7f230406cbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:09.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:09.562+0000 7f2308ee7640 1 --2- 172.21.15.92:0/475773733 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f23040b3ae0 0x7f230406cbe0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f230406e230 tx=0x7f22ec02b910 comp rx=0 tx=0).stop 2023-11-24T06:12:09.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:09.562+0000 7f2308ee7640 1 -- 172.21.15.92:0/475773733 shutdown_connections 2023-11-24T06:12:09.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:09.562+0000 7f2308ee7640 1 --2- 172.21.15.92:0/475773733 >> v2:172.21.15.118:3300/0 conn(0x7f230406d120 0x7f23040cf290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:09.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:09.562+0000 7f2308ee7640 1 --2- 172.21.15.92:0/475773733 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f23040b3ae0 0x7f230406cbe0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:09.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:09.562+0000 7f2308ee7640 1 --2- 172.21.15.92:0/475773733 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f23040ae5a0 0x7f230415e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:09.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:09.562+0000 7f2308ee7640 1 -- 172.21.15.92:0/475773733 >> 172.21.15.92:0/475773733 conn(0x7f230406a800 msgr2=0x7f230406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:09.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:09.562+0000 7f2308ee7640 1 -- 172.21.15.92:0/475773733 shutdown_connections 2023-11-24T06:12:09.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:09.562+0000 7f2308ee7640 1 -- 172.21.15.92:0/475773733 wait complete. 2023-11-24T06:12:09.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:09.564+0000 7f2308ee7640 1 Processor -- start 2023-11-24T06:12:09.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:09.564+0000 7f2308ee7640 1 -- start start 2023-11-24T06:12:09.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:09.565+0000 7f2308ee7640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f230406d120 0x7f23040cb910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:09.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:09.565+0000 7f2308ee7640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f23040ae5a0 0x7f2304155ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:09.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:09.565+0000 7f2308ee7640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f23040b3ae0 0x7f23040cc230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:09.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:09.565+0000 7f2308ee7640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23041572b0 con 0x7f23040ae5a0 2023-11-24T06:12:09.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:09.565+0000 7f2308ee7640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2304157590 con 0x7f230406d120 2023-11-24T06:12:09.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:09.565+0000 7f2308ee7640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2304157870 con 0x7f23040b3ae0 2023-11-24T06:12:09.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:09.565+0000 7f22fbfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f23040ae5a0 0x7f2304155ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:09.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:09.565+0000 7f22fbfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f23040ae5a0 0x7f2304155ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:60406/0 (socket says 172.21.15.92:60406) 2023-11-24T06:12:09.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:09.565+0000 7f22fbfff640 1 -- 172.21.15.92:0/1784382338 learned_addr learned my addr 172.21.15.92:0/1784382338 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:12:09.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:09.566+0000 7f22fbfff640 1 -- 172.21.15.92:0/1784382338 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f23040b3ae0 msgr2=0x7f23040cc230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:09.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:09.566+0000 7f2302d9d640 1 --2- 172.21.15.92:0/1784382338 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f23040b3ae0 0x7f23040cc230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:09.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:09.566+0000 7f230259c640 1 --2- 172.21.15.92:0/1784382338 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f230406d120 0x7f23040cb910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:09.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:09.566+0000 7f22fbfff640 1 --2- 172.21.15.92:0/1784382338 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f23040b3ae0 0x7f23040cc230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:09.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:09.566+0000 7f22fbfff640 1 -- 172.21.15.92:0/1784382338 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f230406d120 msgr2=0x7f23040cb910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:09.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:09.566+0000 7f22fbfff640 1 --2- 172.21.15.92:0/1784382338 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f230406d120 0x7f23040cb910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:09.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:09.566+0000 7f22fbfff640 1 -- 172.21.15.92:0/1784382338 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22ec0094d0 con 0x7f23040ae5a0 2023-11-24T06:12:09.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:09.566+0000 7f22fbfff640 1 --2- 172.21.15.92:0/1784382338 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f23040ae5a0 0x7f2304155ae0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f22ec000c00 tx=0x7f22ec0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:09.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:09.566+0000 7f22faffd640 1 -- 172.21.15.92:0/1784382338 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22ec031840 con 0x7f23040ae5a0 2023-11-24T06:12:09.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:09.566+0000 7f22faffd640 1 -- 172.21.15.92:0/1784382338 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22ec0443f0 con 0x7f23040ae5a0 2023-11-24T06:12:09.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:09.566+0000 7f2308ee7640 1 -- 172.21.15.92:0/1784382338 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2304157b50 con 0x7f23040ae5a0 2023-11-24T06:12:09.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:09.566+0000 7f22faffd640 1 -- 172.21.15.92:0/1784382338 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22ec0447a0 con 0x7f23040ae5a0 2023-11-24T06:12:09.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:09.567+0000 7f2308ee7640 1 -- 172.21.15.92:0/1784382338 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23040cbe50 con 0x7f23040ae5a0 2023-11-24T06:12:09.572 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:09.568+0000 7f2308ee7640 1 -- 172.21.15.92:0/1784382338 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23040675d0 con 0x7f23040ae5a0 2023-11-24T06:12:09.572 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:09.568+0000 7f22faffd640 1 -- 172.21.15.92:0/1784382338 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f22ec04d450 con 0x7f23040ae5a0 2023-11-24T06:12:09.572 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:09.569+0000 7f22faffd640 1 --2- 172.21.15.92:0/1784382338 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f22e406ae40 0x7f22e406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:09.572 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:09.569+0000 7f22faffd640 1 -- 172.21.15.92:0/1784382338 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f22ec02d040 con 0x7f23040ae5a0 2023-11-24T06:12:09.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:09.570+0000 7f230259c640 1 --2- 172.21.15.92:0/1784382338 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f22e406ae40 0x7f22e406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:09.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:09.570+0000 7f230259c640 1 --2- 172.21.15.92:0/1784382338 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f22e406ae40 0x7f22e406d300 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f22f4008850 tx=0x7f22f4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:09.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:09.574+0000 7f22faffd640 1 -- 172.21.15.92:0/1784382338 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f22ec076e00 con 0x7f23040ae5a0 2023-11-24T06:12:09.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:09.765+0000 7f2308ee7640 1 -- 172.21.15.92:0/1784382338 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2304070800 con 0x7f23040ae5a0 2023-11-24T06:12:09.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:09.766+0000 7f22faffd640 1 -- 172.21.15.92:0/1784382338 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+16650 (secure 0 0 0) 0x7f22ec035020 con 0x7f23040ae5a0 2023-11-24T06:12:09.770 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-24T06:12:09.770 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":40,"fsid":"2d6dee5f-8aa3-470f-931a-50ade963b1cb","created":"2023-11-24T06:10:04.707204+0000","modified":"2023-11-24T06:12:06.456223+0000","last_up_change":"2023-11-24T06:10:17.598352+0000","last_in_change":"2023-11-24T06:10:09.040793+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:10:17.733098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:11:42.241681+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-11-24T06:11:43.522720+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ff37a9fb-135f-471f-b4b0-f7b5772006c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6803","nonce":20560714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6805","nonce":20560714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6809","nonce":20560714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6807","nonce":20560714}]},"public_addr":"172.21.15.92:6803/20560714","cluster_addr":"172.21.15.92:6805/20560714","heartbeat_back_addr":"172.21.15.92:6809/20560714","heartbeat_front_addr":"172.21.15.92:6807/20560714","state":["exists","up"]},{"osd":1,"uuid":"03cdd353-dcf1-4f38-94fe-a795137c06b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6819","nonce":2451239906}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6821","nonce":2451239906}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6825","nonce":2451239906}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6823","nonce":2451239906}]},"public_addr":"172.21.15.92:6819/2451239906","cluster_addr":"172.21.15.92:6821/2451239906","heartbeat_back_addr":"172.21.15.92:6825/2451239906","heartbeat_front_addr":"172.21.15.92:6823/2451239906","state":["exists","up"]},{"osd":2,"uuid":"a4a20d80-5d57-484b-becd-9aa8070dd3aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6811","nonce":1124979277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6813","nonce":1124979277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6817","nonce":1124979277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6815","nonce":1124979277}]},"public_addr":"172.21.15.92:6811/1124979277","cluster_addr":"172.21.15.92:6813/1124979277","heartbeat_back_addr":"172.21.15.92:6817/1124979277","heartbeat_front_addr":"172.21.15.92:6815/1124979277","state":["exists","up"]},{"osd":3,"uuid":"4648d7ce-9b29-4583-9425-4ba8fd2b1903","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6827","nonce":1734115782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6829","nonce":1734115782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6833","nonce":1734115782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6831","nonce":1734115782}]},"public_addr":"172.21.15.92:6827/1734115782","cluster_addr":"172.21.15.92:6829/1734115782","heartbeat_back_addr":"172.21.15.92:6833/1734115782","heartbeat_front_addr":"172.21.15.92:6831/1734115782","state":["exists","up"]},{"osd":4,"uuid":"3bc9d7eb-c6fa-4fcf-a3ec-ade604e05aac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6819","nonce":3425937229}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6822","nonce":3425937229}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6830","nonce":3425937229}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6826","nonce":3425937229}]},"public_addr":"172.21.15.118:6819/3425937229","cluster_addr":"172.21.15.118:6822/3425937229","heartbeat_back_addr":"172.21.15.118:6830/3425937229","heartbeat_front_addr":"172.21.15.118:6826/3425937229","state":["exists","up"]},{"osd":5,"uuid":"dce9ba6d-59ed-46d1-84fc-a84b13de10c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6801","nonce":827455984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6803","nonce":827455984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6807","nonce":827455984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6805","nonce":827455984}]},"public_addr":"172.21.15.118:6801/827455984","cluster_addr":"172.21.15.118:6803/827455984","heartbeat_back_addr":"172.21.15.118:6807/827455984","heartbeat_front_addr":"172.21.15.118:6805/827455984","state":["exists","up"]},{"osd":6,"uuid":"82aec504-ae58-4709-8d30-46f2b12081b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6809","nonce":2946986141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6811","nonce":2946986141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6815","nonce":2946986141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6813","nonce":2946986141}]},"public_addr":"172.21.15.118:6809/2946986141","cluster_addr":"172.21.15.118:6811/2946986141","heartbeat_back_addr":"172.21.15.118:6815/2946986141","heartbeat_front_addr":"172.21.15.118:6813/2946986141","state":["exists","up"]},{"osd":7,"uuid":"4ed4c624-c88f-4d35-bdb1-bf209b5dffb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6817","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6818","nonce":48122059}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6821","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6823","nonce":48122059}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6829","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6831","nonce":48122059}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6825","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6827","nonce":48122059}]},"public_addr":"172.21.15.118:6818/48122059","cluster_addr":"172.21.15.118:6823/48122059","heartbeat_back_addr":"172.21.15.118:6831/48122059","heartbeat_front_addr":"172.21.15.118:6827/48122059","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.536664+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.368324+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.271201+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.342971+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.621832+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.387110+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.632946+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.416032+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.92:6836/3889219158":"2023-11-25T06:12:03.443855+0000","172.21.15.92:6834/3889219158":"2023-11-25T06:12:03.443855+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:12:09.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:09.769+0000 7f2308ee7640 1 -- 172.21.15.92:0/1784382338 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f22e406ae40 msgr2=0x7f22e406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:09.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:09.769+0000 7f2308ee7640 1 --2- 172.21.15.92:0/1784382338 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f22e406ae40 0x7f22e406d300 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f22f4008850 tx=0x7f22f4008040 comp rx=0 tx=0).stop 2023-11-24T06:12:09.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:09.769+0000 7f2308ee7640 1 -- 172.21.15.92:0/1784382338 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f23040ae5a0 msgr2=0x7f2304155ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:09.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:09.769+0000 7f2308ee7640 1 --2- 172.21.15.92:0/1784382338 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f23040ae5a0 0x7f2304155ae0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f22ec000c00 tx=0x7f22ec0046c0 comp rx=0 tx=0).stop 2023-11-24T06:12:09.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:09.769+0000 7f2308ee7640 1 -- 172.21.15.92:0/1784382338 shutdown_connections 2023-11-24T06:12:09.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:09.769+0000 7f2308ee7640 1 --2- 172.21.15.92:0/1784382338 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f22e406ae40 0x7f22e406d300 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:09.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:09.769+0000 7f2308ee7640 1 --2- 172.21.15.92:0/1784382338 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f23040b3ae0 0x7f23040cc230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:09.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:09.769+0000 7f2308ee7640 1 --2- 172.21.15.92:0/1784382338 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f23040ae5a0 0x7f2304155ae0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:09.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:09.769+0000 7f2308ee7640 1 --2- 172.21.15.92:0/1784382338 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f230406d120 0x7f23040cb910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:09.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:09.769+0000 7f2308ee7640 1 -- 172.21.15.92:0/1784382338 >> 172.21.15.92:0/1784382338 conn(0x7f230406a800 msgr2=0x7f230406abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:09.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:09.769+0000 7f2308ee7640 1 -- 172.21.15.92:0/1784382338 shutdown_connections 2023-11-24T06:12:09.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:09.770+0000 7f2308ee7640 1 -- 172.21.15.92:0/1784382338 wait complete. 2023-11-24T06:12:09.793 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T06:10:17.733098+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T06:11:42.241681+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '32', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T06:11:43.522720+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '33', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T06:12:09.793 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T06:12:10.006 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.002+0000 7f5b517ce640 1 Processor -- start 2023-11-24T06:12:10.006 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.003+0000 7f5b517ce640 1 -- start start 2023-11-24T06:12:10.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.005+0000 7f5b517ce640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5b4c15da10 0x7f5b4c072230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:10.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.005+0000 7f5b517ce640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f5b4c0b2bd0 0x7f5b4c0b3080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:10.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.005+0000 7f5b517ce640 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b4c071190 con 0x7f5b4c150760 2023-11-24T06:12:10.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.005+0000 7f5b517ce640 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b4c071470 con 0x7f5b4c0b2bd0 2023-11-24T06:12:10.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.005+0000 7f5b517ce640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b4c071750 con 0x7f5b4c15da10 2023-11-24T06:12:10.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.005+0000 7f5b43fff640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5b4c15da10 0x7f5b4c072230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:10.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.005+0000 7f5b4b7fe640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f5b4c0b2bd0 0x7f5b4c0b3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:10.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.005+0000 7f5b43fff640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5b4c15da10 0x7f5b4c072230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3301/0 says I am v2:172.21.15.92:37788/0 (socket says 172.21.15.92:37788) 2023-11-24T06:12:10.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.005+0000 7f5b43fff640 1 -- 172.21.15.92:0/160394854 learned_addr learned my addr 172.21.15.92:0/160394854 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:12:10.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.006+0000 7f5b43fff640 1 -- 172.21.15.92:0/160394854 >> v2:172.21.15.118:3300/0 conn(0x7f5b4c0b2bd0 msgr2=0x7f5b4c0b3080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:10.010 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.006+0000 7f5b43fff640 1 --2- 172.21.15.92:0/160394854 >> v2:172.21.15.118:3300/0 conn(0x7f5b4c0b2bd0 0x7f5b4c0b3080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:10.010 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.006+0000 7f5b43fff640 1 -- 172.21.15.92:0/160394854 >> v1:172.21.15.118:6789/0 conn(0x7f5b4c150760 legacy=0x7f5b4c15d300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:10.010 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.006+0000 7f5b43fff640 1 -- 172.21.15.92:0/160394854 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b4c071a30 con 0x7f5b4c15da10 2023-11-24T06:12:10.010 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.008+0000 7f5b43fff640 1 --2- 172.21.15.92:0/160394854 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5b4c15da10 0x7f5b4c072230 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f5b3400bb00 tx=0x7f5b3402df30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5c98d2a077f66979 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:10.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.008+0000 7f5b4a7fc640 1 -- 172.21.15.92:0/160394854 <== mon.2 v2:172.21.15.118:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b3402ece0 con 0x7f5b4c15da10 2023-11-24T06:12:10.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.008+0000 7f5b4a7fc640 1 -- 172.21.15.92:0/160394854 <== mon.2 v2:172.21.15.118:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b34032070 con 0x7f5b4c15da10 2023-11-24T06:12:10.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.008+0000 7f5b4a7fc640 1 -- 172.21.15.92:0/160394854 <== mon.2 v2:172.21.15.118:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b34038620 con 0x7f5b4c15da10 2023-11-24T06:12:10.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.009+0000 7f5b517ce640 1 -- 172.21.15.92:0/160394854 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5b4c15da10 msgr2=0x7f5b4c072230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:10.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.009+0000 7f5b517ce640 1 --2- 172.21.15.92:0/160394854 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5b4c15da10 0x7f5b4c072230 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f5b3400bb00 tx=0x7f5b3402df30 comp rx=0 tx=0).stop 2023-11-24T06:12:10.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.009+0000 7f5b517ce640 1 -- 172.21.15.92:0/160394854 shutdown_connections 2023-11-24T06:12:10.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.009+0000 7f5b517ce640 1 --2- 172.21.15.92:0/160394854 >> v2:172.21.15.118:3300/0 conn(0x7f5b4c0b2bd0 0x7f5b4c0b3080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:10.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.009+0000 7f5b517ce640 1 --2- 172.21.15.92:0/160394854 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5b4c15da10 0x7f5b4c072230 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:10.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.009+0000 7f5b517ce640 1 -- 172.21.15.92:0/160394854 >> 172.21.15.92:0/160394854 conn(0x7f5b4c06a800 msgr2=0x7f5b4c06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:10.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.009+0000 7f5b517ce640 1 -- 172.21.15.92:0/160394854 shutdown_connections 2023-11-24T06:12:10.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.009+0000 7f5b517ce640 1 -- 172.21.15.92:0/160394854 wait complete. 2023-11-24T06:12:10.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.011+0000 7f5b517ce640 1 Processor -- start 2023-11-24T06:12:10.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.011+0000 7f5b517ce640 1 -- start start 2023-11-24T06:12:10.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.012+0000 7f5b517ce640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5b4c0b2bd0 0x7f5b4c14fa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:10.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.012+0000 7f5b517ce640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5b4c150760 0x7f5b4c158b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:10.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.012+0000 7f5b517ce640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5b4c15da10 0x7f5b4c0ac8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:10.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.012+0000 7f5b517ce640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b4c14ffa0 con 0x7f5b4c15da10 2023-11-24T06:12:10.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.012+0000 7f5b517ce640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b4c1f3380 con 0x7f5b4c0b2bd0 2023-11-24T06:12:10.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.012+0000 7f5b517ce640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b4c1590a0 con 0x7f5b4c150760 2023-11-24T06:12:10.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.012+0000 7f5b4b7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5b4c15da10 0x7f5b4c0ac8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:10.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.012+0000 7f5b4b7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5b4c15da10 0x7f5b4c0ac8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:60410/0 (socket says 172.21.15.92:60410) 2023-11-24T06:12:10.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.012+0000 7f5b4b7fe640 1 -- 172.21.15.92:0/3247480878 learned_addr learned my addr 172.21.15.92:0/3247480878 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:12:10.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.013+0000 7f5b4affd640 1 --2- 172.21.15.92:0/3247480878 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5b4c0b2bd0 0x7f5b4c14fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:10.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.013+0000 7f5b43fff640 1 --2- 172.21.15.92:0/3247480878 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5b4c150760 0x7f5b4c158b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:10.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.013+0000 7f5b4b7fe640 1 -- 172.21.15.92:0/3247480878 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5b4c150760 msgr2=0x7f5b4c158b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:10.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.013+0000 7f5b4b7fe640 1 --2- 172.21.15.92:0/3247480878 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5b4c150760 0x7f5b4c158b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:10.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.013+0000 7f5b4b7fe640 1 -- 172.21.15.92:0/3247480878 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5b4c0b2bd0 msgr2=0x7f5b4c14fa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:10.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.013+0000 7f5b4b7fe640 1 --2- 172.21.15.92:0/3247480878 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5b4c0b2bd0 0x7f5b4c14fa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:10.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.013+0000 7f5b4b7fe640 1 -- 172.21.15.92:0/3247480878 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b3400b760 con 0x7f5b4c15da10 2023-11-24T06:12:10.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.013+0000 7f5b4affd640 1 --2- 172.21.15.92:0/3247480878 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5b4c0b2bd0 0x7f5b4c14fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:12:10.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.013+0000 7f5b4b7fe640 1 --2- 172.21.15.92:0/3247480878 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5b4c15da10 0x7f5b4c0ac8d0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f5b4400efc0 tx=0x7f5b4400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:10.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.013+0000 7f5b437fe640 1 -- 172.21.15.92:0/3247480878 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b44014480 con 0x7f5b4c15da10 2023-11-24T06:12:10.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.013+0000 7f5b437fe640 1 -- 172.21.15.92:0/3247480878 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b4400f040 con 0x7f5b4c15da10 2023-11-24T06:12:10.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.014+0000 7f5b517ce640 1 -- 172.21.15.92:0/3247480878 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b4c14cbf0 con 0x7f5b4c15da10 2023-11-24T06:12:10.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.014+0000 7f5b437fe640 1 -- 172.21.15.92:0/3247480878 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b44014820 con 0x7f5b4c15da10 2023-11-24T06:12:10.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.014+0000 7f5b517ce640 1 -- 172.21.15.92:0/3247480878 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b4c14d1b0 con 0x7f5b4c15da10 2023-11-24T06:12:10.019 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.015+0000 7f5b517ce640 1 -- 172.21.15.92:0/3247480878 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b4c1f34f0 con 0x7f5b4c15da10 2023-11-24T06:12:10.019 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.016+0000 7f5b437fe640 1 -- 172.21.15.92:0/3247480878 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5b44014980 con 0x7f5b4c15da10 2023-11-24T06:12:10.020 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.017+0000 7f5b437fe640 1 --2- 172.21.15.92:0/3247480878 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f5b2c06ae40 0x7f5b2c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:10.020 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.017+0000 7f5b437fe640 1 -- 172.21.15.92:0/3247480878 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f5b4408a520 con 0x7f5b4c15da10 2023-11-24T06:12:10.020 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.017+0000 7f5b4affd640 1 --2- 172.21.15.92:0/3247480878 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f5b2c06ae40 0x7f5b2c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:10.020 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.018+0000 7f5b4affd640 1 --2- 172.21.15.92:0/3247480878 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f5b2c06ae40 0x7f5b2c06d300 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f5b3c000f60 tx=0x7f5b3c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:10.024 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.021+0000 7f5b437fe640 1 -- 172.21.15.92:0/3247480878 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5b44055560 con 0x7f5b4c15da10 2023-11-24T06:12:10.212 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.209+0000 7f5b517ce640 1 -- 172.21.15.92:0/3247480878 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5b4c070f00 con 0x7f5b4c15da10 2023-11-24T06:12:10.213 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.210+0000 7f5b437fe640 1 -- 172.21.15.92:0/3247480878 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v40) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5b44058b80 con 0x7f5b4c15da10 2023-11-24T06:12:10.213 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 1 2023-11-24T06:12:10.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.213+0000 7f5b517ce640 1 -- 172.21.15.92:0/3247480878 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f5b2c06ae40 msgr2=0x7f5b2c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:10.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.213+0000 7f5b517ce640 1 --2- 172.21.15.92:0/3247480878 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f5b2c06ae40 0x7f5b2c06d300 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f5b3c000f60 tx=0x7f5b3c009040 comp rx=0 tx=0).stop 2023-11-24T06:12:10.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.213+0000 7f5b517ce640 1 -- 172.21.15.92:0/3247480878 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5b4c15da10 msgr2=0x7f5b4c0ac8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:10.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.213+0000 7f5b517ce640 1 --2- 172.21.15.92:0/3247480878 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5b4c15da10 0x7f5b4c0ac8d0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f5b4400efc0 tx=0x7f5b4400c490 comp rx=0 tx=0).stop 2023-11-24T06:12:10.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.213+0000 7f5b517ce640 1 -- 172.21.15.92:0/3247480878 shutdown_connections 2023-11-24T06:12:10.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.213+0000 7f5b517ce640 1 --2- 172.21.15.92:0/3247480878 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f5b2c06ae40 0x7f5b2c06d300 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:10.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.213+0000 7f5b517ce640 1 --2- 172.21.15.92:0/3247480878 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5b4c15da10 0x7f5b4c0ac8d0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:10.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.213+0000 7f5b517ce640 1 --2- 172.21.15.92:0/3247480878 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5b4c150760 0x7f5b4c158b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:10.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.213+0000 7f5b517ce640 1 --2- 172.21.15.92:0/3247480878 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5b4c0b2bd0 0x7f5b4c14fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:10.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.213+0000 7f5b517ce640 1 -- 172.21.15.92:0/3247480878 >> 172.21.15.92:0/3247480878 conn(0x7f5b4c06a800 msgr2=0x7f5b4c15a6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:10.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.213+0000 7f5b517ce640 1 -- 172.21.15.92:0/3247480878 shutdown_connections 2023-11-24T06:12:10.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.213+0000 7f5b517ce640 1 -- 172.21.15.92:0/3247480878 wait complete. 2023-11-24T06:12:10.236 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T06:12:10.445 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.442+0000 7f2381c6d640 1 Processor -- start 2023-11-24T06:12:10.445 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.442+0000 7f2381c6d640 1 -- start start 2023-11-24T06:12:10.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.444+0000 7f2381c6d640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f237c06c730 0x7f237c0b2e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:10.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.444+0000 7f2381c6d640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f237c070160 0x7f237c071970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:10.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.444+0000 7f2381c6d640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f237c06cb10 0x7f237c06cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:10.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.444+0000 7f2381c6d640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f237c0708d0 con 0x7f237c06cb10 2023-11-24T06:12:10.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.444+0000 7f2381c6d640 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f237c070bb0 con 0x7f237c06c730 2023-11-24T06:12:10.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.444+0000 7f2381c6d640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f237c070e90 con 0x7f237c070160 2023-11-24T06:12:10.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.444+0000 7f238146c640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f237c06cb10 0x7f237c06cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:10.448 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.444+0000 7f238146c640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f237c06cb10 0x7f237c06cfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:60420/0 (socket says 172.21.15.92:60420) 2023-11-24T06:12:10.448 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.444+0000 7f238146c640 1 -- 172.21.15.92:0/2733065168 learned_addr learned my addr 172.21.15.92:0/2733065168 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:12:10.448 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.444+0000 7f2380c6b640 1 --2- 172.21.15.92:0/2733065168 >> v2:172.21.15.118:3300/0 conn(0x7f237c06c730 0x7f237c0b2e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:10.448 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.444+0000 7f237bfff640 1 --2- 172.21.15.92:0/2733065168 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f237c070160 0x7f237c071970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:10.449 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.445+0000 7f238146c640 1 -- 172.21.15.92:0/2733065168 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f237c070160 msgr2=0x7f237c071970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:10.449 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.445+0000 7f238146c640 1 --2- 172.21.15.92:0/2733065168 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f237c070160 0x7f237c071970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:10.449 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.445+0000 7f238146c640 1 -- 172.21.15.92:0/2733065168 >> v2:172.21.15.118:3300/0 conn(0x7f237c06c730 msgr2=0x7f237c0b2e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:10.449 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.445+0000 7f238146c640 1 --2- 172.21.15.92:0/2733065168 >> v2:172.21.15.118:3300/0 conn(0x7f237c06c730 0x7f237c0b2e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:10.449 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.445+0000 7f238146c640 1 -- 172.21.15.92:0/2733065168 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f237c071170 con 0x7f237c06cb10 2023-11-24T06:12:10.450 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.446+0000 7f238146c640 1 --2- 172.21.15.92:0/2733065168 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f237c06cb10 0x7f237c06cfa0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f237c0727c0 tx=0x7f237402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8eacc2f9a5f25b1 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:10.450 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.446+0000 7f237b7fe640 1 -- 172.21.15.92:0/2733065168 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f237402ca70 con 0x7f237c06cb10 2023-11-24T06:12:10.450 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.446+0000 7f237b7fe640 1 -- 172.21.15.92:0/2733065168 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f237402cbd0 con 0x7f237c06cb10 2023-11-24T06:12:10.450 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.446+0000 7f237b7fe640 1 -- 172.21.15.92:0/2733065168 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23740343f0 con 0x7f237c06cb10 2023-11-24T06:12:10.450 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.447+0000 7f2381c6d640 1 -- 172.21.15.92:0/2733065168 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f237c06cb10 msgr2=0x7f237c06cfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:10.450 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.447+0000 7f2381c6d640 1 --2- 172.21.15.92:0/2733065168 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f237c06cb10 0x7f237c06cfa0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f237c0727c0 tx=0x7f237402bed0 comp rx=0 tx=0).stop 2023-11-24T06:12:10.450 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.447+0000 7f2381c6d640 1 -- 172.21.15.92:0/2733065168 shutdown_connections 2023-11-24T06:12:10.451 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.447+0000 7f2381c6d640 1 --2- 172.21.15.92:0/2733065168 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f237c06cb10 0x7f237c06cfa0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:10.451 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.447+0000 7f2381c6d640 1 --2- 172.21.15.92:0/2733065168 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f237c070160 0x7f237c071970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:10.451 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.447+0000 7f2381c6d640 1 --2- 172.21.15.92:0/2733065168 >> v2:172.21.15.118:3300/0 conn(0x7f237c06c730 0x7f237c0b2e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:10.451 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.447+0000 7f2381c6d640 1 -- 172.21.15.92:0/2733065168 >> 172.21.15.92:0/2733065168 conn(0x7f237c06a7d0 msgr2=0x7f237c06abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:10.451 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.447+0000 7f2381c6d640 1 -- 172.21.15.92:0/2733065168 shutdown_connections 2023-11-24T06:12:10.451 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.447+0000 7f2381c6d640 1 -- 172.21.15.92:0/2733065168 wait complete. 2023-11-24T06:12:10.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.449+0000 7f2381c6d640 1 Processor -- start 2023-11-24T06:12:10.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.449+0000 7f2381c6d640 1 -- start start 2023-11-24T06:12:10.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.450+0000 7f2381c6d640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f237c06c730 0x7f237c1fa2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:10.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.450+0000 7f2381c6d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f237c06cb10 0x7f237c145fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:10.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.450+0000 7f2381c6d640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f237c070160 0x7f237c0cd350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:10.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.450+0000 7f2381c6d640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f237c15aca0 con 0x7f237c06cb10 2023-11-24T06:12:10.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.450+0000 7f2381c6d640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f237c15a120 con 0x7f237c070160 2023-11-24T06:12:10.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.450+0000 7f2381c6d640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f237c15a400 con 0x7f237c06c730 2023-11-24T06:12:10.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.450+0000 7f237bfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f237c06cb10 0x7f237c145fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:10.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.450+0000 7f237bfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f237c06cb10 0x7f237c145fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:60430/0 (socket says 172.21.15.92:60430) 2023-11-24T06:12:10.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.450+0000 7f237bfff640 1 -- 172.21.15.92:0/1197061857 learned_addr learned my addr 172.21.15.92:0/1197061857 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:12:10.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.450+0000 7f2380c6b640 1 --2- 172.21.15.92:0/1197061857 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f237c06c730 0x7f237c1fa2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:10.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.450+0000 7f238146c640 1 --2- 172.21.15.92:0/1197061857 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f237c070160 0x7f237c0cd350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:10.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.450+0000 7f237bfff640 1 -- 172.21.15.92:0/1197061857 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f237c06c730 msgr2=0x7f237c1fa2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:10.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.450+0000 7f237bfff640 1 --2- 172.21.15.92:0/1197061857 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f237c06c730 0x7f237c1fa2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:10.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.450+0000 7f237bfff640 1 -- 172.21.15.92:0/1197061857 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f237c070160 msgr2=0x7f237c0cd350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:10.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.450+0000 7f237bfff640 1 --2- 172.21.15.92:0/1197061857 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f237c070160 0x7f237c0cd350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:10.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.450+0000 7f237bfff640 1 -- 172.21.15.92:0/1197061857 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2374009b30 con 0x7f237c06cb10 2023-11-24T06:12:10.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.451+0000 7f237bfff640 1 --2- 172.21.15.92:0/1197061857 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f237c06cb10 0x7f237c145fe0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f236800d800 tx=0x7f236800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:10.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.451+0000 7f23797fa640 1 -- 172.21.15.92:0/1197061857 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f236800b6c0 con 0x7f237c06cb10 2023-11-24T06:12:10.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.451+0000 7f23797fa640 1 -- 172.21.15.92:0/1197061857 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f236800b820 con 0x7f237c06cb10 2023-11-24T06:12:10.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.451+0000 7f23797fa640 1 -- 172.21.15.92:0/1197061857 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f236800bbb0 con 0x7f237c06cb10 2023-11-24T06:12:10.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.451+0000 7f2381c6d640 1 -- 172.21.15.92:0/1197061857 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f237c15a6e0 con 0x7f237c06cb10 2023-11-24T06:12:10.456 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.451+0000 7f2381c6d640 1 -- 172.21.15.92:0/1197061857 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f237c139510 con 0x7f237c06cb10 2023-11-24T06:12:10.456 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.452+0000 7f2381c6d640 1 -- 172.21.15.92:0/1197061857 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f237c067450 con 0x7f237c06cb10 2023-11-24T06:12:10.456 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.453+0000 7f23797fa640 1 -- 172.21.15.92:0/1197061857 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f236800bd10 con 0x7f237c06cb10 2023-11-24T06:12:10.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.454+0000 7f23797fa640 1 --2- 172.21.15.92:0/1197061857 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f236006ae40 0x7f236006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:10.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.454+0000 7f23797fa640 1 -- 172.21.15.92:0/1197061857 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f2368089be0 con 0x7f237c06cb10 2023-11-24T06:12:10.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.454+0000 7f2380c6b640 1 --2- 172.21.15.92:0/1197061857 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f236006ae40 0x7f236006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:10.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.455+0000 7f2380c6b640 1 --2- 172.21.15.92:0/1197061857 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f236006ae40 0x7f236006d300 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f2370006fd0 tx=0x7f2370008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:10.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.458+0000 7f23797fa640 1 -- 172.21.15.92:0/1197061857 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2368054c20 con 0x7f237c06cb10 2023-11-24T06:12:10.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.638+0000 7f2381c6d640 1 -- 172.21.15.92:0/1197061857 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f237c0b2120 con 0x7f237c06cb10 2023-11-24T06:12:10.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.639+0000 7f23797fa640 1 -- 172.21.15.92:0/1197061857 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v40) v1 ==== 104+0+11 (secure 0 0 0) 0x7f2368058240 con 0x7f237c06cb10 2023-11-24T06:12:10.642 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 32 2023-11-24T06:12:10.645 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.642+0000 7f2381c6d640 1 -- 172.21.15.92:0/1197061857 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f236006ae40 msgr2=0x7f236006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:10.645 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.642+0000 7f2381c6d640 1 --2- 172.21.15.92:0/1197061857 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f236006ae40 0x7f236006d300 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f2370006fd0 tx=0x7f2370008040 comp rx=0 tx=0).stop 2023-11-24T06:12:10.645 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.642+0000 7f2381c6d640 1 -- 172.21.15.92:0/1197061857 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f237c06cb10 msgr2=0x7f237c145fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:10.645 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.642+0000 7f2381c6d640 1 --2- 172.21.15.92:0/1197061857 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f237c06cb10 0x7f237c145fe0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f236800d800 tx=0x7f236800dcc0 comp rx=0 tx=0).stop 2023-11-24T06:12:10.645 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.642+0000 7f2381c6d640 1 -- 172.21.15.92:0/1197061857 shutdown_connections 2023-11-24T06:12:10.645 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.642+0000 7f2381c6d640 1 --2- 172.21.15.92:0/1197061857 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f236006ae40 0x7f236006d300 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:10.645 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.642+0000 7f2381c6d640 1 --2- 172.21.15.92:0/1197061857 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f237c070160 0x7f237c0cd350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:10.646 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.642+0000 7f2381c6d640 1 --2- 172.21.15.92:0/1197061857 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f237c06cb10 0x7f237c145fe0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:10.646 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.642+0000 7f2381c6d640 1 --2- 172.21.15.92:0/1197061857 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f237c06c730 0x7f237c1fa2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:10.646 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.642+0000 7f2381c6d640 1 -- 172.21.15.92:0/1197061857 >> 172.21.15.92:0/1197061857 conn(0x7f237c06a7d0 msgr2=0x7f237c0b0300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:10.646 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.642+0000 7f2381c6d640 1 -- 172.21.15.92:0/1197061857 shutdown_connections 2023-11-24T06:12:10.646 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.642+0000 7f2381c6d640 1 -- 172.21.15.92:0/1197061857 wait complete. 2023-11-24T06:12:10.661 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T06:12:10.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.872+0000 7fbb57ccf640 1 Processor -- start 2023-11-24T06:12:10.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.872+0000 7fbb57ccf640 1 -- start start 2023-11-24T06:12:10.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.873+0000 7fbb57ccf640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fbb5014b2c0 0x7fbb500cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:10.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.874+0000 7fbb57ccf640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb500cfbf0 con 0x7fbb500cf7d0 2023-11-24T06:12:10.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.874+0000 7fbb57ccf640 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb50071690 con 0x7fbb50071070 2023-11-24T06:12:10.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.874+0000 7fbb57ccf640 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb5014b6a0 con 0x7fbb5014b2c0 2023-11-24T06:12:10.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.874+0000 7fbb5526a640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fbb500cf7d0 0x7fbb50070960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:33780/0 (socket says 172.21.15.92:33780) 2023-11-24T06:12:10.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.874+0000 7fbb5526a640 1 -- 172.21.15.92:0/793158672 learned_addr learned my addr 172.21.15.92:0/793158672 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-24T06:12:10.879 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.874+0000 7fbb54a69640 1 -- 172.21.15.92:0/793158672 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4238764185 0 0) 0x7fbb500cfbf0 con 0x7fbb500cf7d0 2023-11-24T06:12:10.879 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.874+0000 7fbb55a6b640 1 --2- 172.21.15.92:0/793158672 >> v2:172.21.15.118:3300/0 conn(0x7fbb5014b2c0 0x7fbb500cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:10.879 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.874+0000 7fbb54a69640 1 -- 172.21.15.92:0/793158672 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbb3c003520 con 0x7fbb500cf7d0 2023-11-24T06:12:10.879 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.875+0000 7fbb54a69640 1 -- 172.21.15.92:0/793158672 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 561787941 0 0) 0x7fbb3c003520 con 0x7fbb500cf7d0 2023-11-24T06:12:10.879 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.875+0000 7fbb54a69640 1 -- 172.21.15.92:0/793158672 >> v2:172.21.15.118:3300/0 conn(0x7fbb5014b2c0 msgr2=0x7fbb500cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:10.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.875+0000 7fbb54a69640 1 --2- 172.21.15.92:0/793158672 >> v2:172.21.15.118:3300/0 conn(0x7fbb5014b2c0 0x7fbb500cf290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:10.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.875+0000 7fbb54a69640 1 -- 172.21.15.92:0/793158672 >> v1:172.21.15.118:6789/0 conn(0x7fbb50071070 legacy=0x7fbb500cc160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:10.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.875+0000 7fbb54a69640 1 -- 172.21.15.92:0/793158672 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb50071970 con 0x7fbb500cf7d0 2023-11-24T06:12:10.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.875+0000 7fbb54a69640 1 -- 172.21.15.92:0/793158672 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2607661350 0 0) 0x7fbb40001670 con 0x7fbb500cf7d0 2023-11-24T06:12:10.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.875+0000 7fbb54a69640 1 -- 172.21.15.92:0/793158672 >> v1:172.21.15.92:6789/0 conn(0x7fbb500cf7d0 legacy=0x7fbb50070960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:10.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.875+0000 7fbb54a69640 1 --2- 172.21.15.92:0/793158672 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbb3c0039c0 0x7fbb3c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:10.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.875+0000 7fbb54a69640 1 --2- 172.21.15.92:0/793158672 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fbb3c0047c0 0x7fbb3c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:10.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.875+0000 7fbb54a69640 1 --2- 172.21.15.92:0/793158672 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbb3c005180 0x7fbb3c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:10.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.875+0000 7fbb54a69640 1 -- 172.21.15.92:0/793158672 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb5014b6a0 con 0x7fbb3c005180 2023-11-24T06:12:10.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.875+0000 7fbb54a69640 1 -- 172.21.15.92:0/793158672 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb3c00c080 con 0x7fbb3c0039c0 2023-11-24T06:12:10.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.875+0000 7fbb54a69640 1 -- 172.21.15.92:0/793158672 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb3c00c240 con 0x7fbb3c0047c0 2023-11-24T06:12:10.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.875+0000 7fbb5626c640 1 --2- 172.21.15.92:0/793158672 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbb3c005180 0x7fbb3c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:10.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.875+0000 7fbb55a6b640 1 --2- 172.21.15.92:0/793158672 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbb3c0039c0 0x7fbb3c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:10.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.876+0000 7fbb5526a640 1 --2- 172.21.15.92:0/793158672 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fbb3c0047c0 0x7fbb3c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:10.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.876+0000 7fbb5626c640 1 -- 172.21.15.92:0/793158672 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fbb3c0047c0 msgr2=0x7fbb3c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:10.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.876+0000 7fbb5626c640 1 --2- 172.21.15.92:0/793158672 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fbb3c0047c0 0x7fbb3c004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:10.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.876+0000 7fbb5626c640 1 -- 172.21.15.92:0/793158672 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbb3c0039c0 msgr2=0x7fbb3c003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:10.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.876+0000 7fbb5626c640 1 --2- 172.21.15.92:0/793158672 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbb3c0039c0 0x7fbb3c003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:10.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.876+0000 7fbb5626c640 1 -- 172.21.15.92:0/793158672 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb3c003520 con 0x7fbb3c005180 2023-11-24T06:12:10.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.877+0000 7fbb5626c640 1 -- 172.21.15.92:0/793158672 reap_dead start 2023-11-24T06:12:10.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.877+0000 7fbb5626c640 1 --2- 172.21.15.92:0/793158672 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbb3c005180 0x7fbb3c00b980 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fbb48001e50 tx=0x7fbb3c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3539057190456133 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:10.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.877+0000 7fbb54a69640 1 -- 172.21.15.92:0/793158672 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb48003b70 con 0x7fbb3c005180 2023-11-24T06:12:10.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.877+0000 7fbb54a69640 1 -- 172.21.15.92:0/793158672 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb48003d10 con 0x7fbb3c005180 2023-11-24T06:12:10.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.877+0000 7fbb54a69640 1 -- 172.21.15.92:0/793158672 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb48005310 con 0x7fbb3c005180 2023-11-24T06:12:10.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.877+0000 7fbb57ccf640 1 -- 172.21.15.92:0/793158672 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbb3c005180 msgr2=0x7fbb3c00b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:10.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.877+0000 7fbb57ccf640 1 --2- 172.21.15.92:0/793158672 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbb3c005180 0x7fbb3c00b980 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fbb48001e50 tx=0x7fbb3c0057f0 comp rx=0 tx=0).stop 2023-11-24T06:12:10.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.878+0000 7fbb57ccf640 1 -- 172.21.15.92:0/793158672 shutdown_connections 2023-11-24T06:12:10.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.878+0000 7fbb57ccf640 1 --2- 172.21.15.92:0/793158672 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbb3c005180 0x7fbb3c00b980 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:10.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.878+0000 7fbb57ccf640 1 -- 172.21.15.92:0/793158672 >> 172.21.15.92:0/793158672 conn(0x7fbb5006a800 msgr2=0x7fbb5006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:10.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.878+0000 7fbb57ccf640 1 -- 172.21.15.92:0/793158672 shutdown_connections 2023-11-24T06:12:10.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.878+0000 7fbb57ccf640 1 -- 172.21.15.92:0/793158672 wait complete. 2023-11-24T06:12:10.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.879+0000 7fbb57ccf640 1 Processor -- start 2023-11-24T06:12:10.885 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.880+0000 7fbb57ccf640 1 -- start start 2023-11-24T06:12:10.885 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.881+0000 7fbb57ccf640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbb3c005180 0x7fbb50163e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:10.885 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.881+0000 7fbb57ccf640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fbb50148120 0x7fbb501485a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:10.885 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.881+0000 7fbb57ccf640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbb501623a0 0x7fbb50162820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:10.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.881+0000 7fbb57ccf640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb50164ec0 con 0x7fbb501623a0 2023-11-24T06:12:10.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.881+0000 7fbb57ccf640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb50148ae0 con 0x7fbb3c005180 2023-11-24T06:12:10.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.881+0000 7fbb57ccf640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb50164350 con 0x7fbb50148120 2023-11-24T06:12:10.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.881+0000 7fbb5626c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbb501623a0 0x7fbb50162820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:10.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.881+0000 7fbb5626c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbb501623a0 0x7fbb50162820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:60446/0 (socket says 172.21.15.92:60446) 2023-11-24T06:12:10.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.881+0000 7fbb5626c640 1 -- 172.21.15.92:0/2331040877 learned_addr learned my addr 172.21.15.92:0/2331040877 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:12:10.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.881+0000 7fbb55a6b640 1 --2- 172.21.15.92:0/2331040877 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbb3c005180 0x7fbb50163e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:10.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.881+0000 7fbb5526a640 1 --2- 172.21.15.92:0/2331040877 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fbb50148120 0x7fbb501485a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:10.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.881+0000 7fbb5626c640 1 -- 172.21.15.92:0/2331040877 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fbb50148120 msgr2=0x7fbb501485a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:10.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.881+0000 7fbb5626c640 1 --2- 172.21.15.92:0/2331040877 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fbb50148120 0x7fbb501485a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:10.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.881+0000 7fbb5626c640 1 -- 172.21.15.92:0/2331040877 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbb3c005180 msgr2=0x7fbb50163e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:10.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.881+0000 7fbb5626c640 1 --2- 172.21.15.92:0/2331040877 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbb3c005180 0x7fbb50163e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:10.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.881+0000 7fbb5626c640 1 -- 172.21.15.92:0/2331040877 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb50164620 con 0x7fbb501623a0 2023-11-24T06:12:10.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.881+0000 7fbb55a6b640 1 --2- 172.21.15.92:0/2331040877 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbb3c005180 0x7fbb50163e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:12:10.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.882+0000 7fbb5626c640 1 --2- 172.21.15.92:0/2331040877 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbb501623a0 0x7fbb50162820 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fbb48001f70 tx=0x7fbb480014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:10.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.882+0000 7fbb467fc640 1 -- 172.21.15.92:0/2331040877 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb48006d50 con 0x7fbb501623a0 2023-11-24T06:12:10.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.882+0000 7fbb467fc640 1 -- 172.21.15.92:0/2331040877 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb48039480 con 0x7fbb501623a0 2023-11-24T06:12:10.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.882+0000 7fbb57ccf640 1 -- 172.21.15.92:0/2331040877 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb50164900 con 0x7fbb501623a0 2023-11-24T06:12:10.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.882+0000 7fbb467fc640 1 -- 172.21.15.92:0/2331040877 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb48039830 con 0x7fbb501623a0 2023-11-24T06:12:10.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.882+0000 7fbb57ccf640 1 -- 172.21.15.92:0/2331040877 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb50152410 con 0x7fbb501623a0 2023-11-24T06:12:10.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.883+0000 7fbb57ccf640 1 -- 172.21.15.92:0/2331040877 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb50152270 con 0x7fbb501623a0 2023-11-24T06:12:10.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.884+0000 7fbb467fc640 1 -- 172.21.15.92:0/2331040877 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fbb48039990 con 0x7fbb501623a0 2023-11-24T06:12:10.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.885+0000 7fbb467fc640 1 --2- 172.21.15.92:0/2331040877 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fbb3406ae70 0x7fbb3406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:10.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.885+0000 7fbb467fc640 1 -- 172.21.15.92:0/2331040877 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fbb480ab620 con 0x7fbb501623a0 2023-11-24T06:12:10.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.885+0000 7fbb55a6b640 1 --2- 172.21.15.92:0/2331040877 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fbb3406ae70 0x7fbb3406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:10.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.886+0000 7fbb55a6b640 1 --2- 172.21.15.92:0/2331040877 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fbb3406ae70 0x7fbb3406d330 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fbb4c009980 tx=0x7fbb4c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:10.892 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:10.889+0000 7fbb467fc640 1 -- 172.21.15.92:0/2331040877 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fbb50152270 con 0x7fbb501623a0 2023-11-24T06:12:11.082 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:11.078+0000 7fbb57ccf640 1 -- 172.21.15.92:0/2331040877 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fbb50152270 con 0x7fbb501623a0 2023-11-24T06:12:11.082 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:11.079+0000 7fbb467fc640 1 -- 172.21.15.92:0/2331040877 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v40) v1 ==== 100+0+11 (secure 0 0 0) 0x7fbb50152270 con 0x7fbb501623a0 2023-11-24T06:12:11.082 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 64 2023-11-24T06:12:11.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:11.082+0000 7fbb57ccf640 1 -- 172.21.15.92:0/2331040877 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fbb3406ae70 msgr2=0x7fbb3406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:11.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:11.082+0000 7fbb57ccf640 1 --2- 172.21.15.92:0/2331040877 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fbb3406ae70 0x7fbb3406d330 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fbb4c009980 tx=0x7fbb4c008040 comp rx=0 tx=0).stop 2023-11-24T06:12:11.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:11.082+0000 7fbb57ccf640 1 -- 172.21.15.92:0/2331040877 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbb501623a0 msgr2=0x7fbb50162820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:11.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:11.082+0000 7fbb57ccf640 1 --2- 172.21.15.92:0/2331040877 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbb501623a0 0x7fbb50162820 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fbb48001f70 tx=0x7fbb480014a0 comp rx=0 tx=0).stop 2023-11-24T06:12:11.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:11.082+0000 7fbb57ccf640 1 -- 172.21.15.92:0/2331040877 shutdown_connections 2023-11-24T06:12:11.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:11.082+0000 7fbb57ccf640 1 --2- 172.21.15.92:0/2331040877 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fbb3406ae70 0x7fbb3406d330 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:11.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:11.082+0000 7fbb57ccf640 1 --2- 172.21.15.92:0/2331040877 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbb501623a0 0x7fbb50162820 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:11.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:11.082+0000 7fbb57ccf640 1 --2- 172.21.15.92:0/2331040877 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fbb50148120 0x7fbb501485a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:11.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:11.082+0000 7fbb57ccf640 1 --2- 172.21.15.92:0/2331040877 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbb3c005180 0x7fbb50163e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:11.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:11.082+0000 7fbb57ccf640 1 -- 172.21.15.92:0/2331040877 >> 172.21.15.92:0/2331040877 conn(0x7fbb5006a800 msgr2=0x7fbb500aee80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:11.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:11.082+0000 7fbb57ccf640 1 -- 172.21.15.92:0/2331040877 shutdown_connections 2023-11-24T06:12:11.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:11.082+0000 7fbb57ccf640 1 -- 172.21.15.92:0/2331040877 wait complete. 2023-11-24T06:12:11.104 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-24T06:12:11.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:11.306+0000 7f6be9e2b640 1 Processor -- start 2023-11-24T06:12:11.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:11.306+0000 7f6be9e2b640 1 -- start start 2023-11-24T06:12:11.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:11.308+0000 7f6be9e2b640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6be40b3ae0 0x7f6be415e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:11.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:11.308+0000 7f6be9e2b640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f6be40cf290 0x7f6be40cf720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:11.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:11.308+0000 7f6be9e2b640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6be4070c40 con 0x7f6be40ae5a0 2023-11-24T06:12:11.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:11.308+0000 7f6be9e2b640 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6be4071450 con 0x7f6be40cf290 2023-11-24T06:12:11.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:11.308+0000 7f6be9e2b640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6be40cfc60 con 0x7f6be40b3ae0 2023-11-24T06:12:11.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:11.308+0000 7f6be37fe640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f6be40ae5a0 0x7f6be406cbe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:33782/0 (socket says 172.21.15.92:33782) 2023-11-24T06:12:11.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:11.308+0000 7f6be37fe640 1 -- 172.21.15.92:0/928044935 learned_addr learned my addr 172.21.15.92:0/928044935 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-24T06:12:11.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:11.309+0000 7f6be3fff640 1 --2- 172.21.15.92:0/928044935 >> v2:172.21.15.118:3300/0 conn(0x7f6be40cf290 0x7f6be40cf720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:11.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:11.309+0000 7f6be2ffd640 1 --2- 172.21.15.92:0/928044935 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6be40b3ae0 0x7f6be415e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:11.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:11.309+0000 7f6be27fc640 1 -- 172.21.15.92:0/928044935 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1525068656 0 0) 0x7f6be4070c40 con 0x7f6be40ae5a0 2023-11-24T06:12:11.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:11.309+0000 7f6be27fc640 1 -- 172.21.15.92:0/928044935 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6bd0003520 con 0x7f6be40ae5a0 2023-11-24T06:12:11.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:11.309+0000 7f6be27fc640 1 -- 172.21.15.92:0/928044935 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1288660222 0 0) 0x7f6bd0003520 con 0x7f6be40ae5a0 2023-11-24T06:12:11.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:11.309+0000 7f6be27fc640 1 -- 172.21.15.92:0/928044935 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6be40b3ae0 msgr2=0x7f6be415e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:11.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:11.309+0000 7f6be27fc640 1 --2- 172.21.15.92:0/928044935 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6be40b3ae0 0x7f6be415e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:11.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:11.309+0000 7f6be27fc640 1 -- 172.21.15.92:0/928044935 >> v2:172.21.15.118:3300/0 conn(0x7f6be40cf290 msgr2=0x7f6be40cf720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:11.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:11.309+0000 7f6be27fc640 1 --2- 172.21.15.92:0/928044935 >> v2:172.21.15.118:3300/0 conn(0x7f6be40cf290 0x7f6be40cf720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:11.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:11.309+0000 7f6be27fc640 1 -- 172.21.15.92:0/928044935 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6be406c470 con 0x7f6be40ae5a0 2023-11-24T06:12:11.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:11.309+0000 7f6be27fc640 1 -- 172.21.15.92:0/928044935 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2607661350 0 0) 0x7f6bd4001670 con 0x7f6be40ae5a0 2023-11-24T06:12:11.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:11.309+0000 7f6be27fc640 1 -- 172.21.15.92:0/928044935 >> v1:172.21.15.92:6789/0 conn(0x7f6be40ae5a0 legacy=0x7f6be406cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:11.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:11.309+0000 7f6be27fc640 1 --2- 172.21.15.92:0/928044935 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6be40b3ae0 0x7f6bd0003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:11.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:11.309+0000 7f6be27fc640 1 --2- 172.21.15.92:0/928044935 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6bd0003ef0 0x7f6bd00047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:11.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:11.309+0000 7f6be27fc640 1 --2- 172.21.15.92:0/928044935 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6bd0004d00 0x7f6bd0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:11.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:11.309+0000 7f6be27fc640 1 -- 172.21.15.92:0/928044935 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6be4071450 con 0x7f6bd0004d00 2023-11-24T06:12:11.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:11.309+0000 7f6be27fc640 1 -- 172.21.15.92:0/928044935 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6be40cfc60 con 0x7f6bd0003ef0 2023-11-24T06:12:11.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:11.309+0000 7f6be27fc640 1 -- 172.21.15.92:0/928044935 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bd000b9b0 con 0x7f6be40b3ae0 2023-11-24T06:12:11.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:11.310+0000 7f6be3fff640 1 --2- 172.21.15.92:0/928044935 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6bd0004d00 0x7f6bd0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:11.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:11.310+0000 7f6be37fe640 1 --2- 172.21.15.92:0/928044935 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6be40b3ae0 0x7f6bd0003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:11.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:11.310+0000 7f6be3fff640 1 -- 172.21.15.92:0/928044935 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6be40b3ae0 msgr2=0x7f6bd0003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:11.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:11.310+0000 7f6be3fff640 1 --2- 172.21.15.92:0/928044935 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6be40b3ae0 0x7f6bd0003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:11.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:11.310+0000 7f6be2ffd640 1 --2- 172.21.15.92:0/928044935 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6bd0003ef0 0x7f6bd00047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:11.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:11.310+0000 7f6be3fff640 1 -- 172.21.15.92:0/928044935 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6bd0003ef0 msgr2=0x7f6bd00047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:11.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:11.310+0000 7f6be3fff640 1 --2- 172.21.15.92:0/928044935 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6bd0003ef0 0x7f6bd00047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:11.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:11.310+0000 7f6be3fff640 1 -- 172.21.15.92:0/928044935 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bd0003520 con 0x7f6bd0004d00 2023-11-24T06:12:11.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:11.310+0000 7f6be37fe640 1 --2- 172.21.15.92:0/928044935 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6be40b3ae0 0x7f6bd0003970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:12:11.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:11.311+0000 7f6be3fff640 1 --2- 172.21.15.92:0/928044935 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6bd0004d00 0x7f6bd0005180 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f6bdc009fc0 tx=0x7f6bd00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=742e4804443cde38 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:11.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:11.311+0000 7f6be27fc640 1 -- 172.21.15.92:0/928044935 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bdc02cc30 con 0x7f6bd0004d00 2023-11-24T06:12:11.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:11.311+0000 7f6be27fc640 1 -- 172.21.15.92:0/928044935 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bdc00f040 con 0x7f6bd0004d00 2023-11-24T06:12:11.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:11.311+0000 7f6be27fc640 1 -- 172.21.15.92:0/928044935 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bdc034440 con 0x7f6bd0004d00 2023-11-24T06:12:11.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:11.312+0000 7f6be9e2b640 1 -- 172.21.15.92:0/928044935 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6bd0004d00 msgr2=0x7f6bd0005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:11.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:11.312+0000 7f6be9e2b640 1 --2- 172.21.15.92:0/928044935 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6bd0004d00 0x7f6bd0005180 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f6bdc009fc0 tx=0x7f6bd00057f0 comp rx=0 tx=0).stop 2023-11-24T06:12:11.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:11.312+0000 7f6be3fff640 1 -- 172.21.15.92:0/928044935 reap_dead start 2023-11-24T06:12:11.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:11.312+0000 7f6be9e2b640 1 -- 172.21.15.92:0/928044935 shutdown_connections 2023-11-24T06:12:11.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:11.312+0000 7f6be9e2b640 1 -- 172.21.15.92:0/928044935 >> 172.21.15.92:0/928044935 conn(0x7f6be406a800 msgr2=0x7f6be406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:11.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:11.312+0000 7f6be9e2b640 1 -- 172.21.15.92:0/928044935 shutdown_connections 2023-11-24T06:12:11.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:11.312+0000 7f6be9e2b640 1 -- 172.21.15.92:0/928044935 wait complete. 2023-11-24T06:12:11.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:11.314+0000 7f6be9e2b640 1 Processor -- start 2023-11-24T06:12:11.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:11.314+0000 7f6be9e2b640 1 -- start start 2023-11-24T06:12:11.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:11.315+0000 7f6be9e2b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6bd0004d00 0x7f6be415cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:11.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:11.315+0000 7f6be9e2b640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6be415eaf0 0x7f6be414d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:11.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:11.315+0000 7f6be9e2b640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6be413f2c0 0x7f6be4157e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:11.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:11.315+0000 7f6be9e2b640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6be415d4d0 con 0x7f6bd0004d00 2023-11-24T06:12:11.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:11.315+0000 7f6be9e2b640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6be415eff0 con 0x7f6be413f2c0 2023-11-24T06:12:11.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:11.315+0000 7f6be9e2b640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6be415f2d0 con 0x7f6be415eaf0 2023-11-24T06:12:11.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:11.315+0000 7f6be37fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6bd0004d00 0x7f6be415cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:11.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:11.315+0000 7f6be37fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6bd0004d00 0x7f6be415cf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:60456/0 (socket says 172.21.15.92:60456) 2023-11-24T06:12:11.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:11.315+0000 7f6be37fe640 1 -- 172.21.15.92:0/1788652916 learned_addr learned my addr 172.21.15.92:0/1788652916 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:12:11.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:11.315+0000 7f6be37fe640 1 -- 172.21.15.92:0/1788652916 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6be415eaf0 msgr2=0x7f6be414d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:11.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:11.315+0000 7f6be37fe640 1 --2- 172.21.15.92:0/1788652916 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6be415eaf0 0x7f6be414d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:11.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:11.315+0000 7f6be37fe640 1 -- 172.21.15.92:0/1788652916 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6be413f2c0 msgr2=0x7f6be4157e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:11.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:11.315+0000 7f6be37fe640 1 --2- 172.21.15.92:0/1788652916 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6be413f2c0 0x7f6be4157e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:11.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:11.315+0000 7f6be37fe640 1 -- 172.21.15.92:0/1788652916 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6be415f5b0 con 0x7f6bd0004d00 2023-11-24T06:12:11.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:11.316+0000 7f6be37fe640 1 --2- 172.21.15.92:0/1788652916 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6bd0004d00 0x7f6be415cf90 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f6bd400d8d0 tx=0x7f6bd400dd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:11.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:11.316+0000 7f6bdbfff640 1 -- 172.21.15.92:0/1788652916 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bd40074e0 con 0x7f6bd0004d00 2023-11-24T06:12:11.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:11.316+0000 7f6bdbfff640 1 -- 172.21.15.92:0/1788652916 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bd4007640 con 0x7f6bd0004d00 2023-11-24T06:12:11.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:11.316+0000 7f6bdbfff640 1 -- 172.21.15.92:0/1788652916 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bd4004a00 con 0x7f6bd0004d00 2023-11-24T06:12:11.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:11.316+0000 7f6be9e2b640 1 -- 172.21.15.92:0/1788652916 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6be415f890 con 0x7f6bd0004d00 2023-11-24T06:12:11.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:11.316+0000 7f6be9e2b640 1 -- 172.21.15.92:0/1788652916 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6be414d820 con 0x7f6bd0004d00 2023-11-24T06:12:11.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:11.317+0000 7f6be9e2b640 1 -- 172.21.15.92:0/1788652916 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6be40675d0 con 0x7f6bd0004d00 2023-11-24T06:12:11.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:11.318+0000 7f6bdbfff640 1 -- 172.21.15.92:0/1788652916 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6bd4004b60 con 0x7f6bd0004d00 2023-11-24T06:12:11.323 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:11.320+0000 7f6bdbfff640 1 --2- 172.21.15.92:0/1788652916 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f6bc406ae70 0x7f6bc406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:11.323 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:11.320+0000 7f6bdbfff640 1 -- 172.21.15.92:0/1788652916 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f6bd4089140 con 0x7f6bd0004d00 2023-11-24T06:12:11.323 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:11.320+0000 7f6be2ffd640 1 --2- 172.21.15.92:0/1788652916 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f6bc406ae70 0x7f6bc406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:11.323 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:11.320+0000 7f6be2ffd640 1 --2- 172.21.15.92:0/1788652916 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f6bc406ae70 0x7f6bc406d330 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f6be41422e0 tx=0x7f6bcc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:11.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:11.324+0000 7f6bdbfff640 1 -- 172.21.15.92:0/1788652916 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6bd4054060 con 0x7f6bd0004d00 2023-11-24T06:12:11.514 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:11.511+0000 7f6be9e2b640 1 -- 172.21.15.92:0/1788652916 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f6be4159500 con 0x7f6bd0004d00 2023-11-24T06:12:11.515 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:11.512+0000 7f6bdbfff640 1 -- 172.21.15.92:0/1788652916 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v40) v1 ==== 80+0+122 (secure 0 0 0) 0x7f6bd4019080 con 0x7f6bd0004d00 2023-11-24T06:12:11.515 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.15.92:6836/3889219158 2023-11-25T06:12:03.443855+0000 2023-11-24T06:12:11.515 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.15.92:6834/3889219158 2023-11-25T06:12:03.443855+0000 2023-11-24T06:12:11.515 INFO:teuthology.orchestra.run.smithi092.stderr:listed 2 entries 2023-11-24T06:12:11.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:11.514+0000 7f6be9e2b640 1 -- 172.21.15.92:0/1788652916 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f6bc406ae70 msgr2=0x7f6bc406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:11.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:11.514+0000 7f6be9e2b640 1 --2- 172.21.15.92:0/1788652916 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f6bc406ae70 0x7f6bc406d330 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f6be41422e0 tx=0x7f6bcc008040 comp rx=0 tx=0).stop 2023-11-24T06:12:11.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:11.514+0000 7f6be9e2b640 1 -- 172.21.15.92:0/1788652916 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6bd0004d00 msgr2=0x7f6be415cf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:11.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:11.514+0000 7f6be9e2b640 1 --2- 172.21.15.92:0/1788652916 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6bd0004d00 0x7f6be415cf90 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f6bd400d8d0 tx=0x7f6bd400dd90 comp rx=0 tx=0).stop 2023-11-24T06:12:11.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:11.514+0000 7f6be9e2b640 1 -- 172.21.15.92:0/1788652916 shutdown_connections 2023-11-24T06:12:11.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:11.514+0000 7f6be9e2b640 1 --2- 172.21.15.92:0/1788652916 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f6bc406ae70 0x7f6bc406d330 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:11.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:11.514+0000 7f6be9e2b640 1 --2- 172.21.15.92:0/1788652916 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6be413f2c0 0x7f6be4157e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:11.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:11.514+0000 7f6be9e2b640 1 --2- 172.21.15.92:0/1788652916 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6be415eaf0 0x7f6be414d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:11.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:11.514+0000 7f6be9e2b640 1 --2- 172.21.15.92:0/1788652916 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6bd0004d00 0x7f6be415cf90 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:11.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:11.514+0000 7f6be9e2b640 1 -- 172.21.15.92:0/1788652916 >> 172.21.15.92:0/1788652916 conn(0x7f6be406a800 msgr2=0x7f6be4159a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:11.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:11.514+0000 7f6be9e2b640 1 -- 172.21.15.92:0/1788652916 shutdown_connections 2023-11-24T06:12:11.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:11.514+0000 7f6be9e2b640 1 -- 172.21.15.92:0/1788652916 wait complete. 2023-11-24T06:12:11.533 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi118.front.sepia.ceph.com... 2023-11-24T06:12:11.534 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:12:11.534 DEBUG:teuthology.orchestra.run.smithi118:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2023-11-24T06:12:11.630 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi118.stderr:ceph-fuse[94335]: fuse finished with error 0 and tester_r 0 2023-11-24T06:12:11.634 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi118.front.sepia.ceph.com 2023-11-24T06:12:11.634 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:12:11.634 DEBUG:teuthology.orchestra.run.smithi118:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-11-24T06:12:11.814 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-11-24T06:12:11.815 DEBUG:teuthology.orchestra.run.smithi092:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:12:11.875 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054298,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498824k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249408k,nr_inodes=812352,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498824k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-24T06:12:11.875 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi092.front.sepia.ceph.com 2023-11-24T06:12:11.875 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:12:11.875 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-11-24T06:12:11.929 INFO:teuthology.orchestra.run.smithi092.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-11-24T06:12:11.930 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-24T06:12:11.930 DEBUG:teuthology.orchestra.run.smithi118:> set -ex 2023-11-24T06:12:11.930 DEBUG:teuthology.orchestra.run.smithi118:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:12:11.991 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054284,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249396k,nr_inodes=812349,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-24T06:12:11.991 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi118.front.sepia.ceph.com 2023-11-24T06:12:11.991 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:12:11.992 DEBUG:teuthology.orchestra.run.smithi118:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-11-24T06:12:12.046 INFO:teuthology.orchestra.run.smithi118.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-11-24T06:12:12.047 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-24T06:12:12.047 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T06:12:12.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:12.267+0000 7fd7609c1640 1 Processor -- start 2023-11-24T06:12:12.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:12.268+0000 7fd7609c1640 1 -- start start 2023-11-24T06:12:12.273 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:12.269+0000 7fd7609c1640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fd75c0d0220 0x7fd75c0d0e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:12.273 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:12.269+0000 7fd7609c1640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd75c0ce9f0 0x7fd75c0cee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:12.273 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:12.269+0000 7fd7609c1640 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd75c0b23c0 con 0x7fd75c15d3c0 2023-11-24T06:12:12.273 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:12.269+0000 7fd7609c1640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd75c0ce5d0 con 0x7fd75c0d0220 2023-11-24T06:12:12.273 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:12.269+0000 7fd7609c1640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd75c0d0800 con 0x7fd75c0ce9f0 2023-11-24T06:12:12.273 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:12.269+0000 7fd75affd640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fd75c0d0220 0x7fd75c0d0e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:12.273 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:12.270+0000 7fd75affd640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fd75c0d0220 0x7fd75c0d0e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:60458/0 (socket says 172.21.15.92:60458) 2023-11-24T06:12:12.273 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:12.270+0000 7fd75affd640 1 -- 172.21.15.92:0/1369199684 learned_addr learned my addr 172.21.15.92:0/1369199684 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:12:12.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:12.270+0000 7fd75bfff640 1 --2- 172.21.15.92:0/1369199684 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd75c0ce9f0 0x7fd75c0cee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:12.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:12.270+0000 7fd75affd640 1 -- 172.21.15.92:0/1369199684 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd75c0ce9f0 msgr2=0x7fd75c0cee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:12.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:12.270+0000 7fd75affd640 1 --2- 172.21.15.92:0/1369199684 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd75c0ce9f0 0x7fd75c0cee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:12.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:12.270+0000 7fd75affd640 1 -- 172.21.15.92:0/1369199684 >> v1:172.21.15.118:6789/0 conn(0x7fd75c15d3c0 legacy=0x7fd75c0afc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:12.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:12.270+0000 7fd75affd640 1 -- 172.21.15.92:0/1369199684 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd75c0cdf10 con 0x7fd75c0d0220 2023-11-24T06:12:12.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:12.271+0000 7fd75affd640 1 --2- 172.21.15.92:0/1369199684 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd75c0d0220 0x7fd75c0d0e90 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fd744009870 tx=0x7fd74402b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=325dfe46a84a4ad7 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:12.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:12.271+0000 7fd75a7fc640 1 -- 172.21.15.92:0/1369199684 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7440317c0 con 0x7fd75c0d0220 2023-11-24T06:12:12.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:12.271+0000 7fd75a7fc640 1 -- 172.21.15.92:0/1369199684 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd744031920 con 0x7fd75c0d0220 2023-11-24T06:12:12.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:12.271+0000 7fd75a7fc640 1 -- 172.21.15.92:0/1369199684 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd744031cb0 con 0x7fd75c0d0220 2023-11-24T06:12:12.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:12.272+0000 7fd7609c1640 1 -- 172.21.15.92:0/1369199684 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd75c0d0220 msgr2=0x7fd75c0d0e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:12.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:12.272+0000 7fd7609c1640 1 --2- 172.21.15.92:0/1369199684 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd75c0d0220 0x7fd75c0d0e90 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fd744009870 tx=0x7fd74402b860 comp rx=0 tx=0).stop 2023-11-24T06:12:12.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:12.272+0000 7fd7609c1640 1 -- 172.21.15.92:0/1369199684 shutdown_connections 2023-11-24T06:12:12.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:12.272+0000 7fd7609c1640 1 --2- 172.21.15.92:0/1369199684 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd75c0ce9f0 0x7fd75c0cee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:12.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:12.272+0000 7fd7609c1640 1 --2- 172.21.15.92:0/1369199684 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd75c0d0220 0x7fd75c0d0e90 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:12.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:12.272+0000 7fd7609c1640 1 -- 172.21.15.92:0/1369199684 >> 172.21.15.92:0/1369199684 conn(0x7fd75c0d30b0 msgr2=0x7fd75c0d34c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:12.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:12.272+0000 7fd7609c1640 1 -- 172.21.15.92:0/1369199684 shutdown_connections 2023-11-24T06:12:12.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:12.272+0000 7fd7609c1640 1 -- 172.21.15.92:0/1369199684 wait complete. 2023-11-24T06:12:12.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:12.274+0000 7fd7609c1640 1 Processor -- start 2023-11-24T06:12:12.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:12.274+0000 7fd7609c1640 1 -- start start 2023-11-24T06:12:12.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:12.275+0000 7fd7609c1640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd75c0ce9f0 0x7fd75c150030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:12.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:12.275+0000 7fd7609c1640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd75c0d0220 0x7fd75c0c94a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:12.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:12.275+0000 7fd7609c1640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd75c15d3c0 0x7fd75c14c020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:12.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:12.275+0000 7fd7609c1640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd75c14fe50 con 0x7fd75c15d3c0 2023-11-24T06:12:12.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:12.275+0000 7fd7609c1640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd75c0cc8e0 con 0x7fd75c0ce9f0 2023-11-24T06:12:12.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:12.275+0000 7fd7609c1640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd75c0ca6f0 con 0x7fd75c0d0220 2023-11-24T06:12:12.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:12.275+0000 7fd75bfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd75c15d3c0 0x7fd75c14c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:12.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:12.275+0000 7fd75bfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd75c15d3c0 0x7fd75c14c020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:60462/0 (socket says 172.21.15.92:60462) 2023-11-24T06:12:12.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:12.275+0000 7fd75bfff640 1 -- 172.21.15.92:0/1813605240 learned_addr learned my addr 172.21.15.92:0/1813605240 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:12:12.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:12.276+0000 7fd75affd640 1 --2- 172.21.15.92:0/1813605240 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd75c0d0220 0x7fd75c0c94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:12.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:12.276+0000 7fd75bfff640 1 -- 172.21.15.92:0/1813605240 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd75c0d0220 msgr2=0x7fd75c0c94a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:12.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:12.276+0000 7fd75bfff640 1 --2- 172.21.15.92:0/1813605240 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd75c0d0220 0x7fd75c0c94a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:12.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:12.276+0000 7fd75bfff640 1 -- 172.21.15.92:0/1813605240 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd75c0ce9f0 msgr2=0x7fd75c150030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:12.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:12.276+0000 7fd75b7fe640 1 --2- 172.21.15.92:0/1813605240 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd75c0ce9f0 0x7fd75c150030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:12.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:12.276+0000 7fd75bfff640 1 --2- 172.21.15.92:0/1813605240 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd75c0ce9f0 0x7fd75c150030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:12.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:12.276+0000 7fd75bfff640 1 -- 172.21.15.92:0/1813605240 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7440094d0 con 0x7fd75c15d3c0 2023-11-24T06:12:12.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:12.276+0000 7fd75b7fe640 1 --2- 172.21.15.92:0/1813605240 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd75c0ce9f0 0x7fd75c150030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:12:12.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:12.276+0000 7fd75bfff640 1 --2- 172.21.15.92:0/1813605240 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd75c15d3c0 0x7fd75c14c020 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fd75400d800 tx=0x7fd75400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:12.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:12.276+0000 7fd753fff640 1 -- 172.21.15.92:0/1813605240 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd75400b6c0 con 0x7fd75c15d3c0 2023-11-24T06:12:12.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:12.276+0000 7fd753fff640 1 -- 172.21.15.92:0/1813605240 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd75400b840 con 0x7fd75c15d3c0 2023-11-24T06:12:12.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:12.276+0000 7fd7609c1640 1 -- 172.21.15.92:0/1813605240 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd75c0ccb30 con 0x7fd75c15d3c0 2023-11-24T06:12:12.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:12.276+0000 7fd753fff640 1 -- 172.21.15.92:0/1813605240 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd75400bbf0 con 0x7fd75c15d3c0 2023-11-24T06:12:12.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:12.277+0000 7fd7609c1640 1 -- 172.21.15.92:0/1813605240 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd75c0cd0f0 con 0x7fd75c15d3c0 2023-11-24T06:12:12.282 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:12.278+0000 7fd7609c1640 1 -- 172.21.15.92:0/1813605240 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd75c0cd2e0 con 0x7fd75c15d3c0 2023-11-24T06:12:12.282 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:12.278+0000 7fd753fff640 1 -- 172.21.15.92:0/1813605240 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd75400bd50 con 0x7fd75c15d3c0 2023-11-24T06:12:12.282 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:12.279+0000 7fd753fff640 1 --2- 172.21.15.92:0/1813605240 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fd73c06ae40 0x7fd73c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:12.283 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:12.279+0000 7fd753fff640 1 -- 172.21.15.92:0/1813605240 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fd754089b70 con 0x7fd75c15d3c0 2023-11-24T06:12:12.283 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:12.280+0000 7fd75b7fe640 1 --2- 172.21.15.92:0/1813605240 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fd73c06ae40 0x7fd73c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:12.283 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:12.280+0000 7fd75b7fe640 1 --2- 172.21.15.92:0/1813605240 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fd73c06ae40 0x7fd73c06d300 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fd74c007e90 tx=0x7fd74c0073a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:12.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:12.284+0000 7fd753fff640 1 -- 172.21.15.92:0/1813605240 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd754054c50 con 0x7fd75c15d3c0 2023-11-24T06:12:12.512 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:12.509+0000 7fd7609c1640 1 -- 172.21.15.92:0/1813605240 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd75c067b60 con 0x7fd75c15d3c0 2023-11-24T06:12:12.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:12.513+0000 7fd753fff640 1 -- 172.21.15.92:0/1813605240 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+4919 (secure 0 0 0) 0x7fd754058270 con 0x7fd75c15d3c0 2023-11-24T06:12:12.517 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-24T06:12:12.517 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":23,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4637,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6834/2665242481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":2665242481},{"type":"v1","addr":"172.21.15.118:6834","nonce":2665242481}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":4640,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6835/2822230615","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6833","nonce":2822230615},{"type":"v1","addr":"172.21.15.118:6835","nonce":2822230615}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5499,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6836/2918354210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":2918354210},{"type":"v1","addr":"172.21.15.92:6836","nonce":2918354210}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":23}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T06:11:44.503003+0000","modified":"2023-11-24T06:11:47.998422+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5235},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5235":{"gid":5235,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.92:6837/1413573590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":1413573590},{"type":"v1","addr":"172.21.15.92:6837","nonce":1413573590}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-24T06:12:12.517 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 23 2023-11-24T06:12:12.519 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:12.516+0000 7fd7609c1640 1 -- 172.21.15.92:0/1813605240 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fd73c06ae40 msgr2=0x7fd73c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:12.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:12.516+0000 7fd7609c1640 1 --2- 172.21.15.92:0/1813605240 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fd73c06ae40 0x7fd73c06d300 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fd74c007e90 tx=0x7fd74c0073a0 comp rx=0 tx=0).stop 2023-11-24T06:12:12.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:12.516+0000 7fd7609c1640 1 -- 172.21.15.92:0/1813605240 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd75c15d3c0 msgr2=0x7fd75c14c020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:12.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:12.516+0000 7fd7609c1640 1 --2- 172.21.15.92:0/1813605240 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd75c15d3c0 0x7fd75c14c020 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fd75400d800 tx=0x7fd75400dcc0 comp rx=0 tx=0).stop 2023-11-24T06:12:12.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:12.516+0000 7fd7609c1640 1 -- 172.21.15.92:0/1813605240 shutdown_connections 2023-11-24T06:12:12.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:12.516+0000 7fd7609c1640 1 --2- 172.21.15.92:0/1813605240 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fd73c06ae40 0x7fd73c06d300 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:12.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:12.516+0000 7fd7609c1640 1 --2- 172.21.15.92:0/1813605240 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd75c15d3c0 0x7fd75c14c020 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:12.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:12.516+0000 7fd7609c1640 1 --2- 172.21.15.92:0/1813605240 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd75c0d0220 0x7fd75c0c94a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:12.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:12.516+0000 7fd7609c1640 1 --2- 172.21.15.92:0/1813605240 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd75c0ce9f0 0x7fd75c150030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:12.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:12.516+0000 7fd7609c1640 1 -- 172.21.15.92:0/1813605240 >> 172.21.15.92:0/1813605240 conn(0x7fd75c0d30b0 msgr2=0x7fd75c15db30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:12.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:12.516+0000 7fd7609c1640 1 -- 172.21.15.92:0/1813605240 shutdown_connections 2023-11-24T06:12:12.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:12.516+0000 7fd7609c1640 1 -- 172.21.15.92:0/1813605240 wait complete. 2023-11-24T06:12:12.537 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:12:12.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:12.746+0000 7f28b2a2b640 1 Processor -- start 2023-11-24T06:12:12.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:12.746+0000 7f28b2a2b640 1 -- start start 2023-11-24T06:12:12.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:12.747+0000 7f28b2a2b640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f28ac143ae0 0x7f28ac0cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:12.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:12.747+0000 7f28b2a2b640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f28ac0cf7d0 0x7f28ac15dca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:12.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:12.747+0000 7f28b2a2b640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f28ac15e1e0 0x7f28ac15a670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:12.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:12.747+0000 7f28b2a2b640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28ac0cfc30 con 0x7f28ac0cf7d0 2023-11-24T06:12:12.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:12.748+0000 7f28b2a2b640 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28ac0b1ee0 con 0x7f28ac143ae0 2023-11-24T06:12:12.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:12.748+0000 7f28b2a2b640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28ac0b2780 con 0x7f28ac15e1e0 2023-11-24T06:12:12.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:12.748+0000 7f28a3fff640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f28ac0cf7d0 0x7f28ac15dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:12.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:12.748+0000 7f28a3fff640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f28ac0cf7d0 0x7f28ac15dca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:60478/0 (socket says 172.21.15.92:60478) 2023-11-24T06:12:12.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:12.748+0000 7f28a3fff640 1 -- 172.21.15.92:0/3148889436 learned_addr learned my addr 172.21.15.92:0/3148889436 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:12:12.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:12.748+0000 7f28abfff640 1 --2- 172.21.15.92:0/3148889436 >> v2:172.21.15.118:3300/0 conn(0x7f28ac143ae0 0x7f28ac0cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:12.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:12.748+0000 7f28a3fff640 1 -- 172.21.15.92:0/3148889436 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f28ac15e1e0 msgr2=0x7f28ac15a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:12.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:12.748+0000 7f28a3fff640 1 --2- 172.21.15.92:0/3148889436 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f28ac15e1e0 0x7f28ac15a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:12.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:12.748+0000 7f28a3fff640 1 -- 172.21.15.92:0/3148889436 >> v2:172.21.15.118:3300/0 conn(0x7f28ac143ae0 msgr2=0x7f28ac0cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:12.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:12.748+0000 7f28a3fff640 1 --2- 172.21.15.92:0/3148889436 >> v2:172.21.15.118:3300/0 conn(0x7f28ac143ae0 0x7f28ac0cf290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:12.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:12.748+0000 7f28a3fff640 1 -- 172.21.15.92:0/3148889436 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28ac15e990 con 0x7f28ac0cf7d0 2023-11-24T06:12:12.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:12.749+0000 7f28a3fff640 1 --2- 172.21.15.92:0/3148889436 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f28ac0cf7d0 0x7f28ac15dca0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f28ac0cfea0 tx=0x7f289402e0d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9888b3328b390c1a server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:12.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:12.750+0000 7f28ab7fe640 1 -- 172.21.15.92:0/3148889436 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f289402ee80 con 0x7f28ac0cf7d0 2023-11-24T06:12:12.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:12.750+0000 7f28ab7fe640 1 -- 172.21.15.92:0/3148889436 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2894036440 con 0x7f28ac0cf7d0 2023-11-24T06:12:12.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:12.750+0000 7f28ab7fe640 1 -- 172.21.15.92:0/3148889436 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28940367b0 con 0x7f28ac0cf7d0 2023-11-24T06:12:12.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:12.750+0000 7f28b2a2b640 1 -- 172.21.15.92:0/3148889436 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f28ac0cf7d0 msgr2=0x7f28ac15dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:12.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:12.750+0000 7f28b2a2b640 1 --2- 172.21.15.92:0/3148889436 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f28ac0cf7d0 0x7f28ac15dca0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f28ac0cfea0 tx=0x7f289402e0d0 comp rx=0 tx=0).stop 2023-11-24T06:12:12.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:12.750+0000 7f28b2a2b640 1 -- 172.21.15.92:0/3148889436 shutdown_connections 2023-11-24T06:12:12.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:12.750+0000 7f28b2a2b640 1 --2- 172.21.15.92:0/3148889436 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f28ac15e1e0 0x7f28ac15a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:12.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:12.750+0000 7f28b2a2b640 1 --2- 172.21.15.92:0/3148889436 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f28ac0cf7d0 0x7f28ac15dca0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:12.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:12.750+0000 7f28b2a2b640 1 --2- 172.21.15.92:0/3148889436 >> v2:172.21.15.118:3300/0 conn(0x7f28ac143ae0 0x7f28ac0cf290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:12.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:12.750+0000 7f28b2a2b640 1 -- 172.21.15.92:0/3148889436 >> 172.21.15.92:0/3148889436 conn(0x7f28ac06a800 msgr2=0x7f28ac06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:12.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:12.750+0000 7f28b2a2b640 1 -- 172.21.15.92:0/3148889436 shutdown_connections 2023-11-24T06:12:12.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:12.750+0000 7f28b2a2b640 1 -- 172.21.15.92:0/3148889436 wait complete. 2023-11-24T06:12:12.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:12.752+0000 7f28b2a2b640 1 Processor -- start 2023-11-24T06:12:12.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:12.752+0000 7f28b2a2b640 1 -- start start 2023-11-24T06:12:12.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:12.753+0000 7f28b2a2b640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f28ac0cf7d0 0x7f28ac141900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:12.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:12.753+0000 7f28b2a2b640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f28ac143ae0 0x7f28ac0cd470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:12.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:12.753+0000 7f28b2a2b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f28ac15e1e0 0x7f28ac0cd9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:12.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:12.753+0000 7f28b2a2b640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28ac1f3350 con 0x7f28ac15e1e0 2023-11-24T06:12:12.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:12.753+0000 7f28b2a2b640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28ac0adcd0 con 0x7f28ac0cf7d0 2023-11-24T06:12:12.758 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:12.753+0000 7f28b2a2b640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28ac0adfb0 con 0x7f28ac143ae0 2023-11-24T06:12:12.758 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:12.754+0000 7f28b0fc8640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f28ac15e1e0 0x7f28ac0cd9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:12.758 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:12.754+0000 7f28b0fc8640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f28ac15e1e0 0x7f28ac0cd9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:60488/0 (socket says 172.21.15.92:60488) 2023-11-24T06:12:12.758 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:12.754+0000 7f28b0fc8640 1 -- 172.21.15.92:0/3742151489 learned_addr learned my addr 172.21.15.92:0/3742151489 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:12:12.758 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:12.754+0000 7f28abfff640 1 --2- 172.21.15.92:0/3742151489 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f28ac0cf7d0 0x7f28ac141900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:12.758 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:12.754+0000 7f28a3fff640 1 --2- 172.21.15.92:0/3742151489 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f28ac143ae0 0x7f28ac0cd470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:12.758 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:12.754+0000 7f28b0fc8640 1 -- 172.21.15.92:0/3742151489 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f28ac143ae0 msgr2=0x7f28ac0cd470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:12.758 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:12.754+0000 7f28b0fc8640 1 --2- 172.21.15.92:0/3742151489 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f28ac143ae0 0x7f28ac0cd470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:12.758 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:12.754+0000 7f28b0fc8640 1 -- 172.21.15.92:0/3742151489 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f28ac0cf7d0 msgr2=0x7f28ac141900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:12.758 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:12.754+0000 7f28b0fc8640 1 --2- 172.21.15.92:0/3742151489 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f28ac0cf7d0 0x7f28ac141900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:12.758 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:12.754+0000 7f28b0fc8640 1 -- 172.21.15.92:0/3742151489 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f289400b770 con 0x7f28ac15e1e0 2023-11-24T06:12:12.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:12.755+0000 7f28b0fc8640 1 --2- 172.21.15.92:0/3742151489 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f28ac15e1e0 0x7f28ac0cd9b0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f28a400b970 tx=0x7f28a400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:12.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:12.755+0000 7f28a97fa640 1 -- 172.21.15.92:0/3742151489 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28a400cb40 con 0x7f28ac15e1e0 2023-11-24T06:12:12.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:12.755+0000 7f28a97fa640 1 -- 172.21.15.92:0/3742151489 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28a400ccc0 con 0x7f28ac15e1e0 2023-11-24T06:12:12.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:12.755+0000 7f28a97fa640 1 -- 172.21.15.92:0/3742151489 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28a4014680 con 0x7f28ac15e1e0 2023-11-24T06:12:12.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:12.755+0000 7f28b2a2b640 1 -- 172.21.15.92:0/3742151489 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28ac0ae290 con 0x7f28ac15e1e0 2023-11-24T06:12:12.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:12.755+0000 7f28b2a2b640 1 -- 172.21.15.92:0/3742151489 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28ac141e40 con 0x7f28ac15e1e0 2023-11-24T06:12:12.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:12.756+0000 7f28b2a2b640 1 -- 172.21.15.92:0/3742151489 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28ac142030 con 0x7f28ac15e1e0 2023-11-24T06:12:12.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:12.757+0000 7f28a97fa640 1 -- 172.21.15.92:0/3742151489 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f28a401d3b0 con 0x7f28ac15e1e0 2023-11-24T06:12:12.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:12.758+0000 7f28a97fa640 1 --2- 172.21.15.92:0/3742151489 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f288c06ae40 0x7f288c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:12.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:12.759+0000 7f28a97fa640 1 -- 172.21.15.92:0/3742151489 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f28a40896c0 con 0x7f28ac15e1e0 2023-11-24T06:12:12.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:12.759+0000 7f28abfff640 1 --2- 172.21.15.92:0/3742151489 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f288c06ae40 0x7f288c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:12.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:12.759+0000 7f28abfff640 1 --2- 172.21.15.92:0/3742151489 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f288c06ae40 0x7f288c06d300 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f289c00a970 tx=0x7f289c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:12.766 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:12.763+0000 7f28a97fa640 1 -- 172.21.15.92:0/3742151489 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f28a4054670 con 0x7f28ac15e1e0 2023-11-24T06:12:12.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:12.955+0000 7f28b2a2b640 1 -- 172.21.15.92:0/3742151489 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f28ac142940 con 0x7f28ac15e1e0 2023-11-24T06:12:12.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:12.956+0000 7f28a97fa640 1 -- 172.21.15.92:0/3742151489 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+16650 (secure 0 0 0) 0x7f28a4057c90 con 0x7f28ac15e1e0 2023-11-24T06:12:12.960 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-24T06:12:12.960 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":40,"fsid":"2d6dee5f-8aa3-470f-931a-50ade963b1cb","created":"2023-11-24T06:10:04.707204+0000","modified":"2023-11-24T06:12:06.456223+0000","last_up_change":"2023-11-24T06:10:17.598352+0000","last_in_change":"2023-11-24T06:10:09.040793+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:10:17.733098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:11:42.241681+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-11-24T06:11:43.522720+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ff37a9fb-135f-471f-b4b0-f7b5772006c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6803","nonce":20560714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6805","nonce":20560714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6809","nonce":20560714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6807","nonce":20560714}]},"public_addr":"172.21.15.92:6803/20560714","cluster_addr":"172.21.15.92:6805/20560714","heartbeat_back_addr":"172.21.15.92:6809/20560714","heartbeat_front_addr":"172.21.15.92:6807/20560714","state":["exists","up"]},{"osd":1,"uuid":"03cdd353-dcf1-4f38-94fe-a795137c06b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6819","nonce":2451239906}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6821","nonce":2451239906}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6825","nonce":2451239906}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6823","nonce":2451239906}]},"public_addr":"172.21.15.92:6819/2451239906","cluster_addr":"172.21.15.92:6821/2451239906","heartbeat_back_addr":"172.21.15.92:6825/2451239906","heartbeat_front_addr":"172.21.15.92:6823/2451239906","state":["exists","up"]},{"osd":2,"uuid":"a4a20d80-5d57-484b-becd-9aa8070dd3aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6811","nonce":1124979277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6813","nonce":1124979277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6817","nonce":1124979277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6815","nonce":1124979277}]},"public_addr":"172.21.15.92:6811/1124979277","cluster_addr":"172.21.15.92:6813/1124979277","heartbeat_back_addr":"172.21.15.92:6817/1124979277","heartbeat_front_addr":"172.21.15.92:6815/1124979277","state":["exists","up"]},{"osd":3,"uuid":"4648d7ce-9b29-4583-9425-4ba8fd2b1903","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6827","nonce":1734115782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6829","nonce":1734115782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6833","nonce":1734115782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6831","nonce":1734115782}]},"public_addr":"172.21.15.92:6827/1734115782","cluster_addr":"172.21.15.92:6829/1734115782","heartbeat_back_addr":"172.21.15.92:6833/1734115782","heartbeat_front_addr":"172.21.15.92:6831/1734115782","state":["exists","up"]},{"osd":4,"uuid":"3bc9d7eb-c6fa-4fcf-a3ec-ade604e05aac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6819","nonce":3425937229}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6822","nonce":3425937229}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6830","nonce":3425937229}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6826","nonce":3425937229}]},"public_addr":"172.21.15.118:6819/3425937229","cluster_addr":"172.21.15.118:6822/3425937229","heartbeat_back_addr":"172.21.15.118:6830/3425937229","heartbeat_front_addr":"172.21.15.118:6826/3425937229","state":["exists","up"]},{"osd":5,"uuid":"dce9ba6d-59ed-46d1-84fc-a84b13de10c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6801","nonce":827455984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6803","nonce":827455984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6807","nonce":827455984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6805","nonce":827455984}]},"public_addr":"172.21.15.118:6801/827455984","cluster_addr":"172.21.15.118:6803/827455984","heartbeat_back_addr":"172.21.15.118:6807/827455984","heartbeat_front_addr":"172.21.15.118:6805/827455984","state":["exists","up"]},{"osd":6,"uuid":"82aec504-ae58-4709-8d30-46f2b12081b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6809","nonce":2946986141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6811","nonce":2946986141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6815","nonce":2946986141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6813","nonce":2946986141}]},"public_addr":"172.21.15.118:6809/2946986141","cluster_addr":"172.21.15.118:6811/2946986141","heartbeat_back_addr":"172.21.15.118:6815/2946986141","heartbeat_front_addr":"172.21.15.118:6813/2946986141","state":["exists","up"]},{"osd":7,"uuid":"4ed4c624-c88f-4d35-bdb1-bf209b5dffb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6817","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6818","nonce":48122059}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6821","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6823","nonce":48122059}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6829","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6831","nonce":48122059}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6825","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6827","nonce":48122059}]},"public_addr":"172.21.15.118:6818/48122059","cluster_addr":"172.21.15.118:6823/48122059","heartbeat_back_addr":"172.21.15.118:6831/48122059","heartbeat_front_addr":"172.21.15.118:6827/48122059","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.536664+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.368324+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.271201+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.342971+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.621832+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.387110+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.632946+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.416032+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.92:6836/3889219158":"2023-11-25T06:12:03.443855+0000","172.21.15.92:6834/3889219158":"2023-11-25T06:12:03.443855+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:12:12.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:12.959+0000 7f28b2a2b640 1 -- 172.21.15.92:0/3742151489 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f288c06ae40 msgr2=0x7f288c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:12.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:12.959+0000 7f28b2a2b640 1 --2- 172.21.15.92:0/3742151489 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f288c06ae40 0x7f288c06d300 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f289c00a970 tx=0x7f289c008040 comp rx=0 tx=0).stop 2023-11-24T06:12:12.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:12.959+0000 7f28b2a2b640 1 -- 172.21.15.92:0/3742151489 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f28ac15e1e0 msgr2=0x7f28ac0cd9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:12.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:12.959+0000 7f28b2a2b640 1 --2- 172.21.15.92:0/3742151489 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f28ac15e1e0 0x7f28ac0cd9b0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f28a400b970 tx=0x7f28a400be30 comp rx=0 tx=0).stop 2023-11-24T06:12:12.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:12.959+0000 7f28b2a2b640 1 -- 172.21.15.92:0/3742151489 shutdown_connections 2023-11-24T06:12:12.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:12.959+0000 7f28b2a2b640 1 --2- 172.21.15.92:0/3742151489 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f288c06ae40 0x7f288c06d300 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:12.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:12.959+0000 7f28b2a2b640 1 --2- 172.21.15.92:0/3742151489 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f28ac15e1e0 0x7f28ac0cd9b0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:12.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:12.959+0000 7f28b2a2b640 1 --2- 172.21.15.92:0/3742151489 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f28ac143ae0 0x7f28ac0cd470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:12.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:12.959+0000 7f28b2a2b640 1 --2- 172.21.15.92:0/3742151489 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f28ac0cf7d0 0x7f28ac141900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:12.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:12.959+0000 7f28b2a2b640 1 -- 172.21.15.92:0/3742151489 >> 172.21.15.92:0/3742151489 conn(0x7f28ac06a800 msgr2=0x7f28ac15a4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:12.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:12.959+0000 7f28b2a2b640 1 -- 172.21.15.92:0/3742151489 shutdown_connections 2023-11-24T06:12:12.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:12.959+0000 7f28b2a2b640 1 -- 172.21.15.92:0/3742151489 wait complete. 2023-11-24T06:12:12.981 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T06:10:17.733098+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T06:11:42.241681+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '32', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T06:11:43.522720+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '33', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T06:12:12.982 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T06:12:13.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:13.193+0000 7ffbb2cca640 1 Processor -- start 2023-11-24T06:12:13.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:13.193+0000 7ffbb2cca640 1 -- start start 2023-11-24T06:12:13.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:13.194+0000 7ffbb2cca640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ffbac13cf10 0x7ffbac13f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:13.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:13.195+0000 7ffbb2cca640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ffbac13fae0 0x7ffbac0cd6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:13.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:13.195+0000 7ffbb2cca640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7ffbac0cdc20 0x7ffbac15d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:13.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:13.195+0000 7ffbb2cca640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffbac13ff60 con 0x7ffbac13cf10 2023-11-24T06:12:13.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:13.195+0000 7ffbb2cca640 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffbac0ce440 con 0x7ffbac0cdc20 2023-11-24T06:12:13.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:13.195+0000 7ffbb2cca640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbac0ce720 con 0x7ffbac13fae0 2023-11-24T06:12:13.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:13.195+0000 7ffbb1cc8640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ffbac13cf10 0x7ffbac13f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:13.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:13.195+0000 7ffbb1cc8640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ffbac13cf10 0x7ffbac13f5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:60498/0 (socket says 172.21.15.92:60498) 2023-11-24T06:12:13.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:13.195+0000 7ffbb1cc8640 1 -- 172.21.15.92:0/1047988679 learned_addr learned my addr 172.21.15.92:0/1047988679 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:12:13.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:13.195+0000 7ffbb14c7640 1 --2- 172.21.15.92:0/1047988679 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ffbac13fae0 0x7ffbac0cd6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:13.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:13.195+0000 7ffbb1cc8640 1 -- 172.21.15.92:0/1047988679 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ffbac13fae0 msgr2=0x7ffbac0cd6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:13.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:13.195+0000 7ffbb1cc8640 1 --2- 172.21.15.92:0/1047988679 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ffbac13fae0 0x7ffbac0cd6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:13.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:13.195+0000 7ffbb1cc8640 1 -- 172.21.15.92:0/1047988679 >> v2:172.21.15.118:3300/0 conn(0x7ffbac0cdc20 msgr2=0x7ffbac15d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:13.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:13.195+0000 7ffbb24c9640 1 --2- 172.21.15.92:0/1047988679 >> v2:172.21.15.118:3300/0 conn(0x7ffbac0cdc20 0x7ffbac15d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:13.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:13.195+0000 7ffbb1cc8640 1 --2- 172.21.15.92:0/1047988679 >> v2:172.21.15.118:3300/0 conn(0x7ffbac0cdc20 0x7ffbac15d9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:13.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:13.195+0000 7ffbb1cc8640 1 -- 172.21.15.92:0/1047988679 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbac0cea00 con 0x7ffbac13cf10 2023-11-24T06:12:13.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:13.195+0000 7ffbb24c9640 1 --2- 172.21.15.92:0/1047988679 >> v2:172.21.15.118:3300/0 conn(0x7ffbac0cdc20 0x7ffbac15d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:12:13.201 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:13.196+0000 7ffbb1cc8640 1 --2- 172.21.15.92:0/1047988679 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffbac13cf10 0x7ffbac13f5a0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7ffbac154a00 tx=0x7ffba802b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e805cf9c0b09845 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:13.201 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:13.197+0000 7ffbb0cc6640 1 -- 172.21.15.92:0/1047988679 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffba8031860 con 0x7ffbac13cf10 2023-11-24T06:12:13.201 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:13.197+0000 7ffbb0cc6640 1 -- 172.21.15.92:0/1047988679 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffba80319c0 con 0x7ffbac13cf10 2023-11-24T06:12:13.201 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:13.197+0000 7ffbb0cc6640 1 -- 172.21.15.92:0/1047988679 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffba8031d50 con 0x7ffbac13cf10 2023-11-24T06:12:13.201 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:13.197+0000 7ffbb2cca640 1 -- 172.21.15.92:0/1047988679 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffbac13cf10 msgr2=0x7ffbac13f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:13.202 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:13.197+0000 7ffbb2cca640 1 --2- 172.21.15.92:0/1047988679 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffbac13cf10 0x7ffbac13f5a0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7ffbac154a00 tx=0x7ffba802b900 comp rx=0 tx=0).stop 2023-11-24T06:12:13.202 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:13.197+0000 7ffbb2cca640 1 -- 172.21.15.92:0/1047988679 shutdown_connections 2023-11-24T06:12:13.202 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:13.197+0000 7ffbb2cca640 1 --2- 172.21.15.92:0/1047988679 >> v2:172.21.15.118:3300/0 conn(0x7ffbac0cdc20 0x7ffbac15d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:13.202 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:13.197+0000 7ffbb2cca640 1 --2- 172.21.15.92:0/1047988679 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ffbac13fae0 0x7ffbac0cd6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:13.202 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:13.197+0000 7ffbb2cca640 1 --2- 172.21.15.92:0/1047988679 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffbac13cf10 0x7ffbac13f5a0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:13.202 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:13.197+0000 7ffbb2cca640 1 -- 172.21.15.92:0/1047988679 >> 172.21.15.92:0/1047988679 conn(0x7ffbac06a7d0 msgr2=0x7ffbac06abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:13.202 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:13.198+0000 7ffbb2cca640 1 -- 172.21.15.92:0/1047988679 shutdown_connections 2023-11-24T06:12:13.202 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:13.198+0000 7ffbb2cca640 1 -- 172.21.15.92:0/1047988679 wait complete. 2023-11-24T06:12:13.203 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:13.199+0000 7ffbb2cca640 1 Processor -- start 2023-11-24T06:12:13.203 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:13.199+0000 7ffbb2cca640 1 -- start start 2023-11-24T06:12:13.204 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:13.200+0000 7ffbb2cca640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffbac0cdc20 0x7ffbac13aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:13.205 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:13.200+0000 7ffbb2cca640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ffbac13cf10 0x7ffbac139060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:13.205 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:13.200+0000 7ffbb2cca640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ffbac13fae0 0x7ffbac138110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:13.205 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:13.200+0000 7ffbb2cca640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbac15bcc0 con 0x7ffbac0cdc20 2023-11-24T06:12:13.205 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:13.200+0000 7ffbb2cca640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbac1f2360 con 0x7ffbac13cf10 2023-11-24T06:12:13.205 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:13.200+0000 7ffbb2cca640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbac1f2640 con 0x7ffbac13fae0 2023-11-24T06:12:13.205 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:13.201+0000 7ffbb1cc8640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffbac0cdc20 0x7ffbac13aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:13.205 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:13.201+0000 7ffbb1cc8640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffbac0cdc20 0x7ffbac13aa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:60504/0 (socket says 172.21.15.92:60504) 2023-11-24T06:12:13.205 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:13.201+0000 7ffbb1cc8640 1 -- 172.21.15.92:0/2195557704 learned_addr learned my addr 172.21.15.92:0/2195557704 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:12:13.205 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:13.201+0000 7ffbb1cc8640 1 -- 172.21.15.92:0/2195557704 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ffbac13fae0 msgr2=0x7ffbac138110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:13.205 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:13.201+0000 7ffbb1cc8640 1 --2- 172.21.15.92:0/2195557704 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ffbac13fae0 0x7ffbac138110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:13.206 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:13.201+0000 7ffbb1cc8640 1 -- 172.21.15.92:0/2195557704 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ffbac13cf10 msgr2=0x7ffbac139060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:13.206 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:13.201+0000 7ffbb1cc8640 1 --2- 172.21.15.92:0/2195557704 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ffbac13cf10 0x7ffbac139060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:13.206 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:13.201+0000 7ffbb1cc8640 1 -- 172.21.15.92:0/2195557704 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffba80094d0 con 0x7ffbac0cdc20 2023-11-24T06:12:13.206 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:13.201+0000 7ffbb1cc8640 1 --2- 172.21.15.92:0/2195557704 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffbac0cdc20 0x7ffbac13aa70 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7ffba802b940 tx=0x7ffba80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:13.206 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:13.201+0000 7ffba27fc640 1 -- 172.21.15.92:0/2195557704 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffba8031850 con 0x7ffbac0cdc20 2023-11-24T06:12:13.206 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:13.202+0000 7ffba27fc640 1 -- 172.21.15.92:0/2195557704 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffba80443f0 con 0x7ffbac0cdc20 2023-11-24T06:12:13.206 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:13.202+0000 7ffba27fc640 1 -- 172.21.15.92:0/2195557704 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffba80447a0 con 0x7ffbac0cdc20 2023-11-24T06:12:13.206 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:13.202+0000 7ffbb2cca640 1 -- 172.21.15.92:0/2195557704 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffbac15b600 con 0x7ffbac0cdc20 2023-11-24T06:12:13.206 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:13.202+0000 7ffbb2cca640 1 -- 172.21.15.92:0/2195557704 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbac1395a0 con 0x7ffbac0cdc20 2023-11-24T06:12:13.207 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:13.203+0000 7ffbb2cca640 1 -- 172.21.15.92:0/2195557704 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffbac15be00 con 0x7ffbac0cdc20 2023-11-24T06:12:13.207 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:13.204+0000 7ffba27fc640 1 -- 172.21.15.92:0/2195557704 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ffba8044900 con 0x7ffbac0cdc20 2023-11-24T06:12:13.208 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:13.205+0000 7ffba27fc640 1 --2- 172.21.15.92:0/2195557704 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7ffb9006ae40 0x7ffb9006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:13.208 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:13.205+0000 7ffba27fc640 1 -- 172.21.15.92:0/2195557704 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5795+0+0 (secure 0 0 0) 0x7ffba80abe30 con 0x7ffbac0cdc20 2023-11-24T06:12:13.208 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:13.205+0000 7ffbb14c7640 1 --2- 172.21.15.92:0/2195557704 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7ffb9006ae40 0x7ffb9006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:13.208 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:13.206+0000 7ffbb14c7640 1 --2- 172.21.15.92:0/2195557704 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7ffb9006ae40 0x7ffb9006d300 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7ffbac15b030 tx=0x7ffb9c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:13.212 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:13.209+0000 7ffba27fc640 1 -- 172.21.15.92:0/2195557704 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ffba8076f90 con 0x7ffbac0cdc20 2023-11-24T06:12:13.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:13.399+0000 7ffbb2cca640 1 -- 172.21.15.92:0/2195557704 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ffbac067450 con 0x7ffbac0cdc20 2023-11-24T06:12:13.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:13.400+0000 7ffba27fc640 1 -- 172.21.15.92:0/2195557704 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v40) v1 ==== 93+0+10 (secure 0 0 0) 0x7ffba8035020 con 0x7ffbac0cdc20 2023-11-24T06:12:13.403 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 1 2023-11-24T06:12:13.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:13.402+0000 7ffbb2cca640 1 -- 172.21.15.92:0/2195557704 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7ffb9006ae40 msgr2=0x7ffb9006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:13.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:13.402+0000 7ffbb2cca640 1 --2- 172.21.15.92:0/2195557704 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7ffb9006ae40 0x7ffb9006d300 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7ffbac15b030 tx=0x7ffb9c008040 comp rx=0 tx=0).stop 2023-11-24T06:12:13.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:13.402+0000 7ffbb2cca640 1 -- 172.21.15.92:0/2195557704 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffbac0cdc20 msgr2=0x7ffbac13aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:13.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:13.402+0000 7ffbb2cca640 1 --2- 172.21.15.92:0/2195557704 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffbac0cdc20 0x7ffbac13aa70 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7ffba802b940 tx=0x7ffba80046c0 comp rx=0 tx=0).stop 2023-11-24T06:12:13.406 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:13.402+0000 7ffbb2cca640 1 -- 172.21.15.92:0/2195557704 shutdown_connections 2023-11-24T06:12:13.406 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:13.402+0000 7ffbb2cca640 1 --2- 172.21.15.92:0/2195557704 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7ffb9006ae40 0x7ffb9006d300 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:13.406 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:13.402+0000 7ffbb2cca640 1 --2- 172.21.15.92:0/2195557704 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ffbac13fae0 0x7ffbac138110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:13.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:13.402+0000 7ffbb2cca640 1 --2- 172.21.15.92:0/2195557704 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ffbac13cf10 0x7ffbac139060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:13.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:13.402+0000 7ffbb2cca640 1 --2- 172.21.15.92:0/2195557704 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffbac0cdc20 0x7ffbac13aa70 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:13.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:13.402+0000 7ffbb2cca640 1 -- 172.21.15.92:0/2195557704 >> 172.21.15.92:0/2195557704 conn(0x7ffbac06a7d0 msgr2=0x7ffbac15a430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:13.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:13.403+0000 7ffbb2cca640 1 -- 172.21.15.92:0/2195557704 shutdown_connections 2023-11-24T06:12:13.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:13.403+0000 7ffbb2cca640 1 -- 172.21.15.92:0/2195557704 wait complete. 2023-11-24T06:12:13.423 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T06:12:13.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:13.626+0000 7ff74acd0640 1 Processor -- start 2023-11-24T06:12:13.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:13.627+0000 7ff74acd0640 1 -- start start 2023-11-24T06:12:13.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:13.628+0000 7ff74acd0640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ff744143ae0 0x7ff7440cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:13.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:13.628+0000 7ff74acd0640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff7440cf7d0 0x7ff74415dca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:13.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:13.628+0000 7ff74acd0640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7440cfc30 con 0x7ff74415e1e0 2023-11-24T06:12:13.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:13.628+0000 7ff74acd0640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7440b1ee0 con 0x7ff744143ae0 2023-11-24T06:12:13.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:13.628+0000 7ff74acd0640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7440b2780 con 0x7ff7440cf7d0 2023-11-24T06:12:13.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:13.629+0000 7ff748a6c640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ff744143ae0 0x7ff7440cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:13.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:13.629+0000 7ff748a6c640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ff744143ae0 0x7ff7440cf290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:60512/0 (socket says 172.21.15.92:60512) 2023-11-24T06:12:13.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:13.629+0000 7ff748a6c640 1 -- 172.21.15.92:0/3031464524 learned_addr learned my addr 172.21.15.92:0/3031464524 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:12:13.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:13.629+0000 7ff743fff640 1 --2- 172.21.15.92:0/3031464524 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff7440cf7d0 0x7ff74415dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:13.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:13.629+0000 7ff748a6c640 1 -- 172.21.15.92:0/3031464524 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff7440cf7d0 msgr2=0x7ff74415dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:13.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:13.629+0000 7ff748a6c640 1 --2- 172.21.15.92:0/3031464524 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff7440cf7d0 0x7ff74415dca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:13.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:13.629+0000 7ff748a6c640 1 -- 172.21.15.92:0/3031464524 >> v1:172.21.15.92:6789/0 conn(0x7ff74415e1e0 legacy=0x7ff74415a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:13.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:13.629+0000 7ff748a6c640 1 -- 172.21.15.92:0/3031464524 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff74415e990 con 0x7ff744143ae0 2023-11-24T06:12:13.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:13.630+0000 7ff748a6c640 1 --2- 172.21.15.92:0/3031464524 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff744143ae0 0x7ff7440cf290 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7ff738009870 tx=0x7ff73802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6baf7251683876c3 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:13.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:13.630+0000 7ff7437fe640 1 -- 172.21.15.92:0/3031464524 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff738030840 con 0x7ff744143ae0 2023-11-24T06:12:13.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:13.630+0000 7ff7437fe640 1 -- 172.21.15.92:0/3031464524 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7380309a0 con 0x7ff744143ae0 2023-11-24T06:12:13.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:13.630+0000 7ff7437fe640 1 -- 172.21.15.92:0/3031464524 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff738030d30 con 0x7ff744143ae0 2023-11-24T06:12:13.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:13.631+0000 7ff74acd0640 1 -- 172.21.15.92:0/3031464524 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff744143ae0 msgr2=0x7ff7440cf290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:13.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:13.631+0000 7ff74acd0640 1 --2- 172.21.15.92:0/3031464524 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff744143ae0 0x7ff7440cf290 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7ff738009870 tx=0x7ff73802b980 comp rx=0 tx=0).stop 2023-11-24T06:12:13.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:13.631+0000 7ff74acd0640 1 -- 172.21.15.92:0/3031464524 shutdown_connections 2023-11-24T06:12:13.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:13.631+0000 7ff74acd0640 1 --2- 172.21.15.92:0/3031464524 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff7440cf7d0 0x7ff74415dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:13.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:13.631+0000 7ff74acd0640 1 --2- 172.21.15.92:0/3031464524 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff744143ae0 0x7ff7440cf290 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:13.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:13.631+0000 7ff74acd0640 1 -- 172.21.15.92:0/3031464524 >> 172.21.15.92:0/3031464524 conn(0x7ff74406a800 msgr2=0x7ff74406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:13.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:13.631+0000 7ff74acd0640 1 -- 172.21.15.92:0/3031464524 shutdown_connections 2023-11-24T06:12:13.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:13.631+0000 7ff74acd0640 1 -- 172.21.15.92:0/3031464524 wait complete. 2023-11-24T06:12:13.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:13.633+0000 7ff74acd0640 1 Processor -- start 2023-11-24T06:12:13.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:13.633+0000 7ff74acd0640 1 -- start start 2023-11-24T06:12:13.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:13.634+0000 7ff74acd0640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff7440cf7d0 0x7ff744142350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:13.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:13.634+0000 7ff74acd0640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff744143ae0 0x7ff744141d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:13.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:13.634+0000 7ff74acd0640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff74415e1e0 0x7ff74415bf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:13.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:13.634+0000 7ff74acd0640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7441e3d00 con 0x7ff744143ae0 2023-11-24T06:12:13.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:13.634+0000 7ff74acd0640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff74415c480 con 0x7ff7440cf7d0 2023-11-24T06:12:13.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:13.634+0000 7ff74acd0640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff74415c700 con 0x7ff74415e1e0 2023-11-24T06:12:13.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:13.634+0000 7ff743fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff744143ae0 0x7ff744141d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:13.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:13.635+0000 7ff743fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff744143ae0 0x7ff744141d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:60522/0 (socket says 172.21.15.92:60522) 2023-11-24T06:12:13.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:13.635+0000 7ff743fff640 1 -- 172.21.15.92:0/3916537336 learned_addr learned my addr 172.21.15.92:0/3916537336 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:12:13.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:13.635+0000 7ff748a6c640 1 --2- 172.21.15.92:0/3916537336 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff7440cf7d0 0x7ff744142350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:13.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:13.635+0000 7ff743fff640 1 -- 172.21.15.92:0/3916537336 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff74415e1e0 msgr2=0x7ff74415bf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:13.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:13.635+0000 7ff74926d640 1 --2- 172.21.15.92:0/3916537336 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff74415e1e0 0x7ff74415bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:13.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:13.635+0000 7ff743fff640 1 --2- 172.21.15.92:0/3916537336 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff74415e1e0 0x7ff74415bf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:13.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:13.635+0000 7ff743fff640 1 -- 172.21.15.92:0/3916537336 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff7440cf7d0 msgr2=0x7ff744142350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:13.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:13.635+0000 7ff743fff640 1 --2- 172.21.15.92:0/3916537336 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff7440cf7d0 0x7ff744142350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:13.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:13.635+0000 7ff743fff640 1 -- 172.21.15.92:0/3916537336 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7380094d0 con 0x7ff744143ae0 2023-11-24T06:12:13.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:13.635+0000 7ff74926d640 1 --2- 172.21.15.92:0/3916537336 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff74415e1e0 0x7ff74415bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:12:13.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:13.635+0000 7ff748a6c640 1 --2- 172.21.15.92:0/3916537336 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff7440cf7d0 0x7ff744142350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:12:13.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:13.635+0000 7ff743fff640 1 --2- 172.21.15.92:0/3916537336 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff744143ae0 0x7ff744141d60 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7ff73000c7f0 tx=0x7ff73000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:13.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:13.636+0000 7ff7417fa640 1 -- 172.21.15.92:0/3916537336 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff730004220 con 0x7ff744143ae0 2023-11-24T06:12:13.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:13.636+0000 7ff7417fa640 1 -- 172.21.15.92:0/3916537336 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7300043a0 con 0x7ff744143ae0 2023-11-24T06:12:13.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:13.636+0000 7ff7417fa640 1 -- 172.21.15.92:0/3916537336 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff730007860 con 0x7ff744143ae0 2023-11-24T06:12:13.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:13.636+0000 7ff74acd0640 1 -- 172.21.15.92:0/3916537336 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff74415c9e0 con 0x7ff744143ae0 2023-11-24T06:12:13.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:13.636+0000 7ff74acd0640 1 -- 172.21.15.92:0/3916537336 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7441e3f10 con 0x7ff744143ae0 2023-11-24T06:12:13.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:13.637+0000 7ff74acd0640 1 -- 172.21.15.92:0/3916537336 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7440b2bf0 con 0x7ff744143ae0 2023-11-24T06:12:13.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:13.638+0000 7ff7417fa640 1 -- 172.21.15.92:0/3916537336 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ff730007bd0 con 0x7ff744143ae0 2023-11-24T06:12:13.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:13.639+0000 7ff7417fa640 1 --2- 172.21.15.92:0/3916537336 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7ff72806ae40 0x7ff72806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:13.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:13.639+0000 7ff7417fa640 1 -- 172.21.15.92:0/3916537336 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5795+0+0 (secure 0 0 0) 0x7ff730089610 con 0x7ff744143ae0 2023-11-24T06:12:13.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:13.639+0000 7ff748a6c640 1 --2- 172.21.15.92:0/3916537336 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7ff72806ae40 0x7ff72806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:13.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:13.640+0000 7ff748a6c640 1 --2- 172.21.15.92:0/3916537336 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7ff72806ae40 0x7ff72806d300 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7ff738036000 tx=0x7ff73803b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:13.646 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:13.643+0000 7ff7417fa640 1 -- 172.21.15.92:0/3916537336 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff7440b2bf0 con 0x7ff744143ae0 2023-11-24T06:12:13.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:13.833+0000 7ff74acd0640 1 -- 172.21.15.92:0/3916537336 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff7440b2bf0 con 0x7ff744143ae0 2023-11-24T06:12:13.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:13.834+0000 7ff7417fa640 1 -- 172.21.15.92:0/3916537336 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v40) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff7440b2bf0 con 0x7ff744143ae0 2023-11-24T06:12:13.837 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 32 2023-11-24T06:12:13.839 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:13.836+0000 7ff74acd0640 1 -- 172.21.15.92:0/3916537336 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7ff72806ae40 msgr2=0x7ff72806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:13.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:13.836+0000 7ff74acd0640 1 --2- 172.21.15.92:0/3916537336 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7ff72806ae40 0x7ff72806d300 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7ff738036000 tx=0x7ff73803b040 comp rx=0 tx=0).stop 2023-11-24T06:12:13.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:13.836+0000 7ff74acd0640 1 -- 172.21.15.92:0/3916537336 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff744143ae0 msgr2=0x7ff744141d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:13.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:13.836+0000 7ff74acd0640 1 --2- 172.21.15.92:0/3916537336 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff744143ae0 0x7ff744141d60 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7ff73000c7f0 tx=0x7ff73000ccb0 comp rx=0 tx=0).stop 2023-11-24T06:12:13.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:13.836+0000 7ff74acd0640 1 -- 172.21.15.92:0/3916537336 shutdown_connections 2023-11-24T06:12:13.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:13.836+0000 7ff74acd0640 1 --2- 172.21.15.92:0/3916537336 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7ff72806ae40 0x7ff72806d300 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:13.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:13.836+0000 7ff74acd0640 1 --2- 172.21.15.92:0/3916537336 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff74415e1e0 0x7ff74415bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:13.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:13.836+0000 7ff74acd0640 1 --2- 172.21.15.92:0/3916537336 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff744143ae0 0x7ff744141d60 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:13.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:13.836+0000 7ff74acd0640 1 --2- 172.21.15.92:0/3916537336 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff7440cf7d0 0x7ff744142350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:13.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:13.836+0000 7ff74acd0640 1 -- 172.21.15.92:0/3916537336 >> 172.21.15.92:0/3916537336 conn(0x7ff74406a800 msgr2=0x7ff7440ce620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:13.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:13.836+0000 7ff74acd0640 1 -- 172.21.15.92:0/3916537336 shutdown_connections 2023-11-24T06:12:13.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:13.837+0000 7ff74acd0640 1 -- 172.21.15.92:0/3916537336 wait complete. 2023-11-24T06:12:13.858 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T06:12:14.070 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.067+0000 7fcac6b7c640 1 Processor -- start 2023-11-24T06:12:14.071 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.067+0000 7fcac6b7c640 1 -- start start 2023-11-24T06:12:14.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.069+0000 7fcac6b7c640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fcac013e360 0x7fcac00cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:14.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.069+0000 7fcac6b7c640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcac015c5f0 con 0x7fcac015f9c0 2023-11-24T06:12:14.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.069+0000 7fcac6b7c640 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcac006c470 con 0x7fcac013df80 2023-11-24T06:12:14.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.069+0000 7fcac6b7c640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcac006c750 con 0x7fcac013e360 2023-11-24T06:12:14.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.069+0000 7fcac5119640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fcac015f9c0 0x7fcac00cd840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:33800/0 (socket says 172.21.15.92:33800) 2023-11-24T06:12:14.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.069+0000 7fcac5119640 1 -- 172.21.15.92:0/1433402741 learned_addr learned my addr 172.21.15.92:0/1433402741 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-24T06:12:14.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.070+0000 7fcabffff640 1 --2- 172.21.15.92:0/1433402741 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fcac013e360 0x7fcac00cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:14.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.070+0000 7fcabf7fe640 1 -- 172.21.15.92:0/1433402741 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 597705421 0 0) 0x7fcac015c5f0 con 0x7fcac015f9c0 2023-11-24T06:12:14.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.070+0000 7fcabf7fe640 1 -- 172.21.15.92:0/1433402741 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcab0003520 con 0x7fcac015f9c0 2023-11-24T06:12:14.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.070+0000 7fcabf7fe640 1 -- 172.21.15.92:0/1433402741 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4210183483 0 0) 0x7fcab0003520 con 0x7fcac015f9c0 2023-11-24T06:12:14.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.070+0000 7fcabf7fe640 1 -- 172.21.15.92:0/1433402741 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fcac013e360 msgr2=0x7fcac00cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:14.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.070+0000 7fcabf7fe640 1 --2- 172.21.15.92:0/1433402741 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fcac013e360 0x7fcac00cf290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:14.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.070+0000 7fcabf7fe640 1 -- 172.21.15.92:0/1433402741 >> v1:172.21.15.118:6789/0 conn(0x7fcac013df80 legacy=0x7fcac015bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:14.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.070+0000 7fcabf7fe640 1 -- 172.21.15.92:0/1433402741 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcac006ca30 con 0x7fcac015f9c0 2023-11-24T06:12:14.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.070+0000 7fcabf7fe640 1 -- 172.21.15.92:0/1433402741 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2607661350 0 0) 0x7fcab80019f0 con 0x7fcac015f9c0 2023-11-24T06:12:14.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.070+0000 7fcabf7fe640 1 -- 172.21.15.92:0/1433402741 >> v1:172.21.15.92:6789/0 conn(0x7fcac015f9c0 legacy=0x7fcac00cd840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:14.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.070+0000 7fcabf7fe640 1 --2- 172.21.15.92:0/1433402741 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fcab00039a0 0x7fcab0003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:14.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.070+0000 7fcabf7fe640 1 --2- 172.21.15.92:0/1433402741 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fcac013e360 0x7fcab00047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:14.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.070+0000 7fcabf7fe640 1 --2- 172.21.15.92:0/1433402741 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcab0004d00 0x7fcab0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:14.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.070+0000 7fcabf7fe640 1 -- 172.21.15.92:0/1433402741 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcac006c750 con 0x7fcab0004d00 2023-11-24T06:12:14.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.070+0000 7fcabf7fe640 1 -- 172.21.15.92:0/1433402741 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcab000ba90 con 0x7fcab00039a0 2023-11-24T06:12:14.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.070+0000 7fcabf7fe640 1 -- 172.21.15.92:0/1433402741 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcab000bc50 con 0x7fcac013e360 2023-11-24T06:12:14.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.071+0000 7fcac5119640 1 --2- 172.21.15.92:0/1433402741 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcab0004d00 0x7fcab0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:14.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.071+0000 7fcac4918640 1 --2- 172.21.15.92:0/1433402741 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fcab00039a0 0x7fcab0003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:14.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.071+0000 7fcabffff640 1 --2- 172.21.15.92:0/1433402741 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fcac013e360 0x7fcab00047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:14.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.071+0000 7fcac5119640 1 -- 172.21.15.92:0/1433402741 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fcac013e360 msgr2=0x7fcab00047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:14.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.071+0000 7fcac5119640 1 --2- 172.21.15.92:0/1433402741 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fcac013e360 0x7fcab00047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:14.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.071+0000 7fcac5119640 1 -- 172.21.15.92:0/1433402741 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fcab00039a0 msgr2=0x7fcab0003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:14.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.071+0000 7fcac5119640 1 --2- 172.21.15.92:0/1433402741 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fcab00039a0 0x7fcab0003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:14.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.071+0000 7fcac5119640 1 -- 172.21.15.92:0/1433402741 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcab0003520 con 0x7fcab0004d00 2023-11-24T06:12:14.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.072+0000 7fcac5119640 1 --2- 172.21.15.92:0/1433402741 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcab0004d00 0x7fcab0005180 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fcab8002230 tx=0x7fcab00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd3dbe508c3fe9d8 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:14.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.072+0000 7fcabf7fe640 1 -- 172.21.15.92:0/1433402741 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcab8034630 con 0x7fcab0004d00 2023-11-24T06:12:14.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.072+0000 7fcabf7fe640 1 -- 172.21.15.92:0/1433402741 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcab802f070 con 0x7fcab0004d00 2023-11-24T06:12:14.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.072+0000 7fcabf7fe640 1 -- 172.21.15.92:0/1433402741 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcab8034a10 con 0x7fcab0004d00 2023-11-24T06:12:14.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.073+0000 7fcac6b7c640 1 -- 172.21.15.92:0/1433402741 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcab0004d00 msgr2=0x7fcab0005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:14.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.073+0000 7fcac6b7c640 1 --2- 172.21.15.92:0/1433402741 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcab0004d00 0x7fcab0005180 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fcab8002230 tx=0x7fcab00057f0 comp rx=0 tx=0).stop 2023-11-24T06:12:14.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.073+0000 7fcac5119640 1 -- 172.21.15.92:0/1433402741 reap_dead start 2023-11-24T06:12:14.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.073+0000 7fcac6b7c640 1 -- 172.21.15.92:0/1433402741 shutdown_connections 2023-11-24T06:12:14.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.073+0000 7fcac6b7c640 1 -- 172.21.15.92:0/1433402741 >> 172.21.15.92:0/1433402741 conn(0x7fcac006a800 msgr2=0x7fcac006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:14.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.073+0000 7fcac6b7c640 1 -- 172.21.15.92:0/1433402741 shutdown_connections 2023-11-24T06:12:14.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.073+0000 7fcac6b7c640 1 -- 172.21.15.92:0/1433402741 wait complete. 2023-11-24T06:12:14.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.075+0000 7fcac6b7c640 1 Processor -- start 2023-11-24T06:12:14.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.075+0000 7fcac6b7c640 1 -- start start 2023-11-24T06:12:14.081 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.076+0000 7fcac6b7c640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fcab0004d00 0x7fcac013d280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:14.081 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.076+0000 7fcac6b7c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcac01f3140 0x7fcac01f35c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:14.081 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.076+0000 7fcac6b7c640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fcac014e3f0 0x7fcac014e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:14.081 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.076+0000 7fcac6b7c640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcac014edb0 con 0x7fcac01f3140 2023-11-24T06:12:14.081 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.076+0000 7fcac6b7c640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcac013d7c0 con 0x7fcac014e3f0 2023-11-24T06:12:14.081 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.076+0000 7fcac6b7c640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcac0152900 con 0x7fcab0004d00 2023-11-24T06:12:14.081 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.076+0000 7fcabffff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcac01f3140 0x7fcac01f35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:14.081 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.076+0000 7fcabffff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcac01f3140 0x7fcac01f35c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:60536/0 (socket says 172.21.15.92:60536) 2023-11-24T06:12:14.081 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.076+0000 7fcabffff640 1 -- 172.21.15.92:0/1768022002 learned_addr learned my addr 172.21.15.92:0/1768022002 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:12:14.082 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.077+0000 7fcac4918640 1 --2- 172.21.15.92:0/1768022002 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fcab0004d00 0x7fcac013d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:14.082 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.077+0000 7fcac5119640 1 --2- 172.21.15.92:0/1768022002 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fcac014e3f0 0x7fcac014e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:14.082 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.077+0000 7fcabffff640 1 -- 172.21.15.92:0/1768022002 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fcab0004d00 msgr2=0x7fcac013d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:14.082 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.077+0000 7fcabffff640 1 --2- 172.21.15.92:0/1768022002 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fcab0004d00 0x7fcac013d280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:14.082 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.077+0000 7fcabffff640 1 -- 172.21.15.92:0/1768022002 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fcac014e3f0 msgr2=0x7fcac014e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:14.082 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.077+0000 7fcabffff640 1 --2- 172.21.15.92:0/1768022002 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fcac014e3f0 0x7fcac014e870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:14.082 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.077+0000 7fcabffff640 1 -- 172.21.15.92:0/1768022002 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcac0152ba0 con 0x7fcac01f3140 2023-11-24T06:12:14.082 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.077+0000 7fcabffff640 1 --2- 172.21.15.92:0/1768022002 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcac01f3140 0x7fcac01f35c0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fcaac010860 tx=0x7fcaac010d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:14.082 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.077+0000 7fcabd7fa640 1 -- 172.21.15.92:0/1768022002 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaac009610 con 0x7fcac01f3140 2023-11-24T06:12:14.082 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.077+0000 7fcabd7fa640 1 -- 172.21.15.92:0/1768022002 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcaac00eb60 con 0x7fcac01f3140 2023-11-24T06:12:14.082 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.077+0000 7fcabd7fa640 1 -- 172.21.15.92:0/1768022002 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaac01c400 con 0x7fcac01f3140 2023-11-24T06:12:14.083 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.077+0000 7fcac6b7c640 1 -- 172.21.15.92:0/1768022002 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcac0152e80 con 0x7fcac01f3140 2023-11-24T06:12:14.083 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.077+0000 7fcac6b7c640 1 -- 172.21.15.92:0/1768022002 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcac0153440 con 0x7fcac01f3140 2023-11-24T06:12:14.083 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.079+0000 7fcac6b7c640 1 -- 172.21.15.92:0/1768022002 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcac00675d0 con 0x7fcac01f3140 2023-11-24T06:12:14.084 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.079+0000 7fcabd7fa640 1 -- 172.21.15.92:0/1768022002 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fcaac01c560 con 0x7fcac01f3140 2023-11-24T06:12:14.084 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.080+0000 7fcabd7fa640 1 --2- 172.21.15.92:0/1768022002 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fcaa406ae40 0x7fcaa406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:14.084 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.080+0000 7fcabd7fa640 1 -- 172.21.15.92:0/1768022002 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fcaac089b60 con 0x7fcac01f3140 2023-11-24T06:12:14.084 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.081+0000 7fcac4918640 1 --2- 172.21.15.92:0/1768022002 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fcaa406ae40 0x7fcaa406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:14.084 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.081+0000 7fcac4918640 1 --2- 172.21.15.92:0/1768022002 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fcaa406ae40 0x7fcaa406d300 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fcab4007fd0 tx=0x7fcab4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:14.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.085+0000 7fcabd7fa640 1 -- 172.21.15.92:0/1768022002 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fcaac054cf0 con 0x7fcac01f3140 2023-11-24T06:12:14.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.275+0000 7fcac6b7c640 1 -- 172.21.15.92:0/1768022002 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fcac014e1d0 con 0x7fcac01f3140 2023-11-24T06:12:14.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.276+0000 7fcabd7fa640 1 -- 172.21.15.92:0/1768022002 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v40) v1 ==== 100+0+11 (secure 0 0 0) 0x7fcaac058310 con 0x7fcac01f3140 2023-11-24T06:12:14.279 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 64 2023-11-24T06:12:14.282 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.278+0000 7fcac6b7c640 1 -- 172.21.15.92:0/1768022002 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fcaa406ae40 msgr2=0x7fcaa406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:14.282 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.278+0000 7fcac6b7c640 1 --2- 172.21.15.92:0/1768022002 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fcaa406ae40 0x7fcaa406d300 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fcab4007fd0 tx=0x7fcab4009040 comp rx=0 tx=0).stop 2023-11-24T06:12:14.282 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.278+0000 7fcac6b7c640 1 -- 172.21.15.92:0/1768022002 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcac01f3140 msgr2=0x7fcac01f35c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:14.282 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.278+0000 7fcac6b7c640 1 --2- 172.21.15.92:0/1768022002 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcac01f3140 0x7fcac01f35c0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fcaac010860 tx=0x7fcaac010d20 comp rx=0 tx=0).stop 2023-11-24T06:12:14.282 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.278+0000 7fcac6b7c640 1 -- 172.21.15.92:0/1768022002 shutdown_connections 2023-11-24T06:12:14.282 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.278+0000 7fcac6b7c640 1 --2- 172.21.15.92:0/1768022002 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fcaa406ae40 0x7fcaa406d300 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:14.282 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.278+0000 7fcac6b7c640 1 --2- 172.21.15.92:0/1768022002 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fcac014e3f0 0x7fcac014e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:14.282 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.278+0000 7fcac6b7c640 1 --2- 172.21.15.92:0/1768022002 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcac01f3140 0x7fcac01f35c0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:14.282 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.278+0000 7fcac6b7c640 1 --2- 172.21.15.92:0/1768022002 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fcab0004d00 0x7fcac013d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:14.282 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.278+0000 7fcac6b7c640 1 -- 172.21.15.92:0/1768022002 >> 172.21.15.92:0/1768022002 conn(0x7fcac006a800 msgr2=0x7fcac00aea60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:14.283 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.278+0000 7fcac6b7c640 1 -- 172.21.15.92:0/1768022002 shutdown_connections 2023-11-24T06:12:14.283 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.279+0000 7fcac6b7c640 1 -- 172.21.15.92:0/1768022002 wait complete. 2023-11-24T06:12:14.301 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T06:12:14.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.502+0000 7f9339e94640 1 Processor -- start 2023-11-24T06:12:14.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.502+0000 7f9339e94640 1 -- start start 2023-11-24T06:12:14.506 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.503+0000 7f9339e94640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f9334143ae0 0x7f93340cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:14.506 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.503+0000 7f9339e94640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f93340cf7d0 0x7f933415dca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:14.507 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.503+0000 7f9339e94640 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93340cfc30 con 0x7f933415e1e0 2023-11-24T06:12:14.507 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.503+0000 7f9339e94640 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93340b1ee0 con 0x7f9334143ae0 2023-11-24T06:12:14.507 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.503+0000 7f9339e94640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93340b2780 con 0x7f93340cf7d0 2023-11-24T06:12:14.507 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.504+0000 7f932bfff640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f93340cf7d0 0x7f933415dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:14.507 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.504+0000 7f93337fe640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f9334143ae0 0x7f93340cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:14.508 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.504+0000 7f93337fe640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f9334143ae0 0x7f93340cf290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.92:50150/0 (socket says 172.21.15.92:50150) 2023-11-24T06:12:14.508 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.504+0000 7f932bfff640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f93340cf7d0 0x7f933415dca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3301/0 says I am v2:172.21.15.92:38012/0 (socket says 172.21.15.92:38012) 2023-11-24T06:12:14.508 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.504+0000 7f93337fe640 1 -- 172.21.15.92:0/3668264637 learned_addr learned my addr 172.21.15.92:0/3668264637 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:12:14.508 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.505+0000 7f932bfff640 1 -- 172.21.15.92:0/3668264637 >> v2:172.21.15.118:3300/0 conn(0x7f9334143ae0 msgr2=0x7f93340cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:14.508 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.505+0000 7f932bfff640 1 --2- 172.21.15.92:0/3668264637 >> v2:172.21.15.118:3300/0 conn(0x7f9334143ae0 0x7f93340cf290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:14.509 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.505+0000 7f932bfff640 1 -- 172.21.15.92:0/3668264637 >> v1:172.21.15.118:6789/0 conn(0x7f933415e1e0 legacy=0x7f933415a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:14.509 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.505+0000 7f932bfff640 1 -- 172.21.15.92:0/3668264637 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f933415e990 con 0x7f93340cf7d0 2023-11-24T06:12:14.509 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.505+0000 7f93337fe640 1 --2- 172.21.15.92:0/3668264637 >> v2:172.21.15.118:3300/0 conn(0x7f9334143ae0 0x7f93340cf290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:12:14.509 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.506+0000 7f932bfff640 1 --2- 172.21.15.92:0/3668264637 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f93340cf7d0 0x7f933415dca0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f931c0097a0 tx=0x7f931c02b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b951ac1d5cb1a44f server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:14.510 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.507+0000 7f9332ffd640 1 -- 172.21.15.92:0/3668264637 <== mon.2 v2:172.21.15.118:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f931c02d040 con 0x7f93340cf7d0 2023-11-24T06:12:14.510 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.507+0000 7f9332ffd640 1 -- 172.21.15.92:0/3668264637 <== mon.2 v2:172.21.15.118:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f931c031810 con 0x7f93340cf7d0 2023-11-24T06:12:14.510 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.507+0000 7f9332ffd640 1 -- 172.21.15.92:0/3668264637 <== mon.2 v2:172.21.15.118:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f931c031ba0 con 0x7f93340cf7d0 2023-11-24T06:12:14.511 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.507+0000 7f9339e94640 1 -- 172.21.15.92:0/3668264637 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f93340cf7d0 msgr2=0x7f933415dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:14.511 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.507+0000 7f9339e94640 1 --2- 172.21.15.92:0/3668264637 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f93340cf7d0 0x7f933415dca0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f931c0097a0 tx=0x7f931c02b8b0 comp rx=0 tx=0).stop 2023-11-24T06:12:14.511 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.507+0000 7f9339e94640 1 -- 172.21.15.92:0/3668264637 shutdown_connections 2023-11-24T06:12:14.511 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.507+0000 7f9339e94640 1 --2- 172.21.15.92:0/3668264637 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f93340cf7d0 0x7f933415dca0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:14.511 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.507+0000 7f9339e94640 1 --2- 172.21.15.92:0/3668264637 >> v2:172.21.15.118:3300/0 conn(0x7f9334143ae0 0x7f93340cf290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:14.511 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.507+0000 7f9339e94640 1 -- 172.21.15.92:0/3668264637 >> 172.21.15.92:0/3668264637 conn(0x7f933406a800 msgr2=0x7f933406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:14.511 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.508+0000 7f9339e94640 1 -- 172.21.15.92:0/3668264637 shutdown_connections 2023-11-24T06:12:14.511 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.508+0000 7f9339e94640 1 -- 172.21.15.92:0/3668264637 wait complete. 2023-11-24T06:12:14.512 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.510+0000 7f9339e94640 1 Processor -- start 2023-11-24T06:12:14.513 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.510+0000 7f9339e94640 1 -- start start 2023-11-24T06:12:14.515 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.511+0000 7f9339e94640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f93340cf7d0 0x7f93341405f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:14.515 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.511+0000 7f9339e94640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9334143ae0 0x7f93341645d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:14.515 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.511+0000 7f9339e94640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f933415e1e0 0x7f933415caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:14.515 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.511+0000 7f9339e94640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93341f6ba0 con 0x7f9334143ae0 2023-11-24T06:12:14.515 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.511+0000 7f9339e94640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93341f6e80 con 0x7f93340cf7d0 2023-11-24T06:12:14.515 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.511+0000 7f9339e94640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93341f7160 con 0x7f933415e1e0 2023-11-24T06:12:14.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.511+0000 7f932bfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9334143ae0 0x7f93341645d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:14.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.511+0000 7f932bfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9334143ae0 0x7f93341645d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:60542/0 (socket says 172.21.15.92:60542) 2023-11-24T06:12:14.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.511+0000 7f932bfff640 1 -- 172.21.15.92:0/3013777848 learned_addr learned my addr 172.21.15.92:0/3013777848 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:12:14.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.512+0000 7f93337fe640 1 --2- 172.21.15.92:0/3013777848 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f93340cf7d0 0x7f93341405f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:14.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.512+0000 7f932bfff640 1 -- 172.21.15.92:0/3013777848 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f933415e1e0 msgr2=0x7f933415caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:14.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.512+0000 7f9333fff640 1 --2- 172.21.15.92:0/3013777848 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f933415e1e0 0x7f933415caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:14.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.512+0000 7f932bfff640 1 --2- 172.21.15.92:0/3013777848 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f933415e1e0 0x7f933415caf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:14.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.512+0000 7f932bfff640 1 -- 172.21.15.92:0/3013777848 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f93340cf7d0 msgr2=0x7f93341405f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:14.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.512+0000 7f932bfff640 1 --2- 172.21.15.92:0/3013777848 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f93340cf7d0 0x7f93341405f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:14.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.512+0000 7f932bfff640 1 -- 172.21.15.92:0/3013777848 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93240094d0 con 0x7f9334143ae0 2023-11-24T06:12:14.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.512+0000 7f932bfff640 1 --2- 172.21.15.92:0/3013777848 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9334143ae0 0x7f93341645d0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f931c0098c0 tx=0x7f931c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:14.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.515+0000 7f9330ff9640 1 -- 172.21.15.92:0/3013777848 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f931c006e10 con 0x7f9334143ae0 2023-11-24T06:12:14.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.515+0000 7f9330ff9640 1 -- 172.21.15.92:0/3013777848 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f931c03ad60 con 0x7f9334143ae0 2023-11-24T06:12:14.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.515+0000 7f9339e94640 1 -- 172.21.15.92:0/3013777848 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f931c009400 con 0x7f9334143ae0 2023-11-24T06:12:14.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.515+0000 7f9330ff9640 1 -- 172.21.15.92:0/3013777848 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f931c044620 con 0x7f9334143ae0 2023-11-24T06:12:14.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.515+0000 7f9339e94640 1 -- 172.21.15.92:0/3013777848 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93341f7720 con 0x7f9334143ae0 2023-11-24T06:12:14.519 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.516+0000 7f9339e94640 1 -- 172.21.15.92:0/3013777848 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93341f78e0 con 0x7f9334143ae0 2023-11-24T06:12:14.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.517+0000 7f9330ff9640 1 -- 172.21.15.92:0/3013777848 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f931c04d440 con 0x7f9334143ae0 2023-11-24T06:12:14.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.518+0000 7f9330ff9640 1 --2- 172.21.15.92:0/3013777848 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f931406ae40 0x7f931406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:14.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.518+0000 7f9330ff9640 1 -- 172.21.15.92:0/3013777848 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f931c0aba00 con 0x7f9334143ae0 2023-11-24T06:12:14.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.518+0000 7f93337fe640 1 --2- 172.21.15.92:0/3013777848 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f931406ae40 0x7f931406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:14.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.519+0000 7f93337fe640 1 --2- 172.21.15.92:0/3013777848 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f931406ae40 0x7f931406d300 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f93340cbd50 tx=0x7f9324008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:14.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.522+0000 7f9330ff9640 1 -- 172.21.15.92:0/3013777848 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f931c0769b0 con 0x7f9334143ae0 2023-11-24T06:12:14.756 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.753+0000 7f9339e94640 1 -- 172.21.15.92:0/3013777848 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f93340675d0 con 0x7f9334143ae0 2023-11-24T06:12:14.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.754+0000 7f9330ff9640 1 -- 172.21.15.92:0/3013777848 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+4919 (secure 0 0 0) 0x7f931c035090 con 0x7f9334143ae0 2023-11-24T06:12:14.758 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-24T06:12:14.758 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":23,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4637,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6834/2665242481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":2665242481},{"type":"v1","addr":"172.21.15.118:6834","nonce":2665242481}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":4640,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6835/2822230615","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6833","nonce":2822230615},{"type":"v1","addr":"172.21.15.118:6835","nonce":2822230615}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5499,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6836/2918354210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":2918354210},{"type":"v1","addr":"172.21.15.92:6836","nonce":2918354210}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":23}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T06:11:44.503003+0000","modified":"2023-11-24T06:11:47.998422+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5235},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5235":{"gid":5235,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.92:6837/1413573590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":1413573590},{"type":"v1","addr":"172.21.15.92:6837","nonce":1413573590}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-24T06:12:14.758 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 23 2023-11-24T06:12:14.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.757+0000 7f9339e94640 1 -- 172.21.15.92:0/3013777848 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f931406ae40 msgr2=0x7f931406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:14.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.757+0000 7f9339e94640 1 --2- 172.21.15.92:0/3013777848 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f931406ae40 0x7f931406d300 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f93340cbd50 tx=0x7f9324008040 comp rx=0 tx=0).stop 2023-11-24T06:12:14.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.757+0000 7f9339e94640 1 -- 172.21.15.92:0/3013777848 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9334143ae0 msgr2=0x7f93341645d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:14.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.757+0000 7f9339e94640 1 --2- 172.21.15.92:0/3013777848 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9334143ae0 0x7f93341645d0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f931c0098c0 tx=0x7f931c0046c0 comp rx=0 tx=0).stop 2023-11-24T06:12:14.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.757+0000 7f9339e94640 1 -- 172.21.15.92:0/3013777848 shutdown_connections 2023-11-24T06:12:14.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.757+0000 7f9339e94640 1 --2- 172.21.15.92:0/3013777848 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f931406ae40 0x7f931406d300 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:14.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.757+0000 7f9339e94640 1 --2- 172.21.15.92:0/3013777848 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f933415e1e0 0x7f933415caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:14.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.757+0000 7f9339e94640 1 --2- 172.21.15.92:0/3013777848 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9334143ae0 0x7f93341645d0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:14.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.757+0000 7f9339e94640 1 --2- 172.21.15.92:0/3013777848 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f93340cf7d0 0x7f93341405f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:14.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.757+0000 7f9339e94640 1 -- 172.21.15.92:0/3013777848 >> 172.21.15.92:0/3013777848 conn(0x7f933406a800 msgr2=0x7f9334165240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:14.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.757+0000 7f9339e94640 1 -- 172.21.15.92:0/3013777848 shutdown_connections 2023-11-24T06:12:14.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.757+0000 7f9339e94640 1 -- 172.21.15.92:0/3013777848 wait complete. 2023-11-24T06:12:14.777 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:12:14.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.973+0000 7f9161b01640 1 Processor -- start 2023-11-24T06:12:14.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.973+0000 7f9161b01640 1 -- start start 2023-11-24T06:12:14.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.975+0000 7f9161b01640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f915c15e3b0 0x7f915c164de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:14.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.975+0000 7f9161b01640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f915c0b1ee0 con 0x7f915c0cf9a0 2023-11-24T06:12:14.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.975+0000 7f9161b01640 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f915c0b2780 con 0x7f915c143ae0 2023-11-24T06:12:14.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.975+0000 7f9161b01640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f915c15e990 con 0x7f915c15e3b0 2023-11-24T06:12:14.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.975+0000 7f9153fff640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f915c0cf9a0 0x7f915c15dca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:33808/0 (socket says 172.21.15.92:33808) 2023-11-24T06:12:14.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.975+0000 7f9153fff640 1 -- 172.21.15.92:0/1957071153 learned_addr learned my addr 172.21.15.92:0/1957071153 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-24T06:12:14.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.975+0000 7f915bfff640 1 --2- 172.21.15.92:0/1957071153 >> v2:172.21.15.92:3300/0 conn(0x7f915c15e3b0 0x7f915c164de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:14.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.976+0000 7f915bfff640 1 -- 172.21.15.92:0/1957071153 >> v1:172.21.15.118:6789/0 conn(0x7f915c143ae0 legacy=0x7f915c0cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:14.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.976+0000 7f915bfff640 1 -- 172.21.15.92:0/1957071153 >> v1:172.21.15.92:6789/0 conn(0x7f915c0cf9a0 legacy=0x7f915c15dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:14.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.976+0000 7f915bfff640 1 -- 172.21.15.92:0/1957071153 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f915c0b2130 con 0x7f915c15e3b0 2023-11-24T06:12:14.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.977+0000 7f915bfff640 1 --2- 172.21.15.92:0/1957071153 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f915c15e3b0 0x7f915c164de0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f9154009d90 tx=0x7f915402beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=416b65255dce7565 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:14.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.977+0000 7f915affd640 1 -- 172.21.15.92:0/1957071153 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f915402cc60 con 0x7f915c15e3b0 2023-11-24T06:12:14.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.977+0000 7f915affd640 1 -- 172.21.15.92:0/1957071153 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f915402cdc0 con 0x7f915c15e3b0 2023-11-24T06:12:14.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.977+0000 7f915affd640 1 -- 172.21.15.92:0/1957071153 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9154034660 con 0x7f915c15e3b0 2023-11-24T06:12:14.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.978+0000 7f9161b01640 1 -- 172.21.15.92:0/1957071153 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f915c15e3b0 msgr2=0x7f915c164de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:14.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.978+0000 7f9161b01640 1 --2- 172.21.15.92:0/1957071153 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f915c15e3b0 0x7f915c164de0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f9154009d90 tx=0x7f915402beb0 comp rx=0 tx=0).stop 2023-11-24T06:12:14.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.978+0000 7f9161b01640 1 -- 172.21.15.92:0/1957071153 shutdown_connections 2023-11-24T06:12:14.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.978+0000 7f9161b01640 1 --2- 172.21.15.92:0/1957071153 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f915c15e3b0 0x7f915c164de0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:14.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.978+0000 7f9161b01640 1 -- 172.21.15.92:0/1957071153 >> 172.21.15.92:0/1957071153 conn(0x7f915c06a800 msgr2=0x7f915c06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:14.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.978+0000 7f9161b01640 1 -- 172.21.15.92:0/1957071153 shutdown_connections 2023-11-24T06:12:14.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.978+0000 7f9161b01640 1 -- 172.21.15.92:0/1957071153 wait complete. 2023-11-24T06:12:14.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.979+0000 7f9161b01640 1 Processor -- start 2023-11-24T06:12:14.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.980+0000 7f9161b01640 1 -- start start 2023-11-24T06:12:14.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.981+0000 7f9161b01640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f915c0cf9a0 0x7f915c1f5bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:14.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.981+0000 7f9161b01640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f915c143ae0 0x7f915c1f2de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:14.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.981+0000 7f9161b01640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f915c15e3b0 0x7f915c0af240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:14.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.981+0000 7f9161b01640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f915c0cd2c0 con 0x7f915c143ae0 2023-11-24T06:12:14.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.981+0000 7f9161b01640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f915c0cc6b0 con 0x7f915c15e3b0 2023-11-24T06:12:14.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.981+0000 7f9161b01640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f915c0cc990 con 0x7f915c0cf9a0 2023-11-24T06:12:14.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.981+0000 7f9153fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f915c143ae0 0x7f915c1f2de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:14.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.981+0000 7f9153fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f915c143ae0 0x7f915c1f2de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:60568/0 (socket says 172.21.15.92:60568) 2023-11-24T06:12:14.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.981+0000 7f9153fff640 1 -- 172.21.15.92:0/2456973471 learned_addr learned my addr 172.21.15.92:0/2456973471 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:12:14.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.981+0000 7f915b7fe640 1 --2- 172.21.15.92:0/2456973471 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f915c0cf9a0 0x7f915c1f5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:14.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.981+0000 7f915bfff640 1 --2- 172.21.15.92:0/2456973471 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f915c15e3b0 0x7f915c0af240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:14.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.981+0000 7f9153fff640 1 -- 172.21.15.92:0/2456973471 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f915c0cf9a0 msgr2=0x7f915c1f5bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:14.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.981+0000 7f9153fff640 1 --2- 172.21.15.92:0/2456973471 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f915c0cf9a0 0x7f915c1f5bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:14.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.981+0000 7f9153fff640 1 -- 172.21.15.92:0/2456973471 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f915c15e3b0 msgr2=0x7f915c0af240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:14.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.981+0000 7f9153fff640 1 --2- 172.21.15.92:0/2456973471 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f915c15e3b0 0x7f915c0af240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:14.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.981+0000 7f9153fff640 1 -- 172.21.15.92:0/2456973471 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9154009a70 con 0x7f915c143ae0 2023-11-24T06:12:14.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.982+0000 7f9153fff640 1 --2- 172.21.15.92:0/2456973471 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f915c143ae0 0x7f915c1f2de0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f914400c4a0 tx=0x7f914400c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:14.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.982+0000 7f9158ff9640 1 -- 172.21.15.92:0/2456973471 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9144005ce0 con 0x7f915c143ae0 2023-11-24T06:12:14.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.982+0000 7f9158ff9640 1 -- 172.21.15.92:0/2456973471 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9144005e40 con 0x7f915c143ae0 2023-11-24T06:12:14.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.982+0000 7f9158ff9640 1 -- 172.21.15.92:0/2456973471 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9144007270 con 0x7f915c143ae0 2023-11-24T06:12:14.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.982+0000 7f9161b01640 1 -- 172.21.15.92:0/2456973471 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f915c0ccc70 con 0x7f915c143ae0 2023-11-24T06:12:14.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.982+0000 7f9161b01640 1 -- 172.21.15.92:0/2456973471 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f915c1f6100 con 0x7f915c143ae0 2023-11-24T06:12:14.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.983+0000 7f9161b01640 1 -- 172.21.15.92:0/2456973471 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f915c0cd430 con 0x7f915c143ae0 2023-11-24T06:12:14.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.984+0000 7f9158ff9640 1 -- 172.21.15.92:0/2456973471 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f91440073d0 con 0x7f915c143ae0 2023-11-24T06:12:14.988 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.985+0000 7f9158ff9640 1 --2- 172.21.15.92:0/2456973471 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f913c06ae70 0x7f913c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:14.988 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.985+0000 7f9158ff9640 1 -- 172.21.15.92:0/2456973471 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f9144088b50 con 0x7f915c143ae0 2023-11-24T06:12:14.988 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.985+0000 7f915b7fe640 1 --2- 172.21.15.92:0/2456973471 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f913c06ae70 0x7f913c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:14.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.986+0000 7f915b7fe640 1 --2- 172.21.15.92:0/2456973471 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f913c06ae70 0x7f913c06d330 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f914c003d90 tx=0x7f914c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:14.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:14.989+0000 7f9158ff9640 1 -- 172.21.15.92:0/2456973471 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9144053bb0 con 0x7f915c143ae0 2023-11-24T06:12:15.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:15.165+0000 7f9161b01640 1 -- 172.21.15.92:0/2456973471 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f915c0675d0 con 0x7f915c143ae0 2023-11-24T06:12:15.170 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:15.167+0000 7f9158ff9640 1 -- 172.21.15.92:0/2456973471 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+16650 (secure 0 0 0) 0x7f9144017020 con 0x7f915c143ae0 2023-11-24T06:12:15.171 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-24T06:12:15.171 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":40,"fsid":"2d6dee5f-8aa3-470f-931a-50ade963b1cb","created":"2023-11-24T06:10:04.707204+0000","modified":"2023-11-24T06:12:06.456223+0000","last_up_change":"2023-11-24T06:10:17.598352+0000","last_in_change":"2023-11-24T06:10:09.040793+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:10:17.733098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:11:42.241681+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-11-24T06:11:43.522720+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ff37a9fb-135f-471f-b4b0-f7b5772006c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6803","nonce":20560714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6805","nonce":20560714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6809","nonce":20560714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6807","nonce":20560714}]},"public_addr":"172.21.15.92:6803/20560714","cluster_addr":"172.21.15.92:6805/20560714","heartbeat_back_addr":"172.21.15.92:6809/20560714","heartbeat_front_addr":"172.21.15.92:6807/20560714","state":["exists","up"]},{"osd":1,"uuid":"03cdd353-dcf1-4f38-94fe-a795137c06b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6819","nonce":2451239906}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6821","nonce":2451239906}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6825","nonce":2451239906}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6823","nonce":2451239906}]},"public_addr":"172.21.15.92:6819/2451239906","cluster_addr":"172.21.15.92:6821/2451239906","heartbeat_back_addr":"172.21.15.92:6825/2451239906","heartbeat_front_addr":"172.21.15.92:6823/2451239906","state":["exists","up"]},{"osd":2,"uuid":"a4a20d80-5d57-484b-becd-9aa8070dd3aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6811","nonce":1124979277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6813","nonce":1124979277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6817","nonce":1124979277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6815","nonce":1124979277}]},"public_addr":"172.21.15.92:6811/1124979277","cluster_addr":"172.21.15.92:6813/1124979277","heartbeat_back_addr":"172.21.15.92:6817/1124979277","heartbeat_front_addr":"172.21.15.92:6815/1124979277","state":["exists","up"]},{"osd":3,"uuid":"4648d7ce-9b29-4583-9425-4ba8fd2b1903","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6827","nonce":1734115782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6829","nonce":1734115782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6833","nonce":1734115782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6831","nonce":1734115782}]},"public_addr":"172.21.15.92:6827/1734115782","cluster_addr":"172.21.15.92:6829/1734115782","heartbeat_back_addr":"172.21.15.92:6833/1734115782","heartbeat_front_addr":"172.21.15.92:6831/1734115782","state":["exists","up"]},{"osd":4,"uuid":"3bc9d7eb-c6fa-4fcf-a3ec-ade604e05aac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6819","nonce":3425937229}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6822","nonce":3425937229}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6830","nonce":3425937229}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6826","nonce":3425937229}]},"public_addr":"172.21.15.118:6819/3425937229","cluster_addr":"172.21.15.118:6822/3425937229","heartbeat_back_addr":"172.21.15.118:6830/3425937229","heartbeat_front_addr":"172.21.15.118:6826/3425937229","state":["exists","up"]},{"osd":5,"uuid":"dce9ba6d-59ed-46d1-84fc-a84b13de10c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6801","nonce":827455984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6803","nonce":827455984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6807","nonce":827455984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6805","nonce":827455984}]},"public_addr":"172.21.15.118:6801/827455984","cluster_addr":"172.21.15.118:6803/827455984","heartbeat_back_addr":"172.21.15.118:6807/827455984","heartbeat_front_addr":"172.21.15.118:6805/827455984","state":["exists","up"]},{"osd":6,"uuid":"82aec504-ae58-4709-8d30-46f2b12081b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6809","nonce":2946986141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6811","nonce":2946986141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6815","nonce":2946986141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6813","nonce":2946986141}]},"public_addr":"172.21.15.118:6809/2946986141","cluster_addr":"172.21.15.118:6811/2946986141","heartbeat_back_addr":"172.21.15.118:6815/2946986141","heartbeat_front_addr":"172.21.15.118:6813/2946986141","state":["exists","up"]},{"osd":7,"uuid":"4ed4c624-c88f-4d35-bdb1-bf209b5dffb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6817","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6818","nonce":48122059}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6821","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6823","nonce":48122059}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6829","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6831","nonce":48122059}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6825","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6827","nonce":48122059}]},"public_addr":"172.21.15.118:6818/48122059","cluster_addr":"172.21.15.118:6823/48122059","heartbeat_back_addr":"172.21.15.118:6831/48122059","heartbeat_front_addr":"172.21.15.118:6827/48122059","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.536664+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.368324+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.271201+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.342971+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.621832+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.387110+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.632946+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.416032+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.92:6836/3889219158":"2023-11-25T06:12:03.443855+0000","172.21.15.92:6834/3889219158":"2023-11-25T06:12:03.443855+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:12:15.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:15.170+0000 7f9161b01640 1 -- 172.21.15.92:0/2456973471 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f913c06ae70 msgr2=0x7f913c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:15.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:15.170+0000 7f9161b01640 1 --2- 172.21.15.92:0/2456973471 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f913c06ae70 0x7f913c06d330 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f914c003d90 tx=0x7f914c009040 comp rx=0 tx=0).stop 2023-11-24T06:12:15.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:15.170+0000 7f9161b01640 1 -- 172.21.15.92:0/2456973471 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f915c143ae0 msgr2=0x7f915c1f2de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:15.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:15.170+0000 7f9161b01640 1 --2- 172.21.15.92:0/2456973471 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f915c143ae0 0x7f915c1f2de0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f914400c4a0 tx=0x7f914400c960 comp rx=0 tx=0).stop 2023-11-24T06:12:15.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:15.170+0000 7f9161b01640 1 -- 172.21.15.92:0/2456973471 shutdown_connections 2023-11-24T06:12:15.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:15.170+0000 7f9161b01640 1 --2- 172.21.15.92:0/2456973471 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f913c06ae70 0x7f913c06d330 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:15.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:15.170+0000 7f9161b01640 1 --2- 172.21.15.92:0/2456973471 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f915c15e3b0 0x7f915c0af240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:15.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:15.170+0000 7f9161b01640 1 --2- 172.21.15.92:0/2456973471 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f915c143ae0 0x7f915c1f2de0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:15.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:15.170+0000 7f9161b01640 1 --2- 172.21.15.92:0/2456973471 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f915c0cf9a0 0x7f915c1f5bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:15.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:15.170+0000 7f9161b01640 1 -- 172.21.15.92:0/2456973471 >> 172.21.15.92:0/2456973471 conn(0x7f915c06a800 msgr2=0x7f915c15bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:15.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:15.170+0000 7f9161b01640 1 -- 172.21.15.92:0/2456973471 shutdown_connections 2023-11-24T06:12:15.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:15.170+0000 7f9161b01640 1 -- 172.21.15.92:0/2456973471 wait complete. 2023-11-24T06:12:15.192 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-24T06:12:15.192 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T06:12:15.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:15.399+0000 7f818e6b4640 1 Processor -- start 2023-11-24T06:12:15.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:15.399+0000 7f818e6b4640 1 -- start start 2023-11-24T06:12:15.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:15.400+0000 7f818e6b4640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8188143ae0 0x7f81880cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:15.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:15.400+0000 7f818e6b4640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f81880cf7d0 0x7f818815dca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:15.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:15.401+0000 7f818e6b4640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81880cfc30 con 0x7f818815e1e0 2023-11-24T06:12:15.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:15.401+0000 7f818e6b4640 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81880b1ee0 con 0x7f81880cf7d0 2023-11-24T06:12:15.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:15.401+0000 7f818e6b4640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81880b2780 con 0x7f8188143ae0 2023-11-24T06:12:15.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:15.401+0000 7f818cc51640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f818815e1e0 0x7f818815a670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:33812/0 (socket says 172.21.15.92:33812) 2023-11-24T06:12:15.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:15.401+0000 7f818cc51640 1 -- 172.21.15.92:0/3718076496 learned_addr learned my addr 172.21.15.92:0/3718076496 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-24T06:12:15.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:15.401+0000 7f8187fff640 1 --2- 172.21.15.92:0/3718076496 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8188143ae0 0x7f81880cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:15.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:15.401+0000 7f81877fe640 1 --2- 172.21.15.92:0/3718076496 >> v2:172.21.15.118:3300/0 conn(0x7f81880cf7d0 0x7f818815dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:15.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:15.401+0000 7f8186ffd640 1 -- 172.21.15.92:0/3718076496 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3881103800 0 0) 0x7f81880cfc30 con 0x7f818815e1e0 2023-11-24T06:12:15.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:15.401+0000 7f8186ffd640 1 -- 172.21.15.92:0/3718076496 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8170003520 con 0x7f818815e1e0 2023-11-24T06:12:15.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:15.402+0000 7f8186ffd640 1 -- 172.21.15.92:0/3718076496 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4023065835 0 0) 0x7f8170003520 con 0x7f818815e1e0 2023-11-24T06:12:15.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:15.402+0000 7f8186ffd640 1 -- 172.21.15.92:0/3718076496 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8188143ae0 msgr2=0x7f81880cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:15.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:15.402+0000 7f8186ffd640 1 --2- 172.21.15.92:0/3718076496 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8188143ae0 0x7f81880cf290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:15.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:15.402+0000 7f8186ffd640 1 -- 172.21.15.92:0/3718076496 >> v2:172.21.15.118:3300/0 conn(0x7f81880cf7d0 msgr2=0x7f818815dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:15.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:15.402+0000 7f8186ffd640 1 --2- 172.21.15.92:0/3718076496 >> v2:172.21.15.118:3300/0 conn(0x7f81880cf7d0 0x7f818815dca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:15.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:15.402+0000 7f8186ffd640 1 -- 172.21.15.92:0/3718076496 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f818815e990 con 0x7f818815e1e0 2023-11-24T06:12:15.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:15.402+0000 7f8186ffd640 1 -- 172.21.15.92:0/3718076496 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2607661350 0 0) 0x7f8180001670 con 0x7f818815e1e0 2023-11-24T06:12:15.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:15.402+0000 7f8186ffd640 1 -- 172.21.15.92:0/3718076496 >> v1:172.21.15.92:6789/0 conn(0x7f818815e1e0 legacy=0x7f818815a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:15.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:15.402+0000 7f8186ffd640 1 --2- 172.21.15.92:0/3718076496 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8170003970 0x7f8170003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:15.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:15.402+0000 7f8186ffd640 1 --2- 172.21.15.92:0/3718076496 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f81700047c0 0x7f8170004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:15.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:15.402+0000 7f8186ffd640 1 --2- 172.21.15.92:0/3718076496 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8188143ae0 0x7f8170005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:15.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:15.402+0000 7f8186ffd640 1 -- 172.21.15.92:0/3718076496 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81880b1ee0 con 0x7f8170003970 2023-11-24T06:12:15.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:15.402+0000 7f8186ffd640 1 -- 172.21.15.92:0/3718076496 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81880b2780 con 0x7f81700047c0 2023-11-24T06:12:15.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:15.402+0000 7f8186ffd640 1 -- 172.21.15.92:0/3718076496 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f817000db90 con 0x7f8188143ae0 2023-11-24T06:12:15.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:15.402+0000 7f8187fff640 1 --2- 172.21.15.92:0/3718076496 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8170003970 0x7f8170003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:15.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:15.403+0000 7f818cc51640 1 --2- 172.21.15.92:0/3718076496 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8188143ae0 0x7f8170005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:15.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:15.403+0000 7f81877fe640 1 --2- 172.21.15.92:0/3718076496 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f81700047c0 0x7f8170004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:15.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:15.403+0000 7f8187fff640 1 -- 172.21.15.92:0/3718076496 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8188143ae0 msgr2=0x7f8170005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:15.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:15.403+0000 7f8187fff640 1 --2- 172.21.15.92:0/3718076496 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8188143ae0 0x7f8170005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:15.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:15.403+0000 7f8187fff640 1 -- 172.21.15.92:0/3718076496 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f81700047c0 msgr2=0x7f8170004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:15.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:15.403+0000 7f8187fff640 1 --2- 172.21.15.92:0/3718076496 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f81700047c0 0x7f8170004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:15.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:15.403+0000 7f8187fff640 1 -- 172.21.15.92:0/3718076496 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8170003520 con 0x7f8170003970 2023-11-24T06:12:15.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:15.404+0000 7f8187fff640 1 --2- 172.21.15.92:0/3718076496 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8170003970 0x7f8170003d90 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f8178009fc0 tx=0x7f81700057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db51b7353662c76c server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:15.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:15.404+0000 7f8186ffd640 1 -- 172.21.15.92:0/3718076496 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f817802cc00 con 0x7f8170003970 2023-11-24T06:12:15.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:15.404+0000 7f8186ffd640 1 -- 172.21.15.92:0/3718076496 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f817800f040 con 0x7f8170003970 2023-11-24T06:12:15.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:15.404+0000 7f8186ffd640 1 -- 172.21.15.92:0/3718076496 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8178034440 con 0x7f8170003970 2023-11-24T06:12:15.412 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:15.404+0000 7f818e6b4640 1 -- 172.21.15.92:0/3718076496 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8170003970 msgr2=0x7f8170003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:15.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:15.405+0000 7f818e6b4640 1 --2- 172.21.15.92:0/3718076496 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8170003970 0x7f8170003d90 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f8178009fc0 tx=0x7f81700057f0 comp rx=0 tx=0).stop 2023-11-24T06:12:15.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:15.405+0000 7f818cc51640 1 -- 172.21.15.92:0/3718076496 reap_dead start 2023-11-24T06:12:15.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:15.405+0000 7f818e6b4640 1 -- 172.21.15.92:0/3718076496 shutdown_connections 2023-11-24T06:12:15.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:15.405+0000 7f818e6b4640 1 -- 172.21.15.92:0/3718076496 >> 172.21.15.92:0/3718076496 conn(0x7f818806a800 msgr2=0x7f818806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:15.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:15.405+0000 7f818e6b4640 1 -- 172.21.15.92:0/3718076496 shutdown_connections 2023-11-24T06:12:15.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:15.405+0000 7f818e6b4640 1 -- 172.21.15.92:0/3718076496 wait complete. 2023-11-24T06:12:15.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:15.406+0000 7f818e6b4640 1 Processor -- start 2023-11-24T06:12:15.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:15.407+0000 7f818e6b4640 1 -- start start 2023-11-24T06:12:15.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:15.407+0000 7f818e6b4640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8170003970 0x7f81880c85c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:15.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:15.407+0000 7f818e6b4640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8188137570 0x7f8188159b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:15.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:15.407+0000 7f818e6b4640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f818815c810 0x7f8188139bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:15.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:15.407+0000 7f818e6b4640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8188138680 con 0x7f8188137570 2023-11-24T06:12:15.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:15.407+0000 7f818e6b4640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8188137a70 con 0x7f818815c810 2023-11-24T06:12:15.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:15.407+0000 7f818e6b4640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8188137d50 con 0x7f8170003970 2023-11-24T06:12:15.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:15.408+0000 7f81877fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8188137570 0x7f8188159b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:15.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:15.408+0000 7f81877fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8188137570 0x7f8188159b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:60586/0 (socket says 172.21.15.92:60586) 2023-11-24T06:12:15.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:15.408+0000 7f81877fe640 1 -- 172.21.15.92:0/894690898 learned_addr learned my addr 172.21.15.92:0/894690898 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:12:15.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:15.408+0000 7f8187fff640 1 --2- 172.21.15.92:0/894690898 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8170003970 0x7f81880c85c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:15.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:15.408+0000 7f818cc51640 1 --2- 172.21.15.92:0/894690898 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f818815c810 0x7f8188139bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:15.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:15.408+0000 7f81877fe640 1 -- 172.21.15.92:0/894690898 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8170003970 msgr2=0x7f81880c85c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:15.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:15.408+0000 7f81877fe640 1 --2- 172.21.15.92:0/894690898 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8170003970 0x7f81880c85c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:15.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:15.408+0000 7f81877fe640 1 -- 172.21.15.92:0/894690898 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f818815c810 msgr2=0x7f8188139bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:15.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:15.408+0000 7f81877fe640 1 --2- 172.21.15.92:0/894690898 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f818815c810 0x7f8188139bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:15.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:15.408+0000 7f81877fe640 1 -- 172.21.15.92:0/894690898 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8188138030 con 0x7f8188137570 2023-11-24T06:12:15.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:15.409+0000 7f81877fe640 1 --2- 172.21.15.92:0/894690898 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8188137570 0x7f8188159b90 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f817c0108e0 tx=0x7f817c010da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:15.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:15.409+0000 7f8184ff9640 1 -- 172.21.15.92:0/894690898 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f817c00eb60 con 0x7f8188137570 2023-11-24T06:12:15.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:15.409+0000 7f8184ff9640 1 -- 172.21.15.92:0/894690898 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f817c00ecc0 con 0x7f8188137570 2023-11-24T06:12:15.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:15.409+0000 7f8184ff9640 1 -- 172.21.15.92:0/894690898 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f817c01c400 con 0x7f8188137570 2023-11-24T06:12:15.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:15.409+0000 7f818e6b4640 1 -- 172.21.15.92:0/894690898 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8188138310 con 0x7f8188137570 2023-11-24T06:12:15.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:15.409+0000 7f818e6b4640 1 -- 172.21.15.92:0/894690898 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f818815cc90 con 0x7f8188137570 2023-11-24T06:12:15.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:15.410+0000 7f818e6b4640 1 -- 172.21.15.92:0/894690898 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81881387f0 con 0x7f8188137570 2023-11-24T06:12:15.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:15.411+0000 7f8184ff9640 1 -- 172.21.15.92:0/894690898 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f817c01c560 con 0x7f8188137570 2023-11-24T06:12:15.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:15.412+0000 7f8184ff9640 1 --2- 172.21.15.92:0/894690898 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f817406ae70 0x7f817406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:15.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:15.412+0000 7f8184ff9640 1 -- 172.21.15.92:0/894690898 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f817c089cd0 con 0x7f8188137570 2023-11-24T06:12:15.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:15.412+0000 7f8187fff640 1 --2- 172.21.15.92:0/894690898 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f817406ae70 0x7f817406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:15.417 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:15.413+0000 7f8187fff640 1 --2- 172.21.15.92:0/894690898 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f817406ae70 0x7f817406d330 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f8178036000 tx=0x7f817802e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:15.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:15.417+0000 7f8184ff9640 1 -- 172.21.15.92:0/894690898 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f817c054d30 con 0x7f8188137570 2023-11-24T06:12:15.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:15.644+0000 7f818e6b4640 1 -- 172.21.15.92:0/894690898 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f81880675d0 con 0x7f8188137570 2023-11-24T06:12:15.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:15.645+0000 7f8184ff9640 1 -- 172.21.15.92:0/894690898 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+4919 (secure 0 0 0) 0x7f817c058350 con 0x7f8188137570 2023-11-24T06:12:15.648 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-24T06:12:15.649 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":23,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4637,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6834/2665242481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":2665242481},{"type":"v1","addr":"172.21.15.118:6834","nonce":2665242481}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":4640,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6835/2822230615","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6833","nonce":2822230615},{"type":"v1","addr":"172.21.15.118:6835","nonce":2822230615}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5499,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6836/2918354210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":2918354210},{"type":"v1","addr":"172.21.15.92:6836","nonce":2918354210}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":23}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T06:11:44.503003+0000","modified":"2023-11-24T06:11:47.998422+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5235},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5235":{"gid":5235,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.92:6837/1413573590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":1413573590},{"type":"v1","addr":"172.21.15.92:6837","nonce":1413573590}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-24T06:12:15.649 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 23 2023-11-24T06:12:15.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:15.647+0000 7f818e6b4640 1 -- 172.21.15.92:0/894690898 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f817406ae70 msgr2=0x7f817406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:15.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:15.647+0000 7f818e6b4640 1 --2- 172.21.15.92:0/894690898 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f817406ae70 0x7f817406d330 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f8178036000 tx=0x7f817802e000 comp rx=0 tx=0).stop 2023-11-24T06:12:15.652 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:15.648+0000 7f818e6b4640 1 -- 172.21.15.92:0/894690898 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8188137570 msgr2=0x7f8188159b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:15.652 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:15.648+0000 7f818e6b4640 1 --2- 172.21.15.92:0/894690898 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8188137570 0x7f8188159b90 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f817c0108e0 tx=0x7f817c010da0 comp rx=0 tx=0).stop 2023-11-24T06:12:15.652 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:15.648+0000 7f818e6b4640 1 -- 172.21.15.92:0/894690898 shutdown_connections 2023-11-24T06:12:15.652 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:15.648+0000 7f818e6b4640 1 --2- 172.21.15.92:0/894690898 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f817406ae70 0x7f817406d330 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:15.652 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:15.648+0000 7f818e6b4640 1 --2- 172.21.15.92:0/894690898 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f818815c810 0x7f8188139bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:15.652 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:15.648+0000 7f818e6b4640 1 --2- 172.21.15.92:0/894690898 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8188137570 0x7f8188159b90 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:15.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:15.648+0000 7f818e6b4640 1 --2- 172.21.15.92:0/894690898 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8170003970 0x7f81880c85c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:15.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:15.648+0000 7f818e6b4640 1 -- 172.21.15.92:0/894690898 >> 172.21.15.92:0/894690898 conn(0x7f818806a800 msgr2=0x7f81880ad990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:15.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:15.648+0000 7f818e6b4640 1 -- 172.21.15.92:0/894690898 shutdown_connections 2023-11-24T06:12:15.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:15.648+0000 7f818e6b4640 1 -- 172.21.15.92:0/894690898 wait complete. 2023-11-24T06:12:15.670 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T06:12:15.885 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:15.881+0000 7f2caadce640 1 Processor -- start 2023-11-24T06:12:15.885 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:15.882+0000 7f2caadce640 1 -- start start 2023-11-24T06:12:15.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:15.883+0000 7f2caadce640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f2ca40c9870 0x7f2ca40ceb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:15.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:15.883+0000 7f2caadce640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f2ca40d0560 0x7f2ca4161030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:15.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:15.883+0000 7f2caadce640 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ca40cf4c0 con 0x7f2ca40d2260 2023-11-24T06:12:15.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:15.883+0000 7f2caadce640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ca40d2ef0 con 0x7f2ca40d0560 2023-11-24T06:12:15.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:15.883+0000 7f2caadce640 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ca40c9c50 con 0x7f2ca40c9870 2023-11-24T06:12:15.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:15.883+0000 7f2ca3fff640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f2ca40d0560 0x7f2ca4161030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:15.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:15.883+0000 7f2ca3fff640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f2ca40d0560 0x7f2ca4161030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:60602/0 (socket says 172.21.15.92:60602) 2023-11-24T06:12:15.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:15.883+0000 7f2ca3fff640 1 -- 172.21.15.92:0/3443089895 learned_addr learned my addr 172.21.15.92:0/3443089895 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:12:15.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:15.884+0000 7f2ca8b6a640 1 --2- 172.21.15.92:0/3443089895 >> v2:172.21.15.118:3300/0 conn(0x7f2ca40c9870 0x7f2ca40ceb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:15.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:15.884+0000 7f2ca3fff640 1 -- 172.21.15.92:0/3443089895 >> v2:172.21.15.118:3300/0 conn(0x7f2ca40c9870 msgr2=0x7f2ca40ceb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:15.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:15.884+0000 7f2ca3fff640 1 --2- 172.21.15.92:0/3443089895 >> v2:172.21.15.118:3300/0 conn(0x7f2ca40c9870 0x7f2ca40ceb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:15.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:15.884+0000 7f2ca3fff640 1 -- 172.21.15.92:0/3443089895 >> v1:172.21.15.118:6789/0 conn(0x7f2ca40d2260 legacy=0x7f2ca40d0e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:15.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:15.884+0000 7f2ca3fff640 1 -- 172.21.15.92:0/3443089895 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ca40cf7a0 con 0x7f2ca40d0560 2023-11-24T06:12:15.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:15.885+0000 7f2ca3fff640 1 --2- 172.21.15.92:0/3443089895 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2ca40d0560 0x7f2ca4161030 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f2ca40b5de0 tx=0x7f2c8c02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16a7a4add5bafd30 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:15.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:15.885+0000 7f2ca37fe640 1 -- 172.21.15.92:0/3443089895 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c8c031640 con 0x7f2ca40d0560 2023-11-24T06:12:15.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:15.885+0000 7f2ca37fe640 1 -- 172.21.15.92:0/3443089895 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c8c0317a0 con 0x7f2ca40d0560 2023-11-24T06:12:15.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:15.885+0000 7f2ca37fe640 1 -- 172.21.15.92:0/3443089895 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c8c031b30 con 0x7f2ca40d0560 2023-11-24T06:12:15.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:15.886+0000 7f2caadce640 1 -- 172.21.15.92:0/3443089895 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2ca40d0560 msgr2=0x7f2ca4161030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:15.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:15.886+0000 7f2caadce640 1 --2- 172.21.15.92:0/3443089895 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2ca40d0560 0x7f2ca4161030 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f2ca40b5de0 tx=0x7f2c8c02b8f0 comp rx=0 tx=0).stop 2023-11-24T06:12:15.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:15.886+0000 7f2caadce640 1 -- 172.21.15.92:0/3443089895 shutdown_connections 2023-11-24T06:12:15.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:15.886+0000 7f2caadce640 1 --2- 172.21.15.92:0/3443089895 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2ca40d0560 0x7f2ca4161030 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:15.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:15.886+0000 7f2caadce640 1 --2- 172.21.15.92:0/3443089895 >> v2:172.21.15.118:3300/0 conn(0x7f2ca40c9870 0x7f2ca40ceb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:15.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:15.886+0000 7f2caadce640 1 -- 172.21.15.92:0/3443089895 >> 172.21.15.92:0/3443089895 conn(0x7f2ca409c560 msgr2=0x7f2ca40d3250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:15.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:15.886+0000 7f2caadce640 1 -- 172.21.15.92:0/3443089895 shutdown_connections 2023-11-24T06:12:15.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:15.886+0000 7f2caadce640 1 -- 172.21.15.92:0/3443089895 wait complete. 2023-11-24T06:12:15.891 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:15.888+0000 7f2caadce640 1 Processor -- start 2023-11-24T06:12:15.891 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:15.888+0000 7f2caadce640 1 -- start start 2023-11-24T06:12:15.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:15.889+0000 7f2caadce640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2ca40c9870 0x7f2ca40cb4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:15.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:15.889+0000 7f2caadce640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2ca40d0560 0x7f2ca4157e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:15.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:15.889+0000 7f2caadce640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2ca40d2260 0x7f2ca40b8630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:15.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:15.889+0000 7f2caadce640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ca40cb060 con 0x7f2ca40c9870 2023-11-24T06:12:15.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:15.889+0000 7f2caadce640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ca4158350 con 0x7f2ca40d0560 2023-11-24T06:12:15.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:15.889+0000 7f2caadce640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ca40ca450 con 0x7f2ca40d2260 2023-11-24T06:12:15.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:15.889+0000 7f2ca8b6a640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2ca40c9870 0x7f2ca40cb4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:15.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:15.889+0000 7f2ca8b6a640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2ca40c9870 0x7f2ca40cb4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:60618/0 (socket says 172.21.15.92:60618) 2023-11-24T06:12:15.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:15.889+0000 7f2ca8b6a640 1 -- 172.21.15.92:0/790817234 learned_addr learned my addr 172.21.15.92:0/790817234 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:12:15.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:15.889+0000 7f2ca936b640 1 --2- 172.21.15.92:0/790817234 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2ca40d2260 0x7f2ca40b8630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:15.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:15.889+0000 7f2ca3fff640 1 --2- 172.21.15.92:0/790817234 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2ca40d0560 0x7f2ca4157e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:15.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:15.889+0000 7f2ca8b6a640 1 -- 172.21.15.92:0/790817234 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2ca40d2260 msgr2=0x7f2ca40b8630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:15.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:15.889+0000 7f2ca8b6a640 1 --2- 172.21.15.92:0/790817234 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2ca40d2260 0x7f2ca40b8630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:15.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:15.889+0000 7f2ca8b6a640 1 -- 172.21.15.92:0/790817234 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2ca40d0560 msgr2=0x7f2ca4157e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:15.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:15.889+0000 7f2ca8b6a640 1 --2- 172.21.15.92:0/790817234 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2ca40d0560 0x7f2ca4157e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:15.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:15.889+0000 7f2ca8b6a640 1 -- 172.21.15.92:0/790817234 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c8c0094d0 con 0x7f2ca40c9870 2023-11-24T06:12:15.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:15.889+0000 7f2ca936b640 1 --2- 172.21.15.92:0/790817234 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2ca40d2260 0x7f2ca40b8630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:12:15.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:15.890+0000 7f2ca8b6a640 1 --2- 172.21.15.92:0/790817234 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2ca40c9870 0x7f2ca40cb4f0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f2c94007ef0 tx=0x7f2c9400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:15.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:15.890+0000 7f2ca17fa640 1 -- 172.21.15.92:0/790817234 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c94014620 con 0x7f2ca40c9870 2023-11-24T06:12:15.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:15.890+0000 7f2ca17fa640 1 -- 172.21.15.92:0/790817234 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c94014780 con 0x7f2ca40c9870 2023-11-24T06:12:15.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:15.890+0000 7f2caadce640 1 -- 172.21.15.92:0/790817234 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ca40ca730 con 0x7f2ca40c9870 2023-11-24T06:12:15.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:15.890+0000 7f2ca17fa640 1 -- 172.21.15.92:0/790817234 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c94014b30 con 0x7f2ca40c9870 2023-11-24T06:12:15.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:15.890+0000 7f2caadce640 1 -- 172.21.15.92:0/790817234 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ca40cacf0 con 0x7f2ca40c9870 2023-11-24T06:12:15.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:15.891+0000 7f2caadce640 1 -- 172.21.15.92:0/790817234 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ca40675d0 con 0x7f2ca40c9870 2023-11-24T06:12:15.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:15.892+0000 7f2ca17fa640 1 -- 172.21.15.92:0/790817234 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2c94014c90 con 0x7f2ca40c9870 2023-11-24T06:12:15.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:15.893+0000 7f2ca17fa640 1 --2- 172.21.15.92:0/790817234 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f2c8406ae40 0x7f2c8406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:15.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:15.893+0000 7f2ca17fa640 1 -- 172.21.15.92:0/790817234 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f2c94089fd0 con 0x7f2ca40c9870 2023-11-24T06:12:15.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:15.893+0000 7f2ca3fff640 1 --2- 172.21.15.92:0/790817234 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f2c8406ae40 0x7f2c8406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:15.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:15.894+0000 7f2ca3fff640 1 --2- 172.21.15.92:0/790817234 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f2c8406ae40 0x7f2c8406d300 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f2c8c036000 tx=0x7f2c8c03c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:15.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:15.897+0000 7f2ca17fa640 1 -- 172.21.15.92:0/790817234 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2c940550b0 con 0x7f2ca40c9870 2023-11-24T06:12:16.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:16.115+0000 7f2caadce640 1 -- 172.21.15.92:0/790817234 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2ca40d0a20 con 0x7f2ca40c9870 2023-11-24T06:12:16.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:16.116+0000 7f2ca17fa640 1 -- 172.21.15.92:0/790817234 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+4919 (secure 0 0 0) 0x7f2c940586d0 con 0x7f2ca40c9870 2023-11-24T06:12:16.119 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-24T06:12:16.119 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":23,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4637,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6834/2665242481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":2665242481},{"type":"v1","addr":"172.21.15.118:6834","nonce":2665242481}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":4640,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6835/2822230615","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6833","nonce":2822230615},{"type":"v1","addr":"172.21.15.118:6835","nonce":2822230615}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5499,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6836/2918354210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":2918354210},{"type":"v1","addr":"172.21.15.92:6836","nonce":2918354210}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":23}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T06:11:44.503003+0000","modified":"2023-11-24T06:11:47.998422+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5235},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5235":{"gid":5235,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.92:6837/1413573590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":1413573590},{"type":"v1","addr":"172.21.15.92:6837","nonce":1413573590}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-24T06:12:16.120 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 23 2023-11-24T06:12:16.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:16.118+0000 7f2caadce640 1 -- 172.21.15.92:0/790817234 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f2c8406ae40 msgr2=0x7f2c8406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:16.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:16.118+0000 7f2caadce640 1 --2- 172.21.15.92:0/790817234 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f2c8406ae40 0x7f2c8406d300 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f2c8c036000 tx=0x7f2c8c03c040 comp rx=0 tx=0).stop 2023-11-24T06:12:16.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:16.118+0000 7f2caadce640 1 -- 172.21.15.92:0/790817234 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2ca40c9870 msgr2=0x7f2ca40cb4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:16.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:16.118+0000 7f2caadce640 1 --2- 172.21.15.92:0/790817234 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2ca40c9870 0x7f2ca40cb4f0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f2c94007ef0 tx=0x7f2c9400e4b0 comp rx=0 tx=0).stop 2023-11-24T06:12:16.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:16.118+0000 7f2caadce640 1 -- 172.21.15.92:0/790817234 shutdown_connections 2023-11-24T06:12:16.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:16.118+0000 7f2caadce640 1 --2- 172.21.15.92:0/790817234 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f2c8406ae40 0x7f2c8406d300 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:16.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:16.118+0000 7f2caadce640 1 --2- 172.21.15.92:0/790817234 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2ca40d2260 0x7f2ca40b8630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:16.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:16.118+0000 7f2caadce640 1 --2- 172.21.15.92:0/790817234 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2ca40d0560 0x7f2ca4157e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:16.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:16.118+0000 7f2caadce640 1 --2- 172.21.15.92:0/790817234 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2ca40c9870 0x7f2ca40cb4f0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:16.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:16.119+0000 7f2caadce640 1 -- 172.21.15.92:0/790817234 >> 172.21.15.92:0/790817234 conn(0x7f2ca409c560 msgr2=0x7f2ca4160870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:16.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:16.119+0000 7f2caadce640 1 -- 172.21.15.92:0/790817234 shutdown_connections 2023-11-24T06:12:16.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:16.119+0000 7f2caadce640 1 -- 172.21.15.92:0/790817234 wait complete. 2023-11-24T06:12:16.138 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:12:16.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:16.327+0000 7f88b300e640 1 Processor -- start 2023-11-24T06:12:16.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:16.327+0000 7f88b300e640 1 -- start start 2023-11-24T06:12:16.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:16.328+0000 7f88b300e640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f88ac13df80 0x7f88ac15bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:16.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:16.328+0000 7f88b300e640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f88ac13e360 0x7f88ac0cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:16.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:16.328+0000 7f88b300e640 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88ac06c470 con 0x7f88ac15f9c0 2023-11-24T06:12:16.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:16.328+0000 7f88b300e640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88ac06c750 con 0x7f88ac13df80 2023-11-24T06:12:16.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:16.328+0000 7f88b300e640 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88ac06ca30 con 0x7f88ac13e360 2023-11-24T06:12:16.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:16.329+0000 7f88b0daa640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f88ac13df80 0x7f88ac15bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:16.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:16.329+0000 7f88b0daa640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f88ac13df80 0x7f88ac15bee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:60624/0 (socket says 172.21.15.92:60624) 2023-11-24T06:12:16.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:16.329+0000 7f88b0daa640 1 -- 172.21.15.92:0/3917302901 learned_addr learned my addr 172.21.15.92:0/3917302901 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:12:16.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:16.329+0000 7f88abfff640 1 --2- 172.21.15.92:0/3917302901 >> v2:172.21.15.118:3300/0 conn(0x7f88ac13e360 0x7f88ac0cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:16.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:16.329+0000 7f88b0daa640 1 -- 172.21.15.92:0/3917302901 >> v2:172.21.15.118:3300/0 conn(0x7f88ac13e360 msgr2=0x7f88ac0cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:16.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:16.329+0000 7f88b0daa640 1 --2- 172.21.15.92:0/3917302901 >> v2:172.21.15.118:3300/0 conn(0x7f88ac13e360 0x7f88ac0cf290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:16.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:16.329+0000 7f88b0daa640 1 -- 172.21.15.92:0/3917302901 >> v1:172.21.15.118:6789/0 conn(0x7f88ac15f9c0 legacy=0x7f88ac1408d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:16.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:16.329+0000 7f88b0daa640 1 -- 172.21.15.92:0/3917302901 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88ac06cd10 con 0x7f88ac13df80 2023-11-24T06:12:16.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:16.330+0000 7f88b0daa640 1 --2- 172.21.15.92:0/3917302901 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f88ac13df80 0x7f88ac15bee0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f88ac06ded0 tx=0x7f88a002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56489be418c41b02 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:16.335 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:16.331+0000 7f88ab7fe640 1 -- 172.21.15.92:0/3917302901 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88a0031870 con 0x7f88ac13df80 2023-11-24T06:12:16.335 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:16.331+0000 7f88ab7fe640 1 -- 172.21.15.92:0/3917302901 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88a00319d0 con 0x7f88ac13df80 2023-11-24T06:12:16.335 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:16.331+0000 7f88ab7fe640 1 -- 172.21.15.92:0/3917302901 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88a0031d60 con 0x7f88ac13df80 2023-11-24T06:12:16.335 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:16.331+0000 7f88b300e640 1 -- 172.21.15.92:0/3917302901 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f88ac13df80 msgr2=0x7f88ac15bee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:16.335 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:16.331+0000 7f88b300e640 1 --2- 172.21.15.92:0/3917302901 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f88ac13df80 0x7f88ac15bee0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f88ac06ded0 tx=0x7f88a002b910 comp rx=0 tx=0).stop 2023-11-24T06:12:16.335 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:16.331+0000 7f88b300e640 1 -- 172.21.15.92:0/3917302901 shutdown_connections 2023-11-24T06:12:16.335 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:16.331+0000 7f88b300e640 1 --2- 172.21.15.92:0/3917302901 >> v2:172.21.15.118:3300/0 conn(0x7f88ac13e360 0x7f88ac0cf290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:16.335 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:16.331+0000 7f88b300e640 1 --2- 172.21.15.92:0/3917302901 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f88ac13df80 0x7f88ac15bee0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:16.335 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:16.331+0000 7f88b300e640 1 -- 172.21.15.92:0/3917302901 >> 172.21.15.92:0/3917302901 conn(0x7f88ac06a800 msgr2=0x7f88ac06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:16.335 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:16.331+0000 7f88b300e640 1 -- 172.21.15.92:0/3917302901 shutdown_connections 2023-11-24T06:12:16.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:16.331+0000 7f88b300e640 1 -- 172.21.15.92:0/3917302901 wait complete. 2023-11-24T06:12:16.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:16.333+0000 7f88b300e640 1 Processor -- start 2023-11-24T06:12:16.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:16.333+0000 7f88b300e640 1 -- start start 2023-11-24T06:12:16.338 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:16.334+0000 7f88b300e640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f88ac13df80 0x7f88ac14aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:16.338 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:16.334+0000 7f88b300e640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f88ac13e360 0x7f88ac1355d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:16.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:16.334+0000 7f88b300e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f88ac15f9c0 0x7f88ac0af0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:16.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:16.334+0000 7f88b300e640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88ac070d90 con 0x7f88ac15f9c0 2023-11-24T06:12:16.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:16.334+0000 7f88b300e640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88ac14c600 con 0x7f88ac13e360 2023-11-24T06:12:16.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:16.334+0000 7f88b300e640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88ac14b1b0 con 0x7f88ac13df80 2023-11-24T06:12:16.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:16.334+0000 7f88b15ab640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f88ac15f9c0 0x7f88ac0af0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:16.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:16.334+0000 7f88b15ab640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f88ac15f9c0 0x7f88ac0af0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:60630/0 (socket says 172.21.15.92:60630) 2023-11-24T06:12:16.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:16.334+0000 7f88b15ab640 1 -- 172.21.15.92:0/2218754602 learned_addr learned my addr 172.21.15.92:0/2218754602 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:12:16.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:16.335+0000 7f88abfff640 1 --2- 172.21.15.92:0/2218754602 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f88ac13e360 0x7f88ac1355d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:16.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:16.335+0000 7f88b0daa640 1 --2- 172.21.15.92:0/2218754602 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f88ac13df80 0x7f88ac14aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:16.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:16.335+0000 7f88b15ab640 1 -- 172.21.15.92:0/2218754602 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f88ac13df80 msgr2=0x7f88ac14aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:16.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:16.335+0000 7f88b15ab640 1 --2- 172.21.15.92:0/2218754602 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f88ac13df80 0x7f88ac14aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:16.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:16.335+0000 7f88b15ab640 1 -- 172.21.15.92:0/2218754602 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f88ac13e360 msgr2=0x7f88ac1355d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:16.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:16.335+0000 7f88b15ab640 1 --2- 172.21.15.92:0/2218754602 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f88ac13e360 0x7f88ac1355d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:16.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:16.335+0000 7f88b15ab640 1 -- 172.21.15.92:0/2218754602 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88a00094d0 con 0x7f88ac15f9c0 2023-11-24T06:12:16.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:16.335+0000 7f88b15ab640 1 --2- 172.21.15.92:0/2218754602 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f88ac15f9c0 0x7f88ac0af0b0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f88a400c7b0 tx=0x7f88a400cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:16.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:16.335+0000 7f88a97fa640 1 -- 172.21.15.92:0/2218754602 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88a4003d80 con 0x7f88ac15f9c0 2023-11-24T06:12:16.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:16.335+0000 7f88a97fa640 1 -- 172.21.15.92:0/2218754602 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88a400ce60 con 0x7f88ac15f9c0 2023-11-24T06:12:16.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:16.335+0000 7f88b300e640 1 -- 172.21.15.92:0/2218754602 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88ac0703f0 con 0x7f88ac15f9c0 2023-11-24T06:12:16.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:16.335+0000 7f88a97fa640 1 -- 172.21.15.92:0/2218754602 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88a40052b0 con 0x7f88ac15f9c0 2023-11-24T06:12:16.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:16.336+0000 7f88b300e640 1 -- 172.21.15.92:0/2218754602 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88ac0709b0 con 0x7f88ac15f9c0 2023-11-24T06:12:16.341 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:16.337+0000 7f88b300e640 1 -- 172.21.15.92:0/2218754602 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88ac0675d0 con 0x7f88ac15f9c0 2023-11-24T06:12:16.341 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:16.338+0000 7f88a97fa640 1 -- 172.21.15.92:0/2218754602 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f88a4006b10 con 0x7f88ac15f9c0 2023-11-24T06:12:16.341 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:16.338+0000 7f88a97fa640 1 --2- 172.21.15.92:0/2218754602 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f889006ae40 0x7f889006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:16.342 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:16.339+0000 7f88a97fa640 1 -- 172.21.15.92:0/2218754602 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f88a40889c0 con 0x7f88ac15f9c0 2023-11-24T06:12:16.342 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:16.339+0000 7f88b0daa640 1 --2- 172.21.15.92:0/2218754602 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f889006ae40 0x7f889006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:16.342 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:16.339+0000 7f88b0daa640 1 --2- 172.21.15.92:0/2218754602 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f889006ae40 0x7f889006d300 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f88a0036000 tx=0x7f88a003c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:16.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:16.343+0000 7f88a97fa640 1 -- 172.21.15.92:0/2218754602 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f88a4053ab0 con 0x7f88ac15f9c0 2023-11-24T06:12:16.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:16.536+0000 7f88b300e640 1 -- 172.21.15.92:0/2218754602 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f88ac0cc100 con 0x7f88ac15f9c0 2023-11-24T06:12:16.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:16.538+0000 7f88a97fa640 1 -- 172.21.15.92:0/2218754602 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+16650 (secure 0 0 0) 0x7f88a4017070 con 0x7f88ac15f9c0 2023-11-24T06:12:16.542 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-24T06:12:16.542 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":40,"fsid":"2d6dee5f-8aa3-470f-931a-50ade963b1cb","created":"2023-11-24T06:10:04.707204+0000","modified":"2023-11-24T06:12:06.456223+0000","last_up_change":"2023-11-24T06:10:17.598352+0000","last_in_change":"2023-11-24T06:10:09.040793+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:10:17.733098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:11:42.241681+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-11-24T06:11:43.522720+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ff37a9fb-135f-471f-b4b0-f7b5772006c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6803","nonce":20560714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6805","nonce":20560714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6809","nonce":20560714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6807","nonce":20560714}]},"public_addr":"172.21.15.92:6803/20560714","cluster_addr":"172.21.15.92:6805/20560714","heartbeat_back_addr":"172.21.15.92:6809/20560714","heartbeat_front_addr":"172.21.15.92:6807/20560714","state":["exists","up"]},{"osd":1,"uuid":"03cdd353-dcf1-4f38-94fe-a795137c06b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6819","nonce":2451239906}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6821","nonce":2451239906}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6825","nonce":2451239906}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6823","nonce":2451239906}]},"public_addr":"172.21.15.92:6819/2451239906","cluster_addr":"172.21.15.92:6821/2451239906","heartbeat_back_addr":"172.21.15.92:6825/2451239906","heartbeat_front_addr":"172.21.15.92:6823/2451239906","state":["exists","up"]},{"osd":2,"uuid":"a4a20d80-5d57-484b-becd-9aa8070dd3aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6811","nonce":1124979277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6813","nonce":1124979277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6817","nonce":1124979277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6815","nonce":1124979277}]},"public_addr":"172.21.15.92:6811/1124979277","cluster_addr":"172.21.15.92:6813/1124979277","heartbeat_back_addr":"172.21.15.92:6817/1124979277","heartbeat_front_addr":"172.21.15.92:6815/1124979277","state":["exists","up"]},{"osd":3,"uuid":"4648d7ce-9b29-4583-9425-4ba8fd2b1903","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6827","nonce":1734115782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6829","nonce":1734115782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6833","nonce":1734115782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6831","nonce":1734115782}]},"public_addr":"172.21.15.92:6827/1734115782","cluster_addr":"172.21.15.92:6829/1734115782","heartbeat_back_addr":"172.21.15.92:6833/1734115782","heartbeat_front_addr":"172.21.15.92:6831/1734115782","state":["exists","up"]},{"osd":4,"uuid":"3bc9d7eb-c6fa-4fcf-a3ec-ade604e05aac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6819","nonce":3425937229}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6822","nonce":3425937229}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6830","nonce":3425937229}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6826","nonce":3425937229}]},"public_addr":"172.21.15.118:6819/3425937229","cluster_addr":"172.21.15.118:6822/3425937229","heartbeat_back_addr":"172.21.15.118:6830/3425937229","heartbeat_front_addr":"172.21.15.118:6826/3425937229","state":["exists","up"]},{"osd":5,"uuid":"dce9ba6d-59ed-46d1-84fc-a84b13de10c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6801","nonce":827455984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6803","nonce":827455984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6807","nonce":827455984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6805","nonce":827455984}]},"public_addr":"172.21.15.118:6801/827455984","cluster_addr":"172.21.15.118:6803/827455984","heartbeat_back_addr":"172.21.15.118:6807/827455984","heartbeat_front_addr":"172.21.15.118:6805/827455984","state":["exists","up"]},{"osd":6,"uuid":"82aec504-ae58-4709-8d30-46f2b12081b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6809","nonce":2946986141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6811","nonce":2946986141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6815","nonce":2946986141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6813","nonce":2946986141}]},"public_addr":"172.21.15.118:6809/2946986141","cluster_addr":"172.21.15.118:6811/2946986141","heartbeat_back_addr":"172.21.15.118:6815/2946986141","heartbeat_front_addr":"172.21.15.118:6813/2946986141","state":["exists","up"]},{"osd":7,"uuid":"4ed4c624-c88f-4d35-bdb1-bf209b5dffb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6817","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6818","nonce":48122059}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6821","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6823","nonce":48122059}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6829","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6831","nonce":48122059}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6825","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6827","nonce":48122059}]},"public_addr":"172.21.15.118:6818/48122059","cluster_addr":"172.21.15.118:6823/48122059","heartbeat_back_addr":"172.21.15.118:6831/48122059","heartbeat_front_addr":"172.21.15.118:6827/48122059","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.536664+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.368324+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.271201+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.342971+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.621832+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.387110+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.632946+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.416032+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.92:6836/3889219158":"2023-11-25T06:12:03.443855+0000","172.21.15.92:6834/3889219158":"2023-11-25T06:12:03.443855+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:12:16.544 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:16.541+0000 7f88b300e640 1 -- 172.21.15.92:0/2218754602 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f889006ae40 msgr2=0x7f889006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:16.544 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:16.541+0000 7f88b300e640 1 --2- 172.21.15.92:0/2218754602 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f889006ae40 0x7f889006d300 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f88a0036000 tx=0x7f88a003c040 comp rx=0 tx=0).stop 2023-11-24T06:12:16.544 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:16.541+0000 7f88b300e640 1 -- 172.21.15.92:0/2218754602 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f88ac15f9c0 msgr2=0x7f88ac0af0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:16.544 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:16.541+0000 7f88b300e640 1 --2- 172.21.15.92:0/2218754602 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f88ac15f9c0 0x7f88ac0af0b0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f88a400c7b0 tx=0x7f88a400cc70 comp rx=0 tx=0).stop 2023-11-24T06:12:16.544 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:16.541+0000 7f88b300e640 1 -- 172.21.15.92:0/2218754602 shutdown_connections 2023-11-24T06:12:16.544 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:16.541+0000 7f88b300e640 1 --2- 172.21.15.92:0/2218754602 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f889006ae40 0x7f889006d300 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:16.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:16.541+0000 7f88b300e640 1 --2- 172.21.15.92:0/2218754602 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f88ac15f9c0 0x7f88ac0af0b0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:16.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:16.541+0000 7f88b300e640 1 --2- 172.21.15.92:0/2218754602 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f88ac13e360 0x7f88ac1355d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:16.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:16.541+0000 7f88b300e640 1 --2- 172.21.15.92:0/2218754602 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f88ac13df80 0x7f88ac14aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:16.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:16.541+0000 7f88b300e640 1 -- 172.21.15.92:0/2218754602 >> 172.21.15.92:0/2218754602 conn(0x7f88ac06a800 msgr2=0x7f88ac15ce50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:16.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:16.541+0000 7f88b300e640 1 -- 172.21.15.92:0/2218754602 shutdown_connections 2023-11-24T06:12:16.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:16.541+0000 7f88b300e640 1 -- 172.21.15.92:0/2218754602 wait complete. 2023-11-24T06:12:16.564 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-24T06:12:16.760 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T06:12:16.769 DEBUG:teuthology.orchestra.run.smithi118:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T06:12:16.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:16.775+0000 7f51eef0f640 1 Processor -- start 2023-11-24T06:12:16.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:16.775+0000 7f51eef0f640 1 -- start start 2023-11-24T06:12:16.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:16.776+0000 7f51eef0f640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f51e80ced30 0x7f51e80b3110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:16.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:16.776+0000 7f51eef0f640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51e80cf1c0 con 0x7f51e8160f80 2023-11-24T06:12:16.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:16.776+0000 7f51eef0f640 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51e80d0de0 con 0x7f51e80ce8f0 2023-11-24T06:12:16.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:16.776+0000 7f51eef0f640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51e80d10c0 con 0x7f51e80ced30 2023-11-24T06:12:16.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:16.777+0000 7f51eccab640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f51e8160f80 0x7f51e8161360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:33818/0 (socket says 172.21.15.92:33818) 2023-11-24T06:12:16.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:16.777+0000 7f51eccab640 1 -- 172.21.15.92:0/1377409702 learned_addr learned my addr 172.21.15.92:0/1377409702 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-24T06:12:16.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:16.777+0000 7f51ed4ac640 1 --2- 172.21.15.92:0/1377409702 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f51e80ced30 0x7f51e80b3110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:16.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:16.777+0000 7f51e77fe640 1 -- 172.21.15.92:0/1377409702 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 982890869 0 0) 0x7f51dc001110 con 0x7f51e8160f80 2023-11-24T06:12:16.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:16.777+0000 7f51e77fe640 1 -- 172.21.15.92:0/1377409702 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f51d8003520 con 0x7f51e8160f80 2023-11-24T06:12:16.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:16.778+0000 7f51e77fe640 1 -- 172.21.15.92:0/1377409702 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 362768272 0 0) 0x7f51d8003520 con 0x7f51e8160f80 2023-11-24T06:12:16.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:16.778+0000 7f51ed4ac640 1 -- 172.21.15.92:0/1377409702 >> v1:172.21.15.118:6789/0 conn(0x7f51e80ce8f0 legacy=0x7f51e80b2a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:16.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:16.778+0000 7f51ed4ac640 1 -- 172.21.15.92:0/1377409702 >> v1:172.21.15.92:6789/0 conn(0x7f51e8160f80 legacy=0x7f51e8161360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:16.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:16.778+0000 7f51ed4ac640 1 -- 172.21.15.92:0/1377409702 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51e80d13a0 con 0x7f51e80ced30 2023-11-24T06:12:16.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:16.780+0000 7f51ed4ac640 1 --2- 172.21.15.92:0/1377409702 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f51e80ced30 0x7f51e80b3110 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f51e0009dc0 tx=0x7f51e002bde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4873bccbab12063d server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:16.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:16.780+0000 7f51e77fe640 1 -- 172.21.15.92:0/1377409702 <== mon.2 v2:172.21.15.118:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51dc001800 con 0x7f51e80ced30 2023-11-24T06:12:16.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:16.780+0000 7f51e77fe640 1 -- 172.21.15.92:0/1377409702 <== mon.2 v2:172.21.15.118:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51e002c910 con 0x7f51e80ced30 2023-11-24T06:12:16.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:16.780+0000 7f51e77fe640 1 -- 172.21.15.92:0/1377409702 <== mon.2 v2:172.21.15.118:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51e002cca0 con 0x7f51e80ced30 2023-11-24T06:12:16.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:16.781+0000 7f51eef0f640 1 -- 172.21.15.92:0/1377409702 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f51e80ced30 msgr2=0x7f51e80b3110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:16.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:16.781+0000 7f51eef0f640 1 --2- 172.21.15.92:0/1377409702 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f51e80ced30 0x7f51e80b3110 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f51e0009dc0 tx=0x7f51e002bde0 comp rx=0 tx=0).stop 2023-11-24T06:12:16.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:16.781+0000 7f51eef0f640 1 -- 172.21.15.92:0/1377409702 shutdown_connections 2023-11-24T06:12:16.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:16.781+0000 7f51eef0f640 1 --2- 172.21.15.92:0/1377409702 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f51e80ced30 0x7f51e80b3110 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:16.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:16.781+0000 7f51eef0f640 1 -- 172.21.15.92:0/1377409702 >> 172.21.15.92:0/1377409702 conn(0x7f51e809c550 msgr2=0x7f51e80d3210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:16.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:16.781+0000 7f51eef0f640 1 -- 172.21.15.92:0/1377409702 shutdown_connections 2023-11-24T06:12:16.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:16.781+0000 7f51eef0f640 1 -- 172.21.15.92:0/1377409702 wait complete. 2023-11-24T06:12:16.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:16.783+0000 7f51eef0f640 1 Processor -- start 2023-11-24T06:12:16.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:16.783+0000 7f51eef0f640 1 -- start start 2023-11-24T06:12:16.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:16.784+0000 7f51eef0f640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f51e80ce8f0 0x7f51e81582b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:16.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:16.784+0000 7f51eef0f640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f51e80ced30 0x7f51e80c6ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:16.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:16.784+0000 7f51eef0f640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f51e8160f80 0x7f51e80b3b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:16.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:16.784+0000 7f51eef0f640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51e81e6d80 con 0x7f51e8160f80 2023-11-24T06:12:16.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:16.784+0000 7f51eef0f640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51e81e6fd0 con 0x7f51e80ced30 2023-11-24T06:12:16.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:16.784+0000 7f51eef0f640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51e81e72b0 con 0x7f51e80ce8f0 2023-11-24T06:12:16.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:16.784+0000 7f51ed4ac640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f51e8160f80 0x7f51e80b3b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:16.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:16.784+0000 7f51ed4ac640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f51e8160f80 0x7f51e80b3b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:60634/0 (socket says 172.21.15.92:60634) 2023-11-24T06:12:16.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:16.784+0000 7f51ed4ac640 1 -- 172.21.15.92:0/1703566942 learned_addr learned my addr 172.21.15.92:0/1703566942 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:12:16.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:16.784+0000 7f51eccab640 1 --2- 172.21.15.92:0/1703566942 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f51e80ce8f0 0x7f51e81582b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:16.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:16.784+0000 7f51e7fff640 1 --2- 172.21.15.92:0/1703566942 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f51e80ced30 0x7f51e80c6ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:16.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:16.785+0000 7f51ed4ac640 1 -- 172.21.15.92:0/1703566942 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f51e80ce8f0 msgr2=0x7f51e81582b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:16.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:16.785+0000 7f51ed4ac640 1 --2- 172.21.15.92:0/1703566942 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f51e80ce8f0 0x7f51e81582b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:16.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:16.785+0000 7f51ed4ac640 1 -- 172.21.15.92:0/1703566942 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f51e80ced30 msgr2=0x7f51e80c6ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:16.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:16.785+0000 7f51ed4ac640 1 --2- 172.21.15.92:0/1703566942 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f51e80ced30 0x7f51e80c6ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:16.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:16.785+0000 7f51ed4ac640 1 -- 172.21.15.92:0/1703566942 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51e0009a70 con 0x7f51e8160f80 2023-11-24T06:12:16.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:16.785+0000 7f51eccab640 1 --2- 172.21.15.92:0/1703566942 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f51e80ce8f0 0x7f51e81582b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:12:16.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:16.785+0000 7f51ed4ac640 1 --2- 172.21.15.92:0/1703566942 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f51e8160f80 0x7f51e80b3b20 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f51e002be20 tx=0x7f51e0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:16.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:16.785+0000 7f51e57fa640 1 -- 172.21.15.92:0/1703566942 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51e0009450 con 0x7f51e8160f80 2023-11-24T06:12:16.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:16.786+0000 7f51e57fa640 1 -- 172.21.15.92:0/1703566942 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51e003e410 con 0x7f51e8160f80 2023-11-24T06:12:16.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:16.786+0000 7f51eef0f640 1 -- 172.21.15.92:0/1703566942 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51e81e7590 con 0x7f51e8160f80 2023-11-24T06:12:16.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:16.786+0000 7f51e57fa640 1 -- 172.21.15.92:0/1703566942 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51e003e7c0 con 0x7f51e8160f80 2023-11-24T06:12:16.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:16.786+0000 7f51eef0f640 1 -- 172.21.15.92:0/1703566942 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51e81e7b50 con 0x7f51e8160f80 2023-11-24T06:12:16.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:16.787+0000 7f51eef0f640 1 -- 172.21.15.92:0/1703566942 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51e80675d0 con 0x7f51e8160f80 2023-11-24T06:12:16.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:16.788+0000 7f51e57fa640 1 -- 172.21.15.92:0/1703566942 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f51e003b050 con 0x7f51e8160f80 2023-11-24T06:12:16.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:16.789+0000 7f51e57fa640 1 --2- 172.21.15.92:0/1703566942 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f51cc06ae40 0x7f51cc06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:16.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:16.789+0000 7f51e57fa640 1 -- 172.21.15.92:0/1703566942 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f51e00abd00 con 0x7f51e8160f80 2023-11-24T06:12:16.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:16.789+0000 7f51eccab640 1 --2- 172.21.15.92:0/1703566942 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f51cc06ae40 0x7f51cc06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:16.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:16.790+0000 7f51eccab640 1 --2- 172.21.15.92:0/1703566942 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f51cc06ae40 0x7f51cc06d300 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f51dc001370 tx=0x7f51dc009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:16.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:16.793+0000 7f51e57fa640 1 -- 172.21.15.92:0/1703566942 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f51e0076d60 con 0x7f51e8160f80 2023-11-24T06:12:17.025 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:17.022+0000 7f51eef0f640 1 -- 172.21.15.92:0/1703566942 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f51e80d3920 con 0x7f51e8160f80 2023-11-24T06:12:17.026 INFO:tasks.ceph.mon.a.smithi092.stderr:2023-11-24T06:12:17.023+0000 7f83f3ce4640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-24T06:12:17.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:17.026+0000 7f51e57fa640 1 -- 172.21.15.92:0/1703566942 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v24) v1 ==== 161+0+0 (secure 0 0 0) 0x7f51e007a380 con 0x7f51e8160f80 2023-11-24T06:12:17.029 INFO:teuthology.orchestra.run.smithi092.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-24T06:12:17.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:17.029+0000 7f51eef0f640 1 -- 172.21.15.92:0/1703566942 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f51cc06ae40 msgr2=0x7f51cc06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:17.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:17.029+0000 7f51eef0f640 1 --2- 172.21.15.92:0/1703566942 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f51cc06ae40 0x7f51cc06d300 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f51dc001370 tx=0x7f51dc009040 comp rx=0 tx=0).stop 2023-11-24T06:12:17.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:17.029+0000 7f51eef0f640 1 -- 172.21.15.92:0/1703566942 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f51e8160f80 msgr2=0x7f51e80b3b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:17.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:17.029+0000 7f51eef0f640 1 --2- 172.21.15.92:0/1703566942 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f51e8160f80 0x7f51e80b3b20 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f51e002be20 tx=0x7f51e0004a40 comp rx=0 tx=0).stop 2023-11-24T06:12:17.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:17.030+0000 7f51eef0f640 1 -- 172.21.15.92:0/1703566942 shutdown_connections 2023-11-24T06:12:17.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:17.030+0000 7f51eef0f640 1 --2- 172.21.15.92:0/1703566942 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f51cc06ae40 0x7f51cc06d300 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:17.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:17.030+0000 7f51eef0f640 1 --2- 172.21.15.92:0/1703566942 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f51e8160f80 0x7f51e80b3b20 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:17.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:17.030+0000 7f51eef0f640 1 --2- 172.21.15.92:0/1703566942 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f51e80ced30 0x7f51e80c6ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:17.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:17.030+0000 7f51eef0f640 1 --2- 172.21.15.92:0/1703566942 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f51e80ce8f0 0x7f51e81582b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:17.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:17.030+0000 7f51eef0f640 1 -- 172.21.15.92:0/1703566942 >> 172.21.15.92:0/1703566942 conn(0x7f51e809c550 msgr2=0x7f51e80d3210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:17.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:17.030+0000 7f51eef0f640 1 -- 172.21.15.92:0/1703566942 shutdown_connections 2023-11-24T06:12:17.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:17.030+0000 7f51eef0f640 1 -- 172.21.15.92:0/1703566942 wait complete. 2023-11-24T06:12:17.049 INFO:tasks.ceph.mds.c.smithi092.stderr: -1618> 2023-11-24T06:11:46.993+0000 7f16ce26b640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-24T06:12:17.051 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-11-24T06:12:17.109 INFO:tasks.ceph.mds.c.smithi092.stdout:starting mds.c at 2023-11-24T06:12:17.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:17.228+0000 7fe6c0a7c640 1 Processor -- start 2023-11-24T06:12:17.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:17.228+0000 7fe6c0a7c640 1 -- start start 2023-11-24T06:12:17.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:17.229+0000 7fe6c0a7c640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fe6bc150760 0x7fe6bc072230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:17.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:17.230+0000 7fe6c0a7c640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe6bc0b32e0 0x7fe6bc15d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:17.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:17.230+0000 7fe6c0a7c640 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6bc0b3750 con 0x7fe6bc15d300 2023-11-24T06:12:17.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:17.230+0000 7fe6c0a7c640 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6bc15dc80 con 0x7fe6bc150760 2023-11-24T06:12:17.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:17.230+0000 7fe6c0a7c640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6bc071190 con 0x7fe6bc0b32e0 2023-11-24T06:12:17.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:17.230+0000 7fe6bad9d640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe6bc0b32e0 0x7fe6bc15d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:17.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:17.230+0000 7fe6ba59c640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fe6bc150760 0x7fe6bc072230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:17.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:17.230+0000 7fe6bad9d640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe6bc0b32e0 0x7fe6bc15d740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3301/0 says I am v2:172.21.15.92:38098/0 (socket says 172.21.15.92:38098) 2023-11-24T06:12:17.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:17.230+0000 7fe6bad9d640 1 -- 172.21.15.92:0/774104842 learned_addr learned my addr 172.21.15.92:0/774104842 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:12:17.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:17.231+0000 7fe6ba59c640 1 -- 172.21.15.92:0/774104842 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe6bc0b32e0 msgr2=0x7fe6bc15d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:17.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:17.231+0000 7fe6ba59c640 1 --2- 172.21.15.92:0/774104842 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe6bc0b32e0 0x7fe6bc15d740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:17.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:17.231+0000 7fe6ba59c640 1 -- 172.21.15.92:0/774104842 >> v1:172.21.15.118:6789/0 conn(0x7fe6bc15d300 legacy=0x7fe6bc0b2bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:17.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:17.231+0000 7fe6ba59c640 1 -- 172.21.15.92:0/774104842 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6bc071470 con 0x7fe6bc150760 2023-11-24T06:12:17.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:17.231+0000 7fe6bad9d640 1 --2- 172.21.15.92:0/774104842 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe6bc0b32e0 0x7fe6bc15d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:12:17.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:17.233+0000 7fe6ba59c640 1 --2- 172.21.15.92:0/774104842 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe6bc150760 0x7fe6bc072230 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fe6ac0097a0 tx=0x7fe6ac02b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1941e8f98745602e server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:17.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:17.233+0000 7fe6b959a640 1 -- 172.21.15.92:0/774104842 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6ac02d040 con 0x7fe6bc150760 2023-11-24T06:12:17.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:17.233+0000 7fe6b959a640 1 -- 172.21.15.92:0/774104842 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6ac031810 con 0x7fe6bc150760 2023-11-24T06:12:17.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:17.233+0000 7fe6b959a640 1 -- 172.21.15.92:0/774104842 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6ac031ba0 con 0x7fe6bc150760 2023-11-24T06:12:17.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:17.233+0000 7fe6c0a7c640 1 -- 172.21.15.92:0/774104842 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe6bc150760 msgr2=0x7fe6bc072230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:17.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:17.233+0000 7fe6c0a7c640 1 --2- 172.21.15.92:0/774104842 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe6bc150760 0x7fe6bc072230 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fe6ac0097a0 tx=0x7fe6ac02b8b0 comp rx=0 tx=0).stop 2023-11-24T06:12:17.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:17.234+0000 7fe6c0a7c640 1 -- 172.21.15.92:0/774104842 shutdown_connections 2023-11-24T06:12:17.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:17.234+0000 7fe6c0a7c640 1 --2- 172.21.15.92:0/774104842 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe6bc0b32e0 0x7fe6bc15d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:17.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:17.234+0000 7fe6c0a7c640 1 --2- 172.21.15.92:0/774104842 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe6bc150760 0x7fe6bc072230 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:17.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:17.234+0000 7fe6c0a7c640 1 -- 172.21.15.92:0/774104842 >> 172.21.15.92:0/774104842 conn(0x7fe6bc06a800 msgr2=0x7fe6bc06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:17.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:17.234+0000 7fe6c0a7c640 1 -- 172.21.15.92:0/774104842 shutdown_connections 2023-11-24T06:12:17.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:17.234+0000 7fe6c0a7c640 1 -- 172.21.15.92:0/774104842 wait complete. 2023-11-24T06:12:17.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:17.235+0000 7fe6c0a7c640 1 Processor -- start 2023-11-24T06:12:17.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:17.235+0000 7fe6c0a7c640 1 -- start start 2023-11-24T06:12:17.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:17.236+0000 7fe6c0a7c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe6bc0b32e0 0x7fe6bc1470b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:17.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:17.236+0000 7fe6c0a7c640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe6bc150760 0x7fe6bc1590f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:17.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:17.236+0000 7fe6c0a7c640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe6bc15d300 0x7fe6bc0af880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:17.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:17.236+0000 7fe6c0a7c640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6bc1475f0 con 0x7fe6bc0b32e0 2023-11-24T06:12:17.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:17.236+0000 7fe6c0a7c640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6bc0ad520 con 0x7fe6bc15d300 2023-11-24T06:12:17.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:17.236+0000 7fe6c0a7c640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6bc0ad800 con 0x7fe6bc150760 2023-11-24T06:12:17.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:17.237+0000 7fe6ba59c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe6bc0b32e0 0x7fe6bc1470b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:17.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:17.237+0000 7fe6ba59c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe6bc0b32e0 0x7fe6bc1470b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:60650/0 (socket says 172.21.15.92:60650) 2023-11-24T06:12:17.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:17.237+0000 7fe6ba59c640 1 -- 172.21.15.92:0/87906486 learned_addr learned my addr 172.21.15.92:0/87906486 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:12:17.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:17.237+0000 7fe6ba59c640 1 -- 172.21.15.92:0/87906486 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe6bc150760 msgr2=0x7fe6bc1590f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:17.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:17.237+0000 7fe6ba59c640 1 --2- 172.21.15.92:0/87906486 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe6bc150760 0x7fe6bc1590f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:17.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:17.237+0000 7fe6ba59c640 1 -- 172.21.15.92:0/87906486 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe6bc15d300 msgr2=0x7fe6bc0af880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:17.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:17.237+0000 7fe6ba59c640 1 --2- 172.21.15.92:0/87906486 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe6bc15d300 0x7fe6bc0af880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:17.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:17.237+0000 7fe6ba59c640 1 -- 172.21.15.92:0/87906486 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6b400b760 con 0x7fe6bc0b32e0 2023-11-24T06:12:17.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:17.237+0000 7fe6ba59c640 1 --2- 172.21.15.92:0/87906486 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe6bc0b32e0 0x7fe6bc1470b0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fe6ac02b8f0 tx=0x7fe6ac0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:17.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:17.238+0000 7fe6b2ffd640 1 -- 172.21.15.92:0/87906486 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6ac006e10 con 0x7fe6bc0b32e0 2023-11-24T06:12:17.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:17.238+0000 7fe6b2ffd640 1 -- 172.21.15.92:0/87906486 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6ac03ad60 con 0x7fe6bc0b32e0 2023-11-24T06:12:17.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:17.238+0000 7fe6c0a7c640 1 -- 172.21.15.92:0/87906486 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6ac009400 con 0x7fe6bc0b32e0 2023-11-24T06:12:17.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:17.238+0000 7fe6b2ffd640 1 -- 172.21.15.92:0/87906486 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6ac044630 con 0x7fe6bc0b32e0 2023-11-24T06:12:17.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:17.238+0000 7fe6c0a7c640 1 -- 172.21.15.92:0/87906486 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6bc0cc1c0 con 0x7fe6bc0b32e0 2023-11-24T06:12:17.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:17.239+0000 7fe6c0a7c640 1 -- 172.21.15.92:0/87906486 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6bc06d480 con 0x7fe6bc0b32e0 2023-11-24T06:12:17.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:17.240+0000 7fe6b2ffd640 1 -- 172.21.15.92:0/87906486 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe6ac04d450 con 0x7fe6bc0b32e0 2023-11-24T06:12:17.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:17.240+0000 7fe6b2ffd640 1 --2- 172.21.15.92:0/87906486 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fe69c06ae40 0x7fe69c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:17.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:17.241+0000 7fe6b2ffd640 1 -- 172.21.15.92:0/87906486 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 5881+0+0 (secure 0 0 0) 0x7fe6ac0ab9b0 con 0x7fe6bc0b32e0 2023-11-24T06:12:17.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:17.241+0000 7fe6b9d9b640 1 --2- 172.21.15.92:0/87906486 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fe69c06ae40 0x7fe69c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:17.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:17.241+0000 7fe6b9d9b640 1 --2- 172.21.15.92:0/87906486 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fe69c06ae40 0x7fe69c06d300 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fe6a40012e0 tx=0x7fe6a4007d50 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:17.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:17.245+0000 7fe6b2ffd640 1 -- 172.21.15.92:0/87906486 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe6ac076910 con 0x7fe6bc0b32e0 2023-11-24T06:12:17.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:17.473+0000 7fe6c0a7c640 1 -- 172.21.15.92:0/87906486 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fe6bc0675d0 con 0x7fe6bc0b32e0 2023-11-24T06:12:18.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:18.029+0000 7fe6b2ffd640 1 -- 172.21.15.92:0/87906486 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v25) v1 ==== 104+0+0 (secure 0 0 0) 0x7fe6ac079f30 con 0x7fe6bc0b32e0 2023-11-24T06:12:18.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:18.033+0000 7fe6c0a7c640 1 -- 172.21.15.92:0/87906486 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fe69c06ae40 msgr2=0x7fe69c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:18.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:18.033+0000 7fe6c0a7c640 1 --2- 172.21.15.92:0/87906486 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fe69c06ae40 0x7fe69c06d300 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fe6a40012e0 tx=0x7fe6a4007d50 comp rx=0 tx=0).stop 2023-11-24T06:12:18.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:18.033+0000 7fe6c0a7c640 1 -- 172.21.15.92:0/87906486 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe6bc0b32e0 msgr2=0x7fe6bc1470b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:18.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:18.033+0000 7fe6c0a7c640 1 --2- 172.21.15.92:0/87906486 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe6bc0b32e0 0x7fe6bc1470b0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fe6ac02b8f0 tx=0x7fe6ac0046c0 comp rx=0 tx=0).stop 2023-11-24T06:12:18.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:18.033+0000 7fe6c0a7c640 1 -- 172.21.15.92:0/87906486 shutdown_connections 2023-11-24T06:12:18.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:18.033+0000 7fe6c0a7c640 1 --2- 172.21.15.92:0/87906486 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fe69c06ae40 0x7fe69c06d300 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:18.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:18.033+0000 7fe6c0a7c640 1 --2- 172.21.15.92:0/87906486 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe6bc15d300 0x7fe6bc0af880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:18.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:18.033+0000 7fe6c0a7c640 1 --2- 172.21.15.92:0/87906486 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe6bc150760 0x7fe6bc1590f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:18.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:18.033+0000 7fe6c0a7c640 1 --2- 172.21.15.92:0/87906486 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe6bc0b32e0 0x7fe6bc1470b0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:18.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:18.033+0000 7fe6c0a7c640 1 -- 172.21.15.92:0/87906486 >> 172.21.15.92:0/87906486 conn(0x7fe6bc06a800 msgr2=0x7fe6bc147880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:18.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:18.033+0000 7fe6c0a7c640 1 -- 172.21.15.92:0/87906486 shutdown_connections 2023-11-24T06:12:18.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:18.033+0000 7fe6c0a7c640 1 -- 172.21.15.92:0/87906486 wait complete. 2023-11-24T06:12:18.059 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-11-24T06:12:18.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:18.273+0000 7f8181a5c640 1 Processor -- start 2023-11-24T06:12:18.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:18.273+0000 7f8181a5c640 1 -- start start 2023-11-24T06:12:18.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:18.275+0000 7f8181a5c640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f817c13e360 0x7f817c0cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:18.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:18.275+0000 7f8181a5c640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f817c15f9c0 0x7f817c0cf7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:18.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:18.275+0000 7f8181a5c640 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f817c15c5f0 con 0x7f817c13df80 2023-11-24T06:12:18.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:18.275+0000 7f8181a5c640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f817c06c470 con 0x7f817c15f9c0 2023-11-24T06:12:18.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:18.275+0000 7f8181a5c640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f817c06c750 con 0x7f817c13e360 2023-11-24T06:12:18.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:18.275+0000 7f817b7fe640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f817c15f9c0 0x7f817c0cf7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:18.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:18.275+0000 7f817b7fe640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f817c15f9c0 0x7f817c0cf7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:60664/0 (socket says 172.21.15.92:60664) 2023-11-24T06:12:18.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:18.275+0000 7f817b7fe640 1 -- 172.21.15.92:0/3214389235 learned_addr learned my addr 172.21.15.92:0/3214389235 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:12:18.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:18.275+0000 7f817b7fe640 1 -- 172.21.15.92:0/3214389235 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f817c13e360 msgr2=0x7f817c0cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:18.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:18.275+0000 7f817a7fc640 1 --2- 172.21.15.92:0/3214389235 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f817c13e360 0x7f817c0cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:18.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:18.276+0000 7f817b7fe640 1 --2- 172.21.15.92:0/3214389235 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f817c13e360 0x7f817c0cf290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:18.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:18.276+0000 7f817b7fe640 1 -- 172.21.15.92:0/3214389235 >> v1:172.21.15.118:6789/0 conn(0x7f817c13df80 legacy=0x7f817c15bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:18.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:18.276+0000 7f817b7fe640 1 -- 172.21.15.92:0/3214389235 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f817c06ca30 con 0x7f817c15f9c0 2023-11-24T06:12:18.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:18.276+0000 7f817a7fc640 1 --2- 172.21.15.92:0/3214389235 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f817c13e360 0x7f817c0cf290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:12:18.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:18.277+0000 7f817b7fe640 1 --2- 172.21.15.92:0/3214389235 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f817c15f9c0 0x7f817c0cf7d0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f8174009e50 tx=0x7f817402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=395cb1d183ca5932 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:18.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:18.277+0000 7f8179ffb640 1 -- 172.21.15.92:0/3214389235 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f817402ca70 con 0x7f817c15f9c0 2023-11-24T06:12:18.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:18.277+0000 7f8179ffb640 1 -- 172.21.15.92:0/3214389235 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f817402cbd0 con 0x7f817c15f9c0 2023-11-24T06:12:18.282 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:18.277+0000 7f8179ffb640 1 -- 172.21.15.92:0/3214389235 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81740343f0 con 0x7f817c15f9c0 2023-11-24T06:12:18.282 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:18.278+0000 7f8181a5c640 1 -- 172.21.15.92:0/3214389235 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f817c15f9c0 msgr2=0x7f817c0cf7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:18.282 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:18.278+0000 7f8181a5c640 1 --2- 172.21.15.92:0/3214389235 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f817c15f9c0 0x7f817c0cf7d0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f8174009e50 tx=0x7f817402bed0 comp rx=0 tx=0).stop 2023-11-24T06:12:18.282 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:18.278+0000 7f8181a5c640 1 -- 172.21.15.92:0/3214389235 shutdown_connections 2023-11-24T06:12:18.282 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:18.278+0000 7f8181a5c640 1 --2- 172.21.15.92:0/3214389235 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f817c15f9c0 0x7f817c0cf7d0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:18.282 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:18.278+0000 7f8181a5c640 1 --2- 172.21.15.92:0/3214389235 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f817c13e360 0x7f817c0cf290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:18.282 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:18.278+0000 7f8181a5c640 1 -- 172.21.15.92:0/3214389235 >> 172.21.15.92:0/3214389235 conn(0x7f817c06a800 msgr2=0x7f817c06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:18.282 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:18.278+0000 7f8181a5c640 1 -- 172.21.15.92:0/3214389235 shutdown_connections 2023-11-24T06:12:18.282 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:18.278+0000 7f8181a5c640 1 -- 172.21.15.92:0/3214389235 wait complete. 2023-11-24T06:12:18.283 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:18.279+0000 7f8181a5c640 1 Processor -- start 2023-11-24T06:12:18.283 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:18.280+0000 7f8181a5c640 1 -- start start 2023-11-24T06:12:18.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:18.281+0000 7f8181a5c640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f817c13df80 0x7f817c1f6eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:18.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:18.281+0000 7f8181a5c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f817c13e360 0x7f817c13d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:18.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:18.281+0000 7f8181a5c640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f817c15f9c0 0x7f817c1f2f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:18.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:18.281+0000 7f8181a5c640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f817c1f6a90 con 0x7f817c13e360 2023-11-24T06:12:18.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:18.281+0000 7f8181a5c640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f817c1f3ef0 con 0x7f817c15f9c0 2023-11-24T06:12:18.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:18.281+0000 7f8181a5c640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f817c1f73f0 con 0x7f817c13df80 2023-11-24T06:12:18.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:18.281+0000 7f817a7fc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f817c13e360 0x7f817c13d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:18.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:18.281+0000 7f817a7fc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f817c13e360 0x7f817c13d710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:60676/0 (socket says 172.21.15.92:60676) 2023-11-24T06:12:18.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:18.281+0000 7f817a7fc640 1 -- 172.21.15.92:0/2803766272 learned_addr learned my addr 172.21.15.92:0/2803766272 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:12:18.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:18.281+0000 7f817b7fe640 1 --2- 172.21.15.92:0/2803766272 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f817c15f9c0 0x7f817c1f2f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:18.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:18.281+0000 7f817affd640 1 --2- 172.21.15.92:0/2803766272 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f817c13df80 0x7f817c1f6eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:18.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:18.281+0000 7f817a7fc640 1 -- 172.21.15.92:0/2803766272 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f817c13df80 msgr2=0x7f817c1f6eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:18.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:18.281+0000 7f817a7fc640 1 --2- 172.21.15.92:0/2803766272 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f817c13df80 0x7f817c1f6eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:18.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:18.281+0000 7f817a7fc640 1 -- 172.21.15.92:0/2803766272 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f817c15f9c0 msgr2=0x7f817c1f2f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:18.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:18.281+0000 7f817a7fc640 1 --2- 172.21.15.92:0/2803766272 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f817c15f9c0 0x7f817c1f2f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:18.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:18.281+0000 7f817a7fc640 1 -- 172.21.15.92:0/2803766272 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8174009b30 con 0x7f817c13e360 2023-11-24T06:12:18.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:18.282+0000 7f817a7fc640 1 --2- 172.21.15.92:0/2803766272 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f817c13e360 0x7f817c13d710 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f817000c820 tx=0x7f817000cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:18.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:18.282+0000 7f8163fff640 1 -- 172.21.15.92:0/2803766272 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8170007a30 con 0x7f817c13e360 2023-11-24T06:12:18.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:18.282+0000 7f8163fff640 1 -- 172.21.15.92:0/2803766272 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8170007b90 con 0x7f817c13e360 2023-11-24T06:12:18.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:18.282+0000 7f8163fff640 1 -- 172.21.15.92:0/2803766272 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f817000ce70 con 0x7f817c13e360 2023-11-24T06:12:18.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:18.282+0000 7f8181a5c640 1 -- 172.21.15.92:0/2803766272 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f817c1f3930 con 0x7f817c13e360 2023-11-24T06:12:18.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:18.282+0000 7f8181a5c640 1 -- 172.21.15.92:0/2803766272 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f817c1f66b0 con 0x7f817c13e360 2023-11-24T06:12:18.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:18.283+0000 7f8181a5c640 1 -- 172.21.15.92:0/2803766272 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f817c1f6bd0 con 0x7f817c13e360 2023-11-24T06:12:18.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:18.284+0000 7f8163fff640 1 -- 172.21.15.92:0/2803766272 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f81700113f0 con 0x7f817c13e360 2023-11-24T06:12:18.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:18.285+0000 7f8163fff640 1 --2- 172.21.15.92:0/2803766272 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f816806ae40 0x7f816806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:18.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:18.285+0000 7f8163fff640 1 -- 172.21.15.92:0/2803766272 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 5881+0+0 (secure 0 0 0) 0x7f8170089860 con 0x7f817c13e360 2023-11-24T06:12:18.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:18.285+0000 7f817affd640 1 --2- 172.21.15.92:0/2803766272 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f816806ae40 0x7f816806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:18.289 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:18.286+0000 7f817affd640 1 --2- 172.21.15.92:0/2803766272 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f816806ae40 0x7f816806d300 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f816c006fd0 tx=0x7f816c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:18.292 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:18.289+0000 7f8163fff640 1 -- 172.21.15.92:0/2803766272 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f817c1f6bd0 con 0x7f817c13e360 2023-11-24T06:12:18.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:18.481+0000 7f8181a5c640 1 -- 172.21.15.92:0/2803766272 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f817c1f6bd0 con 0x7f817c13e360 2023-11-24T06:12:19.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:19.033+0000 7f8163fff640 1 -- 172.21.15.92:0/2803766272 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v42) v1 ==== 181+0+0 (secure 0 0 0) 0x7f817c1f6bd0 con 0x7f817c13e360 2023-11-24T06:12:19.037 INFO:teuthology.orchestra.run.smithi092.stderr:pool 'cephfs_metadata' removed 2023-11-24T06:12:19.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:19.036+0000 7f8181a5c640 1 -- 172.21.15.92:0/2803766272 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f816806ae40 msgr2=0x7f816806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:19.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:19.036+0000 7f8181a5c640 1 --2- 172.21.15.92:0/2803766272 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f816806ae40 0x7f816806d300 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f816c006fd0 tx=0x7f816c008040 comp rx=0 tx=0).stop 2023-11-24T06:12:19.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:19.036+0000 7f8181a5c640 1 -- 172.21.15.92:0/2803766272 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f817c13e360 msgr2=0x7f817c13d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:19.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:19.036+0000 7f8181a5c640 1 --2- 172.21.15.92:0/2803766272 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f817c13e360 0x7f817c13d710 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f817000c820 tx=0x7f817000cc20 comp rx=0 tx=0).stop 2023-11-24T06:12:19.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:19.037+0000 7f8181a5c640 1 -- 172.21.15.92:0/2803766272 shutdown_connections 2023-11-24T06:12:19.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:19.037+0000 7f8181a5c640 1 --2- 172.21.15.92:0/2803766272 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f816806ae40 0x7f816806d300 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:19.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:19.037+0000 7f8181a5c640 1 --2- 172.21.15.92:0/2803766272 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f817c15f9c0 0x7f817c1f2f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:19.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:19.037+0000 7f8181a5c640 1 --2- 172.21.15.92:0/2803766272 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f817c13e360 0x7f817c13d710 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:19.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:19.037+0000 7f8181a5c640 1 --2- 172.21.15.92:0/2803766272 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f817c13df80 0x7f817c1f6eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:19.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:19.037+0000 7f8181a5c640 1 -- 172.21.15.92:0/2803766272 >> 172.21.15.92:0/2803766272 conn(0x7f817c06a800 msgr2=0x7f817c15bc30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:19.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:19.037+0000 7f8181a5c640 1 -- 172.21.15.92:0/2803766272 shutdown_connections 2023-11-24T06:12:19.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:19.037+0000 7f8181a5c640 1 -- 172.21.15.92:0/2803766272 wait complete. 2023-11-24T06:12:19.062 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-11-24T06:12:19.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:19.276+0000 7f8f70dac640 1 Processor -- start 2023-11-24T06:12:19.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:19.276+0000 7f8f70dac640 1 -- start start 2023-11-24T06:12:19.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:19.278+0000 7f8f70dac640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8f6c13fcb0 0x7f8f6c164d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:19.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:19.278+0000 7f8f70dac640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f6c0cdc40 con 0x7f8f6c15afb0 2023-11-24T06:12:19.282 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:19.278+0000 7f8f70dac640 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f6c0cdec0 con 0x7f8f6c13cfa0 2023-11-24T06:12:19.282 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:19.278+0000 7f8f70dac640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f6c0ce1a0 con 0x7f8f6c13fcb0 2023-11-24T06:12:19.282 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:19.278+0000 7f8f6affd640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f8f6c15afb0 0x7f8f6c13f5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:39548/0 (socket says 172.21.15.92:39548) 2023-11-24T06:12:19.282 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:19.278+0000 7f8f6affd640 1 -- 172.21.15.92:0/1228655474 learned_addr learned my addr 172.21.15.92:0/1228655474 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-24T06:12:19.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:19.279+0000 7f8f6a7fc640 1 -- 172.21.15.92:0/1228655474 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2671222952 0 0) 0x7f8f6c0cdc40 con 0x7f8f6c15afb0 2023-11-24T06:12:19.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:19.279+0000 7f8f6bfff640 1 --2- 172.21.15.92:0/1228655474 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8f6c13fcb0 0x7f8f6c164d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:19.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:19.279+0000 7f8f6a7fc640 1 -- 172.21.15.92:0/1228655474 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8f54003520 con 0x7f8f6c15afb0 2023-11-24T06:12:19.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:19.279+0000 7f8f6a7fc640 1 -- 172.21.15.92:0/1228655474 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3760012964 0 0) 0x7f8f54003520 con 0x7f8f6c15afb0 2023-11-24T06:12:19.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:19.279+0000 7f8f6a7fc640 1 -- 172.21.15.92:0/1228655474 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8f6c13fcb0 msgr2=0x7f8f6c164d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:19.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:19.279+0000 7f8f6a7fc640 1 --2- 172.21.15.92:0/1228655474 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8f6c13fcb0 0x7f8f6c164d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:19.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:19.279+0000 7f8f6a7fc640 1 -- 172.21.15.92:0/1228655474 >> v1:172.21.15.118:6789/0 conn(0x7f8f6c13cfa0 legacy=0x7f8f6c0cd530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:19.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:19.279+0000 7f8f6a7fc640 1 -- 172.21.15.92:0/1228655474 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f6c0ce480 con 0x7f8f6c15afb0 2023-11-24T06:12:19.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:19.279+0000 7f8f6a7fc640 1 -- 172.21.15.92:0/1228655474 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2607661350 0 0) 0x7f8f60001670 con 0x7f8f6c15afb0 2023-11-24T06:12:19.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:19.279+0000 7f8f6a7fc640 1 -- 172.21.15.92:0/1228655474 >> v1:172.21.15.92:6789/0 conn(0x7f8f6c15afb0 legacy=0x7f8f6c13f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:19.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:19.279+0000 7f8f6a7fc640 1 --2- 172.21.15.92:0/1228655474 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8f540039a0 0x7f8f54003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:19.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:19.279+0000 7f8f6a7fc640 1 --2- 172.21.15.92:0/1228655474 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8f6c13fcb0 0x7f8f540047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:19.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:19.279+0000 7f8f6a7fc640 1 --2- 172.21.15.92:0/1228655474 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8f54004d00 0x7f8f54005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:19.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:19.279+0000 7f8f6a7fc640 1 -- 172.21.15.92:0/1228655474 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f6c0ce1a0 con 0x7f8f54004d00 2023-11-24T06:12:19.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:19.279+0000 7f8f6a7fc640 1 -- 172.21.15.92:0/1228655474 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f5400bb40 con 0x7f8f540039a0 2023-11-24T06:12:19.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:19.280+0000 7f8f6a7fc640 1 -- 172.21.15.92:0/1228655474 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f5400df10 con 0x7f8f6c13fcb0 2023-11-24T06:12:19.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:19.280+0000 7f8f6bfff640 1 --2- 172.21.15.92:0/1228655474 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8f54004d00 0x7f8f54005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:19.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:19.280+0000 7f8f6b7fe640 1 --2- 172.21.15.92:0/1228655474 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8f540039a0 0x7f8f54003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:19.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:19.280+0000 7f8f6affd640 1 --2- 172.21.15.92:0/1228655474 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8f6c13fcb0 0x7f8f540047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:19.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:19.280+0000 7f8f6bfff640 1 -- 172.21.15.92:0/1228655474 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8f6c13fcb0 msgr2=0x7f8f540047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:19.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:19.280+0000 7f8f6bfff640 1 --2- 172.21.15.92:0/1228655474 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8f6c13fcb0 0x7f8f540047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:19.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:19.280+0000 7f8f6bfff640 1 -- 172.21.15.92:0/1228655474 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8f540039a0 msgr2=0x7f8f54003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:19.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:19.280+0000 7f8f6bfff640 1 --2- 172.21.15.92:0/1228655474 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8f540039a0 0x7f8f54003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:19.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:19.280+0000 7f8f6bfff640 1 -- 172.21.15.92:0/1228655474 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f54003520 con 0x7f8f54004d00 2023-11-24T06:12:19.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:19.281+0000 7f8f6bfff640 1 --2- 172.21.15.92:0/1228655474 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8f54004d00 0x7f8f54005180 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f8f64009fc0 tx=0x7f8f540057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b76f7cc8931774ce server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:19.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:19.281+0000 7f8f6a7fc640 1 -- 172.21.15.92:0/1228655474 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f6402cc20 con 0x7f8f54004d00 2023-11-24T06:12:19.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:19.282+0000 7f8f6a7fc640 1 -- 172.21.15.92:0/1228655474 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f6400f040 con 0x7f8f54004d00 2023-11-24T06:12:19.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:19.282+0000 7f8f6a7fc640 1 -- 172.21.15.92:0/1228655474 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f64034440 con 0x7f8f54004d00 2023-11-24T06:12:19.290 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:19.282+0000 7f8f70dac640 1 -- 172.21.15.92:0/1228655474 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8f54004d00 msgr2=0x7f8f54005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:19.291 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:19.282+0000 7f8f70dac640 1 --2- 172.21.15.92:0/1228655474 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8f54004d00 0x7f8f54005180 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f8f64009fc0 tx=0x7f8f540057f0 comp rx=0 tx=0).stop 2023-11-24T06:12:19.291 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:19.282+0000 7f8f6bfff640 1 -- 172.21.15.92:0/1228655474 reap_dead start 2023-11-24T06:12:19.291 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:19.282+0000 7f8f70dac640 1 -- 172.21.15.92:0/1228655474 shutdown_connections 2023-11-24T06:12:19.291 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:19.282+0000 7f8f70dac640 1 -- 172.21.15.92:0/1228655474 >> 172.21.15.92:0/1228655474 conn(0x7f8f6c06a800 msgr2=0x7f8f6c06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:19.291 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:19.282+0000 7f8f70dac640 1 -- 172.21.15.92:0/1228655474 shutdown_connections 2023-11-24T06:12:19.291 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:19.282+0000 7f8f70dac640 1 -- 172.21.15.92:0/1228655474 wait complete. 2023-11-24T06:12:19.291 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:19.284+0000 7f8f70dac640 1 Processor -- start 2023-11-24T06:12:19.291 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:19.284+0000 7f8f70dac640 1 -- start start 2023-11-24T06:12:19.291 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:19.285+0000 7f8f70dac640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8f54004d00 0x7f8f6c13c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:19.291 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:19.285+0000 7f8f70dac640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8f6c1e44c0 0x7f8f6c15ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:19.292 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:19.285+0000 7f8f70dac640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8f6c1f33a0 0x7f8f6c1f3820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:19.292 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:19.285+0000 7f8f70dac640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f6c15e2b0 con 0x7f8f6c1f33a0 2023-11-24T06:12:19.292 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:19.285+0000 7f8f70dac640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f6c15d6a0 con 0x7f8f54004d00 2023-11-24T06:12:19.292 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:19.285+0000 7f8f70dac640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f6c15d980 con 0x7f8f6c1e44c0 2023-11-24T06:12:19.292 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:19.286+0000 7f8f6bfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8f6c1f33a0 0x7f8f6c1f3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:19.292 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:19.286+0000 7f8f6bfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8f6c1f33a0 0x7f8f6c1f3820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43250/0 (socket says 172.21.15.92:43250) 2023-11-24T06:12:19.292 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:19.286+0000 7f8f6bfff640 1 -- 172.21.15.92:0/1117624926 learned_addr learned my addr 172.21.15.92:0/1117624926 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:12:19.292 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:19.286+0000 7f8f6b7fe640 1 --2- 172.21.15.92:0/1117624926 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8f54004d00 0x7f8f6c13c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:19.292 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:19.286+0000 7f8f6affd640 1 --2- 172.21.15.92:0/1117624926 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8f6c1e44c0 0x7f8f6c15ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:19.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:19.286+0000 7f8f6bfff640 1 -- 172.21.15.92:0/1117624926 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8f6c1e44c0 msgr2=0x7f8f6c15ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:19.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:19.286+0000 7f8f6bfff640 1 --2- 172.21.15.92:0/1117624926 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8f6c1e44c0 0x7f8f6c15ce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:19.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:19.286+0000 7f8f6bfff640 1 -- 172.21.15.92:0/1117624926 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8f54004d00 msgr2=0x7f8f6c13c5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:19.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:19.286+0000 7f8f6bfff640 1 --2- 172.21.15.92:0/1117624926 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8f54004d00 0x7f8f6c13c5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:19.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:19.286+0000 7f8f6bfff640 1 -- 172.21.15.92:0/1117624926 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f6c15dc60 con 0x7f8f6c1f33a0 2023-11-24T06:12:19.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:19.286+0000 7f8f6b7fe640 1 --2- 172.21.15.92:0/1117624926 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8f54004d00 0x7f8f6c13c5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:12:19.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:19.286+0000 7f8f6bfff640 1 --2- 172.21.15.92:0/1117624926 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8f6c1f33a0 0x7f8f6c1f3820 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f8f64037040 tx=0x7f8f64009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:19.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:19.287+0000 7f8f53fff640 1 -- 172.21.15.92:0/1117624926 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f64034d50 con 0x7f8f6c1f33a0 2023-11-24T06:12:19.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:19.287+0000 7f8f53fff640 1 -- 172.21.15.92:0/1117624926 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f64003990 con 0x7f8f6c1f33a0 2023-11-24T06:12:19.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:19.287+0000 7f8f53fff640 1 -- 172.21.15.92:0/1117624926 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f64003d40 con 0x7f8f6c1f33a0 2023-11-24T06:12:19.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:19.287+0000 7f8f70dac640 1 -- 172.21.15.92:0/1117624926 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f6c15df40 con 0x7f8f6c1f33a0 2023-11-24T06:12:19.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:19.287+0000 7f8f70dac640 1 -- 172.21.15.92:0/1117624926 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f6c13cae0 con 0x7f8f6c1f33a0 2023-11-24T06:12:19.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:19.288+0000 7f8f70dac640 1 -- 172.21.15.92:0/1117624926 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f6c067450 con 0x7f8f6c1f33a0 2023-11-24T06:12:19.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:19.289+0000 7f8f53fff640 1 -- 172.21.15.92:0/1117624926 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f8f64004030 con 0x7f8f6c1f33a0 2023-11-24T06:12:19.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:19.290+0000 7f8f53fff640 1 --2- 172.21.15.92:0/1117624926 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f8f5806ae70 0x7f8f5806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:19.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:19.290+0000 7f8f53fff640 1 -- 172.21.15.92:0/1117624926 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 5443+0+0 (secure 0 0 0) 0x7f8f640aac10 con 0x7f8f6c1f33a0 2023-11-24T06:12:19.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:19.290+0000 7f8f6b7fe640 1 --2- 172.21.15.92:0/1117624926 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f8f5806ae70 0x7f8f5806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:19.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:19.291+0000 7f8f6b7fe640 1 --2- 172.21.15.92:0/1117624926 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f8f5806ae70 0x7f8f5806d330 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f8f5c001670 tx=0x7f8f5c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:19.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:19.294+0000 7f8f53fff640 1 -- 172.21.15.92:0/1117624926 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8f64075dd0 con 0x7f8f6c1f33a0 2023-11-24T06:12:19.493 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:19.490+0000 7f8f70dac640 1 -- 172.21.15.92:0/1117624926 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f8f6c0cbff0 con 0x7f8f6c1f33a0 2023-11-24T06:12:20.047 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:20.044+0000 7f8f53fff640 1 -- 172.21.15.92:0/1117624926 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v43) v1 ==== 169+0+0 (secure 0 0 0) 0x7f8f640390d0 con 0x7f8f6c1f33a0 2023-11-24T06:12:20.048 INFO:teuthology.orchestra.run.smithi092.stderr:pool 'cephfs_data' removed 2023-11-24T06:12:20.052 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:20.049+0000 7f8f70dac640 1 -- 172.21.15.92:0/1117624926 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f8f5806ae70 msgr2=0x7f8f5806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:20.052 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:20.049+0000 7f8f70dac640 1 --2- 172.21.15.92:0/1117624926 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f8f5806ae70 0x7f8f5806d330 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f8f5c001670 tx=0x7f8f5c009040 comp rx=0 tx=0).stop 2023-11-24T06:12:20.052 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:20.049+0000 7f8f70dac640 1 -- 172.21.15.92:0/1117624926 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8f6c1f33a0 msgr2=0x7f8f6c1f3820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:20.052 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:20.049+0000 7f8f70dac640 1 --2- 172.21.15.92:0/1117624926 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8f6c1f33a0 0x7f8f6c1f3820 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f8f64037040 tx=0x7f8f64009750 comp rx=0 tx=0).stop 2023-11-24T06:12:20.053 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:20.049+0000 7f8f70dac640 1 -- 172.21.15.92:0/1117624926 shutdown_connections 2023-11-24T06:12:20.053 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:20.049+0000 7f8f70dac640 1 --2- 172.21.15.92:0/1117624926 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f8f5806ae70 0x7f8f5806d330 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:20.053 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:20.049+0000 7f8f70dac640 1 --2- 172.21.15.92:0/1117624926 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8f6c1f33a0 0x7f8f6c1f3820 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:20.054 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:20.050+0000 7f8f70dac640 1 --2- 172.21.15.92:0/1117624926 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8f6c1e44c0 0x7f8f6c15ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:20.054 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:20.050+0000 7f8f70dac640 1 --2- 172.21.15.92:0/1117624926 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8f54004d00 0x7f8f6c13c5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:20.054 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:20.050+0000 7f8f70dac640 1 -- 172.21.15.92:0/1117624926 >> 172.21.15.92:0/1117624926 conn(0x7f8f6c06a800 msgr2=0x7f8f6c0aeb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:20.054 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:20.050+0000 7f8f70dac640 1 -- 172.21.15.92:0/1117624926 shutdown_connections 2023-11-24T06:12:20.054 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:20.050+0000 7f8f70dac640 1 -- 172.21.15.92:0/1117624926 wait complete. 2023-11-24T06:12:20.077 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-11-24T06:12:20.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:20.280+0000 7fe07617a640 1 Processor -- start 2023-11-24T06:12:20.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:20.281+0000 7fe07617a640 1 -- start start 2023-11-24T06:12:20.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:20.282+0000 7fe07617a640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fe0700cffa0 0x7fe07015d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:20.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:20.282+0000 7fe07617a640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0700b2630 con 0x7fe0700d1ea0 2023-11-24T06:12:20.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:20.282+0000 7fe07617a640 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0700ceca0 con 0x7fe07015d2c0 2023-11-24T06:12:20.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:20.282+0000 7fe07617a640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0700cef80 con 0x7fe0700cffa0 2023-11-24T06:12:20.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:20.282+0000 7fe074977640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fe0700cffa0 0x7fe07015d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:20.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:20.282+0000 7fe075979640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fe0700d1ea0 0x7fe0700b3080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:39552/0 (socket says 172.21.15.92:39552) 2023-11-24T06:12:20.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:20.282+0000 7fe075979640 1 -- 172.21.15.92:0/3486427122 learned_addr learned my addr 172.21.15.92:0/3486427122 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-24T06:12:20.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:20.283+0000 7fe074977640 1 -- 172.21.15.92:0/3486427122 >> v1:172.21.15.118:6789/0 conn(0x7fe07015d2c0 legacy=0x7fe0700cf890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:20.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:20.283+0000 7fe074977640 1 -- 172.21.15.92:0/3486427122 >> v1:172.21.15.92:6789/0 conn(0x7fe0700d1ea0 legacy=0x7fe0700b3080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:20.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:20.283+0000 7fe06ffff640 1 -- 172.21.15.92:0/3486427122 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3504614513 0 0) 0x7fe0700b2630 con 0x7fe0700d1ea0 2023-11-24T06:12:20.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:20.283+0000 7fe074977640 1 -- 172.21.15.92:0/3486427122 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0700b2880 con 0x7fe0700cffa0 2023-11-24T06:12:20.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:20.284+0000 7fe074977640 1 --2- 172.21.15.92:0/3486427122 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe0700cffa0 0x7fe07015d6a0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fe05c0097d0 tx=0x7fe05c02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77f416ed144baacb server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:20.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:20.284+0000 7fe06ffff640 1 -- 172.21.15.92:0/3486427122 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe05c02e070 con 0x7fe0700cffa0 2023-11-24T06:12:20.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:20.285+0000 7fe06ffff640 1 -- 172.21.15.92:0/3486427122 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe05c0315a0 con 0x7fe0700cffa0 2023-11-24T06:12:20.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:20.285+0000 7fe06ffff640 1 -- 172.21.15.92:0/3486427122 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe05c031930 con 0x7fe0700cffa0 2023-11-24T06:12:20.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:20.285+0000 7fe07617a640 1 -- 172.21.15.92:0/3486427122 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe0700cffa0 msgr2=0x7fe07015d6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:20.289 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:20.285+0000 7fe07617a640 1 --2- 172.21.15.92:0/3486427122 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe0700cffa0 0x7fe07015d6a0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fe05c0097d0 tx=0x7fe05c02b840 comp rx=0 tx=0).stop 2023-11-24T06:12:20.289 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:20.285+0000 7fe07617a640 1 -- 172.21.15.92:0/3486427122 shutdown_connections 2023-11-24T06:12:20.289 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:20.285+0000 7fe07617a640 1 --2- 172.21.15.92:0/3486427122 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe0700cffa0 0x7fe07015d6a0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:20.289 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:20.285+0000 7fe07617a640 1 -- 172.21.15.92:0/3486427122 >> 172.21.15.92:0/3486427122 conn(0x7fe0700d30b0 msgr2=0x7fe0700d34c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:20.289 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:20.285+0000 7fe07617a640 1 -- 172.21.15.92:0/3486427122 shutdown_connections 2023-11-24T06:12:20.289 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:20.285+0000 7fe07617a640 1 -- 172.21.15.92:0/3486427122 wait complete. 2023-11-24T06:12:20.290 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:20.287+0000 7fe07617a640 1 Processor -- start 2023-11-24T06:12:20.290 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:20.287+0000 7fe07617a640 1 -- start start 2023-11-24T06:12:20.291 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:20.288+0000 7fe07617a640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe0700cffa0 0x7fe07009f910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:20.291 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:20.288+0000 7fe07617a640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe0700d1ea0 0x7fe0700b0120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:20.291 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:20.288+0000 7fe07617a640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe07015d2c0 0x7fe07014d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:20.291 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:20.288+0000 7fe07617a640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0700a0c50 con 0x7fe0700d1ea0 2023-11-24T06:12:20.291 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:20.288+0000 7fe07617a640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe07009ffd0 con 0x7fe07015d2c0 2023-11-24T06:12:20.292 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:20.288+0000 7fe07617a640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0700a02b0 con 0x7fe0700cffa0 2023-11-24T06:12:20.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:20.288+0000 7fe074977640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe0700d1ea0 0x7fe0700b0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:20.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:20.289+0000 7fe074977640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe0700d1ea0 0x7fe0700b0120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43274/0 (socket says 172.21.15.92:43274) 2023-11-24T06:12:20.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:20.289+0000 7fe074977640 1 -- 172.21.15.92:0/1061356399 learned_addr learned my addr 172.21.15.92:0/1061356399 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:12:20.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:20.289+0000 7fe075979640 1 --2- 172.21.15.92:0/1061356399 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe07015d2c0 0x7fe07014d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:20.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:20.289+0000 7fe075178640 1 --2- 172.21.15.92:0/1061356399 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe0700cffa0 0x7fe07009f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:20.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:20.289+0000 7fe074977640 1 -- 172.21.15.92:0/1061356399 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe0700cffa0 msgr2=0x7fe07009f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:20.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:20.289+0000 7fe074977640 1 --2- 172.21.15.92:0/1061356399 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe0700cffa0 0x7fe07009f910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:20.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:20.289+0000 7fe074977640 1 -- 172.21.15.92:0/1061356399 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe07015d2c0 msgr2=0x7fe07014d0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:20.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:20.289+0000 7fe074977640 1 --2- 172.21.15.92:0/1061356399 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe07015d2c0 0x7fe07014d0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:20.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:20.289+0000 7fe074977640 1 -- 172.21.15.92:0/1061356399 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe05c009400 con 0x7fe0700d1ea0 2023-11-24T06:12:20.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:20.289+0000 7fe075979640 1 --2- 172.21.15.92:0/1061356399 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe07015d2c0 0x7fe07014d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:12:20.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:20.289+0000 7fe074977640 1 --2- 172.21.15.92:0/1061356399 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe0700d1ea0 0x7fe0700b0120 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fe05c034040 tx=0x7fe05c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:20.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:20.290+0000 7fe06dffb640 1 -- 172.21.15.92:0/1061356399 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe05c032900 con 0x7fe0700d1ea0 2023-11-24T06:12:20.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:20.290+0000 7fe06dffb640 1 -- 172.21.15.92:0/1061356399 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe05c032a60 con 0x7fe0700d1ea0 2023-11-24T06:12:20.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:20.290+0000 7fe06dffb640 1 -- 172.21.15.92:0/1061356399 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe05c032e10 con 0x7fe0700d1ea0 2023-11-24T06:12:20.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:20.290+0000 7fe07617a640 1 -- 172.21.15.92:0/1061356399 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0700a0590 con 0x7fe0700d1ea0 2023-11-24T06:12:20.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:20.290+0000 7fe07617a640 1 -- 172.21.15.92:0/1061356399 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe07015aee0 con 0x7fe0700d1ea0 2023-11-24T06:12:20.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:20.291+0000 7fe07617a640 1 -- 172.21.15.92:0/1061356399 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe070067450 con 0x7fe0700d1ea0 2023-11-24T06:12:20.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:20.292+0000 7fe06dffb640 1 -- 172.21.15.92:0/1061356399 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe05c03b3f0 con 0x7fe0700d1ea0 2023-11-24T06:12:20.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:20.293+0000 7fe06dffb640 1 --2- 172.21.15.92:0/1061356399 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fe05406ae40 0x7fe05406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:20.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:20.293+0000 7fe06dffb640 1 -- 172.21.15.92:0/1061356399 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5029+0+0 (secure 0 0 0) 0x7fe05c0ab450 con 0x7fe0700d1ea0 2023-11-24T06:12:20.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:20.293+0000 7fe075178640 1 --2- 172.21.15.92:0/1061356399 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fe05406ae40 0x7fe05406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:20.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:20.294+0000 7fe075178640 1 --2- 172.21.15.92:0/1061356399 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fe05406ae40 0x7fe05406d300 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fe064007bf0 tx=0x7fe064007240 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:20.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:20.298+0000 7fe06dffb640 1 -- 172.21.15.92:0/1061356399 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe05c076790 con 0x7fe0700d1ea0 2023-11-24T06:12:20.492 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:20.489+0000 7fe07617a640 1 -- 172.21.15.92:0/1061356399 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fe0700b23c0 con 0x7fe0700d1ea0 2023-11-24T06:12:20.499 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:20.496+0000 7fe06dffb640 1 -- 172.21.15.92:0/1061356399 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v11) v1 ==== 104+0+0 (secure 0 0 0) 0x7fe05c079db0 con 0x7fe0700d1ea0 2023-11-24T06:12:20.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:20.499+0000 7fe07617a640 1 -- 172.21.15.92:0/1061356399 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fe05406ae40 msgr2=0x7fe05406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:20.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:20.499+0000 7fe07617a640 1 --2- 172.21.15.92:0/1061356399 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fe05406ae40 0x7fe05406d300 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fe064007bf0 tx=0x7fe064007240 comp rx=0 tx=0).stop 2023-11-24T06:12:20.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:20.499+0000 7fe07617a640 1 -- 172.21.15.92:0/1061356399 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe0700d1ea0 msgr2=0x7fe0700b0120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:20.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:20.499+0000 7fe07617a640 1 --2- 172.21.15.92:0/1061356399 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe0700d1ea0 0x7fe0700b0120 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fe05c034040 tx=0x7fe05c0046c0 comp rx=0 tx=0).stop 2023-11-24T06:12:20.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:20.499+0000 7fe07617a640 1 -- 172.21.15.92:0/1061356399 shutdown_connections 2023-11-24T06:12:20.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:20.499+0000 7fe07617a640 1 --2- 172.21.15.92:0/1061356399 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fe05406ae40 0x7fe05406d300 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:20.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:20.499+0000 7fe07617a640 1 --2- 172.21.15.92:0/1061356399 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe07015d2c0 0x7fe07014d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:20.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:20.499+0000 7fe07617a640 1 --2- 172.21.15.92:0/1061356399 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe0700d1ea0 0x7fe0700b0120 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:20.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:20.499+0000 7fe07617a640 1 --2- 172.21.15.92:0/1061356399 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe0700cffa0 0x7fe07009f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:20.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:20.499+0000 7fe07617a640 1 -- 172.21.15.92:0/1061356399 >> 172.21.15.92:0/1061356399 conn(0x7fe0700d30b0 msgr2=0x7fe0700b3080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:20.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:20.499+0000 7fe07617a640 1 -- 172.21.15.92:0/1061356399 shutdown_connections 2023-11-24T06:12:20.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:20.499+0000 7fe07617a640 1 -- 172.21.15.92:0/1061356399 wait complete. 2023-11-24T06:12:20.522 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestVolumes.test_volume_create' 2023-11-24T06:12:20.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:20.733+0000 7fd6b4dbe640 1 Processor -- start 2023-11-24T06:12:20.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:20.733+0000 7fd6b4dbe640 1 -- start start 2023-11-24T06:12:20.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:20.734+0000 7fd6b4dbe640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fd6b013df80 0x7fd6b015bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:20.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:20.734+0000 7fd6b4dbe640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fd6b00cf9a0 0x7fd6b00cd840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:20.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:20.734+0000 7fd6b4dbe640 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6b006c470 con 0x7fd6b013e360 2023-11-24T06:12:20.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:20.734+0000 7fd6b4dbe640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6b006c750 con 0x7fd6b013df80 2023-11-24T06:12:20.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:20.734+0000 7fd6b4dbe640 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6b006ca30 con 0x7fd6b00cf9a0 2023-11-24T06:12:20.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:20.734+0000 7fd6ae59c640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fd6b013df80 0x7fd6b015bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:20.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:20.734+0000 7fd6ae59c640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fd6b013df80 0x7fd6b015bee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43278/0 (socket says 172.21.15.92:43278) 2023-11-24T06:12:20.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:20.734+0000 7fd6ae59c640 1 -- 172.21.15.92:0/4221654763 learned_addr learned my addr 172.21.15.92:0/4221654763 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:12:20.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:20.735+0000 7fd6ae59c640 1 -- 172.21.15.92:0/4221654763 >> v2:172.21.15.118:3300/0 conn(0x7fd6b00cf9a0 msgr2=0x7fd6b00cd840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:20.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:20.735+0000 7fd6ae59c640 1 --2- 172.21.15.92:0/4221654763 >> v2:172.21.15.118:3300/0 conn(0x7fd6b00cf9a0 0x7fd6b00cd840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:20.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:20.735+0000 7fd6ae59c640 1 -- 172.21.15.92:0/4221654763 >> v1:172.21.15.118:6789/0 conn(0x7fd6b013e360 legacy=0x7fd6b00cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:20.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:20.735+0000 7fd6ae59c640 1 -- 172.21.15.92:0/4221654763 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6b006cd10 con 0x7fd6b013df80 2023-11-24T06:12:20.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:20.736+0000 7fd6ae59c640 1 --2- 172.21.15.92:0/4221654763 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd6b013df80 0x7fd6b015bee0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fd6b00ccb10 tx=0x7fd69802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c5eebac660a162d server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:20.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:20.736+0000 7fd6add9b640 1 -- 172.21.15.92:0/4221654763 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd698031870 con 0x7fd6b013df80 2023-11-24T06:12:20.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:20.736+0000 7fd6add9b640 1 -- 172.21.15.92:0/4221654763 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6980319d0 con 0x7fd6b013df80 2023-11-24T06:12:20.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:20.736+0000 7fd6add9b640 1 -- 172.21.15.92:0/4221654763 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd698031d60 con 0x7fd6b013df80 2023-11-24T06:12:20.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:20.737+0000 7fd6b4dbe640 1 -- 172.21.15.92:0/4221654763 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd6b013df80 msgr2=0x7fd6b015bee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:20.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:20.737+0000 7fd6b4dbe640 1 --2- 172.21.15.92:0/4221654763 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd6b013df80 0x7fd6b015bee0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fd6b00ccb10 tx=0x7fd69802b910 comp rx=0 tx=0).stop 2023-11-24T06:12:20.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:20.737+0000 7fd6b4dbe640 1 -- 172.21.15.92:0/4221654763 shutdown_connections 2023-11-24T06:12:20.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:20.737+0000 7fd6b4dbe640 1 --2- 172.21.15.92:0/4221654763 >> v2:172.21.15.118:3300/0 conn(0x7fd6b00cf9a0 0x7fd6b00cd840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:20.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:20.737+0000 7fd6b4dbe640 1 --2- 172.21.15.92:0/4221654763 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd6b013df80 0x7fd6b015bee0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:20.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:20.737+0000 7fd6b4dbe640 1 -- 172.21.15.92:0/4221654763 >> 172.21.15.92:0/4221654763 conn(0x7fd6b006a800 msgr2=0x7fd6b006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:20.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:20.737+0000 7fd6b4dbe640 1 -- 172.21.15.92:0/4221654763 shutdown_connections 2023-11-24T06:12:20.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:20.737+0000 7fd6b4dbe640 1 -- 172.21.15.92:0/4221654763 wait complete. 2023-11-24T06:12:20.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:20.739+0000 7fd6b4dbe640 1 Processor -- start 2023-11-24T06:12:20.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:20.739+0000 7fd6b4dbe640 1 -- start start 2023-11-24T06:12:20.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:20.740+0000 7fd6b4dbe640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd6b00cf9a0 0x7fd6b013d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:20.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:20.740+0000 7fd6b4dbe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd6b013df80 0x7fd6b00ade50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:20.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:20.740+0000 7fd6b4dbe640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd6b013e360 0x7fd6b015a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:20.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:20.740+0000 7fd6b4dbe640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6b00ae390 con 0x7fd6b013df80 2023-11-24T06:12:20.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:20.740+0000 7fd6b4dbe640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6b0070770 con 0x7fd6b00cf9a0 2023-11-24T06:12:20.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:20.740+0000 7fd6b4dbe640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6b00adc00 con 0x7fd6b013e360 2023-11-24T06:12:20.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:20.740+0000 7fd6a5d9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd6b013df80 0x7fd6b00ade50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:20.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:20.740+0000 7fd6a5d9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd6b013df80 0x7fd6b00ade50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43282/0 (socket says 172.21.15.92:43282) 2023-11-24T06:12:20.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:20.740+0000 7fd6a5d9b640 1 -- 172.21.15.92:0/699108518 learned_addr learned my addr 172.21.15.92:0/699108518 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:12:20.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:20.740+0000 7fd6ae59c640 1 --2- 172.21.15.92:0/699108518 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd6b00cf9a0 0x7fd6b013d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:20.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:20.741+0000 7fd6aed9d640 1 --2- 172.21.15.92:0/699108518 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd6b013e360 0x7fd6b015a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:20.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:20.741+0000 7fd6a5d9b640 1 -- 172.21.15.92:0/699108518 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd6b013e360 msgr2=0x7fd6b015a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:20.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:20.741+0000 7fd6a5d9b640 1 --2- 172.21.15.92:0/699108518 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd6b013e360 0x7fd6b015a180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:20.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:20.741+0000 7fd6a5d9b640 1 -- 172.21.15.92:0/699108518 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd6b00cf9a0 msgr2=0x7fd6b013d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:20.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:20.741+0000 7fd6a5d9b640 1 --2- 172.21.15.92:0/699108518 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd6b00cf9a0 0x7fd6b013d960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:20.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:20.741+0000 7fd6a5d9b640 1 -- 172.21.15.92:0/699108518 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6980094d0 con 0x7fd6b013df80 2023-11-24T06:12:20.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:20.741+0000 7fd6aed9d640 1 --2- 172.21.15.92:0/699108518 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd6b013e360 0x7fd6b015a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:12:20.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:20.741+0000 7fd6a5d9b640 1 --2- 172.21.15.92:0/699108518 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd6b013df80 0x7fd6b00ade50 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fd6a000c430 tx=0x7fd6a000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:20.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:20.741+0000 7fd6a77fe640 1 -- 172.21.15.92:0/699108518 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6a00035f0 con 0x7fd6b013df80 2023-11-24T06:12:20.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:20.741+0000 7fd6a77fe640 1 -- 172.21.15.92:0/699108518 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6a0003750 con 0x7fd6b013df80 2023-11-24T06:12:20.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:20.741+0000 7fd6a77fe640 1 -- 172.21.15.92:0/699108518 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6a0003b00 con 0x7fd6b013df80 2023-11-24T06:12:20.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:20.741+0000 7fd6b4dbe640 1 -- 172.21.15.92:0/699108518 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6b01f3b00 con 0x7fd6b013df80 2023-11-24T06:12:20.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:20.741+0000 7fd6b4dbe640 1 -- 172.21.15.92:0/699108518 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6b01f40c0 con 0x7fd6b013df80 2023-11-24T06:12:20.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:20.743+0000 7fd6b4dbe640 1 -- 172.21.15.92:0/699108518 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6b00675d0 con 0x7fd6b013df80 2023-11-24T06:12:20.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:20.743+0000 7fd6a77fe640 1 -- 172.21.15.92:0/699108518 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd6a0003c60 con 0x7fd6b013df80 2023-11-24T06:12:20.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:20.744+0000 7fd6a77fe640 1 --2- 172.21.15.92:0/699108518 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fd69006ae70 0x7fd69006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:20.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:20.744+0000 7fd6a77fe640 1 -- 172.21.15.92:0/699108518 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5029+0+0 (secure 0 0 0) 0x7fd6a00886a0 con 0x7fd6b013df80 2023-11-24T06:12:20.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:20.745+0000 7fd6ae59c640 1 --2- 172.21.15.92:0/699108518 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fd69006ae70 0x7fd69006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:20.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:20.745+0000 7fd6ae59c640 1 --2- 172.21.15.92:0/699108518 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fd69006ae70 0x7fd69006d330 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fd698036000 tx=0x7fd69803c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:20.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:20.749+0000 7fd6a77fe640 1 -- 172.21.15.92:0/699108518 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd6a0053990 con 0x7fd6b013df80 2023-11-24T06:12:21.010 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:21.007+0000 7fd6b4dbe640 1 -- 172.21.15.92:0/699108518 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestVolumes.test_volume_create"]} v 0) v1 -- 0x7fd6b00b0350 con 0x7fd6b013df80 2023-11-24T06:12:21.050 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:21.047+0000 7fd6a77fe640 1 -- 172.21.15.92:0/699108518 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestVolumes.test_volume_create"]}]=0 v128) v1 ==== 135+0+0 (secure 0 0 0) 0x7fd6a0056fb0 con 0x7fd6b013df80 2023-11-24T06:12:21.053 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:21.050+0000 7fd6b4dbe640 1 -- 172.21.15.92:0/699108518 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fd69006ae70 msgr2=0x7fd69006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:21.053 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:21.050+0000 7fd6b4dbe640 1 --2- 172.21.15.92:0/699108518 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fd69006ae70 0x7fd69006d330 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fd698036000 tx=0x7fd69803c040 comp rx=0 tx=0).stop 2023-11-24T06:12:21.053 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:21.050+0000 7fd6b4dbe640 1 -- 172.21.15.92:0/699108518 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd6b013df80 msgr2=0x7fd6b00ade50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:21.054 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:21.050+0000 7fd6b4dbe640 1 --2- 172.21.15.92:0/699108518 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd6b013df80 0x7fd6b00ade50 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fd6a000c430 tx=0x7fd6a000c8f0 comp rx=0 tx=0).stop 2023-11-24T06:12:21.054 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:21.050+0000 7fd6b4dbe640 1 -- 172.21.15.92:0/699108518 shutdown_connections 2023-11-24T06:12:21.054 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:21.050+0000 7fd6b4dbe640 1 --2- 172.21.15.92:0/699108518 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fd69006ae70 0x7fd69006d330 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:21.054 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:21.050+0000 7fd6b4dbe640 1 --2- 172.21.15.92:0/699108518 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd6b013e360 0x7fd6b015a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:21.054 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:21.050+0000 7fd6b4dbe640 1 --2- 172.21.15.92:0/699108518 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd6b013df80 0x7fd6b00ade50 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:21.054 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:21.050+0000 7fd6b4dbe640 1 --2- 172.21.15.92:0/699108518 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd6b00cf9a0 0x7fd6b013d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:21.054 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:21.050+0000 7fd6b4dbe640 1 -- 172.21.15.92:0/699108518 >> 172.21.15.92:0/699108518 conn(0x7fd6b006a800 msgr2=0x7fd6b015cf80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:21.054 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:21.050+0000 7fd6b4dbe640 1 -- 172.21.15.92:0/699108518 shutdown_connections 2023-11-24T06:12:21.054 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:21.050+0000 7fd6b4dbe640 1 -- 172.21.15.92:0/699108518 wait complete. 2023-11-24T06:12:21.071 INFO:tasks.cephfs_test_runner:That the volume can be created and then cleans up ... ok 2023-11-24T06:12:21.072 INFO:tasks.cephfs_test_runner:Starting test: test_volume_info (tasks.cephfs.test_volumes.TestVolumes) Tests the 'fs volume info' command 2023-11-24T06:12:21.072 INFO:tasks.cephfs_test_runner:test_volume_info (tasks.cephfs.test_volumes.TestVolumes) 2023-11-24T06:12:21.072 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:12:21.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:21.277+0000 7f24660ea640 1 Processor -- start 2023-11-24T06:12:21.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:21.278+0000 7f24660ea640 1 -- start start 2023-11-24T06:12:21.283 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:21.279+0000 7f24660ea640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f246013cfa0 0x7f246015afb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:21.283 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:21.279+0000 7f24660ea640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f24600cd530 0x7f246013f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:21.283 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:21.279+0000 7f24660ea640 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24600cdec0 con 0x7f24600cd9b0 2023-11-24T06:12:21.283 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:21.279+0000 7f24660ea640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24600ce1a0 con 0x7f24600cd530 2023-11-24T06:12:21.283 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:21.279+0000 7f24660ea640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24600ce480 con 0x7f246013cfa0 2023-11-24T06:12:21.283 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:21.280+0000 7f24648e7640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f24600cd530 0x7f246013f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:21.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:21.280+0000 7f24648e7640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f24600cd530 0x7f246013f5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43286/0 (socket says 172.21.15.92:43286) 2023-11-24T06:12:21.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:21.280+0000 7f24648e7640 1 -- 172.21.15.92:0/729328173 learned_addr learned my addr 172.21.15.92:0/729328173 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:12:21.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:21.280+0000 7f24648e7640 1 -- 172.21.15.92:0/729328173 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f246013cfa0 msgr2=0x7f246015afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:21.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:21.280+0000 7f24648e7640 1 --2- 172.21.15.92:0/729328173 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f246013cfa0 0x7f246015afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:21.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:21.280+0000 7f24648e7640 1 -- 172.21.15.92:0/729328173 >> v1:172.21.15.118:6789/0 conn(0x7f24600cd9b0 legacy=0x7f246015d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:21.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:21.280+0000 7f24648e7640 1 -- 172.21.15.92:0/729328173 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24600ce760 con 0x7f24600cd530 2023-11-24T06:12:21.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:21.281+0000 7f24648e7640 1 --2- 172.21.15.92:0/729328173 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f24600cd530 0x7f246013f5a0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f246006e2a0 tx=0x7f245002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0cf7a69a7a117bb server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:21.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:21.282+0000 7f2457fff640 1 -- 172.21.15.92:0/729328173 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f245002e070 con 0x7f24600cd530 2023-11-24T06:12:21.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:21.282+0000 7f2457fff640 1 -- 172.21.15.92:0/729328173 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24500315a0 con 0x7f24600cd530 2023-11-24T06:12:21.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:21.282+0000 7f2457fff640 1 -- 172.21.15.92:0/729328173 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2450031930 con 0x7f24600cd530 2023-11-24T06:12:21.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:21.282+0000 7f24660ea640 1 -- 172.21.15.92:0/729328173 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f24600cd530 msgr2=0x7f246013f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:21.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:21.282+0000 7f24660ea640 1 --2- 172.21.15.92:0/729328173 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f24600cd530 0x7f246013f5a0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f246006e2a0 tx=0x7f245002b840 comp rx=0 tx=0).stop 2023-11-24T06:12:21.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:21.282+0000 7f24660ea640 1 -- 172.21.15.92:0/729328173 shutdown_connections 2023-11-24T06:12:21.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:21.282+0000 7f24660ea640 1 --2- 172.21.15.92:0/729328173 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f24600cd530 0x7f246013f5a0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:21.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:21.282+0000 7f24660ea640 1 --2- 172.21.15.92:0/729328173 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f246013cfa0 0x7f246015afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:21.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:21.282+0000 7f24660ea640 1 -- 172.21.15.92:0/729328173 >> 172.21.15.92:0/729328173 conn(0x7f246006a800 msgr2=0x7f246006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:21.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:21.282+0000 7f24660ea640 1 -- 172.21.15.92:0/729328173 shutdown_connections 2023-11-24T06:12:21.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:21.282+0000 7f24660ea640 1 -- 172.21.15.92:0/729328173 wait complete. 2023-11-24T06:12:21.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:21.284+0000 7f24660ea640 1 Processor -- start 2023-11-24T06:12:21.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:21.284+0000 7f24660ea640 1 -- start start 2023-11-24T06:12:21.289 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:21.285+0000 7f24660ea640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f24600cd530 0x7f246014f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:21.289 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:21.285+0000 7f24660ea640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f24600cd9b0 0x7f2460137370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:21.289 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:21.285+0000 7f24660ea640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f246013cfa0 0x7f246014ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:21.289 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:21.285+0000 7f24660ea640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f246015d470 con 0x7f24600cd9b0 2023-11-24T06:12:21.290 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:21.285+0000 7f24660ea640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f246014e160 con 0x7f24600cd530 2023-11-24T06:12:21.290 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:21.285+0000 7f24660ea640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f246014e400 con 0x7f246013cfa0 2023-11-24T06:12:21.290 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:21.286+0000 7f24648e7640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f24600cd9b0 0x7f2460137370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:21.290 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:21.286+0000 7f24648e7640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f24600cd9b0 0x7f2460137370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43302/0 (socket says 172.21.15.92:43302) 2023-11-24T06:12:21.290 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:21.286+0000 7f24648e7640 1 -- 172.21.15.92:0/1742683206 learned_addr learned my addr 172.21.15.92:0/1742683206 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:12:21.290 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:21.286+0000 7f24648e7640 1 -- 172.21.15.92:0/1742683206 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f246013cfa0 msgr2=0x7f246014ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:21.290 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:21.286+0000 7f24658e9640 1 --2- 172.21.15.92:0/1742683206 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f246013cfa0 0x7f246014ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:21.290 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:21.286+0000 7f24650e8640 1 --2- 172.21.15.92:0/1742683206 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f24600cd530 0x7f246014f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:21.290 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:21.286+0000 7f24648e7640 1 --2- 172.21.15.92:0/1742683206 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f246013cfa0 0x7f246014ce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:21.290 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:21.286+0000 7f24648e7640 1 -- 172.21.15.92:0/1742683206 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f24600cd530 msgr2=0x7f246014f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:21.291 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:21.286+0000 7f24648e7640 1 --2- 172.21.15.92:0/1742683206 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f24600cd530 0x7f246014f1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:21.291 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:21.286+0000 7f24648e7640 1 -- 172.21.15.92:0/1742683206 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24500094d0 con 0x7f24600cd9b0 2023-11-24T06:12:21.291 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:21.286+0000 7f24648e7640 1 --2- 172.21.15.92:0/1742683206 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f24600cd9b0 0x7f2460137370 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f2450000c00 tx=0x7f24500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:21.291 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:21.287+0000 7f2455ffb640 1 -- 172.21.15.92:0/1742683206 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2450032840 con 0x7f24600cd9b0 2023-11-24T06:12:21.291 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:21.287+0000 7f2455ffb640 1 -- 172.21.15.92:0/1742683206 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24500329a0 con 0x7f24600cd9b0 2023-11-24T06:12:21.292 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:21.287+0000 7f24660ea640 1 -- 172.21.15.92:0/1742683206 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f246014e6e0 con 0x7f24600cd9b0 2023-11-24T06:12:21.292 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:21.287+0000 7f2455ffb640 1 -- 172.21.15.92:0/1742683206 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2450032d50 con 0x7f24600cd9b0 2023-11-24T06:12:21.292 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:21.287+0000 7f24660ea640 1 -- 172.21.15.92:0/1742683206 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f246014eca0 con 0x7f24600cd9b0 2023-11-24T06:12:21.292 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:21.288+0000 7f24660ea640 1 -- 172.21.15.92:0/1742683206 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f246014ee90 con 0x7f24600cd9b0 2023-11-24T06:12:21.292 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:21.289+0000 7f2455ffb640 1 -- 172.21.15.92:0/1742683206 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f245003b3f0 con 0x7f24600cd9b0 2023-11-24T06:12:21.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:21.290+0000 7f2455ffb640 1 --2- 172.21.15.92:0/1742683206 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f244406ae40 0x7f244406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:21.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:21.291+0000 7f2455ffb640 1 -- 172.21.15.92:0/1742683206 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f24500ab4e0 con 0x7f24600cd9b0 2023-11-24T06:12:21.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:21.291+0000 7f24650e8640 1 --2- 172.21.15.92:0/1742683206 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f244406ae40 0x7f244406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:21.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:21.291+0000 7f24650e8640 1 --2- 172.21.15.92:0/1742683206 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f244406ae40 0x7f244406d300 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f2460070940 tx=0x7f245c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:21.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:21.295+0000 7f2455ffb640 1 -- 172.21.15.92:0/1742683206 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2450076790 con 0x7f24600cd9b0 2023-11-24T06:12:21.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:21.488+0000 7f24660ea640 1 -- 172.21.15.92:0/1742683206 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2460067450 con 0x7f24600cd9b0 2023-11-24T06:12:21.493 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:21.490+0000 7f2455ffb640 1 -- 172.21.15.92:0/1742683206 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v43) v1 ==== 74+0+13068 (secure 0 0 0) 0x7f2450079db0 con 0x7f24600cd9b0 2023-11-24T06:12:21.493 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-24T06:12:21.493 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":43,"fsid":"2d6dee5f-8aa3-470f-931a-50ade963b1cb","created":"2023-11-24T06:10:04.707204+0000","modified":"2023-11-24T06:12:20.035051+0000","last_up_change":"2023-11-24T06:10:17.598352+0000","last_in_change":"2023-11-24T06:10:09.040793+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:10:17.733098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ff37a9fb-135f-471f-b4b0-f7b5772006c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6803","nonce":20560714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6805","nonce":20560714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6809","nonce":20560714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6807","nonce":20560714}]},"public_addr":"172.21.15.92:6803/20560714","cluster_addr":"172.21.15.92:6805/20560714","heartbeat_back_addr":"172.21.15.92:6809/20560714","heartbeat_front_addr":"172.21.15.92:6807/20560714","state":["exists","up"]},{"osd":1,"uuid":"03cdd353-dcf1-4f38-94fe-a795137c06b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6819","nonce":2451239906}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6821","nonce":2451239906}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6825","nonce":2451239906}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6823","nonce":2451239906}]},"public_addr":"172.21.15.92:6819/2451239906","cluster_addr":"172.21.15.92:6821/2451239906","heartbeat_back_addr":"172.21.15.92:6825/2451239906","heartbeat_front_addr":"172.21.15.92:6823/2451239906","state":["exists","up"]},{"osd":2,"uuid":"a4a20d80-5d57-484b-becd-9aa8070dd3aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6811","nonce":1124979277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6813","nonce":1124979277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6817","nonce":1124979277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6815","nonce":1124979277}]},"public_addr":"172.21.15.92:6811/1124979277","cluster_addr":"172.21.15.92:6813/1124979277","heartbeat_back_addr":"172.21.15.92:6817/1124979277","heartbeat_front_addr":"172.21.15.92:6815/1124979277","state":["exists","up"]},{"osd":3,"uuid":"4648d7ce-9b29-4583-9425-4ba8fd2b1903","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6827","nonce":1734115782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6829","nonce":1734115782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6833","nonce":1734115782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6831","nonce":1734115782}]},"public_addr":"172.21.15.92:6827/1734115782","cluster_addr":"172.21.15.92:6829/1734115782","heartbeat_back_addr":"172.21.15.92:6833/1734115782","heartbeat_front_addr":"172.21.15.92:6831/1734115782","state":["exists","up"]},{"osd":4,"uuid":"3bc9d7eb-c6fa-4fcf-a3ec-ade604e05aac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6819","nonce":3425937229}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6822","nonce":3425937229}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6830","nonce":3425937229}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6826","nonce":3425937229}]},"public_addr":"172.21.15.118:6819/3425937229","cluster_addr":"172.21.15.118:6822/3425937229","heartbeat_back_addr":"172.21.15.118:6830/3425937229","heartbeat_front_addr":"172.21.15.118:6826/3425937229","state":["exists","up"]},{"osd":5,"uuid":"dce9ba6d-59ed-46d1-84fc-a84b13de10c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6801","nonce":827455984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6803","nonce":827455984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6807","nonce":827455984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6805","nonce":827455984}]},"public_addr":"172.21.15.118:6801/827455984","cluster_addr":"172.21.15.118:6803/827455984","heartbeat_back_addr":"172.21.15.118:6807/827455984","heartbeat_front_addr":"172.21.15.118:6805/827455984","state":["exists","up"]},{"osd":6,"uuid":"82aec504-ae58-4709-8d30-46f2b12081b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6809","nonce":2946986141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6811","nonce":2946986141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6815","nonce":2946986141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6813","nonce":2946986141}]},"public_addr":"172.21.15.118:6809/2946986141","cluster_addr":"172.21.15.118:6811/2946986141","heartbeat_back_addr":"172.21.15.118:6815/2946986141","heartbeat_front_addr":"172.21.15.118:6813/2946986141","state":["exists","up"]},{"osd":7,"uuid":"4ed4c624-c88f-4d35-bdb1-bf209b5dffb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6817","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6818","nonce":48122059}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6821","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6823","nonce":48122059}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6829","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6831","nonce":48122059}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6825","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6827","nonce":48122059}]},"public_addr":"172.21.15.118:6818/48122059","cluster_addr":"172.21.15.118:6823/48122059","heartbeat_back_addr":"172.21.15.118:6831/48122059","heartbeat_front_addr":"172.21.15.118:6827/48122059","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.536664+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.368324+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.271201+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.342971+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.621832+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.387110+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.632946+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.416032+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.92:6835/1413573590":"2023-11-25T06:12:17.023972+0000","172.21.15.92:6837/1413573590":"2023-11-25T06:12:17.023972+0000","172.21.15.92:6836/3889219158":"2023-11-25T06:12:03.443855+0000","172.21.15.92:6834/3889219158":"2023-11-25T06:12:03.443855+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:12:21.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:21.493+0000 7f24660ea640 1 -- 172.21.15.92:0/1742683206 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f244406ae40 msgr2=0x7f244406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:21.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:21.493+0000 7f24660ea640 1 --2- 172.21.15.92:0/1742683206 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f244406ae40 0x7f244406d300 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f2460070940 tx=0x7f245c008040 comp rx=0 tx=0).stop 2023-11-24T06:12:21.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:21.493+0000 7f24660ea640 1 -- 172.21.15.92:0/1742683206 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f24600cd9b0 msgr2=0x7f2460137370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:21.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:21.493+0000 7f24660ea640 1 --2- 172.21.15.92:0/1742683206 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f24600cd9b0 0x7f2460137370 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f2450000c00 tx=0x7f24500046c0 comp rx=0 tx=0).stop 2023-11-24T06:12:21.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:21.493+0000 7f24660ea640 1 -- 172.21.15.92:0/1742683206 shutdown_connections 2023-11-24T06:12:21.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:21.493+0000 7f24660ea640 1 --2- 172.21.15.92:0/1742683206 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f244406ae40 0x7f244406d300 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:21.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:21.493+0000 7f24660ea640 1 --2- 172.21.15.92:0/1742683206 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f246013cfa0 0x7f246014ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:21.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:21.493+0000 7f24660ea640 1 --2- 172.21.15.92:0/1742683206 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f24600cd9b0 0x7f2460137370 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:21.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:21.493+0000 7f24660ea640 1 --2- 172.21.15.92:0/1742683206 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f24600cd530 0x7f246014f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:21.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:21.493+0000 7f24660ea640 1 -- 172.21.15.92:0/1742683206 >> 172.21.15.92:0/1742683206 conn(0x7f246006a800 msgr2=0x7f24600b0230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:21.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:21.493+0000 7f24660ea640 1 -- 172.21.15.92:0/1742683206 shutdown_connections 2023-11-24T06:12:21.498 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:21.493+0000 7f24660ea640 1 -- 172.21.15.92:0/1742683206 wait complete. 2023-11-24T06:12:21.515 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T06:10:17.733098+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T06:12:21.516 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T06:12:21.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:21.725+0000 7f2aad9d5640 1 Processor -- start 2023-11-24T06:12:21.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:21.725+0000 7f2aad9d5640 1 -- start start 2023-11-24T06:12:21.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:21.727+0000 7f2aad9d5640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2aa813df80 0x7f2aa815bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:21.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:21.727+0000 7f2aad9d5640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f2aa813e360 0x7f2aa80cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:21.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:21.727+0000 7f2aad9d5640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2aa806c470 con 0x7f2aa815f9c0 2023-11-24T06:12:21.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:21.727+0000 7f2aad9d5640 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2aa806c750 con 0x7f2aa813e360 2023-11-24T06:12:21.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:21.727+0000 7f2aad9d5640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2aa806ca30 con 0x7f2aa813df80 2023-11-24T06:12:21.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:21.728+0000 7f2aa77fe640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f2aa815f9c0 0x7f2aa81408d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:39564/0 (socket says 172.21.15.92:39564) 2023-11-24T06:12:21.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:21.728+0000 7f2aa77fe640 1 -- 172.21.15.92:0/1964286849 learned_addr learned my addr 172.21.15.92:0/1964286849 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-24T06:12:21.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:21.728+0000 7f2a9e7fc640 1 --2- 172.21.15.92:0/1964286849 >> v2:172.21.15.118:3300/0 conn(0x7f2aa813e360 0x7f2aa80cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:21.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:21.728+0000 7f2aa6ffd640 1 --2- 172.21.15.92:0/1964286849 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2aa813df80 0x7f2aa815bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:21.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:21.728+0000 7f2aa67fc640 1 -- 172.21.15.92:0/1964286849 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1320875487 0 0) 0x7f2aa806c470 con 0x7f2aa815f9c0 2023-11-24T06:12:21.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:21.728+0000 7f2aa67fc640 1 -- 172.21.15.92:0/1964286849 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2a94003520 con 0x7f2aa815f9c0 2023-11-24T06:12:21.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:21.729+0000 7f2aa67fc640 1 -- 172.21.15.92:0/1964286849 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2444246780 0 0) 0x7f2a94003520 con 0x7f2aa815f9c0 2023-11-24T06:12:21.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:21.729+0000 7f2aa67fc640 1 -- 172.21.15.92:0/1964286849 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2aa813df80 msgr2=0x7f2aa815bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:21.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:21.729+0000 7f2aa67fc640 1 --2- 172.21.15.92:0/1964286849 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2aa813df80 0x7f2aa815bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:21.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:21.729+0000 7f2aa67fc640 1 -- 172.21.15.92:0/1964286849 >> v2:172.21.15.118:3300/0 conn(0x7f2aa813e360 msgr2=0x7f2aa80cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:21.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:21.729+0000 7f2aa67fc640 1 --2- 172.21.15.92:0/1964286849 >> v2:172.21.15.118:3300/0 conn(0x7f2aa813e360 0x7f2aa80cf290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:21.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:21.729+0000 7f2aa67fc640 1 -- 172.21.15.92:0/1964286849 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2aa806cd10 con 0x7f2aa815f9c0 2023-11-24T06:12:21.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:21.729+0000 7f2aa67fc640 1 -- 172.21.15.92:0/1964286849 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2607661350 0 0) 0x7f2aa00019f0 con 0x7f2aa815f9c0 2023-11-24T06:12:21.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:21.729+0000 7f2aa67fc640 1 -- 172.21.15.92:0/1964286849 >> v1:172.21.15.92:6789/0 conn(0x7f2aa815f9c0 legacy=0x7f2aa81408d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:21.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:21.729+0000 7f2aa67fc640 1 --2- 172.21.15.92:0/1964286849 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2a94003970 0x7f2a94003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:21.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:21.729+0000 7f2aa67fc640 1 --2- 172.21.15.92:0/1964286849 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2a940047c0 0x7f2a94004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:21.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:21.729+0000 7f2aa67fc640 1 --2- 172.21.15.92:0/1964286849 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2aa813df80 0x7f2a94005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:21.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:21.729+0000 7f2aa67fc640 1 -- 172.21.15.92:0/1964286849 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2aa806c750 con 0x7f2a940047c0 2023-11-24T06:12:21.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:21.729+0000 7f2aa67fc640 1 -- 172.21.15.92:0/1964286849 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2aa806ca30 con 0x7f2a94003970 2023-11-24T06:12:21.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:21.729+0000 7f2aa67fc640 1 -- 172.21.15.92:0/1964286849 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a9400b980 con 0x7f2aa813df80 2023-11-24T06:12:21.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:21.729+0000 7f2a9e7fc640 1 --2- 172.21.15.92:0/1964286849 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2a940047c0 0x7f2a94004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:21.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:21.730+0000 7f2aa6ffd640 1 --2- 172.21.15.92:0/1964286849 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2a94003970 0x7f2a94003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:21.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:21.730+0000 7f2aa77fe640 1 --2- 172.21.15.92:0/1964286849 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2aa813df80 0x7f2a94005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:21.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:21.730+0000 7f2a9e7fc640 1 -- 172.21.15.92:0/1964286849 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2aa813df80 msgr2=0x7f2a94005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:21.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:21.730+0000 7f2a9e7fc640 1 --2- 172.21.15.92:0/1964286849 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2aa813df80 0x7f2a94005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:21.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:21.730+0000 7f2a9e7fc640 1 -- 172.21.15.92:0/1964286849 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2a94003970 msgr2=0x7f2a94003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:21.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:21.730+0000 7f2a9e7fc640 1 --2- 172.21.15.92:0/1964286849 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2a94003970 0x7f2a94003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:21.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:21.730+0000 7f2a9e7fc640 1 -- 172.21.15.92:0/1964286849 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a94003520 con 0x7f2a940047c0 2023-11-24T06:12:21.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:21.730+0000 7f2aa6ffd640 1 --2- 172.21.15.92:0/1964286849 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2a94003970 0x7f2a94003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:12:21.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:21.731+0000 7f2a9e7fc640 1 --2- 172.21.15.92:0/1964286849 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2a940047c0 0x7f2a94004c40 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f2a90009ac0 tx=0x7f2a940057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a66fd19354938567 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:21.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:21.731+0000 7f2aa67fc640 1 -- 172.21.15.92:0/1964286849 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a9002c710 con 0x7f2a940047c0 2023-11-24T06:12:21.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:21.731+0000 7f2aa67fc640 1 -- 172.21.15.92:0/1964286849 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a9002c870 con 0x7f2a940047c0 2023-11-24T06:12:21.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:21.731+0000 7f2aa67fc640 1 -- 172.21.15.92:0/1964286849 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a9002cc00 con 0x7f2a940047c0 2023-11-24T06:12:21.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:21.732+0000 7f2aad9d5640 1 -- 172.21.15.92:0/1964286849 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2a940047c0 msgr2=0x7f2a94004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:21.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:21.732+0000 7f2aad9d5640 1 --2- 172.21.15.92:0/1964286849 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2a940047c0 0x7f2a94004c40 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f2a90009ac0 tx=0x7f2a940057f0 comp rx=0 tx=0).stop 2023-11-24T06:12:21.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:21.732+0000 7f2aa77fe640 1 -- 172.21.15.92:0/1964286849 reap_dead start 2023-11-24T06:12:21.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:21.732+0000 7f2aad9d5640 1 -- 172.21.15.92:0/1964286849 shutdown_connections 2023-11-24T06:12:21.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:21.732+0000 7f2aad9d5640 1 -- 172.21.15.92:0/1964286849 >> 172.21.15.92:0/1964286849 conn(0x7f2aa806a800 msgr2=0x7f2aa806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:21.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:21.732+0000 7f2aad9d5640 1 -- 172.21.15.92:0/1964286849 shutdown_connections 2023-11-24T06:12:21.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:21.732+0000 7f2aad9d5640 1 -- 172.21.15.92:0/1964286849 wait complete. 2023-11-24T06:12:21.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:21.734+0000 7f2aad9d5640 1 Processor -- start 2023-11-24T06:12:21.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:21.734+0000 7f2aad9d5640 1 -- start start 2023-11-24T06:12:21.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:21.735+0000 7f2aad9d5640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2a940047c0 0x7f2aa814f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:21.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:21.735+0000 7f2aad9d5640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2aa814d530 0x7f2aa814de10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:21.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:21.735+0000 7f2aad9d5640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2aa815a140 0x7f2aa815a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:21.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:21.735+0000 7f2aad9d5640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2aa815ab00 con 0x7f2a940047c0 2023-11-24T06:12:21.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:21.735+0000 7f2aad9d5640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2aa8151030 con 0x7f2aa814d530 2023-11-24T06:12:21.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:21.735+0000 7f2aad9d5640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2aa814ff90 con 0x7f2aa815a140 2023-11-24T06:12:21.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:21.735+0000 7f2aa6ffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2a940047c0 0x7f2aa814f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:21.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:21.735+0000 7f2aa6ffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2a940047c0 0x7f2aa814f8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43320/0 (socket says 172.21.15.92:43320) 2023-11-24T06:12:21.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:21.735+0000 7f2aa6ffd640 1 -- 172.21.15.92:0/1956144868 learned_addr learned my addr 172.21.15.92:0/1956144868 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:12:21.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:21.736+0000 7f2aa6ffd640 1 -- 172.21.15.92:0/1956144868 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2aa815a140 msgr2=0x7f2aa815a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:21.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:21.736+0000 7f2aa6ffd640 1 --2- 172.21.15.92:0/1956144868 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2aa815a140 0x7f2aa815a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:21.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:21.736+0000 7f2aa6ffd640 1 -- 172.21.15.92:0/1956144868 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2aa814d530 msgr2=0x7f2aa814de10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:21.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:21.736+0000 7f2aa6ffd640 1 --2- 172.21.15.92:0/1956144868 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2aa814d530 0x7f2aa814de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:21.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:21.736+0000 7f2aa6ffd640 1 -- 172.21.15.92:0/1956144868 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2aa8150270 con 0x7f2a940047c0 2023-11-24T06:12:21.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:21.736+0000 7f2aa6ffd640 1 --2- 172.21.15.92:0/1956144868 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2a940047c0 0x7f2aa814f8f0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f2a98011700 tx=0x7f2a98011bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:21.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:21.736+0000 7f2aa4ff9640 1 -- 172.21.15.92:0/1956144868 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a9800f8e0 con 0x7f2a940047c0 2023-11-24T06:12:21.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:21.736+0000 7f2aa4ff9640 1 -- 172.21.15.92:0/1956144868 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a9800fa40 con 0x7f2a940047c0 2023-11-24T06:12:21.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:21.736+0000 7f2aa4ff9640 1 -- 172.21.15.92:0/1956144868 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a9800fdf0 con 0x7f2a940047c0 2023-11-24T06:12:21.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:21.736+0000 7f2aad9d5640 1 -- 172.21.15.92:0/1956144868 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2aa8150550 con 0x7f2a940047c0 2023-11-24T06:12:21.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:21.737+0000 7f2aad9d5640 1 -- 172.21.15.92:0/1956144868 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2aa8150b10 con 0x7f2a940047c0 2023-11-24T06:12:21.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:21.738+0000 7f2aad9d5640 1 -- 172.21.15.92:0/1956144868 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2aa8150cd0 con 0x7f2a940047c0 2023-11-24T06:12:21.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:21.739+0000 7f2aa4ff9640 1 -- 172.21.15.92:0/1956144868 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2a98015460 con 0x7f2a940047c0 2023-11-24T06:12:21.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:21.739+0000 7f2aa4ff9640 1 --2- 172.21.15.92:0/1956144868 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f2a8806ae40 0x7f2a8806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:21.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:21.740+0000 7f2aa4ff9640 1 -- 172.21.15.92:0/1956144868 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f2a98089b20 con 0x7f2a940047c0 2023-11-24T06:12:21.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:21.740+0000 7f2a9e7fc640 1 --2- 172.21.15.92:0/1956144868 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f2a8806ae40 0x7f2a8806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:21.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:21.740+0000 7f2a9e7fc640 1 --2- 172.21.15.92:0/1956144868 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f2a8806ae40 0x7f2a8806d300 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f2a90036000 tx=0x7f2a9002e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:21.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:21.744+0000 7f2aa4ff9640 1 -- 172.21.15.92:0/1956144868 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2a98054de0 con 0x7f2a940047c0 2023-11-24T06:12:21.936 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:21.933+0000 7f2aad9d5640 1 -- 172.21.15.92:0/1956144868 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2aa81546d0 con 0x7f2a940047c0 2023-11-24T06:12:21.937 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:21.934+0000 7f2aa4ff9640 1 -- 172.21.15.92:0/1956144868 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v43) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2a98022020 con 0x7f2a940047c0 2023-11-24T06:12:21.937 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 1 2023-11-24T06:12:21.940 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:21.936+0000 7f2aad9d5640 1 -- 172.21.15.92:0/1956144868 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f2a8806ae40 msgr2=0x7f2a8806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:21.940 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:21.936+0000 7f2aad9d5640 1 --2- 172.21.15.92:0/1956144868 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f2a8806ae40 0x7f2a8806d300 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f2a90036000 tx=0x7f2a9002e000 comp rx=0 tx=0).stop 2023-11-24T06:12:21.940 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:21.936+0000 7f2aad9d5640 1 -- 172.21.15.92:0/1956144868 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2a940047c0 msgr2=0x7f2aa814f8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:21.940 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:21.936+0000 7f2aad9d5640 1 --2- 172.21.15.92:0/1956144868 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2a940047c0 0x7f2aa814f8f0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f2a98011700 tx=0x7f2a98011bc0 comp rx=0 tx=0).stop 2023-11-24T06:12:21.940 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:21.936+0000 7f2aad9d5640 1 -- 172.21.15.92:0/1956144868 shutdown_connections 2023-11-24T06:12:21.940 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:21.936+0000 7f2aad9d5640 1 --2- 172.21.15.92:0/1956144868 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f2a8806ae40 0x7f2a8806d300 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:21.940 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:21.936+0000 7f2aad9d5640 1 --2- 172.21.15.92:0/1956144868 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2aa815a140 0x7f2aa815a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:21.940 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:21.936+0000 7f2aad9d5640 1 --2- 172.21.15.92:0/1956144868 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2aa814d530 0x7f2aa814de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:21.941 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:21.936+0000 7f2aad9d5640 1 --2- 172.21.15.92:0/1956144868 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2a940047c0 0x7f2aa814f8f0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:21.941 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:21.936+0000 7f2aad9d5640 1 -- 172.21.15.92:0/1956144868 >> 172.21.15.92:0/1956144868 conn(0x7f2aa806a800 msgr2=0x7f2aa8153fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:21.941 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:21.937+0000 7f2aad9d5640 1 -- 172.21.15.92:0/1956144868 shutdown_connections 2023-11-24T06:12:21.941 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:21.937+0000 7f2aad9d5640 1 -- 172.21.15.92:0/1956144868 wait complete. 2023-11-24T06:12:21.958 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_info' 2023-11-24T06:12:22.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:22.169+0000 7fea12ada640 1 Processor -- start 2023-11-24T06:12:22.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:22.169+0000 7fea12ada640 1 -- start start 2023-11-24T06:12:22.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:22.170+0000 7fea12ada640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fea0c13df80 0x7fea0c15bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:22.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:22.171+0000 7fea12ada640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fea0c13e360 0x7fea0c0cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:22.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:22.171+0000 7fea12ada640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea0c06c470 con 0x7fea0c15f9c0 2023-11-24T06:12:22.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:22.171+0000 7fea12ada640 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea0c06c750 con 0x7fea0c13df80 2023-11-24T06:12:22.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:22.171+0000 7fea12ada640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea0c06ca30 con 0x7fea0c13e360 2023-11-24T06:12:22.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:22.171+0000 7fea11077640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fea0c15f9c0 0x7fea0c1408d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:39566/0 (socket says 172.21.15.92:39566) 2023-11-24T06:12:22.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:22.171+0000 7fea11077640 1 -- 172.21.15.92:0/2683789653 learned_addr learned my addr 172.21.15.92:0/2683789653 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-24T06:12:22.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:22.171+0000 7fea10876640 1 --2- 172.21.15.92:0/2683789653 >> v2:172.21.15.118:3300/0 conn(0x7fea0c13df80 0x7fea0c15bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:22.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:22.171+0000 7fea0bfff640 1 --2- 172.21.15.92:0/2683789653 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fea0c13e360 0x7fea0c0cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:22.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:22.171+0000 7fea0b7fe640 1 -- 172.21.15.92:0/2683789653 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 601003114 0 0) 0x7fea0c06c470 con 0x7fea0c15f9c0 2023-11-24T06:12:22.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:22.172+0000 7fea0b7fe640 1 -- 172.21.15.92:0/2683789653 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe9fc003520 con 0x7fea0c15f9c0 2023-11-24T06:12:22.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:22.172+0000 7fea0b7fe640 1 -- 172.21.15.92:0/2683789653 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2449205334 0 0) 0x7fe9fc003520 con 0x7fea0c15f9c0 2023-11-24T06:12:22.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:22.172+0000 7fea0b7fe640 1 -- 172.21.15.92:0/2683789653 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fea0c13e360 msgr2=0x7fea0c0cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:22.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:22.172+0000 7fea0b7fe640 1 --2- 172.21.15.92:0/2683789653 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fea0c13e360 0x7fea0c0cf290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:22.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:22.172+0000 7fea0b7fe640 1 -- 172.21.15.92:0/2683789653 >> v2:172.21.15.118:3300/0 conn(0x7fea0c13df80 msgr2=0x7fea0c15bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:22.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:22.172+0000 7fea0b7fe640 1 --2- 172.21.15.92:0/2683789653 >> v2:172.21.15.118:3300/0 conn(0x7fea0c13df80 0x7fea0c15bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:22.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:22.172+0000 7fea0b7fe640 1 -- 172.21.15.92:0/2683789653 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea0c06cd10 con 0x7fea0c15f9c0 2023-11-24T06:12:22.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:22.172+0000 7fea0b7fe640 1 -- 172.21.15.92:0/2683789653 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2607661350 0 0) 0x7fea040019f0 con 0x7fea0c15f9c0 2023-11-24T06:12:22.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:22.172+0000 7fea0b7fe640 1 -- 172.21.15.92:0/2683789653 >> v1:172.21.15.92:6789/0 conn(0x7fea0c15f9c0 legacy=0x7fea0c1408d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:22.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:22.172+0000 7fea0b7fe640 1 --2- 172.21.15.92:0/2683789653 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fea0c13e360 0x7fe9fc003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:22.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:22.172+0000 7fea0b7fe640 1 --2- 172.21.15.92:0/2683789653 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe9fc003f40 0x7fe9fc0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:22.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:22.172+0000 7fea0b7fe640 1 --2- 172.21.15.92:0/2683789653 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe9fc004d00 0x7fe9fc005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:22.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:22.172+0000 7fea0b7fe640 1 -- 172.21.15.92:0/2683789653 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea0c06c750 con 0x7fe9fc003f40 2023-11-24T06:12:22.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:22.172+0000 7fea0b7fe640 1 -- 172.21.15.92:0/2683789653 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea0c06ca30 con 0x7fe9fc004d00 2023-11-24T06:12:22.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:22.172+0000 7fea0b7fe640 1 -- 172.21.15.92:0/2683789653 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9fc00b980 con 0x7fea0c13e360 2023-11-24T06:12:22.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:22.172+0000 7fea0bfff640 1 --2- 172.21.15.92:0/2683789653 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe9fc003f40 0x7fe9fc0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:22.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:22.173+0000 7fea0bfff640 1 -- 172.21.15.92:0/2683789653 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fea0c13e360 msgr2=0x7fe9fc003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:22.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:22.173+0000 7fea0bfff640 1 --2- 172.21.15.92:0/2683789653 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fea0c13e360 0x7fe9fc003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:22.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:22.173+0000 7fea11077640 1 --2- 172.21.15.92:0/2683789653 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe9fc004d00 0x7fe9fc005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:22.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:22.173+0000 7fea0bfff640 1 -- 172.21.15.92:0/2683789653 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe9fc004d00 msgr2=0x7fe9fc005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:22.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:22.173+0000 7fea0bfff640 1 --2- 172.21.15.92:0/2683789653 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe9fc004d00 0x7fe9fc005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:22.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:22.173+0000 7fea0bfff640 1 -- 172.21.15.92:0/2683789653 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9fc003520 con 0x7fe9fc003f40 2023-11-24T06:12:22.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:22.173+0000 7fea11077640 1 --2- 172.21.15.92:0/2683789653 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe9fc004d00 0x7fe9fc005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:12:22.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:22.174+0000 7fea0bfff640 1 --2- 172.21.15.92:0/2683789653 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe9fc003f40 0x7fe9fc0047c0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fe9f8009af0 tx=0x7fe9fc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=289fe0da8e477ba1 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:22.180 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:22.174+0000 7fea0b7fe640 1 -- 172.21.15.92:0/2683789653 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9f802c710 con 0x7fe9fc003f40 2023-11-24T06:12:22.180 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:22.174+0000 7fea0b7fe640 1 -- 172.21.15.92:0/2683789653 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9f802c870 con 0x7fe9fc003f40 2023-11-24T06:12:22.180 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:22.174+0000 7fea0b7fe640 1 -- 172.21.15.92:0/2683789653 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9f802cc00 con 0x7fe9fc003f40 2023-11-24T06:12:22.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:22.175+0000 7fea12ada640 1 -- 172.21.15.92:0/2683789653 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe9fc003f40 msgr2=0x7fe9fc0047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:22.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:22.175+0000 7fea12ada640 1 --2- 172.21.15.92:0/2683789653 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe9fc003f40 0x7fe9fc0047c0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fe9f8009af0 tx=0x7fe9fc0057f0 comp rx=0 tx=0).stop 2023-11-24T06:12:22.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:22.175+0000 7fea11077640 1 -- 172.21.15.92:0/2683789653 reap_dead start 2023-11-24T06:12:22.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:22.175+0000 7fea12ada640 1 -- 172.21.15.92:0/2683789653 shutdown_connections 2023-11-24T06:12:22.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:22.175+0000 7fea12ada640 1 -- 172.21.15.92:0/2683789653 >> 172.21.15.92:0/2683789653 conn(0x7fea0c06a800 msgr2=0x7fea0c06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:22.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:22.175+0000 7fea12ada640 1 -- 172.21.15.92:0/2683789653 shutdown_connections 2023-11-24T06:12:22.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:22.175+0000 7fea12ada640 1 -- 172.21.15.92:0/2683789653 wait complete. 2023-11-24T06:12:22.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:22.177+0000 7fea12ada640 1 Processor -- start 2023-11-24T06:12:22.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:22.177+0000 7fea12ada640 1 -- start start 2023-11-24T06:12:22.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:22.178+0000 7fea12ada640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe9fc003f40 0x7fea0c071af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:22.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:22.178+0000 7fea12ada640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fea0c1f6dd0 0x7fea0c14c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:22.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:22.178+0000 7fea12ada640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fea0c1f2b80 0x7fea0c152a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:22.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:22.178+0000 7fea12ada640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea0c1f34f0 con 0x7fea0c1f2b80 2023-11-24T06:12:22.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:22.178+0000 7fea12ada640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea0c1f37d0 con 0x7fe9fc003f40 2023-11-24T06:12:22.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:22.178+0000 7fea12ada640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea0c1f3ab0 con 0x7fea0c1f6dd0 2023-11-24T06:12:22.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:22.178+0000 7fea11077640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fea0c1f2b80 0x7fea0c152a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:22.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:22.178+0000 7fea11077640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fea0c1f2b80 0x7fea0c152a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43334/0 (socket says 172.21.15.92:43334) 2023-11-24T06:12:22.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:22.178+0000 7fea11077640 1 -- 172.21.15.92:0/3210066278 learned_addr learned my addr 172.21.15.92:0/3210066278 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:12:22.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:22.179+0000 7fea10876640 1 --2- 172.21.15.92:0/3210066278 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe9fc003f40 0x7fea0c071af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:22.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:22.179+0000 7fea0bfff640 1 --2- 172.21.15.92:0/3210066278 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fea0c1f6dd0 0x7fea0c14c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:22.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:22.179+0000 7fea11077640 1 -- 172.21.15.92:0/3210066278 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fea0c1f6dd0 msgr2=0x7fea0c14c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:22.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:22.179+0000 7fea11077640 1 --2- 172.21.15.92:0/3210066278 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fea0c1f6dd0 0x7fea0c14c000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:22.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:22.179+0000 7fea11077640 1 -- 172.21.15.92:0/3210066278 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe9fc003f40 msgr2=0x7fea0c071af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:22.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:22.179+0000 7fea11077640 1 --2- 172.21.15.92:0/3210066278 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe9fc003f40 0x7fea0c071af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:22.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:22.179+0000 7fea11077640 1 -- 172.21.15.92:0/3210066278 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea0c14b800 con 0x7fea0c1f2b80 2023-11-24T06:12:22.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:22.179+0000 7fea10876640 1 --2- 172.21.15.92:0/3210066278 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe9fc003f40 0x7fea0c071af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:12:22.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:22.179+0000 7fea11077640 1 --2- 172.21.15.92:0/3210066278 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fea0c1f2b80 0x7fea0c152a30 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fea0c15b6b0 tx=0x7fea0400b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:22.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:22.179+0000 7fea097fa640 1 -- 172.21.15.92:0/3210066278 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea0400bb80 con 0x7fea0c1f2b80 2023-11-24T06:12:22.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:22.179+0000 7fea097fa640 1 -- 172.21.15.92:0/3210066278 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea0400bd00 con 0x7fea0c1f2b80 2023-11-24T06:12:22.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:22.179+0000 7fea12ada640 1 -- 172.21.15.92:0/3210066278 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea0c14bae0 con 0x7fea0c1f2b80 2023-11-24T06:12:22.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:22.179+0000 7fea097fa640 1 -- 172.21.15.92:0/3210066278 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea04007350 con 0x7fea0c1f2b80 2023-11-24T06:12:22.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:22.179+0000 7fea12ada640 1 -- 172.21.15.92:0/3210066278 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea0c14c540 con 0x7fea0c1f2b80 2023-11-24T06:12:22.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:22.180+0000 7fea12ada640 1 -- 172.21.15.92:0/3210066278 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea0c14bcd0 con 0x7fea0c1f2b80 2023-11-24T06:12:22.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:22.181+0000 7fea097fa640 1 -- 172.21.15.92:0/3210066278 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fea040074b0 con 0x7fea0c1f2b80 2023-11-24T06:12:22.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:22.182+0000 7fea097fa640 1 --2- 172.21.15.92:0/3210066278 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fe9f006ae40 0x7fe9f006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:22.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:22.182+0000 7fea097fa640 1 -- 172.21.15.92:0/3210066278 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5029+0+0 (secure 0 0 0) 0x7fea04089890 con 0x7fea0c1f2b80 2023-11-24T06:12:22.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:22.183+0000 7fea10876640 1 --2- 172.21.15.92:0/3210066278 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fe9f006ae40 0x7fe9f006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:22.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:22.183+0000 7fea10876640 1 --2- 172.21.15.92:0/3210066278 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fe9f006ae40 0x7fe9f006d300 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fea0c15b2b0 tx=0x7fea00008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:22.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:22.187+0000 7fea097fa640 1 -- 172.21.15.92:0/3210066278 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fea04054bd0 con 0x7fea0c1f2b80 2023-11-24T06:12:22.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:22.442+0000 7fea12ada640 1 -- 172.21.15.92:0/3210066278 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_info"]} v 0) v1 -- 0x7fea0c0675d0 con 0x7fea0c1f2b80 2023-11-24T06:12:23.064 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:23.061+0000 7fea097fa640 1 -- 172.21.15.92:0/3210066278 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_info"]}]=0 v130) v1 ==== 136+0+0 (secure 0 0 0) 0x7fea040581f0 con 0x7fea0c1f2b80 2023-11-24T06:12:23.067 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:23.064+0000 7fea12ada640 1 -- 172.21.15.92:0/3210066278 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fe9f006ae40 msgr2=0x7fe9f006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:23.067 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:23.064+0000 7fea12ada640 1 --2- 172.21.15.92:0/3210066278 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fe9f006ae40 0x7fe9f006d300 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fea0c15b2b0 tx=0x7fea00008040 comp rx=0 tx=0).stop 2023-11-24T06:12:23.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:23.064+0000 7fea12ada640 1 -- 172.21.15.92:0/3210066278 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fea0c1f2b80 msgr2=0x7fea0c152a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:23.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:23.064+0000 7fea12ada640 1 --2- 172.21.15.92:0/3210066278 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fea0c1f2b80 0x7fea0c152a30 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fea0c15b6b0 tx=0x7fea0400b4a0 comp rx=0 tx=0).stop 2023-11-24T06:12:23.069 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:23.065+0000 7fea12ada640 1 -- 172.21.15.92:0/3210066278 shutdown_connections 2023-11-24T06:12:23.069 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:23.065+0000 7fea12ada640 1 --2- 172.21.15.92:0/3210066278 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fe9f006ae40 0x7fe9f006d300 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:23.069 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:23.065+0000 7fea12ada640 1 --2- 172.21.15.92:0/3210066278 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fea0c1f2b80 0x7fea0c152a30 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:23.069 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:23.065+0000 7fea12ada640 1 --2- 172.21.15.92:0/3210066278 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fea0c1f6dd0 0x7fea0c14c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:23.069 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:23.065+0000 7fea12ada640 1 --2- 172.21.15.92:0/3210066278 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe9fc003f40 0x7fea0c071af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:23.069 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:23.065+0000 7fea12ada640 1 -- 172.21.15.92:0/3210066278 >> 172.21.15.92:0/3210066278 conn(0x7fea0c06a800 msgr2=0x7fea0c15d8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:23.069 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:23.065+0000 7fea12ada640 1 -- 172.21.15.92:0/3210066278 shutdown_connections 2023-11-24T06:12:23.069 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:23.065+0000 7fea12ada640 1 -- 172.21.15.92:0/3210066278 wait complete. 2023-11-24T06:12:23.089 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-24T06:12:23.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:23.295+0000 7f33f6c7c640 1 Processor -- start 2023-11-24T06:12:23.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:23.295+0000 7f33f6c7c640 1 -- start start 2023-11-24T06:12:23.300 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:23.297+0000 7f33f6c7c640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f33f00cf9a0 0x7f33f015e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:23.300 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:23.297+0000 7f33f6c7c640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33f0070c40 con 0x7f33f00ae5a0 2023-11-24T06:12:23.300 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:23.297+0000 7f33f6c7c640 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33f006c470 con 0x7f33f00b3ae0 2023-11-24T06:12:23.300 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:23.297+0000 7f33f6c7c640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33f0071450 con 0x7f33f00cf9a0 2023-11-24T06:12:23.300 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:23.297+0000 7f33f4a18640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f33f00ae5a0 0x7f33f006cbe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:39570/0 (socket says 172.21.15.92:39570) 2023-11-24T06:12:23.300 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:23.297+0000 7f33f4a18640 1 -- 172.21.15.92:0/751785759 learned_addr learned my addr 172.21.15.92:0/751785759 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-24T06:12:23.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:23.297+0000 7f33f5219640 1 --2- 172.21.15.92:0/751785759 >> v2:172.21.15.92:3300/0 conn(0x7f33f00cf9a0 0x7f33f015e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:23.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:23.298+0000 7f33ef7fe640 1 -- 172.21.15.92:0/751785759 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2882218152 0 0) 0x7f33f0070c40 con 0x7f33f00ae5a0 2023-11-24T06:12:23.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:23.298+0000 7f33ef7fe640 1 -- 172.21.15.92:0/751785759 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f33e0003520 con 0x7f33f00ae5a0 2023-11-24T06:12:23.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:23.298+0000 7f33f5219640 1 -- 172.21.15.92:0/751785759 >> v1:172.21.15.118:6789/0 conn(0x7f33f00b3ae0 legacy=0x7f33f00cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:23.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:23.298+0000 7f33f5219640 1 -- 172.21.15.92:0/751785759 >> v1:172.21.15.92:6789/0 conn(0x7f33f00ae5a0 legacy=0x7f33f006cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:23.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:23.298+0000 7f33f5219640 1 -- 172.21.15.92:0/751785759 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33f006c750 con 0x7f33f00cf9a0 2023-11-24T06:12:23.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:23.299+0000 7f33f5219640 1 --2- 172.21.15.92:0/751785759 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f33f00cf9a0 0x7f33f015e430 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f33e8009d90 tx=0x7f33e802bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a572a8dcff909dc8 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:23.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:23.299+0000 7f33ef7fe640 1 -- 172.21.15.92:0/751785759 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33e802cd00 con 0x7f33f00cf9a0 2023-11-24T06:12:23.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:23.299+0000 7f33ef7fe640 1 -- 172.21.15.92:0/751785759 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33e802ce60 con 0x7f33f00cf9a0 2023-11-24T06:12:23.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:23.299+0000 7f33ef7fe640 1 -- 172.21.15.92:0/751785759 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33e8034660 con 0x7f33f00cf9a0 2023-11-24T06:12:23.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:23.300+0000 7f33f6c7c640 1 -- 172.21.15.92:0/751785759 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f33f00cf9a0 msgr2=0x7f33f015e430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:23.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:23.300+0000 7f33f6c7c640 1 --2- 172.21.15.92:0/751785759 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f33f00cf9a0 0x7f33f015e430 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f33e8009d90 tx=0x7f33e802bf50 comp rx=0 tx=0).stop 2023-11-24T06:12:23.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:23.300+0000 7f33f6c7c640 1 -- 172.21.15.92:0/751785759 shutdown_connections 2023-11-24T06:12:23.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:23.300+0000 7f33f6c7c640 1 --2- 172.21.15.92:0/751785759 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f33f00cf9a0 0x7f33f015e430 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:23.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:23.300+0000 7f33f6c7c640 1 -- 172.21.15.92:0/751785759 >> 172.21.15.92:0/751785759 conn(0x7f33f006a800 msgr2=0x7f33f006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:23.304 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:23.300+0000 7f33f6c7c640 1 -- 172.21.15.92:0/751785759 shutdown_connections 2023-11-24T06:12:23.304 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:23.300+0000 7f33f6c7c640 1 -- 172.21.15.92:0/751785759 wait complete. 2023-11-24T06:12:23.304 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:23.301+0000 7f33f6c7c640 1 Processor -- start 2023-11-24T06:12:23.305 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:23.302+0000 7f33f6c7c640 1 -- start start 2023-11-24T06:12:23.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:23.302+0000 7f33f6c7c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f33f00ae5a0 0x7f33f0148bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:23.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:23.303+0000 7f33f6c7c640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f33f00b3ae0 0x7f33f0158700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:23.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:23.303+0000 7f33f6c7c640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f33f00cf9a0 0x7f33f0147d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:23.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:23.303+0000 7f33f6c7c640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33f01494d0 con 0x7f33f00ae5a0 2023-11-24T06:12:23.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:23.303+0000 7f33f6c7c640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33f0158c40 con 0x7f33f00b3ae0 2023-11-24T06:12:23.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:23.303+0000 7f33f6c7c640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33f00cb8a0 con 0x7f33f00cf9a0 2023-11-24T06:12:23.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:23.303+0000 7f33f4a18640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f33f00ae5a0 0x7f33f0148bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:23.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:23.303+0000 7f33f4a18640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f33f00ae5a0 0x7f33f0148bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43356/0 (socket says 172.21.15.92:43356) 2023-11-24T06:12:23.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:23.303+0000 7f33f4a18640 1 -- 172.21.15.92:0/2447319526 learned_addr learned my addr 172.21.15.92:0/2447319526 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:12:23.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:23.303+0000 7f33f4a18640 1 -- 172.21.15.92:0/2447319526 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f33f00cf9a0 msgr2=0x7f33f0147d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:23.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:23.303+0000 7f33f4a18640 1 --2- 172.21.15.92:0/2447319526 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f33f00cf9a0 0x7f33f0147d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:23.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:23.303+0000 7f33effff640 1 --2- 172.21.15.92:0/2447319526 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f33f00b3ae0 0x7f33f0158700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:23.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:23.303+0000 7f33f4a18640 1 -- 172.21.15.92:0/2447319526 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f33f00b3ae0 msgr2=0x7f33f0158700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:23.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:23.303+0000 7f33f4a18640 1 --2- 172.21.15.92:0/2447319526 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f33f00b3ae0 0x7f33f0158700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:23.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:23.303+0000 7f33f4a18640 1 -- 172.21.15.92:0/2447319526 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33e8009a70 con 0x7f33f00ae5a0 2023-11-24T06:12:23.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:23.304+0000 7f33f4a18640 1 --2- 172.21.15.92:0/2447319526 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f33f00ae5a0 0x7f33f0148bd0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f33e400c500 tx=0x7f33e400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:23.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:23.304+0000 7f33ed7fa640 1 -- 172.21.15.92:0/2447319526 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33e4005ce0 con 0x7f33f00ae5a0 2023-11-24T06:12:23.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:23.304+0000 7f33ed7fa640 1 -- 172.21.15.92:0/2447319526 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33e4005e40 con 0x7f33f00ae5a0 2023-11-24T06:12:23.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:23.304+0000 7f33f6c7c640 1 -- 172.21.15.92:0/2447319526 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33f0158e70 con 0x7f33f00ae5a0 2023-11-24T06:12:23.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:23.304+0000 7f33ed7fa640 1 -- 172.21.15.92:0/2447319526 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33e4007330 con 0x7f33f00ae5a0 2023-11-24T06:12:23.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:23.304+0000 7f33f6c7c640 1 -- 172.21.15.92:0/2447319526 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33f00cbaf0 con 0x7f33f00ae5a0 2023-11-24T06:12:23.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:23.305+0000 7f33f6c7c640 1 -- 172.21.15.92:0/2447319526 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33f00675d0 con 0x7f33f00ae5a0 2023-11-24T06:12:23.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:23.306+0000 7f33ed7fa640 1 -- 172.21.15.92:0/2447319526 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f33e4007550 con 0x7f33f00ae5a0 2023-11-24T06:12:23.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:23.307+0000 7f33ed7fa640 1 --2- 172.21.15.92:0/2447319526 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f33d406ae40 0x7f33d406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:23.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:23.307+0000 7f33ed7fa640 1 -- 172.21.15.92:0/2447319526 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f33e4088a20 con 0x7f33f00ae5a0 2023-11-24T06:12:23.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:23.307+0000 7f33effff640 1 --2- 172.21.15.92:0/2447319526 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f33d406ae40 0x7f33d406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:23.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:23.308+0000 7f33effff640 1 --2- 172.21.15.92:0/2447319526 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f33d406ae40 0x7f33d406d300 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f33f00b2010 tx=0x7f33dc009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:23.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:23.311+0000 7f33ed7fa640 1 -- 172.21.15.92:0/2447319526 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f33e4053e20 con 0x7f33f00ae5a0 2023-11-24T06:12:23.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:23.486+0000 7f33f6c7c640 1 -- 172.21.15.92:0/2447319526 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f33f00cf5e0 con 0x7f33f00ae5a0 2023-11-24T06:12:23.498 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:23.495+0000 7f33ed7fa640 1 -- 172.21.15.92:0/2447319526 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v12)=0 v12) v1 ==== 122+0+0 (secure 0 0 0) 0x7f33e4017020 con 0x7f33f00ae5a0 2023-11-24T06:12:23.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:23.497+0000 7f33f6c7c640 1 -- 172.21.15.92:0/2447319526 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f33d406ae40 msgr2=0x7f33d406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:23.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:23.497+0000 7f33f6c7c640 1 --2- 172.21.15.92:0/2447319526 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f33d406ae40 0x7f33d406d300 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f33f00b2010 tx=0x7f33dc009040 comp rx=0 tx=0).stop 2023-11-24T06:12:23.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:23.497+0000 7f33f6c7c640 1 -- 172.21.15.92:0/2447319526 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f33f00ae5a0 msgr2=0x7f33f0148bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:23.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:23.497+0000 7f33f6c7c640 1 --2- 172.21.15.92:0/2447319526 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f33f00ae5a0 0x7f33f0148bd0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f33e400c500 tx=0x7f33e400c9c0 comp rx=0 tx=0).stop 2023-11-24T06:12:23.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:23.497+0000 7f33f6c7c640 1 -- 172.21.15.92:0/2447319526 shutdown_connections 2023-11-24T06:12:23.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:23.497+0000 7f33f6c7c640 1 --2- 172.21.15.92:0/2447319526 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f33d406ae40 0x7f33d406d300 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:23.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:23.497+0000 7f33f6c7c640 1 --2- 172.21.15.92:0/2447319526 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f33f00cf9a0 0x7f33f0147d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:23.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:23.498+0000 7f33f6c7c640 1 --2- 172.21.15.92:0/2447319526 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f33f00b3ae0 0x7f33f0158700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:23.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:23.498+0000 7f33f6c7c640 1 --2- 172.21.15.92:0/2447319526 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f33f00ae5a0 0x7f33f0148bd0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:23.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:23.498+0000 7f33f6c7c640 1 -- 172.21.15.92:0/2447319526 >> 172.21.15.92:0/2447319526 conn(0x7f33f006a800 msgr2=0x7f33f00adb50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:23.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:23.498+0000 7f33f6c7c640 1 -- 172.21.15.92:0/2447319526 shutdown_connections 2023-11-24T06:12:23.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:23.498+0000 7f33f6c7c640 1 -- 172.21.15.92:0/2447319526 wait complete. 2023-11-24T06:12:23.518 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-24T06:12:23.519 DEBUG:teuthology.orchestra.run.smithi092:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-24T06:12:23.621 DEBUG:teuthology.orchestra.run.smithi118:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-24T06:12:23.743 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-24T06:12:23.744 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-11-24T06:12:23.744 DEBUG:teuthology.orchestra.run.smithi092:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:12:23.802 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054298,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498824k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249408k,nr_inodes=812352,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498824k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-24T06:12:23.802 DEBUG:teuthology.orchestra.run.smithi118:> set -ex 2023-11-24T06:12:23.803 DEBUG:teuthology.orchestra.run.smithi118:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:12:23.862 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054284,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249396k,nr_inodes=812349,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-24T06:12:23.863 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-11-24T06:12:23.863 DEBUG:teuthology.orchestra.run.smithi092:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:12:23.922 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054298,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498824k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249408k,nr_inodes=812352,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498824k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-24T06:12:23.922 DEBUG:teuthology.orchestra.run.smithi118:> set -ex 2023-11-24T06:12:23.922 DEBUG:teuthology.orchestra.run.smithi118:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:12:23.982 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054284,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249396k,nr_inodes=812349,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-24T06:12:23.982 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-11-24T06:12:23.982 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T06:12:24.207 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:24.204+0000 7f4cd0a7c640 1 Processor -- start 2023-11-24T06:12:24.207 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:24.204+0000 7f4cd0a7c640 1 -- start start 2023-11-24T06:12:24.209 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:24.205+0000 7f4cd0a7c640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f4ccc0723e0 0x7f4ccc166170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:24.209 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:24.205+0000 7f4cd0a7c640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ccc0ae8a0 con 0x7f4ccc0ae3f0 2023-11-24T06:12:24.209 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:24.205+0000 7f4cd0a7c640 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ccc15fd40 con 0x7f4ccc15b3f0 2023-11-24T06:12:24.209 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:24.206+0000 7f4cd0a7c640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ccc071930 con 0x7f4ccc0723e0 2023-11-24T06:12:24.209 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:24.206+0000 7f4ccad9d640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f4ccc0723e0 0x7f4ccc166170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:24.209 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:24.206+0000 7f4cc3fff640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f4ccc0ae3f0 0x7f4ccc071cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:39578/0 (socket says 172.21.15.92:39578) 2023-11-24T06:12:24.209 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:24.206+0000 7f4cc3fff640 1 -- 172.21.15.92:0/1642746273 learned_addr learned my addr 172.21.15.92:0/1642746273 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-24T06:12:24.210 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:24.206+0000 7f4ccad9d640 1 -- 172.21.15.92:0/1642746273 >> v1:172.21.15.118:6789/0 conn(0x7f4ccc15b3f0 legacy=0x7f4ccc0adce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:24.210 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:24.206+0000 7f4ccad9d640 1 -- 172.21.15.92:0/1642746273 >> v1:172.21.15.92:6789/0 conn(0x7f4ccc0ae3f0 legacy=0x7f4ccc071cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:24.210 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:24.206+0000 7f4ccad9d640 1 -- 172.21.15.92:0/1642746273 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ccc15ef80 con 0x7f4ccc0723e0 2023-11-24T06:12:24.210 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:24.206+0000 7f4cc9d9b640 1 -- 172.21.15.92:0/1642746273 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 990762641 0 0) 0x7f4ccc0ae8a0 con 0x7f4ccc0ae3f0 2023-11-24T06:12:24.210 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:24.208+0000 7f4ccad9d640 1 --2- 172.21.15.92:0/1642746273 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4ccc0723e0 0x7f4ccc166170 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f4cc4009d90 tx=0x7f4cc402beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=698a742cdb7a5083 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:24.211 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:24.208+0000 7f4cc9d9b640 1 -- 172.21.15.92:0/1642746273 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cc402cc60 con 0x7f4ccc0723e0 2023-11-24T06:12:24.211 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:24.208+0000 7f4cc9d9b640 1 -- 172.21.15.92:0/1642746273 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cc402cdc0 con 0x7f4ccc0723e0 2023-11-24T06:12:24.211 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:24.208+0000 7f4cc9d9b640 1 -- 172.21.15.92:0/1642746273 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cc4034660 con 0x7f4ccc0723e0 2023-11-24T06:12:24.212 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:24.209+0000 7f4cd0a7c640 1 -- 172.21.15.92:0/1642746273 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4ccc0723e0 msgr2=0x7f4ccc166170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:24.212 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:24.209+0000 7f4cd0a7c640 1 --2- 172.21.15.92:0/1642746273 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4ccc0723e0 0x7f4ccc166170 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f4cc4009d90 tx=0x7f4cc402beb0 comp rx=0 tx=0).stop 2023-11-24T06:12:24.212 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:24.209+0000 7f4cd0a7c640 1 -- 172.21.15.92:0/1642746273 shutdown_connections 2023-11-24T06:12:24.212 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:24.209+0000 7f4cd0a7c640 1 --2- 172.21.15.92:0/1642746273 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4ccc0723e0 0x7f4ccc166170 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:24.212 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:24.209+0000 7f4cd0a7c640 1 -- 172.21.15.92:0/1642746273 >> 172.21.15.92:0/1642746273 conn(0x7f4ccc0b0c00 msgr2=0x7f4ccc0b1010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:24.212 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:24.209+0000 7f4cd0a7c640 1 -- 172.21.15.92:0/1642746273 shutdown_connections 2023-11-24T06:12:24.213 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:24.209+0000 7f4cd0a7c640 1 -- 172.21.15.92:0/1642746273 wait complete. 2023-11-24T06:12:24.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:24.211+0000 7f4cd0a7c640 1 Processor -- start 2023-11-24T06:12:24.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:24.211+0000 7f4cd0a7c640 1 -- start start 2023-11-24T06:12:24.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:24.212+0000 7f4cd0a7c640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4ccc0723e0 0x7f4ccc0b3260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:24.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:24.212+0000 7f4cd0a7c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4ccc0ae3f0 0x7f4ccc09be50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:24.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:24.212+0000 7f4cd0a7c640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4ccc15b3f0 0x7f4ccc15b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:24.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:24.212+0000 7f4cd0a7c640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ccc09a990 con 0x7f4ccc0ae3f0 2023-11-24T06:12:24.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:24.212+0000 7f4cd0a7c640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ccc09a6b0 con 0x7f4ccc15b3f0 2023-11-24T06:12:24.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:24.212+0000 7f4cd0a7c640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ccc09b370 con 0x7f4ccc0723e0 2023-11-24T06:12:24.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:24.213+0000 7f4cc3fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4ccc0ae3f0 0x7f4ccc09be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:24.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:24.213+0000 7f4cc3fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4ccc0ae3f0 0x7f4ccc09be50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43374/0 (socket says 172.21.15.92:43374) 2023-11-24T06:12:24.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:24.213+0000 7f4cc3fff640 1 -- 172.21.15.92:0/2302072038 learned_addr learned my addr 172.21.15.92:0/2302072038 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:12:24.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:24.213+0000 7f4cc3fff640 1 -- 172.21.15.92:0/2302072038 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4ccc0723e0 msgr2=0x7f4ccc0b3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:24.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:24.213+0000 7f4cc3fff640 1 --2- 172.21.15.92:0/2302072038 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4ccc0723e0 0x7f4ccc0b3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:24.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:24.213+0000 7f4cc3fff640 1 -- 172.21.15.92:0/2302072038 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4ccc15b3f0 msgr2=0x7f4ccc15b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:24.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:24.213+0000 7f4ccad9d640 1 --2- 172.21.15.92:0/2302072038 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4ccc15b3f0 0x7f4ccc15b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:24.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:24.213+0000 7f4cc3fff640 1 --2- 172.21.15.92:0/2302072038 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4ccc15b3f0 0x7f4ccc15b7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:24.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:24.213+0000 7f4cc3fff640 1 -- 172.21.15.92:0/2302072038 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cc4009a70 con 0x7f4ccc0ae3f0 2023-11-24T06:12:24.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:24.213+0000 7f4cc3fff640 1 --2- 172.21.15.92:0/2302072038 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4ccc0ae3f0 0x7f4ccc09be50 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f4cb400c470 tx=0x7f4cb400c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:24.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:24.214+0000 7f4cc2ffd640 1 -- 172.21.15.92:0/2302072038 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cb40074e0 con 0x7f4ccc0ae3f0 2023-11-24T06:12:24.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:24.214+0000 7f4cc2ffd640 1 -- 172.21.15.92:0/2302072038 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cb4007640 con 0x7f4ccc0ae3f0 2023-11-24T06:12:24.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:24.214+0000 7f4cc2ffd640 1 -- 172.21.15.92:0/2302072038 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cb4004a20 con 0x7f4ccc0ae3f0 2023-11-24T06:12:24.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:24.214+0000 7f4cd0a7c640 1 -- 172.21.15.92:0/2302072038 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ccc09b650 con 0x7f4ccc0ae3f0 2023-11-24T06:12:24.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:24.214+0000 7f4cd0a7c640 1 -- 172.21.15.92:0/2302072038 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ccc0b37a0 con 0x7f4ccc0ae3f0 2023-11-24T06:12:24.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:24.215+0000 7f4cd0a7c640 1 -- 172.21.15.92:0/2302072038 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ccc09bb70 con 0x7f4ccc0ae3f0 2023-11-24T06:12:24.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:24.216+0000 7f4cc2ffd640 1 -- 172.21.15.92:0/2302072038 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f4cb4004b80 con 0x7f4ccc0ae3f0 2023-11-24T06:12:24.220 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:24.217+0000 7f4cc2ffd640 1 --2- 172.21.15.92:0/2302072038 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f4cac06ae40 0x7f4cac06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:24.220 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:24.217+0000 7f4cc2ffd640 1 -- 172.21.15.92:0/2302072038 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f4cb4088690 con 0x7f4ccc0ae3f0 2023-11-24T06:12:24.220 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:24.217+0000 7f4cca59c640 1 --2- 172.21.15.92:0/2302072038 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f4cac06ae40 0x7f4cac06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:24.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:24.218+0000 7f4cca59c640 1 --2- 172.21.15.92:0/2302072038 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f4cac06ae40 0x7f4cac06d300 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f4cbc006fd0 tx=0x7f4cbc006bc0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:24.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:24.222+0000 7f4cc2ffd640 1 -- 172.21.15.92:0/2302072038 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4ccc09bb70 con 0x7f4ccc0ae3f0 2023-11-24T06:12:24.451 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:24.448+0000 7f4cd0a7c640 1 -- 172.21.15.92:0/2302072038 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4ccc09bb70 con 0x7f4ccc0ae3f0 2023-11-24T06:12:24.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:24.449+0000 7f4cc2ffd640 1 -- 172.21.15.92:0/2302072038 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+3705 (secure 0 0 0) 0x7f4ccc09bb70 con 0x7f4ccc0ae3f0 2023-11-24T06:12:24.452 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-24T06:12:24.453 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":25,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4637,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6834/2665242481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":2665242481},{"type":"v1","addr":"172.21.15.118:6834","nonce":2665242481}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":4640,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6835/2822230615","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6833","nonce":2822230615},{"type":"v1","addr":"172.21.15.118:6835","nonce":2822230615}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5499,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6836/2918354210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":2918354210},{"type":"v1","addr":"172.21.15.92:6836","nonce":2918354210}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":23},{"gid":5631,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/4066957863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":4066957863},{"type":"v1","addr":"172.21.15.92:6837","nonce":4066957863}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":25}],"filesystems":[]} 2023-11-24T06:12:24.453 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 25 2023-11-24T06:12:24.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:24.452+0000 7f4cd0a7c640 1 -- 172.21.15.92:0/2302072038 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f4cac06ae40 msgr2=0x7f4cac06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:24.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:24.452+0000 7f4cd0a7c640 1 --2- 172.21.15.92:0/2302072038 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f4cac06ae40 0x7f4cac06d300 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f4cbc006fd0 tx=0x7f4cbc006bc0 comp rx=0 tx=0).stop 2023-11-24T06:12:24.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:24.452+0000 7f4cd0a7c640 1 -- 172.21.15.92:0/2302072038 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4ccc0ae3f0 msgr2=0x7f4ccc09be50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:24.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:24.452+0000 7f4cd0a7c640 1 --2- 172.21.15.92:0/2302072038 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4ccc0ae3f0 0x7f4ccc09be50 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f4cb400c470 tx=0x7f4cb400c930 comp rx=0 tx=0).stop 2023-11-24T06:12:24.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:24.452+0000 7f4cd0a7c640 1 -- 172.21.15.92:0/2302072038 shutdown_connections 2023-11-24T06:12:24.456 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:24.452+0000 7f4cd0a7c640 1 --2- 172.21.15.92:0/2302072038 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f4cac06ae40 0x7f4cac06d300 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:24.456 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:24.452+0000 7f4cd0a7c640 1 --2- 172.21.15.92:0/2302072038 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4ccc15b3f0 0x7f4ccc15b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:24.456 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:24.452+0000 7f4cd0a7c640 1 --2- 172.21.15.92:0/2302072038 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4ccc0ae3f0 0x7f4ccc09be50 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:24.456 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:24.452+0000 7f4cd0a7c640 1 --2- 172.21.15.92:0/2302072038 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4ccc0723e0 0x7f4ccc0b3260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:24.456 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:24.452+0000 7f4cd0a7c640 1 -- 172.21.15.92:0/2302072038 >> 172.21.15.92:0/2302072038 conn(0x7f4ccc0b0c00 msgr2=0x7f4ccc0aef50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:24.456 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:24.452+0000 7f4cd0a7c640 1 -- 172.21.15.92:0/2302072038 shutdown_connections 2023-11-24T06:12:24.456 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:24.452+0000 7f4cd0a7c640 1 -- 172.21.15.92:0/2302072038 wait complete. 2023-11-24T06:12:24.473 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-24T06:12:24.473 INFO:tasks.ceph.mds.a:Stopping old one... 2023-11-24T06:12:24.473 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-24T06:12:24.473 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T06:12:24.474 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-24T06:12:24.474 INFO:tasks.ceph.mds.c:Stopping old one... 2023-11-24T06:12:24.474 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-11-24T06:12:24.474 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T06:12:24.474 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-24T06:12:24.474 INFO:tasks.ceph.mds.b:Stopping old one... 2023-11-24T06:12:24.475 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-24T06:12:24.475 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T06:12:24.475 INFO:tasks.ceph.mds.d:Restarting daemon 2023-11-24T06:12:24.475 INFO:tasks.ceph.mds.d:Stopping old one... 2023-11-24T06:12:24.475 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-11-24T06:12:24.475 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T06:12:24.491 INFO:tasks.ceph.mds.a:Stopped 2023-11-24T06:12:24.491 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-24T06:12:24.494 INFO:tasks.ceph.mds.c:Stopped 2023-11-24T06:12:24.495 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-24T06:12:24.496 INFO:tasks.ceph.mds.d:Stopped 2023-11-24T06:12:24.496 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-11-24T06:12:24.506 INFO:tasks.ceph.mds.b:Stopped 2023-11-24T06:12:24.506 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-24T06:12:24.543 INFO:tasks.ceph.mds.a:Started 2023-11-24T06:12:24.543 DEBUG:teuthology.parallel:result is None 2023-11-24T06:12:24.551 INFO:tasks.ceph.mds.c:Started 2023-11-24T06:12:24.551 DEBUG:teuthology.parallel:result is None 2023-11-24T06:12:24.554 INFO:tasks.ceph.mds.d:Started 2023-11-24T06:12:24.554 DEBUG:teuthology.parallel:result is None 2023-11-24T06:12:24.563 INFO:tasks.ceph.mds.b:Started 2023-11-24T06:12:24.563 DEBUG:teuthology.parallel:result is None 2023-11-24T06:12:24.563 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-11-24T06:12:24.708 INFO:tasks.ceph.mds.c.smithi092.stdout:starting mds.c at 2023-11-24T06:12:24.708 INFO:tasks.ceph.mds.a.smithi092.stdout:starting mds.a at 2023-11-24T06:12:24.717 INFO:tasks.ceph.mds.d.smithi118.stdout:starting mds.d at 2023-11-24T06:12:24.723 INFO:tasks.ceph.mds.b.smithi118.stdout:starting mds.b at 2023-11-24T06:12:24.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:24.759+0000 7eff0f51d640 1 Processor -- start 2023-11-24T06:12:24.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:24.759+0000 7eff0f51d640 1 -- start start 2023-11-24T06:12:24.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:24.760+0000 7eff0f51d640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7eff080ae5a0 0x7eff0815e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:24.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:24.760+0000 7eff0f51d640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7eff080b3ae0 0x7eff0806cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:24.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:24.761+0000 7eff0f51d640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff08070c40 con 0x7eff0806d120 2023-11-24T06:12:24.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:24.761+0000 7eff0f51d640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff08071450 con 0x7eff080b3ae0 2023-11-24T06:12:24.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:24.761+0000 7eff0f51d640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff0806c470 con 0x7eff080ae5a0 2023-11-24T06:12:24.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:24.761+0000 7eff0cab8640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7eff080b3ae0 0x7eff0806cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:24.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:24.761+0000 7eff0daba640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7eff0806d120 0x7eff080cf290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:39602/0 (socket says 172.21.15.92:39602) 2023-11-24T06:12:24.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:24.761+0000 7eff0daba640 1 -- 172.21.15.92:0/1915660099 learned_addr learned my addr 172.21.15.92:0/1915660099 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-24T06:12:24.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:24.761+0000 7eff0d2b9640 1 --2- 172.21.15.92:0/1915660099 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7eff080ae5a0 0x7eff0815e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:24.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:24.761+0000 7eff0cab8640 1 -- 172.21.15.92:0/1915660099 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7eff080ae5a0 msgr2=0x7eff0815e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:24.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:24.761+0000 7eff0cab8640 1 --2- 172.21.15.92:0/1915660099 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7eff080ae5a0 0x7eff0815e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:24.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:24.761+0000 7eff0cab8640 1 -- 172.21.15.92:0/1915660099 >> v1:172.21.15.92:6789/0 conn(0x7eff0806d120 legacy=0x7eff080cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:24.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:24.761+0000 7eff0cab8640 1 -- 172.21.15.92:0/1915660099 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff0806c750 con 0x7eff080b3ae0 2023-11-24T06:12:24.766 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:24.761+0000 7efefffff640 1 -- 172.21.15.92:0/1915660099 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 144451739 0 0) 0x7eff08070c40 con 0x7eff0806d120 2023-11-24T06:12:24.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:24.762+0000 7eff0cab8640 1 --2- 172.21.15.92:0/1915660099 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7eff080b3ae0 0x7eff0806cbe0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7efef80097a0 tx=0x7efef802b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62132e8f2a604f2e server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:24.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:24.763+0000 7efefffff640 1 -- 172.21.15.92:0/1915660099 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efef802d040 con 0x7eff080b3ae0 2023-11-24T06:12:24.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:24.763+0000 7efefffff640 1 -- 172.21.15.92:0/1915660099 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efef8031810 con 0x7eff080b3ae0 2023-11-24T06:12:24.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:24.763+0000 7efefffff640 1 -- 172.21.15.92:0/1915660099 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efef8031ba0 con 0x7eff080b3ae0 2023-11-24T06:12:24.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:24.763+0000 7eff0f51d640 1 -- 172.21.15.92:0/1915660099 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7eff080b3ae0 msgr2=0x7eff0806cbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:24.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:24.763+0000 7eff0f51d640 1 --2- 172.21.15.92:0/1915660099 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7eff080b3ae0 0x7eff0806cbe0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7efef80097a0 tx=0x7efef802b8b0 comp rx=0 tx=0).stop 2023-11-24T06:12:24.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:24.763+0000 7eff0f51d640 1 -- 172.21.15.92:0/1915660099 shutdown_connections 2023-11-24T06:12:24.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:24.763+0000 7eff0f51d640 1 --2- 172.21.15.92:0/1915660099 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7eff080b3ae0 0x7eff0806cbe0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:24.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:24.763+0000 7eff0f51d640 1 --2- 172.21.15.92:0/1915660099 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7eff080ae5a0 0x7eff0815e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:24.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:24.763+0000 7eff0f51d640 1 -- 172.21.15.92:0/1915660099 >> 172.21.15.92:0/1915660099 conn(0x7eff0806a800 msgr2=0x7eff0806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:24.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:24.763+0000 7eff0f51d640 1 -- 172.21.15.92:0/1915660099 shutdown_connections 2023-11-24T06:12:24.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:24.763+0000 7eff0f51d640 1 -- 172.21.15.92:0/1915660099 wait complete. 2023-11-24T06:12:24.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:24.765+0000 7eff0f51d640 1 Processor -- start 2023-11-24T06:12:24.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:24.765+0000 7eff0f51d640 1 -- start start 2023-11-24T06:12:24.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:24.766+0000 7eff0f51d640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7eff0806d120 0x7eff080cbc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:24.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:24.766+0000 7eff0f51d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7eff080ae5a0 0x7eff081520c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:24.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:24.766+0000 7eff0f51d640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7eff080b3ae0 0x7eff081f3810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:24.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:24.766+0000 7eff0f51d640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff080cc2d0 con 0x7eff080ae5a0 2023-11-24T06:12:24.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:24.766+0000 7eff0f51d640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff080cc5b0 con 0x7eff080b3ae0 2023-11-24T06:12:24.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:24.766+0000 7eff0f51d640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff081f3010 con 0x7eff0806d120 2023-11-24T06:12:24.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:24.767+0000 7eff0cab8640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7eff080ae5a0 0x7eff081520c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:24.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:24.767+0000 7eff0cab8640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7eff080ae5a0 0x7eff081520c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43424/0 (socket says 172.21.15.92:43424) 2023-11-24T06:12:24.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:24.767+0000 7eff0cab8640 1 -- 172.21.15.92:0/3649472629 learned_addr learned my addr 172.21.15.92:0/3649472629 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:12:24.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:24.767+0000 7eff0d2b9640 1 --2- 172.21.15.92:0/3649472629 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7eff0806d120 0x7eff080cbc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:24.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:24.767+0000 7eff0daba640 1 --2- 172.21.15.92:0/3649472629 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7eff080b3ae0 0x7eff081f3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:24.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:24.767+0000 7eff0cab8640 1 -- 172.21.15.92:0/3649472629 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7eff0806d120 msgr2=0x7eff080cbc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:24.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:24.767+0000 7eff0cab8640 1 --2- 172.21.15.92:0/3649472629 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7eff0806d120 0x7eff080cbc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:24.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:24.767+0000 7eff0cab8640 1 -- 172.21.15.92:0/3649472629 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7eff080b3ae0 msgr2=0x7eff081f3810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:24.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:24.767+0000 7eff0cab8640 1 --2- 172.21.15.92:0/3649472629 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7eff080b3ae0 0x7eff081f3810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:24.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:24.767+0000 7eff0cab8640 1 -- 172.21.15.92:0/3649472629 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efef8009400 con 0x7eff080ae5a0 2023-11-24T06:12:24.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:24.767+0000 7eff0cab8640 1 --2- 172.21.15.92:0/3649472629 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7eff080ae5a0 0x7eff081520c0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7efef8000c00 tx=0x7efef80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:24.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:24.768+0000 7efefdffb640 1 -- 172.21.15.92:0/3649472629 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efef803ad60 con 0x7eff080ae5a0 2023-11-24T06:12:24.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:24.768+0000 7efefdffb640 1 -- 172.21.15.92:0/3649472629 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efef80443f0 con 0x7eff080ae5a0 2023-11-24T06:12:24.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:24.768+0000 7eff0f51d640 1 -- 172.21.15.92:0/3649472629 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff081f32f0 con 0x7eff080ae5a0 2023-11-24T06:12:24.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:24.768+0000 7efefdffb640 1 -- 172.21.15.92:0/3649472629 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efef80447a0 con 0x7eff080ae5a0 2023-11-24T06:12:24.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:24.768+0000 7eff0f51d640 1 -- 172.21.15.92:0/3649472629 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff0815a290 con 0x7eff080ae5a0 2023-11-24T06:12:24.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:24.769+0000 7eff0f51d640 1 -- 172.21.15.92:0/3649472629 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff080675d0 con 0x7eff080ae5a0 2023-11-24T06:12:24.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:24.770+0000 7efefdffb640 1 -- 172.21.15.92:0/3649472629 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7efef804d450 con 0x7eff080ae5a0 2023-11-24T06:12:24.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:24.771+0000 7efefdffb640 1 --2- 172.21.15.92:0/3649472629 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7efeec06ae40 0x7efeec06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:24.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:24.771+0000 7efefdffb640 1 -- 172.21.15.92:0/3649472629 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5029+0+0 (secure 0 0 0) 0x7efef80ab560 con 0x7eff080ae5a0 2023-11-24T06:12:24.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:24.772+0000 7eff0d2b9640 1 --2- 172.21.15.92:0/3649472629 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7efeec06ae40 0x7efeec06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:24.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:24.772+0000 7eff0d2b9640 1 --2- 172.21.15.92:0/3649472629 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7efeec06ae40 0x7efeec06d300 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7eff04006fd0 tx=0x7eff04008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:24.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:24.776+0000 7efefdffb640 1 -- 172.21.15.92:0/3649472629 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7efef8076950 con 0x7eff080ae5a0 2023-11-24T06:12:24.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:24.979+0000 7eff0f51d640 1 -- 172.21.15.92:0/3649472629 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7eff081591a0 con 0x7eff080ae5a0 2023-11-24T06:12:26.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:26.100+0000 7efefdffb640 1 -- 172.21.15.92:0/3649472629 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v48) v1 ==== 97+0+0 (secure 0 0 0) 0x7efef8079f70 con 0x7eff080ae5a0 2023-11-24T06:12:26.103 INFO:teuthology.orchestra.run.smithi092.stderr: removed all blocklist entries 2023-11-24T06:12:26.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:26.103+0000 7eff0f51d640 1 -- 172.21.15.92:0/3649472629 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7efeec06ae40 msgr2=0x7efeec06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:26.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:26.103+0000 7eff0f51d640 1 --2- 172.21.15.92:0/3649472629 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7efeec06ae40 0x7efeec06d300 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7eff04006fd0 tx=0x7eff04008040 comp rx=0 tx=0).stop 2023-11-24T06:12:26.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:26.103+0000 7eff0f51d640 1 -- 172.21.15.92:0/3649472629 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7eff080ae5a0 msgr2=0x7eff081520c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:26.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:26.103+0000 7eff0f51d640 1 --2- 172.21.15.92:0/3649472629 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7eff080ae5a0 0x7eff081520c0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7efef8000c00 tx=0x7efef80046c0 comp rx=0 tx=0).stop 2023-11-24T06:12:26.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:26.104+0000 7eff0f51d640 1 -- 172.21.15.92:0/3649472629 shutdown_connections 2023-11-24T06:12:26.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:26.104+0000 7eff0f51d640 1 --2- 172.21.15.92:0/3649472629 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7efeec06ae40 0x7efeec06d300 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:26.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:26.104+0000 7eff0f51d640 1 --2- 172.21.15.92:0/3649472629 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7eff080b3ae0 0x7eff081f3810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:26.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:26.104+0000 7eff0f51d640 1 --2- 172.21.15.92:0/3649472629 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7eff080ae5a0 0x7eff081520c0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:26.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:26.104+0000 7eff0f51d640 1 --2- 172.21.15.92:0/3649472629 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7eff0806d120 0x7eff080cbc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:26.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:26.104+0000 7eff0f51d640 1 -- 172.21.15.92:0/3649472629 >> 172.21.15.92:0/3649472629 conn(0x7eff0806a800 msgr2=0x7eff080b28f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:26.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:26.104+0000 7eff0f51d640 1 -- 172.21.15.92:0/3649472629 shutdown_connections 2023-11-24T06:12:26.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:26.104+0000 7eff0f51d640 1 -- 172.21.15.92:0/3649472629 wait complete. 2023-11-24T06:12:26.127 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-11-24T06:12:26.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:26.331+0000 7f67a8f5a640 1 Processor -- start 2023-11-24T06:12:26.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:26.331+0000 7f67a8f5a640 1 -- start start 2023-11-24T06:12:26.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:26.332+0000 7f67a8f5a640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f67a4150760 0x7f67a4072230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:26.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:26.332+0000 7f67a8f5a640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f67a415d300 0x7f67a415d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:26.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:26.332+0000 7f67a8f5a640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f67a40b2bd0 0x7f67a40b3040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:26.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:26.332+0000 7f67a8f5a640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67a415dc80 con 0x7f67a4150760 2023-11-24T06:12:26.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:26.332+0000 7f67a8f5a640 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67a4071190 con 0x7f67a415d300 2023-11-24T06:12:26.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:26.332+0000 7f67a8f5a640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67a4071470 con 0x7f67a40b2bd0 2023-11-24T06:12:26.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:26.333+0000 7f67a259c640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f67a4150760 0x7f67a4072230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:26.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:26.333+0000 7f67a259c640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f67a4150760 0x7f67a4072230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43434/0 (socket says 172.21.15.92:43434) 2023-11-24T06:12:26.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:26.333+0000 7f67a259c640 1 -- 172.21.15.92:0/4218609149 learned_addr learned my addr 172.21.15.92:0/4218609149 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:12:26.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:26.333+0000 7f67a2d9d640 1 --2- 172.21.15.92:0/4218609149 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f67a40b2bd0 0x7f67a40b3040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:26.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:26.333+0000 7f67a259c640 1 -- 172.21.15.92:0/4218609149 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f67a40b2bd0 msgr2=0x7f67a40b3040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:26.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:26.333+0000 7f67a1d9b640 1 --2- 172.21.15.92:0/4218609149 >> v2:172.21.15.118:3300/0 conn(0x7f67a415d300 0x7f67a415d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:26.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:26.333+0000 7f67a259c640 1 --2- 172.21.15.92:0/4218609149 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f67a40b2bd0 0x7f67a40b3040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:26.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:26.333+0000 7f67a259c640 1 -- 172.21.15.92:0/4218609149 >> v2:172.21.15.118:3300/0 conn(0x7f67a415d300 msgr2=0x7f67a415d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:26.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:26.333+0000 7f67a259c640 1 --2- 172.21.15.92:0/4218609149 >> v2:172.21.15.118:3300/0 conn(0x7f67a415d300 0x7f67a415d740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:26.338 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:26.333+0000 7f67a259c640 1 -- 172.21.15.92:0/4218609149 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67a4071750 con 0x7f67a4150760 2023-11-24T06:12:26.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:26.334+0000 7f67a259c640 1 --2- 172.21.15.92:0/4218609149 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f67a4150760 0x7f67a4072230 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f67a40b1ca0 tx=0x7f679402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17dad1d4dea100b1 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:26.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:26.335+0000 7f67a159a640 1 -- 172.21.15.92:0/4218609149 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6794031870 con 0x7f67a4150760 2023-11-24T06:12:26.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:26.335+0000 7f67a159a640 1 -- 172.21.15.92:0/4218609149 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67940319d0 con 0x7f67a4150760 2023-11-24T06:12:26.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:26.335+0000 7f67a159a640 1 -- 172.21.15.92:0/4218609149 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6794031d60 con 0x7f67a4150760 2023-11-24T06:12:26.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:26.335+0000 7f67a8f5a640 1 -- 172.21.15.92:0/4218609149 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f67a4150760 msgr2=0x7f67a4072230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:26.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:26.335+0000 7f67a8f5a640 1 --2- 172.21.15.92:0/4218609149 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f67a4150760 0x7f67a4072230 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f67a40b1ca0 tx=0x7f679402b910 comp rx=0 tx=0).stop 2023-11-24T06:12:26.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:26.335+0000 7f67a8f5a640 1 -- 172.21.15.92:0/4218609149 shutdown_connections 2023-11-24T06:12:26.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:26.335+0000 7f67a8f5a640 1 --2- 172.21.15.92:0/4218609149 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f67a40b2bd0 0x7f67a40b3040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:26.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:26.335+0000 7f67a8f5a640 1 --2- 172.21.15.92:0/4218609149 >> v2:172.21.15.118:3300/0 conn(0x7f67a415d300 0x7f67a415d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:26.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:26.335+0000 7f67a8f5a640 1 --2- 172.21.15.92:0/4218609149 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f67a4150760 0x7f67a4072230 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:26.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:26.335+0000 7f67a8f5a640 1 -- 172.21.15.92:0/4218609149 >> 172.21.15.92:0/4218609149 conn(0x7f67a406a800 msgr2=0x7f67a406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:26.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:26.335+0000 7f67a8f5a640 1 -- 172.21.15.92:0/4218609149 shutdown_connections 2023-11-24T06:12:26.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:26.335+0000 7f67a8f5a640 1 -- 172.21.15.92:0/4218609149 wait complete. 2023-11-24T06:12:26.341 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:26.337+0000 7f67a8f5a640 1 Processor -- start 2023-11-24T06:12:26.341 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:26.337+0000 7f67a8f5a640 1 -- start start 2023-11-24T06:12:26.343 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:26.338+0000 7f67a8f5a640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f67a40b2bd0 0x7f67a414d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:26.343 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:26.338+0000 7f67a8f5a640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f67a4150760 0x7f67a41588f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:26.343 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:26.338+0000 7f67a8f5a640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f67a415d300 0x7f67a414e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:26.343 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:26.338+0000 7f67a8f5a640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67a4159fa0 con 0x7f67a40b2bd0 2023-11-24T06:12:26.343 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:26.338+0000 7f67a8f5a640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67a4159390 con 0x7f67a4150760 2023-11-24T06:12:26.343 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:26.338+0000 7f67a8f5a640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67a4159670 con 0x7f67a415d300 2023-11-24T06:12:26.343 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:26.338+0000 7f67a259c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f67a40b2bd0 0x7f67a414d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:26.343 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:26.339+0000 7f67a259c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f67a40b2bd0 0x7f67a414d710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43446/0 (socket says 172.21.15.92:43446) 2023-11-24T06:12:26.344 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:26.339+0000 7f67a259c640 1 -- 172.21.15.92:0/3884990174 learned_addr learned my addr 172.21.15.92:0/3884990174 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:12:26.344 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:26.339+0000 7f67a1d9b640 1 --2- 172.21.15.92:0/3884990174 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f67a4150760 0x7f67a41588f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:26.344 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:26.339+0000 7f67a2d9d640 1 --2- 172.21.15.92:0/3884990174 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f67a415d300 0x7f67a414e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:26.344 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:26.339+0000 7f67a259c640 1 -- 172.21.15.92:0/3884990174 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f67a415d300 msgr2=0x7f67a414e9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:26.344 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:26.339+0000 7f67a259c640 1 --2- 172.21.15.92:0/3884990174 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f67a415d300 0x7f67a414e9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:26.344 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:26.339+0000 7f67a259c640 1 -- 172.21.15.92:0/3884990174 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f67a4150760 msgr2=0x7f67a41588f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:26.344 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:26.339+0000 7f67a259c640 1 --2- 172.21.15.92:0/3884990174 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f67a4150760 0x7f67a41588f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:26.344 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:26.339+0000 7f67a259c640 1 -- 172.21.15.92:0/3884990174 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67940094d0 con 0x7f67a40b2bd0 2023-11-24T06:12:26.344 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:26.339+0000 7f67a259c640 1 --2- 172.21.15.92:0/3884990174 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f67a40b2bd0 0x7f67a414d710 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f67940097c0 tx=0x7f67940046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:26.344 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:26.340+0000 7f6792ffd640 1 -- 172.21.15.92:0/3884990174 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6794031820 con 0x7f67a40b2bd0 2023-11-24T06:12:26.344 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:26.340+0000 7f6792ffd640 1 -- 172.21.15.92:0/3884990174 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67940443f0 con 0x7f67a40b2bd0 2023-11-24T06:12:26.344 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:26.340+0000 7f6792ffd640 1 -- 172.21.15.92:0/3884990174 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67940447a0 con 0x7f67a40b2bd0 2023-11-24T06:12:26.344 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:26.340+0000 7f67a8f5a640 1 -- 172.21.15.92:0/3884990174 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67a4159950 con 0x7f67a40b2bd0 2023-11-24T06:12:26.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:26.340+0000 7f67a8f5a640 1 -- 172.21.15.92:0/3884990174 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67a4150f30 con 0x7f67a40b2bd0 2023-11-24T06:12:26.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:26.341+0000 7f67a8f5a640 1 -- 172.21.15.92:0/3884990174 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67a40675d0 con 0x7f67a40b2bd0 2023-11-24T06:12:26.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:26.342+0000 7f6792ffd640 1 -- 172.21.15.92:0/3884990174 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f679404d450 con 0x7f67a40b2bd0 2023-11-24T06:12:26.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:26.343+0000 7f6792ffd640 1 --2- 172.21.15.92:0/3884990174 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f678406ae40 0x7f678406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:26.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:26.343+0000 7f6792ffd640 1 -- 172.21.15.92:0/3884990174 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f679402d040 con 0x7f67a40b2bd0 2023-11-24T06:12:26.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:26.343+0000 7f67a1d9b640 1 --2- 172.21.15.92:0/3884990174 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f678406ae40 0x7f678406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:26.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:26.344+0000 7f67a1d9b640 1 --2- 172.21.15.92:0/3884990174 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f678406ae40 0x7f678406d300 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f6798006fd0 tx=0x7f6798008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:26.351 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:26.348+0000 7f6792ffd640 1 -- 172.21.15.92:0/3884990174 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6794076db0 con 0x7f67a40b2bd0 2023-11-24T06:12:26.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:26.605+0000 7f67a8f5a640 1 -- 172.21.15.92:0/3884990174 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f67a40af1b0 con 0x7f67a40b2bd0 2023-11-24T06:12:26.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:26.606+0000 7f6792ffd640 1 -- 172.21.15.92:0/3884990174 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v9) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f6794035090 con 0x7f67a40b2bd0 2023-11-24T06:12:26.609 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-24T06:12:26.609 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-24T06:12:26.609 INFO:teuthology.orchestra.run.smithi092.stdout: "auth_dump": [ 2023-11-24T06:12:26.609 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-24T06:12:26.610 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "mds.a", 2023-11-24T06:12:26.610 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQAQPmBl501xBxAAEtXTFSRjPeIoKJpcPILQwA==", 2023-11-24T06:12:26.610 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-11-24T06:12:26.610 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-11-24T06:12:26.610 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow *", 2023-11-24T06:12:26.610 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow *", 2023-11-24T06:12:26.610 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-11-24T06:12:26.610 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-24T06:12:26.610 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-24T06:12:26.610 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-24T06:12:26.610 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "mds.b", 2023-11-24T06:12:26.610 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQAQPmBlaY/AKhAArTolS+ALdzAB8M0lQ3/Mjg==", 2023-11-24T06:12:26.611 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-11-24T06:12:26.611 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-11-24T06:12:26.611 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow *", 2023-11-24T06:12:26.611 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow *", 2023-11-24T06:12:26.611 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-11-24T06:12:26.611 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-24T06:12:26.611 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-24T06:12:26.611 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-24T06:12:26.611 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "mds.c", 2023-11-24T06:12:26.611 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQAQPmBl4VbbGBAAkJwrHXAC4d5S2hdPIcE/Cg==", 2023-11-24T06:12:26.611 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-11-24T06:12:26.612 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-11-24T06:12:26.612 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow *", 2023-11-24T06:12:26.612 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow *", 2023-11-24T06:12:26.612 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-11-24T06:12:26.612 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-24T06:12:26.612 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-24T06:12:26.612 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-24T06:12:26.612 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "mds.d", 2023-11-24T06:12:26.612 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQARPmBl00MmABAAvqEvZ0bL8D1oHg4gcKXEyA==", 2023-11-24T06:12:26.612 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-11-24T06:12:26.612 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-11-24T06:12:26.612 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow *", 2023-11-24T06:12:26.613 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow *", 2023-11-24T06:12:26.613 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-11-24T06:12:26.613 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-24T06:12:26.613 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-24T06:12:26.613 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-24T06:12:26.613 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.0", 2023-11-24T06:12:26.613 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQAVPmBlr+4mLRAAgsX9kvv5frsFc+E1VoO8Fw==", 2023-11-24T06:12:26.613 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-11-24T06:12:26.613 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-11-24T06:12:26.613 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-11-24T06:12:26.613 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-11-24T06:12:26.613 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-24T06:12:26.613 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-24T06:12:26.614 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-24T06:12:26.614 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.1", 2023-11-24T06:12:26.614 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQAZPmBl7IUIChAA/DL645CltqHdzRS8r4Lksg==", 2023-11-24T06:12:26.614 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-11-24T06:12:26.614 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-11-24T06:12:26.614 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-11-24T06:12:26.614 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-11-24T06:12:26.614 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-24T06:12:26.614 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-24T06:12:26.614 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-24T06:12:26.614 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.2", 2023-11-24T06:12:26.614 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQAcPmBlJ5VbIhAAEnc+QX3whpwM0Ok9WD8dcA==", 2023-11-24T06:12:26.615 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-11-24T06:12:26.615 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-11-24T06:12:26.615 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-11-24T06:12:26.615 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-11-24T06:12:26.615 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-24T06:12:26.615 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-24T06:12:26.615 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-24T06:12:26.615 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.3", 2023-11-24T06:12:26.615 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQAfPmBlKdS9KxAAbdaBTd1CEf/A6RyKxc9t6A==", 2023-11-24T06:12:26.615 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-11-24T06:12:26.615 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-11-24T06:12:26.616 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-11-24T06:12:26.616 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-11-24T06:12:26.616 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-24T06:12:26.616 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-24T06:12:26.616 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-24T06:12:26.616 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.4", 2023-11-24T06:12:26.616 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQAkPmBlnOJ6KxAAuLTyFpOBStxC5FevOc4pKw==", 2023-11-24T06:12:26.616 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-11-24T06:12:26.616 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-11-24T06:12:26.616 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-11-24T06:12:26.616 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-11-24T06:12:26.616 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-24T06:12:26.617 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-24T06:12:26.617 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-24T06:12:26.617 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.5", 2023-11-24T06:12:26.617 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQAoPmBlrkDXCBAAwv3u79yJeeTjreYWfr/yvQ==", 2023-11-24T06:12:26.617 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-11-24T06:12:26.617 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-11-24T06:12:26.617 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-11-24T06:12:26.617 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-11-24T06:12:26.617 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-24T06:12:26.617 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-24T06:12:26.617 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-24T06:12:26.617 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.6", 2023-11-24T06:12:26.617 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQArPmBlP1AoIRAA8D8nUCwXZRHZmB9d73fYbQ==", 2023-11-24T06:12:26.617 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-11-24T06:12:26.618 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-11-24T06:12:26.618 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-11-24T06:12:26.618 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-11-24T06:12:26.618 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-24T06:12:26.618 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-24T06:12:26.618 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-24T06:12:26.618 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.7", 2023-11-24T06:12:26.618 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQAuPmBlUBKMKRAA3obYjr9s13SuYnbFt+Y0bQ==", 2023-11-24T06:12:26.618 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-11-24T06:12:26.618 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-11-24T06:12:26.618 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-11-24T06:12:26.618 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-11-24T06:12:26.619 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-24T06:12:26.619 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-24T06:12:26.619 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-24T06:12:26.619 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "client.0", 2023-11-24T06:12:26.619 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQARPmBlGMUsDhAAlgrJvA/TT5hpUYd5o9OzIQ==", 2023-11-24T06:12:26.619 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-11-24T06:12:26.619 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-11-24T06:12:26.619 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow r", 2023-11-24T06:12:26.619 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-11-24T06:12:26.619 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-24T06:12:26.620 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-24T06:12:26.620 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-24T06:12:26.620 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "client.1", 2023-11-24T06:12:26.620 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQARPmBlOeviIxAA/egv9DYEDJ+1yKDFbsY1pw==", 2023-11-24T06:12:26.620 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-11-24T06:12:26.620 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-11-24T06:12:26.620 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow r", 2023-11-24T06:12:26.620 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow rw", 2023-11-24T06:12:26.620 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow rwx" 2023-11-24T06:12:26.620 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-24T06:12:26.620 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-24T06:12:26.620 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-24T06:12:26.621 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "client.2", 2023-11-24T06:12:26.621 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQARPmBlIqJ9GBAA46UmMl1hl/522waZSndXhw==", 2023-11-24T06:12:26.621 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-11-24T06:12:26.621 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-11-24T06:12:26.621 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow r", 2023-11-24T06:12:26.621 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow rw", 2023-11-24T06:12:26.621 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow rwx" 2023-11-24T06:12:26.621 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-24T06:12:26.621 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-24T06:12:26.621 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-24T06:12:26.622 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "client.3", 2023-11-24T06:12:26.622 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQARPmBlQRCDLhAA80Oc0yKHPMD5EjVxqurfPw==", 2023-11-24T06:12:26.622 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-11-24T06:12:26.622 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-11-24T06:12:26.622 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow r", 2023-11-24T06:12:26.622 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow rw", 2023-11-24T06:12:26.622 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow rwx" 2023-11-24T06:12:26.622 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-24T06:12:26.622 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-24T06:12:26.622 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-24T06:12:26.622 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "client.admin", 2023-11-24T06:12:26.623 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQAOPmBl/8h5MxAAs//puGUeKqNyBdlTaWY/1w==", 2023-11-24T06:12:26.623 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-11-24T06:12:26.623 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow *", 2023-11-24T06:12:26.623 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow *", 2023-11-24T06:12:26.623 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow *", 2023-11-24T06:12:26.623 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-11-24T06:12:26.623 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-24T06:12:26.623 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-24T06:12:26.623 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-24T06:12:26.623 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "mgr.x", 2023-11-24T06:12:26.623 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQAPPmBl1LwlNxAAsk8YjJh9jW0OCcGdfnBA1w==", 2023-11-24T06:12:26.623 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-11-24T06:12:26.623 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow *", 2023-11-24T06:12:26.624 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile mgr", 2023-11-24T06:12:26.624 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-11-24T06:12:26.624 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-24T06:12:26.624 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-24T06:12:26.624 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-24T06:12:26.624 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "mgr.y", 2023-11-24T06:12:26.624 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQAPPmBlyGxUKhAAWOTBKusZ4JOcJ1SaHJ6ZmQ==", 2023-11-24T06:12:26.624 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-11-24T06:12:26.624 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow *", 2023-11-24T06:12:26.624 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile mgr", 2023-11-24T06:12:26.624 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-11-24T06:12:26.624 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-24T06:12:26.625 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-24T06:12:26.625 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-24T06:12:26.625 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-24T06:12:26.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:26.608+0000 7f67a8f5a640 1 -- 172.21.15.92:0/3884990174 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f678406ae40 msgr2=0x7f678406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:26.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:26.608+0000 7f67a8f5a640 1 --2- 172.21.15.92:0/3884990174 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f678406ae40 0x7f678406d300 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f6798006fd0 tx=0x7f6798008040 comp rx=0 tx=0).stop 2023-11-24T06:12:26.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:26.608+0000 7f67a8f5a640 1 -- 172.21.15.92:0/3884990174 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f67a40b2bd0 msgr2=0x7f67a414d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:26.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:26.608+0000 7f67a8f5a640 1 --2- 172.21.15.92:0/3884990174 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f67a40b2bd0 0x7f67a414d710 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f67940097c0 tx=0x7f67940046c0 comp rx=0 tx=0).stop 2023-11-24T06:12:26.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:26.608+0000 7f67a8f5a640 1 -- 172.21.15.92:0/3884990174 shutdown_connections 2023-11-24T06:12:26.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:26.608+0000 7f67a8f5a640 1 --2- 172.21.15.92:0/3884990174 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f678406ae40 0x7f678406d300 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:26.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:26.608+0000 7f67a8f5a640 1 --2- 172.21.15.92:0/3884990174 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f67a415d300 0x7f67a414e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:26.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:26.608+0000 7f67a8f5a640 1 --2- 172.21.15.92:0/3884990174 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f67a4150760 0x7f67a41588f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:26.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:26.608+0000 7f67a8f5a640 1 --2- 172.21.15.92:0/3884990174 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f67a40b2bd0 0x7f67a414d710 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:26.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:26.608+0000 7f67a8f5a640 1 -- 172.21.15.92:0/3884990174 >> 172.21.15.92:0/3884990174 conn(0x7f67a406a800 msgr2=0x7f67a40b1fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:26.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:26.609+0000 7f67a8f5a640 1 -- 172.21.15.92:0/3884990174 shutdown_connections 2023-11-24T06:12:26.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:26.609+0000 7f67a8f5a640 1 -- 172.21.15.92:0/3884990174 wait complete. 2023-11-24T06:12:26.631 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:12:26.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:26.840+0000 7fa699343640 1 Processor -- start 2023-11-24T06:12:26.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:26.840+0000 7fa699343640 1 -- start start 2023-11-24T06:12:26.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:26.842+0000 7fa699343640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fa694143ae0 0x7fa6940cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:26.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:26.842+0000 7fa699343640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fa69415e3b0 0x7fa6940702c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:26.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:26.842+0000 7fa699343640 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6940cfc30 con 0x7fa6940cf7d0 2023-11-24T06:12:26.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:26.842+0000 7fa699343640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6940b1ee0 con 0x7fa69415e3b0 2023-11-24T06:12:26.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:26.842+0000 7fa699343640 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6940b2780 con 0x7fa694143ae0 2023-11-24T06:12:26.846 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:26.842+0000 7fa6937fe640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fa69415e3b0 0x7fa6940702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:26.846 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:26.842+0000 7fa6937fe640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fa69415e3b0 0x7fa6940702c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43452/0 (socket says 172.21.15.92:43452) 2023-11-24T06:12:26.846 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:26.842+0000 7fa6937fe640 1 -- 172.21.15.92:0/3956257389 learned_addr learned my addr 172.21.15.92:0/3956257389 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:12:26.846 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:26.842+0000 7fa692ffd640 1 --2- 172.21.15.92:0/3956257389 >> v2:172.21.15.118:3300/0 conn(0x7fa694143ae0 0x7fa6940cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:26.846 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:26.843+0000 7fa6937fe640 1 -- 172.21.15.92:0/3956257389 >> v2:172.21.15.118:3300/0 conn(0x7fa694143ae0 msgr2=0x7fa6940cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:26.846 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:26.843+0000 7fa6937fe640 1 --2- 172.21.15.92:0/3956257389 >> v2:172.21.15.118:3300/0 conn(0x7fa694143ae0 0x7fa6940cf290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:26.846 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:26.843+0000 7fa6937fe640 1 -- 172.21.15.92:0/3956257389 >> v1:172.21.15.118:6789/0 conn(0x7fa6940cf7d0 legacy=0x7fa69415dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:26.846 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:26.843+0000 7fa6937fe640 1 -- 172.21.15.92:0/3956257389 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa69415e990 con 0x7fa69415e3b0 2023-11-24T06:12:26.847 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:26.844+0000 7fa6937fe640 1 --2- 172.21.15.92:0/3956257389 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa69415e3b0 0x7fa6940702c0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fa6940cfea0 tx=0x7fa68c02e0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88f997ca50d4966f server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:26.847 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:26.844+0000 7fa691ffb640 1 -- 172.21.15.92:0/3956257389 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa68c02ec90 con 0x7fa69415e3b0 2023-11-24T06:12:26.847 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:26.844+0000 7fa691ffb640 1 -- 172.21.15.92:0/3956257389 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa68c02edf0 con 0x7fa69415e3b0 2023-11-24T06:12:26.847 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:26.844+0000 7fa691ffb640 1 -- 172.21.15.92:0/3956257389 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa68c036610 con 0x7fa69415e3b0 2023-11-24T06:12:26.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:26.845+0000 7fa699343640 1 -- 172.21.15.92:0/3956257389 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa69415e3b0 msgr2=0x7fa6940702c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:26.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:26.845+0000 7fa699343640 1 --2- 172.21.15.92:0/3956257389 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa69415e3b0 0x7fa6940702c0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fa6940cfea0 tx=0x7fa68c02e0f0 comp rx=0 tx=0).stop 2023-11-24T06:12:26.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:26.845+0000 7fa699343640 1 -- 172.21.15.92:0/3956257389 shutdown_connections 2023-11-24T06:12:26.849 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:26.845+0000 7fa699343640 1 --2- 172.21.15.92:0/3956257389 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa69415e3b0 0x7fa6940702c0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:26.849 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:26.845+0000 7fa699343640 1 --2- 172.21.15.92:0/3956257389 >> v2:172.21.15.118:3300/0 conn(0x7fa694143ae0 0x7fa6940cf290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:26.849 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:26.845+0000 7fa699343640 1 -- 172.21.15.92:0/3956257389 >> 172.21.15.92:0/3956257389 conn(0x7fa69406a800 msgr2=0x7fa69406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:26.849 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:26.845+0000 7fa699343640 1 -- 172.21.15.92:0/3956257389 shutdown_connections 2023-11-24T06:12:26.849 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:26.845+0000 7fa699343640 1 -- 172.21.15.92:0/3956257389 wait complete. 2023-11-24T06:12:26.850 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:26.847+0000 7fa699343640 1 Processor -- start 2023-11-24T06:12:26.850 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:26.847+0000 7fa699343640 1 -- start start 2023-11-24T06:12:26.852 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:26.848+0000 7fa699343640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa6940cf7d0 0x7fa6940adb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:26.852 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:26.848+0000 7fa699343640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa694143ae0 0x7fa694140af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:26.852 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:26.848+0000 7fa699343640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa69415e3b0 0x7fa694141030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:26.852 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:26.848+0000 7fa699343640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa69413f140 con 0x7fa6940cf7d0 2023-11-24T06:12:26.852 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:26.848+0000 7fa699343640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa69413ffa0 con 0x7fa69415e3b0 2023-11-24T06:12:26.852 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:26.848+0000 7fa699343640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa69413f390 con 0x7fa694143ae0 2023-11-24T06:12:26.852 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:26.848+0000 7fa692ffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa6940cf7d0 0x7fa6940adb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:26.852 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:26.848+0000 7fa692ffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa6940cf7d0 0x7fa6940adb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43454/0 (socket says 172.21.15.92:43454) 2023-11-24T06:12:26.852 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:26.848+0000 7fa692ffd640 1 -- 172.21.15.92:0/1745020838 learned_addr learned my addr 172.21.15.92:0/1745020838 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:12:26.852 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:26.848+0000 7fa6927fc640 1 --2- 172.21.15.92:0/1745020838 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa694143ae0 0x7fa694140af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:26.852 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:26.848+0000 7fa692ffd640 1 -- 172.21.15.92:0/1745020838 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa694143ae0 msgr2=0x7fa694140af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:26.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:26.848+0000 7fa692ffd640 1 --2- 172.21.15.92:0/1745020838 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa694143ae0 0x7fa694140af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:26.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:26.848+0000 7fa692ffd640 1 -- 172.21.15.92:0/1745020838 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa69415e3b0 msgr2=0x7fa694141030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:26.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:26.848+0000 7fa6937fe640 1 --2- 172.21.15.92:0/1745020838 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa69415e3b0 0x7fa694141030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:26.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:26.848+0000 7fa692ffd640 1 --2- 172.21.15.92:0/1745020838 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa69415e3b0 0x7fa694141030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:26.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:26.848+0000 7fa692ffd640 1 -- 172.21.15.92:0/1745020838 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa68c009b30 con 0x7fa6940cf7d0 2023-11-24T06:12:26.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:26.849+0000 7fa692ffd640 1 --2- 172.21.15.92:0/1745020838 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa6940cf7d0 0x7fa6940adb90 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fa68400d800 tx=0x7fa68400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:26.854 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:26.849+0000 7fa67b7fe640 1 -- 172.21.15.92:0/1745020838 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa68400b6c0 con 0x7fa6940cf7d0 2023-11-24T06:12:26.854 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:26.849+0000 7fa67b7fe640 1 -- 172.21.15.92:0/1745020838 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa68400b820 con 0x7fa6940cf7d0 2023-11-24T06:12:26.854 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:26.849+0000 7fa67b7fe640 1 -- 172.21.15.92:0/1745020838 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa68400bbb0 con 0x7fa6940cf7d0 2023-11-24T06:12:26.854 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:26.849+0000 7fa699343640 1 -- 172.21.15.92:0/1745020838 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa69413f670 con 0x7fa6940cf7d0 2023-11-24T06:12:26.854 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:26.849+0000 7fa699343640 1 -- 172.21.15.92:0/1745020838 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa69413fc30 con 0x7fa6940cf7d0 2023-11-24T06:12:26.854 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:26.850+0000 7fa699343640 1 -- 172.21.15.92:0/1745020838 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa69413fdc0 con 0x7fa6940cf7d0 2023-11-24T06:12:26.854 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:26.851+0000 7fa67b7fe640 1 -- 172.21.15.92:0/1745020838 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa68400bd10 con 0x7fa6940cf7d0 2023-11-24T06:12:26.856 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:26.852+0000 7fa67b7fe640 1 --2- 172.21.15.92:0/1745020838 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fa68006ae40 0x7fa68006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:26.856 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:26.853+0000 7fa67b7fe640 1 -- 172.21.15.92:0/1745020838 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fa684089900 con 0x7fa6940cf7d0 2023-11-24T06:12:26.856 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:26.853+0000 7fa6927fc640 1 --2- 172.21.15.92:0/1745020838 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fa68006ae40 0x7fa68006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:26.856 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:26.853+0000 7fa6927fc640 1 --2- 172.21.15.92:0/1745020838 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fa68006ae40 0x7fa68006d300 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fa694140350 tx=0x7fa688009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:26.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:26.857+0000 7fa67b7fe640 1 -- 172.21.15.92:0/1745020838 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa684054c60 con 0x7fa6940cf7d0 2023-11-24T06:12:27.051 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:27.048+0000 7fa699343640 1 -- 172.21.15.92:0/1745020838 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa6940675d0 con 0x7fa6940cf7d0 2023-11-24T06:12:27.053 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:27.050+0000 7fa67b7fe640 1 -- 172.21.15.92:0/1745020838 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v48) v1 ==== 74+0+12809 (secure 0 0 0) 0x7fa684058280 con 0x7fa6940cf7d0 2023-11-24T06:12:27.054 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-24T06:12:27.054 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":48,"fsid":"2d6dee5f-8aa3-470f-931a-50ade963b1cb","created":"2023-11-24T06:10:04.707204+0000","modified":"2023-11-24T06:12:26.097755+0000","last_up_change":"2023-11-24T06:10:17.598352+0000","last_in_change":"2023-11-24T06:10:09.040793+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:10:17.733098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ff37a9fb-135f-471f-b4b0-f7b5772006c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6803","nonce":20560714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6805","nonce":20560714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6809","nonce":20560714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6807","nonce":20560714}]},"public_addr":"172.21.15.92:6803/20560714","cluster_addr":"172.21.15.92:6805/20560714","heartbeat_back_addr":"172.21.15.92:6809/20560714","heartbeat_front_addr":"172.21.15.92:6807/20560714","state":["exists","up"]},{"osd":1,"uuid":"03cdd353-dcf1-4f38-94fe-a795137c06b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6819","nonce":2451239906}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6821","nonce":2451239906}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6825","nonce":2451239906}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6823","nonce":2451239906}]},"public_addr":"172.21.15.92:6819/2451239906","cluster_addr":"172.21.15.92:6821/2451239906","heartbeat_back_addr":"172.21.15.92:6825/2451239906","heartbeat_front_addr":"172.21.15.92:6823/2451239906","state":["exists","up"]},{"osd":2,"uuid":"a4a20d80-5d57-484b-becd-9aa8070dd3aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6811","nonce":1124979277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6813","nonce":1124979277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6817","nonce":1124979277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6815","nonce":1124979277}]},"public_addr":"172.21.15.92:6811/1124979277","cluster_addr":"172.21.15.92:6813/1124979277","heartbeat_back_addr":"172.21.15.92:6817/1124979277","heartbeat_front_addr":"172.21.15.92:6815/1124979277","state":["exists","up"]},{"osd":3,"uuid":"4648d7ce-9b29-4583-9425-4ba8fd2b1903","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6827","nonce":1734115782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6829","nonce":1734115782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6833","nonce":1734115782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6831","nonce":1734115782}]},"public_addr":"172.21.15.92:6827/1734115782","cluster_addr":"172.21.15.92:6829/1734115782","heartbeat_back_addr":"172.21.15.92:6833/1734115782","heartbeat_front_addr":"172.21.15.92:6831/1734115782","state":["exists","up"]},{"osd":4,"uuid":"3bc9d7eb-c6fa-4fcf-a3ec-ade604e05aac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6819","nonce":3425937229}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6822","nonce":3425937229}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6830","nonce":3425937229}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6826","nonce":3425937229}]},"public_addr":"172.21.15.118:6819/3425937229","cluster_addr":"172.21.15.118:6822/3425937229","heartbeat_back_addr":"172.21.15.118:6830/3425937229","heartbeat_front_addr":"172.21.15.118:6826/3425937229","state":["exists","up"]},{"osd":5,"uuid":"dce9ba6d-59ed-46d1-84fc-a84b13de10c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6801","nonce":827455984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6803","nonce":827455984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6807","nonce":827455984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6805","nonce":827455984}]},"public_addr":"172.21.15.118:6801/827455984","cluster_addr":"172.21.15.118:6803/827455984","heartbeat_back_addr":"172.21.15.118:6807/827455984","heartbeat_front_addr":"172.21.15.118:6805/827455984","state":["exists","up"]},{"osd":6,"uuid":"82aec504-ae58-4709-8d30-46f2b12081b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6809","nonce":2946986141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6811","nonce":2946986141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6815","nonce":2946986141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6813","nonce":2946986141}]},"public_addr":"172.21.15.118:6809/2946986141","cluster_addr":"172.21.15.118:6811/2946986141","heartbeat_back_addr":"172.21.15.118:6815/2946986141","heartbeat_front_addr":"172.21.15.118:6813/2946986141","state":["exists","up"]},{"osd":7,"uuid":"4ed4c624-c88f-4d35-bdb1-bf209b5dffb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6817","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6818","nonce":48122059}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6821","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6823","nonce":48122059}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6829","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6831","nonce":48122059}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6825","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6827","nonce":48122059}]},"public_addr":"172.21.15.118:6818/48122059","cluster_addr":"172.21.15.118:6823/48122059","heartbeat_back_addr":"172.21.15.118:6831/48122059","heartbeat_front_addr":"172.21.15.118:6827/48122059","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.536664+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.368324+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.271201+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.342971+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.621832+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.387110+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.632946+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.416032+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:12:27.056 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:27.053+0000 7fa699343640 1 -- 172.21.15.92:0/1745020838 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fa68006ae40 msgr2=0x7fa68006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:27.056 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:27.053+0000 7fa699343640 1 --2- 172.21.15.92:0/1745020838 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fa68006ae40 0x7fa68006d300 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fa694140350 tx=0x7fa688009040 comp rx=0 tx=0).stop 2023-11-24T06:12:27.056 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:27.053+0000 7fa699343640 1 -- 172.21.15.92:0/1745020838 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa6940cf7d0 msgr2=0x7fa6940adb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:27.056 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:27.053+0000 7fa699343640 1 --2- 172.21.15.92:0/1745020838 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa6940cf7d0 0x7fa6940adb90 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fa68400d800 tx=0x7fa68400dcc0 comp rx=0 tx=0).stop 2023-11-24T06:12:27.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:27.053+0000 7fa699343640 1 -- 172.21.15.92:0/1745020838 shutdown_connections 2023-11-24T06:12:27.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:27.053+0000 7fa699343640 1 --2- 172.21.15.92:0/1745020838 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fa68006ae40 0x7fa68006d300 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:27.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:27.053+0000 7fa699343640 1 --2- 172.21.15.92:0/1745020838 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa69415e3b0 0x7fa694141030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:27.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:27.053+0000 7fa699343640 1 --2- 172.21.15.92:0/1745020838 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa694143ae0 0x7fa694140af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:27.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:27.053+0000 7fa699343640 1 --2- 172.21.15.92:0/1745020838 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa6940cf7d0 0x7fa6940adb90 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:27.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:27.053+0000 7fa699343640 1 -- 172.21.15.92:0/1745020838 >> 172.21.15.92:0/1745020838 conn(0x7fa69406a800 msgr2=0x7fa69415b870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:27.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:27.053+0000 7fa699343640 1 -- 172.21.15.92:0/1745020838 shutdown_connections 2023-11-24T06:12:27.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:27.053+0000 7fa699343640 1 -- 172.21.15.92:0/1745020838 wait complete. 2023-11-24T06:12:27.074 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T06:10:17.733098+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T06:12:27.074 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T06:12:27.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:27.272+0000 7f990f446640 1 Processor -- start 2023-11-24T06:12:27.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:27.272+0000 7f990f446640 1 -- start start 2023-11-24T06:12:27.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:27.273+0000 7f990f446640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f99080cf9a0 0x7f9908166030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:27.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:27.273+0000 7f990f446640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f990815c5f0 con 0x7f990813df80 2023-11-24T06:12:27.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:27.273+0000 7f990f446640 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f990806c470 con 0x7f990813e360 2023-11-24T06:12:27.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:27.273+0000 7f990f446640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f990806c750 con 0x7f99080cf9a0 2023-11-24T06:12:27.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:27.274+0000 7f990d1e2640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f990813df80 0x7f990815bee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:39604/0 (socket says 172.21.15.92:39604) 2023-11-24T06:12:27.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:27.274+0000 7f990d1e2640 1 -- 172.21.15.92:0/751805522 learned_addr learned my addr 172.21.15.92:0/751805522 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-24T06:12:27.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:27.274+0000 7f990d9e3640 1 --2- 172.21.15.92:0/751805522 >> v2:172.21.15.92:3300/0 conn(0x7f99080cf9a0 0x7f9908166030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:27.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:27.274+0000 7f990d9e3640 1 -- 172.21.15.92:0/751805522 >> v1:172.21.15.118:6789/0 conn(0x7f990813e360 legacy=0x7f99080cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:27.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:27.274+0000 7f990d9e3640 1 -- 172.21.15.92:0/751805522 >> v1:172.21.15.92:6789/0 conn(0x7f990813df80 legacy=0x7f990815bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:27.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:27.274+0000 7f98fffff640 1 -- 172.21.15.92:0/751805522 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2140468101 0 0) 0x7f990815c5f0 con 0x7f990813df80 2023-11-24T06:12:27.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:27.274+0000 7f990d9e3640 1 -- 172.21.15.92:0/751805522 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f990806ca30 con 0x7f99080cf9a0 2023-11-24T06:12:27.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:27.275+0000 7f990d9e3640 1 --2- 172.21.15.92:0/751805522 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f99080cf9a0 0x7f9908166030 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f9900009d90 tx=0x7f990002beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f1f1fcf6c48cf3e server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:27.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:27.275+0000 7f98fffff640 1 -- 172.21.15.92:0/751805522 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f990002cc60 con 0x7f99080cf9a0 2023-11-24T06:12:27.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:27.275+0000 7f98fffff640 1 -- 172.21.15.92:0/751805522 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f990002cdc0 con 0x7f99080cf9a0 2023-11-24T06:12:27.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:27.275+0000 7f98fffff640 1 -- 172.21.15.92:0/751805522 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9900034660 con 0x7f99080cf9a0 2023-11-24T06:12:27.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:27.276+0000 7f990f446640 1 -- 172.21.15.92:0/751805522 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f99080cf9a0 msgr2=0x7f9908166030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:27.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:27.276+0000 7f990f446640 1 --2- 172.21.15.92:0/751805522 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f99080cf9a0 0x7f9908166030 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f9900009d90 tx=0x7f990002beb0 comp rx=0 tx=0).stop 2023-11-24T06:12:27.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:27.276+0000 7f990f446640 1 -- 172.21.15.92:0/751805522 shutdown_connections 2023-11-24T06:12:27.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:27.276+0000 7f990f446640 1 --2- 172.21.15.92:0/751805522 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f99080cf9a0 0x7f9908166030 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:27.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:27.276+0000 7f990f446640 1 -- 172.21.15.92:0/751805522 >> 172.21.15.92:0/751805522 conn(0x7f990806a800 msgr2=0x7f990806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:27.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:27.276+0000 7f990f446640 1 -- 172.21.15.92:0/751805522 shutdown_connections 2023-11-24T06:12:27.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:27.276+0000 7f990f446640 1 -- 172.21.15.92:0/751805522 wait complete. 2023-11-24T06:12:27.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:27.278+0000 7f990f446640 1 Processor -- start 2023-11-24T06:12:27.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:27.278+0000 7f990f446640 1 -- start start 2023-11-24T06:12:27.283 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:27.279+0000 7f990f446640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f99080cf9a0 0x7f9908150c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:27.283 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:27.279+0000 7f990f446640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f990813df80 0x7f990814d240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:27.283 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:27.279+0000 7f990f446640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f990813e360 0x7f99080b2f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:27.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:27.279+0000 7f990f446640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f990814d780 con 0x7f990813df80 2023-11-24T06:12:27.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:27.279+0000 7f990f446640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f990814cdb0 con 0x7f99080cf9a0 2023-11-24T06:12:27.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:27.279+0000 7f990f446640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f990814f790 con 0x7f990813e360 2023-11-24T06:12:27.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:27.279+0000 7f990c9e1640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f990813df80 0x7f990814d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:27.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:27.279+0000 7f990c9e1640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f990813df80 0x7f990814d240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43472/0 (socket says 172.21.15.92:43472) 2023-11-24T06:12:27.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:27.279+0000 7f990c9e1640 1 -- 172.21.15.92:0/1502124952 learned_addr learned my addr 172.21.15.92:0/1502124952 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:12:27.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:27.279+0000 7f990d1e2640 1 --2- 172.21.15.92:0/1502124952 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f99080cf9a0 0x7f9908150c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:27.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:27.279+0000 7f990d9e3640 1 --2- 172.21.15.92:0/1502124952 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f990813e360 0x7f99080b2f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:27.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:27.279+0000 7f990c9e1640 1 -- 172.21.15.92:0/1502124952 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f990813e360 msgr2=0x7f99080b2f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:27.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:27.279+0000 7f990c9e1640 1 --2- 172.21.15.92:0/1502124952 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f990813e360 0x7f99080b2f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:27.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:27.280+0000 7f990c9e1640 1 -- 172.21.15.92:0/1502124952 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f99080cf9a0 msgr2=0x7f9908150c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:27.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:27.280+0000 7f990c9e1640 1 --2- 172.21.15.92:0/1502124952 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f99080cf9a0 0x7f9908150c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:27.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:27.280+0000 7f990c9e1640 1 -- 172.21.15.92:0/1502124952 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9900009a70 con 0x7f990813df80 2023-11-24T06:12:27.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:27.280+0000 7f990c9e1640 1 --2- 172.21.15.92:0/1502124952 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f990813df80 0x7f990814d240 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f98f800c400 tx=0x7f98f800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:27.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:27.280+0000 7f98fdffb640 1 -- 172.21.15.92:0/1502124952 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98f8011790 con 0x7f990813df80 2023-11-24T06:12:27.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:27.280+0000 7f98fdffb640 1 -- 172.21.15.92:0/1502124952 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98f8011930 con 0x7f990813df80 2023-11-24T06:12:27.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:27.280+0000 7f990f446640 1 -- 172.21.15.92:0/1502124952 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f990814c1a0 con 0x7f990813df80 2023-11-24T06:12:27.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:27.280+0000 7f98fdffb640 1 -- 172.21.15.92:0/1502124952 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98f8011ce0 con 0x7f990813df80 2023-11-24T06:12:27.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:27.280+0000 7f990f446640 1 -- 172.21.15.92:0/1502124952 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f990814c760 con 0x7f990813df80 2023-11-24T06:12:27.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:27.281+0000 7f990f446640 1 -- 172.21.15.92:0/1502124952 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f990814cf20 con 0x7f990813df80 2023-11-24T06:12:27.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:27.282+0000 7f98fdffb640 1 -- 172.21.15.92:0/1502124952 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f98f801c460 con 0x7f990813df80 2023-11-24T06:12:27.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:27.283+0000 7f98fdffb640 1 --2- 172.21.15.92:0/1502124952 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f98ec06ae40 0x7f98ec06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:27.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:27.283+0000 7f98fdffb640 1 -- 172.21.15.92:0/1502124952 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f98f80892e0 con 0x7f990813df80 2023-11-24T06:12:27.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:27.283+0000 7f990d1e2640 1 --2- 172.21.15.92:0/1502124952 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f98ec06ae40 0x7f98ec06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:27.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:27.284+0000 7f990d1e2640 1 --2- 172.21.15.92:0/1502124952 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f98ec06ae40 0x7f98ec06d300 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f9904000f60 tx=0x7f9904009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:27.290 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:27.287+0000 7f98fdffb640 1 -- 172.21.15.92:0/1502124952 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f98f8054640 con 0x7f990813df80 2023-11-24T06:12:27.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:27.464+0000 7f990f446640 1 -- 172.21.15.92:0/1502124952 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f99080675d0 con 0x7f990813df80 2023-11-24T06:12:27.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:27.465+0000 7f98fdffb640 1 -- 172.21.15.92:0/1502124952 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v48) v1 ==== 93+0+10 (secure 0 0 0) 0x7f98f8057c60 con 0x7f990813df80 2023-11-24T06:12:27.468 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 1 2023-11-24T06:12:27.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:27.467+0000 7f990f446640 1 -- 172.21.15.92:0/1502124952 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f98ec06ae40 msgr2=0x7f98ec06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:27.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:27.467+0000 7f990f446640 1 --2- 172.21.15.92:0/1502124952 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f98ec06ae40 0x7f98ec06d300 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f9904000f60 tx=0x7f9904009040 comp rx=0 tx=0).stop 2023-11-24T06:12:27.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:27.467+0000 7f990f446640 1 -- 172.21.15.92:0/1502124952 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f990813df80 msgr2=0x7f990814d240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:27.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:27.467+0000 7f990f446640 1 --2- 172.21.15.92:0/1502124952 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f990813df80 0x7f990814d240 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f98f800c400 tx=0x7f98f800c8c0 comp rx=0 tx=0).stop 2023-11-24T06:12:27.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:27.468+0000 7f990f446640 1 -- 172.21.15.92:0/1502124952 shutdown_connections 2023-11-24T06:12:27.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:27.468+0000 7f990f446640 1 --2- 172.21.15.92:0/1502124952 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f98ec06ae40 0x7f98ec06d300 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:27.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:27.468+0000 7f990f446640 1 --2- 172.21.15.92:0/1502124952 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f990813e360 0x7f99080b2f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:27.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:27.468+0000 7f990f446640 1 --2- 172.21.15.92:0/1502124952 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f990813df80 0x7f990814d240 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:27.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:27.468+0000 7f990f446640 1 --2- 172.21.15.92:0/1502124952 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f99080cf9a0 0x7f9908150c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:27.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:27.468+0000 7f990f446640 1 -- 172.21.15.92:0/1502124952 >> 172.21.15.92:0/1502124952 conn(0x7f990806a800 msgr2=0x7f99080cd840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:27.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:27.468+0000 7f990f446640 1 -- 172.21.15.92:0/1502124952 shutdown_connections 2023-11-24T06:12:27.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:27.468+0000 7f990f446640 1 -- 172.21.15.92:0/1502124952 wait complete. 2023-11-24T06:12:27.489 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-11-24T06:12:27.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:27.688+0000 7effa1c7c640 1 Processor -- start 2023-11-24T06:12:27.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:27.688+0000 7effa1c7c640 1 -- start start 2023-11-24T06:12:27.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:27.689+0000 7effa1c7c640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7eff9c0cf9a0 0x7eff9c15dca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:27.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:27.689+0000 7effa1c7c640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7eff9c15e1e0 0x7eff9c0702c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:27.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:27.690+0000 7effa1c7c640 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff9c0b1ee0 con 0x7eff9c143ae0 2023-11-24T06:12:27.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:27.690+0000 7effa1c7c640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff9c0b2780 con 0x7eff9c15e1e0 2023-11-24T06:12:27.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:27.690+0000 7effa1c7c640 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff9c15e990 con 0x7eff9c0cf9a0 2023-11-24T06:12:27.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:27.690+0000 7eff9bfff640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7eff9c15e1e0 0x7eff9c0702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:27.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:27.690+0000 7eff9bfff640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7eff9c15e1e0 0x7eff9c0702c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43476/0 (socket says 172.21.15.92:43476) 2023-11-24T06:12:27.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:27.690+0000 7eff9bfff640 1 -- 172.21.15.92:0/3646269301 learned_addr learned my addr 172.21.15.92:0/3646269301 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:12:27.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:27.690+0000 7eff9affd640 1 --2- 172.21.15.92:0/3646269301 >> v2:172.21.15.118:3300/0 conn(0x7eff9c0cf9a0 0x7eff9c15dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:27.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:27.690+0000 7eff9bfff640 1 -- 172.21.15.92:0/3646269301 >> v2:172.21.15.118:3300/0 conn(0x7eff9c0cf9a0 msgr2=0x7eff9c15dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:27.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:27.690+0000 7eff9bfff640 1 --2- 172.21.15.92:0/3646269301 >> v2:172.21.15.118:3300/0 conn(0x7eff9c0cf9a0 0x7eff9c15dca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:27.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:27.690+0000 7eff9bfff640 1 -- 172.21.15.92:0/3646269301 >> v1:172.21.15.118:6789/0 conn(0x7eff9c143ae0 legacy=0x7eff9c0cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:27.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:27.690+0000 7eff9bfff640 1 -- 172.21.15.92:0/3646269301 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff9c0b2130 con 0x7eff9c15e1e0 2023-11-24T06:12:27.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:27.690+0000 7eff9affd640 1 --2- 172.21.15.92:0/3646269301 >> v2:172.21.15.118:3300/0 conn(0x7eff9c0cf9a0 0x7eff9c15dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:12:27.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:27.691+0000 7eff9bfff640 1 --2- 172.21.15.92:0/3646269301 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7eff9c15e1e0 0x7eff9c0702c0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7eff9c0cb830 tx=0x7eff9402bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcafc875be67c6f8 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:27.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:27.692+0000 7eff9a7fc640 1 -- 172.21.15.92:0/3646269301 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff9402ca80 con 0x7eff9c15e1e0 2023-11-24T06:12:27.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:27.692+0000 7eff9a7fc640 1 -- 172.21.15.92:0/3646269301 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff9402cbe0 con 0x7eff9c15e1e0 2023-11-24T06:12:27.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:27.692+0000 7eff9a7fc640 1 -- 172.21.15.92:0/3646269301 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff940343f0 con 0x7eff9c15e1e0 2023-11-24T06:12:27.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:27.692+0000 7effa1c7c640 1 -- 172.21.15.92:0/3646269301 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7eff9c15e1e0 msgr2=0x7eff9c0702c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:27.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:27.692+0000 7effa1c7c640 1 --2- 172.21.15.92:0/3646269301 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7eff9c15e1e0 0x7eff9c0702c0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7eff9c0cb830 tx=0x7eff9402bee0 comp rx=0 tx=0).stop 2023-11-24T06:12:27.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:27.692+0000 7effa1c7c640 1 -- 172.21.15.92:0/3646269301 shutdown_connections 2023-11-24T06:12:27.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:27.692+0000 7effa1c7c640 1 --2- 172.21.15.92:0/3646269301 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7eff9c15e1e0 0x7eff9c0702c0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:27.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:27.692+0000 7effa1c7c640 1 --2- 172.21.15.92:0/3646269301 >> v2:172.21.15.118:3300/0 conn(0x7eff9c0cf9a0 0x7eff9c15dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:27.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:27.692+0000 7effa1c7c640 1 -- 172.21.15.92:0/3646269301 >> 172.21.15.92:0/3646269301 conn(0x7eff9c06a800 msgr2=0x7eff9c06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:27.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:27.693+0000 7effa1c7c640 1 -- 172.21.15.92:0/3646269301 shutdown_connections 2023-11-24T06:12:27.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:27.693+0000 7effa1c7c640 1 -- 172.21.15.92:0/3646269301 wait complete. 2023-11-24T06:12:27.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:27.694+0000 7effa1c7c640 1 Processor -- start 2023-11-24T06:12:27.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:27.694+0000 7effa1c7c640 1 -- start start 2023-11-24T06:12:27.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:27.695+0000 7effa1c7c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7eff9c0cf9a0 0x7eff9c156bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:27.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:27.695+0000 7effa1c7c640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7eff9c143ae0 0x7eff9c13eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:27.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:27.695+0000 7effa1c7c640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7eff9c15e1e0 0x7eff9c13c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:27.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:27.695+0000 7effa1c7c640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff9c13f300 con 0x7eff9c0cf9a0 2023-11-24T06:12:27.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:27.695+0000 7effa1c7c640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff9c142e40 con 0x7eff9c15e1e0 2023-11-24T06:12:27.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:27.695+0000 7effa1c7c640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff9c1422b0 con 0x7eff9c143ae0 2023-11-24T06:12:27.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:27.695+0000 7eff9b7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7eff9c0cf9a0 0x7eff9c156bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:27.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:27.695+0000 7eff9b7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7eff9c0cf9a0 0x7eff9c156bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43488/0 (socket says 172.21.15.92:43488) 2023-11-24T06:12:27.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:27.695+0000 7eff9b7fe640 1 -- 172.21.15.92:0/1412794613 learned_addr learned my addr 172.21.15.92:0/1412794613 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:12:27.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:27.696+0000 7eff9b7fe640 1 -- 172.21.15.92:0/1412794613 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7eff9c143ae0 msgr2=0x7eff9c13eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:27.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:27.696+0000 7eff9b7fe640 1 --2- 172.21.15.92:0/1412794613 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7eff9c143ae0 0x7eff9c13eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:27.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:27.696+0000 7eff9b7fe640 1 -- 172.21.15.92:0/1412794613 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7eff9c15e1e0 msgr2=0x7eff9c13c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:27.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:27.696+0000 7eff9b7fe640 1 --2- 172.21.15.92:0/1412794613 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7eff9c15e1e0 0x7eff9c13c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:27.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:27.696+0000 7eff9b7fe640 1 -- 172.21.15.92:0/1412794613 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff94009b30 con 0x7eff9c0cf9a0 2023-11-24T06:12:27.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:27.696+0000 7eff9b7fe640 1 --2- 172.21.15.92:0/1412794613 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7eff9c0cf9a0 0x7eff9c156bf0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7eff8c00c410 tx=0x7eff8c00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:27.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:27.696+0000 7eff83fff640 1 -- 172.21.15.92:0/1412794613 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff8c0035f0 con 0x7eff9c0cf9a0 2023-11-24T06:12:27.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:27.696+0000 7eff83fff640 1 -- 172.21.15.92:0/1412794613 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff8c003750 con 0x7eff9c0cf9a0 2023-11-24T06:12:27.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:27.696+0000 7effa1c7c640 1 -- 172.21.15.92:0/1412794613 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff9c142550 con 0x7eff9c0cf9a0 2023-11-24T06:12:27.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:27.696+0000 7eff83fff640 1 -- 172.21.15.92:0/1412794613 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff8c003b00 con 0x7eff9c0cf9a0 2023-11-24T06:12:27.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:27.697+0000 7effa1c7c640 1 -- 172.21.15.92:0/1412794613 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff9c142ad0 con 0x7eff9c0cf9a0 2023-11-24T06:12:27.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:27.698+0000 7effa1c7c640 1 -- 172.21.15.92:0/1412794613 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff9c0702c0 con 0x7eff9c0cf9a0 2023-11-24T06:12:27.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:27.698+0000 7eff83fff640 1 -- 172.21.15.92:0/1412794613 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7eff8c003c60 con 0x7eff9c0cf9a0 2023-11-24T06:12:27.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:27.699+0000 7eff83fff640 1 --2- 172.21.15.92:0/1412794613 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7eff8806ae40 0x7eff8806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:27.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:27.699+0000 7eff83fff640 1 -- 172.21.15.92:0/1412794613 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 4857+0+0 (secure 0 0 0) 0x7eff8c0889a0 con 0x7eff9c0cf9a0 2023-11-24T06:12:27.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:27.699+0000 7eff9affd640 1 --2- 172.21.15.92:0/1412794613 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7eff8806ae40 0x7eff8806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:27.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:27.700+0000 7eff9affd640 1 --2- 172.21.15.92:0/1412794613 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7eff8806ae40 0x7eff8806d300 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7eff90006fd0 tx=0x7eff90008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:27.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:27.704+0000 7eff83fff640 1 -- 172.21.15.92:0/1412794613 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7eff8c053d00 con 0x7eff9c0cf9a0 2023-11-24T06:12:27.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:27.895+0000 7effa1c7c640 1 -- 172.21.15.92:0/1412794613 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7eff9c0719e0 con 0x7eff9c0cf9a0 2023-11-24T06:12:27.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:27.895+0000 7eff83fff640 1 -- 172.21.15.92:0/1412794613 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v48) v1 ==== 84+0+65 (secure 0 0 0) 0x7eff8c017020 con 0x7eff9c0cf9a0 2023-11-24T06:12:27.899 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-24T06:12:27.899 INFO:teuthology.orchestra.run.smithi092.stdout:[ 2023-11-24T06:12:27.899 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-24T06:12:27.899 INFO:teuthology.orchestra.run.smithi092.stdout: "poolnum": 1, 2023-11-24T06:12:27.899 INFO:teuthology.orchestra.run.smithi092.stdout: "poolname": ".mgr" 2023-11-24T06:12:27.899 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-24T06:12:27.899 INFO:teuthology.orchestra.run.smithi092.stdout:] 2023-11-24T06:12:27.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:27.898+0000 7effa1c7c640 1 -- 172.21.15.92:0/1412794613 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7eff8806ae40 msgr2=0x7eff8806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:27.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:27.898+0000 7effa1c7c640 1 --2- 172.21.15.92:0/1412794613 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7eff8806ae40 0x7eff8806d300 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7eff90006fd0 tx=0x7eff90008040 comp rx=0 tx=0).stop 2023-11-24T06:12:27.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:27.898+0000 7effa1c7c640 1 -- 172.21.15.92:0/1412794613 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7eff9c0cf9a0 msgr2=0x7eff9c156bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:27.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:27.898+0000 7effa1c7c640 1 --2- 172.21.15.92:0/1412794613 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7eff9c0cf9a0 0x7eff9c156bf0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7eff8c00c410 tx=0x7eff8c00c8d0 comp rx=0 tx=0).stop 2023-11-24T06:12:27.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:27.898+0000 7effa1c7c640 1 -- 172.21.15.92:0/1412794613 shutdown_connections 2023-11-24T06:12:27.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:27.898+0000 7effa1c7c640 1 --2- 172.21.15.92:0/1412794613 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7eff8806ae40 0x7eff8806d300 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:27.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:27.898+0000 7effa1c7c640 1 --2- 172.21.15.92:0/1412794613 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7eff9c15e1e0 0x7eff9c13c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:27.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:27.898+0000 7effa1c7c640 1 --2- 172.21.15.92:0/1412794613 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7eff9c143ae0 0x7eff9c13eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:27.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:27.898+0000 7effa1c7c640 1 --2- 172.21.15.92:0/1412794613 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7eff9c0cf9a0 0x7eff9c156bf0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:27.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:27.898+0000 7effa1c7c640 1 -- 172.21.15.92:0/1412794613 >> 172.21.15.92:0/1412794613 conn(0x7eff9c06a800 msgr2=0x7eff9c06abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:27.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:27.898+0000 7effa1c7c640 1 -- 172.21.15.92:0/1412794613 shutdown_connections 2023-11-24T06:12:27.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:27.898+0000 7effa1c7c640 1 -- 172.21.15.92:0/1412794613 wait complete. 2023-11-24T06:12:27.920 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-11-24T06:12:27.921 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-11-24T06:12:28.140 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:28.137+0000 7fcab297d640 1 Processor -- start 2023-11-24T06:12:28.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:28.137+0000 7fcab297d640 1 -- start start 2023-11-24T06:12:28.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:28.138+0000 7fcab297d640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fcaac160f80 0x7fcaac161360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:28.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:28.138+0000 7fcab297d640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fcaac0ced50 0x7fcaac0d02f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:28.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:28.138+0000 7fcab297d640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcaac1618a0 con 0x7fcaac0ce8f0 2023-11-24T06:12:28.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:28.138+0000 7fcab297d640 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcaac0cf1e0 con 0x7fcaac160f80 2023-11-24T06:12:28.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:28.138+0000 7fcab297d640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcaac0d0de0 con 0x7fcaac0ced50 2023-11-24T06:12:28.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:28.139+0000 7fcaab7fe640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fcaac0ce8f0 0x7fcaac0cfbe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:39612/0 (socket says 172.21.15.92:39612) 2023-11-24T06:12:28.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:28.139+0000 7fcaab7fe640 1 -- 172.21.15.92:0/2404914884 learned_addr learned my addr 172.21.15.92:0/2404914884 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-24T06:12:28.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:28.139+0000 7fcab0f1a640 1 --2- 172.21.15.92:0/2404914884 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fcaac0ced50 0x7fcaac0d02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:28.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:28.139+0000 7fcaabfff640 1 --2- 172.21.15.92:0/2404914884 >> v2:172.21.15.118:3300/0 conn(0x7fcaac160f80 0x7fcaac161360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:28.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:28.139+0000 7fcaaaffd640 1 -- 172.21.15.92:0/2404914884 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1799546461 0 0) 0x7fcaac1618a0 con 0x7fcaac0ce8f0 2023-11-24T06:12:28.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:28.139+0000 7fcaaaffd640 1 -- 172.21.15.92:0/2404914884 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fca8c003520 con 0x7fcaac0ce8f0 2023-11-24T06:12:28.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:28.139+0000 7fcaaaffd640 1 -- 172.21.15.92:0/2404914884 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 757224066 0 0) 0x7fca8c003520 con 0x7fcaac0ce8f0 2023-11-24T06:12:28.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:28.139+0000 7fcaaaffd640 1 -- 172.21.15.92:0/2404914884 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fcaac0ced50 msgr2=0x7fcaac0d02f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:28.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:28.139+0000 7fcaaaffd640 1 --2- 172.21.15.92:0/2404914884 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fcaac0ced50 0x7fcaac0d02f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:28.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:28.139+0000 7fcaaaffd640 1 -- 172.21.15.92:0/2404914884 >> v2:172.21.15.118:3300/0 conn(0x7fcaac160f80 msgr2=0x7fcaac161360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:28.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:28.139+0000 7fcaaaffd640 1 --2- 172.21.15.92:0/2404914884 >> v2:172.21.15.118:3300/0 conn(0x7fcaac160f80 0x7fcaac161360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:28.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:28.139+0000 7fcaaaffd640 1 -- 172.21.15.92:0/2404914884 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcaac0d10c0 con 0x7fcaac0ce8f0 2023-11-24T06:12:28.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:28.139+0000 7fcaaaffd640 1 -- 172.21.15.92:0/2404914884 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2607661350 0 0) 0x7fca940019f0 con 0x7fcaac0ce8f0 2023-11-24T06:12:28.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:28.139+0000 7fcaaaffd640 1 -- 172.21.15.92:0/2404914884 >> v1:172.21.15.92:6789/0 conn(0x7fcaac0ce8f0 legacy=0x7fcaac0cfbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:28.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:28.140+0000 7fcaaaffd640 1 --2- 172.21.15.92:0/2404914884 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fca8c003970 0x7fca8c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:28.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:28.140+0000 7fcaaaffd640 1 --2- 172.21.15.92:0/2404914884 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fcaac0ced50 0x7fca8c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:28.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:28.140+0000 7fcaaaffd640 1 --2- 172.21.15.92:0/2404914884 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fca8c004d00 0x7fca8c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:28.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:28.140+0000 7fcaaaffd640 1 -- 172.21.15.92:0/2404914884 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaac0cf1e0 con 0x7fca8c003970 2023-11-24T06:12:28.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:28.140+0000 7fcaaaffd640 1 -- 172.21.15.92:0/2404914884 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaac0d0de0 con 0x7fca8c004d00 2023-11-24T06:12:28.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:28.140+0000 7fcaaaffd640 1 -- 172.21.15.92:0/2404914884 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca8c00b9b0 con 0x7fcaac0ced50 2023-11-24T06:12:28.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:28.140+0000 7fcaabfff640 1 --2- 172.21.15.92:0/2404914884 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fca8c003970 0x7fca8c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:28.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:28.140+0000 7fcaab7fe640 1 --2- 172.21.15.92:0/2404914884 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fcaac0ced50 0x7fca8c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:28.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:28.140+0000 7fcaabfff640 1 -- 172.21.15.92:0/2404914884 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fcaac0ced50 msgr2=0x7fca8c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:28.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:28.140+0000 7fcaabfff640 1 --2- 172.21.15.92:0/2404914884 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fcaac0ced50 0x7fca8c0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:28.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:28.140+0000 7fcaabfff640 1 -- 172.21.15.92:0/2404914884 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fca8c004d00 msgr2=0x7fca8c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:28.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:28.140+0000 7fcab0f1a640 1 --2- 172.21.15.92:0/2404914884 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fca8c004d00 0x7fca8c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:28.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:28.140+0000 7fcaabfff640 1 --2- 172.21.15.92:0/2404914884 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fca8c004d00 0x7fca8c005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:28.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:28.140+0000 7fcaabfff640 1 -- 172.21.15.92:0/2404914884 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca8c003520 con 0x7fca8c003970 2023-11-24T06:12:28.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:28.141+0000 7fcaabfff640 1 --2- 172.21.15.92:0/2404914884 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fca8c003970 0x7fca8c003d90 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fca9c009ac0 tx=0x7fca8c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=866b18b7289d1f52 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:28.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:28.142+0000 7fcaaaffd640 1 -- 172.21.15.92:0/2404914884 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca9c02c710 con 0x7fca8c003970 2023-11-24T06:12:28.180 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:28.142+0000 7fcaaaffd640 1 -- 172.21.15.92:0/2404914884 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca9c02c870 con 0x7fca8c003970 2023-11-24T06:12:28.180 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:28.142+0000 7fcaaaffd640 1 -- 172.21.15.92:0/2404914884 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca9c02cc00 con 0x7fca8c003970 2023-11-24T06:12:28.180 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:28.142+0000 7fcab297d640 1 -- 172.21.15.92:0/2404914884 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fca8c003970 msgr2=0x7fca8c003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:28.180 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:28.142+0000 7fcab297d640 1 --2- 172.21.15.92:0/2404914884 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fca8c003970 0x7fca8c003d90 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fca9c009ac0 tx=0x7fca8c0057f0 comp rx=0 tx=0).stop 2023-11-24T06:12:28.180 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:28.142+0000 7fcab0f1a640 1 -- 172.21.15.92:0/2404914884 reap_dead start 2023-11-24T06:12:28.180 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:28.142+0000 7fcab297d640 1 -- 172.21.15.92:0/2404914884 shutdown_connections 2023-11-24T06:12:28.180 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:28.142+0000 7fcab297d640 1 -- 172.21.15.92:0/2404914884 >> 172.21.15.92:0/2404914884 conn(0x7fcaac09c550 msgr2=0x7fcaac0d3210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:28.180 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:28.142+0000 7fcab297d640 1 -- 172.21.15.92:0/2404914884 shutdown_connections 2023-11-24T06:12:28.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:28.143+0000 7fcab297d640 1 -- 172.21.15.92:0/2404914884 wait complete. 2023-11-24T06:12:28.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:28.144+0000 7fcab297d640 1 Processor -- start 2023-11-24T06:12:28.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:28.144+0000 7fcab297d640 1 -- start start 2023-11-24T06:12:28.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:28.145+0000 7fcab297d640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fca8c003970 0x7fcaac14dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:28.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:28.145+0000 7fcab297d640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fcaac1f40e0 0x7fcaac0ca2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:28.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:28.145+0000 7fcab297d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcaac0ca820 0x7fcaac0cc1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:28.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:28.145+0000 7fcab297d640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaac0cacd0 con 0x7fcaac0ca820 2023-11-24T06:12:28.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:28.145+0000 7fcab297d640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaac1f32e0 con 0x7fca8c003970 2023-11-24T06:12:28.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:28.145+0000 7fcab297d640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcaac14e520 con 0x7fcaac1f40e0 2023-11-24T06:12:28.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:28.146+0000 7fcab0f1a640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcaac0ca820 0x7fcaac0cc1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:28.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:28.146+0000 7fcab0f1a640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcaac0ca820 0x7fcaac0cc1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43510/0 (socket says 172.21.15.92:43510) 2023-11-24T06:12:28.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:28.146+0000 7fcab0f1a640 1 -- 172.21.15.92:0/425151187 learned_addr learned my addr 172.21.15.92:0/425151187 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:12:28.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:28.146+0000 7fcaabfff640 1 --2- 172.21.15.92:0/425151187 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fca8c003970 0x7fcaac14dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:28.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:28.146+0000 7fcaab7fe640 1 --2- 172.21.15.92:0/425151187 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fcaac1f40e0 0x7fcaac0ca2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:28.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:28.146+0000 7fcab0f1a640 1 -- 172.21.15.92:0/425151187 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fcaac1f40e0 msgr2=0x7fcaac0ca2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:28.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:28.146+0000 7fcab0f1a640 1 --2- 172.21.15.92:0/425151187 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fcaac1f40e0 0x7fcaac0ca2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:28.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:28.146+0000 7fcab0f1a640 1 -- 172.21.15.92:0/425151187 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fca8c003970 msgr2=0x7fcaac14dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:28.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:28.146+0000 7fcab0f1a640 1 --2- 172.21.15.92:0/425151187 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fca8c003970 0x7fcaac14dfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:28.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:28.146+0000 7fcab0f1a640 1 -- 172.21.15.92:0/425151187 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcaac1f3ef0 con 0x7fcaac0ca820 2023-11-24T06:12:28.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:28.146+0000 7fcaabfff640 1 --2- 172.21.15.92:0/425151187 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fca8c003970 0x7fcaac14dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:12:28.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:28.146+0000 7fcab0f1a640 1 --2- 172.21.15.92:0/425151187 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcaac0ca820 0x7fcaac0cc1b0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fcaac0c6c90 tx=0x7fcaa400e830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:28.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:28.147+0000 7fcaa8ff9640 1 -- 172.21.15.92:0/425151187 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaa400c070 con 0x7fcaac0ca820 2023-11-24T06:12:28.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:28.147+0000 7fcaa8ff9640 1 -- 172.21.15.92:0/425151187 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcaa4013610 con 0x7fcaac0ca820 2023-11-24T06:12:28.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:28.147+0000 7fcaa8ff9640 1 -- 172.21.15.92:0/425151187 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaa40139c0 con 0x7fcaac0ca820 2023-11-24T06:12:28.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:28.147+0000 7fcab297d640 1 -- 172.21.15.92:0/425151187 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcaac0cce90 con 0x7fcaac0ca820 2023-11-24T06:12:28.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:28.147+0000 7fcab297d640 1 -- 172.21.15.92:0/425151187 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcaac158f10 con 0x7fcaac0ca820 2023-11-24T06:12:28.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:28.148+0000 7fcab297d640 1 -- 172.21.15.92:0/425151187 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcaac0675d0 con 0x7fcaac0ca820 2023-11-24T06:12:28.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:28.149+0000 7fcaa8ff9640 1 -- 172.21.15.92:0/425151187 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fcaa4013b20 con 0x7fcaac0ca820 2023-11-24T06:12:28.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:28.150+0000 7fcaa8ff9640 1 --2- 172.21.15.92:0/425151187 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fca9006ae40 0x7fca9006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:28.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:28.150+0000 7fcaa8ff9640 1 -- 172.21.15.92:0/425151187 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fcaa4088dd0 con 0x7fcaac0ca820 2023-11-24T06:12:28.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:28.150+0000 7fcaabfff640 1 --2- 172.21.15.92:0/425151187 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fca9006ae40 0x7fca9006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:28.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:28.151+0000 7fcaabfff640 1 --2- 172.21.15.92:0/425151187 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fca9006ae40 0x7fca9006d300 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fca9c009ac0 tx=0x7fca9c030000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:28.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:28.155+0000 7fcaa8ff9640 1 -- 172.21.15.92:0/425151187 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fcaa4054260 con 0x7fcaac0ca820 2023-11-24T06:12:28.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:28.354+0000 7fcab297d640 1 -- 172.21.15.92:0/425151187 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fcaac0d3920 con 0x7fcaac0ca820 2023-11-24T06:12:29.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:29.126+0000 7fcaa8ff9640 1 -- 172.21.15.92:0/425151187 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v49) v1 ==== 138+0+0 (secure 0 0 0) 0x7fcaa4057880 con 0x7fcaac0ca820 2023-11-24T06:12:29.130 INFO:teuthology.orchestra.run.smithi092.stderr:pool 'cephfs_metadata' created 2023-11-24T06:12:29.133 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:29.130+0000 7fcab297d640 1 -- 172.21.15.92:0/425151187 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fca9006ae40 msgr2=0x7fca9006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:29.133 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:29.130+0000 7fcab297d640 1 --2- 172.21.15.92:0/425151187 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fca9006ae40 0x7fca9006d300 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fca9c009ac0 tx=0x7fca9c030000 comp rx=0 tx=0).stop 2023-11-24T06:12:29.133 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:29.130+0000 7fcab297d640 1 -- 172.21.15.92:0/425151187 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcaac0ca820 msgr2=0x7fcaac0cc1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:29.133 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:29.130+0000 7fcab297d640 1 --2- 172.21.15.92:0/425151187 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcaac0ca820 0x7fcaac0cc1b0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fcaac0c6c90 tx=0x7fcaa400e830 comp rx=0 tx=0).stop 2023-11-24T06:12:29.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:29.130+0000 7fcab297d640 1 -- 172.21.15.92:0/425151187 shutdown_connections 2023-11-24T06:12:29.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:29.130+0000 7fcab297d640 1 --2- 172.21.15.92:0/425151187 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fca9006ae40 0x7fca9006d300 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:29.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:29.130+0000 7fcab297d640 1 --2- 172.21.15.92:0/425151187 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcaac0ca820 0x7fcaac0cc1b0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:29.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:29.130+0000 7fcab297d640 1 --2- 172.21.15.92:0/425151187 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fcaac1f40e0 0x7fcaac0ca2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:29.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:29.130+0000 7fcab297d640 1 --2- 172.21.15.92:0/425151187 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fca8c003970 0x7fcaac14dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:29.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:29.130+0000 7fcab297d640 1 -- 172.21.15.92:0/425151187 >> 172.21.15.92:0/425151187 conn(0x7fcaac09c550 msgr2=0x7fcaac15b780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:29.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:29.130+0000 7fcab297d640 1 -- 172.21.15.92:0/425151187 shutdown_connections 2023-11-24T06:12:29.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:29.130+0000 7fcab297d640 1 -- 172.21.15.92:0/425151187 wait complete. 2023-11-24T06:12:29.157 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-11-24T06:12:29.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:29.358+0000 7f57a1009640 1 Processor -- start 2023-11-24T06:12:29.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:29.358+0000 7f57a1009640 1 -- start start 2023-11-24T06:12:29.363 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:29.359+0000 7f57a1009640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f579c15d640 0x7f579c0d1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:29.363 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:29.359+0000 7f57a1009640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f579c0d0140 0x7f579c0b0f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:29.363 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:29.359+0000 7f57a1009640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f579c0b22a0 con 0x7f579c15da20 2023-11-24T06:12:29.363 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:29.360+0000 7f57a1009640 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f579c0afec0 con 0x7f579c15d640 2023-11-24T06:12:29.363 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:29.360+0000 7f57a1009640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f579c0b01a0 con 0x7f579c0d0140 2023-11-24T06:12:29.363 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:29.360+0000 7f579a59c640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f579c15da20 0x7f579c0cfa30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:49828/0 (socket says 172.21.15.92:49828) 2023-11-24T06:12:29.363 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:29.360+0000 7f579a59c640 1 -- 172.21.15.92:0/2325641553 learned_addr learned my addr 172.21.15.92:0/2325641553 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-24T06:12:29.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:29.360+0000 7f579ad9d640 1 --2- 172.21.15.92:0/2325641553 >> v2:172.21.15.118:3300/0 conn(0x7f579c15d640 0x7f579c0d1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:29.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:29.360+0000 7f579b59e640 1 --2- 172.21.15.92:0/2325641553 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f579c0d0140 0x7f579c0b0f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:29.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:29.360+0000 7f5799d9b640 1 -- 172.21.15.92:0/2325641553 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3414453080 0 0) 0x7f579c0b22a0 con 0x7f579c15da20 2023-11-24T06:12:29.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:29.360+0000 7f5799d9b640 1 -- 172.21.15.92:0/2325641553 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5788003520 con 0x7f579c15da20 2023-11-24T06:12:29.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:29.361+0000 7f5799d9b640 1 -- 172.21.15.92:0/2325641553 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2501677870 0 0) 0x7f5788003520 con 0x7f579c15da20 2023-11-24T06:12:29.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:29.361+0000 7f5799d9b640 1 -- 172.21.15.92:0/2325641553 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f579c0d0140 msgr2=0x7f579c0b0f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:29.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:29.361+0000 7f5799d9b640 1 --2- 172.21.15.92:0/2325641553 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f579c0d0140 0x7f579c0b0f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:29.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:29.361+0000 7f5799d9b640 1 -- 172.21.15.92:0/2325641553 >> v2:172.21.15.118:3300/0 conn(0x7f579c15d640 msgr2=0x7f579c0d1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:29.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:29.361+0000 7f5799d9b640 1 --2- 172.21.15.92:0/2325641553 >> v2:172.21.15.118:3300/0 conn(0x7f579c15d640 0x7f579c0d1b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:29.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:29.361+0000 7f5799d9b640 1 -- 172.21.15.92:0/2325641553 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f579c0b0480 con 0x7f579c15da20 2023-11-24T06:12:29.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:29.361+0000 7f5799d9b640 1 -- 172.21.15.92:0/2325641553 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2607661350 0 0) 0x7f578c001670 con 0x7f579c15da20 2023-11-24T06:12:29.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:29.361+0000 7f5799d9b640 1 -- 172.21.15.92:0/2325641553 >> v1:172.21.15.92:6789/0 conn(0x7f579c15da20 legacy=0x7f579c0cfa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:29.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:29.361+0000 7f5799d9b640 1 --2- 172.21.15.92:0/2325641553 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5788003970 0x7f5788003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:29.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:29.361+0000 7f5799d9b640 1 --2- 172.21.15.92:0/2325641553 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f57880047c0 0x7f5788004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:29.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:29.361+0000 7f5799d9b640 1 --2- 172.21.15.92:0/2325641553 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f579c0d0140 0x7f5788005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:29.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:29.361+0000 7f5799d9b640 1 -- 172.21.15.92:0/2325641553 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f579c0afec0 con 0x7f5788003970 2023-11-24T06:12:29.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:29.361+0000 7f5799d9b640 1 -- 172.21.15.92:0/2325641553 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f579c0b01a0 con 0x7f57880047c0 2023-11-24T06:12:29.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:29.361+0000 7f5799d9b640 1 -- 172.21.15.92:0/2325641553 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f578800b980 con 0x7f579c0d0140 2023-11-24T06:12:29.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:29.361+0000 7f579ad9d640 1 --2- 172.21.15.92:0/2325641553 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5788003970 0x7f5788003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:29.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:29.362+0000 7f579ad9d640 1 -- 172.21.15.92:0/2325641553 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f579c0d0140 msgr2=0x7f5788005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:29.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:29.362+0000 7f579ad9d640 1 --2- 172.21.15.92:0/2325641553 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f579c0d0140 0x7f5788005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:29.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:29.362+0000 7f579ad9d640 1 -- 172.21.15.92:0/2325641553 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f57880047c0 msgr2=0x7f5788004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:29.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:29.362+0000 7f579ad9d640 1 --2- 172.21.15.92:0/2325641553 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f57880047c0 0x7f5788004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:29.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:29.362+0000 7f579ad9d640 1 -- 172.21.15.92:0/2325641553 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5788003520 con 0x7f5788003970 2023-11-24T06:12:29.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:29.363+0000 7f579ad9d640 1 --2- 172.21.15.92:0/2325641553 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5788003970 0x7f5788003d90 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f5784009ac0 tx=0x7f57880057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5dff702ae27be36 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:29.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:29.363+0000 7f5799d9b640 1 -- 172.21.15.92:0/2325641553 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f578402c710 con 0x7f5788003970 2023-11-24T06:12:29.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:29.363+0000 7f5799d9b640 1 -- 172.21.15.92:0/2325641553 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f578402c870 con 0x7f5788003970 2023-11-24T06:12:29.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:29.363+0000 7f5799d9b640 1 -- 172.21.15.92:0/2325641553 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f578402cc00 con 0x7f5788003970 2023-11-24T06:12:29.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:29.364+0000 7f57a1009640 1 -- 172.21.15.92:0/2325641553 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5788003970 msgr2=0x7f5788003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:29.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:29.364+0000 7f57a1009640 1 --2- 172.21.15.92:0/2325641553 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5788003970 0x7f5788003d90 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f5784009ac0 tx=0x7f57880057f0 comp rx=0 tx=0).stop 2023-11-24T06:12:29.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:29.364+0000 7f579b59e640 1 -- 172.21.15.92:0/2325641553 reap_dead start 2023-11-24T06:12:29.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:29.364+0000 7f57a1009640 1 -- 172.21.15.92:0/2325641553 shutdown_connections 2023-11-24T06:12:29.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:29.364+0000 7f57a1009640 1 -- 172.21.15.92:0/2325641553 >> 172.21.15.92:0/2325641553 conn(0x7f579c09c560 msgr2=0x7f579c0d3230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:29.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:29.364+0000 7f57a1009640 1 -- 172.21.15.92:0/2325641553 shutdown_connections 2023-11-24T06:12:29.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:29.364+0000 7f57a1009640 1 -- 172.21.15.92:0/2325641553 wait complete. 2023-11-24T06:12:29.369 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:29.366+0000 7f57a1009640 1 Processor -- start 2023-11-24T06:12:29.369 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:29.366+0000 7f57a1009640 1 -- start start 2023-11-24T06:12:29.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:29.367+0000 7f57a1009640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5788003970 0x7f579c0b1830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:29.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:29.367+0000 7f57a1009640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f579c0b8b20 0x7f579c1f6310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:29.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:29.367+0000 7f57a1009640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f579c1f6850 0x7f579c0b6cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:29.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:29.367+0000 7f57a1009640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f579c1f6d00 con 0x7f579c0b8b20 2023-11-24T06:12:29.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:29.367+0000 7f57a1009640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f579c0b21a0 con 0x7f579c1f6850 2023-11-24T06:12:29.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:29.367+0000 7f57a1009640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f579c0b5f10 con 0x7f5788003970 2023-11-24T06:12:29.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:29.367+0000 7f579a59c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f579c0b8b20 0x7f579c1f6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:29.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:29.367+0000 7f579a59c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f579c0b8b20 0x7f579c1f6310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:49778/0 (socket says 172.21.15.92:49778) 2023-11-24T06:12:29.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:29.367+0000 7f579a59c640 1 -- 172.21.15.92:0/925759830 learned_addr learned my addr 172.21.15.92:0/925759830 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:12:29.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:29.367+0000 7f579ad9d640 1 --2- 172.21.15.92:0/925759830 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5788003970 0x7f579c0b1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:29.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:29.367+0000 7f579b59e640 1 --2- 172.21.15.92:0/925759830 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f579c1f6850 0x7f579c0b6cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:29.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:29.368+0000 7f579a59c640 1 -- 172.21.15.92:0/925759830 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5788003970 msgr2=0x7f579c0b1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:29.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:29.368+0000 7f579a59c640 1 --2- 172.21.15.92:0/925759830 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5788003970 0x7f579c0b1830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:29.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:29.368+0000 7f579a59c640 1 -- 172.21.15.92:0/925759830 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f579c1f6850 msgr2=0x7f579c0b6cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:29.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:29.368+0000 7f579a59c640 1 --2- 172.21.15.92:0/925759830 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f579c1f6850 0x7f579c0b6cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:29.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:29.368+0000 7f579a59c640 1 -- 172.21.15.92:0/925759830 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f579c0b61f0 con 0x7f579c0b8b20 2023-11-24T06:12:29.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:29.368+0000 7f579a59c640 1 --2- 172.21.15.92:0/925759830 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f579c0b8b20 0x7f579c1f6310 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f578c00c710 tx=0x7f578c00cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:29.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:29.368+0000 7f57937fe640 1 -- 172.21.15.92:0/925759830 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f578c0074e0 con 0x7f579c0b8b20 2023-11-24T06:12:29.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:29.368+0000 7f57937fe640 1 -- 172.21.15.92:0/925759830 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f578c007640 con 0x7f579c0b8b20 2023-11-24T06:12:29.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:29.368+0000 7f57937fe640 1 -- 172.21.15.92:0/925759830 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f578c004a00 con 0x7f579c0b8b20 2023-11-24T06:12:29.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:29.368+0000 7f57a1009640 1 -- 172.21.15.92:0/925759830 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f579c0b64d0 con 0x7f579c0b8b20 2023-11-24T06:12:29.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:29.368+0000 7f57a1009640 1 -- 172.21.15.92:0/925759830 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f579c0b1d70 con 0x7f579c0b8b20 2023-11-24T06:12:29.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:29.369+0000 7f57a1009640 1 -- 172.21.15.92:0/925759830 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f579c0b69f0 con 0x7f579c0b8b20 2023-11-24T06:12:29.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:29.371+0000 7f57937fe640 1 -- 172.21.15.92:0/925759830 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f578c004b60 con 0x7f579c0b8b20 2023-11-24T06:12:29.374 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:29.371+0000 7f57937fe640 1 --2- 172.21.15.92:0/925759830 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f577c06ae40 0x7f577c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:29.374 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:29.371+0000 7f57937fe640 1 -- 172.21.15.92:0/925759830 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f578c088900 con 0x7f579c0b8b20 2023-11-24T06:12:29.375 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:29.372+0000 7f579ad9d640 1 --2- 172.21.15.92:0/925759830 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f577c06ae40 0x7f577c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:29.375 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:29.372+0000 7f579ad9d640 1 --2- 172.21.15.92:0/925759830 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f577c06ae40 0x7f577c06d300 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f5784036000 tx=0x7f5784030000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:29.379 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:29.377+0000 7f57937fe640 1 -- 172.21.15.92:0/925759830 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f579c0b69f0 con 0x7f579c0b8b20 2023-11-24T06:12:29.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:29.570+0000 7f57a1009640 1 -- 172.21.15.92:0/925759830 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f579c0b69f0 con 0x7f579c0b8b20 2023-11-24T06:12:30.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:30.133+0000 7f57937fe640 1 -- 172.21.15.92:0/925759830 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v50) v1 ==== 170+0+0 (secure 0 0 0) 0x7f579c0b69f0 con 0x7f579c0b8b20 2023-11-24T06:12:30.136 INFO:teuthology.orchestra.run.smithi092.stderr:pool 'cephfs_data' created 2023-11-24T06:12:30.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:30.136+0000 7f57a1009640 1 -- 172.21.15.92:0/925759830 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f577c06ae40 msgr2=0x7f577c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:30.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:30.136+0000 7f57a1009640 1 --2- 172.21.15.92:0/925759830 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f577c06ae40 0x7f577c06d300 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f5784036000 tx=0x7f5784030000 comp rx=0 tx=0).stop 2023-11-24T06:12:30.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:30.136+0000 7f57a1009640 1 -- 172.21.15.92:0/925759830 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f579c0b8b20 msgr2=0x7f579c1f6310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:30.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:30.136+0000 7f57a1009640 1 --2- 172.21.15.92:0/925759830 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f579c0b8b20 0x7f579c1f6310 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f578c00c710 tx=0x7f578c00cbd0 comp rx=0 tx=0).stop 2023-11-24T06:12:30.140 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:30.136+0000 7f57a1009640 1 -- 172.21.15.92:0/925759830 shutdown_connections 2023-11-24T06:12:30.140 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:30.137+0000 7f57a1009640 1 --2- 172.21.15.92:0/925759830 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f577c06ae40 0x7f577c06d300 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:30.140 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:30.137+0000 7f57a1009640 1 --2- 172.21.15.92:0/925759830 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f579c1f6850 0x7f579c0b6cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:30.140 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:30.137+0000 7f57a1009640 1 --2- 172.21.15.92:0/925759830 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f579c0b8b20 0x7f579c1f6310 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:30.140 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:30.137+0000 7f57a1009640 1 --2- 172.21.15.92:0/925759830 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f5788003970 0x7f579c0b1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:30.140 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:30.137+0000 7f57a1009640 1 -- 172.21.15.92:0/925759830 >> 172.21.15.92:0/925759830 conn(0x7f579c09c560 msgr2=0x7f579c15dee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:30.140 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:30.137+0000 7f57a1009640 1 -- 172.21.15.92:0/925759830 shutdown_connections 2023-11-24T06:12:30.140 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:30.137+0000 7f57a1009640 1 -- 172.21.15.92:0/925759830 wait complete. 2023-11-24T06:12:30.171 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-11-24T06:12:30.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:30.358+0000 7f3e852ec640 1 Processor -- start 2023-11-24T06:12:30.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:30.358+0000 7f3e852ec640 1 -- start start 2023-11-24T06:12:30.363 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:30.360+0000 7f3e852ec640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f3e8013e360 0x7f3e800cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:30.363 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:30.360+0000 7f3e852ec640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3e8015f9c0 0x7f3e800cf7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:30.363 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:30.360+0000 7f3e852ec640 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e8015c5f0 con 0x7f3e8013df80 2023-11-24T06:12:30.363 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:30.360+0000 7f3e852ec640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e8006c470 con 0x7f3e8013e360 2023-11-24T06:12:30.363 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:30.360+0000 7f3e852ec640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e8006c750 con 0x7f3e8015f9c0 2023-11-24T06:12:30.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:30.360+0000 7f3e7e7fc640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f3e8013e360 0x7f3e800cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:30.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:30.360+0000 7f3e7e7fc640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f3e8013e360 0x7f3e800cf290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:49792/0 (socket says 172.21.15.92:49792) 2023-11-24T06:12:30.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:30.360+0000 7f3e7e7fc640 1 -- 172.21.15.92:0/1138008612 learned_addr learned my addr 172.21.15.92:0/1138008612 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:12:30.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:30.361+0000 7f3e7f7fe640 1 --2- 172.21.15.92:0/1138008612 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3e8015f9c0 0x7f3e800cf7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:30.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:30.361+0000 7f3e7e7fc640 1 -- 172.21.15.92:0/1138008612 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3e8015f9c0 msgr2=0x7f3e800cf7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:30.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:30.361+0000 7f3e7e7fc640 1 --2- 172.21.15.92:0/1138008612 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3e8015f9c0 0x7f3e800cf7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:30.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:30.361+0000 7f3e7e7fc640 1 -- 172.21.15.92:0/1138008612 >> v1:172.21.15.118:6789/0 conn(0x7f3e8013df80 legacy=0x7f3e8015bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:30.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:30.361+0000 7f3e7e7fc640 1 -- 172.21.15.92:0/1138008612 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e8006ca30 con 0x7f3e8013e360 2023-11-24T06:12:30.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:30.362+0000 7f3e7e7fc640 1 --2- 172.21.15.92:0/1138008612 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3e8013e360 0x7f3e800cf290 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f3e8006ded0 tx=0x7f3e7402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2711cd316726502f server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:30.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:30.362+0000 7f3e7dffb640 1 -- 172.21.15.92:0/1138008612 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e74031830 con 0x7f3e8013e360 2023-11-24T06:12:30.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:30.362+0000 7f3e7dffb640 1 -- 172.21.15.92:0/1138008612 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e74031990 con 0x7f3e8013e360 2023-11-24T06:12:30.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:30.362+0000 7f3e7dffb640 1 -- 172.21.15.92:0/1138008612 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e74031d20 con 0x7f3e8013e360 2023-11-24T06:12:30.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:30.363+0000 7f3e852ec640 1 -- 172.21.15.92:0/1138008612 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3e8013e360 msgr2=0x7f3e800cf290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:30.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:30.363+0000 7f3e852ec640 1 --2- 172.21.15.92:0/1138008612 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3e8013e360 0x7f3e800cf290 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f3e8006ded0 tx=0x7f3e7402b8d0 comp rx=0 tx=0).stop 2023-11-24T06:12:30.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:30.363+0000 7f3e852ec640 1 -- 172.21.15.92:0/1138008612 shutdown_connections 2023-11-24T06:12:30.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:30.363+0000 7f3e852ec640 1 --2- 172.21.15.92:0/1138008612 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3e8015f9c0 0x7f3e800cf7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:30.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:30.363+0000 7f3e852ec640 1 --2- 172.21.15.92:0/1138008612 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3e8013e360 0x7f3e800cf290 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:30.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:30.363+0000 7f3e852ec640 1 -- 172.21.15.92:0/1138008612 >> 172.21.15.92:0/1138008612 conn(0x7f3e8006a800 msgr2=0x7f3e8006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:30.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:30.363+0000 7f3e852ec640 1 -- 172.21.15.92:0/1138008612 shutdown_connections 2023-11-24T06:12:30.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:30.363+0000 7f3e852ec640 1 -- 172.21.15.92:0/1138008612 wait complete. 2023-11-24T06:12:30.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:30.365+0000 7f3e852ec640 1 Processor -- start 2023-11-24T06:12:30.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:30.365+0000 7f3e852ec640 1 -- start start 2023-11-24T06:12:30.369 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:30.367+0000 7f3e852ec640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3e8013df80 0x7f3e8013d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:30.369 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:30.367+0000 7f3e852ec640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3e8013e360 0x7f3e8014b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:30.369 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:30.367+0000 7f3e852ec640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3e8015f9c0 0x7f3e801f0d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:30.370 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:30.367+0000 7f3e852ec640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e801f1db0 con 0x7f3e8013e360 2023-11-24T06:12:30.370 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:30.367+0000 7f3e852ec640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e8013de10 con 0x7f3e8015f9c0 2023-11-24T06:12:30.370 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:30.367+0000 7f3e852ec640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e801f17f0 con 0x7f3e8013df80 2023-11-24T06:12:30.370 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:30.367+0000 7f3e7e7fc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3e8013e360 0x7f3e8014b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:30.370 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:30.367+0000 7f3e7e7fc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3e8013e360 0x7f3e8014b3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:49796/0 (socket says 172.21.15.92:49796) 2023-11-24T06:12:30.370 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:30.367+0000 7f3e7e7fc640 1 -- 172.21.15.92:0/3048936979 learned_addr learned my addr 172.21.15.92:0/3048936979 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:12:30.370 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:30.367+0000 7f3e7effd640 1 --2- 172.21.15.92:0/3048936979 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3e8013df80 0x7f3e8013d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:30.370 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:30.367+0000 7f3e7f7fe640 1 --2- 172.21.15.92:0/3048936979 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3e8015f9c0 0x7f3e801f0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:30.370 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:30.367+0000 7f3e7e7fc640 1 -- 172.21.15.92:0/3048936979 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3e8013df80 msgr2=0x7f3e8013d8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:30.370 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:30.367+0000 7f3e7e7fc640 1 --2- 172.21.15.92:0/3048936979 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3e8013df80 0x7f3e8013d8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:30.370 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:30.367+0000 7f3e7e7fc640 1 -- 172.21.15.92:0/3048936979 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3e8015f9c0 msgr2=0x7f3e801f0d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:30.370 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:30.367+0000 7f3e7e7fc640 1 --2- 172.21.15.92:0/3048936979 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3e8015f9c0 0x7f3e801f0d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:30.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:30.367+0000 7f3e7e7fc640 1 -- 172.21.15.92:0/3048936979 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e740094d0 con 0x7f3e8013e360 2023-11-24T06:12:30.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:30.367+0000 7f3e7f7fe640 1 --2- 172.21.15.92:0/3048936979 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3e8015f9c0 0x7f3e801f0d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:12:30.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:30.368+0000 7f3e7e7fc640 1 --2- 172.21.15.92:0/3048936979 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3e8013e360 0x7f3e8014b3c0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f3e74000c00 tx=0x7f3e740046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:30.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:30.368+0000 7f3e677fe640 1 -- 172.21.15.92:0/3048936979 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e740317d0 con 0x7f3e8013e360 2023-11-24T06:12:30.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:30.368+0000 7f3e677fe640 1 -- 172.21.15.92:0/3048936979 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e7403aa60 con 0x7f3e8013e360 2023-11-24T06:12:30.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:30.368+0000 7f3e677fe640 1 -- 172.21.15.92:0/3048936979 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e7403ae10 con 0x7f3e8013e360 2023-11-24T06:12:30.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:30.368+0000 7f3e852ec640 1 -- 172.21.15.92:0/3048936979 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e801f1ad0 con 0x7f3e8013e360 2023-11-24T06:12:30.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:30.368+0000 7f3e852ec640 1 -- 172.21.15.92:0/3048936979 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e8014aea0 con 0x7f3e8013e360 2023-11-24T06:12:30.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:30.369+0000 7f3e852ec640 1 -- 172.21.15.92:0/3048936979 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e8014aa20 con 0x7f3e8013e360 2023-11-24T06:12:30.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:30.370+0000 7f3e677fe640 1 -- 172.21.15.92:0/3048936979 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f3e7404d450 con 0x7f3e8013e360 2023-11-24T06:12:30.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:30.371+0000 7f3e677fe640 1 --2- 172.21.15.92:0/3048936979 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f3e6c06ae40 0x7f3e6c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:30.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:30.371+0000 7f3e677fe640 1 -- 172.21.15.92:0/3048936979 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f3e740ab760 con 0x7f3e8013e360 2023-11-24T06:12:30.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:30.372+0000 7f3e7effd640 1 --2- 172.21.15.92:0/3048936979 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f3e6c06ae40 0x7f3e6c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:30.374 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:30.372+0000 7f3e7effd640 1 --2- 172.21.15.92:0/3048936979 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f3e6c06ae40 0x7f3e6c06d300 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f3e70006fd0 tx=0x7f3e70008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:30.380 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:30.378+0000 7f3e677fe640 1 -- 172.21.15.92:0/3048936979 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3e8014aa20 con 0x7f3e8013e360 2023-11-24T06:12:30.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:30.603+0000 7f3e852ec640 1 -- 172.21.15.92:0/3048936979 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f3e8014aa20 con 0x7f3e8013e360 2023-11-24T06:12:31.508 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:31.506+0000 7f3e677fe640 1 -- 172.21.15.92:0/3048936979 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 8 and data pool 9 v27) v1 ==== 172+0+0 (secure 0 0 0) 0x7f3e8014aa20 con 0x7f3e8013e360 2023-11-24T06:12:31.508 INFO:teuthology.orchestra.run.smithi092.stderr:new fs with metadata pool 8 and data pool 9 2023-11-24T06:12:31.512 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:31.509+0000 7f3e852ec640 1 -- 172.21.15.92:0/3048936979 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f3e6c06ae40 msgr2=0x7f3e6c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:31.512 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:31.509+0000 7f3e852ec640 1 --2- 172.21.15.92:0/3048936979 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f3e6c06ae40 0x7f3e6c06d300 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f3e70006fd0 tx=0x7f3e70008040 comp rx=0 tx=0).stop 2023-11-24T06:12:31.512 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:31.509+0000 7f3e852ec640 1 -- 172.21.15.92:0/3048936979 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3e8013e360 msgr2=0x7f3e8014b3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:31.512 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:31.509+0000 7f3e852ec640 1 --2- 172.21.15.92:0/3048936979 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3e8013e360 0x7f3e8014b3c0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f3e74000c00 tx=0x7f3e740046c0 comp rx=0 tx=0).stop 2023-11-24T06:12:31.512 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:31.510+0000 7f3e852ec640 1 -- 172.21.15.92:0/3048936979 shutdown_connections 2023-11-24T06:12:31.512 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:31.510+0000 7f3e852ec640 1 --2- 172.21.15.92:0/3048936979 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f3e6c06ae40 0x7f3e6c06d300 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:31.512 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:31.510+0000 7f3e852ec640 1 --2- 172.21.15.92:0/3048936979 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3e8015f9c0 0x7f3e801f0d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:31.512 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:31.510+0000 7f3e852ec640 1 --2- 172.21.15.92:0/3048936979 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3e8013e360 0x7f3e8014b3c0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:31.512 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:31.510+0000 7f3e852ec640 1 --2- 172.21.15.92:0/3048936979 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3e8013df80 0x7f3e8013d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:31.512 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:31.510+0000 7f3e852ec640 1 -- 172.21.15.92:0/3048936979 >> 172.21.15.92:0/3048936979 conn(0x7f3e8006a800 msgr2=0x7f3e8015de70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:31.513 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:31.510+0000 7f3e852ec640 1 -- 172.21.15.92:0/3048936979 shutdown_connections 2023-11-24T06:12:31.513 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:31.510+0000 7f3e852ec640 1 -- 172.21.15.92:0/3048936979 wait complete. 2023-11-24T06:12:31.535 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:12:31.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:31.715+0000 7f9fbf5a9640 1 Processor -- start 2023-11-24T06:12:31.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:31.715+0000 7f9fbf5a9640 1 -- start start 2023-11-24T06:12:31.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:31.717+0000 7f9fbf5a9640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f9fb80cd240 0x7f9fb80b3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:31.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:31.717+0000 7f9fbf5a9640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f9fb806c6b0 0x7f9fb8099a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:31.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:31.717+0000 7f9fbf5a9640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9fb806cea0 con 0x7f9fb8099fa0 2023-11-24T06:12:31.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:31.717+0000 7f9fbf5a9640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9fb809a410 con 0x7f9fb806c6b0 2023-11-24T06:12:31.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:31.717+0000 7f9fbf5a9640 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9fb806cbc0 con 0x7f9fb80cd240 2023-11-24T06:12:31.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:31.717+0000 7f9fbcb44640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f9fb806c6b0 0x7f9fb8099a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:31.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:31.717+0000 7f9fbcb44640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f9fb806c6b0 0x7f9fb8099a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:49808/0 (socket says 172.21.15.92:49808) 2023-11-24T06:12:31.720 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:31.717+0000 7f9fbcb44640 1 -- 172.21.15.92:0/2972712243 learned_addr learned my addr 172.21.15.92:0/2972712243 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:12:31.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:31.717+0000 7f9fbd345640 1 --2- 172.21.15.92:0/2972712243 >> v2:172.21.15.118:3300/0 conn(0x7f9fb80cd240 0x7f9fb80b3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:31.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:31.717+0000 7f9fbcb44640 1 -- 172.21.15.92:0/2972712243 >> v2:172.21.15.118:3300/0 conn(0x7f9fb80cd240 msgr2=0x7f9fb80b3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:31.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:31.717+0000 7f9fbcb44640 1 --2- 172.21.15.92:0/2972712243 >> v2:172.21.15.118:3300/0 conn(0x7f9fb80cd240 0x7f9fb80b3820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:31.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:31.717+0000 7f9fbcb44640 1 -- 172.21.15.92:0/2972712243 >> v1:172.21.15.92:6789/0 conn(0x7f9fb8099fa0 legacy=0x7f9fb806b4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:31.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:31.718+0000 7f9fbcb44640 1 -- 172.21.15.92:0/2972712243 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fb806f350 con 0x7f9fb806c6b0 2023-11-24T06:12:31.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:31.718+0000 7f9faffff640 1 -- 172.21.15.92:0/2972712243 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 743207559 0 0) 0x7f9fb806cea0 con 0x7f9fb8099fa0 2023-11-24T06:12:31.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:31.719+0000 7f9fbcb44640 1 --2- 172.21.15.92:0/2972712243 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9fb806c6b0 0x7f9fb8099a60 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f9fb8070a10 tx=0x7f9fa802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a77b9732a827529 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:31.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:31.719+0000 7f9faffff640 1 -- 172.21.15.92:0/2972712243 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fa8030860 con 0x7f9fb806c6b0 2023-11-24T06:12:31.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:31.719+0000 7f9faffff640 1 -- 172.21.15.92:0/2972712243 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fa80309c0 con 0x7f9fb806c6b0 2023-11-24T06:12:31.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:31.719+0000 7f9faffff640 1 -- 172.21.15.92:0/2972712243 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fa8030d50 con 0x7f9fb806c6b0 2023-11-24T06:12:31.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:31.719+0000 7f9fbf5a9640 1 -- 172.21.15.92:0/2972712243 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9fb806c6b0 msgr2=0x7f9fb8099a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:31.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:31.719+0000 7f9fbf5a9640 1 --2- 172.21.15.92:0/2972712243 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9fb806c6b0 0x7f9fb8099a60 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f9fb8070a10 tx=0x7f9fa802b9a0 comp rx=0 tx=0).stop 2023-11-24T06:12:31.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:31.720+0000 7f9fbf5a9640 1 -- 172.21.15.92:0/2972712243 shutdown_connections 2023-11-24T06:12:31.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:31.720+0000 7f9fbf5a9640 1 --2- 172.21.15.92:0/2972712243 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9fb806c6b0 0x7f9fb8099a60 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:31.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:31.720+0000 7f9fbf5a9640 1 --2- 172.21.15.92:0/2972712243 >> v2:172.21.15.118:3300/0 conn(0x7f9fb80cd240 0x7f9fb80b3820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:31.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:31.720+0000 7f9fbf5a9640 1 -- 172.21.15.92:0/2972712243 >> 172.21.15.92:0/2972712243 conn(0x7f9fb80b2d00 msgr2=0x7f9fb80b3110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:31.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:31.720+0000 7f9fbf5a9640 1 -- 172.21.15.92:0/2972712243 shutdown_connections 2023-11-24T06:12:31.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:31.720+0000 7f9fbf5a9640 1 -- 172.21.15.92:0/2972712243 wait complete. 2023-11-24T06:12:31.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:31.721+0000 7f9fbf5a9640 1 Processor -- start 2023-11-24T06:12:31.724 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:31.721+0000 7f9fbf5a9640 1 -- start start 2023-11-24T06:12:31.724 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:31.722+0000 7f9fbf5a9640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9fb806c6b0 0x7f9fb80c5d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:31.725 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:31.722+0000 7f9fbf5a9640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f9fb8099fa0 0x7f9fb80c7550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:31.725 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:31.722+0000 7f9fbf5a9640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9fb80cd240 0x7f9fb80d0310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:31.725 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:31.722+0000 7f9fbf5a9640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fb80caef0 con 0x7f9fb80cd240 2023-11-24T06:12:31.725 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:31.722+0000 7f9fbf5a9640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fb80c82e0 con 0x7f9fb806c6b0 2023-11-24T06:12:31.725 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:31.722+0000 7f9fbf5a9640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fb80c6330 con 0x7f9fb8099fa0 2023-11-24T06:12:31.725 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:31.723+0000 7f9fbdb46640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9fb80cd240 0x7f9fb80d0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:31.725 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:31.723+0000 7f9fbdb46640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9fb80cd240 0x7f9fb80d0310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:49816/0 (socket says 172.21.15.92:49816) 2023-11-24T06:12:31.725 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:31.723+0000 7f9fbdb46640 1 -- 172.21.15.92:0/3551775322 learned_addr learned my addr 172.21.15.92:0/3551775322 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:12:31.726 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:31.723+0000 7f9fbd345640 1 --2- 172.21.15.92:0/3551775322 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9fb806c6b0 0x7f9fb80c5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:31.726 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:31.723+0000 7f9fbcb44640 1 --2- 172.21.15.92:0/3551775322 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f9fb8099fa0 0x7f9fb80c7550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:31.726 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:31.723+0000 7f9fbdb46640 1 -- 172.21.15.92:0/3551775322 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f9fb8099fa0 msgr2=0x7f9fb80c7550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:31.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:31.723+0000 7f9fbdb46640 1 --2- 172.21.15.92:0/3551775322 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f9fb8099fa0 0x7f9fb80c7550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:31.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:31.723+0000 7f9fbdb46640 1 -- 172.21.15.92:0/3551775322 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9fb806c6b0 msgr2=0x7f9fb80c5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:31.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:31.723+0000 7f9fbdb46640 1 --2- 172.21.15.92:0/3551775322 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9fb806c6b0 0x7f9fb80c5d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:31.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:31.723+0000 7f9fbdb46640 1 -- 172.21.15.92:0/3551775322 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fa80094d0 con 0x7f9fb80cd240 2023-11-24T06:12:31.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:31.723+0000 7f9fbdb46640 1 --2- 172.21.15.92:0/3551775322 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9fb80cd240 0x7f9fb80d0310 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f9fb0001560 tx=0x7f9fb000cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:31.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:31.724+0000 7f9fadffb640 1 -- 172.21.15.92:0/3551775322 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fb0003da0 con 0x7f9fb80cd240 2023-11-24T06:12:31.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:31.724+0000 7f9fadffb640 1 -- 172.21.15.92:0/3551775322 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fb0005070 con 0x7f9fb80cd240 2023-11-24T06:12:31.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:31.724+0000 7f9fadffb640 1 -- 172.21.15.92:0/3551775322 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fb0005420 con 0x7f9fb80cd240 2023-11-24T06:12:31.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:31.724+0000 7f9fbf5a9640 1 -- 172.21.15.92:0/3551775322 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fb80c6610 con 0x7f9fb80cd240 2023-11-24T06:12:31.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:31.724+0000 7f9fbf5a9640 1 -- 172.21.15.92:0/3551775322 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fb80c6bd0 con 0x7f9fb80cd240 2023-11-24T06:12:31.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:31.725+0000 7f9fbf5a9640 1 -- 172.21.15.92:0/3551775322 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9fb806d0d0 con 0x7f9fb80cd240 2023-11-24T06:12:31.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:31.726+0000 7f9fadffb640 1 -- 172.21.15.92:0/3551775322 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f9fb0005580 con 0x7f9fb80cd240 2023-11-24T06:12:31.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:31.726+0000 7f9fadffb640 1 --2- 172.21.15.92:0/3551775322 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f9f9c06ae40 0x7f9f9c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:31.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:31.727+0000 7f9fadffb640 1 -- 172.21.15.92:0/3551775322 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9fb0088ff0 con 0x7f9fb80cd240 2023-11-24T06:12:31.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:31.727+0000 7f9fbd345640 1 --2- 172.21.15.92:0/3551775322 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f9f9c06ae40 0x7f9f9c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:31.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:31.727+0000 7f9fbd345640 1 --2- 172.21.15.92:0/3551775322 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f9f9c06ae40 0x7f9f9c06d300 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f9fb4006fd0 tx=0x7f9fb4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:31.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:31.731+0000 7f9fadffb640 1 -- 172.21.15.92:0/3551775322 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9fb0053f70 con 0x7f9fb80cd240 2023-11-24T06:12:31.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:31.906+0000 7f9fbf5a9640 1 -- 172.21.15.92:0/3551775322 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9fb80c8490 con 0x7f9fb80cd240 2023-11-24T06:12:31.910 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:31.908+0000 7f9fadffb640 1 -- 172.21.15.92:0/3551775322 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v51) v1 ==== 74+0+16530 (secure 0 0 0) 0x7f9fb80c8490 con 0x7f9fb80cd240 2023-11-24T06:12:31.910 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-24T06:12:31.910 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":51,"fsid":"2d6dee5f-8aa3-470f-931a-50ade963b1cb","created":"2023-11-24T06:10:04.707204+0000","modified":"2023-11-24T06:12:30.604926+0000","last_up_change":"2023-11-24T06:10:17.598352+0000","last_in_change":"2023-11-24T06:10:09.040793+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:10:17.733098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:12:28.410305+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-11-24T06:12:29.625599+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ff37a9fb-135f-471f-b4b0-f7b5772006c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6803","nonce":20560714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6805","nonce":20560714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6809","nonce":20560714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6807","nonce":20560714}]},"public_addr":"172.21.15.92:6803/20560714","cluster_addr":"172.21.15.92:6805/20560714","heartbeat_back_addr":"172.21.15.92:6809/20560714","heartbeat_front_addr":"172.21.15.92:6807/20560714","state":["exists","up"]},{"osd":1,"uuid":"03cdd353-dcf1-4f38-94fe-a795137c06b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6819","nonce":2451239906}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6821","nonce":2451239906}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6825","nonce":2451239906}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6823","nonce":2451239906}]},"public_addr":"172.21.15.92:6819/2451239906","cluster_addr":"172.21.15.92:6821/2451239906","heartbeat_back_addr":"172.21.15.92:6825/2451239906","heartbeat_front_addr":"172.21.15.92:6823/2451239906","state":["exists","up"]},{"osd":2,"uuid":"a4a20d80-5d57-484b-becd-9aa8070dd3aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6811","nonce":1124979277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6813","nonce":1124979277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6817","nonce":1124979277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6815","nonce":1124979277}]},"public_addr":"172.21.15.92:6811/1124979277","cluster_addr":"172.21.15.92:6813/1124979277","heartbeat_back_addr":"172.21.15.92:6817/1124979277","heartbeat_front_addr":"172.21.15.92:6815/1124979277","state":["exists","up"]},{"osd":3,"uuid":"4648d7ce-9b29-4583-9425-4ba8fd2b1903","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6827","nonce":1734115782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6829","nonce":1734115782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6833","nonce":1734115782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6831","nonce":1734115782}]},"public_addr":"172.21.15.92:6827/1734115782","cluster_addr":"172.21.15.92:6829/1734115782","heartbeat_back_addr":"172.21.15.92:6833/1734115782","heartbeat_front_addr":"172.21.15.92:6831/1734115782","state":["exists","up"]},{"osd":4,"uuid":"3bc9d7eb-c6fa-4fcf-a3ec-ade604e05aac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6819","nonce":3425937229}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6822","nonce":3425937229}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6830","nonce":3425937229}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6826","nonce":3425937229}]},"public_addr":"172.21.15.118:6819/3425937229","cluster_addr":"172.21.15.118:6822/3425937229","heartbeat_back_addr":"172.21.15.118:6830/3425937229","heartbeat_front_addr":"172.21.15.118:6826/3425937229","state":["exists","up"]},{"osd":5,"uuid":"dce9ba6d-59ed-46d1-84fc-a84b13de10c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6801","nonce":827455984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6803","nonce":827455984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6807","nonce":827455984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6805","nonce":827455984}]},"public_addr":"172.21.15.118:6801/827455984","cluster_addr":"172.21.15.118:6803/827455984","heartbeat_back_addr":"172.21.15.118:6807/827455984","heartbeat_front_addr":"172.21.15.118:6805/827455984","state":["exists","up"]},{"osd":6,"uuid":"82aec504-ae58-4709-8d30-46f2b12081b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6809","nonce":2946986141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6811","nonce":2946986141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6815","nonce":2946986141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6813","nonce":2946986141}]},"public_addr":"172.21.15.118:6809/2946986141","cluster_addr":"172.21.15.118:6811/2946986141","heartbeat_back_addr":"172.21.15.118:6815/2946986141","heartbeat_front_addr":"172.21.15.118:6813/2946986141","state":["exists","up"]},{"osd":7,"uuid":"4ed4c624-c88f-4d35-bdb1-bf209b5dffb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6817","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6818","nonce":48122059}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6821","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6823","nonce":48122059}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6829","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6831","nonce":48122059}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6825","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6827","nonce":48122059}]},"public_addr":"172.21.15.118:6818/48122059","cluster_addr":"172.21.15.118:6823/48122059","heartbeat_back_addr":"172.21.15.118:6831/48122059","heartbeat_front_addr":"172.21.15.118:6827/48122059","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.536664+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.368324+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.271201+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.342971+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.621832+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.387110+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.632946+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.416032+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:12:31.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:31.910+0000 7f9fbf5a9640 1 -- 172.21.15.92:0/3551775322 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f9f9c06ae40 msgr2=0x7f9f9c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:31.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:31.910+0000 7f9fbf5a9640 1 --2- 172.21.15.92:0/3551775322 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f9f9c06ae40 0x7f9f9c06d300 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f9fb4006fd0 tx=0x7f9fb4008040 comp rx=0 tx=0).stop 2023-11-24T06:12:31.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:31.910+0000 7f9fbf5a9640 1 -- 172.21.15.92:0/3551775322 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9fb80cd240 msgr2=0x7f9fb80d0310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:31.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:31.910+0000 7f9fbf5a9640 1 --2- 172.21.15.92:0/3551775322 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9fb80cd240 0x7f9fb80d0310 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f9fb0001560 tx=0x7f9fb000cc70 comp rx=0 tx=0).stop 2023-11-24T06:12:31.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:31.911+0000 7f9fbf5a9640 1 -- 172.21.15.92:0/3551775322 shutdown_connections 2023-11-24T06:12:31.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:31.911+0000 7f9fbf5a9640 1 --2- 172.21.15.92:0/3551775322 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f9f9c06ae40 0x7f9f9c06d300 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:31.914 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:31.911+0000 7f9fbf5a9640 1 --2- 172.21.15.92:0/3551775322 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9fb80cd240 0x7f9fb80d0310 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:31.914 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:31.911+0000 7f9fbf5a9640 1 --2- 172.21.15.92:0/3551775322 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f9fb8099fa0 0x7f9fb80c7550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:31.914 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:31.911+0000 7f9fbf5a9640 1 --2- 172.21.15.92:0/3551775322 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9fb806c6b0 0x7f9fb80c5d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:31.914 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:31.911+0000 7f9fbf5a9640 1 -- 172.21.15.92:0/3551775322 >> 172.21.15.92:0/3551775322 conn(0x7f9fb80b2d00 msgr2=0x7f9fb80aff90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:31.914 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:31.911+0000 7f9fbf5a9640 1 -- 172.21.15.92:0/3551775322 shutdown_connections 2023-11-24T06:12:31.914 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:31.911+0000 7f9fbf5a9640 1 -- 172.21.15.92:0/3551775322 wait complete. 2023-11-24T06:12:31.930 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:12:32.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:32.141+0000 7fa1e6456640 1 Processor -- start 2023-11-24T06:12:32.144 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:32.141+0000 7fa1e6456640 1 -- start start 2023-11-24T06:12:32.146 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:32.143+0000 7fa1e6456640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa1e013df80 0x7fa1e015bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:32.146 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:32.143+0000 7fa1e6456640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fa1e013e360 0x7fa1e00cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:32.146 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:32.143+0000 7fa1e6456640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1e006c470 con 0x7fa1e015f9c0 2023-11-24T06:12:32.146 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:32.143+0000 7fa1e6456640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1e006c750 con 0x7fa1e013e360 2023-11-24T06:12:32.146 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:32.143+0000 7fa1e6456640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1e006ca30 con 0x7fa1e013df80 2023-11-24T06:12:32.146 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:32.143+0000 7fa1df7fe640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fa1e013e360 0x7fa1e00cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:32.146 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:32.143+0000 7fa1df7fe640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fa1e013e360 0x7fa1e00cf290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:49830/0 (socket says 172.21.15.92:49830) 2023-11-24T06:12:32.146 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:32.143+0000 7fa1e49f3640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fa1e015f9c0 0x7fa1e01408d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:49846/0 (socket says 172.21.15.92:49846) 2023-11-24T06:12:32.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:32.143+0000 7fa1df7fe640 1 -- 172.21.15.92:0/2732584381 learned_addr learned my addr 172.21.15.92:0/2732584381 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:12:32.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:32.143+0000 7fa1dffff640 1 --2- 172.21.15.92:0/2732584381 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa1e013df80 0x7fa1e015bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:32.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:32.144+0000 7fa1df7fe640 1 -- 172.21.15.92:0/2732584381 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa1e013df80 msgr2=0x7fa1e015bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:32.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:32.144+0000 7fa1df7fe640 1 --2- 172.21.15.92:0/2732584381 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa1e013df80 0x7fa1e015bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:32.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:32.144+0000 7fa1df7fe640 1 -- 172.21.15.92:0/2732584381 >> v1:172.21.15.92:6789/0 conn(0x7fa1e015f9c0 legacy=0x7fa1e01408d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:32.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:32.144+0000 7fa1df7fe640 1 -- 172.21.15.92:0/2732584381 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1e006cd10 con 0x7fa1e013e360 2023-11-24T06:12:32.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:32.144+0000 7fa1deffd640 1 -- 172.21.15.92:0/2732584381 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3218772089 0 0) 0x7fa1e006c470 con 0x7fa1e015f9c0 2023-11-24T06:12:32.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:32.145+0000 7fa1df7fe640 1 --2- 172.21.15.92:0/2732584381 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa1e013e360 0x7fa1e00cf290 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fa1c8009870 tx=0x7fa1c802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d27fe81b4cc292dc server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:32.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:32.146+0000 7fa1deffd640 1 -- 172.21.15.92:0/2732584381 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1c8030840 con 0x7fa1e013e360 2023-11-24T06:12:32.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:32.146+0000 7fa1deffd640 1 -- 172.21.15.92:0/2732584381 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1c80309a0 con 0x7fa1e013e360 2023-11-24T06:12:32.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:32.146+0000 7fa1deffd640 1 -- 172.21.15.92:0/2732584381 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1c8030d30 con 0x7fa1e013e360 2023-11-24T06:12:32.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:32.146+0000 7fa1e6456640 1 -- 172.21.15.92:0/2732584381 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa1e013e360 msgr2=0x7fa1e00cf290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:32.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:32.146+0000 7fa1e6456640 1 --2- 172.21.15.92:0/2732584381 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa1e013e360 0x7fa1e00cf290 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fa1c8009870 tx=0x7fa1c802b980 comp rx=0 tx=0).stop 2023-11-24T06:12:32.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:32.146+0000 7fa1e6456640 1 -- 172.21.15.92:0/2732584381 shutdown_connections 2023-11-24T06:12:32.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:32.146+0000 7fa1e6456640 1 --2- 172.21.15.92:0/2732584381 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa1e013e360 0x7fa1e00cf290 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:32.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:32.146+0000 7fa1e6456640 1 --2- 172.21.15.92:0/2732584381 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa1e013df80 0x7fa1e015bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:32.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:32.146+0000 7fa1e6456640 1 -- 172.21.15.92:0/2732584381 >> 172.21.15.92:0/2732584381 conn(0x7fa1e006a800 msgr2=0x7fa1e006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:32.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:32.146+0000 7fa1e6456640 1 -- 172.21.15.92:0/2732584381 shutdown_connections 2023-11-24T06:12:32.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:32.146+0000 7fa1e6456640 1 -- 172.21.15.92:0/2732584381 wait complete. 2023-11-24T06:12:32.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:32.148+0000 7fa1e6456640 1 Processor -- start 2023-11-24T06:12:32.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:32.148+0000 7fa1e6456640 1 -- start start 2023-11-24T06:12:32.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:32.149+0000 7fa1e6456640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa1e013df80 0x7fa1e01336c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:32.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:32.149+0000 7fa1e6456640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa1e013e360 0x7fa1e014f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:32.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:32.149+0000 7fa1e6456640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa1e015f9c0 0x7fa1e01529c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:32.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:32.149+0000 7fa1e6456640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1e0133c00 con 0x7fa1e015f9c0 2023-11-24T06:12:32.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:32.149+0000 7fa1e6456640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1e0150170 con 0x7fa1e013e360 2023-11-24T06:12:32.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:32.149+0000 7fa1e6456640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1e014e490 con 0x7fa1e013df80 2023-11-24T06:12:32.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:32.149+0000 7fa1e49f3640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa1e015f9c0 0x7fa1e01529c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:32.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:32.149+0000 7fa1e49f3640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa1e015f9c0 0x7fa1e01529c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:49844/0 (socket says 172.21.15.92:49844) 2023-11-24T06:12:32.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:32.149+0000 7fa1e49f3640 1 -- 172.21.15.92:0/1257948378 learned_addr learned my addr 172.21.15.92:0/1257948378 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:12:32.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:32.150+0000 7fa1df7fe640 1 --2- 172.21.15.92:0/1257948378 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa1e013e360 0x7fa1e014f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:32.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:32.150+0000 7fa1dffff640 1 --2- 172.21.15.92:0/1257948378 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa1e013df80 0x7fa1e01336c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:32.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:32.150+0000 7fa1e49f3640 1 -- 172.21.15.92:0/1257948378 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa1e013df80 msgr2=0x7fa1e01336c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:32.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:32.150+0000 7fa1e49f3640 1 --2- 172.21.15.92:0/1257948378 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa1e013df80 0x7fa1e01336c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:32.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:32.150+0000 7fa1e49f3640 1 -- 172.21.15.92:0/1257948378 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa1e013e360 msgr2=0x7fa1e014f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:32.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:32.150+0000 7fa1e49f3640 1 --2- 172.21.15.92:0/1257948378 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa1e013e360 0x7fa1e014f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:32.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:32.150+0000 7fa1e49f3640 1 -- 172.21.15.92:0/1257948378 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1c80094d0 con 0x7fa1e015f9c0 2023-11-24T06:12:32.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:32.150+0000 7fa1e49f3640 1 --2- 172.21.15.92:0/1257948378 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa1e015f9c0 0x7fa1e01529c0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fa1d800c450 tx=0x7fa1d800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:32.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:32.150+0000 7fa1dcff9640 1 -- 172.21.15.92:0/1257948378 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1d80074e0 con 0x7fa1e015f9c0 2023-11-24T06:12:32.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:32.150+0000 7fa1dcff9640 1 -- 172.21.15.92:0/1257948378 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1d8007660 con 0x7fa1e015f9c0 2023-11-24T06:12:32.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:32.150+0000 7fa1dcff9640 1 -- 172.21.15.92:0/1257948378 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1d8004a20 con 0x7fa1e015f9c0 2023-11-24T06:12:32.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:32.151+0000 7fa1e6456640 1 -- 172.21.15.92:0/1257948378 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1e014e770 con 0x7fa1e015f9c0 2023-11-24T06:12:32.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:32.151+0000 7fa1e6456640 1 -- 172.21.15.92:0/1257948378 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1e014ed30 con 0x7fa1e015f9c0 2023-11-24T06:12:32.155 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:32.152+0000 7fa1e6456640 1 -- 172.21.15.92:0/1257948378 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1e014ef20 con 0x7fa1e015f9c0 2023-11-24T06:12:32.155 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:32.153+0000 7fa1dcff9640 1 -- 172.21.15.92:0/1257948378 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa1d8004b80 con 0x7fa1e015f9c0 2023-11-24T06:12:32.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:32.154+0000 7fa1dcff9640 1 --2- 172.21.15.92:0/1257948378 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fa1c006ae70 0x7fa1c006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:32.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:32.154+0000 7fa1dcff9640 1 -- 172.21.15.92:0/1257948378 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa1d8088700 con 0x7fa1e015f9c0 2023-11-24T06:12:32.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:32.154+0000 7fa1dffff640 1 --2- 172.21.15.92:0/1257948378 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fa1c006ae70 0x7fa1c006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:32.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:32.154+0000 7fa1dffff640 1 --2- 172.21.15.92:0/1257948378 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fa1c006ae70 0x7fa1c006d330 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fa1d00099c0 tx=0x7fa1d0003990 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:32.160 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:32.158+0000 7fa1dcff9640 1 -- 172.21.15.92:0/1257948378 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa1d8053710 con 0x7fa1e015f9c0 2023-11-24T06:12:32.352 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:32.350+0000 7fa1e6456640 1 -- 172.21.15.92:0/1257948378 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa1e00675d0 con 0x7fa1e015f9c0 2023-11-24T06:12:32.354 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:32.352+0000 7fa1dcff9640 1 -- 172.21.15.92:0/1257948378 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v52) v1 ==== 74+0+16517 (secure 0 0 0) 0x7fa1d8056d30 con 0x7fa1e015f9c0 2023-11-24T06:12:32.355 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-24T06:12:32.355 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":52,"fsid":"2d6dee5f-8aa3-470f-931a-50ade963b1cb","created":"2023-11-24T06:10:04.707204+0000","modified":"2023-11-24T06:12:32.152035+0000","last_up_change":"2023-11-24T06:10:17.598352+0000","last_in_change":"2023-11-24T06:10:09.040793+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:10:17.733098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:12:28.410305+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-11-24T06:12:29.625599+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ff37a9fb-135f-471f-b4b0-f7b5772006c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6803","nonce":20560714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6805","nonce":20560714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6809","nonce":20560714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6807","nonce":20560714}]},"public_addr":"172.21.15.92:6803/20560714","cluster_addr":"172.21.15.92:6805/20560714","heartbeat_back_addr":"172.21.15.92:6809/20560714","heartbeat_front_addr":"172.21.15.92:6807/20560714","state":["exists","up"]},{"osd":1,"uuid":"03cdd353-dcf1-4f38-94fe-a795137c06b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6819","nonce":2451239906}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6821","nonce":2451239906}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6825","nonce":2451239906}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6823","nonce":2451239906}]},"public_addr":"172.21.15.92:6819/2451239906","cluster_addr":"172.21.15.92:6821/2451239906","heartbeat_back_addr":"172.21.15.92:6825/2451239906","heartbeat_front_addr":"172.21.15.92:6823/2451239906","state":["exists","up"]},{"osd":2,"uuid":"a4a20d80-5d57-484b-becd-9aa8070dd3aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6811","nonce":1124979277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6813","nonce":1124979277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6817","nonce":1124979277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6815","nonce":1124979277}]},"public_addr":"172.21.15.92:6811/1124979277","cluster_addr":"172.21.15.92:6813/1124979277","heartbeat_back_addr":"172.21.15.92:6817/1124979277","heartbeat_front_addr":"172.21.15.92:6815/1124979277","state":["exists","up"]},{"osd":3,"uuid":"4648d7ce-9b29-4583-9425-4ba8fd2b1903","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6827","nonce":1734115782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6829","nonce":1734115782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6833","nonce":1734115782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6831","nonce":1734115782}]},"public_addr":"172.21.15.92:6827/1734115782","cluster_addr":"172.21.15.92:6829/1734115782","heartbeat_back_addr":"172.21.15.92:6833/1734115782","heartbeat_front_addr":"172.21.15.92:6831/1734115782","state":["exists","up"]},{"osd":4,"uuid":"3bc9d7eb-c6fa-4fcf-a3ec-ade604e05aac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6819","nonce":3425937229}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6822","nonce":3425937229}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6830","nonce":3425937229}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6826","nonce":3425937229}]},"public_addr":"172.21.15.118:6819/3425937229","cluster_addr":"172.21.15.118:6822/3425937229","heartbeat_back_addr":"172.21.15.118:6830/3425937229","heartbeat_front_addr":"172.21.15.118:6826/3425937229","state":["exists","up"]},{"osd":5,"uuid":"dce9ba6d-59ed-46d1-84fc-a84b13de10c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6801","nonce":827455984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6803","nonce":827455984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6807","nonce":827455984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6805","nonce":827455984}]},"public_addr":"172.21.15.118:6801/827455984","cluster_addr":"172.21.15.118:6803/827455984","heartbeat_back_addr":"172.21.15.118:6807/827455984","heartbeat_front_addr":"172.21.15.118:6805/827455984","state":["exists","up"]},{"osd":6,"uuid":"82aec504-ae58-4709-8d30-46f2b12081b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6809","nonce":2946986141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6811","nonce":2946986141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6815","nonce":2946986141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6813","nonce":2946986141}]},"public_addr":"172.21.15.118:6809/2946986141","cluster_addr":"172.21.15.118:6811/2946986141","heartbeat_back_addr":"172.21.15.118:6815/2946986141","heartbeat_front_addr":"172.21.15.118:6813/2946986141","state":["exists","up"]},{"osd":7,"uuid":"4ed4c624-c88f-4d35-bdb1-bf209b5dffb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6817","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6818","nonce":48122059}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6821","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6823","nonce":48122059}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6829","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6831","nonce":48122059}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6825","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6827","nonce":48122059}]},"public_addr":"172.21.15.118:6818/48122059","cluster_addr":"172.21.15.118:6823/48122059","heartbeat_back_addr":"172.21.15.118:6831/48122059","heartbeat_front_addr":"172.21.15.118:6827/48122059","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.536664+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.368324+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.271201+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.342971+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.621832+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.387110+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.632946+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.416032+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:12:32.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:32.356+0000 7fa1e6456640 1 -- 172.21.15.92:0/1257948378 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fa1c006ae70 msgr2=0x7fa1c006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:32.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:32.356+0000 7fa1e6456640 1 --2- 172.21.15.92:0/1257948378 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fa1c006ae70 0x7fa1c006d330 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fa1d00099c0 tx=0x7fa1d0003990 comp rx=0 tx=0).stop 2023-11-24T06:12:32.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:32.356+0000 7fa1e6456640 1 -- 172.21.15.92:0/1257948378 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa1e015f9c0 msgr2=0x7fa1e01529c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:32.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:32.356+0000 7fa1e6456640 1 --2- 172.21.15.92:0/1257948378 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa1e015f9c0 0x7fa1e01529c0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fa1d800c450 tx=0x7fa1d800c910 comp rx=0 tx=0).stop 2023-11-24T06:12:32.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:32.356+0000 7fa1e6456640 1 -- 172.21.15.92:0/1257948378 shutdown_connections 2023-11-24T06:12:32.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:32.356+0000 7fa1e6456640 1 --2- 172.21.15.92:0/1257948378 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fa1c006ae70 0x7fa1c006d330 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:32.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:32.356+0000 7fa1e6456640 1 --2- 172.21.15.92:0/1257948378 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa1e015f9c0 0x7fa1e01529c0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:32.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:32.356+0000 7fa1e6456640 1 --2- 172.21.15.92:0/1257948378 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa1e013e360 0x7fa1e014f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:32.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:32.356+0000 7fa1e6456640 1 --2- 172.21.15.92:0/1257948378 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa1e013df80 0x7fa1e01336c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:32.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:32.356+0000 7fa1e6456640 1 -- 172.21.15.92:0/1257948378 >> 172.21.15.92:0/1257948378 conn(0x7fa1e006a800 msgr2=0x7fa1e015d700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:32.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:32.357+0000 7fa1e6456640 1 -- 172.21.15.92:0/1257948378 shutdown_connections 2023-11-24T06:12:32.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:32.357+0000 7fa1e6456640 1 -- 172.21.15.92:0/1257948378 wait complete. 2023-11-24T06:12:32.376 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-11-24T06:12:32.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:32.583+0000 7f2d6dd4c640 1 Processor -- start 2023-11-24T06:12:32.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:32.583+0000 7f2d6dd4c640 1 -- start start 2023-11-24T06:12:32.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:32.585+0000 7f2d6dd4c640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f2d6815da10 0x7f2d68072230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:32.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:32.585+0000 7f2d6dd4c640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f2d680b2bd0 0x7f2d680b3020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:32.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:32.585+0000 7f2d6dd4c640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d68071190 con 0x7f2d68150760 2023-11-24T06:12:32.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:32.585+0000 7f2d6dd4c640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d68071470 con 0x7f2d680b2bd0 2023-11-24T06:12:32.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:32.585+0000 7f2d6dd4c640 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d68071750 con 0x7f2d6815da10 2023-11-24T06:12:32.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:32.585+0000 7f2d67fff640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f2d680b2bd0 0x7f2d680b3020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:32.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:32.585+0000 7f2d677fe640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f2d68150760 0x7f2d6815d300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:49862/0 (socket says 172.21.15.92:49862) 2023-11-24T06:12:32.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:32.585+0000 7f2d677fe640 1 -- 172.21.15.92:0/3890908154 learned_addr learned my addr 172.21.15.92:0/3890908154 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-24T06:12:32.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:32.586+0000 7f2d5effd640 1 --2- 172.21.15.92:0/3890908154 >> v2:172.21.15.118:3300/0 conn(0x7f2d6815da10 0x7f2d68072230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:32.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:32.586+0000 7f2d67fff640 1 -- 172.21.15.92:0/3890908154 >> v2:172.21.15.118:3300/0 conn(0x7f2d6815da10 msgr2=0x7f2d68072230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:32.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:32.586+0000 7f2d67fff640 1 --2- 172.21.15.92:0/3890908154 >> v2:172.21.15.118:3300/0 conn(0x7f2d6815da10 0x7f2d68072230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:32.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:32.586+0000 7f2d67fff640 1 -- 172.21.15.92:0/3890908154 >> v1:172.21.15.92:6789/0 conn(0x7f2d68150760 legacy=0x7f2d6815d300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:32.589 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:32.586+0000 7f2d67fff640 1 -- 172.21.15.92:0/3890908154 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d68071a30 con 0x7f2d680b2bd0 2023-11-24T06:12:32.589 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:32.586+0000 7f2d66ffd640 1 -- 172.21.15.92:0/3890908154 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 460678588 0 0) 0x7f2d68071190 con 0x7f2d68150760 2023-11-24T06:12:32.589 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:32.587+0000 7f2d67fff640 1 --2- 172.21.15.92:0/3890908154 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2d680b2bd0 0x7f2d680b3020 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f2d680b1ca0 tx=0x7f2d6002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b55f46482e8c24d server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:32.589 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:32.587+0000 7f2d66ffd640 1 -- 172.21.15.92:0/3890908154 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d60031830 con 0x7f2d680b2bd0 2023-11-24T06:12:32.589 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:32.587+0000 7f2d66ffd640 1 -- 172.21.15.92:0/3890908154 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d60031990 con 0x7f2d680b2bd0 2023-11-24T06:12:32.589 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:32.587+0000 7f2d66ffd640 1 -- 172.21.15.92:0/3890908154 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d60031d20 con 0x7f2d680b2bd0 2023-11-24T06:12:32.590 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:32.588+0000 7f2d6dd4c640 1 -- 172.21.15.92:0/3890908154 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2d680b2bd0 msgr2=0x7f2d680b3020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:32.590 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:32.588+0000 7f2d6dd4c640 1 --2- 172.21.15.92:0/3890908154 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2d680b2bd0 0x7f2d680b3020 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f2d680b1ca0 tx=0x7f2d6002b8d0 comp rx=0 tx=0).stop 2023-11-24T06:12:32.590 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:32.588+0000 7f2d6dd4c640 1 -- 172.21.15.92:0/3890908154 shutdown_connections 2023-11-24T06:12:32.591 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:32.588+0000 7f2d6dd4c640 1 --2- 172.21.15.92:0/3890908154 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2d680b2bd0 0x7f2d680b3020 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:32.591 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:32.588+0000 7f2d6dd4c640 1 --2- 172.21.15.92:0/3890908154 >> v2:172.21.15.118:3300/0 conn(0x7f2d6815da10 0x7f2d68072230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:32.591 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:32.588+0000 7f2d6dd4c640 1 -- 172.21.15.92:0/3890908154 >> 172.21.15.92:0/3890908154 conn(0x7f2d6806a800 msgr2=0x7f2d6806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:32.591 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:32.588+0000 7f2d6dd4c640 1 -- 172.21.15.92:0/3890908154 shutdown_connections 2023-11-24T06:12:32.591 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:32.588+0000 7f2d6dd4c640 1 -- 172.21.15.92:0/3890908154 wait complete. 2023-11-24T06:12:32.592 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:32.590+0000 7f2d6dd4c640 1 Processor -- start 2023-11-24T06:12:32.592 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:32.590+0000 7f2d6dd4c640 1 -- start start 2023-11-24T06:12:32.593 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:32.591+0000 7f2d6dd4c640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2d680b2bd0 0x7f2d68156dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:32.593 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:32.591+0000 7f2d6dd4c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2d68150760 0x7f2d680ac8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:32.593 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:32.591+0000 7f2d6dd4c640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2d6815da10 0x7f2d68155b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:32.593 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:32.591+0000 7f2d6dd4c640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d68156050 con 0x7f2d68150760 2023-11-24T06:12:32.593 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:32.591+0000 7f2d6dd4c640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d68156280 con 0x7f2d6815da10 2023-11-24T06:12:32.593 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:32.591+0000 7f2d6dd4c640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d68157a40 con 0x7f2d680b2bd0 2023-11-24T06:12:32.594 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:32.591+0000 7f2d5effd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2d68150760 0x7f2d680ac8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:32.594 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:32.591+0000 7f2d5effd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2d68150760 0x7f2d680ac8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:49856/0 (socket says 172.21.15.92:49856) 2023-11-24T06:12:32.594 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:32.591+0000 7f2d5effd640 1 -- 172.21.15.92:0/1358844699 learned_addr learned my addr 172.21.15.92:0/1358844699 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:12:32.595 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:32.591+0000 7f2d5effd640 1 -- 172.21.15.92:0/1358844699 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2d680b2bd0 msgr2=0x7f2d68156dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:32.595 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:32.591+0000 7f2d677fe640 1 --2- 172.21.15.92:0/1358844699 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2d680b2bd0 0x7f2d68156dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:32.595 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:32.591+0000 7f2d67fff640 1 --2- 172.21.15.92:0/1358844699 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2d6815da10 0x7f2d68155b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:32.595 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:32.591+0000 7f2d5effd640 1 --2- 172.21.15.92:0/1358844699 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2d680b2bd0 0x7f2d68156dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:32.595 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:32.591+0000 7f2d5effd640 1 -- 172.21.15.92:0/1358844699 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2d6815da10 msgr2=0x7f2d68155b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:32.595 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:32.591+0000 7f2d5effd640 1 --2- 172.21.15.92:0/1358844699 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2d6815da10 0x7f2d68155b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:32.595 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:32.591+0000 7f2d5effd640 1 -- 172.21.15.92:0/1358844699 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d60009400 con 0x7f2d68150760 2023-11-24T06:12:32.595 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:32.592+0000 7f2d5effd640 1 --2- 172.21.15.92:0/1358844699 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2d68150760 0x7f2d680ac8d0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f2d5000d5f0 tx=0x7f2d5000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:32.595 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:32.592+0000 7f2d64ff9640 1 -- 172.21.15.92:0/1358844699 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d500125f0 con 0x7f2d68150760 2023-11-24T06:12:32.595 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:32.592+0000 7f2d64ff9640 1 -- 172.21.15.92:0/1358844699 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d50012750 con 0x7f2d68150760 2023-11-24T06:12:32.596 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:32.592+0000 7f2d64ff9640 1 -- 172.21.15.92:0/1358844699 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d50012b00 con 0x7f2d68150760 2023-11-24T06:12:32.596 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:32.592+0000 7f2d6dd4c640 1 -- 172.21.15.92:0/1358844699 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d68157d20 con 0x7f2d68150760 2023-11-24T06:12:32.596 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:32.592+0000 7f2d6dd4c640 1 -- 172.21.15.92:0/1358844699 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d68142ad0 con 0x7f2d68150760 2023-11-24T06:12:32.596 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:32.594+0000 7f2d6dd4c640 1 -- 172.21.15.92:0/1358844699 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d681578a0 con 0x7f2d68150760 2023-11-24T06:12:32.596 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:32.594+0000 7f2d64ff9640 1 -- 172.21.15.92:0/1358844699 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2d50012c60 con 0x7f2d68150760 2023-11-24T06:12:32.597 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:32.595+0000 7f2d64ff9640 1 --2- 172.21.15.92:0/1358844699 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f2d4806ae70 0x7f2d4806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:32.597 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:32.595+0000 7f2d64ff9640 1 -- 172.21.15.92:0/1358844699 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2d50089a10 con 0x7f2d68150760 2023-11-24T06:12:32.597 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:32.596+0000 7f2d677fe640 1 --2- 172.21.15.92:0/1358844699 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f2d4806ae70 0x7f2d4806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:32.598 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:32.596+0000 7f2d677fe640 1 --2- 172.21.15.92:0/1358844699 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f2d4806ae70 0x7f2d4806d330 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f2d580012e0 tx=0x7f2d58009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:32.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:32.600+0000 7f2d64ff9640 1 -- 172.21.15.92:0/1358844699 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2d681578a0 con 0x7f2d68150760 2023-11-24T06:12:32.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:32.832+0000 7f2d6dd4c640 1 -- 172.21.15.92:0/1358844699 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f2d681578a0 con 0x7f2d68150760 2023-11-24T06:12:33.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:33.527+0000 7f2d64ff9640 1 -- 172.21.15.92:0/1358844699 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v30) v1 ==== 118+0+0 (secure 0 0 0) 0x7f2d681578a0 con 0x7f2d68150760 2023-11-24T06:12:33.534 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:33.531+0000 7f2d6dd4c640 1 -- 172.21.15.92:0/1358844699 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f2d4806ae70 msgr2=0x7f2d4806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:33.534 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:33.531+0000 7f2d6dd4c640 1 --2- 172.21.15.92:0/1358844699 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f2d4806ae70 0x7f2d4806d330 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f2d580012e0 tx=0x7f2d58009040 comp rx=0 tx=0).stop 2023-11-24T06:12:33.534 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:33.531+0000 7f2d6dd4c640 1 -- 172.21.15.92:0/1358844699 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2d68150760 msgr2=0x7f2d680ac8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:33.534 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:33.531+0000 7f2d6dd4c640 1 --2- 172.21.15.92:0/1358844699 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2d68150760 0x7f2d680ac8d0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f2d5000d5f0 tx=0x7f2d5000dab0 comp rx=0 tx=0).stop 2023-11-24T06:12:33.534 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:33.531+0000 7f2d6dd4c640 1 -- 172.21.15.92:0/1358844699 shutdown_connections 2023-11-24T06:12:33.534 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:33.531+0000 7f2d6dd4c640 1 --2- 172.21.15.92:0/1358844699 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f2d4806ae70 0x7f2d4806d330 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:33.534 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:33.531+0000 7f2d6dd4c640 1 --2- 172.21.15.92:0/1358844699 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2d6815da10 0x7f2d68155b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:33.534 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:33.531+0000 7f2d6dd4c640 1 --2- 172.21.15.92:0/1358844699 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2d68150760 0x7f2d680ac8d0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:33.534 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:33.531+0000 7f2d6dd4c640 1 --2- 172.21.15.92:0/1358844699 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2d680b2bd0 0x7f2d68156dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:33.534 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:33.531+0000 7f2d6dd4c640 1 -- 172.21.15.92:0/1358844699 >> 172.21.15.92:0/1358844699 conn(0x7f2d6806a800 msgr2=0x7f2d6806c470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:33.534 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:33.532+0000 7f2d6dd4c640 1 -- 172.21.15.92:0/1358844699 shutdown_connections 2023-11-24T06:12:33.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:33.532+0000 7f2d6dd4c640 1 -- 172.21.15.92:0/1358844699 wait complete. 2023-11-24T06:12:33.554 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-11-24T06:12:33.554 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-11-24T06:12:33.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:33.772+0000 7f0b6e8e7640 1 Processor -- start 2023-11-24T06:12:33.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:33.772+0000 7f0b6e8e7640 1 -- start start 2023-11-24T06:12:33.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:33.773+0000 7f0b6e8e7640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f0b680cd240 0x7f0b680b3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:33.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:33.773+0000 7f0b6e8e7640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0b6806c6b0 0x7f0b68099a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:33.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:33.773+0000 7f0b6e8e7640 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b6806cea0 con 0x7f0b68099fa0 2023-11-24T06:12:33.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:33.773+0000 7f0b6e8e7640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b6809a430 con 0x7f0b680cd240 2023-11-24T06:12:33.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:33.773+0000 7f0b6e8e7640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b6806cbc0 con 0x7f0b6806c6b0 2023-11-24T06:12:33.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:33.774+0000 7f0b67fff640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f0b680cd240 0x7f0b680b3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:33.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:33.774+0000 7f0b67fff640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f0b680cd240 0x7f0b680b3820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:49872/0 (socket says 172.21.15.92:49872) 2023-11-24T06:12:33.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:33.774+0000 7f0b67fff640 1 -- 172.21.15.92:0/40317978 learned_addr learned my addr 172.21.15.92:0/40317978 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:12:33.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:33.774+0000 7f0b67fff640 1 -- 172.21.15.92:0/40317978 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0b6806c6b0 msgr2=0x7f0b68099a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:33.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:33.774+0000 7f0b67fff640 1 --2- 172.21.15.92:0/40317978 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0b6806c6b0 0x7f0b68099a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:33.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:33.774+0000 7f0b67fff640 1 -- 172.21.15.92:0/40317978 >> v1:172.21.15.118:6789/0 conn(0x7f0b68099fa0 legacy=0x7f0b6806b4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:33.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:33.774+0000 7f0b67fff640 1 -- 172.21.15.92:0/40317978 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b6806f350 con 0x7f0b680cd240 2023-11-24T06:12:33.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:33.775+0000 7f0b67fff640 1 --2- 172.21.15.92:0/40317978 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0b680cd240 0x7f0b680b3820 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f0b58009e50 tx=0x7f0b5802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46c4ab44987ede7d server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:33.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:33.775+0000 7f0b66ffd640 1 -- 172.21.15.92:0/40317978 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b5802ca70 con 0x7f0b680cd240 2023-11-24T06:12:33.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:33.775+0000 7f0b66ffd640 1 -- 172.21.15.92:0/40317978 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b5802cbd0 con 0x7f0b680cd240 2023-11-24T06:12:33.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:33.775+0000 7f0b66ffd640 1 -- 172.21.15.92:0/40317978 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b580343f0 con 0x7f0b680cd240 2023-11-24T06:12:33.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:33.776+0000 7f0b6e8e7640 1 -- 172.21.15.92:0/40317978 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0b680cd240 msgr2=0x7f0b680b3820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:33.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:33.776+0000 7f0b6e8e7640 1 --2- 172.21.15.92:0/40317978 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0b680cd240 0x7f0b680b3820 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f0b58009e50 tx=0x7f0b5802bed0 comp rx=0 tx=0).stop 2023-11-24T06:12:33.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:33.776+0000 7f0b6e8e7640 1 -- 172.21.15.92:0/40317978 shutdown_connections 2023-11-24T06:12:33.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:33.776+0000 7f0b6e8e7640 1 --2- 172.21.15.92:0/40317978 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0b6806c6b0 0x7f0b68099a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:33.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:33.776+0000 7f0b6e8e7640 1 --2- 172.21.15.92:0/40317978 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0b680cd240 0x7f0b680b3820 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:33.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:33.776+0000 7f0b6e8e7640 1 -- 172.21.15.92:0/40317978 >> 172.21.15.92:0/40317978 conn(0x7f0b680b2d00 msgr2=0x7f0b680b3110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:33.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:33.776+0000 7f0b6e8e7640 1 -- 172.21.15.92:0/40317978 shutdown_connections 2023-11-24T06:12:33.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:33.776+0000 7f0b6e8e7640 1 -- 172.21.15.92:0/40317978 wait complete. 2023-11-24T06:12:33.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:33.778+0000 7f0b6e8e7640 1 Processor -- start 2023-11-24T06:12:33.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:33.778+0000 7f0b6e8e7640 1 -- start start 2023-11-24T06:12:33.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:33.779+0000 7f0b6e8e7640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0b6806c6b0 0x7f0b680c66f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:33.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:33.779+0000 7f0b6e8e7640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0b68099fa0 0x7f0b68158bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:33.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:33.779+0000 7f0b6e8e7640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0b680cd240 0x7f0b68159110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:33.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:33.779+0000 7f0b6e8e7640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b680b41f0 con 0x7f0b680cd240 2023-11-24T06:12:33.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:33.779+0000 7f0b6e8e7640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b680b4d70 con 0x7f0b68099fa0 2023-11-24T06:12:33.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:33.779+0000 7f0b6e8e7640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b68159650 con 0x7f0b6806c6b0 2023-11-24T06:12:33.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:33.779+0000 7f0b6ce84640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0b680cd240 0x7f0b68159110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:33.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:33.779+0000 7f0b6ce84640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0b680cd240 0x7f0b68159110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:49888/0 (socket says 172.21.15.92:49888) 2023-11-24T06:12:33.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:33.779+0000 7f0b6ce84640 1 -- 172.21.15.92:0/479555292 learned_addr learned my addr 172.21.15.92:0/479555292 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:12:33.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:33.780+0000 7f0b6ce84640 1 -- 172.21.15.92:0/479555292 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0b6806c6b0 msgr2=0x7f0b680c66f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:33.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:33.780+0000 7f0b6ce84640 1 --2- 172.21.15.92:0/479555292 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0b6806c6b0 0x7f0b680c66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:33.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:33.780+0000 7f0b6ce84640 1 -- 172.21.15.92:0/479555292 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0b68099fa0 msgr2=0x7f0b68158bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:33.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:33.780+0000 7f0b677fe640 1 --2- 172.21.15.92:0/479555292 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0b68099fa0 0x7f0b68158bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:33.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:33.780+0000 7f0b6ce84640 1 --2- 172.21.15.92:0/479555292 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0b68099fa0 0x7f0b68158bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:33.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:33.780+0000 7f0b6ce84640 1 -- 172.21.15.92:0/479555292 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b58009b30 con 0x7f0b680cd240 2023-11-24T06:12:33.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:33.780+0000 7f0b6ce84640 1 --2- 172.21.15.92:0/479555292 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0b680cd240 0x7f0b68159110 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f0b6000b4f0 tx=0x7f0b6000b9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:33.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:33.780+0000 7f0b64ff9640 1 -- 172.21.15.92:0/479555292 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b60011830 con 0x7f0b680cd240 2023-11-24T06:12:33.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:33.780+0000 7f0b64ff9640 1 -- 172.21.15.92:0/479555292 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b60011990 con 0x7f0b680cd240 2023-11-24T06:12:33.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:33.780+0000 7f0b64ff9640 1 -- 172.21.15.92:0/479555292 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b60011d40 con 0x7f0b680cd240 2023-11-24T06:12:33.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:33.781+0000 7f0b6e8e7640 1 -- 172.21.15.92:0/479555292 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b680b4440 con 0x7f0b680cd240 2023-11-24T06:12:33.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:33.781+0000 7f0b6e8e7640 1 -- 172.21.15.92:0/479555292 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b680b4a00 con 0x7f0b680cd240 2023-11-24T06:12:33.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:33.782+0000 7f0b6e8e7640 1 -- 172.21.15.92:0/479555292 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b680675d0 con 0x7f0b680cd240 2023-11-24T06:12:33.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:33.783+0000 7f0b64ff9640 1 -- 172.21.15.92:0/479555292 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f0b6001a400 con 0x7f0b680cd240 2023-11-24T06:12:33.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:33.784+0000 7f0b64ff9640 1 --2- 172.21.15.92:0/479555292 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f0b4806ae40 0x7f0b4806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:33.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:33.784+0000 7f0b64ff9640 1 -- 172.21.15.92:0/479555292 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0b600896d0 con 0x7f0b680cd240 2023-11-24T06:12:33.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:33.784+0000 7f0b67fff640 1 --2- 172.21.15.92:0/479555292 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f0b4806ae40 0x7f0b4806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:33.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:33.784+0000 7f0b67fff640 1 --2- 172.21.15.92:0/479555292 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f0b4806ae40 0x7f0b4806d300 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f0b58035000 tx=0x7f0b5802e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:33.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:33.788+0000 7f0b64ff9640 1 -- 172.21.15.92:0/479555292 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0b60054780 con 0x7f0b680cd240 2023-11-24T06:12:34.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:34.014+0000 7f0b6e8e7640 1 -- 172.21.15.92:0/479555292 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f0b680b4eb0 con 0x7f0b680cd240 2023-11-24T06:12:34.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:34.538+0000 7f0b64ff9640 1 -- 172.21.15.92:0/479555292 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v31) v1 ==== 122+0+0 (secure 0 0 0) 0x7f0b680b4eb0 con 0x7f0b680cd240 2023-11-24T06:12:34.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:34.543+0000 7f0b6e8e7640 1 -- 172.21.15.92:0/479555292 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f0b4806ae40 msgr2=0x7f0b4806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:34.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:34.543+0000 7f0b6e8e7640 1 --2- 172.21.15.92:0/479555292 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f0b4806ae40 0x7f0b4806d300 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f0b58035000 tx=0x7f0b5802e000 comp rx=0 tx=0).stop 2023-11-24T06:12:34.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:34.543+0000 7f0b6e8e7640 1 -- 172.21.15.92:0/479555292 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0b680cd240 msgr2=0x7f0b68159110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:34.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:34.543+0000 7f0b6e8e7640 1 --2- 172.21.15.92:0/479555292 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0b680cd240 0x7f0b68159110 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f0b6000b4f0 tx=0x7f0b6000b9b0 comp rx=0 tx=0).stop 2023-11-24T06:12:34.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:34.543+0000 7f0b6e8e7640 1 -- 172.21.15.92:0/479555292 shutdown_connections 2023-11-24T06:12:34.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:34.543+0000 7f0b6e8e7640 1 --2- 172.21.15.92:0/479555292 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f0b4806ae40 0x7f0b4806d300 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:34.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:34.543+0000 7f0b6e8e7640 1 --2- 172.21.15.92:0/479555292 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0b680cd240 0x7f0b68159110 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:34.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:34.543+0000 7f0b6e8e7640 1 --2- 172.21.15.92:0/479555292 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0b68099fa0 0x7f0b68158bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:34.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:34.543+0000 7f0b6e8e7640 1 --2- 172.21.15.92:0/479555292 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0b6806c6b0 0x7f0b680c66f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:34.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:34.543+0000 7f0b6e8e7640 1 -- 172.21.15.92:0/479555292 >> 172.21.15.92:0/479555292 conn(0x7f0b680b2d00 msgr2=0x7f0b681607c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:34.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:34.544+0000 7f0b6e8e7640 1 -- 172.21.15.92:0/479555292 shutdown_connections 2023-11-24T06:12:34.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:34.544+0000 7f0b6e8e7640 1 -- 172.21.15.92:0/479555292 wait complete. 2023-11-24T06:12:34.566 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-11-24T06:12:34.567 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T06:12:34.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:34.777+0000 7f90cca1e640 1 Processor -- start 2023-11-24T06:12:34.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:34.777+0000 7f90cca1e640 1 -- start start 2023-11-24T06:12:34.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:34.778+0000 7f90cca1e640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f90c814b2c0 0x7f90c80cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:34.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:34.778+0000 7f90cca1e640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f90c80cf7d0 0x7f90c8070960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:34.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:34.778+0000 7f90cca1e640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90c80cfbf0 con 0x7f90c8070ea0 2023-11-24T06:12:34.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:34.778+0000 7f90cca1e640 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90c814b6a0 con 0x7f90c80cf7d0 2023-11-24T06:12:34.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:34.778+0000 7f90cca1e640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90c8071690 con 0x7f90c814b2c0 2023-11-24T06:12:34.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:34.779+0000 7f90c6d9d640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f90c8070ea0 0x7f90c80b1ee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:49878/0 (socket says 172.21.15.92:49878) 2023-11-24T06:12:34.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:34.779+0000 7f90c6d9d640 1 -- 172.21.15.92:0/2850887513 learned_addr learned my addr 172.21.15.92:0/2850887513 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-24T06:12:34.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:34.779+0000 7f90bffff640 1 --2- 172.21.15.92:0/2850887513 >> v2:172.21.15.118:3300/0 conn(0x7f90c80cf7d0 0x7f90c8070960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:34.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:34.779+0000 7f90c659c640 1 --2- 172.21.15.92:0/2850887513 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f90c814b2c0 0x7f90c80cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:34.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:34.779+0000 7f90c5d9b640 1 -- 172.21.15.92:0/2850887513 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3337263857 0 0) 0x7f90c80cfbf0 con 0x7f90c8070ea0 2023-11-24T06:12:34.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:34.779+0000 7f90c5d9b640 1 -- 172.21.15.92:0/2850887513 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f90b4003520 con 0x7f90c8070ea0 2023-11-24T06:12:34.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:34.780+0000 7f90c5d9b640 1 -- 172.21.15.92:0/2850887513 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3124047686 0 0) 0x7f90b4003520 con 0x7f90c8070ea0 2023-11-24T06:12:34.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:34.780+0000 7f90c5d9b640 1 -- 172.21.15.92:0/2850887513 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f90c814b2c0 msgr2=0x7f90c80cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:34.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:34.780+0000 7f90c5d9b640 1 --2- 172.21.15.92:0/2850887513 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f90c814b2c0 0x7f90c80cf290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:34.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:34.780+0000 7f90c5d9b640 1 -- 172.21.15.92:0/2850887513 >> v2:172.21.15.118:3300/0 conn(0x7f90c80cf7d0 msgr2=0x7f90c8070960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:34.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:34.780+0000 7f90c5d9b640 1 --2- 172.21.15.92:0/2850887513 >> v2:172.21.15.118:3300/0 conn(0x7f90c80cf7d0 0x7f90c8070960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:34.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:34.780+0000 7f90c5d9b640 1 -- 172.21.15.92:0/2850887513 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90c8071970 con 0x7f90c8070ea0 2023-11-24T06:12:34.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:34.780+0000 7f90c5d9b640 1 -- 172.21.15.92:0/2850887513 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2607661350 0 0) 0x7f90c00019f0 con 0x7f90c8070ea0 2023-11-24T06:12:34.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:34.780+0000 7f90c5d9b640 1 -- 172.21.15.92:0/2850887513 >> v1:172.21.15.92:6789/0 conn(0x7f90c8070ea0 legacy=0x7f90c80b1ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:34.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:34.780+0000 7f90c5d9b640 1 --2- 172.21.15.92:0/2850887513 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f90b4003970 0x7f90b4003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:34.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:34.780+0000 7f90c5d9b640 1 --2- 172.21.15.92:0/2850887513 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f90b40047c0 0x7f90b4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:34.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:34.780+0000 7f90c5d9b640 1 --2- 172.21.15.92:0/2850887513 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f90c814b2c0 0x7f90b4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:34.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:34.780+0000 7f90c5d9b640 1 -- 172.21.15.92:0/2850887513 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90c814b6a0 con 0x7f90b40047c0 2023-11-24T06:12:34.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:34.780+0000 7f90c5d9b640 1 -- 172.21.15.92:0/2850887513 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90c8071690 con 0x7f90b4003970 2023-11-24T06:12:34.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:34.780+0000 7f90c5d9b640 1 -- 172.21.15.92:0/2850887513 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90b400b980 con 0x7f90c814b2c0 2023-11-24T06:12:34.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:34.780+0000 7f90bffff640 1 --2- 172.21.15.92:0/2850887513 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f90b40047c0 0x7f90b4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:34.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:34.780+0000 7f90bffff640 1 -- 172.21.15.92:0/2850887513 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f90c814b2c0 msgr2=0x7f90b4005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:34.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:34.780+0000 7f90bffff640 1 --2- 172.21.15.92:0/2850887513 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f90c814b2c0 0x7f90b4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:34.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:34.780+0000 7f90bffff640 1 -- 172.21.15.92:0/2850887513 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f90b4003970 msgr2=0x7f90b4003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:34.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:34.780+0000 7f90bffff640 1 --2- 172.21.15.92:0/2850887513 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f90b4003970 0x7f90b4003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:34.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:34.780+0000 7f90bffff640 1 -- 172.21.15.92:0/2850887513 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90b4003520 con 0x7f90b40047c0 2023-11-24T06:12:34.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:34.781+0000 7f90bffff640 1 --2- 172.21.15.92:0/2850887513 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f90b40047c0 0x7f90b4004c40 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f90b0009ac0 tx=0x7f90b40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d40a45740c85d985 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:34.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:34.782+0000 7f90c5d9b640 1 -- 172.21.15.92:0/2850887513 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90b002c710 con 0x7f90b40047c0 2023-11-24T06:12:34.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:34.782+0000 7f90c5d9b640 1 -- 172.21.15.92:0/2850887513 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90b002c870 con 0x7f90b40047c0 2023-11-24T06:12:34.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:34.782+0000 7f90c5d9b640 1 -- 172.21.15.92:0/2850887513 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90b002cc00 con 0x7f90b40047c0 2023-11-24T06:12:34.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:34.782+0000 7f90cca1e640 1 -- 172.21.15.92:0/2850887513 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f90b40047c0 msgr2=0x7f90b4004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:34.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:34.782+0000 7f90cca1e640 1 --2- 172.21.15.92:0/2850887513 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f90b40047c0 0x7f90b4004c40 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f90b0009ac0 tx=0x7f90b40057f0 comp rx=0 tx=0).stop 2023-11-24T06:12:34.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:34.782+0000 7f90c6d9d640 1 -- 172.21.15.92:0/2850887513 reap_dead start 2023-11-24T06:12:34.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:34.782+0000 7f90cca1e640 1 -- 172.21.15.92:0/2850887513 shutdown_connections 2023-11-24T06:12:34.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:34.782+0000 7f90cca1e640 1 -- 172.21.15.92:0/2850887513 >> 172.21.15.92:0/2850887513 conn(0x7f90c806a800 msgr2=0x7f90c806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:34.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:34.782+0000 7f90cca1e640 1 -- 172.21.15.92:0/2850887513 shutdown_connections 2023-11-24T06:12:34.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:34.782+0000 7f90cca1e640 1 -- 172.21.15.92:0/2850887513 wait complete. 2023-11-24T06:12:34.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:34.784+0000 7f90cca1e640 1 Processor -- start 2023-11-24T06:12:34.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:34.784+0000 7f90cca1e640 1 -- start start 2023-11-24T06:12:34.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:34.785+0000 7f90cca1e640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f90b40047c0 0x7f90c81643a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:34.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:34.785+0000 7f90cca1e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f90c8146fc0 0x7f90c8140160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:34.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:34.785+0000 7f90cca1e640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f90c8147420 0x7f90c81406a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:34.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:34.785+0000 7f90cca1e640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90c8140be0 con 0x7f90c8146fc0 2023-11-24T06:12:34.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:34.785+0000 7f90cca1e640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90c8164cf0 con 0x7f90b40047c0 2023-11-24T06:12:34.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:34.785+0000 7f90cca1e640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90c8151d40 con 0x7f90c8147420 2023-11-24T06:12:34.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:34.785+0000 7f90bffff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f90c8146fc0 0x7f90c8140160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:34.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:34.785+0000 7f90bffff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f90c8146fc0 0x7f90c8140160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:49908/0 (socket says 172.21.15.92:49908) 2023-11-24T06:12:34.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:34.785+0000 7f90bffff640 1 -- 172.21.15.92:0/4027730044 learned_addr learned my addr 172.21.15.92:0/4027730044 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:12:34.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:34.786+0000 7f90c659c640 1 --2- 172.21.15.92:0/4027730044 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f90b40047c0 0x7f90c81643a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:34.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:34.786+0000 7f90c6d9d640 1 --2- 172.21.15.92:0/4027730044 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f90c8147420 0x7f90c81406a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:34.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:34.786+0000 7f90bffff640 1 -- 172.21.15.92:0/4027730044 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f90c8147420 msgr2=0x7f90c81406a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:34.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:34.786+0000 7f90bffff640 1 --2- 172.21.15.92:0/4027730044 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f90c8147420 0x7f90c81406a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:34.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:34.786+0000 7f90bffff640 1 -- 172.21.15.92:0/4027730044 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f90b40047c0 msgr2=0x7f90c81643a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:34.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:34.786+0000 7f90bffff640 1 --2- 172.21.15.92:0/4027730044 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f90b40047c0 0x7f90c81643a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:34.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:34.786+0000 7f90bffff640 1 -- 172.21.15.92:0/4027730044 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90c8152020 con 0x7f90c8146fc0 2023-11-24T06:12:34.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:34.786+0000 7f90c659c640 1 --2- 172.21.15.92:0/4027730044 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f90b40047c0 0x7f90c81643a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:12:34.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:34.786+0000 7f90bffff640 1 --2- 172.21.15.92:0/4027730044 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f90c8146fc0 0x7f90c8140160 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f90b002c2b0 tx=0x7f90b0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:34.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:34.786+0000 7f90beffd640 1 -- 172.21.15.92:0/4027730044 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90b0003810 con 0x7f90c8146fc0 2023-11-24T06:12:34.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:34.787+0000 7f90beffd640 1 -- 172.21.15.92:0/4027730044 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90b002de90 con 0x7f90c8146fc0 2023-11-24T06:12:34.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:34.787+0000 7f90beffd640 1 -- 172.21.15.92:0/4027730044 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90b0003b70 con 0x7f90c8146fc0 2023-11-24T06:12:34.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:34.787+0000 7f90cca1e640 1 -- 172.21.15.92:0/4027730044 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90c8152300 con 0x7f90c8146fc0 2023-11-24T06:12:34.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:34.787+0000 7f90cca1e640 1 -- 172.21.15.92:0/4027730044 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90c813ec60 con 0x7f90c8146fc0 2023-11-24T06:12:34.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:34.788+0000 7f90cca1e640 1 -- 172.21.15.92:0/4027730044 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90c80675d0 con 0x7f90c8146fc0 2023-11-24T06:12:34.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:34.789+0000 7f90beffd640 1 -- 172.21.15.92:0/4027730044 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f90b0003cd0 con 0x7f90c8146fc0 2023-11-24T06:12:34.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:34.790+0000 7f90beffd640 1 --2- 172.21.15.92:0/4027730044 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f90a806ae40 0x7f90a806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:34.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:34.790+0000 7f90beffd640 1 -- 172.21.15.92:0/4027730044 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f90b00ab670 con 0x7f90c8146fc0 2023-11-24T06:12:34.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:34.790+0000 7f90c659c640 1 --2- 172.21.15.92:0/4027730044 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f90a806ae40 0x7f90a806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:34.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:34.790+0000 7f90c659c640 1 --2- 172.21.15.92:0/4027730044 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f90a806ae40 0x7f90a806d300 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f90b8009950 tx=0x7f90b8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:34.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:34.794+0000 7f90beffd640 1 -- 172.21.15.92:0/4027730044 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f90b0076840 con 0x7f90c8146fc0 2023-11-24T06:12:35.023 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:35.021+0000 7f90cca1e640 1 -- 172.21.15.92:0/4027730044 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f90c80cb450 con 0x7f90c8146fc0 2023-11-24T06:12:35.024 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:35.022+0000 7f90beffd640 1 -- 172.21.15.92:0/4027730044 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) v1 ==== 94+0+4919 (secure 0 0 0) 0x7f90b0079e60 con 0x7f90c8146fc0 2023-11-24T06:12:35.025 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-24T06:12:35.025 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":31,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4778,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6835/1184353034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":1184353034},{"type":"v1","addr":"172.21.15.118:6835","nonce":1184353034}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":26},{"gid":4942,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6833/1256969666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":1256969666},{"type":"v1","addr":"172.21.15.118:6833","nonce":1256969666}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":26},{"gid":5706,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6836/3849408561","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":3849408561},{"type":"v1","addr":"172.21.15.92:6836","nonce":3849408561}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":31,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T06:12:30.605447+0000","modified":"2023-11-24T06:12:34.015371+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5709},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5709":{"gid":5709,"name":"c","rank":0,"incarnation":27,"state":"up:active","state_seq":3,"addr":"172.21.15.92:6837/1958912045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":1958912045},{"type":"v1","addr":"172.21.15.92:6837","nonce":1958912045}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-24T06:12:35.025 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 31 2023-11-24T06:12:35.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:35.025+0000 7f90cca1e640 1 -- 172.21.15.92:0/4027730044 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f90a806ae40 msgr2=0x7f90a806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:35.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:35.025+0000 7f90cca1e640 1 --2- 172.21.15.92:0/4027730044 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f90a806ae40 0x7f90a806d300 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f90b8009950 tx=0x7f90b8008040 comp rx=0 tx=0).stop 2023-11-24T06:12:35.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:35.025+0000 7f90cca1e640 1 -- 172.21.15.92:0/4027730044 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f90c8146fc0 msgr2=0x7f90c8140160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:35.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:35.025+0000 7f90cca1e640 1 --2- 172.21.15.92:0/4027730044 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f90c8146fc0 0x7f90c8140160 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f90b002c2b0 tx=0x7f90b0004730 comp rx=0 tx=0).stop 2023-11-24T06:12:35.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:35.025+0000 7f90cca1e640 1 -- 172.21.15.92:0/4027730044 shutdown_connections 2023-11-24T06:12:35.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:35.025+0000 7f90cca1e640 1 --2- 172.21.15.92:0/4027730044 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f90a806ae40 0x7f90a806d300 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:35.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:35.025+0000 7f90cca1e640 1 --2- 172.21.15.92:0/4027730044 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f90c8147420 0x7f90c81406a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:35.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:35.025+0000 7f90cca1e640 1 --2- 172.21.15.92:0/4027730044 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f90c8146fc0 0x7f90c8140160 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:35.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:35.025+0000 7f90cca1e640 1 --2- 172.21.15.92:0/4027730044 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f90b40047c0 0x7f90c81643a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:35.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:35.025+0000 7f90cca1e640 1 -- 172.21.15.92:0/4027730044 >> 172.21.15.92:0/4027730044 conn(0x7f90c806a800 msgr2=0x7f90c80aef90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:35.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:35.026+0000 7f90cca1e640 1 -- 172.21.15.92:0/4027730044 shutdown_connections 2023-11-24T06:12:35.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:35.026+0000 7f90cca1e640 1 -- 172.21.15.92:0/4027730044 wait complete. 2023-11-24T06:12:35.046 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:12:35.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:35.253+0000 7f3adc333640 1 Processor -- start 2023-11-24T06:12:35.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:35.253+0000 7f3adc333640 1 -- start start 2023-11-24T06:12:35.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:35.255+0000 7f3adc333640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3ad40b3ae0 0x7f3ad415e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:35.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:35.255+0000 7f3adc333640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f3ad40cf290 0x7f3ad40cf720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:35.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:35.255+0000 7f3adc333640 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ad4070c40 con 0x7f3ad40ae5a0 2023-11-24T06:12:35.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:35.255+0000 7f3adc333640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ad4071450 con 0x7f3ad40cf290 2023-11-24T06:12:35.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:35.255+0000 7f3adc333640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ad40cfc60 con 0x7f3ad40b3ae0 2023-11-24T06:12:35.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:35.255+0000 7f3ada8d0640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f3ad40cf290 0x7f3ad40cf720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:35.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:35.255+0000 7f3ada8d0640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f3ad40cf290 0x7f3ad40cf720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:49922/0 (socket says 172.21.15.92:49922) 2023-11-24T06:12:35.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:35.255+0000 7f3ada8d0640 1 -- 172.21.15.92:0/4275036809 learned_addr learned my addr 172.21.15.92:0/4275036809 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:12:35.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:35.255+0000 7f3ad98ce640 1 --2- 172.21.15.92:0/4275036809 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3ad40b3ae0 0x7f3ad415e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:35.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:35.256+0000 7f3ada8d0640 1 -- 172.21.15.92:0/4275036809 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3ad40b3ae0 msgr2=0x7f3ad415e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:35.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:35.256+0000 7f3ada8d0640 1 --2- 172.21.15.92:0/4275036809 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3ad40b3ae0 0x7f3ad415e430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:35.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:35.256+0000 7f3ada8d0640 1 -- 172.21.15.92:0/4275036809 >> v1:172.21.15.118:6789/0 conn(0x7f3ad40ae5a0 legacy=0x7f3ad406cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:35.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:35.256+0000 7f3ada8d0640 1 -- 172.21.15.92:0/4275036809 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ad406c470 con 0x7f3ad40cf290 2023-11-24T06:12:35.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:35.257+0000 7f3ada8d0640 1 --2- 172.21.15.92:0/4275036809 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3ad40cf290 0x7f3ad40cf720 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f3acc009870 tx=0x7f3acc02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=688b048cdb84e997 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:35.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:35.257+0000 7f3ad90cd640 1 -- 172.21.15.92:0/4275036809 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3acc030840 con 0x7f3ad40cf290 2023-11-24T06:12:35.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:35.257+0000 7f3ad90cd640 1 -- 172.21.15.92:0/4275036809 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3acc0309a0 con 0x7f3ad40cf290 2023-11-24T06:12:35.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:35.257+0000 7f3ad90cd640 1 -- 172.21.15.92:0/4275036809 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3acc030d30 con 0x7f3ad40cf290 2023-11-24T06:12:35.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:35.257+0000 7f3adc333640 1 -- 172.21.15.92:0/4275036809 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3ad40cf290 msgr2=0x7f3ad40cf720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:35.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:35.257+0000 7f3adc333640 1 --2- 172.21.15.92:0/4275036809 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3ad40cf290 0x7f3ad40cf720 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f3acc009870 tx=0x7f3acc02b980 comp rx=0 tx=0).stop 2023-11-24T06:12:35.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:35.258+0000 7f3adc333640 1 -- 172.21.15.92:0/4275036809 shutdown_connections 2023-11-24T06:12:35.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:35.258+0000 7f3adc333640 1 --2- 172.21.15.92:0/4275036809 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3ad40cf290 0x7f3ad40cf720 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:35.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:35.258+0000 7f3adc333640 1 --2- 172.21.15.92:0/4275036809 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3ad40b3ae0 0x7f3ad415e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:35.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:35.258+0000 7f3adc333640 1 -- 172.21.15.92:0/4275036809 >> 172.21.15.92:0/4275036809 conn(0x7f3ad406a800 msgr2=0x7f3ad406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:35.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:35.258+0000 7f3adc333640 1 -- 172.21.15.92:0/4275036809 shutdown_connections 2023-11-24T06:12:35.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:35.258+0000 7f3adc333640 1 -- 172.21.15.92:0/4275036809 wait complete. 2023-11-24T06:12:35.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:35.259+0000 7f3adc333640 1 Processor -- start 2023-11-24T06:12:35.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:35.259+0000 7f3adc333640 1 -- start start 2023-11-24T06:12:35.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:35.260+0000 7f3adc333640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3ad40ae5a0 0x7f3ad4153f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:35.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:35.260+0000 7f3adc333640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3ad40b3ae0 0x7f3ad415bf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:35.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:35.260+0000 7f3adc333640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3ad40cf290 0x7f3ad415eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:35.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:35.260+0000 7f3adc333640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ad414f9c0 con 0x7f3ad40cf290 2023-11-24T06:12:35.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:35.260+0000 7f3adc333640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ad4151470 con 0x7f3ad40ae5a0 2023-11-24T06:12:35.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:35.260+0000 7f3adc333640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ad415f710 con 0x7f3ad40b3ae0 2023-11-24T06:12:35.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:35.261+0000 7f3ada8d0640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3ad40cf290 0x7f3ad415eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:35.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:35.261+0000 7f3ada8d0640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3ad40cf290 0x7f3ad415eb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:49932/0 (socket says 172.21.15.92:49932) 2023-11-24T06:12:35.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:35.261+0000 7f3ada8d0640 1 -- 172.21.15.92:0/318714877 learned_addr learned my addr 172.21.15.92:0/318714877 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:12:35.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:35.261+0000 7f3ada0cf640 1 --2- 172.21.15.92:0/318714877 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3ad40ae5a0 0x7f3ad4153f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:35.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:35.261+0000 7f3ad98ce640 1 --2- 172.21.15.92:0/318714877 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3ad40b3ae0 0x7f3ad415bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:35.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:35.261+0000 7f3ada8d0640 1 -- 172.21.15.92:0/318714877 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3ad40b3ae0 msgr2=0x7f3ad415bf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:35.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:35.261+0000 7f3ada8d0640 1 --2- 172.21.15.92:0/318714877 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3ad40b3ae0 0x7f3ad415bf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:35.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:35.261+0000 7f3ada8d0640 1 -- 172.21.15.92:0/318714877 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3ad40ae5a0 msgr2=0x7f3ad4153f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:35.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:35.261+0000 7f3ada8d0640 1 --2- 172.21.15.92:0/318714877 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3ad40ae5a0 0x7f3ad4153f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:35.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:35.261+0000 7f3ada8d0640 1 -- 172.21.15.92:0/318714877 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3acc0094d0 con 0x7f3ad40cf290 2023-11-24T06:12:35.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:35.261+0000 7f3ada0cf640 1 --2- 172.21.15.92:0/318714877 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3ad40ae5a0 0x7f3ad4153f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:12:35.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:35.261+0000 7f3ada8d0640 1 --2- 172.21.15.92:0/318714877 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3ad40cf290 0x7f3ad415eb50 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f3acc02b9c0 tx=0x7f3acc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:35.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:35.262+0000 7f3ad2ffd640 1 -- 172.21.15.92:0/318714877 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3acc039c20 con 0x7f3ad40cf290 2023-11-24T06:12:35.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:35.262+0000 7f3ad2ffd640 1 -- 172.21.15.92:0/318714877 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3acc039da0 con 0x7f3ad40cf290 2023-11-24T06:12:35.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:35.262+0000 7f3ad2ffd640 1 -- 172.21.15.92:0/318714877 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3acc043630 con 0x7f3ad40cf290 2023-11-24T06:12:35.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:35.262+0000 7f3adc333640 1 -- 172.21.15.92:0/318714877 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ad415f9f0 con 0x7f3ad40cf290 2023-11-24T06:12:35.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:35.262+0000 7f3adc333640 1 -- 172.21.15.92:0/318714877 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ad41519a0 con 0x7f3ad40cf290 2023-11-24T06:12:35.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:35.263+0000 7f3adc333640 1 -- 172.21.15.92:0/318714877 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ad4151be0 con 0x7f3ad40cf290 2023-11-24T06:12:35.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:35.264+0000 7f3ad2ffd640 1 -- 172.21.15.92:0/318714877 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f3acc042390 con 0x7f3ad40cf290 2023-11-24T06:12:35.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:35.265+0000 7f3ad2ffd640 1 --2- 172.21.15.92:0/318714877 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f3ab806ae40 0x7f3ab806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:35.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:35.266+0000 7f3ad2ffd640 1 -- 172.21.15.92:0/318714877 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3acc0ab910 con 0x7f3ad40cf290 2023-11-24T06:12:35.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:35.266+0000 7f3ada0cf640 1 --2- 172.21.15.92:0/318714877 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f3ab806ae40 0x7f3ab806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:35.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:35.266+0000 7f3ada0cf640 1 --2- 172.21.15.92:0/318714877 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f3ab806ae40 0x7f3ab806d300 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f3ac8004520 tx=0x7f3ac8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:35.272 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:35.270+0000 7f3ad2ffd640 1 -- 172.21.15.92:0/318714877 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3ad4151be0 con 0x7f3ad40cf290 2023-11-24T06:12:35.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:35.462+0000 7f3adc333640 1 -- 172.21.15.92:0/318714877 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3ad4151be0 con 0x7f3ad40cf290 2023-11-24T06:12:35.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:35.463+0000 7f3ad2ffd640 1 -- 172.21.15.92:0/318714877 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v52) v1 ==== 74+0+16517 (secure 0 0 0) 0x7f3ad4151be0 con 0x7f3ad40cf290 2023-11-24T06:12:35.466 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-24T06:12:35.466 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":52,"fsid":"2d6dee5f-8aa3-470f-931a-50ade963b1cb","created":"2023-11-24T06:10:04.707204+0000","modified":"2023-11-24T06:12:32.152035+0000","last_up_change":"2023-11-24T06:10:17.598352+0000","last_in_change":"2023-11-24T06:10:09.040793+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:10:17.733098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:12:28.410305+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-11-24T06:12:29.625599+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ff37a9fb-135f-471f-b4b0-f7b5772006c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6803","nonce":20560714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6805","nonce":20560714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6809","nonce":20560714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6807","nonce":20560714}]},"public_addr":"172.21.15.92:6803/20560714","cluster_addr":"172.21.15.92:6805/20560714","heartbeat_back_addr":"172.21.15.92:6809/20560714","heartbeat_front_addr":"172.21.15.92:6807/20560714","state":["exists","up"]},{"osd":1,"uuid":"03cdd353-dcf1-4f38-94fe-a795137c06b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6819","nonce":2451239906}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6821","nonce":2451239906}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6825","nonce":2451239906}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6823","nonce":2451239906}]},"public_addr":"172.21.15.92:6819/2451239906","cluster_addr":"172.21.15.92:6821/2451239906","heartbeat_back_addr":"172.21.15.92:6825/2451239906","heartbeat_front_addr":"172.21.15.92:6823/2451239906","state":["exists","up"]},{"osd":2,"uuid":"a4a20d80-5d57-484b-becd-9aa8070dd3aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6811","nonce":1124979277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6813","nonce":1124979277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6817","nonce":1124979277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6815","nonce":1124979277}]},"public_addr":"172.21.15.92:6811/1124979277","cluster_addr":"172.21.15.92:6813/1124979277","heartbeat_back_addr":"172.21.15.92:6817/1124979277","heartbeat_front_addr":"172.21.15.92:6815/1124979277","state":["exists","up"]},{"osd":3,"uuid":"4648d7ce-9b29-4583-9425-4ba8fd2b1903","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6827","nonce":1734115782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6829","nonce":1734115782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6833","nonce":1734115782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6831","nonce":1734115782}]},"public_addr":"172.21.15.92:6827/1734115782","cluster_addr":"172.21.15.92:6829/1734115782","heartbeat_back_addr":"172.21.15.92:6833/1734115782","heartbeat_front_addr":"172.21.15.92:6831/1734115782","state":["exists","up"]},{"osd":4,"uuid":"3bc9d7eb-c6fa-4fcf-a3ec-ade604e05aac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6819","nonce":3425937229}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6822","nonce":3425937229}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6830","nonce":3425937229}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6826","nonce":3425937229}]},"public_addr":"172.21.15.118:6819/3425937229","cluster_addr":"172.21.15.118:6822/3425937229","heartbeat_back_addr":"172.21.15.118:6830/3425937229","heartbeat_front_addr":"172.21.15.118:6826/3425937229","state":["exists","up"]},{"osd":5,"uuid":"dce9ba6d-59ed-46d1-84fc-a84b13de10c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6801","nonce":827455984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6803","nonce":827455984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6807","nonce":827455984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6805","nonce":827455984}]},"public_addr":"172.21.15.118:6801/827455984","cluster_addr":"172.21.15.118:6803/827455984","heartbeat_back_addr":"172.21.15.118:6807/827455984","heartbeat_front_addr":"172.21.15.118:6805/827455984","state":["exists","up"]},{"osd":6,"uuid":"82aec504-ae58-4709-8d30-46f2b12081b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6809","nonce":2946986141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6811","nonce":2946986141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6815","nonce":2946986141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6813","nonce":2946986141}]},"public_addr":"172.21.15.118:6809/2946986141","cluster_addr":"172.21.15.118:6811/2946986141","heartbeat_back_addr":"172.21.15.118:6815/2946986141","heartbeat_front_addr":"172.21.15.118:6813/2946986141","state":["exists","up"]},{"osd":7,"uuid":"4ed4c624-c88f-4d35-bdb1-bf209b5dffb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6817","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6818","nonce":48122059}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6821","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6823","nonce":48122059}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6829","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6831","nonce":48122059}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6825","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6827","nonce":48122059}]},"public_addr":"172.21.15.118:6818/48122059","cluster_addr":"172.21.15.118:6823/48122059","heartbeat_back_addr":"172.21.15.118:6831/48122059","heartbeat_front_addr":"172.21.15.118:6827/48122059","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.536664+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.368324+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.271201+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.342971+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.621832+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.387110+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.632946+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.416032+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:12:35.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:35.466+0000 7f3adc333640 1 -- 172.21.15.92:0/318714877 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f3ab806ae40 msgr2=0x7f3ab806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:35.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:35.466+0000 7f3adc333640 1 --2- 172.21.15.92:0/318714877 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f3ab806ae40 0x7f3ab806d300 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f3ac8004520 tx=0x7f3ac8009040 comp rx=0 tx=0).stop 2023-11-24T06:12:35.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:35.466+0000 7f3adc333640 1 -- 172.21.15.92:0/318714877 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3ad40cf290 msgr2=0x7f3ad415eb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:35.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:35.466+0000 7f3adc333640 1 --2- 172.21.15.92:0/318714877 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3ad40cf290 0x7f3ad415eb50 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f3acc02b9c0 tx=0x7f3acc004730 comp rx=0 tx=0).stop 2023-11-24T06:12:35.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:35.466+0000 7f3adc333640 1 -- 172.21.15.92:0/318714877 shutdown_connections 2023-11-24T06:12:35.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:35.466+0000 7f3adc333640 1 --2- 172.21.15.92:0/318714877 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f3ab806ae40 0x7f3ab806d300 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:35.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:35.466+0000 7f3adc333640 1 --2- 172.21.15.92:0/318714877 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3ad40cf290 0x7f3ad415eb50 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:35.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:35.466+0000 7f3adc333640 1 --2- 172.21.15.92:0/318714877 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3ad40b3ae0 0x7f3ad415bf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:35.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:35.466+0000 7f3adc333640 1 --2- 172.21.15.92:0/318714877 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3ad40ae5a0 0x7f3ad4153f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:35.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:35.466+0000 7f3adc333640 1 -- 172.21.15.92:0/318714877 >> 172.21.15.92:0/318714877 conn(0x7f3ad406a800 msgr2=0x7f3ad40b22f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:35.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:35.466+0000 7f3adc333640 1 -- 172.21.15.92:0/318714877 shutdown_connections 2023-11-24T06:12:35.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:35.466+0000 7f3adc333640 1 -- 172.21.15.92:0/318714877 wait complete. 2023-11-24T06:12:35.488 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-11-24T06:12:35.489 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-24T06:12:35.524 INFO:tasks.ceph.mds.c.smithi092.stderr:2023-11-24T06:12:35.522+0000 7f738eab8640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-24T06:12:35.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:35.699+0000 7f84acbbe640 1 Processor -- start 2023-11-24T06:12:35.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:35.699+0000 7f84acbbe640 1 -- start start 2023-11-24T06:12:35.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:35.701+0000 7f84acbbe640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f84a813df80 0x7f84a815bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:35.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:35.701+0000 7f84acbbe640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f84a813e360 0x7f84a80cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:35.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:35.701+0000 7f84acbbe640 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84a806c470 con 0x7f84a815f9c0 2023-11-24T06:12:35.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:35.701+0000 7f84acbbe640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84a806c750 con 0x7f84a813e360 2023-11-24T06:12:35.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:35.701+0000 7f84acbbe640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84a806ca30 con 0x7f84a813df80 2023-11-24T06:12:35.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:35.701+0000 7f849ffff640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f84a813e360 0x7f84a80cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:35.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:35.701+0000 7f849ffff640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f84a813e360 0x7f84a80cf290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:49938/0 (socket says 172.21.15.92:49938) 2023-11-24T06:12:35.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:35.701+0000 7f849ffff640 1 -- 172.21.15.92:0/1354083648 learned_addr learned my addr 172.21.15.92:0/1354083648 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:12:35.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:35.702+0000 7f849ffff640 1 -- 172.21.15.92:0/1354083648 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f84a813df80 msgr2=0x7f84a815bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:35.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:35.702+0000 7f84a659c640 1 --2- 172.21.15.92:0/1354083648 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f84a813df80 0x7f84a815bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:35.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:35.702+0000 7f849ffff640 1 --2- 172.21.15.92:0/1354083648 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f84a813df80 0x7f84a815bee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:35.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:35.702+0000 7f849ffff640 1 -- 172.21.15.92:0/1354083648 >> v1:172.21.15.118:6789/0 conn(0x7f84a815f9c0 legacy=0x7f84a81408d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:35.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:35.702+0000 7f849ffff640 1 -- 172.21.15.92:0/1354083648 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84a806cd10 con 0x7f84a813e360 2023-11-24T06:12:35.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:35.703+0000 7f849ffff640 1 --2- 172.21.15.92:0/1354083648 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f84a813e360 0x7f84a80cf290 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f8490009870 tx=0x7f849002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e3c37630dc04dee server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:35.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:35.703+0000 7f84a5d9b640 1 -- 172.21.15.92:0/1354083648 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8490031640 con 0x7f84a813e360 2023-11-24T06:12:35.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:35.703+0000 7f84a5d9b640 1 -- 172.21.15.92:0/1354083648 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84900317a0 con 0x7f84a813e360 2023-11-24T06:12:35.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:35.703+0000 7f84a5d9b640 1 -- 172.21.15.92:0/1354083648 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8490031b30 con 0x7f84a813e360 2023-11-24T06:12:35.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:35.704+0000 7f84acbbe640 1 -- 172.21.15.92:0/1354083648 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f84a813e360 msgr2=0x7f84a80cf290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:35.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:35.704+0000 7f84acbbe640 1 --2- 172.21.15.92:0/1354083648 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f84a813e360 0x7f84a80cf290 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f8490009870 tx=0x7f849002b8f0 comp rx=0 tx=0).stop 2023-11-24T06:12:35.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:35.704+0000 7f84acbbe640 1 -- 172.21.15.92:0/1354083648 shutdown_connections 2023-11-24T06:12:35.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:35.704+0000 7f84acbbe640 1 --2- 172.21.15.92:0/1354083648 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f84a813e360 0x7f84a80cf290 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:35.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:35.704+0000 7f84acbbe640 1 --2- 172.21.15.92:0/1354083648 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f84a813df80 0x7f84a815bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:35.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:35.704+0000 7f84acbbe640 1 -- 172.21.15.92:0/1354083648 >> 172.21.15.92:0/1354083648 conn(0x7f84a806a800 msgr2=0x7f84a806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:35.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:35.704+0000 7f84acbbe640 1 -- 172.21.15.92:0/1354083648 shutdown_connections 2023-11-24T06:12:35.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:35.704+0000 7f84acbbe640 1 -- 172.21.15.92:0/1354083648 wait complete. 2023-11-24T06:12:35.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:35.706+0000 7f84acbbe640 1 Processor -- start 2023-11-24T06:12:35.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:35.706+0000 7f84acbbe640 1 -- start start 2023-11-24T06:12:35.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:35.707+0000 7f84acbbe640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f84a813df80 0x7f84a814aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:35.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:35.707+0000 7f84acbbe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f84a813e360 0x7f84a81f6590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:35.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:35.707+0000 7f84acbbe640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f84a815f9c0 0x7f84a80b2670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:35.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:35.707+0000 7f84acbbe640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84a813cc10 con 0x7f84a813e360 2023-11-24T06:12:35.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:35.707+0000 7f84acbbe640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84a813ceb0 con 0x7f84a813df80 2023-11-24T06:12:35.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:35.707+0000 7f84acbbe640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84a81f6ad0 con 0x7f84a815f9c0 2023-11-24T06:12:35.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:35.707+0000 7f849ffff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f84a813e360 0x7f84a81f6590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:35.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:35.707+0000 7f849ffff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f84a813e360 0x7f84a81f6590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:49950/0 (socket says 172.21.15.92:49950) 2023-11-24T06:12:35.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:35.707+0000 7f849ffff640 1 -- 172.21.15.92:0/747408816 learned_addr learned my addr 172.21.15.92:0/747408816 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:12:35.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:35.708+0000 7f849ffff640 1 -- 172.21.15.92:0/747408816 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f84a815f9c0 msgr2=0x7f84a80b2670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:35.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:35.708+0000 7f849ffff640 1 --2- 172.21.15.92:0/747408816 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f84a815f9c0 0x7f84a80b2670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:35.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:35.708+0000 7f849ffff640 1 -- 172.21.15.92:0/747408816 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f84a813df80 msgr2=0x7f84a814aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:35.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:35.708+0000 7f849ffff640 1 --2- 172.21.15.92:0/747408816 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f84a813df80 0x7f84a814aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:35.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:35.708+0000 7f849ffff640 1 -- 172.21.15.92:0/747408816 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84900094d0 con 0x7f84a813e360 2023-11-24T06:12:35.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:35.708+0000 7f849ffff640 1 --2- 172.21.15.92:0/747408816 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f84a813e360 0x7f84a81f6590 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f8490009870 tx=0x7f84900046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:35.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:35.708+0000 7f849effd640 1 -- 172.21.15.92:0/747408816 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f849003ac60 con 0x7f84a813e360 2023-11-24T06:12:35.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:35.708+0000 7f849effd640 1 -- 172.21.15.92:0/747408816 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f849003ade0 con 0x7f84a813e360 2023-11-24T06:12:35.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:35.708+0000 7f849effd640 1 -- 172.21.15.92:0/747408816 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8490044630 con 0x7f84a813e360 2023-11-24T06:12:35.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:35.708+0000 7f84acbbe640 1 -- 172.21.15.92:0/747408816 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84a81f6d70 con 0x7f84a813e360 2023-11-24T06:12:35.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:35.709+0000 7f84acbbe640 1 -- 172.21.15.92:0/747408816 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84a81f7330 con 0x7f84a813e360 2023-11-24T06:12:35.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:35.710+0000 7f84acbbe640 1 -- 172.21.15.92:0/747408816 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84a80675d0 con 0x7f84a813e360 2023-11-24T06:12:35.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:35.711+0000 7f849effd640 1 -- 172.21.15.92:0/747408816 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f849002f050 con 0x7f84a813e360 2023-11-24T06:12:35.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:35.712+0000 7f849effd640 1 --2- 172.21.15.92:0/747408816 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f848806ae40 0x7f848806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:35.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:35.712+0000 7f849effd640 1 -- 172.21.15.92:0/747408816 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f84900ab830 con 0x7f84a813e360 2023-11-24T06:12:35.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:35.712+0000 7f84a659c640 1 --2- 172.21.15.92:0/747408816 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f848806ae40 0x7f848806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:35.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:35.713+0000 7f84a659c640 1 --2- 172.21.15.92:0/747408816 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f848806ae40 0x7f848806d300 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f849800a810 tx=0x7f8498008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:35.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:35.716+0000 7f849effd640 1 -- 172.21.15.92:0/747408816 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8490076840 con 0x7f84a813e360 2023-11-24T06:12:35.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:35.905+0000 7f84acbbe640 1 -- 172.21.15.92:0/747408816 --> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f84a813d020 con 0x7f848806ae40 2023-11-24T06:12:35.917 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:35.915+0000 7f849effd640 1 -- 172.21.15.92:0/747408816 <== mgr.4101 v2:172.21.15.92:6800/784027401 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+287215 (secure 0 0 0) 0x7f84a813d020 con 0x7f848806ae40 2023-11-24T06:12:35.922 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-24T06:12:35.924 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2023-11-24T06:12:35.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:35.921+0000 7f84acbbe640 1 -- 172.21.15.92:0/747408816 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f848806ae40 msgr2=0x7f848806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:35.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:35.921+0000 7f84acbbe640 1 --2- 172.21.15.92:0/747408816 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f848806ae40 0x7f848806d300 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f849800a810 tx=0x7f8498008040 comp rx=0 tx=0).stop 2023-11-24T06:12:35.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:35.921+0000 7f84acbbe640 1 -- 172.21.15.92:0/747408816 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f84a813e360 msgr2=0x7f84a81f6590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:35.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:35.921+0000 7f84acbbe640 1 --2- 172.21.15.92:0/747408816 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f84a813e360 0x7f84a81f6590 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f8490009870 tx=0x7f84900046c0 comp rx=0 tx=0).stop 2023-11-24T06:12:35.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:35.921+0000 7f84acbbe640 1 -- 172.21.15.92:0/747408816 shutdown_connections 2023-11-24T06:12:35.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:35.921+0000 7f84acbbe640 1 --2- 172.21.15.92:0/747408816 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f848806ae40 0x7f848806d300 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:35.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:35.921+0000 7f84acbbe640 1 --2- 172.21.15.92:0/747408816 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f84a815f9c0 0x7f84a80b2670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:35.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:35.921+0000 7f84acbbe640 1 --2- 172.21.15.92:0/747408816 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f84a813e360 0x7f84a81f6590 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:35.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:35.921+0000 7f84acbbe640 1 --2- 172.21.15.92:0/747408816 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f84a813df80 0x7f84a814aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:35.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:35.921+0000 7f84acbbe640 1 -- 172.21.15.92:0/747408816 >> 172.21.15.92:0/747408816 conn(0x7f84a806a800 msgr2=0x7f84a80b0290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:35.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:35.921+0000 7f84acbbe640 1 -- 172.21.15.92:0/747408816 shutdown_connections 2023-11-24T06:12:35.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:35.922+0000 7f84acbbe640 1 -- 172.21.15.92:0/747408816 wait complete. 2023-11-24T06:12:35.941 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":114,"stamp":"2023-11-24T06:12:35.706037+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":54,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":238832,"kb_used_data":6008,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754735888,"statfs":{"total":773094113280,"available":772849549312,"internally_reserved":0,"allocated":6152192,"data_stored":3266904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001874"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-24T06:12:32.717116+0000","last_change":"2023-11-24T06:12:31.198964+0000","last_active":"2023-11-24T06:12:32.717116+0000","last_peered":"2023-11-24T06:12:32.717116+0000","last_clean":"2023-11-24T06:12:32.717116+0000","last_became_active":"2023-11-24T06:12:31.197652+0000","last_became_peered":"2023-11-24T06:12:31.197652+0000","last_unstale":"2023-11-24T06:12:32.717116+0000","last_undegraded":"2023-11-24T06:12:32.717116+0000","last_fullsized":"2023-11-24T06:12:32.717116+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_clean_scrub_stamp":"2023-11-24T06:12:30.127407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:17:30.690945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-24T06:12:32.160974+0000","last_change":"2023-11-24T06:12:31.194790+0000","last_active":"2023-11-24T06:12:32.160974+0000","last_peered":"2023-11-24T06:12:32.160974+0000","last_clean":"2023-11-24T06:12:32.160974+0000","last_became_active":"2023-11-24T06:12:31.194330+0000","last_became_peered":"2023-11-24T06:12:31.194330+0000","last_unstale":"2023-11-24T06:12:32.160974+0000","last_undegraded":"2023-11-24T06:12:32.160974+0000","last_fullsized":"2023-11-24T06:12:32.160974+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_clean_scrub_stamp":"2023-11-24T06:12:30.127407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:00:15.801304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-24T06:12:32.168126+0000","last_change":"2023-11-24T06:12:31.189554+0000","last_active":"2023-11-24T06:12:32.168126+0000","last_peered":"2023-11-24T06:12:32.168126+0000","last_clean":"2023-11-24T06:12:32.168126+0000","last_became_active":"2023-11-24T06:12:31.189217+0000","last_became_peered":"2023-11-24T06:12:31.189217+0000","last_unstale":"2023-11-24T06:12:32.168126+0000","last_undegraded":"2023-11-24T06:12:32.168126+0000","last_fullsized":"2023-11-24T06:12:32.168126+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_clean_scrub_stamp":"2023-11-24T06:12:30.127407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:27:34.505266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-24T06:12:32.716824+0000","last_change":"2023-11-24T06:12:31.188411+0000","last_active":"2023-11-24T06:12:32.716824+0000","last_peered":"2023-11-24T06:12:32.716824+0000","last_clean":"2023-11-24T06:12:32.716824+0000","last_became_active":"2023-11-24T06:12:31.188141+0000","last_became_peered":"2023-11-24T06:12:31.188141+0000","last_unstale":"2023-11-24T06:12:32.716824+0000","last_undegraded":"2023-11-24T06:12:32.716824+0000","last_fullsized":"2023-11-24T06:12:32.716824+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_clean_scrub_stamp":"2023-11-24T06:12:30.127407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:11:10.610649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-24T06:12:32.717176+0000","last_change":"2023-11-24T06:12:31.186880+0000","last_active":"2023-11-24T06:12:32.717176+0000","last_peered":"2023-11-24T06:12:32.717176+0000","last_clean":"2023-11-24T06:12:32.717176+0000","last_became_active":"2023-11-24T06:12:31.186500+0000","last_became_peered":"2023-11-24T06:12:31.186500+0000","last_unstale":"2023-11-24T06:12:32.717176+0000","last_undegraded":"2023-11-24T06:12:32.717176+0000","last_fullsized":"2023-11-24T06:12:32.717176+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_clean_scrub_stamp":"2023-11-24T06:12:30.127407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:27:08.379593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-24T06:12:32.718087+0000","last_change":"2023-11-24T06:12:31.185747+0000","last_active":"2023-11-24T06:12:32.718087+0000","last_peered":"2023-11-24T06:12:32.718087+0000","last_clean":"2023-11-24T06:12:32.718087+0000","last_became_active":"2023-11-24T06:12:31.185468+0000","last_became_peered":"2023-11-24T06:12:31.185468+0000","last_unstale":"2023-11-24T06:12:32.718087+0000","last_undegraded":"2023-11-24T06:12:32.718087+0000","last_fullsized":"2023-11-24T06:12:32.718087+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_clean_scrub_stamp":"2023-11-24T06:12:30.127407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:49:46.518805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-24T06:12:32.717060+0000","last_change":"2023-11-24T06:12:31.172625+0000","last_active":"2023-11-24T06:12:32.717060+0000","last_peered":"2023-11-24T06:12:32.717060+0000","last_clean":"2023-11-24T06:12:32.717060+0000","last_became_active":"2023-11-24T06:12:31.171819+0000","last_became_peered":"2023-11-24T06:12:31.171819+0000","last_unstale":"2023-11-24T06:12:32.717060+0000","last_undegraded":"2023-11-24T06:12:32.717060+0000","last_fullsized":"2023-11-24T06:12:32.717060+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_clean_scrub_stamp":"2023-11-24T06:12:30.127407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:04:42.070586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-24T06:12:32.160714+0000","last_change":"2023-11-24T06:12:31.190733+0000","last_active":"2023-11-24T06:12:32.160714+0000","last_peered":"2023-11-24T06:12:32.160714+0000","last_clean":"2023-11-24T06:12:32.160714+0000","last_became_active":"2023-11-24T06:12:31.187775+0000","last_became_peered":"2023-11-24T06:12:31.187775+0000","last_unstale":"2023-11-24T06:12:32.160714+0000","last_undegraded":"2023-11-24T06:12:32.160714+0000","last_fullsized":"2023-11-24T06:12:32.160714+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_clean_scrub_stamp":"2023-11-24T06:12:30.127407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:56:08.211961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-24T06:12:32.719316+0000","last_change":"2023-11-24T06:12:31.181645+0000","last_active":"2023-11-24T06:12:32.719316+0000","last_peered":"2023-11-24T06:12:32.719316+0000","last_clean":"2023-11-24T06:12:32.719316+0000","last_became_active":"2023-11-24T06:12:31.181322+0000","last_became_peered":"2023-11-24T06:12:31.181322+0000","last_unstale":"2023-11-24T06:12:32.719316+0000","last_undegraded":"2023-11-24T06:12:32.719316+0000","last_fullsized":"2023-11-24T06:12:32.719316+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_clean_scrub_stamp":"2023-11-24T06:12:30.127407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:49:50.435010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-24T06:12:32.715883+0000","last_change":"2023-11-24T06:12:31.180116+0000","last_active":"2023-11-24T06:12:32.715883+0000","last_peered":"2023-11-24T06:12:32.715883+0000","last_clean":"2023-11-24T06:12:32.715883+0000","last_became_active":"2023-11-24T06:12:31.179609+0000","last_became_peered":"2023-11-24T06:12:31.179609+0000","last_unstale":"2023-11-24T06:12:32.715883+0000","last_undegraded":"2023-11-24T06:12:32.715883+0000","last_fullsized":"2023-11-24T06:12:32.715883+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_clean_scrub_stamp":"2023-11-24T06:12:30.127407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:49:50.435010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-24T06:12:32.716508+0000","last_change":"2023-11-24T06:12:31.191319+0000","last_active":"2023-11-24T06:12:32.716508+0000","last_peered":"2023-11-24T06:12:32.716508+0000","last_clean":"2023-11-24T06:12:32.716508+0000","last_became_active":"2023-11-24T06:12:31.191016+0000","last_became_peered":"2023-11-24T06:12:31.191016+0000","last_unstale":"2023-11-24T06:12:32.716508+0000","last_undegraded":"2023-11-24T06:12:32.716508+0000","last_fullsized":"2023-11-24T06:12:32.716508+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_clean_scrub_stamp":"2023-11-24T06:12:30.127407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:38:48.289685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-24T06:12:32.716914+0000","last_change":"2023-11-24T06:12:31.179443+0000","last_active":"2023-11-24T06:12:32.716914+0000","last_peered":"2023-11-24T06:12:32.716914+0000","last_clean":"2023-11-24T06:12:32.716914+0000","last_became_active":"2023-11-24T06:12:31.179128+0000","last_became_peered":"2023-11-24T06:12:31.179128+0000","last_unstale":"2023-11-24T06:12:32.716914+0000","last_undegraded":"2023-11-24T06:12:32.716914+0000","last_fullsized":"2023-11-24T06:12:32.716914+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_clean_scrub_stamp":"2023-11-24T06:12:30.127407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:09:09.186156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-24T06:12:32.166693+0000","last_change":"2023-11-24T06:12:31.182856+0000","last_active":"2023-11-24T06:12:32.166693+0000","last_peered":"2023-11-24T06:12:32.166693+0000","last_clean":"2023-11-24T06:12:32.166693+0000","last_became_active":"2023-11-24T06:12:31.182485+0000","last_became_peered":"2023-11-24T06:12:31.182485+0000","last_unstale":"2023-11-24T06:12:32.166693+0000","last_undegraded":"2023-11-24T06:12:32.166693+0000","last_fullsized":"2023-11-24T06:12:32.166693+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_clean_scrub_stamp":"2023-11-24T06:12:30.127407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:45:48.394942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-24T06:12:32.716837+0000","last_change":"2023-11-24T06:12:31.184693+0000","last_active":"2023-11-24T06:12:32.716837+0000","last_peered":"2023-11-24T06:12:32.716837+0000","last_clean":"2023-11-24T06:12:32.716837+0000","last_became_active":"2023-11-24T06:12:31.184376+0000","last_became_peered":"2023-11-24T06:12:31.184376+0000","last_unstale":"2023-11-24T06:12:32.716837+0000","last_undegraded":"2023-11-24T06:12:32.716837+0000","last_fullsized":"2023-11-24T06:12:32.716837+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_clean_scrub_stamp":"2023-11-24T06:12:30.127407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:34:53.658692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-24T06:12:32.716056+0000","last_change":"2023-11-24T06:12:31.180593+0000","last_active":"2023-11-24T06:12:32.716056+0000","last_peered":"2023-11-24T06:12:32.716056+0000","last_clean":"2023-11-24T06:12:32.716056+0000","last_became_active":"2023-11-24T06:12:31.179053+0000","last_became_peered":"2023-11-24T06:12:31.179053+0000","last_unstale":"2023-11-24T06:12:32.716056+0000","last_undegraded":"2023-11-24T06:12:32.716056+0000","last_fullsized":"2023-11-24T06:12:32.716056+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_clean_scrub_stamp":"2023-11-24T06:12:30.127407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:04:19.413427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-24T06:12:32.716630+0000","last_change":"2023-11-24T06:12:31.188247+0000","last_active":"2023-11-24T06:12:32.716630+0000","last_peered":"2023-11-24T06:12:32.716630+0000","last_clean":"2023-11-24T06:12:32.716630+0000","last_became_active":"2023-11-24T06:12:31.186090+0000","last_became_peered":"2023-11-24T06:12:31.186090+0000","last_unstale":"2023-11-24T06:12:32.716630+0000","last_undegraded":"2023-11-24T06:12:32.716630+0000","last_fullsized":"2023-11-24T06:12:32.716630+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_clean_scrub_stamp":"2023-11-24T06:12:30.127407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:11:10.610649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-24T06:12:32.161136+0000","last_change":"2023-11-24T06:12:31.194663+0000","last_active":"2023-11-24T06:12:32.161136+0000","last_peered":"2023-11-24T06:12:32.161136+0000","last_clean":"2023-11-24T06:12:32.161136+0000","last_became_active":"2023-11-24T06:12:31.194395+0000","last_became_peered":"2023-11-24T06:12:31.194395+0000","last_unstale":"2023-11-24T06:12:32.161136+0000","last_undegraded":"2023-11-24T06:12:32.161136+0000","last_fullsized":"2023-11-24T06:12:32.161136+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_clean_scrub_stamp":"2023-11-24T06:12:30.127407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:32:16.673191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-24T06:12:32.168708+0000","last_change":"2023-11-24T06:12:31.196775+0000","last_active":"2023-11-24T06:12:32.168708+0000","last_peered":"2023-11-24T06:12:32.168708+0000","last_clean":"2023-11-24T06:12:32.168708+0000","last_became_active":"2023-11-24T06:12:31.196208+0000","last_became_peered":"2023-11-24T06:12:31.196208+0000","last_unstale":"2023-11-24T06:12:32.168708+0000","last_undegraded":"2023-11-24T06:12:32.168708+0000","last_fullsized":"2023-11-24T06:12:32.168708+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_clean_scrub_stamp":"2023-11-24T06:12:30.127407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:58:17.965765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-24T06:12:32.718242+0000","last_change":"2023-11-24T06:12:31.198155+0000","last_active":"2023-11-24T06:12:32.718242+0000","last_peered":"2023-11-24T06:12:32.718242+0000","last_clean":"2023-11-24T06:12:32.718242+0000","last_became_active":"2023-11-24T06:12:31.197563+0000","last_became_peered":"2023-11-24T06:12:31.197563+0000","last_unstale":"2023-11-24T06:12:32.718242+0000","last_undegraded":"2023-11-24T06:12:32.718242+0000","last_fullsized":"2023-11-24T06:12:32.718242+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_clean_scrub_stamp":"2023-11-24T06:12:30.127407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:34:02.104830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-24T06:12:32.160965+0000","last_change":"2023-11-24T06:12:31.196369+0000","last_active":"2023-11-24T06:12:32.160965+0000","last_peered":"2023-11-24T06:12:32.160965+0000","last_clean":"2023-11-24T06:12:32.160965+0000","last_became_active":"2023-11-24T06:12:31.195118+0000","last_became_peered":"2023-11-24T06:12:31.195118+0000","last_unstale":"2023-11-24T06:12:32.160965+0000","last_undegraded":"2023-11-24T06:12:32.160965+0000","last_fullsized":"2023-11-24T06:12:32.160965+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_clean_scrub_stamp":"2023-11-24T06:12:30.127407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:49:00.756482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-24T06:12:32.167019+0000","last_change":"2023-11-24T06:12:31.190534+0000","last_active":"2023-11-24T06:12:32.167019+0000","last_peered":"2023-11-24T06:12:32.167019+0000","last_clean":"2023-11-24T06:12:32.167019+0000","last_became_active":"2023-11-24T06:12:31.190219+0000","last_became_peered":"2023-11-24T06:12:31.190219+0000","last_unstale":"2023-11-24T06:12:32.167019+0000","last_undegraded":"2023-11-24T06:12:32.167019+0000","last_fullsized":"2023-11-24T06:12:32.167019+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_clean_scrub_stamp":"2023-11-24T06:12:30.127407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:13:44.410307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-24T06:12:32.160719+0000","last_change":"2023-11-24T06:12:31.196925+0000","last_active":"2023-11-24T06:12:32.160719+0000","last_peered":"2023-11-24T06:12:32.160719+0000","last_clean":"2023-11-24T06:12:32.160719+0000","last_became_active":"2023-11-24T06:12:31.196658+0000","last_became_peered":"2023-11-24T06:12:31.196658+0000","last_unstale":"2023-11-24T06:12:32.160719+0000","last_undegraded":"2023-11-24T06:12:32.160719+0000","last_fullsized":"2023-11-24T06:12:32.160719+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_clean_scrub_stamp":"2023-11-24T06:12:30.127407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:25:39.965248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-24T06:12:32.167805+0000","last_change":"2023-11-24T06:12:31.196652+0000","last_active":"2023-11-24T06:12:32.167805+0000","last_peered":"2023-11-24T06:12:32.167805+0000","last_clean":"2023-11-24T06:12:32.167805+0000","last_became_active":"2023-11-24T06:12:31.196269+0000","last_became_peered":"2023-11-24T06:12:31.196269+0000","last_unstale":"2023-11-24T06:12:32.167805+0000","last_undegraded":"2023-11-24T06:12:32.167805+0000","last_fullsized":"2023-11-24T06:12:32.167805+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_clean_scrub_stamp":"2023-11-24T06:12:30.127407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:40:20.908351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-24T06:12:32.165936+0000","last_change":"2023-11-24T06:12:31.180751+0000","last_active":"2023-11-24T06:12:32.165936+0000","last_peered":"2023-11-24T06:12:32.165936+0000","last_clean":"2023-11-24T06:12:32.165936+0000","last_became_active":"2023-11-24T06:12:31.180394+0000","last_became_peered":"2023-11-24T06:12:31.180394+0000","last_unstale":"2023-11-24T06:12:32.165936+0000","last_undegraded":"2023-11-24T06:12:32.165936+0000","last_fullsized":"2023-11-24T06:12:32.165936+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_clean_scrub_stamp":"2023-11-24T06:12:30.127407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:54:30.238447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-24T06:12:32.717630+0000","last_change":"2023-11-24T06:12:31.187460+0000","last_active":"2023-11-24T06:12:32.717630+0000","last_peered":"2023-11-24T06:12:32.717630+0000","last_clean":"2023-11-24T06:12:32.717630+0000","last_became_active":"2023-11-24T06:12:31.187186+0000","last_became_peered":"2023-11-24T06:12:31.187186+0000","last_unstale":"2023-11-24T06:12:32.717630+0000","last_undegraded":"2023-11-24T06:12:32.717630+0000","last_fullsized":"2023-11-24T06:12:32.717630+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_clean_scrub_stamp":"2023-11-24T06:12:30.127407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:55:23.096616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-24T06:12:32.717193+0000","last_change":"2023-11-24T06:12:31.176738+0000","last_active":"2023-11-24T06:12:32.717193+0000","last_peered":"2023-11-24T06:12:32.717193+0000","last_clean":"2023-11-24T06:12:32.717193+0000","last_became_active":"2023-11-24T06:12:31.176091+0000","last_became_peered":"2023-11-24T06:12:31.176091+0000","last_unstale":"2023-11-24T06:12:32.717193+0000","last_undegraded":"2023-11-24T06:12:32.717193+0000","last_fullsized":"2023-11-24T06:12:32.717193+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_clean_scrub_stamp":"2023-11-24T06:12:30.127407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:47:16.954718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-24T06:12:32.719159+0000","last_change":"2023-11-24T06:12:31.183783+0000","last_active":"2023-11-24T06:12:32.719159+0000","last_peered":"2023-11-24T06:12:32.719159+0000","last_clean":"2023-11-24T06:12:32.719159+0000","last_became_active":"2023-11-24T06:12:31.183086+0000","last_became_peered":"2023-11-24T06:12:31.183086+0000","last_unstale":"2023-11-24T06:12:32.719159+0000","last_undegraded":"2023-11-24T06:12:32.719159+0000","last_fullsized":"2023-11-24T06:12:32.719159+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_clean_scrub_stamp":"2023-11-24T06:12:30.127407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:53:02.969558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-24T06:12:32.717829+0000","last_change":"2023-11-24T06:12:31.190309+0000","last_active":"2023-11-24T06:12:32.717829+0000","last_peered":"2023-11-24T06:12:32.717829+0000","last_clean":"2023-11-24T06:12:32.717829+0000","last_became_active":"2023-11-24T06:12:31.189984+0000","last_became_peered":"2023-11-24T06:12:31.189984+0000","last_unstale":"2023-11-24T06:12:32.717829+0000","last_undegraded":"2023-11-24T06:12:32.717829+0000","last_fullsized":"2023-11-24T06:12:32.717829+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_clean_scrub_stamp":"2023-11-24T06:12:30.127407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:58:40.674890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-24T06:12:32.161213+0000","last_change":"2023-11-24T06:12:31.192915+0000","last_active":"2023-11-24T06:12:32.161213+0000","last_peered":"2023-11-24T06:12:32.161213+0000","last_clean":"2023-11-24T06:12:32.161213+0000","last_became_active":"2023-11-24T06:12:31.192601+0000","last_became_peered":"2023-11-24T06:12:31.192601+0000","last_unstale":"2023-11-24T06:12:32.161213+0000","last_undegraded":"2023-11-24T06:12:32.161213+0000","last_fullsized":"2023-11-24T06:12:32.161213+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_clean_scrub_stamp":"2023-11-24T06:12:30.127407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:57:21.103658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-24T06:12:32.167332+0000","last_change":"2023-11-24T06:12:31.185380+0000","last_active":"2023-11-24T06:12:32.167332+0000","last_peered":"2023-11-24T06:12:32.167332+0000","last_clean":"2023-11-24T06:12:32.167332+0000","last_became_active":"2023-11-24T06:12:31.184245+0000","last_became_peered":"2023-11-24T06:12:31.184245+0000","last_unstale":"2023-11-24T06:12:32.167332+0000","last_undegraded":"2023-11-24T06:12:32.167332+0000","last_fullsized":"2023-11-24T06:12:32.167332+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_clean_scrub_stamp":"2023-11-24T06:12:30.127407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:42:02.177045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-24T06:12:32.717105+0000","last_change":"2023-11-24T06:12:31.189192+0000","last_active":"2023-11-24T06:12:32.717105+0000","last_peered":"2023-11-24T06:12:32.717105+0000","last_clean":"2023-11-24T06:12:32.717105+0000","last_became_active":"2023-11-24T06:12:31.188899+0000","last_became_peered":"2023-11-24T06:12:31.188899+0000","last_unstale":"2023-11-24T06:12:32.717105+0000","last_undegraded":"2023-11-24T06:12:32.717105+0000","last_fullsized":"2023-11-24T06:12:32.717105+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_clean_scrub_stamp":"2023-11-24T06:12:30.127407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:30:35.115592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-24T06:12:32.716762+0000","last_change":"2023-11-24T06:12:31.193298+0000","last_active":"2023-11-24T06:12:32.716762+0000","last_peered":"2023-11-24T06:12:32.716762+0000","last_clean":"2023-11-24T06:12:32.716762+0000","last_became_active":"2023-11-24T06:12:31.192937+0000","last_became_peered":"2023-11-24T06:12:31.192937+0000","last_unstale":"2023-11-24T06:12:32.716762+0000","last_undegraded":"2023-11-24T06:12:32.716762+0000","last_fullsized":"2023-11-24T06:12:32.716762+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_clean_scrub_stamp":"2023-11-24T06:12:30.127407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:09:09.186156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-24T06:12:32.719941+0000","last_change":"2023-11-24T06:12:31.180838+0000","last_active":"2023-11-24T06:12:32.719941+0000","last_peered":"2023-11-24T06:12:32.719941+0000","last_clean":"2023-11-24T06:12:32.719941+0000","last_became_active":"2023-11-24T06:12:31.179135+0000","last_became_peered":"2023-11-24T06:12:31.179135+0000","last_unstale":"2023-11-24T06:12:32.719941+0000","last_undegraded":"2023-11-24T06:12:32.719941+0000","last_fullsized":"2023-11-24T06:12:32.719941+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_clean_scrub_stamp":"2023-11-24T06:12:30.127407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:10:05.057643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"51'4","reported_seq":16,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-24T06:12:32.720108+0000","last_change":"2023-11-24T06:12:30.170926+0000","last_active":"2023-11-24T06:12:32.720108+0000","last_peered":"2023-11-24T06:12:32.720108+0000","last_clean":"2023-11-24T06:12:32.720108+0000","last_became_active":"2023-11-24T06:12:30.169994+0000","last_became_peered":"2023-11-24T06:12:30.169994+0000","last_unstale":"2023-11-24T06:12:32.720108+0000","last_undegraded":"2023-11-24T06:12:32.720108+0000","last_fullsized":"2023-11-24T06:12:32.720108+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:29.113984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:29.113984+0000","last_clean_scrub_stamp":"2023-11-24T06:12:29.113984+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:35:49.905218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-24T06:12:32.715481+0000","last_change":"2023-11-24T06:12:31.189759+0000","last_active":"2023-11-24T06:12:32.715481+0000","last_peered":"2023-11-24T06:12:32.715481+0000","last_clean":"2023-11-24T06:12:32.715481+0000","last_became_active":"2023-11-24T06:12:31.189333+0000","last_became_peered":"2023-11-24T06:12:31.189333+0000","last_unstale":"2023-11-24T06:12:32.715481+0000","last_undegraded":"2023-11-24T06:12:32.715481+0000","last_fullsized":"2023-11-24T06:12:32.715481+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_clean_scrub_stamp":"2023-11-24T06:12:30.127407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:04:42.070586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"51'1","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-24T06:12:32.161038+0000","last_change":"2023-11-24T06:12:30.172656+0000","last_active":"2023-11-24T06:12:32.161038+0000","last_peered":"2023-11-24T06:12:32.161038+0000","last_clean":"2023-11-24T06:12:32.161038+0000","last_became_active":"2023-11-24T06:12:30.172351+0000","last_became_peered":"2023-11-24T06:12:30.172351+0000","last_unstale":"2023-11-24T06:12:32.161038+0000","last_undegraded":"2023-11-24T06:12:32.161038+0000","last_fullsized":"2023-11-24T06:12:32.161038+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:29.113984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:29.113984+0000","last_clean_scrub_stamp":"2023-11-24T06:12:29.113984+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:58:16.952342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-24T06:12:32.718885+0000","last_change":"2023-11-24T06:12:31.183384+0000","last_active":"2023-11-24T06:12:32.718885+0000","last_peered":"2023-11-24T06:12:32.718885+0000","last_clean":"2023-11-24T06:12:32.718885+0000","last_became_active":"2023-11-24T06:12:31.182743+0000","last_became_peered":"2023-11-24T06:12:31.182743+0000","last_unstale":"2023-11-24T06:12:32.718885+0000","last_undegraded":"2023-11-24T06:12:32.718885+0000","last_fullsized":"2023-11-24T06:12:32.718885+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_clean_scrub_stamp":"2023-11-24T06:12:30.127407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:34:48.602430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"51'2","reported_seq":14,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-24T06:12:32.719579+0000","last_change":"2023-11-24T06:12:30.164705+0000","last_active":"2023-11-24T06:12:32.719579+0000","last_peered":"2023-11-24T06:12:32.719579+0000","last_clean":"2023-11-24T06:12:32.719579+0000","last_became_active":"2023-11-24T06:12:30.161249+0000","last_became_peered":"2023-11-24T06:12:30.161249+0000","last_unstale":"2023-11-24T06:12:32.719579+0000","last_undegraded":"2023-11-24T06:12:32.719579+0000","last_fullsized":"2023-11-24T06:12:32.719579+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:29.113984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:29.113984+0000","last_clean_scrub_stamp":"2023-11-24T06:12:29.113984+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:34:00.242285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-24T06:12:32.719478+0000","last_change":"2023-11-24T06:12:31.184020+0000","last_active":"2023-11-24T06:12:32.719478+0000","last_peered":"2023-11-24T06:12:32.719478+0000","last_clean":"2023-11-24T06:12:32.719478+0000","last_became_active":"2023-11-24T06:12:31.183498+0000","last_became_peered":"2023-11-24T06:12:31.183498+0000","last_unstale":"2023-11-24T06:12:32.719478+0000","last_undegraded":"2023-11-24T06:12:32.719478+0000","last_fullsized":"2023-11-24T06:12:32.719478+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_clean_scrub_stamp":"2023-11-24T06:12:30.127407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:59:24.875576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"51'2","reported_seq":14,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-24T06:12:32.717074+0000","last_change":"2023-11-24T06:12:30.170250+0000","last_active":"2023-11-24T06:12:32.717074+0000","last_peered":"2023-11-24T06:12:32.717074+0000","last_clean":"2023-11-24T06:12:32.717074+0000","last_became_active":"2023-11-24T06:12:30.169714+0000","last_became_peered":"2023-11-24T06:12:30.169714+0000","last_unstale":"2023-11-24T06:12:32.717074+0000","last_undegraded":"2023-11-24T06:12:32.717074+0000","last_fullsized":"2023-11-24T06:12:32.717074+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:29.113984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:29.113984+0000","last_clean_scrub_stamp":"2023-11-24T06:12:29.113984+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:52:04.468529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-24T06:12:32.161047+0000","last_change":"2023-11-24T06:12:31.193576+0000","last_active":"2023-11-24T06:12:32.161047+0000","last_peered":"2023-11-24T06:12:32.161047+0000","last_clean":"2023-11-24T06:12:32.161047+0000","last_became_active":"2023-11-24T06:12:31.193239+0000","last_became_peered":"2023-11-24T06:12:31.193239+0000","last_unstale":"2023-11-24T06:12:32.161047+0000","last_undegraded":"2023-11-24T06:12:32.161047+0000","last_fullsized":"2023-11-24T06:12:32.161047+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_clean_scrub_stamp":"2023-11-24T06:12:30.127407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:11:29.067383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"51'1","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-24T06:12:32.717314+0000","last_change":"2023-11-24T06:12:30.168621+0000","last_active":"2023-11-24T06:12:32.717314+0000","last_peered":"2023-11-24T06:12:32.717314+0000","last_clean":"2023-11-24T06:12:32.717314+0000","last_became_active":"2023-11-24T06:12:30.168312+0000","last_became_peered":"2023-11-24T06:12:30.168312+0000","last_unstale":"2023-11-24T06:12:32.717314+0000","last_undegraded":"2023-11-24T06:12:32.717314+0000","last_fullsized":"2023-11-24T06:12:32.717314+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:29.113984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:29.113984+0000","last_clean_scrub_stamp":"2023-11-24T06:12:29.113984+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:40:59.512025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-24T06:12:32.161074+0000","last_change":"2023-11-24T06:12:31.199120+0000","last_active":"2023-11-24T06:12:32.161074+0000","last_peered":"2023-11-24T06:12:32.161074+0000","last_clean":"2023-11-24T06:12:32.161074+0000","last_became_active":"2023-11-24T06:12:31.198767+0000","last_became_peered":"2023-11-24T06:12:31.198767+0000","last_unstale":"2023-11-24T06:12:32.161074+0000","last_undegraded":"2023-11-24T06:12:32.161074+0000","last_fullsized":"2023-11-24T06:12:32.161074+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_clean_scrub_stamp":"2023-11-24T06:12:30.127407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:45:39.815231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"51'1","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-24T06:12:32.718508+0000","last_change":"2023-11-24T06:12:30.162112+0000","last_active":"2023-11-24T06:12:32.718508+0000","last_peered":"2023-11-24T06:12:32.718508+0000","last_clean":"2023-11-24T06:12:32.718508+0000","last_became_active":"2023-11-24T06:12:30.161858+0000","last_became_peered":"2023-11-24T06:12:30.161858+0000","last_unstale":"2023-11-24T06:12:32.718508+0000","last_undegraded":"2023-11-24T06:12:32.718508+0000","last_fullsized":"2023-11-24T06:12:32.718508+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:29.113984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:29.113984+0000","last_clean_scrub_stamp":"2023-11-24T06:12:29.113984+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:04:41.057163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-24T06:12:32.165857+0000","last_change":"2023-11-24T06:12:31.187113+0000","last_active":"2023-11-24T06:12:32.165857+0000","last_peered":"2023-11-24T06:12:32.165857+0000","last_clean":"2023-11-24T06:12:32.165857+0000","last_became_active":"2023-11-24T06:12:31.186840+0000","last_became_peered":"2023-11-24T06:12:31.186840+0000","last_unstale":"2023-11-24T06:12:32.165857+0000","last_undegraded":"2023-11-24T06:12:32.165857+0000","last_fullsized":"2023-11-24T06:12:32.165857+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_clean_scrub_stamp":"2023-11-24T06:12:30.127407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:13:10.229382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-24T06:12:32.716372+0000","last_change":"2023-11-24T06:12:30.167327+0000","last_active":"2023-11-24T06:12:32.716372+0000","last_peered":"2023-11-24T06:12:32.716372+0000","last_clean":"2023-11-24T06:12:32.716372+0000","last_became_active":"2023-11-24T06:12:30.165432+0000","last_became_peered":"2023-11-24T06:12:30.165432+0000","last_unstale":"2023-11-24T06:12:32.716372+0000","last_undegraded":"2023-11-24T06:12:32.716372+0000","last_fullsized":"2023-11-24T06:12:32.716372+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:29.113984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:29.113984+0000","last_clean_scrub_stamp":"2023-11-24T06:12:29.113984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:32:38.868244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-24T06:12:32.161188+0000","last_change":"2023-11-24T06:12:31.195684+0000","last_active":"2023-11-24T06:12:32.161188+0000","last_peered":"2023-11-24T06:12:32.161188+0000","last_clean":"2023-11-24T06:12:32.161188+0000","last_became_active":"2023-11-24T06:12:31.195158+0000","last_became_peered":"2023-11-24T06:12:31.195158+0000","last_unstale":"2023-11-24T06:12:32.161188+0000","last_undegraded":"2023-11-24T06:12:32.161188+0000","last_fullsized":"2023-11-24T06:12:32.161188+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_clean_scrub_stamp":"2023-11-24T06:12:30.127407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:22:56.206609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-24T06:12:32.715693+0000","last_change":"2023-11-24T06:12:30.173265+0000","last_active":"2023-11-24T06:12:32.715693+0000","last_peered":"2023-11-24T06:12:32.715693+0000","last_clean":"2023-11-24T06:12:32.715693+0000","last_became_active":"2023-11-24T06:12:30.173000+0000","last_became_peered":"2023-11-24T06:12:30.173000+0000","last_unstale":"2023-11-24T06:12:32.715693+0000","last_undegraded":"2023-11-24T06:12:32.715693+0000","last_fullsized":"2023-11-24T06:12:32.715693+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:29.113984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:29.113984+0000","last_clean_scrub_stamp":"2023-11-24T06:12:29.113984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:11:06.890822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-24T06:12:32.719276+0000","last_change":"2023-11-24T06:12:31.182308+0000","last_active":"2023-11-24T06:12:32.719276+0000","last_peered":"2023-11-24T06:12:32.719276+0000","last_clean":"2023-11-24T06:12:32.719276+0000","last_became_active":"2023-11-24T06:12:31.181750+0000","last_became_peered":"2023-11-24T06:12:31.181750+0000","last_unstale":"2023-11-24T06:12:32.719276+0000","last_undegraded":"2023-11-24T06:12:32.719276+0000","last_fullsized":"2023-11-24T06:12:32.719276+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_clean_scrub_stamp":"2023-11-24T06:12:30.127407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:38:37.898801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-24T06:12:32.168578+0000","last_change":"2023-11-24T06:12:30.177615+0000","last_active":"2023-11-24T06:12:32.168578+0000","last_peered":"2023-11-24T06:12:32.168578+0000","last_clean":"2023-11-24T06:12:32.168578+0000","last_became_active":"2023-11-24T06:12:30.177282+0000","last_became_peered":"2023-11-24T06:12:30.177282+0000","last_unstale":"2023-11-24T06:12:32.168578+0000","last_undegraded":"2023-11-24T06:12:32.168578+0000","last_fullsized":"2023-11-24T06:12:32.168578+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:29.113984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:29.113984+0000","last_clean_scrub_stamp":"2023-11-24T06:12:29.113984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:43:23.386785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-24T06:12:32.168507+0000","last_change":"2023-11-24T06:12:31.196996+0000","last_active":"2023-11-24T06:12:32.168507+0000","last_peered":"2023-11-24T06:12:32.168507+0000","last_clean":"2023-11-24T06:12:32.168507+0000","last_became_active":"2023-11-24T06:12:31.196397+0000","last_became_peered":"2023-11-24T06:12:31.196397+0000","last_unstale":"2023-11-24T06:12:32.168507+0000","last_undegraded":"2023-11-24T06:12:32.168507+0000","last_fullsized":"2023-11-24T06:12:32.168507+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_clean_scrub_stamp":"2023-11-24T06:12:30.127407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:32:10.194351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-24T06:12:32.161035+0000","last_change":"2023-11-24T06:12:30.169593+0000","last_active":"2023-11-24T06:12:32.161035+0000","last_peered":"2023-11-24T06:12:32.161035+0000","last_clean":"2023-11-24T06:12:32.161035+0000","last_became_active":"2023-11-24T06:12:30.169000+0000","last_became_peered":"2023-11-24T06:12:30.169000+0000","last_unstale":"2023-11-24T06:12:32.161035+0000","last_undegraded":"2023-11-24T06:12:32.161035+0000","last_fullsized":"2023-11-24T06:12:32.161035+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:29.113984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:29.113984+0000","last_clean_scrub_stamp":"2023-11-24T06:12:29.113984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:01:54.756749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-24T06:12:32.715773+0000","last_change":"2023-11-24T06:12:31.180439+0000","last_active":"2023-11-24T06:12:32.715773+0000","last_peered":"2023-11-24T06:12:32.715773+0000","last_clean":"2023-11-24T06:12:32.715773+0000","last_became_active":"2023-11-24T06:12:31.179555+0000","last_became_peered":"2023-11-24T06:12:31.179555+0000","last_unstale":"2023-11-24T06:12:32.715773+0000","last_undegraded":"2023-11-24T06:12:32.715773+0000","last_fullsized":"2023-11-24T06:12:32.715773+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_clean_scrub_stamp":"2023-11-24T06:12:30.127407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:38:37.898801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"51'3","reported_seq":15,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-24T06:12:32.168963+0000","last_change":"2023-11-24T06:12:30.176966+0000","last_active":"2023-11-24T06:12:32.168963+0000","last_peered":"2023-11-24T06:12:32.168963+0000","last_clean":"2023-11-24T06:12:32.168963+0000","last_became_active":"2023-11-24T06:12:30.176673+0000","last_became_peered":"2023-11-24T06:12:30.176673+0000","last_unstale":"2023-11-24T06:12:32.168963+0000","last_undegraded":"2023-11-24T06:12:32.168963+0000","last_fullsized":"2023-11-24T06:12:32.168963+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:29.113984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:29.113984+0000","last_clean_scrub_stamp":"2023-11-24T06:12:29.113984+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:34:47.589007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-24T06:12:32.168534+0000","last_change":"2023-11-24T06:12:31.194642+0000","last_active":"2023-11-24T06:12:32.168534+0000","last_peered":"2023-11-24T06:12:32.168534+0000","last_clean":"2023-11-24T06:12:32.168534+0000","last_became_active":"2023-11-24T06:12:31.194294+0000","last_became_peered":"2023-11-24T06:12:31.194294+0000","last_unstale":"2023-11-24T06:12:32.168534+0000","last_undegraded":"2023-11-24T06:12:32.168534+0000","last_fullsized":"2023-11-24T06:12:32.168534+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_clean_scrub_stamp":"2023-11-24T06:12:30.127407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:24:35.462590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"51'6","reported_seq":18,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-24T06:12:32.718895+0000","last_change":"2023-11-24T06:12:30.164701+0000","last_active":"2023-11-24T06:12:32.718895+0000","last_peered":"2023-11-24T06:12:32.718895+0000","last_clean":"2023-11-24T06:12:32.718895+0000","last_became_active":"2023-11-24T06:12:30.162468+0000","last_became_peered":"2023-11-24T06:12:30.162468+0000","last_unstale":"2023-11-24T06:12:32.718895+0000","last_undegraded":"2023-11-24T06:12:32.718895+0000","last_fullsized":"2023-11-24T06:12:32.718895+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:29.113984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:29.113984+0000","last_clean_scrub_stamp":"2023-11-24T06:12:29.113984+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:22:54.523841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-24T06:12:32.716146+0000","last_change":"2023-11-24T06:12:31.185563+0000","last_active":"2023-11-24T06:12:32.716146+0000","last_peered":"2023-11-24T06:12:32.716146+0000","last_clean":"2023-11-24T06:12:32.716146+0000","last_became_active":"2023-11-24T06:12:31.185210+0000","last_became_peered":"2023-11-24T06:12:31.185210+0000","last_unstale":"2023-11-24T06:12:32.716146+0000","last_undegraded":"2023-11-24T06:12:32.716146+0000","last_fullsized":"2023-11-24T06:12:32.716146+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_clean_scrub_stamp":"2023-11-24T06:12:30.127407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:35:50.918641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"51'2","reported_seq":14,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-24T06:12:32.161065+0000","last_change":"2023-11-24T06:12:30.168367+0000","last_active":"2023-11-24T06:12:32.161065+0000","last_peered":"2023-11-24T06:12:32.161065+0000","last_clean":"2023-11-24T06:12:32.161065+0000","last_became_active":"2023-11-24T06:12:30.168091+0000","last_became_peered":"2023-11-24T06:12:30.168091+0000","last_unstale":"2023-11-24T06:12:32.161065+0000","last_undegraded":"2023-11-24T06:12:32.161065+0000","last_fullsized":"2023-11-24T06:12:32.161065+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:29.113984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:29.113984+0000","last_clean_scrub_stamp":"2023-11-24T06:12:29.113984+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:19:45.435002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-24T06:12:32.167131+0000","last_change":"2023-11-24T06:12:31.193652+0000","last_active":"2023-11-24T06:12:32.167131+0000","last_peered":"2023-11-24T06:12:32.167131+0000","last_clean":"2023-11-24T06:12:32.167131+0000","last_became_active":"2023-11-24T06:12:31.193264+0000","last_became_peered":"2023-11-24T06:12:31.193264+0000","last_unstale":"2023-11-24T06:12:32.167131+0000","last_undegraded":"2023-11-24T06:12:32.167131+0000","last_fullsized":"2023-11-24T06:12:32.167131+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_clean_scrub_stamp":"2023-11-24T06:12:30.127407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:32:07.709266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-24T06:12:32.165966+0000","last_change":"2023-11-24T06:12:30.170269+0000","last_active":"2023-11-24T06:12:32.165966+0000","last_peered":"2023-11-24T06:12:32.165966+0000","last_clean":"2023-11-24T06:12:32.165966+0000","last_became_active":"2023-11-24T06:12:30.169940+0000","last_became_peered":"2023-11-24T06:12:30.169940+0000","last_unstale":"2023-11-24T06:12:32.165966+0000","last_undegraded":"2023-11-24T06:12:32.165966+0000","last_fullsized":"2023-11-24T06:12:32.165966+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:29.113984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:29.113984+0000","last_clean_scrub_stamp":"2023-11-24T06:12:29.113984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:40:51.288335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-24T06:12:32.715676+0000","last_change":"2023-11-24T06:12:31.183845+0000","last_active":"2023-11-24T06:12:32.715676+0000","last_peered":"2023-11-24T06:12:32.715676+0000","last_clean":"2023-11-24T06:12:32.715676+0000","last_became_active":"2023-11-24T06:12:31.183296+0000","last_became_peered":"2023-11-24T06:12:31.183296+0000","last_unstale":"2023-11-24T06:12:32.715676+0000","last_undegraded":"2023-11-24T06:12:32.715676+0000","last_fullsized":"2023-11-24T06:12:32.715676+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_clean_scrub_stamp":"2023-11-24T06:12:30.127407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:14:56.855135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-24T06:12:32.716390+0000","last_change":"2023-11-24T06:12:30.174303+0000","last_active":"2023-11-24T06:12:32.716390+0000","last_peered":"2023-11-24T06:12:32.716390+0000","last_clean":"2023-11-24T06:12:32.716390+0000","last_became_active":"2023-11-24T06:12:30.173929+0000","last_became_peered":"2023-11-24T06:12:30.173929+0000","last_unstale":"2023-11-24T06:12:32.716390+0000","last_undegraded":"2023-11-24T06:12:32.716390+0000","last_fullsized":"2023-11-24T06:12:32.716390+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:29.113984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:29.113984+0000","last_clean_scrub_stamp":"2023-11-24T06:12:29.113984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:14:55.841712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-24T06:12:32.167937+0000","last_change":"2023-11-24T06:12:31.196653+0000","last_active":"2023-11-24T06:12:32.167937+0000","last_peered":"2023-11-24T06:12:32.167937+0000","last_clean":"2023-11-24T06:12:32.167937+0000","last_became_active":"2023-11-24T06:12:31.196275+0000","last_became_peered":"2023-11-24T06:12:31.196275+0000","last_unstale":"2023-11-24T06:12:32.167937+0000","last_undegraded":"2023-11-24T06:12:32.167937+0000","last_fullsized":"2023-11-24T06:12:32.167937+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_clean_scrub_stamp":"2023-11-24T06:12:30.127407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:02:11.931305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"51'8","reported_seq":20,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-24T06:12:32.165954+0000","last_change":"2023-11-24T06:12:30.174960+0000","last_active":"2023-11-24T06:12:32.165954+0000","last_peered":"2023-11-24T06:12:32.165954+0000","last_clean":"2023-11-24T06:12:32.165954+0000","last_became_active":"2023-11-24T06:12:30.174626+0000","last_became_peered":"2023-11-24T06:12:30.174626+0000","last_unstale":"2023-11-24T06:12:32.165954+0000","last_undegraded":"2023-11-24T06:12:32.165954+0000","last_fullsized":"2023-11-24T06:12:32.165954+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:29.113984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:29.113984+0000","last_clean_scrub_stamp":"2023-11-24T06:12:29.113984+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:47:15.941295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-24T06:12:32.168668+0000","last_change":"2023-11-24T06:12:31.191797+0000","last_active":"2023-11-24T06:12:32.168668+0000","last_peered":"2023-11-24T06:12:32.168668+0000","last_clean":"2023-11-24T06:12:32.168668+0000","last_became_active":"2023-11-24T06:12:31.191348+0000","last_became_peered":"2023-11-24T06:12:31.191348+0000","last_unstale":"2023-11-24T06:12:32.168668+0000","last_undegraded":"2023-11-24T06:12:32.168668+0000","last_fullsized":"2023-11-24T06:12:32.168668+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_clean_scrub_stamp":"2023-11-24T06:12:30.127407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:19:46.448425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"51'1","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-24T06:12:32.161359+0000","last_change":"2023-11-24T06:12:30.172737+0000","last_active":"2023-11-24T06:12:32.161359+0000","last_peered":"2023-11-24T06:12:32.161359+0000","last_clean":"2023-11-24T06:12:32.161359+0000","last_became_active":"2023-11-24T06:12:30.172021+0000","last_became_peered":"2023-11-24T06:12:30.172021+0000","last_unstale":"2023-11-24T06:12:32.161359+0000","last_undegraded":"2023-11-24T06:12:32.161359+0000","last_fullsized":"2023-11-24T06:12:32.161359+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:29.113984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:29.113984+0000","last_clean_scrub_stamp":"2023-11-24T06:12:29.113984+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:32:09.180928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-24T06:12:32.717889+0000","last_change":"2023-11-24T06:12:31.185551+0000","last_active":"2023-11-24T06:12:32.717889+0000","last_peered":"2023-11-24T06:12:32.717889+0000","last_clean":"2023-11-24T06:12:32.717889+0000","last_became_active":"2023-11-24T06:12:31.185208+0000","last_became_peered":"2023-11-24T06:12:31.185208+0000","last_unstale":"2023-11-24T06:12:32.717889+0000","last_undegraded":"2023-11-24T06:12:32.717889+0000","last_fullsized":"2023-11-24T06:12:32.717889+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_clean_scrub_stamp":"2023-11-24T06:12:30.127407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:49:46.518805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"51'2","reported_seq":14,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-24T06:12:32.720049+0000","last_change":"2023-11-24T06:12:30.170427+0000","last_active":"2023-11-24T06:12:32.720049+0000","last_peered":"2023-11-24T06:12:32.720049+0000","last_clean":"2023-11-24T06:12:32.720049+0000","last_became_active":"2023-11-24T06:12:30.170062+0000","last_became_peered":"2023-11-24T06:12:30.170062+0000","last_unstale":"2023-11-24T06:12:32.720049+0000","last_undegraded":"2023-11-24T06:12:32.720049+0000","last_fullsized":"2023-11-24T06:12:32.720049+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:29.113984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:29.113984+0000","last_clean_scrub_stamp":"2023-11-24T06:12:29.113984+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:02:22.416616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-24T06:12:32.161122+0000","last_change":"2023-11-24T06:12:31.188508+0000","last_active":"2023-11-24T06:12:32.161122+0000","last_peered":"2023-11-24T06:12:32.161122+0000","last_clean":"2023-11-24T06:12:32.161122+0000","last_became_active":"2023-11-24T06:12:31.188217+0000","last_became_peered":"2023-11-24T06:12:31.188217+0000","last_unstale":"2023-11-24T06:12:32.161122+0000","last_undegraded":"2023-11-24T06:12:32.161122+0000","last_fullsized":"2023-11-24T06:12:32.161122+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_clean_scrub_stamp":"2023-11-24T06:12:30.127407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:35:02.628615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"51'1","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-24T06:12:32.716761+0000","last_change":"2023-11-24T06:12:30.172394+0000","last_active":"2023-11-24T06:12:32.716761+0000","last_peered":"2023-11-24T06:12:32.716761+0000","last_clean":"2023-11-24T06:12:32.716761+0000","last_became_active":"2023-11-24T06:12:30.172115+0000","last_became_peered":"2023-11-24T06:12:30.172115+0000","last_unstale":"2023-11-24T06:12:32.716761+0000","last_undegraded":"2023-11-24T06:12:32.716761+0000","last_fullsized":"2023-11-24T06:12:32.716761+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:29.113984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:29.113984+0000","last_clean_scrub_stamp":"2023-11-24T06:12:29.113984+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:34:00.242285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-24T06:12:32.168881+0000","last_change":"2023-11-24T06:12:31.194745+0000","last_active":"2023-11-24T06:12:32.168881+0000","last_peered":"2023-11-24T06:12:32.168881+0000","last_clean":"2023-11-24T06:12:32.168881+0000","last_became_active":"2023-11-24T06:12:31.194449+0000","last_became_peered":"2023-11-24T06:12:31.194449+0000","last_unstale":"2023-11-24T06:12:32.168881+0000","last_undegraded":"2023-11-24T06:12:32.168881+0000","last_fullsized":"2023-11-24T06:12:32.168881+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_clean_scrub_stamp":"2023-11-24T06:12:30.127407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:01:55.770172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-24T06:12:32.719124+0000","last_change":"2023-11-24T06:12:30.174464+0000","last_active":"2023-11-24T06:12:32.719124+0000","last_peered":"2023-11-24T06:12:32.719124+0000","last_clean":"2023-11-24T06:12:32.719124+0000","last_became_active":"2023-11-24T06:12:30.174177+0000","last_became_peered":"2023-11-24T06:12:30.174177+0000","last_unstale":"2023-11-24T06:12:32.719124+0000","last_undegraded":"2023-11-24T06:12:32.719124+0000","last_fullsized":"2023-11-24T06:12:32.719124+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:29.113984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:29.113984+0000","last_clean_scrub_stamp":"2023-11-24T06:12:29.113984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:03:02.518611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-24T06:12:32.716194+0000","last_change":"2023-11-24T06:12:31.186486+0000","last_active":"2023-11-24T06:12:32.716194+0000","last_peered":"2023-11-24T06:12:32.716194+0000","last_clean":"2023-11-24T06:12:32.716194+0000","last_became_active":"2023-11-24T06:12:31.184729+0000","last_became_peered":"2023-11-24T06:12:31.184729+0000","last_unstale":"2023-11-24T06:12:32.716194+0000","last_undegraded":"2023-11-24T06:12:32.716194+0000","last_fullsized":"2023-11-24T06:12:32.716194+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_clean_scrub_stamp":"2023-11-24T06:12:30.127407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:03:03.532034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"51'2","reported_seq":14,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-24T06:12:32.161188+0000","last_change":"2023-11-24T06:12:30.165766+0000","last_active":"2023-11-24T06:12:32.161188+0000","last_peered":"2023-11-24T06:12:32.161188+0000","last_clean":"2023-11-24T06:12:32.161188+0000","last_became_active":"2023-11-24T06:12:30.165281+0000","last_became_peered":"2023-11-24T06:12:30.165281+0000","last_unstale":"2023-11-24T06:12:32.161188+0000","last_undegraded":"2023-11-24T06:12:32.161188+0000","last_fullsized":"2023-11-24T06:12:32.161188+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:29.113984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:29.113984+0000","last_clean_scrub_stamp":"2023-11-24T06:12:29.113984+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:46:03.597280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-24T06:12:32.717994+0000","last_change":"2023-11-24T06:12:31.187800+0000","last_active":"2023-11-24T06:12:32.717994+0000","last_peered":"2023-11-24T06:12:32.717994+0000","last_clean":"2023-11-24T06:12:32.717994+0000","last_became_active":"2023-11-24T06:12:31.183035+0000","last_became_peered":"2023-11-24T06:12:31.183035+0000","last_unstale":"2023-11-24T06:12:32.717994+0000","last_undegraded":"2023-11-24T06:12:32.717994+0000","last_fullsized":"2023-11-24T06:12:32.717994+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_clean_scrub_stamp":"2023-11-24T06:12:30.127407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:33:17.733215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-24T06:12:32.715511+0000","last_change":"2023-11-24T06:12:30.165882+0000","last_active":"2023-11-24T06:12:32.715511+0000","last_peered":"2023-11-24T06:12:32.715511+0000","last_clean":"2023-11-24T06:12:32.715511+0000","last_became_active":"2023-11-24T06:12:30.165610+0000","last_became_peered":"2023-11-24T06:12:30.165610+0000","last_unstale":"2023-11-24T06:12:32.715511+0000","last_undegraded":"2023-11-24T06:12:32.715511+0000","last_fullsized":"2023-11-24T06:12:32.715511+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:29.113984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:29.113984+0000","last_clean_scrub_stamp":"2023-11-24T06:12:29.113984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:23:52.481214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-24T06:12:32.716939+0000","last_change":"2023-11-24T06:12:31.185398+0000","last_active":"2023-11-24T06:12:32.716939+0000","last_peered":"2023-11-24T06:12:32.716939+0000","last_clean":"2023-11-24T06:12:32.716939+0000","last_became_active":"2023-11-24T06:12:31.185056+0000","last_became_peered":"2023-11-24T06:12:31.185056+0000","last_unstale":"2023-11-24T06:12:32.716939+0000","last_undegraded":"2023-11-24T06:12:32.716939+0000","last_fullsized":"2023-11-24T06:12:32.716939+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_clean_scrub_stamp":"2023-11-24T06:12:30.127407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:47:16.954718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-24T06:12:32.715776+0000","last_change":"2023-11-24T06:12:30.171661+0000","last_active":"2023-11-24T06:12:32.715776+0000","last_peered":"2023-11-24T06:12:32.715776+0000","last_clean":"2023-11-24T06:12:32.715776+0000","last_became_active":"2023-11-24T06:12:30.170946+0000","last_became_peered":"2023-11-24T06:12:30.170946+0000","last_unstale":"2023-11-24T06:12:32.715776+0000","last_undegraded":"2023-11-24T06:12:32.715776+0000","last_fullsized":"2023-11-24T06:12:32.715776+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:29.113984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:29.113984+0000","last_clean_scrub_stamp":"2023-11-24T06:12:29.113984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:20:42.313069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-24T06:12:32.717564+0000","last_change":"2023-11-24T06:12:31.189248+0000","last_active":"2023-11-24T06:12:32.717564+0000","last_peered":"2023-11-24T06:12:32.717564+0000","last_clean":"2023-11-24T06:12:32.717564+0000","last_became_active":"2023-11-24T06:12:31.188097+0000","last_became_peered":"2023-11-24T06:12:31.188097+0000","last_unstale":"2023-11-24T06:12:32.717564+0000","last_undegraded":"2023-11-24T06:12:32.717564+0000","last_fullsized":"2023-11-24T06:12:32.717564+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_clean_scrub_stamp":"2023-11-24T06:12:30.127407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:33:17.733215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"51'2","reported_seq":14,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-24T06:12:32.715672+0000","last_change":"2023-11-24T06:12:30.173946+0000","last_active":"2023-11-24T06:12:32.715672+0000","last_peered":"2023-11-24T06:12:32.715672+0000","last_clean":"2023-11-24T06:12:32.715672+0000","last_became_active":"2023-11-24T06:12:30.173665+0000","last_became_peered":"2023-11-24T06:12:30.173665+0000","last_unstale":"2023-11-24T06:12:32.715672+0000","last_undegraded":"2023-11-24T06:12:32.715672+0000","last_fullsized":"2023-11-24T06:12:32.715672+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:29.113984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:29.113984+0000","last_clean_scrub_stamp":"2023-11-24T06:12:29.113984+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:14:55.841712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-24T06:12:32.718086+0000","last_change":"2023-11-24T06:12:31.187788+0000","last_active":"2023-11-24T06:12:32.718086+0000","last_peered":"2023-11-24T06:12:32.718086+0000","last_clean":"2023-11-24T06:12:32.718086+0000","last_became_active":"2023-11-24T06:12:31.187459+0000","last_became_peered":"2023-11-24T06:12:31.187459+0000","last_unstale":"2023-11-24T06:12:32.718086+0000","last_undegraded":"2023-11-24T06:12:32.718086+0000","last_fullsized":"2023-11-24T06:12:32.718086+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_clean_scrub_stamp":"2023-11-24T06:12:30.127407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:34:48.602430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-24T06:12:32.720152+0000","last_change":"2023-11-24T06:12:30.163488+0000","last_active":"2023-11-24T06:12:32.720152+0000","last_peered":"2023-11-24T06:12:32.720152+0000","last_clean":"2023-11-24T06:12:32.720152+0000","last_became_active":"2023-11-24T06:12:30.163196+0000","last_became_peered":"2023-11-24T06:12:30.163196+0000","last_unstale":"2023-11-24T06:12:32.720152+0000","last_undegraded":"2023-11-24T06:12:32.720152+0000","last_fullsized":"2023-11-24T06:12:32.720152+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:29.113984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:29.113984+0000","last_clean_scrub_stamp":"2023-11-24T06:12:29.113984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:29:09.486165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-24T06:12:32.715395+0000","last_change":"2023-11-24T06:12:31.177668+0000","last_active":"2023-11-24T06:12:32.715395+0000","last_peered":"2023-11-24T06:12:32.715395+0000","last_clean":"2023-11-24T06:12:32.715395+0000","last_became_active":"2023-11-24T06:12:31.177407+0000","last_became_peered":"2023-11-24T06:12:31.177407+0000","last_unstale":"2023-11-24T06:12:32.715395+0000","last_undegraded":"2023-11-24T06:12:32.715395+0000","last_fullsized":"2023-11-24T06:12:32.715395+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_clean_scrub_stamp":"2023-11-24T06:12:30.127407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:34:53.658692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-24T06:12:32.166671+0000","last_change":"2023-11-24T06:12:30.162538+0000","last_active":"2023-11-24T06:12:32.166671+0000","last_peered":"2023-11-24T06:12:32.166671+0000","last_clean":"2023-11-24T06:12:32.166671+0000","last_became_active":"2023-11-24T06:12:30.161061+0000","last_became_peered":"2023-11-24T06:12:30.161061+0000","last_unstale":"2023-11-24T06:12:32.166671+0000","last_undegraded":"2023-11-24T06:12:32.166671+0000","last_fullsized":"2023-11-24T06:12:32.166671+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:29.113984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:29.113984+0000","last_clean_scrub_stamp":"2023-11-24T06:12:29.113984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:56:14.550653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-24T06:12:32.719603+0000","last_change":"2023-11-24T06:12:31.180889+0000","last_active":"2023-11-24T06:12:32.719603+0000","last_peered":"2023-11-24T06:12:32.719603+0000","last_clean":"2023-11-24T06:12:32.719603+0000","last_became_active":"2023-11-24T06:12:31.180089+0000","last_became_peered":"2023-11-24T06:12:31.180089+0000","last_unstale":"2023-11-24T06:12:32.719603+0000","last_undegraded":"2023-11-24T06:12:32.719603+0000","last_fullsized":"2023-11-24T06:12:32.719603+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_clean_scrub_stamp":"2023-11-24T06:12:30.127407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:45:14.012287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-24T06:12:32.717151+0000","last_change":"2023-11-24T06:12:30.165117+0000","last_active":"2023-11-24T06:12:32.717151+0000","last_peered":"2023-11-24T06:12:32.717151+0000","last_clean":"2023-11-24T06:12:32.717151+0000","last_became_active":"2023-11-24T06:12:30.164738+0000","last_became_peered":"2023-11-24T06:12:30.164738+0000","last_unstale":"2023-11-24T06:12:32.717151+0000","last_undegraded":"2023-11-24T06:12:32.717151+0000","last_fullsized":"2023-11-24T06:12:32.717151+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:29.113984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:29.113984+0000","last_clean_scrub_stamp":"2023-11-24T06:12:29.113984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:02:22.416616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-24T06:12:32.160876+0000","last_change":"2023-11-24T06:12:31.193439+0000","last_active":"2023-11-24T06:12:32.160876+0000","last_peered":"2023-11-24T06:12:32.160876+0000","last_clean":"2023-11-24T06:12:32.160876+0000","last_became_active":"2023-11-24T06:12:31.193147+0000","last_became_peered":"2023-11-24T06:12:31.193147+0000","last_unstale":"2023-11-24T06:12:32.160876+0000","last_undegraded":"2023-11-24T06:12:32.160876+0000","last_fullsized":"2023-11-24T06:12:32.160876+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_clean_scrub_stamp":"2023-11-24T06:12:30.127407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:09:42.396697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-24T06:12:32.716399+0000","last_change":"2023-11-24T06:12:30.168562+0000","last_active":"2023-11-24T06:12:32.716399+0000","last_peered":"2023-11-24T06:12:32.716399+0000","last_clean":"2023-11-24T06:12:32.716399+0000","last_became_active":"2023-11-24T06:12:30.168336+0000","last_became_peered":"2023-11-24T06:12:30.168336+0000","last_unstale":"2023-11-24T06:12:32.716399+0000","last_undegraded":"2023-11-24T06:12:32.716399+0000","last_fullsized":"2023-11-24T06:12:32.716399+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:29.113984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:29.113984+0000","last_clean_scrub_stamp":"2023-11-24T06:12:29.113984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:32:38.868244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-24T06:12:32.167394+0000","last_change":"2023-11-24T06:12:31.197065+0000","last_active":"2023-11-24T06:12:32.167394+0000","last_peered":"2023-11-24T06:12:32.167394+0000","last_clean":"2023-11-24T06:12:32.167394+0000","last_became_active":"2023-11-24T06:12:31.196691+0000","last_became_peered":"2023-11-24T06:12:31.196691+0000","last_unstale":"2023-11-24T06:12:32.167394+0000","last_undegraded":"2023-11-24T06:12:32.167394+0000","last_fullsized":"2023-11-24T06:12:32.167394+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_clean_scrub_stamp":"2023-11-24T06:12:30.127407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:25:07.777538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-24T06:12:32.718566+0000","last_change":"2023-11-24T06:12:30.162492+0000","last_active":"2023-11-24T06:12:32.718566+0000","last_peered":"2023-11-24T06:12:32.718566+0000","last_clean":"2023-11-24T06:12:32.718566+0000","last_became_active":"2023-11-24T06:12:30.162161+0000","last_became_peered":"2023-11-24T06:12:30.162161+0000","last_unstale":"2023-11-24T06:12:32.718566+0000","last_undegraded":"2023-11-24T06:12:32.718566+0000","last_fullsized":"2023-11-24T06:12:32.718566+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:29.113984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:29.113984+0000","last_clean_scrub_stamp":"2023-11-24T06:12:29.113984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:32:38.868244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-24T06:12:32.714757+0000","last_change":"2023-11-24T06:12:31.175110+0000","last_active":"2023-11-24T06:12:32.714757+0000","last_peered":"2023-11-24T06:12:32.714757+0000","last_clean":"2023-11-24T06:12:32.714757+0000","last_became_active":"2023-11-24T06:12:31.174422+0000","last_became_peered":"2023-11-24T06:12:31.174422+0000","last_unstale":"2023-11-24T06:12:32.714757+0000","last_undegraded":"2023-11-24T06:12:32.714757+0000","last_fullsized":"2023-11-24T06:12:32.714757+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_clean_scrub_stamp":"2023-11-24T06:12:30.127407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:04:19.413427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-24T06:12:32.715573+0000","last_change":"2023-11-24T06:12:30.159143+0000","last_active":"2023-11-24T06:12:32.715573+0000","last_peered":"2023-11-24T06:12:32.715573+0000","last_clean":"2023-11-24T06:12:32.715573+0000","last_became_active":"2023-11-24T06:12:30.158841+0000","last_became_peered":"2023-11-24T06:12:30.158841+0000","last_unstale":"2023-11-24T06:12:32.715573+0000","last_undegraded":"2023-11-24T06:12:32.715573+0000","last_fullsized":"2023-11-24T06:12:32.715573+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:29.113984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:29.113984+0000","last_clean_scrub_stamp":"2023-11-24T06:12:29.113984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:20:42.313069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-24T06:12:32.160910+0000","last_change":"2023-11-24T06:12:31.195508+0000","last_active":"2023-11-24T06:12:32.160910+0000","last_peered":"2023-11-24T06:12:32.160910+0000","last_clean":"2023-11-24T06:12:32.160910+0000","last_became_active":"2023-11-24T06:12:31.194717+0000","last_became_peered":"2023-11-24T06:12:31.194717+0000","last_unstale":"2023-11-24T06:12:32.160910+0000","last_undegraded":"2023-11-24T06:12:32.160910+0000","last_fullsized":"2023-11-24T06:12:32.160910+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_clean_scrub_stamp":"2023-11-24T06:12:30.127407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:12:48.584142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-24T06:12:32.160811+0000","last_change":"2023-11-24T06:12:30.170567+0000","last_active":"2023-11-24T06:12:32.160811+0000","last_peered":"2023-11-24T06:12:32.160811+0000","last_clean":"2023-11-24T06:12:32.160811+0000","last_became_active":"2023-11-24T06:12:30.169458+0000","last_became_peered":"2023-11-24T06:12:30.169458+0000","last_unstale":"2023-11-24T06:12:32.160811+0000","last_undegraded":"2023-11-24T06:12:32.160811+0000","last_fullsized":"2023-11-24T06:12:32.160811+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:29.113984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:29.113984+0000","last_clean_scrub_stamp":"2023-11-24T06:12:29.113984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:24:34.449167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-24T06:12:32.160698+0000","last_change":"2023-11-24T06:12:31.199006+0000","last_active":"2023-11-24T06:12:32.160698+0000","last_peered":"2023-11-24T06:12:32.160698+0000","last_clean":"2023-11-24T06:12:32.160698+0000","last_became_active":"2023-11-24T06:12:31.198718+0000","last_became_peered":"2023-11-24T06:12:31.198718+0000","last_unstale":"2023-11-24T06:12:32.160698+0000","last_undegraded":"2023-11-24T06:12:32.160698+0000","last_fullsized":"2023-11-24T06:12:32.160698+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_clean_scrub_stamp":"2023-11-24T06:12:30.127407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:19:49.094045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-24T06:12:32.167166+0000","last_change":"2023-11-24T06:12:30.167396+0000","last_active":"2023-11-24T06:12:32.167166+0000","last_peered":"2023-11-24T06:12:32.167166+0000","last_clean":"2023-11-24T06:12:32.167166+0000","last_became_active":"2023-11-24T06:12:30.167076+0000","last_became_peered":"2023-11-24T06:12:30.167076+0000","last_unstale":"2023-11-24T06:12:32.167166+0000","last_undegraded":"2023-11-24T06:12:32.167166+0000","last_fullsized":"2023-11-24T06:12:32.167166+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:29.113984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:29.113984+0000","last_clean_scrub_stamp":"2023-11-24T06:12:29.113984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:34:57.044397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":193,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-24T06:12:31.179110+0000","last_change":"2023-11-24T06:10:20.082677+0000","last_active":"2023-11-24T06:12:31.179110+0000","last_peered":"2023-11-24T06:12:31.179110+0000","last_clean":"2023-11-24T06:12:31.179110+0000","last_became_active":"2023-11-24T06:10:20.080551+0000","last_became_peered":"2023-11-24T06:10:20.080551+0000","last_unstale":"2023-11-24T06:12:31.179110+0000","last_undegraded":"2023-11-24T06:12:31.179110+0000","last_fullsized":"2023-11-24T06:12:31.179110+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:18.602209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:18.602209+0000","last_clean_scrub_stamp":"2023-11-24T06:10:18.602209+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:47:24.216235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574877,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32384,"kb_used_data":1196,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339456,"statfs":{"total":96636764160,"available":96603602944,"internally_reserved":0,"allocated":1224704,"data_stored":855575,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:12:18 2023","interfaces":[{"interface":"back","average":{"1min":1.016,"5min":1.091,"15min":1.103},"min":{"1min":0.75600000000000001,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":1.5580000000000001,"5min":1.9630000000000001,"15min":1.9630000000000001},"last":1.262},{"interface":"front","average":{"1min":1.0529999999999999,"5min":1.056,"15min":1.056},"min":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":1.605,"5min":1.621,"15min":1.621},"last":1.204}]},{"osd":1,"last update":"Fri Nov 24 06:12:18 2023","interfaces":[{"interface":"back","average":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"min":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":1.657,"5min":2.0259999999999998,"15min":2.0259999999999998},"last":0.93400000000000005},{"interface":"front","average":{"1min":1.226,"5min":1.149,"15min":1.1359999999999999},"min":{"1min":0.70699999999999996,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.786,"5min":2.1480000000000001,"15min":2.1480000000000001},"last":1.601}]},{"osd":2,"last update":"Fri Nov 24 06:12:18 2023","interfaces":[{"interface":"back","average":{"1min":1.1000000000000001,"5min":1.1339999999999999,"15min":1.1399999999999999},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.488,"5min":1.7809999999999999,"15min":1.7809999999999999},"last":1.1659999999999999},{"interface":"front","average":{"1min":1.2370000000000001,"5min":1.1850000000000001,"15min":1.1759999999999999},"min":{"1min":0.90900000000000003,"5min":0.86599999999999999,"15min":0.86599999999999999},"max":{"1min":2.0619999999999998,"5min":2.0619999999999998,"15min":2.0619999999999998},"last":1.3500000000000001}]},{"osd":3,"last update":"Fri Nov 24 06:12:18 2023","interfaces":[{"interface":"back","average":{"1min":1.1970000000000001,"5min":1.204,"15min":1.2050000000000001},"min":{"1min":0.89600000000000002,"5min":0.73999999999999999,"15min":0.73999999999999999},"max":{"1min":1.615,"5min":1.615,"15min":1.615},"last":1.002},{"interface":"front","average":{"1min":1.2909999999999999,"5min":1.3069999999999999,"15min":1.3089999999999999},"min":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"max":{"1min":1.9570000000000001,"5min":1.9570000000000001,"15min":1.9570000000000001},"last":1.4530000000000001}]},{"osd":4,"last update":"Fri Nov 24 06:12:18 2023","interfaces":[{"interface":"back","average":{"1min":1.3660000000000001,"5min":1.304,"15min":1.294},"min":{"1min":0.748,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.9570000000000001,"5min":1.9570000000000001,"15min":1.9570000000000001},"last":1.371},{"interface":"front","average":{"1min":1.3320000000000001,"5min":1.329,"15min":1.329},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.8340000000000001,"5min":2.1680000000000001,"15min":2.1680000000000001},"last":1.3200000000000001}]},{"osd":5,"last update":"Fri Nov 24 06:12:18 2023","interfaces":[{"interface":"back","average":{"1min":1.546,"5min":1.466,"15min":1.452},"min":{"1min":0.75900000000000001,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":2.0379999999999998,"5min":2.0379999999999998,"15min":2.0379999999999998},"last":1.4910000000000001},{"interface":"front","average":{"1min":1.5720000000000001,"5min":1.4770000000000001,"15min":1.4610000000000001},"min":{"1min":1.077,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.923,"5min":2.0539999999999998,"15min":2.0539999999999998},"last":1.413}]},{"osd":6,"last update":"Fri Nov 24 06:12:18 2023","interfaces":[{"interface":"back","average":{"1min":1.6559999999999999,"5min":1.548,"15min":1.53},"min":{"1min":0.94499999999999995,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":2.0899999999999999,"5min":2.0899999999999999,"15min":2.0899999999999999},"last":1.675},{"interface":"front","average":{"1min":1.6719999999999999,"5min":1.5580000000000001,"15min":1.5389999999999999},"min":{"1min":1.127,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":2.0070000000000001,"5min":2.0680000000000001,"15min":2.0680000000000001},"last":1.548}]}]},{"osd":6,"up_from":13,"seq":55834574878,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31792,"kb_used_data":608,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340048,"statfs":{"total":96636764160,"available":96604209152,"internally_reserved":0,"allocated":622592,"data_stored":265047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:12:21 2023","interfaces":[{"interface":"back","average":{"1min":1.097,"5min":1.194,"15min":1.21},"min":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":2.032,"5min":2.032,"15min":2.032},"last":0.97499999999999998},{"interface":"front","average":{"1min":1.1639999999999999,"5min":1.212,"15min":1.22},"min":{"1min":0.81599999999999995,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.9199999999999999,"5min":2.0230000000000001,"15min":2.0230000000000001},"last":0.77800000000000002}]},{"osd":1,"last update":"Fri Nov 24 06:12:21 2023","interfaces":[{"interface":"back","average":{"1min":1.149,"5min":1.163,"15min":1.165},"min":{"1min":0.72199999999999998,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":2.1499999999999999,"5min":2.1499999999999999,"15min":2.1499999999999999},"last":0.84699999999999998},{"interface":"front","average":{"1min":1.1970000000000001,"5min":1.2270000000000001,"15min":1.232},"min":{"1min":0.78700000000000003,"5min":0.77500000000000002,"15min":0.77500000000000002},"max":{"1min":1.645,"5min":1.8899999999999999,"15min":1.8899999999999999},"last":0.92300000000000004}]},{"osd":2,"last update":"Fri Nov 24 06:12:21 2023","interfaces":[{"interface":"back","average":{"1min":1.4179999999999999,"5min":1.2050000000000001,"15min":1.169},"min":{"1min":0.88,"5min":0.82299999999999995,"15min":0.82299999999999995},"max":{"1min":4.7359999999999998,"5min":4.7359999999999998,"15min":4.7359999999999998},"last":1.224},{"interface":"front","average":{"1min":1.4970000000000001,"5min":1.45,"15min":1.4419999999999999},"min":{"1min":0.93799999999999994,"5min":0.85899999999999999,"15min":0.85899999999999999},"max":{"1min":3.9700000000000002,"5min":3.9700000000000002,"15min":3.9700000000000002},"last":1.087}]},{"osd":3,"last update":"Fri Nov 24 06:12:21 2023","interfaces":[{"interface":"back","average":{"1min":1.2809999999999999,"5min":1.3069999999999999,"15min":1.3109999999999999},"min":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"max":{"1min":1.798,"5min":1.919,"15min":1.919},"last":1.0569999999999999},{"interface":"front","average":{"1min":1.28,"5min":1.292,"15min":1.294},"min":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":2.3929999999999998,"5min":2.3929999999999998,"15min":2.3929999999999998},"last":1.379}]},{"osd":4,"last update":"Fri Nov 24 06:12:21 2023","interfaces":[{"interface":"back","average":{"1min":1.4470000000000001,"5min":1.4550000000000001,"15min":1.4570000000000001},"min":{"1min":0.873,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":2.4039999999999999,"5min":2.4039999999999999,"15min":2.4039999999999999},"last":1.175},{"interface":"front","average":{"1min":1.5620000000000001,"5min":1.496,"15min":1.4850000000000001},"min":{"1min":0.88400000000000001,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":2.5609999999999999,"5min":2.5609999999999999,"15min":2.5609999999999999},"last":1.1499999999999999}]},{"osd":5,"last update":"Fri Nov 24 06:12:21 2023","interfaces":[{"interface":"back","average":{"1min":1.4610000000000001,"5min":1.4730000000000001,"15min":1.4750000000000001},"min":{"1min":0.90000000000000002,"5min":0.79100000000000004,"15min":0.79100000000000004},"max":{"1min":2.8290000000000002,"5min":2.8290000000000002,"15min":2.8290000000000002},"last":1.4490000000000001},{"interface":"front","average":{"1min":1.639,"5min":1.653,"15min":1.655},"min":{"1min":1.2250000000000001,"5min":0.93400000000000005,"15min":0.93400000000000005},"max":{"1min":2.9420000000000002,"5min":2.9420000000000002,"15min":2.9420000000000002},"last":1.2709999999999999}]},{"osd":7,"last update":"Fri Nov 24 06:12:21 2023","interfaces":[{"interface":"back","average":{"1min":1.6160000000000001,"5min":1.5960000000000001,"15min":1.5920000000000001},"min":{"1min":1.1299999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":2.1859999999999999,"5min":2.5139999999999998,"15min":2.5139999999999998},"last":1.337},{"interface":"front","average":{"1min":1.7509999999999999,"5min":1.5880000000000001,"15min":1.5609999999999999},"min":{"1min":1.0249999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":4.7859999999999996,"5min":4.7859999999999996,"15min":4.7859999999999996},"last":1.504}]}]},{"osd":5,"up_from":13,"seq":55834574878,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31780,"kb_used_data":600,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340060,"statfs":{"total":96636764160,"available":96604221440,"internally_reserved":0,"allocated":614400,"data_stored":258785,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:12:21 2023","interfaces":[{"interface":"back","average":{"1min":0.94099999999999995,"5min":1.1890000000000001,"15min":1.23},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.4750000000000001,"5min":3.0030000000000001,"15min":3.0030000000000001},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.99199999999999999,"5min":1.1799999999999999,"15min":1.2110000000000001},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.5369999999999999,"5min":2.0979999999999999,"15min":2.0979999999999999},"last":1.3069999999999999}]},{"osd":1,"last update":"Fri Nov 24 06:12:21 2023","interfaces":[{"interface":"back","average":{"1min":1.1599999999999999,"5min":1.252,"15min":1.268},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.728,"5min":5.0499999999999998,"15min":5.0499999999999998},"last":0.90300000000000002},{"interface":"front","average":{"1min":1.0529999999999999,"5min":1.3939999999999999,"15min":1.4510000000000001},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.5509999999999999,"5min":3.6930000000000001,"15min":3.6930000000000001},"last":1.4910000000000001}]},{"osd":2,"last update":"Fri Nov 24 06:12:21 2023","interfaces":[{"interface":"back","average":{"1min":1.161,"5min":1.3080000000000001,"15min":1.3320000000000001},"min":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"max":{"1min":1.5389999999999999,"5min":3.085,"15min":3.085},"last":1.5700000000000001},{"interface":"front","average":{"1min":1.1879999999999999,"5min":1.53,"15min":1.587},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.8340000000000001,"5min":4.7359999999999998,"15min":4.7359999999999998},"last":1.5389999999999999}]},{"osd":3,"last update":"Fri Nov 24 06:12:21 2023","interfaces":[{"interface":"back","average":{"1min":1.2450000000000001,"5min":1.385,"15min":1.409},"min":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"max":{"1min":1.7629999999999999,"5min":3.2229999999999999,"15min":3.2229999999999999},"last":1.625},{"interface":"front","average":{"1min":1.2929999999999999,"5min":1.4139999999999999,"15min":1.4339999999999999},"min":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"max":{"1min":1.8069999999999999,"5min":2.3260000000000001,"15min":2.3260000000000001},"last":1.4350000000000001}]},{"osd":4,"last update":"Fri Nov 24 06:12:21 2023","interfaces":[{"interface":"back","average":{"1min":1.3759999999999999,"5min":1.528,"15min":1.554},"min":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"max":{"1min":1.756,"5min":3.1230000000000002,"15min":3.1230000000000002},"last":1.387},{"interface":"front","average":{"1min":1.375,"5min":1.522,"15min":1.546},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":2.044,"5min":2.4809999999999999,"15min":2.4809999999999999},"last":1.7509999999999999}]},{"osd":6,"last update":"Fri Nov 24 06:12:21 2023","interfaces":[{"interface":"back","average":{"1min":1.427,"5min":1.6910000000000001,"15min":1.7350000000000001},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.913,"5min":4.6529999999999996,"15min":4.6529999999999996},"last":1.663},{"interface":"front","average":{"1min":1.4890000000000001,"5min":1.6970000000000001,"15min":1.732},"min":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"max":{"1min":2.266,"5min":4.3529999999999998,"15min":4.3529999999999998},"last":1.6870000000000001}]},{"osd":7,"last update":"Fri Nov 24 06:12:21 2023","interfaces":[{"interface":"back","average":{"1min":1.496,"5min":1.6679999999999999,"15min":1.696},"min":{"1min":0.874,"5min":0.874,"15min":0.874},"max":{"1min":1.9830000000000001,"5min":2.8839999999999999,"15min":2.8839999999999999},"last":1.847},{"interface":"front","average":{"1min":1.575,"5min":1.8080000000000001,"15min":1.847},"min":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":2.0699999999999998,"5min":4.8739999999999997,"15min":4.8739999999999997},"last":1.7190000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574878,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31788,"kb_used_data":608,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340052,"statfs":{"total":96636764160,"available":96604213248,"internally_reserved":0,"allocated":622592,"data_stored":260359,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:12:21 2023","interfaces":[{"interface":"back","average":{"1min":1.127,"5min":1.0589999999999999,"15min":1.0469999999999999},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.585,"5min":1.585,"15min":1.585},"last":0.94099999999999995},{"interface":"front","average":{"1min":1.0429999999999999,"5min":1.1819999999999999,"15min":1.2050000000000001},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.744,"5min":1.8220000000000001,"15min":1.8220000000000001},"last":0.77700000000000002}]},{"osd":1,"last update":"Fri Nov 24 06:12:21 2023","interfaces":[{"interface":"back","average":{"1min":1.1499999999999999,"5min":1.1259999999999999,"15min":1.1220000000000001},"min":{"1min":0.73499999999999999,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":1.97,"5min":1.97,"15min":1.97},"last":0.998},{"interface":"front","average":{"1min":1.1819999999999999,"5min":1.1779999999999999,"15min":1.1779999999999999},"min":{"1min":0.79800000000000004,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":1.917,"5min":1.917,"15min":1.917},"last":1.2470000000000001}]},{"osd":2,"last update":"Fri Nov 24 06:12:21 2023","interfaces":[{"interface":"back","average":{"1min":1.0980000000000001,"5min":1.22,"15min":1.24},"min":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":1.6970000000000001,"5min":2.319,"15min":2.319},"last":0.82399999999999995},{"interface":"front","average":{"1min":1.3320000000000001,"5min":1.3129999999999999,"15min":1.3100000000000001},"min":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"max":{"1min":2.077,"5min":2.0990000000000002,"15min":2.0990000000000002},"last":0.88900000000000001}]},{"osd":3,"last update":"Fri Nov 24 06:12:21 2023","interfaces":[{"interface":"back","average":{"1min":1.206,"5min":1.3700000000000001,"15min":1.3979999999999999},"min":{"1min":0.752,"5min":0.752,"15min":0.752},"max":{"1min":2.048,"5min":2.048,"15min":2.048},"last":1.3919999999999999},{"interface":"front","average":{"1min":1.3340000000000001,"5min":1.4199999999999999,"15min":1.4339999999999999},"min":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"max":{"1min":1.8149999999999999,"5min":2.1720000000000002,"15min":2.1720000000000002},"last":1.091}]},{"osd":5,"last update":"Fri Nov 24 06:12:21 2023","interfaces":[{"interface":"back","average":{"1min":1.2829999999999999,"5min":1.327,"15min":1.3340000000000001},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.835,"5min":1.895,"15min":1.895},"last":1.0349999999999999},{"interface":"front","average":{"1min":1.4359999999999999,"5min":1.528,"15min":1.544},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":2.044,"5min":2.5059999999999998,"15min":2.5059999999999998},"last":1.1919999999999999}]},{"osd":6,"last update":"Fri Nov 24 06:12:21 2023","interfaces":[{"interface":"back","average":{"1min":1.423,"5min":1.522,"15min":1.538},"min":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"max":{"1min":2.1509999999999998,"5min":2.2069999999999999,"15min":2.2069999999999999},"last":1.4690000000000001},{"interface":"front","average":{"1min":1.587,"5min":1.665,"15min":1.6779999999999999},"min":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":2.0139999999999998,"5min":2.4510000000000001,"15min":2.4510000000000001},"last":1.3180000000000001}]},{"osd":7,"last update":"Fri Nov 24 06:12:21 2023","interfaces":[{"interface":"back","average":{"1min":1.5640000000000001,"5min":1.6279999999999999,"15min":1.6379999999999999},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":2.214,"5min":2.4350000000000001,"15min":2.4350000000000001},"last":1.522},{"interface":"front","average":{"1min":1.554,"5min":1.617,"15min":1.627},"min":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":2.1499999999999999,"5min":3.1120000000000001,"15min":3.1120000000000001},"last":1.1200000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574878,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27624,"kb_used_data":604,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344216,"statfs":{"total":96636764160,"available":96608477184,"internally_reserved":0,"allocated":618496,"data_stored":258803,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:12:26 2023","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.98599999999999999,"15min":1},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":1.8360000000000001,"5min":2.766,"15min":2.766},"last":0.93500000000000005},{"interface":"front","average":{"1min":1.0800000000000001,"5min":0.94799999999999995,"15min":0.92600000000000005},"min":{"1min":0.44600000000000001,"5min":0.441,"15min":0.441},"max":{"1min":1.873,"5min":1.873,"15min":1.873},"last":0.76900000000000002}]},{"osd":1,"last update":"Fri Nov 24 06:12:26 2023","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.876,"15min":0.88200000000000001},"min":{"1min":0.46999999999999997,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.2569999999999999,"5min":2.5779999999999998,"15min":2.5779999999999998},"last":1.0209999999999999},{"interface":"front","average":{"1min":1.02,"5min":1.151,"15min":1.173},"min":{"1min":0.50700000000000001,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.7350000000000001,"5min":3.6899999999999999,"15min":3.6899999999999999},"last":1.3620000000000001}]},{"osd":2,"last update":"Fri Nov 24 06:12:26 2023","interfaces":[{"interface":"back","average":{"1min":0.996,"5min":1.1399999999999999,"15min":1.1639999999999999},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.637,"5min":4.0620000000000003,"15min":4.0620000000000003},"last":1.4159999999999999},{"interface":"front","average":{"1min":1.113,"5min":1.204,"15min":1.2190000000000001},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.5309999999999999,"5min":2.9529999999999998,"15min":2.9529999999999998},"last":1.659}]},{"osd":4,"last update":"Fri Nov 24 06:12:26 2023","interfaces":[{"interface":"back","average":{"1min":1.26,"5min":1.3939999999999999,"15min":1.4159999999999999},"min":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"max":{"1min":1.6819999999999999,"5min":4.125,"15min":4.125},"last":1.492},{"interface":"front","average":{"1min":1.351,"5min":1.427,"15min":1.4399999999999999},"min":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"max":{"1min":1.7689999999999999,"5min":3.9409999999999998,"15min":3.9409999999999998},"last":1.5600000000000001}]},{"osd":5,"last update":"Fri Nov 24 06:12:26 2023","interfaces":[{"interface":"back","average":{"1min":1.319,"5min":1.421,"15min":1.4379999999999999},"min":{"1min":0.749,"5min":0.749,"15min":0.749},"max":{"1min":1.6679999999999999,"5min":3.3359999999999999,"15min":3.3359999999999999},"last":1.7809999999999999},{"interface":"front","average":{"1min":1.214,"5min":1.4099999999999999,"15min":1.4430000000000001},"min":{"1min":0.86499999999999999,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.542,"5min":4.1669999999999998,"15min":4.1669999999999998},"last":1.2589999999999999}]},{"osd":6,"last update":"Fri Nov 24 06:12:26 2023","interfaces":[{"interface":"back","average":{"1min":1.2230000000000001,"5min":1.3939999999999999,"15min":1.4219999999999999},"min":{"1min":0.871,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":1.571,"5min":4.0190000000000001,"15min":4.0190000000000001},"last":2.169},{"interface":"front","average":{"1min":1.429,"5min":1.512,"15min":1.526},"min":{"1min":0.95999999999999996,"5min":0.92300000000000004,"15min":0.92300000000000004},"max":{"1min":1.966,"5min":3.2480000000000002,"15min":3.2480000000000002},"last":1.895}]},{"osd":7,"last update":"Fri Nov 24 06:12:26 2023","interfaces":[{"interface":"back","average":{"1min":1.3400000000000001,"5min":1.5580000000000001,"15min":1.5940000000000001},"min":{"1min":0.79500000000000004,"5min":0.76600000000000001,"15min":0.76600000000000001},"max":{"1min":1.8049999999999999,"5min":4.202,"15min":4.202},"last":1.831},{"interface":"front","average":{"1min":1.3480000000000001,"5min":1.5669999999999999,"15min":1.603},"min":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"max":{"1min":1.9199999999999999,"5min":3.9849999999999999,"15min":3.9849999999999999},"last":2.2989999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574878,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27620,"kb_used_data":600,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344220,"statfs":{"total":96636764160,"available":96608481280,"internally_reserved":0,"allocated":614400,"data_stored":258189,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:12:22 2023","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.88,"15min":0.88600000000000001},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":1.3149999999999999,"5min":1.6259999999999999,"15min":1.6259999999999999},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.96799999999999997,"5min":1.0740000000000001,"15min":1.0920000000000001},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.619,"5min":1.7669999999999999,"15min":1.7669999999999999},"last":0.51300000000000001}]},{"osd":1,"last update":"Fri Nov 24 06:12:22 2023","interfaces":[{"interface":"back","average":{"1min":0.99399999999999999,"5min":0.94999999999999996,"15min":0.94199999999999995},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.635,"5min":1.635,"15min":1.635},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.95499999999999996,"5min":0.97599999999999998,"15min":0.97999999999999998},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.498,"5min":1.8140000000000001,"15min":1.8140000000000001},"last":0.56999999999999995}]},{"osd":3,"last update":"Fri Nov 24 06:12:22 2023","interfaces":[{"interface":"back","average":{"1min":0.94499999999999995,"5min":1.0529999999999999,"15min":1.071},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.476,"5min":2.3559999999999999,"15min":2.3559999999999999},"last":0.49099999999999999},{"interface":"front","average":{"1min":1.1719999999999999,"5min":1.1319999999999999,"15min":1.1259999999999999},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.643,"5min":2.1840000000000002,"15min":2.1840000000000002},"last":0.47899999999999998}]},{"osd":4,"last update":"Fri Nov 24 06:12:22 2023","interfaces":[{"interface":"back","average":{"1min":1.411,"5min":1.417,"15min":1.4179999999999999},"min":{"1min":0.72099999999999997,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.9350000000000001,"5min":3.411,"15min":3.411},"last":0.73999999999999999},{"interface":"front","average":{"1min":1.333,"5min":1.4710000000000001,"15min":1.494},"min":{"1min":0.88400000000000001,"5min":0.82699999999999996,"15min":0.82699999999999996},"max":{"1min":1.964,"5min":3.7109999999999999,"15min":3.7109999999999999},"last":1.02}]},{"osd":5,"last update":"Fri Nov 24 06:12:22 2023","interfaces":[{"interface":"back","average":{"1min":1.246,"5min":1.3089999999999999,"15min":1.319},"min":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"max":{"1min":1.5620000000000001,"5min":2.5259999999999998,"15min":2.5259999999999998},"last":1.097},{"interface":"front","average":{"1min":1.444,"5min":1.3300000000000001,"15min":1.3109999999999999},"min":{"1min":0.96799999999999997,"5min":0.90400000000000003,"15min":0.90400000000000003},"max":{"1min":1.8839999999999999,"5min":2.1640000000000001,"15min":2.1640000000000001},"last":0.999}]},{"osd":6,"last update":"Fri Nov 24 06:12:22 2023","interfaces":[{"interface":"back","average":{"1min":1.244,"5min":1.4139999999999999,"15min":1.4419999999999999},"min":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"max":{"1min":1.8400000000000001,"5min":2.3039999999999998,"15min":2.3039999999999998},"last":1.155},{"interface":"front","average":{"1min":1.347,"5min":1.444,"15min":1.46},"min":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"max":{"1min":1.6850000000000001,"5min":2.605,"15min":2.605},"last":0.77100000000000002}]},{"osd":7,"last update":"Fri Nov 24 06:12:22 2023","interfaces":[{"interface":"back","average":{"1min":1.365,"5min":1.5489999999999999,"15min":1.579},"min":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"max":{"1min":1.9079999999999999,"5min":2.3740000000000001,"15min":2.3740000000000001},"last":0.97299999999999998},{"interface":"front","average":{"1min":1.3009999999999999,"5min":1.4330000000000001,"15min":1.4550000000000001},"min":{"1min":0.877,"5min":0.877,"15min":0.877},"max":{"1min":1.575,"5min":2.5699999999999998,"15min":2.5699999999999998},"last":0.89200000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574877,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27628,"kb_used_data":600,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344212,"statfs":{"total":96636764160,"available":96608473088,"internally_reserved":0,"allocated":614400,"data_stored":258837,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:12:21 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.878,"15min":0.90800000000000003},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":1.3740000000000001,"5min":1.9299999999999999,"15min":1.9299999999999999},"last":0.93200000000000005},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.90700000000000003,"15min":0.92500000000000004},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":1.258,"5min":1.7669999999999999,"15min":1.7669999999999999},"last":0.79600000000000004}]},{"osd":2,"last update":"Fri Nov 24 06:12:21 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.96499999999999997,"15min":1.0009999999999999},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.0840000000000001,"5min":2.5329999999999999,"15min":2.5329999999999999},"last":1.2889999999999999},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.99099999999999999,"15min":1.0169999999999999},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.159,"5min":1.657,"15min":1.657},"last":1.0549999999999999}]},{"osd":3,"last update":"Fri Nov 24 06:12:21 2023","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.98899999999999999,"15min":1.0209999999999999},"min":{"1min":0.49399999999999999,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.2729999999999999,"5min":1.841,"15min":1.841},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.98399999999999999,"5min":1.121,"15min":1.1439999999999999},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.5249999999999999,"5min":2.8940000000000001,"15min":2.8940000000000001},"last":1.1850000000000001}]},{"osd":4,"last update":"Fri Nov 24 06:12:21 2023","interfaces":[{"interface":"back","average":{"1min":1.145,"5min":1.254,"15min":1.272},"min":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":1.454,"5min":2.1930000000000001,"15min":2.1930000000000001},"last":1.25},{"interface":"front","average":{"1min":1.1200000000000001,"5min":1.1839999999999999,"15min":1.194},"min":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":1.569,"5min":1.784,"15min":1.784},"last":1.105}]},{"osd":5,"last update":"Fri Nov 24 06:12:21 2023","interfaces":[{"interface":"back","average":{"1min":1.1499999999999999,"5min":1.454,"15min":1.504},"min":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":1.5820000000000001,"5min":5.5449999999999999,"15min":5.5449999999999999},"last":1.4550000000000001},{"interface":"front","average":{"1min":1.167,"5min":1.484,"15min":1.5369999999999999},"min":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":1.538,"5min":6.0030000000000001,"15min":6.0030000000000001},"last":1.153}]},{"osd":6,"last update":"Fri Nov 24 06:12:21 2023","interfaces":[{"interface":"back","average":{"1min":1.2030000000000001,"5min":1.4950000000000001,"15min":1.544},"min":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":1.5189999999999999,"5min":5.2809999999999997,"15min":5.2809999999999997},"last":1.405},{"interface":"front","average":{"1min":1.2849999999999999,"5min":1.4370000000000001,"15min":1.462},"min":{"1min":0.79300000000000004,"5min":0.749,"15min":0.749},"max":{"1min":1.7969999999999999,"5min":4.327,"15min":4.327},"last":1.379}]},{"osd":7,"last update":"Fri Nov 24 06:12:21 2023","interfaces":[{"interface":"back","average":{"1min":1.173,"5min":1.321,"15min":1.3460000000000001},"min":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":1.8540000000000001,"5min":2.3069999999999999,"15min":2.3069999999999999},"last":1.026},{"interface":"front","average":{"1min":1.252,"5min":1.528,"15min":1.5740000000000001},"min":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":1.6819999999999999,"5min":5.4500000000000002,"15min":5.4500000000000002},"last":1.341}]}]},{"osd":0,"up_from":13,"seq":55834574877,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28216,"kb_used_data":1192,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343624,"statfs":{"total":96636764160,"available":96607870976,"internally_reserved":0,"allocated":1220608,"data_stored":851309,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Nov 24 06:12:20 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.90500000000000003,"15min":0.92700000000000005},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":1.675,"5min":2.3999999999999999,"15min":2.3999999999999999},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.79900000000000004,"5min":1.1100000000000001,"15min":1.1619999999999999},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.403,"5min":4.8289999999999997,"15min":4.8289999999999997},"last":0.85899999999999999}]},{"osd":2,"last update":"Fri Nov 24 06:12:20 2023","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":1.0249999999999999,"15min":1.05},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.5960000000000001,"5min":2.5139999999999998,"15min":2.5139999999999998},"last":0.96499999999999997},{"interface":"front","average":{"1min":0.98199999999999998,"5min":1.0860000000000001,"15min":1.103},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.4390000000000001,"5min":2.3599999999999999,"15min":2.3599999999999999},"last":0.58199999999999996}]},{"osd":3,"last update":"Fri Nov 24 06:12:20 2023","interfaces":[{"interface":"back","average":{"1min":1.018,"5min":1.1040000000000001,"15min":1.1180000000000001},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.514,"5min":2.5840000000000001,"15min":2.5840000000000001},"last":0.66000000000000003},{"interface":"front","average":{"1min":1.071,"5min":1.3460000000000001,"15min":1.3919999999999999},"min":{"1min":0.503,"5min":0.495,"15min":0.495},"max":{"1min":1.526,"5min":5.1230000000000002,"15min":5.1230000000000002},"last":0.63700000000000001}]},{"osd":4,"last update":"Fri Nov 24 06:12:20 2023","interfaces":[{"interface":"back","average":{"1min":1.0209999999999999,"5min":1.23,"15min":1.2649999999999999},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.3919999999999999,"5min":2.887,"15min":2.887},"last":0.90100000000000002},{"interface":"front","average":{"1min":1.0760000000000001,"5min":1.284,"15min":1.3180000000000001},"min":{"1min":0.71599999999999997,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":1.6819999999999999,"5min":2.8530000000000002,"15min":2.8530000000000002},"last":0.79600000000000004}]},{"osd":5,"last update":"Fri Nov 24 06:12:20 2023","interfaces":[{"interface":"back","average":{"1min":1.2370000000000001,"5min":1.5069999999999999,"15min":1.552},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.804,"5min":4.6299999999999999,"15min":4.6299999999999999},"last":0.88200000000000001},{"interface":"front","average":{"1min":1.056,"5min":1.2769999999999999,"15min":1.3140000000000001},"min":{"1min":0.72699999999999998,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.3560000000000001,"5min":2.794,"15min":2.794},"last":0.995}]},{"osd":6,"last update":"Fri Nov 24 06:12:20 2023","interfaces":[{"interface":"back","average":{"1min":1.2829999999999999,"5min":1.5940000000000001,"15min":1.6459999999999999},"min":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.6779999999999999,"5min":4.798,"15min":4.798},"last":1.1060000000000001},{"interface":"front","average":{"1min":1.375,"5min":1.5840000000000001,"15min":1.619},"min":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"max":{"1min":1.9350000000000001,"5min":4.7149999999999999,"15min":4.7149999999999999},"last":1.087}]},{"osd":7,"last update":"Fri Nov 24 06:12:20 2023","interfaces":[{"interface":"back","average":{"1min":1.3109999999999999,"5min":1.635,"15min":1.6890000000000001},"min":{"1min":0.73799999999999999,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.9750000000000001,"5min":4.7649999999999997,"15min":4.7649999999999997},"last":1.0680000000000001},{"interface":"front","average":{"1min":1.421,"5min":1.6399999999999999,"15min":1.6759999999999999},"min":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"max":{"1min":2.0430000000000001,"5min":4.6820000000000004,"15min":4.6820000000000004},"last":1.024}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-24T06:12:35.949 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-24T06:12:36.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:36.198+0000 7fc553f97640 1 Processor -- start 2023-11-24T06:12:36.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:36.198+0000 7fc553f97640 1 -- start start 2023-11-24T06:12:36.202 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:36.200+0000 7fc553f97640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fc54c143ae0 0x7fc54c0cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:36.202 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:36.200+0000 7fc553f97640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc54c0cfc30 con 0x7fc54c15e3b0 2023-11-24T06:12:36.202 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:36.200+0000 7fc553f97640 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc54c0b1ee0 con 0x7fc54c0cf7d0 2023-11-24T06:12:36.202 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:36.200+0000 7fc553f97640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc54c0b2780 con 0x7fc54c143ae0 2023-11-24T06:12:36.203 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:36.200+0000 7fc551d33640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fc54c143ae0 0x7fc54c0cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:36.203 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:36.200+0000 7fc551d33640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fc54c143ae0 0x7fc54c0cf290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:49958/0 (socket says 172.21.15.92:49958) 2023-11-24T06:12:36.203 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:36.200+0000 7fc551d33640 1 -- 172.21.15.92:0/1395401725 learned_addr learned my addr 172.21.15.92:0/1395401725 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:12:36.203 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:36.201+0000 7fc551d33640 1 -- 172.21.15.92:0/1395401725 >> v1:172.21.15.118:6789/0 conn(0x7fc54c0cf7d0 legacy=0x7fc54c15dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:36.203 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:36.201+0000 7fc551d33640 1 -- 172.21.15.92:0/1395401725 >> v1:172.21.15.92:6789/0 conn(0x7fc54c15e3b0 legacy=0x7fc54c0702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:36.203 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:36.201+0000 7fc551d33640 1 -- 172.21.15.92:0/1395401725 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc54c15e990 con 0x7fc54c143ae0 2023-11-24T06:12:36.204 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:36.202+0000 7fc551d33640 1 --2- 172.21.15.92:0/1395401725 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc54c143ae0 0x7fc54c0cf290 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fc548009870 tx=0x7fc54802dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bea9658ecbf25919 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:36.204 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:36.202+0000 7fc550d31640 1 -- 172.21.15.92:0/1395401725 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc54802e960 con 0x7fc54c143ae0 2023-11-24T06:12:36.204 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:36.202+0000 7fc550d31640 1 -- 172.21.15.92:0/1395401725 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc54802eac0 con 0x7fc54c143ae0 2023-11-24T06:12:36.204 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:36.202+0000 7fc550d31640 1 -- 172.21.15.92:0/1395401725 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc54802ee50 con 0x7fc54c143ae0 2023-11-24T06:12:36.205 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:36.203+0000 7fc553f97640 1 -- 172.21.15.92:0/1395401725 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc54c143ae0 msgr2=0x7fc54c0cf290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:36.205 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:36.203+0000 7fc553f97640 1 --2- 172.21.15.92:0/1395401725 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc54c143ae0 0x7fc54c0cf290 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fc548009870 tx=0x7fc54802dbb0 comp rx=0 tx=0).stop 2023-11-24T06:12:36.205 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:36.203+0000 7fc553f97640 1 -- 172.21.15.92:0/1395401725 shutdown_connections 2023-11-24T06:12:36.205 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:36.203+0000 7fc553f97640 1 --2- 172.21.15.92:0/1395401725 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc54c143ae0 0x7fc54c0cf290 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:36.205 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:36.203+0000 7fc553f97640 1 -- 172.21.15.92:0/1395401725 >> 172.21.15.92:0/1395401725 conn(0x7fc54c06a800 msgr2=0x7fc54c06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:36.206 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:36.203+0000 7fc553f97640 1 -- 172.21.15.92:0/1395401725 shutdown_connections 2023-11-24T06:12:36.206 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:36.203+0000 7fc553f97640 1 -- 172.21.15.92:0/1395401725 wait complete. 2023-11-24T06:12:36.206 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:36.205+0000 7fc553f97640 1 Processor -- start 2023-11-24T06:12:36.207 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:36.205+0000 7fc553f97640 1 -- start start 2023-11-24T06:12:36.208 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:36.206+0000 7fc553f97640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc54c0cf7d0 0x7fc54c0af640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:36.208 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:36.206+0000 7fc553f97640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc54c143ae0 0x7fc54c0cd040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:36.208 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:36.206+0000 7fc553f97640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc54c15e3b0 0x7fc54c0cd580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:36.208 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:36.206+0000 7fc553f97640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc54c0c3f60 con 0x7fc54c0cf7d0 2023-11-24T06:12:36.209 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:36.206+0000 7fc553f97640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc54c0cdbb0 con 0x7fc54c143ae0 2023-11-24T06:12:36.209 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:36.206+0000 7fc553f97640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc54c0acb20 con 0x7fc54c15e3b0 2023-11-24T06:12:36.209 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:36.206+0000 7fc551d33640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc54c0cf7d0 0x7fc54c0af640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:36.209 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:36.206+0000 7fc551d33640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc54c0cf7d0 0x7fc54c0af640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:49974/0 (socket says 172.21.15.92:49974) 2023-11-24T06:12:36.209 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:36.206+0000 7fc551d33640 1 -- 172.21.15.92:0/3218905133 learned_addr learned my addr 172.21.15.92:0/3218905133 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:12:36.209 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:36.206+0000 7fc551d33640 1 -- 172.21.15.92:0/3218905133 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc54c15e3b0 msgr2=0x7fc54c0cd580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:36.209 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:36.206+0000 7fc551d33640 1 --2- 172.21.15.92:0/3218905133 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc54c15e3b0 0x7fc54c0cd580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:36.209 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:36.206+0000 7fc551d33640 1 -- 172.21.15.92:0/3218905133 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc54c143ae0 msgr2=0x7fc54c0cd040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:36.209 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:36.206+0000 7fc551d33640 1 --2- 172.21.15.92:0/3218905133 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc54c143ae0 0x7fc54c0cd040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:36.209 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:36.206+0000 7fc551d33640 1 -- 172.21.15.92:0/3218905133 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5480094d0 con 0x7fc54c0cf7d0 2023-11-24T06:12:36.210 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:36.207+0000 7fc551d33640 1 --2- 172.21.15.92:0/3218905133 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc54c0cf7d0 0x7fc54c0af640 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fc548009870 tx=0x7fc5480046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:36.210 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:36.207+0000 7fc5427fc640 1 -- 172.21.15.92:0/3218905133 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5480405f0 con 0x7fc54c0cf7d0 2023-11-24T06:12:36.211 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:36.207+0000 7fc5427fc640 1 -- 172.21.15.92:0/3218905133 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc548040750 con 0x7fc54c0cf7d0 2023-11-24T06:12:36.211 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:36.207+0000 7fc5427fc640 1 -- 172.21.15.92:0/3218905133 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc548040ae0 con 0x7fc54c0cf7d0 2023-11-24T06:12:36.211 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:36.207+0000 7fc553f97640 1 -- 172.21.15.92:0/3218905133 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc54c0ace00 con 0x7fc54c0cf7d0 2023-11-24T06:12:36.211 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:36.207+0000 7fc553f97640 1 -- 172.21.15.92:0/3218905133 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc54c0ad3c0 con 0x7fc54c0cf7d0 2023-11-24T06:12:36.211 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:36.208+0000 7fc553f97640 1 -- 172.21.15.92:0/3218905133 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc54c0ad5b0 con 0x7fc54c0cf7d0 2023-11-24T06:12:36.211 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:36.209+0000 7fc5427fc640 1 -- 172.21.15.92:0/3218905133 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc548040c40 con 0x7fc54c0cf7d0 2023-11-24T06:12:36.212 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:36.210+0000 7fc5427fc640 1 --2- 172.21.15.92:0/3218905133 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fc53006ae40 0x7fc53006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:36.212 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:36.210+0000 7fc5427fc640 1 -- 172.21.15.92:0/3218905133 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc5480ac510 con 0x7fc54c0cf7d0 2023-11-24T06:12:36.212 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:36.210+0000 7fc551532640 1 --2- 172.21.15.92:0/3218905133 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fc53006ae40 0x7fc53006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:36.213 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:36.211+0000 7fc551532640 1 --2- 172.21.15.92:0/3218905133 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fc53006ae40 0x7fc53006d300 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fc54c142430 tx=0x7fc53c007990 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:36.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:36.214+0000 7fc5427fc640 1 -- 172.21.15.92:0/3218905133 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc548077650 con 0x7fc54c0cf7d0 2023-11-24T06:12:36.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:36.400+0000 7fc553f97640 1 -- 172.21.15.92:0/3218905133 --> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc54c0b2410 con 0x7fc53006ae40 2023-11-24T06:12:36.424 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:36.422+0000 7fc5427fc640 1 -- 172.21.15.92:0/3218905133 <== mgr.4101 v2:172.21.15.92:6800/784027401 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+287215 (secure 0 0 0) 0x7fc53c0050d0 con 0x7fc53006ae40 2023-11-24T06:12:36.428 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-24T06:12:36.430 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2023-11-24T06:12:36.431 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:36.429+0000 7fc553f97640 1 -- 172.21.15.92:0/3218905133 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fc53006ae40 msgr2=0x7fc53006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:36.431 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:36.429+0000 7fc553f97640 1 --2- 172.21.15.92:0/3218905133 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fc53006ae40 0x7fc53006d300 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fc54c142430 tx=0x7fc53c007990 comp rx=0 tx=0).stop 2023-11-24T06:12:36.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:36.429+0000 7fc553f97640 1 -- 172.21.15.92:0/3218905133 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc54c0cf7d0 msgr2=0x7fc54c0af640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:36.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:36.429+0000 7fc553f97640 1 --2- 172.21.15.92:0/3218905133 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc54c0cf7d0 0x7fc54c0af640 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fc548009870 tx=0x7fc5480046c0 comp rx=0 tx=0).stop 2023-11-24T06:12:36.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:36.429+0000 7fc553f97640 1 -- 172.21.15.92:0/3218905133 shutdown_connections 2023-11-24T06:12:36.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:36.429+0000 7fc553f97640 1 --2- 172.21.15.92:0/3218905133 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fc53006ae40 0x7fc53006d300 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:36.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:36.429+0000 7fc553f97640 1 --2- 172.21.15.92:0/3218905133 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc54c15e3b0 0x7fc54c0cd580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:36.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:36.429+0000 7fc553f97640 1 --2- 172.21.15.92:0/3218905133 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc54c143ae0 0x7fc54c0cd040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:36.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:36.429+0000 7fc553f97640 1 --2- 172.21.15.92:0/3218905133 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc54c0cf7d0 0x7fc54c0af640 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:36.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:36.429+0000 7fc553f97640 1 -- 172.21.15.92:0/3218905133 >> 172.21.15.92:0/3218905133 conn(0x7fc54c06a800 msgr2=0x7fc54c142cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:36.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:36.429+0000 7fc553f97640 1 -- 172.21.15.92:0/3218905133 shutdown_connections 2023-11-24T06:12:36.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:36.429+0000 7fc553f97640 1 -- 172.21.15.92:0/3218905133 wait complete. 2023-11-24T06:12:36.449 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":114,"stamp":"2023-11-24T06:12:35.706037+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":54,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":238832,"kb_used_data":6008,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754735888,"statfs":{"total":773094113280,"available":772849549312,"internally_reserved":0,"allocated":6152192,"data_stored":3266904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001874"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-24T06:12:32.717116+0000","last_change":"2023-11-24T06:12:31.198964+0000","last_active":"2023-11-24T06:12:32.717116+0000","last_peered":"2023-11-24T06:12:32.717116+0000","last_clean":"2023-11-24T06:12:32.717116+0000","last_became_active":"2023-11-24T06:12:31.197652+0000","last_became_peered":"2023-11-24T06:12:31.197652+0000","last_unstale":"2023-11-24T06:12:32.717116+0000","last_undegraded":"2023-11-24T06:12:32.717116+0000","last_fullsized":"2023-11-24T06:12:32.717116+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_clean_scrub_stamp":"2023-11-24T06:12:30.127407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:17:30.690945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-24T06:12:32.160974+0000","last_change":"2023-11-24T06:12:31.194790+0000","last_active":"2023-11-24T06:12:32.160974+0000","last_peered":"2023-11-24T06:12:32.160974+0000","last_clean":"2023-11-24T06:12:32.160974+0000","last_became_active":"2023-11-24T06:12:31.194330+0000","last_became_peered":"2023-11-24T06:12:31.194330+0000","last_unstale":"2023-11-24T06:12:32.160974+0000","last_undegraded":"2023-11-24T06:12:32.160974+0000","last_fullsized":"2023-11-24T06:12:32.160974+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_clean_scrub_stamp":"2023-11-24T06:12:30.127407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:00:15.801304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-24T06:12:32.168126+0000","last_change":"2023-11-24T06:12:31.189554+0000","last_active":"2023-11-24T06:12:32.168126+0000","last_peered":"2023-11-24T06:12:32.168126+0000","last_clean":"2023-11-24T06:12:32.168126+0000","last_became_active":"2023-11-24T06:12:31.189217+0000","last_became_peered":"2023-11-24T06:12:31.189217+0000","last_unstale":"2023-11-24T06:12:32.168126+0000","last_undegraded":"2023-11-24T06:12:32.168126+0000","last_fullsized":"2023-11-24T06:12:32.168126+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_clean_scrub_stamp":"2023-11-24T06:12:30.127407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:27:34.505266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-24T06:12:32.716824+0000","last_change":"2023-11-24T06:12:31.188411+0000","last_active":"2023-11-24T06:12:32.716824+0000","last_peered":"2023-11-24T06:12:32.716824+0000","last_clean":"2023-11-24T06:12:32.716824+0000","last_became_active":"2023-11-24T06:12:31.188141+0000","last_became_peered":"2023-11-24T06:12:31.188141+0000","last_unstale":"2023-11-24T06:12:32.716824+0000","last_undegraded":"2023-11-24T06:12:32.716824+0000","last_fullsized":"2023-11-24T06:12:32.716824+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_clean_scrub_stamp":"2023-11-24T06:12:30.127407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:11:10.610649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-24T06:12:32.717176+0000","last_change":"2023-11-24T06:12:31.186880+0000","last_active":"2023-11-24T06:12:32.717176+0000","last_peered":"2023-11-24T06:12:32.717176+0000","last_clean":"2023-11-24T06:12:32.717176+0000","last_became_active":"2023-11-24T06:12:31.186500+0000","last_became_peered":"2023-11-24T06:12:31.186500+0000","last_unstale":"2023-11-24T06:12:32.717176+0000","last_undegraded":"2023-11-24T06:12:32.717176+0000","last_fullsized":"2023-11-24T06:12:32.717176+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_clean_scrub_stamp":"2023-11-24T06:12:30.127407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:27:08.379593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-24T06:12:32.718087+0000","last_change":"2023-11-24T06:12:31.185747+0000","last_active":"2023-11-24T06:12:32.718087+0000","last_peered":"2023-11-24T06:12:32.718087+0000","last_clean":"2023-11-24T06:12:32.718087+0000","last_became_active":"2023-11-24T06:12:31.185468+0000","last_became_peered":"2023-11-24T06:12:31.185468+0000","last_unstale":"2023-11-24T06:12:32.718087+0000","last_undegraded":"2023-11-24T06:12:32.718087+0000","last_fullsized":"2023-11-24T06:12:32.718087+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_clean_scrub_stamp":"2023-11-24T06:12:30.127407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:49:46.518805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-24T06:12:32.717060+0000","last_change":"2023-11-24T06:12:31.172625+0000","last_active":"2023-11-24T06:12:32.717060+0000","last_peered":"2023-11-24T06:12:32.717060+0000","last_clean":"2023-11-24T06:12:32.717060+0000","last_became_active":"2023-11-24T06:12:31.171819+0000","last_became_peered":"2023-11-24T06:12:31.171819+0000","last_unstale":"2023-11-24T06:12:32.717060+0000","last_undegraded":"2023-11-24T06:12:32.717060+0000","last_fullsized":"2023-11-24T06:12:32.717060+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_clean_scrub_stamp":"2023-11-24T06:12:30.127407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:04:42.070586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-24T06:12:32.160714+0000","last_change":"2023-11-24T06:12:31.190733+0000","last_active":"2023-11-24T06:12:32.160714+0000","last_peered":"2023-11-24T06:12:32.160714+0000","last_clean":"2023-11-24T06:12:32.160714+0000","last_became_active":"2023-11-24T06:12:31.187775+0000","last_became_peered":"2023-11-24T06:12:31.187775+0000","last_unstale":"2023-11-24T06:12:32.160714+0000","last_undegraded":"2023-11-24T06:12:32.160714+0000","last_fullsized":"2023-11-24T06:12:32.160714+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_clean_scrub_stamp":"2023-11-24T06:12:30.127407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:56:08.211961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-24T06:12:32.719316+0000","last_change":"2023-11-24T06:12:31.181645+0000","last_active":"2023-11-24T06:12:32.719316+0000","last_peered":"2023-11-24T06:12:32.719316+0000","last_clean":"2023-11-24T06:12:32.719316+0000","last_became_active":"2023-11-24T06:12:31.181322+0000","last_became_peered":"2023-11-24T06:12:31.181322+0000","last_unstale":"2023-11-24T06:12:32.719316+0000","last_undegraded":"2023-11-24T06:12:32.719316+0000","last_fullsized":"2023-11-24T06:12:32.719316+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_clean_scrub_stamp":"2023-11-24T06:12:30.127407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:49:50.435010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-24T06:12:32.715883+0000","last_change":"2023-11-24T06:12:31.180116+0000","last_active":"2023-11-24T06:12:32.715883+0000","last_peered":"2023-11-24T06:12:32.715883+0000","last_clean":"2023-11-24T06:12:32.715883+0000","last_became_active":"2023-11-24T06:12:31.179609+0000","last_became_peered":"2023-11-24T06:12:31.179609+0000","last_unstale":"2023-11-24T06:12:32.715883+0000","last_undegraded":"2023-11-24T06:12:32.715883+0000","last_fullsized":"2023-11-24T06:12:32.715883+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_clean_scrub_stamp":"2023-11-24T06:12:30.127407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:49:50.435010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-24T06:12:32.716508+0000","last_change":"2023-11-24T06:12:31.191319+0000","last_active":"2023-11-24T06:12:32.716508+0000","last_peered":"2023-11-24T06:12:32.716508+0000","last_clean":"2023-11-24T06:12:32.716508+0000","last_became_active":"2023-11-24T06:12:31.191016+0000","last_became_peered":"2023-11-24T06:12:31.191016+0000","last_unstale":"2023-11-24T06:12:32.716508+0000","last_undegraded":"2023-11-24T06:12:32.716508+0000","last_fullsized":"2023-11-24T06:12:32.716508+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_clean_scrub_stamp":"2023-11-24T06:12:30.127407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:38:48.289685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-24T06:12:32.716914+0000","last_change":"2023-11-24T06:12:31.179443+0000","last_active":"2023-11-24T06:12:32.716914+0000","last_peered":"2023-11-24T06:12:32.716914+0000","last_clean":"2023-11-24T06:12:32.716914+0000","last_became_active":"2023-11-24T06:12:31.179128+0000","last_became_peered":"2023-11-24T06:12:31.179128+0000","last_unstale":"2023-11-24T06:12:32.716914+0000","last_undegraded":"2023-11-24T06:12:32.716914+0000","last_fullsized":"2023-11-24T06:12:32.716914+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_clean_scrub_stamp":"2023-11-24T06:12:30.127407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:09:09.186156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-24T06:12:32.166693+0000","last_change":"2023-11-24T06:12:31.182856+0000","last_active":"2023-11-24T06:12:32.166693+0000","last_peered":"2023-11-24T06:12:32.166693+0000","last_clean":"2023-11-24T06:12:32.166693+0000","last_became_active":"2023-11-24T06:12:31.182485+0000","last_became_peered":"2023-11-24T06:12:31.182485+0000","last_unstale":"2023-11-24T06:12:32.166693+0000","last_undegraded":"2023-11-24T06:12:32.166693+0000","last_fullsized":"2023-11-24T06:12:32.166693+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_clean_scrub_stamp":"2023-11-24T06:12:30.127407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:45:48.394942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-24T06:12:32.716837+0000","last_change":"2023-11-24T06:12:31.184693+0000","last_active":"2023-11-24T06:12:32.716837+0000","last_peered":"2023-11-24T06:12:32.716837+0000","last_clean":"2023-11-24T06:12:32.716837+0000","last_became_active":"2023-11-24T06:12:31.184376+0000","last_became_peered":"2023-11-24T06:12:31.184376+0000","last_unstale":"2023-11-24T06:12:32.716837+0000","last_undegraded":"2023-11-24T06:12:32.716837+0000","last_fullsized":"2023-11-24T06:12:32.716837+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_clean_scrub_stamp":"2023-11-24T06:12:30.127407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:34:53.658692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-24T06:12:32.716056+0000","last_change":"2023-11-24T06:12:31.180593+0000","last_active":"2023-11-24T06:12:32.716056+0000","last_peered":"2023-11-24T06:12:32.716056+0000","last_clean":"2023-11-24T06:12:32.716056+0000","last_became_active":"2023-11-24T06:12:31.179053+0000","last_became_peered":"2023-11-24T06:12:31.179053+0000","last_unstale":"2023-11-24T06:12:32.716056+0000","last_undegraded":"2023-11-24T06:12:32.716056+0000","last_fullsized":"2023-11-24T06:12:32.716056+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_clean_scrub_stamp":"2023-11-24T06:12:30.127407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:04:19.413427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-24T06:12:32.716630+0000","last_change":"2023-11-24T06:12:31.188247+0000","last_active":"2023-11-24T06:12:32.716630+0000","last_peered":"2023-11-24T06:12:32.716630+0000","last_clean":"2023-11-24T06:12:32.716630+0000","last_became_active":"2023-11-24T06:12:31.186090+0000","last_became_peered":"2023-11-24T06:12:31.186090+0000","last_unstale":"2023-11-24T06:12:32.716630+0000","last_undegraded":"2023-11-24T06:12:32.716630+0000","last_fullsized":"2023-11-24T06:12:32.716630+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_clean_scrub_stamp":"2023-11-24T06:12:30.127407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:11:10.610649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-24T06:12:32.161136+0000","last_change":"2023-11-24T06:12:31.194663+0000","last_active":"2023-11-24T06:12:32.161136+0000","last_peered":"2023-11-24T06:12:32.161136+0000","last_clean":"2023-11-24T06:12:32.161136+0000","last_became_active":"2023-11-24T06:12:31.194395+0000","last_became_peered":"2023-11-24T06:12:31.194395+0000","last_unstale":"2023-11-24T06:12:32.161136+0000","last_undegraded":"2023-11-24T06:12:32.161136+0000","last_fullsized":"2023-11-24T06:12:32.161136+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_clean_scrub_stamp":"2023-11-24T06:12:30.127407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:32:16.673191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-24T06:12:32.168708+0000","last_change":"2023-11-24T06:12:31.196775+0000","last_active":"2023-11-24T06:12:32.168708+0000","last_peered":"2023-11-24T06:12:32.168708+0000","last_clean":"2023-11-24T06:12:32.168708+0000","last_became_active":"2023-11-24T06:12:31.196208+0000","last_became_peered":"2023-11-24T06:12:31.196208+0000","last_unstale":"2023-11-24T06:12:32.168708+0000","last_undegraded":"2023-11-24T06:12:32.168708+0000","last_fullsized":"2023-11-24T06:12:32.168708+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_clean_scrub_stamp":"2023-11-24T06:12:30.127407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:58:17.965765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-24T06:12:32.718242+0000","last_change":"2023-11-24T06:12:31.198155+0000","last_active":"2023-11-24T06:12:32.718242+0000","last_peered":"2023-11-24T06:12:32.718242+0000","last_clean":"2023-11-24T06:12:32.718242+0000","last_became_active":"2023-11-24T06:12:31.197563+0000","last_became_peered":"2023-11-24T06:12:31.197563+0000","last_unstale":"2023-11-24T06:12:32.718242+0000","last_undegraded":"2023-11-24T06:12:32.718242+0000","last_fullsized":"2023-11-24T06:12:32.718242+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_clean_scrub_stamp":"2023-11-24T06:12:30.127407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:34:02.104830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-24T06:12:32.160965+0000","last_change":"2023-11-24T06:12:31.196369+0000","last_active":"2023-11-24T06:12:32.160965+0000","last_peered":"2023-11-24T06:12:32.160965+0000","last_clean":"2023-11-24T06:12:32.160965+0000","last_became_active":"2023-11-24T06:12:31.195118+0000","last_became_peered":"2023-11-24T06:12:31.195118+0000","last_unstale":"2023-11-24T06:12:32.160965+0000","last_undegraded":"2023-11-24T06:12:32.160965+0000","last_fullsized":"2023-11-24T06:12:32.160965+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_clean_scrub_stamp":"2023-11-24T06:12:30.127407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:49:00.756482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-24T06:12:32.167019+0000","last_change":"2023-11-24T06:12:31.190534+0000","last_active":"2023-11-24T06:12:32.167019+0000","last_peered":"2023-11-24T06:12:32.167019+0000","last_clean":"2023-11-24T06:12:32.167019+0000","last_became_active":"2023-11-24T06:12:31.190219+0000","last_became_peered":"2023-11-24T06:12:31.190219+0000","last_unstale":"2023-11-24T06:12:32.167019+0000","last_undegraded":"2023-11-24T06:12:32.167019+0000","last_fullsized":"2023-11-24T06:12:32.167019+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_clean_scrub_stamp":"2023-11-24T06:12:30.127407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:13:44.410307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-24T06:12:32.160719+0000","last_change":"2023-11-24T06:12:31.196925+0000","last_active":"2023-11-24T06:12:32.160719+0000","last_peered":"2023-11-24T06:12:32.160719+0000","last_clean":"2023-11-24T06:12:32.160719+0000","last_became_active":"2023-11-24T06:12:31.196658+0000","last_became_peered":"2023-11-24T06:12:31.196658+0000","last_unstale":"2023-11-24T06:12:32.160719+0000","last_undegraded":"2023-11-24T06:12:32.160719+0000","last_fullsized":"2023-11-24T06:12:32.160719+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_clean_scrub_stamp":"2023-11-24T06:12:30.127407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:25:39.965248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-24T06:12:32.167805+0000","last_change":"2023-11-24T06:12:31.196652+0000","last_active":"2023-11-24T06:12:32.167805+0000","last_peered":"2023-11-24T06:12:32.167805+0000","last_clean":"2023-11-24T06:12:32.167805+0000","last_became_active":"2023-11-24T06:12:31.196269+0000","last_became_peered":"2023-11-24T06:12:31.196269+0000","last_unstale":"2023-11-24T06:12:32.167805+0000","last_undegraded":"2023-11-24T06:12:32.167805+0000","last_fullsized":"2023-11-24T06:12:32.167805+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_clean_scrub_stamp":"2023-11-24T06:12:30.127407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:40:20.908351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-24T06:12:32.165936+0000","last_change":"2023-11-24T06:12:31.180751+0000","last_active":"2023-11-24T06:12:32.165936+0000","last_peered":"2023-11-24T06:12:32.165936+0000","last_clean":"2023-11-24T06:12:32.165936+0000","last_became_active":"2023-11-24T06:12:31.180394+0000","last_became_peered":"2023-11-24T06:12:31.180394+0000","last_unstale":"2023-11-24T06:12:32.165936+0000","last_undegraded":"2023-11-24T06:12:32.165936+0000","last_fullsized":"2023-11-24T06:12:32.165936+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_clean_scrub_stamp":"2023-11-24T06:12:30.127407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:54:30.238447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-24T06:12:32.717630+0000","last_change":"2023-11-24T06:12:31.187460+0000","last_active":"2023-11-24T06:12:32.717630+0000","last_peered":"2023-11-24T06:12:32.717630+0000","last_clean":"2023-11-24T06:12:32.717630+0000","last_became_active":"2023-11-24T06:12:31.187186+0000","last_became_peered":"2023-11-24T06:12:31.187186+0000","last_unstale":"2023-11-24T06:12:32.717630+0000","last_undegraded":"2023-11-24T06:12:32.717630+0000","last_fullsized":"2023-11-24T06:12:32.717630+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_clean_scrub_stamp":"2023-11-24T06:12:30.127407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:55:23.096616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-24T06:12:32.717193+0000","last_change":"2023-11-24T06:12:31.176738+0000","last_active":"2023-11-24T06:12:32.717193+0000","last_peered":"2023-11-24T06:12:32.717193+0000","last_clean":"2023-11-24T06:12:32.717193+0000","last_became_active":"2023-11-24T06:12:31.176091+0000","last_became_peered":"2023-11-24T06:12:31.176091+0000","last_unstale":"2023-11-24T06:12:32.717193+0000","last_undegraded":"2023-11-24T06:12:32.717193+0000","last_fullsized":"2023-11-24T06:12:32.717193+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_clean_scrub_stamp":"2023-11-24T06:12:30.127407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:47:16.954718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-24T06:12:32.719159+0000","last_change":"2023-11-24T06:12:31.183783+0000","last_active":"2023-11-24T06:12:32.719159+0000","last_peered":"2023-11-24T06:12:32.719159+0000","last_clean":"2023-11-24T06:12:32.719159+0000","last_became_active":"2023-11-24T06:12:31.183086+0000","last_became_peered":"2023-11-24T06:12:31.183086+0000","last_unstale":"2023-11-24T06:12:32.719159+0000","last_undegraded":"2023-11-24T06:12:32.719159+0000","last_fullsized":"2023-11-24T06:12:32.719159+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_clean_scrub_stamp":"2023-11-24T06:12:30.127407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:53:02.969558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-24T06:12:32.717829+0000","last_change":"2023-11-24T06:12:31.190309+0000","last_active":"2023-11-24T06:12:32.717829+0000","last_peered":"2023-11-24T06:12:32.717829+0000","last_clean":"2023-11-24T06:12:32.717829+0000","last_became_active":"2023-11-24T06:12:31.189984+0000","last_became_peered":"2023-11-24T06:12:31.189984+0000","last_unstale":"2023-11-24T06:12:32.717829+0000","last_undegraded":"2023-11-24T06:12:32.717829+0000","last_fullsized":"2023-11-24T06:12:32.717829+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_clean_scrub_stamp":"2023-11-24T06:12:30.127407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:58:40.674890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-24T06:12:32.161213+0000","last_change":"2023-11-24T06:12:31.192915+0000","last_active":"2023-11-24T06:12:32.161213+0000","last_peered":"2023-11-24T06:12:32.161213+0000","last_clean":"2023-11-24T06:12:32.161213+0000","last_became_active":"2023-11-24T06:12:31.192601+0000","last_became_peered":"2023-11-24T06:12:31.192601+0000","last_unstale":"2023-11-24T06:12:32.161213+0000","last_undegraded":"2023-11-24T06:12:32.161213+0000","last_fullsized":"2023-11-24T06:12:32.161213+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_clean_scrub_stamp":"2023-11-24T06:12:30.127407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:57:21.103658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-24T06:12:32.167332+0000","last_change":"2023-11-24T06:12:31.185380+0000","last_active":"2023-11-24T06:12:32.167332+0000","last_peered":"2023-11-24T06:12:32.167332+0000","last_clean":"2023-11-24T06:12:32.167332+0000","last_became_active":"2023-11-24T06:12:31.184245+0000","last_became_peered":"2023-11-24T06:12:31.184245+0000","last_unstale":"2023-11-24T06:12:32.167332+0000","last_undegraded":"2023-11-24T06:12:32.167332+0000","last_fullsized":"2023-11-24T06:12:32.167332+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_clean_scrub_stamp":"2023-11-24T06:12:30.127407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:42:02.177045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-24T06:12:32.717105+0000","last_change":"2023-11-24T06:12:31.189192+0000","last_active":"2023-11-24T06:12:32.717105+0000","last_peered":"2023-11-24T06:12:32.717105+0000","last_clean":"2023-11-24T06:12:32.717105+0000","last_became_active":"2023-11-24T06:12:31.188899+0000","last_became_peered":"2023-11-24T06:12:31.188899+0000","last_unstale":"2023-11-24T06:12:32.717105+0000","last_undegraded":"2023-11-24T06:12:32.717105+0000","last_fullsized":"2023-11-24T06:12:32.717105+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_clean_scrub_stamp":"2023-11-24T06:12:30.127407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:30:35.115592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-24T06:12:32.716762+0000","last_change":"2023-11-24T06:12:31.193298+0000","last_active":"2023-11-24T06:12:32.716762+0000","last_peered":"2023-11-24T06:12:32.716762+0000","last_clean":"2023-11-24T06:12:32.716762+0000","last_became_active":"2023-11-24T06:12:31.192937+0000","last_became_peered":"2023-11-24T06:12:31.192937+0000","last_unstale":"2023-11-24T06:12:32.716762+0000","last_undegraded":"2023-11-24T06:12:32.716762+0000","last_fullsized":"2023-11-24T06:12:32.716762+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_clean_scrub_stamp":"2023-11-24T06:12:30.127407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:09:09.186156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-24T06:12:32.719941+0000","last_change":"2023-11-24T06:12:31.180838+0000","last_active":"2023-11-24T06:12:32.719941+0000","last_peered":"2023-11-24T06:12:32.719941+0000","last_clean":"2023-11-24T06:12:32.719941+0000","last_became_active":"2023-11-24T06:12:31.179135+0000","last_became_peered":"2023-11-24T06:12:31.179135+0000","last_unstale":"2023-11-24T06:12:32.719941+0000","last_undegraded":"2023-11-24T06:12:32.719941+0000","last_fullsized":"2023-11-24T06:12:32.719941+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_clean_scrub_stamp":"2023-11-24T06:12:30.127407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:10:05.057643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"51'4","reported_seq":16,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-24T06:12:32.720108+0000","last_change":"2023-11-24T06:12:30.170926+0000","last_active":"2023-11-24T06:12:32.720108+0000","last_peered":"2023-11-24T06:12:32.720108+0000","last_clean":"2023-11-24T06:12:32.720108+0000","last_became_active":"2023-11-24T06:12:30.169994+0000","last_became_peered":"2023-11-24T06:12:30.169994+0000","last_unstale":"2023-11-24T06:12:32.720108+0000","last_undegraded":"2023-11-24T06:12:32.720108+0000","last_fullsized":"2023-11-24T06:12:32.720108+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:29.113984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:29.113984+0000","last_clean_scrub_stamp":"2023-11-24T06:12:29.113984+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:35:49.905218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-24T06:12:32.715481+0000","last_change":"2023-11-24T06:12:31.189759+0000","last_active":"2023-11-24T06:12:32.715481+0000","last_peered":"2023-11-24T06:12:32.715481+0000","last_clean":"2023-11-24T06:12:32.715481+0000","last_became_active":"2023-11-24T06:12:31.189333+0000","last_became_peered":"2023-11-24T06:12:31.189333+0000","last_unstale":"2023-11-24T06:12:32.715481+0000","last_undegraded":"2023-11-24T06:12:32.715481+0000","last_fullsized":"2023-11-24T06:12:32.715481+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_clean_scrub_stamp":"2023-11-24T06:12:30.127407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:04:42.070586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"51'1","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-24T06:12:32.161038+0000","last_change":"2023-11-24T06:12:30.172656+0000","last_active":"2023-11-24T06:12:32.161038+0000","last_peered":"2023-11-24T06:12:32.161038+0000","last_clean":"2023-11-24T06:12:32.161038+0000","last_became_active":"2023-11-24T06:12:30.172351+0000","last_became_peered":"2023-11-24T06:12:30.172351+0000","last_unstale":"2023-11-24T06:12:32.161038+0000","last_undegraded":"2023-11-24T06:12:32.161038+0000","last_fullsized":"2023-11-24T06:12:32.161038+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:29.113984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:29.113984+0000","last_clean_scrub_stamp":"2023-11-24T06:12:29.113984+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:58:16.952342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-24T06:12:32.718885+0000","last_change":"2023-11-24T06:12:31.183384+0000","last_active":"2023-11-24T06:12:32.718885+0000","last_peered":"2023-11-24T06:12:32.718885+0000","last_clean":"2023-11-24T06:12:32.718885+0000","last_became_active":"2023-11-24T06:12:31.182743+0000","last_became_peered":"2023-11-24T06:12:31.182743+0000","last_unstale":"2023-11-24T06:12:32.718885+0000","last_undegraded":"2023-11-24T06:12:32.718885+0000","last_fullsized":"2023-11-24T06:12:32.718885+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_clean_scrub_stamp":"2023-11-24T06:12:30.127407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:34:48.602430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"51'2","reported_seq":14,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-24T06:12:32.719579+0000","last_change":"2023-11-24T06:12:30.164705+0000","last_active":"2023-11-24T06:12:32.719579+0000","last_peered":"2023-11-24T06:12:32.719579+0000","last_clean":"2023-11-24T06:12:32.719579+0000","last_became_active":"2023-11-24T06:12:30.161249+0000","last_became_peered":"2023-11-24T06:12:30.161249+0000","last_unstale":"2023-11-24T06:12:32.719579+0000","last_undegraded":"2023-11-24T06:12:32.719579+0000","last_fullsized":"2023-11-24T06:12:32.719579+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:29.113984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:29.113984+0000","last_clean_scrub_stamp":"2023-11-24T06:12:29.113984+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:34:00.242285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-24T06:12:32.719478+0000","last_change":"2023-11-24T06:12:31.184020+0000","last_active":"2023-11-24T06:12:32.719478+0000","last_peered":"2023-11-24T06:12:32.719478+0000","last_clean":"2023-11-24T06:12:32.719478+0000","last_became_active":"2023-11-24T06:12:31.183498+0000","last_became_peered":"2023-11-24T06:12:31.183498+0000","last_unstale":"2023-11-24T06:12:32.719478+0000","last_undegraded":"2023-11-24T06:12:32.719478+0000","last_fullsized":"2023-11-24T06:12:32.719478+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_clean_scrub_stamp":"2023-11-24T06:12:30.127407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:59:24.875576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"51'2","reported_seq":14,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-24T06:12:32.717074+0000","last_change":"2023-11-24T06:12:30.170250+0000","last_active":"2023-11-24T06:12:32.717074+0000","last_peered":"2023-11-24T06:12:32.717074+0000","last_clean":"2023-11-24T06:12:32.717074+0000","last_became_active":"2023-11-24T06:12:30.169714+0000","last_became_peered":"2023-11-24T06:12:30.169714+0000","last_unstale":"2023-11-24T06:12:32.717074+0000","last_undegraded":"2023-11-24T06:12:32.717074+0000","last_fullsized":"2023-11-24T06:12:32.717074+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:29.113984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:29.113984+0000","last_clean_scrub_stamp":"2023-11-24T06:12:29.113984+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:52:04.468529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-24T06:12:32.161047+0000","last_change":"2023-11-24T06:12:31.193576+0000","last_active":"2023-11-24T06:12:32.161047+0000","last_peered":"2023-11-24T06:12:32.161047+0000","last_clean":"2023-11-24T06:12:32.161047+0000","last_became_active":"2023-11-24T06:12:31.193239+0000","last_became_peered":"2023-11-24T06:12:31.193239+0000","last_unstale":"2023-11-24T06:12:32.161047+0000","last_undegraded":"2023-11-24T06:12:32.161047+0000","last_fullsized":"2023-11-24T06:12:32.161047+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_clean_scrub_stamp":"2023-11-24T06:12:30.127407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:11:29.067383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"51'1","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-24T06:12:32.717314+0000","last_change":"2023-11-24T06:12:30.168621+0000","last_active":"2023-11-24T06:12:32.717314+0000","last_peered":"2023-11-24T06:12:32.717314+0000","last_clean":"2023-11-24T06:12:32.717314+0000","last_became_active":"2023-11-24T06:12:30.168312+0000","last_became_peered":"2023-11-24T06:12:30.168312+0000","last_unstale":"2023-11-24T06:12:32.717314+0000","last_undegraded":"2023-11-24T06:12:32.717314+0000","last_fullsized":"2023-11-24T06:12:32.717314+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:29.113984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:29.113984+0000","last_clean_scrub_stamp":"2023-11-24T06:12:29.113984+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:40:59.512025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-24T06:12:32.161074+0000","last_change":"2023-11-24T06:12:31.199120+0000","last_active":"2023-11-24T06:12:32.161074+0000","last_peered":"2023-11-24T06:12:32.161074+0000","last_clean":"2023-11-24T06:12:32.161074+0000","last_became_active":"2023-11-24T06:12:31.198767+0000","last_became_peered":"2023-11-24T06:12:31.198767+0000","last_unstale":"2023-11-24T06:12:32.161074+0000","last_undegraded":"2023-11-24T06:12:32.161074+0000","last_fullsized":"2023-11-24T06:12:32.161074+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_clean_scrub_stamp":"2023-11-24T06:12:30.127407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:45:39.815231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"51'1","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-24T06:12:32.718508+0000","last_change":"2023-11-24T06:12:30.162112+0000","last_active":"2023-11-24T06:12:32.718508+0000","last_peered":"2023-11-24T06:12:32.718508+0000","last_clean":"2023-11-24T06:12:32.718508+0000","last_became_active":"2023-11-24T06:12:30.161858+0000","last_became_peered":"2023-11-24T06:12:30.161858+0000","last_unstale":"2023-11-24T06:12:32.718508+0000","last_undegraded":"2023-11-24T06:12:32.718508+0000","last_fullsized":"2023-11-24T06:12:32.718508+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:29.113984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:29.113984+0000","last_clean_scrub_stamp":"2023-11-24T06:12:29.113984+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:04:41.057163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-24T06:12:32.165857+0000","last_change":"2023-11-24T06:12:31.187113+0000","last_active":"2023-11-24T06:12:32.165857+0000","last_peered":"2023-11-24T06:12:32.165857+0000","last_clean":"2023-11-24T06:12:32.165857+0000","last_became_active":"2023-11-24T06:12:31.186840+0000","last_became_peered":"2023-11-24T06:12:31.186840+0000","last_unstale":"2023-11-24T06:12:32.165857+0000","last_undegraded":"2023-11-24T06:12:32.165857+0000","last_fullsized":"2023-11-24T06:12:32.165857+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_clean_scrub_stamp":"2023-11-24T06:12:30.127407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:13:10.229382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-24T06:12:32.716372+0000","last_change":"2023-11-24T06:12:30.167327+0000","last_active":"2023-11-24T06:12:32.716372+0000","last_peered":"2023-11-24T06:12:32.716372+0000","last_clean":"2023-11-24T06:12:32.716372+0000","last_became_active":"2023-11-24T06:12:30.165432+0000","last_became_peered":"2023-11-24T06:12:30.165432+0000","last_unstale":"2023-11-24T06:12:32.716372+0000","last_undegraded":"2023-11-24T06:12:32.716372+0000","last_fullsized":"2023-11-24T06:12:32.716372+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:29.113984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:29.113984+0000","last_clean_scrub_stamp":"2023-11-24T06:12:29.113984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:32:38.868244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-24T06:12:32.161188+0000","last_change":"2023-11-24T06:12:31.195684+0000","last_active":"2023-11-24T06:12:32.161188+0000","last_peered":"2023-11-24T06:12:32.161188+0000","last_clean":"2023-11-24T06:12:32.161188+0000","last_became_active":"2023-11-24T06:12:31.195158+0000","last_became_peered":"2023-11-24T06:12:31.195158+0000","last_unstale":"2023-11-24T06:12:32.161188+0000","last_undegraded":"2023-11-24T06:12:32.161188+0000","last_fullsized":"2023-11-24T06:12:32.161188+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_clean_scrub_stamp":"2023-11-24T06:12:30.127407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:22:56.206609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-24T06:12:32.715693+0000","last_change":"2023-11-24T06:12:30.173265+0000","last_active":"2023-11-24T06:12:32.715693+0000","last_peered":"2023-11-24T06:12:32.715693+0000","last_clean":"2023-11-24T06:12:32.715693+0000","last_became_active":"2023-11-24T06:12:30.173000+0000","last_became_peered":"2023-11-24T06:12:30.173000+0000","last_unstale":"2023-11-24T06:12:32.715693+0000","last_undegraded":"2023-11-24T06:12:32.715693+0000","last_fullsized":"2023-11-24T06:12:32.715693+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:29.113984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:29.113984+0000","last_clean_scrub_stamp":"2023-11-24T06:12:29.113984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:11:06.890822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-24T06:12:32.719276+0000","last_change":"2023-11-24T06:12:31.182308+0000","last_active":"2023-11-24T06:12:32.719276+0000","last_peered":"2023-11-24T06:12:32.719276+0000","last_clean":"2023-11-24T06:12:32.719276+0000","last_became_active":"2023-11-24T06:12:31.181750+0000","last_became_peered":"2023-11-24T06:12:31.181750+0000","last_unstale":"2023-11-24T06:12:32.719276+0000","last_undegraded":"2023-11-24T06:12:32.719276+0000","last_fullsized":"2023-11-24T06:12:32.719276+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_clean_scrub_stamp":"2023-11-24T06:12:30.127407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:38:37.898801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-24T06:12:32.168578+0000","last_change":"2023-11-24T06:12:30.177615+0000","last_active":"2023-11-24T06:12:32.168578+0000","last_peered":"2023-11-24T06:12:32.168578+0000","last_clean":"2023-11-24T06:12:32.168578+0000","last_became_active":"2023-11-24T06:12:30.177282+0000","last_became_peered":"2023-11-24T06:12:30.177282+0000","last_unstale":"2023-11-24T06:12:32.168578+0000","last_undegraded":"2023-11-24T06:12:32.168578+0000","last_fullsized":"2023-11-24T06:12:32.168578+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:29.113984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:29.113984+0000","last_clean_scrub_stamp":"2023-11-24T06:12:29.113984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:43:23.386785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-24T06:12:32.168507+0000","last_change":"2023-11-24T06:12:31.196996+0000","last_active":"2023-11-24T06:12:32.168507+0000","last_peered":"2023-11-24T06:12:32.168507+0000","last_clean":"2023-11-24T06:12:32.168507+0000","last_became_active":"2023-11-24T06:12:31.196397+0000","last_became_peered":"2023-11-24T06:12:31.196397+0000","last_unstale":"2023-11-24T06:12:32.168507+0000","last_undegraded":"2023-11-24T06:12:32.168507+0000","last_fullsized":"2023-11-24T06:12:32.168507+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_clean_scrub_stamp":"2023-11-24T06:12:30.127407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:32:10.194351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-24T06:12:32.161035+0000","last_change":"2023-11-24T06:12:30.169593+0000","last_active":"2023-11-24T06:12:32.161035+0000","last_peered":"2023-11-24T06:12:32.161035+0000","last_clean":"2023-11-24T06:12:32.161035+0000","last_became_active":"2023-11-24T06:12:30.169000+0000","last_became_peered":"2023-11-24T06:12:30.169000+0000","last_unstale":"2023-11-24T06:12:32.161035+0000","last_undegraded":"2023-11-24T06:12:32.161035+0000","last_fullsized":"2023-11-24T06:12:32.161035+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:29.113984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:29.113984+0000","last_clean_scrub_stamp":"2023-11-24T06:12:29.113984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:01:54.756749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-24T06:12:32.715773+0000","last_change":"2023-11-24T06:12:31.180439+0000","last_active":"2023-11-24T06:12:32.715773+0000","last_peered":"2023-11-24T06:12:32.715773+0000","last_clean":"2023-11-24T06:12:32.715773+0000","last_became_active":"2023-11-24T06:12:31.179555+0000","last_became_peered":"2023-11-24T06:12:31.179555+0000","last_unstale":"2023-11-24T06:12:32.715773+0000","last_undegraded":"2023-11-24T06:12:32.715773+0000","last_fullsized":"2023-11-24T06:12:32.715773+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_clean_scrub_stamp":"2023-11-24T06:12:30.127407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:38:37.898801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"51'3","reported_seq":15,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-24T06:12:32.168963+0000","last_change":"2023-11-24T06:12:30.176966+0000","last_active":"2023-11-24T06:12:32.168963+0000","last_peered":"2023-11-24T06:12:32.168963+0000","last_clean":"2023-11-24T06:12:32.168963+0000","last_became_active":"2023-11-24T06:12:30.176673+0000","last_became_peered":"2023-11-24T06:12:30.176673+0000","last_unstale":"2023-11-24T06:12:32.168963+0000","last_undegraded":"2023-11-24T06:12:32.168963+0000","last_fullsized":"2023-11-24T06:12:32.168963+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:29.113984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:29.113984+0000","last_clean_scrub_stamp":"2023-11-24T06:12:29.113984+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:34:47.589007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-24T06:12:32.168534+0000","last_change":"2023-11-24T06:12:31.194642+0000","last_active":"2023-11-24T06:12:32.168534+0000","last_peered":"2023-11-24T06:12:32.168534+0000","last_clean":"2023-11-24T06:12:32.168534+0000","last_became_active":"2023-11-24T06:12:31.194294+0000","last_became_peered":"2023-11-24T06:12:31.194294+0000","last_unstale":"2023-11-24T06:12:32.168534+0000","last_undegraded":"2023-11-24T06:12:32.168534+0000","last_fullsized":"2023-11-24T06:12:32.168534+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_clean_scrub_stamp":"2023-11-24T06:12:30.127407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:24:35.462590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"51'6","reported_seq":18,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-24T06:12:32.718895+0000","last_change":"2023-11-24T06:12:30.164701+0000","last_active":"2023-11-24T06:12:32.718895+0000","last_peered":"2023-11-24T06:12:32.718895+0000","last_clean":"2023-11-24T06:12:32.718895+0000","last_became_active":"2023-11-24T06:12:30.162468+0000","last_became_peered":"2023-11-24T06:12:30.162468+0000","last_unstale":"2023-11-24T06:12:32.718895+0000","last_undegraded":"2023-11-24T06:12:32.718895+0000","last_fullsized":"2023-11-24T06:12:32.718895+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:29.113984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:29.113984+0000","last_clean_scrub_stamp":"2023-11-24T06:12:29.113984+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:22:54.523841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-24T06:12:32.716146+0000","last_change":"2023-11-24T06:12:31.185563+0000","last_active":"2023-11-24T06:12:32.716146+0000","last_peered":"2023-11-24T06:12:32.716146+0000","last_clean":"2023-11-24T06:12:32.716146+0000","last_became_active":"2023-11-24T06:12:31.185210+0000","last_became_peered":"2023-11-24T06:12:31.185210+0000","last_unstale":"2023-11-24T06:12:32.716146+0000","last_undegraded":"2023-11-24T06:12:32.716146+0000","last_fullsized":"2023-11-24T06:12:32.716146+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_clean_scrub_stamp":"2023-11-24T06:12:30.127407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:35:50.918641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"51'2","reported_seq":14,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-24T06:12:32.161065+0000","last_change":"2023-11-24T06:12:30.168367+0000","last_active":"2023-11-24T06:12:32.161065+0000","last_peered":"2023-11-24T06:12:32.161065+0000","last_clean":"2023-11-24T06:12:32.161065+0000","last_became_active":"2023-11-24T06:12:30.168091+0000","last_became_peered":"2023-11-24T06:12:30.168091+0000","last_unstale":"2023-11-24T06:12:32.161065+0000","last_undegraded":"2023-11-24T06:12:32.161065+0000","last_fullsized":"2023-11-24T06:12:32.161065+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:29.113984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:29.113984+0000","last_clean_scrub_stamp":"2023-11-24T06:12:29.113984+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:19:45.435002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-24T06:12:32.167131+0000","last_change":"2023-11-24T06:12:31.193652+0000","last_active":"2023-11-24T06:12:32.167131+0000","last_peered":"2023-11-24T06:12:32.167131+0000","last_clean":"2023-11-24T06:12:32.167131+0000","last_became_active":"2023-11-24T06:12:31.193264+0000","last_became_peered":"2023-11-24T06:12:31.193264+0000","last_unstale":"2023-11-24T06:12:32.167131+0000","last_undegraded":"2023-11-24T06:12:32.167131+0000","last_fullsized":"2023-11-24T06:12:32.167131+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_clean_scrub_stamp":"2023-11-24T06:12:30.127407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:32:07.709266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-24T06:12:32.165966+0000","last_change":"2023-11-24T06:12:30.170269+0000","last_active":"2023-11-24T06:12:32.165966+0000","last_peered":"2023-11-24T06:12:32.165966+0000","last_clean":"2023-11-24T06:12:32.165966+0000","last_became_active":"2023-11-24T06:12:30.169940+0000","last_became_peered":"2023-11-24T06:12:30.169940+0000","last_unstale":"2023-11-24T06:12:32.165966+0000","last_undegraded":"2023-11-24T06:12:32.165966+0000","last_fullsized":"2023-11-24T06:12:32.165966+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:29.113984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:29.113984+0000","last_clean_scrub_stamp":"2023-11-24T06:12:29.113984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:40:51.288335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-24T06:12:32.715676+0000","last_change":"2023-11-24T06:12:31.183845+0000","last_active":"2023-11-24T06:12:32.715676+0000","last_peered":"2023-11-24T06:12:32.715676+0000","last_clean":"2023-11-24T06:12:32.715676+0000","last_became_active":"2023-11-24T06:12:31.183296+0000","last_became_peered":"2023-11-24T06:12:31.183296+0000","last_unstale":"2023-11-24T06:12:32.715676+0000","last_undegraded":"2023-11-24T06:12:32.715676+0000","last_fullsized":"2023-11-24T06:12:32.715676+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_clean_scrub_stamp":"2023-11-24T06:12:30.127407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:14:56.855135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-24T06:12:32.716390+0000","last_change":"2023-11-24T06:12:30.174303+0000","last_active":"2023-11-24T06:12:32.716390+0000","last_peered":"2023-11-24T06:12:32.716390+0000","last_clean":"2023-11-24T06:12:32.716390+0000","last_became_active":"2023-11-24T06:12:30.173929+0000","last_became_peered":"2023-11-24T06:12:30.173929+0000","last_unstale":"2023-11-24T06:12:32.716390+0000","last_undegraded":"2023-11-24T06:12:32.716390+0000","last_fullsized":"2023-11-24T06:12:32.716390+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:29.113984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:29.113984+0000","last_clean_scrub_stamp":"2023-11-24T06:12:29.113984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:14:55.841712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-24T06:12:32.167937+0000","last_change":"2023-11-24T06:12:31.196653+0000","last_active":"2023-11-24T06:12:32.167937+0000","last_peered":"2023-11-24T06:12:32.167937+0000","last_clean":"2023-11-24T06:12:32.167937+0000","last_became_active":"2023-11-24T06:12:31.196275+0000","last_became_peered":"2023-11-24T06:12:31.196275+0000","last_unstale":"2023-11-24T06:12:32.167937+0000","last_undegraded":"2023-11-24T06:12:32.167937+0000","last_fullsized":"2023-11-24T06:12:32.167937+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_clean_scrub_stamp":"2023-11-24T06:12:30.127407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:02:11.931305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"51'8","reported_seq":20,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-24T06:12:32.165954+0000","last_change":"2023-11-24T06:12:30.174960+0000","last_active":"2023-11-24T06:12:32.165954+0000","last_peered":"2023-11-24T06:12:32.165954+0000","last_clean":"2023-11-24T06:12:32.165954+0000","last_became_active":"2023-11-24T06:12:30.174626+0000","last_became_peered":"2023-11-24T06:12:30.174626+0000","last_unstale":"2023-11-24T06:12:32.165954+0000","last_undegraded":"2023-11-24T06:12:32.165954+0000","last_fullsized":"2023-11-24T06:12:32.165954+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:29.113984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:29.113984+0000","last_clean_scrub_stamp":"2023-11-24T06:12:29.113984+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:47:15.941295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-24T06:12:32.168668+0000","last_change":"2023-11-24T06:12:31.191797+0000","last_active":"2023-11-24T06:12:32.168668+0000","last_peered":"2023-11-24T06:12:32.168668+0000","last_clean":"2023-11-24T06:12:32.168668+0000","last_became_active":"2023-11-24T06:12:31.191348+0000","last_became_peered":"2023-11-24T06:12:31.191348+0000","last_unstale":"2023-11-24T06:12:32.168668+0000","last_undegraded":"2023-11-24T06:12:32.168668+0000","last_fullsized":"2023-11-24T06:12:32.168668+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_clean_scrub_stamp":"2023-11-24T06:12:30.127407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:19:46.448425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"51'1","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-24T06:12:32.161359+0000","last_change":"2023-11-24T06:12:30.172737+0000","last_active":"2023-11-24T06:12:32.161359+0000","last_peered":"2023-11-24T06:12:32.161359+0000","last_clean":"2023-11-24T06:12:32.161359+0000","last_became_active":"2023-11-24T06:12:30.172021+0000","last_became_peered":"2023-11-24T06:12:30.172021+0000","last_unstale":"2023-11-24T06:12:32.161359+0000","last_undegraded":"2023-11-24T06:12:32.161359+0000","last_fullsized":"2023-11-24T06:12:32.161359+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:29.113984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:29.113984+0000","last_clean_scrub_stamp":"2023-11-24T06:12:29.113984+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:32:09.180928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-24T06:12:32.717889+0000","last_change":"2023-11-24T06:12:31.185551+0000","last_active":"2023-11-24T06:12:32.717889+0000","last_peered":"2023-11-24T06:12:32.717889+0000","last_clean":"2023-11-24T06:12:32.717889+0000","last_became_active":"2023-11-24T06:12:31.185208+0000","last_became_peered":"2023-11-24T06:12:31.185208+0000","last_unstale":"2023-11-24T06:12:32.717889+0000","last_undegraded":"2023-11-24T06:12:32.717889+0000","last_fullsized":"2023-11-24T06:12:32.717889+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_clean_scrub_stamp":"2023-11-24T06:12:30.127407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:49:46.518805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"51'2","reported_seq":14,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-24T06:12:32.720049+0000","last_change":"2023-11-24T06:12:30.170427+0000","last_active":"2023-11-24T06:12:32.720049+0000","last_peered":"2023-11-24T06:12:32.720049+0000","last_clean":"2023-11-24T06:12:32.720049+0000","last_became_active":"2023-11-24T06:12:30.170062+0000","last_became_peered":"2023-11-24T06:12:30.170062+0000","last_unstale":"2023-11-24T06:12:32.720049+0000","last_undegraded":"2023-11-24T06:12:32.720049+0000","last_fullsized":"2023-11-24T06:12:32.720049+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:29.113984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:29.113984+0000","last_clean_scrub_stamp":"2023-11-24T06:12:29.113984+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:02:22.416616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-24T06:12:32.161122+0000","last_change":"2023-11-24T06:12:31.188508+0000","last_active":"2023-11-24T06:12:32.161122+0000","last_peered":"2023-11-24T06:12:32.161122+0000","last_clean":"2023-11-24T06:12:32.161122+0000","last_became_active":"2023-11-24T06:12:31.188217+0000","last_became_peered":"2023-11-24T06:12:31.188217+0000","last_unstale":"2023-11-24T06:12:32.161122+0000","last_undegraded":"2023-11-24T06:12:32.161122+0000","last_fullsized":"2023-11-24T06:12:32.161122+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_clean_scrub_stamp":"2023-11-24T06:12:30.127407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:35:02.628615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"51'1","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-24T06:12:32.716761+0000","last_change":"2023-11-24T06:12:30.172394+0000","last_active":"2023-11-24T06:12:32.716761+0000","last_peered":"2023-11-24T06:12:32.716761+0000","last_clean":"2023-11-24T06:12:32.716761+0000","last_became_active":"2023-11-24T06:12:30.172115+0000","last_became_peered":"2023-11-24T06:12:30.172115+0000","last_unstale":"2023-11-24T06:12:32.716761+0000","last_undegraded":"2023-11-24T06:12:32.716761+0000","last_fullsized":"2023-11-24T06:12:32.716761+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:29.113984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:29.113984+0000","last_clean_scrub_stamp":"2023-11-24T06:12:29.113984+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:34:00.242285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-24T06:12:32.168881+0000","last_change":"2023-11-24T06:12:31.194745+0000","last_active":"2023-11-24T06:12:32.168881+0000","last_peered":"2023-11-24T06:12:32.168881+0000","last_clean":"2023-11-24T06:12:32.168881+0000","last_became_active":"2023-11-24T06:12:31.194449+0000","last_became_peered":"2023-11-24T06:12:31.194449+0000","last_unstale":"2023-11-24T06:12:32.168881+0000","last_undegraded":"2023-11-24T06:12:32.168881+0000","last_fullsized":"2023-11-24T06:12:32.168881+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_clean_scrub_stamp":"2023-11-24T06:12:30.127407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:01:55.770172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-24T06:12:32.719124+0000","last_change":"2023-11-24T06:12:30.174464+0000","last_active":"2023-11-24T06:12:32.719124+0000","last_peered":"2023-11-24T06:12:32.719124+0000","last_clean":"2023-11-24T06:12:32.719124+0000","last_became_active":"2023-11-24T06:12:30.174177+0000","last_became_peered":"2023-11-24T06:12:30.174177+0000","last_unstale":"2023-11-24T06:12:32.719124+0000","last_undegraded":"2023-11-24T06:12:32.719124+0000","last_fullsized":"2023-11-24T06:12:32.719124+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:29.113984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:29.113984+0000","last_clean_scrub_stamp":"2023-11-24T06:12:29.113984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:03:02.518611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-24T06:12:32.716194+0000","last_change":"2023-11-24T06:12:31.186486+0000","last_active":"2023-11-24T06:12:32.716194+0000","last_peered":"2023-11-24T06:12:32.716194+0000","last_clean":"2023-11-24T06:12:32.716194+0000","last_became_active":"2023-11-24T06:12:31.184729+0000","last_became_peered":"2023-11-24T06:12:31.184729+0000","last_unstale":"2023-11-24T06:12:32.716194+0000","last_undegraded":"2023-11-24T06:12:32.716194+0000","last_fullsized":"2023-11-24T06:12:32.716194+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_clean_scrub_stamp":"2023-11-24T06:12:30.127407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:03:03.532034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"51'2","reported_seq":14,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-24T06:12:32.161188+0000","last_change":"2023-11-24T06:12:30.165766+0000","last_active":"2023-11-24T06:12:32.161188+0000","last_peered":"2023-11-24T06:12:32.161188+0000","last_clean":"2023-11-24T06:12:32.161188+0000","last_became_active":"2023-11-24T06:12:30.165281+0000","last_became_peered":"2023-11-24T06:12:30.165281+0000","last_unstale":"2023-11-24T06:12:32.161188+0000","last_undegraded":"2023-11-24T06:12:32.161188+0000","last_fullsized":"2023-11-24T06:12:32.161188+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:29.113984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:29.113984+0000","last_clean_scrub_stamp":"2023-11-24T06:12:29.113984+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:46:03.597280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-24T06:12:32.717994+0000","last_change":"2023-11-24T06:12:31.187800+0000","last_active":"2023-11-24T06:12:32.717994+0000","last_peered":"2023-11-24T06:12:32.717994+0000","last_clean":"2023-11-24T06:12:32.717994+0000","last_became_active":"2023-11-24T06:12:31.183035+0000","last_became_peered":"2023-11-24T06:12:31.183035+0000","last_unstale":"2023-11-24T06:12:32.717994+0000","last_undegraded":"2023-11-24T06:12:32.717994+0000","last_fullsized":"2023-11-24T06:12:32.717994+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_clean_scrub_stamp":"2023-11-24T06:12:30.127407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:33:17.733215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-24T06:12:32.715511+0000","last_change":"2023-11-24T06:12:30.165882+0000","last_active":"2023-11-24T06:12:32.715511+0000","last_peered":"2023-11-24T06:12:32.715511+0000","last_clean":"2023-11-24T06:12:32.715511+0000","last_became_active":"2023-11-24T06:12:30.165610+0000","last_became_peered":"2023-11-24T06:12:30.165610+0000","last_unstale":"2023-11-24T06:12:32.715511+0000","last_undegraded":"2023-11-24T06:12:32.715511+0000","last_fullsized":"2023-11-24T06:12:32.715511+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:29.113984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:29.113984+0000","last_clean_scrub_stamp":"2023-11-24T06:12:29.113984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:23:52.481214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-24T06:12:32.716939+0000","last_change":"2023-11-24T06:12:31.185398+0000","last_active":"2023-11-24T06:12:32.716939+0000","last_peered":"2023-11-24T06:12:32.716939+0000","last_clean":"2023-11-24T06:12:32.716939+0000","last_became_active":"2023-11-24T06:12:31.185056+0000","last_became_peered":"2023-11-24T06:12:31.185056+0000","last_unstale":"2023-11-24T06:12:32.716939+0000","last_undegraded":"2023-11-24T06:12:32.716939+0000","last_fullsized":"2023-11-24T06:12:32.716939+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_clean_scrub_stamp":"2023-11-24T06:12:30.127407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:47:16.954718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-24T06:12:32.715776+0000","last_change":"2023-11-24T06:12:30.171661+0000","last_active":"2023-11-24T06:12:32.715776+0000","last_peered":"2023-11-24T06:12:32.715776+0000","last_clean":"2023-11-24T06:12:32.715776+0000","last_became_active":"2023-11-24T06:12:30.170946+0000","last_became_peered":"2023-11-24T06:12:30.170946+0000","last_unstale":"2023-11-24T06:12:32.715776+0000","last_undegraded":"2023-11-24T06:12:32.715776+0000","last_fullsized":"2023-11-24T06:12:32.715776+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:29.113984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:29.113984+0000","last_clean_scrub_stamp":"2023-11-24T06:12:29.113984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:20:42.313069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-24T06:12:32.717564+0000","last_change":"2023-11-24T06:12:31.189248+0000","last_active":"2023-11-24T06:12:32.717564+0000","last_peered":"2023-11-24T06:12:32.717564+0000","last_clean":"2023-11-24T06:12:32.717564+0000","last_became_active":"2023-11-24T06:12:31.188097+0000","last_became_peered":"2023-11-24T06:12:31.188097+0000","last_unstale":"2023-11-24T06:12:32.717564+0000","last_undegraded":"2023-11-24T06:12:32.717564+0000","last_fullsized":"2023-11-24T06:12:32.717564+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_clean_scrub_stamp":"2023-11-24T06:12:30.127407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:33:17.733215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"51'2","reported_seq":14,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-24T06:12:32.715672+0000","last_change":"2023-11-24T06:12:30.173946+0000","last_active":"2023-11-24T06:12:32.715672+0000","last_peered":"2023-11-24T06:12:32.715672+0000","last_clean":"2023-11-24T06:12:32.715672+0000","last_became_active":"2023-11-24T06:12:30.173665+0000","last_became_peered":"2023-11-24T06:12:30.173665+0000","last_unstale":"2023-11-24T06:12:32.715672+0000","last_undegraded":"2023-11-24T06:12:32.715672+0000","last_fullsized":"2023-11-24T06:12:32.715672+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:29.113984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:29.113984+0000","last_clean_scrub_stamp":"2023-11-24T06:12:29.113984+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:14:55.841712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-24T06:12:32.718086+0000","last_change":"2023-11-24T06:12:31.187788+0000","last_active":"2023-11-24T06:12:32.718086+0000","last_peered":"2023-11-24T06:12:32.718086+0000","last_clean":"2023-11-24T06:12:32.718086+0000","last_became_active":"2023-11-24T06:12:31.187459+0000","last_became_peered":"2023-11-24T06:12:31.187459+0000","last_unstale":"2023-11-24T06:12:32.718086+0000","last_undegraded":"2023-11-24T06:12:32.718086+0000","last_fullsized":"2023-11-24T06:12:32.718086+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_clean_scrub_stamp":"2023-11-24T06:12:30.127407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:34:48.602430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-24T06:12:32.720152+0000","last_change":"2023-11-24T06:12:30.163488+0000","last_active":"2023-11-24T06:12:32.720152+0000","last_peered":"2023-11-24T06:12:32.720152+0000","last_clean":"2023-11-24T06:12:32.720152+0000","last_became_active":"2023-11-24T06:12:30.163196+0000","last_became_peered":"2023-11-24T06:12:30.163196+0000","last_unstale":"2023-11-24T06:12:32.720152+0000","last_undegraded":"2023-11-24T06:12:32.720152+0000","last_fullsized":"2023-11-24T06:12:32.720152+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:29.113984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:29.113984+0000","last_clean_scrub_stamp":"2023-11-24T06:12:29.113984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:29:09.486165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-24T06:12:32.715395+0000","last_change":"2023-11-24T06:12:31.177668+0000","last_active":"2023-11-24T06:12:32.715395+0000","last_peered":"2023-11-24T06:12:32.715395+0000","last_clean":"2023-11-24T06:12:32.715395+0000","last_became_active":"2023-11-24T06:12:31.177407+0000","last_became_peered":"2023-11-24T06:12:31.177407+0000","last_unstale":"2023-11-24T06:12:32.715395+0000","last_undegraded":"2023-11-24T06:12:32.715395+0000","last_fullsized":"2023-11-24T06:12:32.715395+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_clean_scrub_stamp":"2023-11-24T06:12:30.127407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:34:53.658692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-24T06:12:32.166671+0000","last_change":"2023-11-24T06:12:30.162538+0000","last_active":"2023-11-24T06:12:32.166671+0000","last_peered":"2023-11-24T06:12:32.166671+0000","last_clean":"2023-11-24T06:12:32.166671+0000","last_became_active":"2023-11-24T06:12:30.161061+0000","last_became_peered":"2023-11-24T06:12:30.161061+0000","last_unstale":"2023-11-24T06:12:32.166671+0000","last_undegraded":"2023-11-24T06:12:32.166671+0000","last_fullsized":"2023-11-24T06:12:32.166671+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:29.113984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:29.113984+0000","last_clean_scrub_stamp":"2023-11-24T06:12:29.113984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:56:14.550653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-24T06:12:32.719603+0000","last_change":"2023-11-24T06:12:31.180889+0000","last_active":"2023-11-24T06:12:32.719603+0000","last_peered":"2023-11-24T06:12:32.719603+0000","last_clean":"2023-11-24T06:12:32.719603+0000","last_became_active":"2023-11-24T06:12:31.180089+0000","last_became_peered":"2023-11-24T06:12:31.180089+0000","last_unstale":"2023-11-24T06:12:32.719603+0000","last_undegraded":"2023-11-24T06:12:32.719603+0000","last_fullsized":"2023-11-24T06:12:32.719603+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_clean_scrub_stamp":"2023-11-24T06:12:30.127407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:45:14.012287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-24T06:12:32.717151+0000","last_change":"2023-11-24T06:12:30.165117+0000","last_active":"2023-11-24T06:12:32.717151+0000","last_peered":"2023-11-24T06:12:32.717151+0000","last_clean":"2023-11-24T06:12:32.717151+0000","last_became_active":"2023-11-24T06:12:30.164738+0000","last_became_peered":"2023-11-24T06:12:30.164738+0000","last_unstale":"2023-11-24T06:12:32.717151+0000","last_undegraded":"2023-11-24T06:12:32.717151+0000","last_fullsized":"2023-11-24T06:12:32.717151+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:29.113984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:29.113984+0000","last_clean_scrub_stamp":"2023-11-24T06:12:29.113984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:02:22.416616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-24T06:12:32.160876+0000","last_change":"2023-11-24T06:12:31.193439+0000","last_active":"2023-11-24T06:12:32.160876+0000","last_peered":"2023-11-24T06:12:32.160876+0000","last_clean":"2023-11-24T06:12:32.160876+0000","last_became_active":"2023-11-24T06:12:31.193147+0000","last_became_peered":"2023-11-24T06:12:31.193147+0000","last_unstale":"2023-11-24T06:12:32.160876+0000","last_undegraded":"2023-11-24T06:12:32.160876+0000","last_fullsized":"2023-11-24T06:12:32.160876+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_clean_scrub_stamp":"2023-11-24T06:12:30.127407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:09:42.396697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-24T06:12:32.716399+0000","last_change":"2023-11-24T06:12:30.168562+0000","last_active":"2023-11-24T06:12:32.716399+0000","last_peered":"2023-11-24T06:12:32.716399+0000","last_clean":"2023-11-24T06:12:32.716399+0000","last_became_active":"2023-11-24T06:12:30.168336+0000","last_became_peered":"2023-11-24T06:12:30.168336+0000","last_unstale":"2023-11-24T06:12:32.716399+0000","last_undegraded":"2023-11-24T06:12:32.716399+0000","last_fullsized":"2023-11-24T06:12:32.716399+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:29.113984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:29.113984+0000","last_clean_scrub_stamp":"2023-11-24T06:12:29.113984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:32:38.868244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-24T06:12:32.167394+0000","last_change":"2023-11-24T06:12:31.197065+0000","last_active":"2023-11-24T06:12:32.167394+0000","last_peered":"2023-11-24T06:12:32.167394+0000","last_clean":"2023-11-24T06:12:32.167394+0000","last_became_active":"2023-11-24T06:12:31.196691+0000","last_became_peered":"2023-11-24T06:12:31.196691+0000","last_unstale":"2023-11-24T06:12:32.167394+0000","last_undegraded":"2023-11-24T06:12:32.167394+0000","last_fullsized":"2023-11-24T06:12:32.167394+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_clean_scrub_stamp":"2023-11-24T06:12:30.127407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:25:07.777538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-24T06:12:32.718566+0000","last_change":"2023-11-24T06:12:30.162492+0000","last_active":"2023-11-24T06:12:32.718566+0000","last_peered":"2023-11-24T06:12:32.718566+0000","last_clean":"2023-11-24T06:12:32.718566+0000","last_became_active":"2023-11-24T06:12:30.162161+0000","last_became_peered":"2023-11-24T06:12:30.162161+0000","last_unstale":"2023-11-24T06:12:32.718566+0000","last_undegraded":"2023-11-24T06:12:32.718566+0000","last_fullsized":"2023-11-24T06:12:32.718566+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:29.113984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:29.113984+0000","last_clean_scrub_stamp":"2023-11-24T06:12:29.113984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:32:38.868244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-24T06:12:32.714757+0000","last_change":"2023-11-24T06:12:31.175110+0000","last_active":"2023-11-24T06:12:32.714757+0000","last_peered":"2023-11-24T06:12:32.714757+0000","last_clean":"2023-11-24T06:12:32.714757+0000","last_became_active":"2023-11-24T06:12:31.174422+0000","last_became_peered":"2023-11-24T06:12:31.174422+0000","last_unstale":"2023-11-24T06:12:32.714757+0000","last_undegraded":"2023-11-24T06:12:32.714757+0000","last_fullsized":"2023-11-24T06:12:32.714757+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_clean_scrub_stamp":"2023-11-24T06:12:30.127407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:04:19.413427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-24T06:12:32.715573+0000","last_change":"2023-11-24T06:12:30.159143+0000","last_active":"2023-11-24T06:12:32.715573+0000","last_peered":"2023-11-24T06:12:32.715573+0000","last_clean":"2023-11-24T06:12:32.715573+0000","last_became_active":"2023-11-24T06:12:30.158841+0000","last_became_peered":"2023-11-24T06:12:30.158841+0000","last_unstale":"2023-11-24T06:12:32.715573+0000","last_undegraded":"2023-11-24T06:12:32.715573+0000","last_fullsized":"2023-11-24T06:12:32.715573+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:29.113984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:29.113984+0000","last_clean_scrub_stamp":"2023-11-24T06:12:29.113984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:20:42.313069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-24T06:12:32.160910+0000","last_change":"2023-11-24T06:12:31.195508+0000","last_active":"2023-11-24T06:12:32.160910+0000","last_peered":"2023-11-24T06:12:32.160910+0000","last_clean":"2023-11-24T06:12:32.160910+0000","last_became_active":"2023-11-24T06:12:31.194717+0000","last_became_peered":"2023-11-24T06:12:31.194717+0000","last_unstale":"2023-11-24T06:12:32.160910+0000","last_undegraded":"2023-11-24T06:12:32.160910+0000","last_fullsized":"2023-11-24T06:12:32.160910+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_clean_scrub_stamp":"2023-11-24T06:12:30.127407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:12:48.584142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-24T06:12:32.160811+0000","last_change":"2023-11-24T06:12:30.170567+0000","last_active":"2023-11-24T06:12:32.160811+0000","last_peered":"2023-11-24T06:12:32.160811+0000","last_clean":"2023-11-24T06:12:32.160811+0000","last_became_active":"2023-11-24T06:12:30.169458+0000","last_became_peered":"2023-11-24T06:12:30.169458+0000","last_unstale":"2023-11-24T06:12:32.160811+0000","last_undegraded":"2023-11-24T06:12:32.160811+0000","last_fullsized":"2023-11-24T06:12:32.160811+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:29.113984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:29.113984+0000","last_clean_scrub_stamp":"2023-11-24T06:12:29.113984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:24:34.449167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-24T06:12:32.160698+0000","last_change":"2023-11-24T06:12:31.199006+0000","last_active":"2023-11-24T06:12:32.160698+0000","last_peered":"2023-11-24T06:12:32.160698+0000","last_clean":"2023-11-24T06:12:32.160698+0000","last_became_active":"2023-11-24T06:12:31.198718+0000","last_became_peered":"2023-11-24T06:12:31.198718+0000","last_unstale":"2023-11-24T06:12:32.160698+0000","last_undegraded":"2023-11-24T06:12:32.160698+0000","last_fullsized":"2023-11-24T06:12:32.160698+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:30.127407+0000","last_clean_scrub_stamp":"2023-11-24T06:12:30.127407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:19:49.094045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-24T06:12:32.167166+0000","last_change":"2023-11-24T06:12:30.167396+0000","last_active":"2023-11-24T06:12:32.167166+0000","last_peered":"2023-11-24T06:12:32.167166+0000","last_clean":"2023-11-24T06:12:32.167166+0000","last_became_active":"2023-11-24T06:12:30.167076+0000","last_became_peered":"2023-11-24T06:12:30.167076+0000","last_unstale":"2023-11-24T06:12:32.167166+0000","last_undegraded":"2023-11-24T06:12:32.167166+0000","last_fullsized":"2023-11-24T06:12:32.167166+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:29.113984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:29.113984+0000","last_clean_scrub_stamp":"2023-11-24T06:12:29.113984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:34:57.044397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":193,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-24T06:12:31.179110+0000","last_change":"2023-11-24T06:10:20.082677+0000","last_active":"2023-11-24T06:12:31.179110+0000","last_peered":"2023-11-24T06:12:31.179110+0000","last_clean":"2023-11-24T06:12:31.179110+0000","last_became_active":"2023-11-24T06:10:20.080551+0000","last_became_peered":"2023-11-24T06:10:20.080551+0000","last_unstale":"2023-11-24T06:12:31.179110+0000","last_undegraded":"2023-11-24T06:12:31.179110+0000","last_fullsized":"2023-11-24T06:12:31.179110+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:18.602209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:18.602209+0000","last_clean_scrub_stamp":"2023-11-24T06:10:18.602209+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:47:24.216235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574877,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32384,"kb_used_data":1196,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339456,"statfs":{"total":96636764160,"available":96603602944,"internally_reserved":0,"allocated":1224704,"data_stored":855575,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:12:18 2023","interfaces":[{"interface":"back","average":{"1min":1.016,"5min":1.091,"15min":1.103},"min":{"1min":0.75600000000000001,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":1.5580000000000001,"5min":1.9630000000000001,"15min":1.9630000000000001},"last":1.262},{"interface":"front","average":{"1min":1.0529999999999999,"5min":1.056,"15min":1.056},"min":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":1.605,"5min":1.621,"15min":1.621},"last":1.204}]},{"osd":1,"last update":"Fri Nov 24 06:12:18 2023","interfaces":[{"interface":"back","average":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"min":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":1.657,"5min":2.0259999999999998,"15min":2.0259999999999998},"last":0.93400000000000005},{"interface":"front","average":{"1min":1.226,"5min":1.149,"15min":1.1359999999999999},"min":{"1min":0.70699999999999996,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.786,"5min":2.1480000000000001,"15min":2.1480000000000001},"last":1.601}]},{"osd":2,"last update":"Fri Nov 24 06:12:18 2023","interfaces":[{"interface":"back","average":{"1min":1.1000000000000001,"5min":1.1339999999999999,"15min":1.1399999999999999},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.488,"5min":1.7809999999999999,"15min":1.7809999999999999},"last":1.1659999999999999},{"interface":"front","average":{"1min":1.2370000000000001,"5min":1.1850000000000001,"15min":1.1759999999999999},"min":{"1min":0.90900000000000003,"5min":0.86599999999999999,"15min":0.86599999999999999},"max":{"1min":2.0619999999999998,"5min":2.0619999999999998,"15min":2.0619999999999998},"last":1.3500000000000001}]},{"osd":3,"last update":"Fri Nov 24 06:12:18 2023","interfaces":[{"interface":"back","average":{"1min":1.1970000000000001,"5min":1.204,"15min":1.2050000000000001},"min":{"1min":0.89600000000000002,"5min":0.73999999999999999,"15min":0.73999999999999999},"max":{"1min":1.615,"5min":1.615,"15min":1.615},"last":1.002},{"interface":"front","average":{"1min":1.2909999999999999,"5min":1.3069999999999999,"15min":1.3089999999999999},"min":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"max":{"1min":1.9570000000000001,"5min":1.9570000000000001,"15min":1.9570000000000001},"last":1.4530000000000001}]},{"osd":4,"last update":"Fri Nov 24 06:12:18 2023","interfaces":[{"interface":"back","average":{"1min":1.3660000000000001,"5min":1.304,"15min":1.294},"min":{"1min":0.748,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.9570000000000001,"5min":1.9570000000000001,"15min":1.9570000000000001},"last":1.371},{"interface":"front","average":{"1min":1.3320000000000001,"5min":1.329,"15min":1.329},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.8340000000000001,"5min":2.1680000000000001,"15min":2.1680000000000001},"last":1.3200000000000001}]},{"osd":5,"last update":"Fri Nov 24 06:12:18 2023","interfaces":[{"interface":"back","average":{"1min":1.546,"5min":1.466,"15min":1.452},"min":{"1min":0.75900000000000001,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":2.0379999999999998,"5min":2.0379999999999998,"15min":2.0379999999999998},"last":1.4910000000000001},{"interface":"front","average":{"1min":1.5720000000000001,"5min":1.4770000000000001,"15min":1.4610000000000001},"min":{"1min":1.077,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.923,"5min":2.0539999999999998,"15min":2.0539999999999998},"last":1.413}]},{"osd":6,"last update":"Fri Nov 24 06:12:18 2023","interfaces":[{"interface":"back","average":{"1min":1.6559999999999999,"5min":1.548,"15min":1.53},"min":{"1min":0.94499999999999995,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":2.0899999999999999,"5min":2.0899999999999999,"15min":2.0899999999999999},"last":1.675},{"interface":"front","average":{"1min":1.6719999999999999,"5min":1.5580000000000001,"15min":1.5389999999999999},"min":{"1min":1.127,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":2.0070000000000001,"5min":2.0680000000000001,"15min":2.0680000000000001},"last":1.548}]}]},{"osd":6,"up_from":13,"seq":55834574878,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31792,"kb_used_data":608,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340048,"statfs":{"total":96636764160,"available":96604209152,"internally_reserved":0,"allocated":622592,"data_stored":265047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:12:21 2023","interfaces":[{"interface":"back","average":{"1min":1.097,"5min":1.194,"15min":1.21},"min":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":2.032,"5min":2.032,"15min":2.032},"last":0.97499999999999998},{"interface":"front","average":{"1min":1.1639999999999999,"5min":1.212,"15min":1.22},"min":{"1min":0.81599999999999995,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.9199999999999999,"5min":2.0230000000000001,"15min":2.0230000000000001},"last":0.77800000000000002}]},{"osd":1,"last update":"Fri Nov 24 06:12:21 2023","interfaces":[{"interface":"back","average":{"1min":1.149,"5min":1.163,"15min":1.165},"min":{"1min":0.72199999999999998,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":2.1499999999999999,"5min":2.1499999999999999,"15min":2.1499999999999999},"last":0.84699999999999998},{"interface":"front","average":{"1min":1.1970000000000001,"5min":1.2270000000000001,"15min":1.232},"min":{"1min":0.78700000000000003,"5min":0.77500000000000002,"15min":0.77500000000000002},"max":{"1min":1.645,"5min":1.8899999999999999,"15min":1.8899999999999999},"last":0.92300000000000004}]},{"osd":2,"last update":"Fri Nov 24 06:12:21 2023","interfaces":[{"interface":"back","average":{"1min":1.4179999999999999,"5min":1.2050000000000001,"15min":1.169},"min":{"1min":0.88,"5min":0.82299999999999995,"15min":0.82299999999999995},"max":{"1min":4.7359999999999998,"5min":4.7359999999999998,"15min":4.7359999999999998},"last":1.224},{"interface":"front","average":{"1min":1.4970000000000001,"5min":1.45,"15min":1.4419999999999999},"min":{"1min":0.93799999999999994,"5min":0.85899999999999999,"15min":0.85899999999999999},"max":{"1min":3.9700000000000002,"5min":3.9700000000000002,"15min":3.9700000000000002},"last":1.087}]},{"osd":3,"last update":"Fri Nov 24 06:12:21 2023","interfaces":[{"interface":"back","average":{"1min":1.2809999999999999,"5min":1.3069999999999999,"15min":1.3109999999999999},"min":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"max":{"1min":1.798,"5min":1.919,"15min":1.919},"last":1.0569999999999999},{"interface":"front","average":{"1min":1.28,"5min":1.292,"15min":1.294},"min":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":2.3929999999999998,"5min":2.3929999999999998,"15min":2.3929999999999998},"last":1.379}]},{"osd":4,"last update":"Fri Nov 24 06:12:21 2023","interfaces":[{"interface":"back","average":{"1min":1.4470000000000001,"5min":1.4550000000000001,"15min":1.4570000000000001},"min":{"1min":0.873,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":2.4039999999999999,"5min":2.4039999999999999,"15min":2.4039999999999999},"last":1.175},{"interface":"front","average":{"1min":1.5620000000000001,"5min":1.496,"15min":1.4850000000000001},"min":{"1min":0.88400000000000001,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":2.5609999999999999,"5min":2.5609999999999999,"15min":2.5609999999999999},"last":1.1499999999999999}]},{"osd":5,"last update":"Fri Nov 24 06:12:21 2023","interfaces":[{"interface":"back","average":{"1min":1.4610000000000001,"5min":1.4730000000000001,"15min":1.4750000000000001},"min":{"1min":0.90000000000000002,"5min":0.79100000000000004,"15min":0.79100000000000004},"max":{"1min":2.8290000000000002,"5min":2.8290000000000002,"15min":2.8290000000000002},"last":1.4490000000000001},{"interface":"front","average":{"1min":1.639,"5min":1.653,"15min":1.655},"min":{"1min":1.2250000000000001,"5min":0.93400000000000005,"15min":0.93400000000000005},"max":{"1min":2.9420000000000002,"5min":2.9420000000000002,"15min":2.9420000000000002},"last":1.2709999999999999}]},{"osd":7,"last update":"Fri Nov 24 06:12:21 2023","interfaces":[{"interface":"back","average":{"1min":1.6160000000000001,"5min":1.5960000000000001,"15min":1.5920000000000001},"min":{"1min":1.1299999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":2.1859999999999999,"5min":2.5139999999999998,"15min":2.5139999999999998},"last":1.337},{"interface":"front","average":{"1min":1.7509999999999999,"5min":1.5880000000000001,"15min":1.5609999999999999},"min":{"1min":1.0249999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":4.7859999999999996,"5min":4.7859999999999996,"15min":4.7859999999999996},"last":1.504}]}]},{"osd":5,"up_from":13,"seq":55834574878,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31780,"kb_used_data":600,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340060,"statfs":{"total":96636764160,"available":96604221440,"internally_reserved":0,"allocated":614400,"data_stored":258785,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:12:21 2023","interfaces":[{"interface":"back","average":{"1min":0.94099999999999995,"5min":1.1890000000000001,"15min":1.23},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.4750000000000001,"5min":3.0030000000000001,"15min":3.0030000000000001},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.99199999999999999,"5min":1.1799999999999999,"15min":1.2110000000000001},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.5369999999999999,"5min":2.0979999999999999,"15min":2.0979999999999999},"last":1.3069999999999999}]},{"osd":1,"last update":"Fri Nov 24 06:12:21 2023","interfaces":[{"interface":"back","average":{"1min":1.1599999999999999,"5min":1.252,"15min":1.268},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.728,"5min":5.0499999999999998,"15min":5.0499999999999998},"last":0.90300000000000002},{"interface":"front","average":{"1min":1.0529999999999999,"5min":1.3939999999999999,"15min":1.4510000000000001},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.5509999999999999,"5min":3.6930000000000001,"15min":3.6930000000000001},"last":1.4910000000000001}]},{"osd":2,"last update":"Fri Nov 24 06:12:21 2023","interfaces":[{"interface":"back","average":{"1min":1.161,"5min":1.3080000000000001,"15min":1.3320000000000001},"min":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"max":{"1min":1.5389999999999999,"5min":3.085,"15min":3.085},"last":1.5700000000000001},{"interface":"front","average":{"1min":1.1879999999999999,"5min":1.53,"15min":1.587},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.8340000000000001,"5min":4.7359999999999998,"15min":4.7359999999999998},"last":1.5389999999999999}]},{"osd":3,"last update":"Fri Nov 24 06:12:21 2023","interfaces":[{"interface":"back","average":{"1min":1.2450000000000001,"5min":1.385,"15min":1.409},"min":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"max":{"1min":1.7629999999999999,"5min":3.2229999999999999,"15min":3.2229999999999999},"last":1.625},{"interface":"front","average":{"1min":1.2929999999999999,"5min":1.4139999999999999,"15min":1.4339999999999999},"min":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"max":{"1min":1.8069999999999999,"5min":2.3260000000000001,"15min":2.3260000000000001},"last":1.4350000000000001}]},{"osd":4,"last update":"Fri Nov 24 06:12:21 2023","interfaces":[{"interface":"back","average":{"1min":1.3759999999999999,"5min":1.528,"15min":1.554},"min":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"max":{"1min":1.756,"5min":3.1230000000000002,"15min":3.1230000000000002},"last":1.387},{"interface":"front","average":{"1min":1.375,"5min":1.522,"15min":1.546},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":2.044,"5min":2.4809999999999999,"15min":2.4809999999999999},"last":1.7509999999999999}]},{"osd":6,"last update":"Fri Nov 24 06:12:21 2023","interfaces":[{"interface":"back","average":{"1min":1.427,"5min":1.6910000000000001,"15min":1.7350000000000001},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.913,"5min":4.6529999999999996,"15min":4.6529999999999996},"last":1.663},{"interface":"front","average":{"1min":1.4890000000000001,"5min":1.6970000000000001,"15min":1.732},"min":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"max":{"1min":2.266,"5min":4.3529999999999998,"15min":4.3529999999999998},"last":1.6870000000000001}]},{"osd":7,"last update":"Fri Nov 24 06:12:21 2023","interfaces":[{"interface":"back","average":{"1min":1.496,"5min":1.6679999999999999,"15min":1.696},"min":{"1min":0.874,"5min":0.874,"15min":0.874},"max":{"1min":1.9830000000000001,"5min":2.8839999999999999,"15min":2.8839999999999999},"last":1.847},{"interface":"front","average":{"1min":1.575,"5min":1.8080000000000001,"15min":1.847},"min":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":2.0699999999999998,"5min":4.8739999999999997,"15min":4.8739999999999997},"last":1.7190000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574878,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31788,"kb_used_data":608,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340052,"statfs":{"total":96636764160,"available":96604213248,"internally_reserved":0,"allocated":622592,"data_stored":260359,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:12:21 2023","interfaces":[{"interface":"back","average":{"1min":1.127,"5min":1.0589999999999999,"15min":1.0469999999999999},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.585,"5min":1.585,"15min":1.585},"last":0.94099999999999995},{"interface":"front","average":{"1min":1.0429999999999999,"5min":1.1819999999999999,"15min":1.2050000000000001},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.744,"5min":1.8220000000000001,"15min":1.8220000000000001},"last":0.77700000000000002}]},{"osd":1,"last update":"Fri Nov 24 06:12:21 2023","interfaces":[{"interface":"back","average":{"1min":1.1499999999999999,"5min":1.1259999999999999,"15min":1.1220000000000001},"min":{"1min":0.73499999999999999,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":1.97,"5min":1.97,"15min":1.97},"last":0.998},{"interface":"front","average":{"1min":1.1819999999999999,"5min":1.1779999999999999,"15min":1.1779999999999999},"min":{"1min":0.79800000000000004,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":1.917,"5min":1.917,"15min":1.917},"last":1.2470000000000001}]},{"osd":2,"last update":"Fri Nov 24 06:12:21 2023","interfaces":[{"interface":"back","average":{"1min":1.0980000000000001,"5min":1.22,"15min":1.24},"min":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":1.6970000000000001,"5min":2.319,"15min":2.319},"last":0.82399999999999995},{"interface":"front","average":{"1min":1.3320000000000001,"5min":1.3129999999999999,"15min":1.3100000000000001},"min":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"max":{"1min":2.077,"5min":2.0990000000000002,"15min":2.0990000000000002},"last":0.88900000000000001}]},{"osd":3,"last update":"Fri Nov 24 06:12:21 2023","interfaces":[{"interface":"back","average":{"1min":1.206,"5min":1.3700000000000001,"15min":1.3979999999999999},"min":{"1min":0.752,"5min":0.752,"15min":0.752},"max":{"1min":2.048,"5min":2.048,"15min":2.048},"last":1.3919999999999999},{"interface":"front","average":{"1min":1.3340000000000001,"5min":1.4199999999999999,"15min":1.4339999999999999},"min":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"max":{"1min":1.8149999999999999,"5min":2.1720000000000002,"15min":2.1720000000000002},"last":1.091}]},{"osd":5,"last update":"Fri Nov 24 06:12:21 2023","interfaces":[{"interface":"back","average":{"1min":1.2829999999999999,"5min":1.327,"15min":1.3340000000000001},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.835,"5min":1.895,"15min":1.895},"last":1.0349999999999999},{"interface":"front","average":{"1min":1.4359999999999999,"5min":1.528,"15min":1.544},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":2.044,"5min":2.5059999999999998,"15min":2.5059999999999998},"last":1.1919999999999999}]},{"osd":6,"last update":"Fri Nov 24 06:12:21 2023","interfaces":[{"interface":"back","average":{"1min":1.423,"5min":1.522,"15min":1.538},"min":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"max":{"1min":2.1509999999999998,"5min":2.2069999999999999,"15min":2.2069999999999999},"last":1.4690000000000001},{"interface":"front","average":{"1min":1.587,"5min":1.665,"15min":1.6779999999999999},"min":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":2.0139999999999998,"5min":2.4510000000000001,"15min":2.4510000000000001},"last":1.3180000000000001}]},{"osd":7,"last update":"Fri Nov 24 06:12:21 2023","interfaces":[{"interface":"back","average":{"1min":1.5640000000000001,"5min":1.6279999999999999,"15min":1.6379999999999999},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":2.214,"5min":2.4350000000000001,"15min":2.4350000000000001},"last":1.522},{"interface":"front","average":{"1min":1.554,"5min":1.617,"15min":1.627},"min":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":2.1499999999999999,"5min":3.1120000000000001,"15min":3.1120000000000001},"last":1.1200000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574878,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27624,"kb_used_data":604,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344216,"statfs":{"total":96636764160,"available":96608477184,"internally_reserved":0,"allocated":618496,"data_stored":258803,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:12:26 2023","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.98599999999999999,"15min":1},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":1.8360000000000001,"5min":2.766,"15min":2.766},"last":0.93500000000000005},{"interface":"front","average":{"1min":1.0800000000000001,"5min":0.94799999999999995,"15min":0.92600000000000005},"min":{"1min":0.44600000000000001,"5min":0.441,"15min":0.441},"max":{"1min":1.873,"5min":1.873,"15min":1.873},"last":0.76900000000000002}]},{"osd":1,"last update":"Fri Nov 24 06:12:26 2023","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.876,"15min":0.88200000000000001},"min":{"1min":0.46999999999999997,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.2569999999999999,"5min":2.5779999999999998,"15min":2.5779999999999998},"last":1.0209999999999999},{"interface":"front","average":{"1min":1.02,"5min":1.151,"15min":1.173},"min":{"1min":0.50700000000000001,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.7350000000000001,"5min":3.6899999999999999,"15min":3.6899999999999999},"last":1.3620000000000001}]},{"osd":2,"last update":"Fri Nov 24 06:12:26 2023","interfaces":[{"interface":"back","average":{"1min":0.996,"5min":1.1399999999999999,"15min":1.1639999999999999},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.637,"5min":4.0620000000000003,"15min":4.0620000000000003},"last":1.4159999999999999},{"interface":"front","average":{"1min":1.113,"5min":1.204,"15min":1.2190000000000001},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.5309999999999999,"5min":2.9529999999999998,"15min":2.9529999999999998},"last":1.659}]},{"osd":4,"last update":"Fri Nov 24 06:12:26 2023","interfaces":[{"interface":"back","average":{"1min":1.26,"5min":1.3939999999999999,"15min":1.4159999999999999},"min":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"max":{"1min":1.6819999999999999,"5min":4.125,"15min":4.125},"last":1.492},{"interface":"front","average":{"1min":1.351,"5min":1.427,"15min":1.4399999999999999},"min":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"max":{"1min":1.7689999999999999,"5min":3.9409999999999998,"15min":3.9409999999999998},"last":1.5600000000000001}]},{"osd":5,"last update":"Fri Nov 24 06:12:26 2023","interfaces":[{"interface":"back","average":{"1min":1.319,"5min":1.421,"15min":1.4379999999999999},"min":{"1min":0.749,"5min":0.749,"15min":0.749},"max":{"1min":1.6679999999999999,"5min":3.3359999999999999,"15min":3.3359999999999999},"last":1.7809999999999999},{"interface":"front","average":{"1min":1.214,"5min":1.4099999999999999,"15min":1.4430000000000001},"min":{"1min":0.86499999999999999,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.542,"5min":4.1669999999999998,"15min":4.1669999999999998},"last":1.2589999999999999}]},{"osd":6,"last update":"Fri Nov 24 06:12:26 2023","interfaces":[{"interface":"back","average":{"1min":1.2230000000000001,"5min":1.3939999999999999,"15min":1.4219999999999999},"min":{"1min":0.871,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":1.571,"5min":4.0190000000000001,"15min":4.0190000000000001},"last":2.169},{"interface":"front","average":{"1min":1.429,"5min":1.512,"15min":1.526},"min":{"1min":0.95999999999999996,"5min":0.92300000000000004,"15min":0.92300000000000004},"max":{"1min":1.966,"5min":3.2480000000000002,"15min":3.2480000000000002},"last":1.895}]},{"osd":7,"last update":"Fri Nov 24 06:12:26 2023","interfaces":[{"interface":"back","average":{"1min":1.3400000000000001,"5min":1.5580000000000001,"15min":1.5940000000000001},"min":{"1min":0.79500000000000004,"5min":0.76600000000000001,"15min":0.76600000000000001},"max":{"1min":1.8049999999999999,"5min":4.202,"15min":4.202},"last":1.831},{"interface":"front","average":{"1min":1.3480000000000001,"5min":1.5669999999999999,"15min":1.603},"min":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"max":{"1min":1.9199999999999999,"5min":3.9849999999999999,"15min":3.9849999999999999},"last":2.2989999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574878,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27620,"kb_used_data":600,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344220,"statfs":{"total":96636764160,"available":96608481280,"internally_reserved":0,"allocated":614400,"data_stored":258189,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:12:22 2023","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.88,"15min":0.88600000000000001},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":1.3149999999999999,"5min":1.6259999999999999,"15min":1.6259999999999999},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.96799999999999997,"5min":1.0740000000000001,"15min":1.0920000000000001},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.619,"5min":1.7669999999999999,"15min":1.7669999999999999},"last":0.51300000000000001}]},{"osd":1,"last update":"Fri Nov 24 06:12:22 2023","interfaces":[{"interface":"back","average":{"1min":0.99399999999999999,"5min":0.94999999999999996,"15min":0.94199999999999995},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.635,"5min":1.635,"15min":1.635},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.95499999999999996,"5min":0.97599999999999998,"15min":0.97999999999999998},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.498,"5min":1.8140000000000001,"15min":1.8140000000000001},"last":0.56999999999999995}]},{"osd":3,"last update":"Fri Nov 24 06:12:22 2023","interfaces":[{"interface":"back","average":{"1min":0.94499999999999995,"5min":1.0529999999999999,"15min":1.071},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.476,"5min":2.3559999999999999,"15min":2.3559999999999999},"last":0.49099999999999999},{"interface":"front","average":{"1min":1.1719999999999999,"5min":1.1319999999999999,"15min":1.1259999999999999},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.643,"5min":2.1840000000000002,"15min":2.1840000000000002},"last":0.47899999999999998}]},{"osd":4,"last update":"Fri Nov 24 06:12:22 2023","interfaces":[{"interface":"back","average":{"1min":1.411,"5min":1.417,"15min":1.4179999999999999},"min":{"1min":0.72099999999999997,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.9350000000000001,"5min":3.411,"15min":3.411},"last":0.73999999999999999},{"interface":"front","average":{"1min":1.333,"5min":1.4710000000000001,"15min":1.494},"min":{"1min":0.88400000000000001,"5min":0.82699999999999996,"15min":0.82699999999999996},"max":{"1min":1.964,"5min":3.7109999999999999,"15min":3.7109999999999999},"last":1.02}]},{"osd":5,"last update":"Fri Nov 24 06:12:22 2023","interfaces":[{"interface":"back","average":{"1min":1.246,"5min":1.3089999999999999,"15min":1.319},"min":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"max":{"1min":1.5620000000000001,"5min":2.5259999999999998,"15min":2.5259999999999998},"last":1.097},{"interface":"front","average":{"1min":1.444,"5min":1.3300000000000001,"15min":1.3109999999999999},"min":{"1min":0.96799999999999997,"5min":0.90400000000000003,"15min":0.90400000000000003},"max":{"1min":1.8839999999999999,"5min":2.1640000000000001,"15min":2.1640000000000001},"last":0.999}]},{"osd":6,"last update":"Fri Nov 24 06:12:22 2023","interfaces":[{"interface":"back","average":{"1min":1.244,"5min":1.4139999999999999,"15min":1.4419999999999999},"min":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"max":{"1min":1.8400000000000001,"5min":2.3039999999999998,"15min":2.3039999999999998},"last":1.155},{"interface":"front","average":{"1min":1.347,"5min":1.444,"15min":1.46},"min":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"max":{"1min":1.6850000000000001,"5min":2.605,"15min":2.605},"last":0.77100000000000002}]},{"osd":7,"last update":"Fri Nov 24 06:12:22 2023","interfaces":[{"interface":"back","average":{"1min":1.365,"5min":1.5489999999999999,"15min":1.579},"min":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"max":{"1min":1.9079999999999999,"5min":2.3740000000000001,"15min":2.3740000000000001},"last":0.97299999999999998},{"interface":"front","average":{"1min":1.3009999999999999,"5min":1.4330000000000001,"15min":1.4550000000000001},"min":{"1min":0.877,"5min":0.877,"15min":0.877},"max":{"1min":1.575,"5min":2.5699999999999998,"15min":2.5699999999999998},"last":0.89200000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574877,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27628,"kb_used_data":600,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344212,"statfs":{"total":96636764160,"available":96608473088,"internally_reserved":0,"allocated":614400,"data_stored":258837,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:12:21 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.878,"15min":0.90800000000000003},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":1.3740000000000001,"5min":1.9299999999999999,"15min":1.9299999999999999},"last":0.93200000000000005},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.90700000000000003,"15min":0.92500000000000004},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":1.258,"5min":1.7669999999999999,"15min":1.7669999999999999},"last":0.79600000000000004}]},{"osd":2,"last update":"Fri Nov 24 06:12:21 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.96499999999999997,"15min":1.0009999999999999},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.0840000000000001,"5min":2.5329999999999999,"15min":2.5329999999999999},"last":1.2889999999999999},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.99099999999999999,"15min":1.0169999999999999},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.159,"5min":1.657,"15min":1.657},"last":1.0549999999999999}]},{"osd":3,"last update":"Fri Nov 24 06:12:21 2023","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.98899999999999999,"15min":1.0209999999999999},"min":{"1min":0.49399999999999999,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.2729999999999999,"5min":1.841,"15min":1.841},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.98399999999999999,"5min":1.121,"15min":1.1439999999999999},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.5249999999999999,"5min":2.8940000000000001,"15min":2.8940000000000001},"last":1.1850000000000001}]},{"osd":4,"last update":"Fri Nov 24 06:12:21 2023","interfaces":[{"interface":"back","average":{"1min":1.145,"5min":1.254,"15min":1.272},"min":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":1.454,"5min":2.1930000000000001,"15min":2.1930000000000001},"last":1.25},{"interface":"front","average":{"1min":1.1200000000000001,"5min":1.1839999999999999,"15min":1.194},"min":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":1.569,"5min":1.784,"15min":1.784},"last":1.105}]},{"osd":5,"last update":"Fri Nov 24 06:12:21 2023","interfaces":[{"interface":"back","average":{"1min":1.1499999999999999,"5min":1.454,"15min":1.504},"min":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":1.5820000000000001,"5min":5.5449999999999999,"15min":5.5449999999999999},"last":1.4550000000000001},{"interface":"front","average":{"1min":1.167,"5min":1.484,"15min":1.5369999999999999},"min":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":1.538,"5min":6.0030000000000001,"15min":6.0030000000000001},"last":1.153}]},{"osd":6,"last update":"Fri Nov 24 06:12:21 2023","interfaces":[{"interface":"back","average":{"1min":1.2030000000000001,"5min":1.4950000000000001,"15min":1.544},"min":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":1.5189999999999999,"5min":5.2809999999999997,"15min":5.2809999999999997},"last":1.405},{"interface":"front","average":{"1min":1.2849999999999999,"5min":1.4370000000000001,"15min":1.462},"min":{"1min":0.79300000000000004,"5min":0.749,"15min":0.749},"max":{"1min":1.7969999999999999,"5min":4.327,"15min":4.327},"last":1.379}]},{"osd":7,"last update":"Fri Nov 24 06:12:21 2023","interfaces":[{"interface":"back","average":{"1min":1.173,"5min":1.321,"15min":1.3460000000000001},"min":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":1.8540000000000001,"5min":2.3069999999999999,"15min":2.3069999999999999},"last":1.026},{"interface":"front","average":{"1min":1.252,"5min":1.528,"15min":1.5740000000000001},"min":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":1.6819999999999999,"5min":5.4500000000000002,"15min":5.4500000000000002},"last":1.341}]}]},{"osd":0,"up_from":13,"seq":55834574877,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28216,"kb_used_data":1192,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343624,"statfs":{"total":96636764160,"available":96607870976,"internally_reserved":0,"allocated":1220608,"data_stored":851309,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Nov 24 06:12:20 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.90500000000000003,"15min":0.92700000000000005},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":1.675,"5min":2.3999999999999999,"15min":2.3999999999999999},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.79900000000000004,"5min":1.1100000000000001,"15min":1.1619999999999999},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.403,"5min":4.8289999999999997,"15min":4.8289999999999997},"last":0.85899999999999999}]},{"osd":2,"last update":"Fri Nov 24 06:12:20 2023","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":1.0249999999999999,"15min":1.05},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.5960000000000001,"5min":2.5139999999999998,"15min":2.5139999999999998},"last":0.96499999999999997},{"interface":"front","average":{"1min":0.98199999999999998,"5min":1.0860000000000001,"15min":1.103},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.4390000000000001,"5min":2.3599999999999999,"15min":2.3599999999999999},"last":0.58199999999999996}]},{"osd":3,"last update":"Fri Nov 24 06:12:20 2023","interfaces":[{"interface":"back","average":{"1min":1.018,"5min":1.1040000000000001,"15min":1.1180000000000001},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.514,"5min":2.5840000000000001,"15min":2.5840000000000001},"last":0.66000000000000003},{"interface":"front","average":{"1min":1.071,"5min":1.3460000000000001,"15min":1.3919999999999999},"min":{"1min":0.503,"5min":0.495,"15min":0.495},"max":{"1min":1.526,"5min":5.1230000000000002,"15min":5.1230000000000002},"last":0.63700000000000001}]},{"osd":4,"last update":"Fri Nov 24 06:12:20 2023","interfaces":[{"interface":"back","average":{"1min":1.0209999999999999,"5min":1.23,"15min":1.2649999999999999},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.3919999999999999,"5min":2.887,"15min":2.887},"last":0.90100000000000002},{"interface":"front","average":{"1min":1.0760000000000001,"5min":1.284,"15min":1.3180000000000001},"min":{"1min":0.71599999999999997,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":1.6819999999999999,"5min":2.8530000000000002,"15min":2.8530000000000002},"last":0.79600000000000004}]},{"osd":5,"last update":"Fri Nov 24 06:12:20 2023","interfaces":[{"interface":"back","average":{"1min":1.2370000000000001,"5min":1.5069999999999999,"15min":1.552},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.804,"5min":4.6299999999999999,"15min":4.6299999999999999},"last":0.88200000000000001},{"interface":"front","average":{"1min":1.056,"5min":1.2769999999999999,"15min":1.3140000000000001},"min":{"1min":0.72699999999999998,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.3560000000000001,"5min":2.794,"15min":2.794},"last":0.995}]},{"osd":6,"last update":"Fri Nov 24 06:12:20 2023","interfaces":[{"interface":"back","average":{"1min":1.2829999999999999,"5min":1.5940000000000001,"15min":1.6459999999999999},"min":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.6779999999999999,"5min":4.798,"15min":4.798},"last":1.1060000000000001},{"interface":"front","average":{"1min":1.375,"5min":1.5840000000000001,"15min":1.619},"min":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"max":{"1min":1.9350000000000001,"5min":4.7149999999999999,"15min":4.7149999999999999},"last":1.087}]},{"osd":7,"last update":"Fri Nov 24 06:12:20 2023","interfaces":[{"interface":"back","average":{"1min":1.3109999999999999,"5min":1.635,"15min":1.6890000000000001},"min":{"1min":0.73799999999999999,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.9750000000000001,"5min":4.7649999999999997,"15min":4.7649999999999997},"last":1.0680000000000001},{"interface":"front","average":{"1min":1.421,"5min":1.6399999999999999,"15min":1.6759999999999999},"min":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"max":{"1min":2.0430000000000001,"5min":4.6820000000000004,"15min":4.6820000000000004},"last":1.024}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-24T06:12:36.454 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-11-24T06:12:36.455 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-11-24T06:12:36.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:36.720+0000 7f3486361640 1 Processor -- start 2023-11-24T06:12:36.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:36.720+0000 7f3486361640 1 -- start start 2023-11-24T06:12:36.724 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:36.721+0000 7f3486361640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f348015e1a0 0x7f348015c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:36.724 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:36.721+0000 7f3486361640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f348015ba40 con 0x7f348015dd40 2023-11-24T06:12:36.724 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:36.722+0000 7f3486361640 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f348015eb10 con 0x7f3480157dc0 2023-11-24T06:12:36.724 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:36.722+0000 7f3486361640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34801581a0 con 0x7f348015e1a0 2023-11-24T06:12:36.724 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:36.722+0000 7f347f7fe640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f348015dd40 0x7f348015bed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:49896/0 (socket says 172.21.15.92:49896) 2023-11-24T06:12:36.724 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:36.722+0000 7f347f7fe640 1 -- 172.21.15.92:0/2383587967 learned_addr learned my addr 172.21.15.92:0/2383587967 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-24T06:12:36.726 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:36.722+0000 7f347effd640 1 -- 172.21.15.92:0/2383587967 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2226651711 0 0) 0x7f348015ba40 con 0x7f348015dd40 2023-11-24T06:12:36.726 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:36.722+0000 7f347effd640 1 -- 172.21.15.92:0/2383587967 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f346c005730 con 0x7f348015dd40 2023-11-24T06:12:36.726 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:36.722+0000 7f34848fe640 1 --2- 172.21.15.92:0/2383587967 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f348015e1a0 0x7f348015c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:36.726 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:36.722+0000 7f347effd640 1 -- 172.21.15.92:0/2383587967 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 98818079 0 0) 0x7f346c005730 con 0x7f348015dd40 2023-11-24T06:12:36.726 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:36.723+0000 7f347effd640 1 -- 172.21.15.92:0/2383587967 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f348015e1a0 msgr2=0x7f348015c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:36.726 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:36.723+0000 7f347effd640 1 --2- 172.21.15.92:0/2383587967 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f348015e1a0 0x7f348015c5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:36.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:36.723+0000 7f347effd640 1 -- 172.21.15.92:0/2383587967 >> v1:172.21.15.118:6789/0 conn(0x7f3480157dc0 legacy=0x7f3480070cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:36.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:36.723+0000 7f347effd640 1 -- 172.21.15.92:0/2383587967 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3480071860 con 0x7f348015dd40 2023-11-24T06:12:36.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:36.723+0000 7f347effd640 1 -- 172.21.15.92:0/2383587967 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2607661350 0 0) 0x7f3468001670 con 0x7f348015dd40 2023-11-24T06:12:36.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:36.723+0000 7f347effd640 1 -- 172.21.15.92:0/2383587967 >> v1:172.21.15.92:6789/0 conn(0x7f348015dd40 legacy=0x7f348015bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:36.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:36.723+0000 7f347effd640 1 --2- 172.21.15.92:0/2383587967 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f348015e1a0 0x7f346c005bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:36.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:36.723+0000 7f347effd640 1 --2- 172.21.15.92:0/2383587967 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f346c006230 0x7f346c0069d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:36.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:36.723+0000 7f347effd640 1 --2- 172.21.15.92:0/2383587967 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f346c006f10 0x7f346c007390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:36.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:36.723+0000 7f347effd640 1 -- 172.21.15.92:0/2383587967 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34801581a0 con 0x7f346c006f10 2023-11-24T06:12:36.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:36.723+0000 7f347effd640 1 -- 172.21.15.92:0/2383587967 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f346c00ddd0 con 0x7f346c006230 2023-11-24T06:12:36.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:36.723+0000 7f347effd640 1 -- 172.21.15.92:0/2383587967 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f346c00df90 con 0x7f348015e1a0 2023-11-24T06:12:36.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:36.723+0000 7f34848fe640 1 --2- 172.21.15.92:0/2383587967 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f346c006f10 0x7f346c007390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:36.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:36.723+0000 7f347ffff640 1 --2- 172.21.15.92:0/2383587967 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f348015e1a0 0x7f346c005bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:36.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:36.723+0000 7f347f7fe640 1 --2- 172.21.15.92:0/2383587967 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f346c006230 0x7f346c0069d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:36.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:36.723+0000 7f34848fe640 1 -- 172.21.15.92:0/2383587967 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f348015e1a0 msgr2=0x7f346c005bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:36.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:36.723+0000 7f34848fe640 1 --2- 172.21.15.92:0/2383587967 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f348015e1a0 0x7f346c005bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:36.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:36.723+0000 7f34848fe640 1 -- 172.21.15.92:0/2383587967 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f346c006230 msgr2=0x7f346c0069d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:36.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:36.723+0000 7f34848fe640 1 --2- 172.21.15.92:0/2383587967 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f346c006230 0x7f346c0069d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:36.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:36.724+0000 7f34848fe640 1 -- 172.21.15.92:0/2383587967 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f346c005730 con 0x7f346c006f10 2023-11-24T06:12:36.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:36.725+0000 7f34848fe640 1 --2- 172.21.15.92:0/2383587967 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f346c006f10 0x7f346c007390 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f347800ba30 tx=0x7f346c007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c307539f9fec2325 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:36.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:36.725+0000 7f347effd640 1 -- 172.21.15.92:0/2383587967 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3478032640 con 0x7f346c006f10 2023-11-24T06:12:36.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:36.725+0000 7f347effd640 1 -- 172.21.15.92:0/2383587967 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34780327a0 con 0x7f346c006f10 2023-11-24T06:12:36.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:36.725+0000 7f347effd640 1 -- 172.21.15.92:0/2383587967 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3478032b30 con 0x7f346c006f10 2023-11-24T06:12:36.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:36.725+0000 7f3486361640 1 -- 172.21.15.92:0/2383587967 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f346c006f10 msgr2=0x7f346c007390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:36.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:36.725+0000 7f3486361640 1 --2- 172.21.15.92:0/2383587967 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f346c006f10 0x7f346c007390 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f347800ba30 tx=0x7f346c007a00 comp rx=0 tx=0).stop 2023-11-24T06:12:36.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:36.725+0000 7f34848fe640 1 -- 172.21.15.92:0/2383587967 reap_dead start 2023-11-24T06:12:36.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:36.725+0000 7f3486361640 1 -- 172.21.15.92:0/2383587967 shutdown_connections 2023-11-24T06:12:36.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:36.726+0000 7f3486361640 1 -- 172.21.15.92:0/2383587967 >> 172.21.15.92:0/2383587967 conn(0x7f34800b0be0 msgr2=0x7f34800b0fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:36.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:36.726+0000 7f3486361640 1 -- 172.21.15.92:0/2383587967 shutdown_connections 2023-11-24T06:12:36.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:36.726+0000 7f3486361640 1 -- 172.21.15.92:0/2383587967 wait complete. 2023-11-24T06:12:36.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:36.727+0000 7f3486361640 1 Processor -- start 2023-11-24T06:12:36.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:36.728+0000 7f3486361640 1 -- start start 2023-11-24T06:12:36.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:36.728+0000 7f3486361640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f346c006f10 0x7f3480145a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:36.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:36.728+0000 7f3486361640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f34800cc820 0x7f34800acd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:36.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:36.729+0000 7f3486361640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f348015c590 0x7f348015ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:36.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:36.729+0000 7f3486361640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34800ae300 con 0x7f348015c590 2023-11-24T06:12:36.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:36.729+0000 7f3486361640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f348015cf80 con 0x7f346c006f10 2023-11-24T06:12:36.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:36.729+0000 7f3486361640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34800ad680 con 0x7f34800cc820 2023-11-24T06:12:36.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:36.729+0000 7f34848fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f348015c590 0x7f348015ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:36.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:36.729+0000 7f34848fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f348015c590 0x7f348015ca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:49992/0 (socket says 172.21.15.92:49992) 2023-11-24T06:12:36.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:36.729+0000 7f34848fe640 1 -- 172.21.15.92:0/588471911 learned_addr learned my addr 172.21.15.92:0/588471911 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:12:36.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:36.729+0000 7f347f7fe640 1 --2- 172.21.15.92:0/588471911 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f34800cc820 0x7f34800acd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:36.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:36.729+0000 7f347ffff640 1 --2- 172.21.15.92:0/588471911 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f346c006f10 0x7f3480145a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:36.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:36.729+0000 7f34848fe640 1 -- 172.21.15.92:0/588471911 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f34800cc820 msgr2=0x7f34800acd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:36.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:36.729+0000 7f34848fe640 1 --2- 172.21.15.92:0/588471911 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f34800cc820 0x7f34800acd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:36.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:36.729+0000 7f34848fe640 1 -- 172.21.15.92:0/588471911 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f346c006f10 msgr2=0x7f3480145a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:36.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:36.729+0000 7f34848fe640 1 --2- 172.21.15.92:0/588471911 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f346c006f10 0x7f3480145a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:36.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:36.729+0000 7f34848fe640 1 -- 172.21.15.92:0/588471911 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34800ad960 con 0x7f348015c590 2023-11-24T06:12:36.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:36.730+0000 7f34848fe640 1 --2- 172.21.15.92:0/588471911 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f348015c590 0x7f348015ca40 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f3478035040 tx=0x7f3478004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:36.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:36.730+0000 7f347cff9640 1 -- 172.21.15.92:0/588471911 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3478004220 con 0x7f348015c590 2023-11-24T06:12:36.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:36.730+0000 7f347cff9640 1 -- 172.21.15.92:0/588471911 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3478040070 con 0x7f348015c590 2023-11-24T06:12:36.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:36.730+0000 7f347cff9640 1 -- 172.21.15.92:0/588471911 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f347803c700 con 0x7f348015c590 2023-11-24T06:12:36.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:36.730+0000 7f3486361640 1 -- 172.21.15.92:0/588471911 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34800adc40 con 0x7f348015c590 2023-11-24T06:12:36.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:36.730+0000 7f3486361640 1 -- 172.21.15.92:0/588471911 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3480145f40 con 0x7f348015c590 2023-11-24T06:12:36.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:36.731+0000 7f3486361640 1 -- 172.21.15.92:0/588471911 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34800ae440 con 0x7f348015c590 2023-11-24T06:12:36.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:36.732+0000 7f347cff9640 1 -- 172.21.15.92:0/588471911 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f347803c860 con 0x7f348015c590 2023-11-24T06:12:36.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:36.733+0000 7f347cff9640 1 --2- 172.21.15.92:0/588471911 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f346006ae40 0x7f346006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:36.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:36.734+0000 7f347cff9640 1 -- 172.21.15.92:0/588471911 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3478042070 con 0x7f348015c590 2023-11-24T06:12:36.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:36.734+0000 7f347ffff640 1 --2- 172.21.15.92:0/588471911 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f346006ae40 0x7f346006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:36.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:36.734+0000 7f347ffff640 1 --2- 172.21.15.92:0/588471911 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f346006ae40 0x7f346006d300 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f3470001c60 tx=0x7f3470009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:36.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:36.738+0000 7f347cff9640 1 -- 172.21.15.92:0/588471911 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f34800ae440 con 0x7f348015c590 2023-11-24T06:12:36.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:36.991+0000 7f3486361640 1 -- 172.21.15.92:0/588471911 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f34800ae440 con 0x7f348015c590 2023-11-24T06:12:36.999 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:36.996+0000 7f347cff9640 1 -- 172.21.15.92:0/588471911 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v10) v1 ==== 190+0+147 (secure 0 0 0) 0x7f34800ae440 con 0x7f348015c590 2023-11-24T06:12:36.999 INFO:teuthology.orchestra.run.smithi092.stdout:[client.0] 2023-11-24T06:12:36.999 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQARPmBlGMUsDhAAlgrJvA/TT5hpUYd5o9OzIQ== 2023-11-24T06:12:36.999 INFO:teuthology.orchestra.run.smithi092.stdout: caps mds = "allow" 2023-11-24T06:12:36.999 INFO:teuthology.orchestra.run.smithi092.stdout: caps mon = "allow r" 2023-11-24T06:12:36.999 INFO:teuthology.orchestra.run.smithi092.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-11-24T06:12:36.999 INFO:teuthology.orchestra.run.smithi092.stderr:updated caps for client.0 2023-11-24T06:12:37.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:36.999+0000 7f3486361640 1 -- 172.21.15.92:0/588471911 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f346006ae40 msgr2=0x7f346006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:37.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:36.999+0000 7f3486361640 1 --2- 172.21.15.92:0/588471911 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f346006ae40 0x7f346006d300 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f3470001c60 tx=0x7f3470009040 comp rx=0 tx=0).stop 2023-11-24T06:12:37.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:36.999+0000 7f3486361640 1 -- 172.21.15.92:0/588471911 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f348015c590 msgr2=0x7f348015ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:37.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:36.999+0000 7f3486361640 1 --2- 172.21.15.92:0/588471911 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f348015c590 0x7f348015ca40 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f3478035040 tx=0x7f3478004730 comp rx=0 tx=0).stop 2023-11-24T06:12:37.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:36.999+0000 7f3486361640 1 -- 172.21.15.92:0/588471911 shutdown_connections 2023-11-24T06:12:37.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:36.999+0000 7f3486361640 1 --2- 172.21.15.92:0/588471911 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f346006ae40 0x7f346006d300 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:37.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:36.999+0000 7f3486361640 1 --2- 172.21.15.92:0/588471911 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f348015c590 0x7f348015ca40 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:37.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:36.999+0000 7f3486361640 1 --2- 172.21.15.92:0/588471911 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f34800cc820 0x7f34800acd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:37.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:36.999+0000 7f3486361640 1 --2- 172.21.15.92:0/588471911 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f346c006f10 0x7f3480145a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:37.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:36.999+0000 7f3486361640 1 -- 172.21.15.92:0/588471911 >> 172.21.15.92:0/588471911 conn(0x7f34800b0be0 msgr2=0x7f348006cbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:37.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:37.000+0000 7f3486361640 1 -- 172.21.15.92:0/588471911 shutdown_connections 2023-11-24T06:12:37.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:37.000+0000 7f3486361640 1 -- 172.21.15.92:0/588471911 wait complete. 2023-11-24T06:12:37.019 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T06:12:37.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:37.225+0000 7ff7eb5a9640 1 Processor -- start 2023-11-24T06:12:37.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:37.225+0000 7ff7eb5a9640 1 -- start start 2023-11-24T06:12:37.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:37.227+0000 7ff7eb5a9640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ff7e4143ae0 0x7ff7e40cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:37.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:37.227+0000 7ff7eb5a9640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7ff7e40cf7d0 0x7ff7e415dca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:37.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:37.227+0000 7ff7eb5a9640 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7e40cfc30 con 0x7ff7e415e1e0 2023-11-24T06:12:37.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:37.227+0000 7ff7eb5a9640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7e40b1ee0 con 0x7ff7e4143ae0 2023-11-24T06:12:37.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:37.227+0000 7ff7eb5a9640 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7e40b2780 con 0x7ff7e40cf7d0 2023-11-24T06:12:37.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:37.227+0000 7ff7e9345640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ff7e4143ae0 0x7ff7e40cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:37.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:37.227+0000 7ff7e9345640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ff7e4143ae0 0x7ff7e40cf290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:49998/0 (socket says 172.21.15.92:49998) 2023-11-24T06:12:37.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:37.227+0000 7ff7e9345640 1 -- 172.21.15.92:0/2488032674 learned_addr learned my addr 172.21.15.92:0/2488032674 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:12:37.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:37.227+0000 7ff7e8b44640 1 --2- 172.21.15.92:0/2488032674 >> v2:172.21.15.118:3300/0 conn(0x7ff7e40cf7d0 0x7ff7e415dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:37.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:37.228+0000 7ff7e9345640 1 -- 172.21.15.92:0/2488032674 >> v2:172.21.15.118:3300/0 conn(0x7ff7e40cf7d0 msgr2=0x7ff7e415dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:37.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:37.228+0000 7ff7e9345640 1 --2- 172.21.15.92:0/2488032674 >> v2:172.21.15.118:3300/0 conn(0x7ff7e40cf7d0 0x7ff7e415dca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:37.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:37.228+0000 7ff7e9345640 1 -- 172.21.15.92:0/2488032674 >> v1:172.21.15.118:6789/0 conn(0x7ff7e415e1e0 legacy=0x7ff7e415a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:37.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:37.228+0000 7ff7e9345640 1 -- 172.21.15.92:0/2488032674 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7e415e990 con 0x7ff7e4143ae0 2023-11-24T06:12:37.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:37.229+0000 7ff7e9345640 1 --2- 172.21.15.92:0/2488032674 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff7e4143ae0 0x7ff7e40cf290 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7ff7e40cb830 tx=0x7ff7e002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5792a2c359ae7770 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:37.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:37.229+0000 7ff7dbfff640 1 -- 172.21.15.92:0/2488032674 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7e0031870 con 0x7ff7e4143ae0 2023-11-24T06:12:37.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:37.230+0000 7ff7dbfff640 1 -- 172.21.15.92:0/2488032674 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7e00319d0 con 0x7ff7e4143ae0 2023-11-24T06:12:37.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:37.230+0000 7ff7dbfff640 1 -- 172.21.15.92:0/2488032674 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7e0031d60 con 0x7ff7e4143ae0 2023-11-24T06:12:37.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:37.230+0000 7ff7eb5a9640 1 -- 172.21.15.92:0/2488032674 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff7e4143ae0 msgr2=0x7ff7e40cf290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:37.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:37.230+0000 7ff7eb5a9640 1 --2- 172.21.15.92:0/2488032674 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff7e4143ae0 0x7ff7e40cf290 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7ff7e40cb830 tx=0x7ff7e002b910 comp rx=0 tx=0).stop 2023-11-24T06:12:37.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:37.230+0000 7ff7eb5a9640 1 -- 172.21.15.92:0/2488032674 shutdown_connections 2023-11-24T06:12:37.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:37.230+0000 7ff7eb5a9640 1 --2- 172.21.15.92:0/2488032674 >> v2:172.21.15.118:3300/0 conn(0x7ff7e40cf7d0 0x7ff7e415dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:37.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:37.230+0000 7ff7eb5a9640 1 --2- 172.21.15.92:0/2488032674 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff7e4143ae0 0x7ff7e40cf290 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:37.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:37.230+0000 7ff7eb5a9640 1 -- 172.21.15.92:0/2488032674 >> 172.21.15.92:0/2488032674 conn(0x7ff7e406a800 msgr2=0x7ff7e406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:37.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:37.230+0000 7ff7eb5a9640 1 -- 172.21.15.92:0/2488032674 shutdown_connections 2023-11-24T06:12:37.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:37.230+0000 7ff7eb5a9640 1 -- 172.21.15.92:0/2488032674 wait complete. 2023-11-24T06:12:37.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:37.233+0000 7ff7eb5a9640 1 Processor -- start 2023-11-24T06:12:37.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:37.233+0000 7ff7eb5a9640 1 -- start start 2023-11-24T06:12:37.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:37.234+0000 7ff7eb5a9640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff7e40cf7d0 0x7ff7e4159970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:37.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:37.234+0000 7ff7eb5a9640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff7e4143ae0 0x7ff7e4139a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:37.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:37.234+0000 7ff7eb5a9640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff7e415e1e0 0x7ff7e40cc610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:37.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:37.234+0000 7ff7eb5a9640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7e4159eb0 con 0x7ff7e40cf7d0 2023-11-24T06:12:37.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:37.234+0000 7ff7eb5a9640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7e413cdd0 con 0x7ff7e4143ae0 2023-11-24T06:12:37.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:37.234+0000 7ff7eb5a9640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7e40aca00 con 0x7ff7e415e1e0 2023-11-24T06:12:37.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:37.234+0000 7ff7e9345640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff7e40cf7d0 0x7ff7e4159970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:37.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:37.234+0000 7ff7e9345640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff7e40cf7d0 0x7ff7e4159970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50014/0 (socket says 172.21.15.92:50014) 2023-11-24T06:12:37.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:37.234+0000 7ff7e9345640 1 -- 172.21.15.92:0/3393013287 learned_addr learned my addr 172.21.15.92:0/3393013287 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:12:37.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:37.235+0000 7ff7e9345640 1 -- 172.21.15.92:0/3393013287 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff7e415e1e0 msgr2=0x7ff7e40cc610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:37.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:37.235+0000 7ff7e9345640 1 --2- 172.21.15.92:0/3393013287 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff7e415e1e0 0x7ff7e40cc610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:37.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:37.235+0000 7ff7e9345640 1 -- 172.21.15.92:0/3393013287 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff7e4143ae0 msgr2=0x7ff7e4139a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:37.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:37.235+0000 7ff7e9345640 1 --2- 172.21.15.92:0/3393013287 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff7e4143ae0 0x7ff7e4139a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:37.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:37.235+0000 7ff7e9345640 1 -- 172.21.15.92:0/3393013287 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7e00094d0 con 0x7ff7e40cf7d0 2023-11-24T06:12:37.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:37.235+0000 7ff7e9345640 1 --2- 172.21.15.92:0/3393013287 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff7e40cf7d0 0x7ff7e4159970 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7ff7e002b950 tx=0x7ff7e00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:37.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:37.235+0000 7ff7d9ffb640 1 -- 172.21.15.92:0/3393013287 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7e003ad30 con 0x7ff7e40cf7d0 2023-11-24T06:12:37.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:37.235+0000 7ff7d9ffb640 1 -- 172.21.15.92:0/3393013287 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7e00443f0 con 0x7ff7e40cf7d0 2023-11-24T06:12:37.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:37.235+0000 7ff7d9ffb640 1 -- 172.21.15.92:0/3393013287 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7e00447a0 con 0x7ff7e40cf7d0 2023-11-24T06:12:37.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:37.235+0000 7ff7eb5a9640 1 -- 172.21.15.92:0/3393013287 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7e4139fa0 con 0x7ff7e40cf7d0 2023-11-24T06:12:37.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:37.235+0000 7ff7eb5a9640 1 -- 172.21.15.92:0/3393013287 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7e413c810 con 0x7ff7e40cf7d0 2023-11-24T06:12:37.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:37.237+0000 7ff7eb5a9640 1 -- 172.21.15.92:0/3393013287 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7e40675d0 con 0x7ff7e40cf7d0 2023-11-24T06:12:37.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:37.237+0000 7ff7d9ffb640 1 -- 172.21.15.92:0/3393013287 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ff7e004d450 con 0x7ff7e40cf7d0 2023-11-24T06:12:37.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:37.238+0000 7ff7d9ffb640 1 --2- 172.21.15.92:0/3393013287 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7ff7c806ae40 0x7ff7c806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:37.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:37.238+0000 7ff7d9ffb640 1 -- 172.21.15.92:0/3393013287 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff7e002d040 con 0x7ff7e40cf7d0 2023-11-24T06:12:37.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:37.238+0000 7ff7e8b44640 1 --2- 172.21.15.92:0/3393013287 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7ff7c806ae40 0x7ff7c806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:37.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:37.239+0000 7ff7e8b44640 1 --2- 172.21.15.92:0/3393013287 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7ff7c806ae40 0x7ff7c806d300 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7ff7d4006fd0 tx=0x7ff7d4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:37.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:37.243+0000 7ff7d9ffb640 1 -- 172.21.15.92:0/3393013287 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff7e0076c80 con 0x7ff7e40cf7d0 2023-11-24T06:12:37.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:37.467+0000 7ff7eb5a9640 1 -- 172.21.15.92:0/3393013287 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff7e40ccb50 con 0x7ff7e40cf7d0 2023-11-24T06:12:37.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:37.468+0000 7ff7d9ffb640 1 -- 172.21.15.92:0/3393013287 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) v1 ==== 94+0+4919 (secure 0 0 0) 0x7ff7e0035020 con 0x7ff7e40cf7d0 2023-11-24T06:12:37.471 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-24T06:12:37.471 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":31,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4778,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6835/1184353034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":1184353034},{"type":"v1","addr":"172.21.15.118:6835","nonce":1184353034}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":26},{"gid":4942,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6833/1256969666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":1256969666},{"type":"v1","addr":"172.21.15.118:6833","nonce":1256969666}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":26},{"gid":5706,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6836/3849408561","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":3849408561},{"type":"v1","addr":"172.21.15.92:6836","nonce":3849408561}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":31,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T06:12:30.605447+0000","modified":"2023-11-24T06:12:34.015371+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5709},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5709":{"gid":5709,"name":"c","rank":0,"incarnation":27,"state":"up:active","state_seq":3,"addr":"172.21.15.92:6837/1958912045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":1958912045},{"type":"v1","addr":"172.21.15.92:6837","nonce":1958912045}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-24T06:12:37.471 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 31 2023-11-24T06:12:37.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:37.471+0000 7ff7eb5a9640 1 -- 172.21.15.92:0/3393013287 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7ff7c806ae40 msgr2=0x7ff7c806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:37.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:37.471+0000 7ff7eb5a9640 1 --2- 172.21.15.92:0/3393013287 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7ff7c806ae40 0x7ff7c806d300 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7ff7d4006fd0 tx=0x7ff7d4008040 comp rx=0 tx=0).stop 2023-11-24T06:12:37.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:37.471+0000 7ff7eb5a9640 1 -- 172.21.15.92:0/3393013287 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff7e40cf7d0 msgr2=0x7ff7e4159970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:37.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:37.471+0000 7ff7eb5a9640 1 --2- 172.21.15.92:0/3393013287 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff7e40cf7d0 0x7ff7e4159970 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7ff7e002b950 tx=0x7ff7e00046c0 comp rx=0 tx=0).stop 2023-11-24T06:12:37.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:37.471+0000 7ff7eb5a9640 1 -- 172.21.15.92:0/3393013287 shutdown_connections 2023-11-24T06:12:37.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:37.471+0000 7ff7eb5a9640 1 --2- 172.21.15.92:0/3393013287 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7ff7c806ae40 0x7ff7c806d300 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:37.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:37.471+0000 7ff7eb5a9640 1 --2- 172.21.15.92:0/3393013287 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff7e415e1e0 0x7ff7e40cc610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:37.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:37.471+0000 7ff7eb5a9640 1 --2- 172.21.15.92:0/3393013287 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff7e4143ae0 0x7ff7e4139a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:37.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:37.471+0000 7ff7eb5a9640 1 --2- 172.21.15.92:0/3393013287 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff7e40cf7d0 0x7ff7e4159970 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:37.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:37.472+0000 7ff7eb5a9640 1 -- 172.21.15.92:0/3393013287 >> 172.21.15.92:0/3393013287 conn(0x7ff7e406a800 msgr2=0x7ff7e415b7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:37.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:37.472+0000 7ff7eb5a9640 1 -- 172.21.15.92:0/3393013287 shutdown_connections 2023-11-24T06:12:37.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:37.472+0000 7ff7eb5a9640 1 -- 172.21.15.92:0/3393013287 wait complete. 2023-11-24T06:12:37.894 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 31, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-24T06:12:30.605447+0000', 'modified': '2023-11-24T06:12:34.015371+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5709}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5709': {'gid': 5709, 'name': 'c', 'rank': 0, 'incarnation': 27, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.92:6837/1958912045', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6835', 'nonce': 1958912045}, {'type': 'v1', 'addr': '172.21.15.92:6837', 'nonce': 1958912045}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [9], 'metadata_pool': 8, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-24T06:12:37.894 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-24T06:12:37.894 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-11-24T06:12:38.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.112+0000 7fd9da97c640 1 Processor -- start 2023-11-24T06:12:38.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.112+0000 7fd9da97c640 1 -- start start 2023-11-24T06:12:38.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.114+0000 7fd9da97c640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fd9d415afb0 0x7fd9d413f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:38.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.114+0000 7fd9da97c640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fd9d413fae0 0x7fd9d40b2040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:38.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.114+0000 7fd9da97c640 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9d413ff30 con 0x7fd9d413cfa0 2023-11-24T06:12:38.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.114+0000 7fd9da97c640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9d40cdc40 con 0x7fd9d415afb0 2023-11-24T06:12:38.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.114+0000 7fd9da97c640 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9d40cdec0 con 0x7fd9d413fae0 2023-11-24T06:12:38.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.114+0000 7fd9d9179640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fd9d415afb0 0x7fd9d413f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:38.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.114+0000 7fd9d9179640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fd9d415afb0 0x7fd9d413f5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50016/0 (socket says 172.21.15.92:50016) 2023-11-24T06:12:38.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.114+0000 7fd9d9179640 1 -- 172.21.15.92:0/3915683235 learned_addr learned my addr 172.21.15.92:0/3915683235 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:12:38.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.114+0000 7fd9d9179640 1 -- 172.21.15.92:0/3915683235 >> v2:172.21.15.118:3300/0 conn(0x7fd9d413fae0 msgr2=0x7fd9d40b2040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:38.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.114+0000 7fd9da17b640 1 --2- 172.21.15.92:0/3915683235 >> v2:172.21.15.118:3300/0 conn(0x7fd9d413fae0 0x7fd9d40b2040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:38.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.115+0000 7fd9d9179640 1 --2- 172.21.15.92:0/3915683235 >> v2:172.21.15.118:3300/0 conn(0x7fd9d413fae0 0x7fd9d40b2040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:38.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.115+0000 7fd9d9179640 1 -- 172.21.15.92:0/3915683235 >> v1:172.21.15.118:6789/0 conn(0x7fd9d413cfa0 legacy=0x7fd9d40cd530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:38.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.115+0000 7fd9d9179640 1 -- 172.21.15.92:0/3915683235 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9d40ce1a0 con 0x7fd9d415afb0 2023-11-24T06:12:38.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.116+0000 7fd9d9179640 1 --2- 172.21.15.92:0/3915683235 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd9d415afb0 0x7fd9d413f5a0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fd9d4154a90 tx=0x7fd9c402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52e9c9fd19f235c7 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:38.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.116+0000 7fd9d8978640 1 -- 172.21.15.92:0/3915683235 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9c40317e0 con 0x7fd9d415afb0 2023-11-24T06:12:38.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.116+0000 7fd9d8978640 1 -- 172.21.15.92:0/3915683235 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9c4031940 con 0x7fd9d415afb0 2023-11-24T06:12:38.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.116+0000 7fd9d8978640 1 -- 172.21.15.92:0/3915683235 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9c4031cd0 con 0x7fd9d415afb0 2023-11-24T06:12:38.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.117+0000 7fd9da97c640 1 -- 172.21.15.92:0/3915683235 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd9d415afb0 msgr2=0x7fd9d413f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:38.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.117+0000 7fd9da97c640 1 --2- 172.21.15.92:0/3915683235 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd9d415afb0 0x7fd9d413f5a0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fd9d4154a90 tx=0x7fd9c402b880 comp rx=0 tx=0).stop 2023-11-24T06:12:38.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.117+0000 7fd9da97c640 1 -- 172.21.15.92:0/3915683235 shutdown_connections 2023-11-24T06:12:38.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.117+0000 7fd9da97c640 1 --2- 172.21.15.92:0/3915683235 >> v2:172.21.15.118:3300/0 conn(0x7fd9d413fae0 0x7fd9d40b2040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:38.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.117+0000 7fd9da97c640 1 --2- 172.21.15.92:0/3915683235 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd9d415afb0 0x7fd9d413f5a0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:38.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.117+0000 7fd9da97c640 1 -- 172.21.15.92:0/3915683235 >> 172.21.15.92:0/3915683235 conn(0x7fd9d406a800 msgr2=0x7fd9d406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:38.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.117+0000 7fd9da97c640 1 -- 172.21.15.92:0/3915683235 shutdown_connections 2023-11-24T06:12:38.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.117+0000 7fd9da97c640 1 -- 172.21.15.92:0/3915683235 wait complete. 2023-11-24T06:12:38.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.119+0000 7fd9da97c640 1 Processor -- start 2023-11-24T06:12:38.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.119+0000 7fd9da97c640 1 -- start start 2023-11-24T06:12:38.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.120+0000 7fd9da97c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd9d413cfa0 0x7fd9d41f6740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:38.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.120+0000 7fd9da97c640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd9d413fae0 0x7fd9d41548d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:38.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.120+0000 7fd9da97c640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd9d415afb0 0x7fd9d40ca1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:38.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.120+0000 7fd9da97c640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9d4155290 con 0x7fd9d413cfa0 2023-11-24T06:12:38.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.120+0000 7fd9da97c640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9d4155570 con 0x7fd9d413fae0 2023-11-24T06:12:38.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.120+0000 7fd9da97c640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9d4155850 con 0x7fd9d415afb0 2023-11-24T06:12:38.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.120+0000 7fd9d997a640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd9d413cfa0 0x7fd9d41f6740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:38.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.120+0000 7fd9d997a640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd9d413cfa0 0x7fd9d41f6740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50028/0 (socket says 172.21.15.92:50028) 2023-11-24T06:12:38.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.120+0000 7fd9d997a640 1 -- 172.21.15.92:0/1785968076 learned_addr learned my addr 172.21.15.92:0/1785968076 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:12:38.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.120+0000 7fd9d997a640 1 -- 172.21.15.92:0/1785968076 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd9d415afb0 msgr2=0x7fd9d40ca1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:38.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.120+0000 7fd9d997a640 1 --2- 172.21.15.92:0/1785968076 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd9d415afb0 0x7fd9d40ca1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:38.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.120+0000 7fd9d997a640 1 -- 172.21.15.92:0/1785968076 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd9d413fae0 msgr2=0x7fd9d41548d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:38.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.120+0000 7fd9d9179640 1 --2- 172.21.15.92:0/1785968076 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd9d413fae0 0x7fd9d41548d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:38.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.120+0000 7fd9d997a640 1 --2- 172.21.15.92:0/1785968076 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd9d413fae0 0x7fd9d41548d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:38.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.120+0000 7fd9d997a640 1 -- 172.21.15.92:0/1785968076 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9c40094d0 con 0x7fd9d413cfa0 2023-11-24T06:12:38.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.121+0000 7fd9d997a640 1 --2- 172.21.15.92:0/1785968076 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd9d413cfa0 0x7fd9d41f6740 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fd9d000c410 tx=0x7fd9d000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:38.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.121+0000 7fd9ca7fc640 1 -- 172.21.15.92:0/1785968076 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9d00035f0 con 0x7fd9d413cfa0 2023-11-24T06:12:38.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.121+0000 7fd9ca7fc640 1 -- 172.21.15.92:0/1785968076 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9d0003750 con 0x7fd9d413cfa0 2023-11-24T06:12:38.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.121+0000 7fd9ca7fc640 1 -- 172.21.15.92:0/1785968076 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9d0003b40 con 0x7fd9d413cfa0 2023-11-24T06:12:38.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.121+0000 7fd9da97c640 1 -- 172.21.15.92:0/1785968076 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9d41580c0 con 0x7fd9d413cfa0 2023-11-24T06:12:38.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.121+0000 7fd9da97c640 1 -- 172.21.15.92:0/1785968076 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9d41388a0 con 0x7fd9d413cfa0 2023-11-24T06:12:38.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.123+0000 7fd9ca7fc640 1 -- 172.21.15.92:0/1785968076 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd9d0003ca0 con 0x7fd9d413cfa0 2023-11-24T06:12:38.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.124+0000 7fd9ca7fc640 1 --2- 172.21.15.92:0/1785968076 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fd9b806ae40 0x7fd9b806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:38.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.124+0000 7fd9ca7fc640 1 -- 172.21.15.92:0/1785968076 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd9d00888f0 con 0x7fd9d413cfa0 2023-11-24T06:12:38.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.124+0000 7fd9d9179640 1 --2- 172.21.15.92:0/1785968076 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fd9b806ae40 0x7fd9b806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:38.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.125+0000 7fd9de77b740 1 Processor -- start 2023-11-24T06:12:38.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.125+0000 7fd9d9179640 1 --2- 172.21.15.92:0/1785968076 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fd9b806ae40 0x7fd9b806d300 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fd9c4036000 tx=0x7fd9c403d040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:38.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.125+0000 7fd9de77b740 1 -- start start 2023-11-24T06:12:38.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.126+0000 7fd9de77b740 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x55ba1f1c5070 0x55ba1f1c5450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:38.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.126+0000 7fd9de77b740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ba1f1bfd70 0x55ba1f1c4270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:38.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.126+0000 7fd9de77b740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ba1f1c47b0 0x55ba1f208730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:38.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.126+0000 7fd9de77b740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba1f123880 con 0x55ba1f1bfd70 2023-11-24T06:12:38.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.126+0000 7fd9de77b740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba1f132ae0 con 0x55ba1f1c47b0 2023-11-24T06:12:38.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.126+0000 7fd9de77b740 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ba1f1301d0 con 0x55ba1f1c5070 2023-11-24T06:12:38.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.126+0000 7fd9d9179640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ba1f1bfd70 0x55ba1f1c4270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:38.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.126+0000 7fd9d9179640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ba1f1bfd70 0x55ba1f1c4270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50032/0 (socket says 172.21.15.92:50032) 2023-11-24T06:12:38.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.126+0000 7fd9d9179640 1 -- 172.21.15.92:0/2728705801 learned_addr learned my addr 172.21.15.92:0/2728705801 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:12:38.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.127+0000 7fd9d997a640 1 --2- 172.21.15.92:0/2728705801 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x55ba1f1c5070 0x55ba1f1c5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:38.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.127+0000 7fd9da17b640 1 --2- 172.21.15.92:0/2728705801 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ba1f1c47b0 0x55ba1f208730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:38.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.127+0000 7fd9d9179640 1 -- 172.21.15.92:0/2728705801 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x55ba1f1c5070 msgr2=0x55ba1f1c5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:38.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.127+0000 7fd9d9179640 1 --2- 172.21.15.92:0/2728705801 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x55ba1f1c5070 0x55ba1f1c5450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:38.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.127+0000 7fd9d9179640 1 -- 172.21.15.92:0/2728705801 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ba1f1c47b0 msgr2=0x55ba1f208730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:38.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.127+0000 7fd9d9179640 1 --2- 172.21.15.92:0/2728705801 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ba1f1c47b0 0x55ba1f208730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:38.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.127+0000 7fd9d9179640 1 -- 172.21.15.92:0/2728705801 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ba1f0f3860 con 0x55ba1f1bfd70 2023-11-24T06:12:38.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.127+0000 7fd9da17b640 1 --2- 172.21.15.92:0/2728705801 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ba1f1c47b0 0x55ba1f208730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:12:38.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.127+0000 7fd9d9179640 1 --2- 172.21.15.92:0/2728705801 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ba1f1bfd70 0x55ba1f1c4270 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fd9c40317b0 tx=0x7fd9c4030000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eca8a146b34b840 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:38.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.127+0000 7fd9b77fe640 1 -- 172.21.15.92:0/2728705801 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9c4032c30 con 0x55ba1f1bfd70 2023-11-24T06:12:38.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.127+0000 7fd9b77fe640 1 -- 172.21.15.92:0/2728705801 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9c4032d90 con 0x55ba1f1bfd70 2023-11-24T06:12:38.132 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.127+0000 7fd9b77fe640 1 -- 172.21.15.92:0/2728705801 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9c404e400 con 0x55ba1f1bfd70 2023-11-24T06:12:38.132 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.128+0000 7fd9de77b740 1 -- 172.21.15.92:0/2728705801 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ba1f1bfd70 msgr2=0x55ba1f1c4270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:38.132 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.128+0000 7fd9de77b740 1 --2- 172.21.15.92:0/2728705801 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ba1f1bfd70 0x55ba1f1c4270 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fd9c40317b0 tx=0x7fd9c4030000 comp rx=0 tx=0).stop 2023-11-24T06:12:38.132 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.128+0000 7fd9de77b740 1 -- 172.21.15.92:0/2728705801 shutdown_connections 2023-11-24T06:12:38.132 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.128+0000 7fd9de77b740 1 --2- 172.21.15.92:0/2728705801 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ba1f1c47b0 0x55ba1f208730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:38.132 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.128+0000 7fd9de77b740 1 --2- 172.21.15.92:0/2728705801 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ba1f1bfd70 0x55ba1f1c4270 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:38.132 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.128+0000 7fd9de77b740 1 --2- 172.21.15.92:0/2728705801 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x55ba1f1c5070 0x55ba1f1c5450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:38.132 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.128+0000 7fd9de77b740 1 -- 172.21.15.92:0/2728705801 >> 172.21.15.92:0/2728705801 conn(0x55ba1f186c70 msgr2=0x55ba1f20ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:38.132 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.128+0000 7fd9de77b740 1 -- 172.21.15.92:0/2728705801 shutdown_connections 2023-11-24T06:12:38.132 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.128+0000 7fd9de77b740 1 -- 172.21.15.92:0/2728705801 wait complete. 2023-11-24T06:12:38.138 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.129+0000 7fd9de77b740 1 -- start start 2023-11-24T06:12:38.138 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.129+0000 7fd9de77b740 1 Processor -- start 2023-11-24T06:12:38.138 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.130+0000 7fd9de77b740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ba1f1bfd70 0x55ba1f212b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:38.138 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.130+0000 7fd9de77b740 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x55ba1f1c47b0 0x55ba1f211f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:38.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.130+0000 7fd9de77b740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ba1f1c5070 0x55ba1f212470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:38.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.130+0000 7fd9de77b740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba1f12bac0 con 0x55ba1f1bfd70 2023-11-24T06:12:38.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.130+0000 7fd9de77b740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba1f0c82f0 con 0x55ba1f1c5070 2023-11-24T06:12:38.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.130+0000 7fd9de77b740 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ba1f0e5c70 con 0x55ba1f1c47b0 2023-11-24T06:12:38.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.130+0000 7fd9de77b740 10 client.0 _ll_register_callbacks cb 0x55ba1f0ea3b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-24T06:12:38.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.130+0000 7fd9d997a640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ba1f1bfd70 0x55ba1f212b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:38.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.131+0000 7fd9d997a640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ba1f1bfd70 0x55ba1f212b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50044/0 (socket says 172.21.15.92:50044) 2023-11-24T06:12:38.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.131+0000 7fd9d997a640 1 -- 172.21.15.92:0/1874318052 learned_addr learned my addr 172.21.15.92:0/1874318052 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:12:38.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.131+0000 7fd9d997a640 1 -- 172.21.15.92:0/1874318052 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x55ba1f1c47b0 msgr2=0x55ba1f211f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:38.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.131+0000 7fd9d997a640 1 --2- 172.21.15.92:0/1874318052 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x55ba1f1c47b0 0x55ba1f211f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:38.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.131+0000 7fd9d997a640 1 -- 172.21.15.92:0/1874318052 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ba1f1c5070 msgr2=0x55ba1f212470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:38.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.131+0000 7fd9d997a640 1 --2- 172.21.15.92:0/1874318052 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ba1f1c5070 0x55ba1f212470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:38.140 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.131+0000 7fd9d997a640 1 -- 172.21.15.92:0/1874318052 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9d40ce1a0 con 0x55ba1f1bfd70 2023-11-24T06:12:38.140 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.131+0000 7fd9d997a640 1 --2- 172.21.15.92:0/1874318052 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ba1f1bfd70 0x55ba1f212b80 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fd9d0053ac0 tx=0x7fd9d0057e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=115bd51802973465 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:38.140 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.131+0000 7fd9b4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-11-24T06:12:38.140 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.131+0000 7fd9b4ff9640 1 -- 172.21.15.92:0/1874318052 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9d005f430 con 0x55ba1f1bfd70 2023-11-24T06:12:38.140 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.131+0000 7fd9b4ff9640 1 -- 172.21.15.92:0/1874318052 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(52..52 src has 1..52) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd9d005f590 con 0x55ba1f1bfd70 2023-11-24T06:12:38.140 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.131+0000 7fd9de77b740 1 -- 172.21.15.92:0/1874318052 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ba1f124aa0 con 0x55ba1f1bfd70 2023-11-24T06:12:38.140 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.132+0000 7fd9b4ff9640 1 -- 172.21.15.92:0/1874318052 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd9d00533b0 con 0x55ba1f1bfd70 2023-11-24T06:12:38.140 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.132+0000 7fd9de77b740 10 client.5856 fetch_fsmap learned FSMap version 31 2023-11-24T06:12:38.140 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.132+0000 7fd9de77b740 1 -- 172.21.15.92:0/1874318052 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=2+}) v3 -- 0x55ba1f12a140 con 0x55ba1f1bfd70 2023-11-24T06:12:38.140 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.132+0000 7fd9b4ff9640 1 -- 172.21.15.92:0/1874318052 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ba1f124aa0 con 0x55ba1f1bfd70 2023-11-24T06:12:38.140 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.132+0000 7fd9b4ff9640 1 -- 172.21.15.92:0/1874318052 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 31) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fd9d0074070 con 0x55ba1f1bfd70 2023-11-24T06:12:38.140 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.132+0000 7fd9de77b740 10 client.5856 fetch_fsmap finished waiting for FSMap version 31 2023-11-24T06:12:38.140 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.132+0000 7fd9de77b740 10 client.5856 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-24T06:12:38.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.132+0000 7fd9de77b740 20 client.5856 populate_metadata read hostname 'smithi092' 2023-11-24T06:12:38.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.132+0000 7fd9de77b740 1 --2- 172.21.15.92:0/1874318052 >> [v2:172.21.15.92:6835/1958912045,v1:172.21.15.92:6837/1958912045] conn(0x55ba1f1eaeb0 0x55ba1f228060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:38.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.132+0000 7fd9de77b740 4 client.5856 mds_command: new command op to 5709 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-24T06:12:38.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.132+0000 7fd9de77b740 1 -- 172.21.15.92:0/1874318052 --> [v2:172.21.15.92:6835/1958912045,v1:172.21.15.92:6837/1958912045] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ba1f1247c0 con 0x55ba1f1eaeb0 2023-11-24T06:12:38.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.132+0000 7fd9d9179640 1 --2- 172.21.15.92:0/1874318052 >> [v2:172.21.15.92:6835/1958912045,v1:172.21.15.92:6837/1958912045] conn(0x55ba1f1eaeb0 0x55ba1f228060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:38.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.133+0000 7fd9d9179640 1 --2- 172.21.15.92:0/1874318052 >> [v2:172.21.15.92:6835/1958912045,v1:172.21.15.92:6837/1958912045] conn(0x55ba1f1eaeb0 0x55ba1f228060 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7ea548e3b015880f server_cookie=14259e0412bc0f79 in_seq=0 out_seq=0 2023-11-24T06:12:38.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.133+0000 7fd9b4ff9640 10 client.5856 ms_handle_connect on v2:172.21.15.92:6835/1958912045 2023-11-24T06:12:38.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.133+0000 7fd9b4ff9640 1 -- 172.21.15.92:0/1874318052 <== mds.0 v2:172.21.15.92:6835/1958912045 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ba1f1247c0 con 0x55ba1f1eaeb0 2023-11-24T06:12:38.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.133+0000 7fd9b4ff9640 10 client.5856 handle_command_reply: tid=0 2023-11-24T06:12:38.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.133+0000 7fd9de77b740 1 client.5856 shutdown 2023-11-24T06:12:38.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.134+0000 7fd9de77b740 1 -- 172.21.15.92:0/1874318052 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ba1f1bfd70 msgr2=0x55ba1f212b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:38.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.134+0000 7fd9de77b740 1 --2- 172.21.15.92:0/1874318052 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ba1f1bfd70 0x55ba1f212b80 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fd9d0053ac0 tx=0x7fd9d0057e00 comp rx=0 tx=0).stop 2023-11-24T06:12:38.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.134+0000 7fd9de77b740 1 -- 172.21.15.92:0/1874318052 shutdown_connections 2023-11-24T06:12:38.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.134+0000 7fd9de77b740 1 --2- 172.21.15.92:0/1874318052 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ba1f1c5070 0x55ba1f212470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:38.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.134+0000 7fd9de77b740 1 --2- 172.21.15.92:0/1874318052 >> [v2:172.21.15.92:6835/1958912045,v1:172.21.15.92:6837/1958912045] conn(0x55ba1f1eaeb0 0x55ba1f228060 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:38.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.134+0000 7fd9de77b740 1 --2- 172.21.15.92:0/1874318052 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x55ba1f1c47b0 0x55ba1f211f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:38.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.134+0000 7fd9de77b740 1 --2- 172.21.15.92:0/1874318052 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ba1f1bfd70 0x55ba1f212b80 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:38.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.134+0000 7fd9b4ff9640 0 client.5856 ms_handle_reset on v2:172.21.15.92:6835/1958912045 2023-11-24T06:12:38.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.134+0000 7fd9de77b740 1 -- 172.21.15.92:0/1874318052 >> 172.21.15.92:0/1874318052 conn(0x55ba1f186c70 msgr2=0x55ba1f1c5450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:38.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.134+0000 7fd9de77b740 1 -- 172.21.15.92:0/1874318052 shutdown_connections 2023-11-24T06:12:38.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.134+0000 7fd9de77b740 1 -- 172.21.15.92:0/1874318052 wait complete. 2023-11-24T06:12:38.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.134+0000 7fd9de77b740 20 client.5856 trim_cache size 0 max 16384 2023-11-24T06:12:38.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.151+0000 7fd9de77b740 1 Processor -- start 2023-11-24T06:12:38.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.151+0000 7fd9de77b740 1 -- start start 2023-11-24T06:12:38.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.152+0000 7fd9de77b740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ba1f1bfd70 0x55ba1f1c3960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:38.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.152+0000 7fd9de77b740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ba1f1c47b0 0x55ba1f22c750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:38.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.152+0000 7fd9de77b740 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x55ba1f1c5070 0x55ba1f1bedd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:38.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.152+0000 7fd9de77b740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba1ee8ef20 con 0x55ba1f1c47b0 2023-11-24T06:12:38.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.152+0000 7fd9de77b740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba1f21dc80 con 0x55ba1f1bfd70 2023-11-24T06:12:38.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.152+0000 7fd9de77b740 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ba1f204c50 con 0x55ba1f1c5070 2023-11-24T06:12:38.155 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.152+0000 7fd9d9179640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ba1f1c47b0 0x55ba1f22c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:38.155 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.152+0000 7fd9d9179640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ba1f1c47b0 0x55ba1f22c750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50056/0 (socket says 172.21.15.92:50056) 2023-11-24T06:12:38.155 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.152+0000 7fd9d9179640 1 -- 172.21.15.92:0/618882856 learned_addr learned my addr 172.21.15.92:0/618882856 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:12:38.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.152+0000 7fd9da17b640 1 --2- 172.21.15.92:0/618882856 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x55ba1f1c5070 0x55ba1f1bedd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:38.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.152+0000 7fd9d9179640 1 -- 172.21.15.92:0/618882856 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x55ba1f1c5070 msgr2=0x55ba1f1bedd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:38.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.152+0000 7fd9d997a640 1 --2- 172.21.15.92:0/618882856 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ba1f1bfd70 0x55ba1f1c3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:38.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.153+0000 7fd9d9179640 1 --2- 172.21.15.92:0/618882856 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x55ba1f1c5070 0x55ba1f1bedd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:38.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.153+0000 7fd9d9179640 1 -- 172.21.15.92:0/618882856 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ba1f1bfd70 msgr2=0x55ba1f1c3960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:38.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.153+0000 7fd9d9179640 1 --2- 172.21.15.92:0/618882856 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ba1f1bfd70 0x55ba1f1c3960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:38.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.153+0000 7fd9d9179640 1 -- 172.21.15.92:0/618882856 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ba1f0c7b80 con 0x55ba1f1c47b0 2023-11-24T06:12:38.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.153+0000 7fd9da17b640 1 --2- 172.21.15.92:0/618882856 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x55ba1f1c5070 0x55ba1f1bedd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:12:38.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.153+0000 7fd9d997a640 1 --2- 172.21.15.92:0/618882856 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ba1f1bfd70 0x55ba1f1c3960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:12:38.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.153+0000 7fd9d9179640 1 --2- 172.21.15.92:0/618882856 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ba1f1c47b0 0x55ba1f22c750 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fd9c40090b0 tx=0x7fd9c402e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8572d35e559847b server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:38.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.153+0000 7fd9b7fff640 1 -- 172.21.15.92:0/618882856 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9c4057480 con 0x55ba1f1c47b0 2023-11-24T06:12:38.158 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.153+0000 7fd9b7fff640 1 -- 172.21.15.92:0/618882856 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9c40575e0 con 0x55ba1f1c47b0 2023-11-24T06:12:38.158 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.153+0000 7fd9b7fff640 1 -- 172.21.15.92:0/618882856 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9c4004030 con 0x55ba1f1c47b0 2023-11-24T06:12:38.158 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.154+0000 7fd9de77b740 1 -- 172.21.15.92:0/618882856 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ba1f1c47b0 msgr2=0x55ba1f22c750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:38.158 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.154+0000 7fd9de77b740 1 --2- 172.21.15.92:0/618882856 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ba1f1c47b0 0x55ba1f22c750 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fd9c40090b0 tx=0x7fd9c402e000 comp rx=0 tx=0).stop 2023-11-24T06:12:38.158 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.154+0000 7fd9de77b740 1 -- 172.21.15.92:0/618882856 shutdown_connections 2023-11-24T06:12:38.158 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.154+0000 7fd9de77b740 1 --2- 172.21.15.92:0/618882856 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x55ba1f1c5070 0x55ba1f1bedd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:38.158 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.154+0000 7fd9de77b740 1 --2- 172.21.15.92:0/618882856 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ba1f1c47b0 0x55ba1f22c750 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:38.158 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.154+0000 7fd9de77b740 1 --2- 172.21.15.92:0/618882856 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ba1f1bfd70 0x55ba1f1c3960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:38.158 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.154+0000 7fd9de77b740 1 -- 172.21.15.92:0/618882856 >> 172.21.15.92:0/618882856 conn(0x55ba1f186c70 msgr2=0x55ba1f1bb4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:38.158 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.154+0000 7fd9de77b740 1 -- 172.21.15.92:0/618882856 shutdown_connections 2023-11-24T06:12:38.158 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.154+0000 7fd9de77b740 1 -- 172.21.15.92:0/618882856 wait complete. 2023-11-24T06:12:38.159 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.154+0000 7fd9de77b740 1 -- start start 2023-11-24T06:12:38.159 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.155+0000 7fd9de77b740 1 Processor -- start 2023-11-24T06:12:38.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.156+0000 7fd9de77b740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ba1f1bfd70 0x55ba1f27d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:38.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.156+0000 7fd9de77b740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ba1f1c47b0 0x55ba1f1dc1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:38.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.156+0000 7fd9de77b740 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x55ba1f1c5070 0x55ba1f1e8910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:38.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.156+0000 7fd9de77b740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba1f21d8c0 con 0x55ba1f1c47b0 2023-11-24T06:12:38.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.156+0000 7fd9de77b740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba1f211890 con 0x55ba1f1bfd70 2023-11-24T06:12:38.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.156+0000 7fd9de77b740 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ba1f211ae0 con 0x55ba1f1c5070 2023-11-24T06:12:38.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.156+0000 7fd9de77b740 10 client.0 _ll_register_callbacks cb 0x7fd9d0053900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-24T06:12:38.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.156+0000 7fd9d9179640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ba1f1c47b0 0x55ba1f1dc1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:38.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.156+0000 7fd9d9179640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ba1f1c47b0 0x55ba1f1dc1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50072/0 (socket says 172.21.15.92:50072) 2023-11-24T06:12:38.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.156+0000 7fd9d9179640 1 -- 172.21.15.92:0/2261852263 learned_addr learned my addr 172.21.15.92:0/2261852263 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:12:38.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.156+0000 7fd9da17b640 1 --2- 172.21.15.92:0/2261852263 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x55ba1f1c5070 0x55ba1f1e8910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:38.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.156+0000 7fd9d9179640 1 -- 172.21.15.92:0/2261852263 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x55ba1f1c5070 msgr2=0x55ba1f1e8910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:38.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.156+0000 7fd9d997a640 1 --2- 172.21.15.92:0/2261852263 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ba1f1bfd70 0x55ba1f27d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:38.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.156+0000 7fd9d9179640 1 --2- 172.21.15.92:0/2261852263 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x55ba1f1c5070 0x55ba1f1e8910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:38.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.156+0000 7fd9d9179640 1 -- 172.21.15.92:0/2261852263 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ba1f1bfd70 msgr2=0x55ba1f27d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:38.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.156+0000 7fd9d9179640 1 --2- 172.21.15.92:0/2261852263 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ba1f1bfd70 0x55ba1f27d4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:38.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.156+0000 7fd9d9179640 1 -- 172.21.15.92:0/2261852263 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ba1f0f3860 con 0x55ba1f1c47b0 2023-11-24T06:12:38.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.157+0000 7fd9d9179640 1 --2- 172.21.15.92:0/2261852263 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ba1f1c47b0 0x55ba1f1dc1c0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fd9c4006b10 tx=0x7fd9c4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8655fe9cd65940a server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:38.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.157+0000 7fd9b4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-11-24T06:12:38.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.157+0000 7fd9b4ff9640 1 -- 172.21.15.92:0/2261852263 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9c4004030 con 0x55ba1f1c47b0 2023-11-24T06:12:38.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.157+0000 7fd9b4ff9640 1 -- 172.21.15.92:0/2261852263 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(52..52 src has 1..52) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd9c4004190 con 0x55ba1f1c47b0 2023-11-24T06:12:38.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.157+0000 7fd9de77b740 1 -- 172.21.15.92:0/2261852263 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd9c4032d90 con 0x55ba1f1c47b0 2023-11-24T06:12:38.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.157+0000 7fd9b4ff9640 1 -- 172.21.15.92:0/2261852263 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd9c4004350 con 0x55ba1f1c47b0 2023-11-24T06:12:38.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.157+0000 7fd9de77b740 10 client.5862 fetch_fsmap learned FSMap version 31 2023-11-24T06:12:38.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.157+0000 7fd9de77b740 1 -- 172.21.15.92:0/2261852263 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=2+}) v3 -- 0x55ba1f0ed170 con 0x55ba1f1c47b0 2023-11-24T06:12:38.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.157+0000 7fd9b4ff9640 1 -- 172.21.15.92:0/2261852263 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9c4032d90 con 0x55ba1f1c47b0 2023-11-24T06:12:38.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.158+0000 7fd9b4ff9640 1 -- 172.21.15.92:0/2261852263 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 31) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fd9c4003610 con 0x55ba1f1c47b0 2023-11-24T06:12:38.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.158+0000 7fd9de77b740 10 client.5862 fetch_fsmap finished waiting for FSMap version 31 2023-11-24T06:12:38.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.158+0000 7fd9de77b740 10 client.5862 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-24T06:12:38.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.158+0000 7fd9de77b740 20 client.5862 populate_metadata read hostname 'smithi092' 2023-11-24T06:12:38.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.158+0000 7fd9de77b740 1 --2- 172.21.15.92:0/2261852263 >> [v2:172.21.15.92:6835/1958912045,v1:172.21.15.92:6837/1958912045] conn(0x55ba1f1bedd0 0x55ba1f1bf1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:38.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.158+0000 7fd9de77b740 4 client.5862 mds_command: new command op to 5709 tid=0 multi_id=0 [{"prefix": "status"}] 2023-11-24T06:12:38.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.158+0000 7fd9de77b740 1 -- 172.21.15.92:0/2261852263 --> [v2:172.21.15.92:6835/1958912045,v1:172.21.15.92:6837/1958912045] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55ba1ee2c6c0 con 0x55ba1f1bedd0 2023-11-24T06:12:38.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.158+0000 7fd9d997a640 1 --2- 172.21.15.92:0/2261852263 >> [v2:172.21.15.92:6835/1958912045,v1:172.21.15.92:6837/1958912045] conn(0x55ba1f1bedd0 0x55ba1f1bf1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:38.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.158+0000 7fd9b4ff9640 10 client.5862 ms_handle_connect on v2:172.21.15.92:6835/1958912045 2023-11-24T06:12:38.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.158+0000 7fd9d997a640 1 --2- 172.21.15.92:0/2261852263 >> [v2:172.21.15.92:6835/1958912045,v1:172.21.15.92:6837/1958912045] conn(0x55ba1f1bedd0 0x55ba1f1bf1b0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7ba5604ede180ae9 server_cookie=8e7e974bf805fdad in_seq=0 out_seq=0 2023-11-24T06:12:38.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.159+0000 7fd9b4ff9640 1 -- 172.21.15.92:0/2261852263 <== mds.0 v2:172.21.15.92:6835/1958912045 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x55ba1ee2c6c0 con 0x55ba1f1bedd0 2023-11-24T06:12:38.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.159+0000 7fd9b4ff9640 10 client.5862 handle_command_reply: tid=0 2023-11-24T06:12:38.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.159+0000 7fd9de77b740 1 client.5862 shutdown 2023-11-24T06:12:38.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.159+0000 7fd9de77b740 1 -- 172.21.15.92:0/2261852263 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ba1f1c47b0 msgr2=0x55ba1f1dc1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:38.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.159+0000 7fd9de77b740 1 --2- 172.21.15.92:0/2261852263 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ba1f1c47b0 0x55ba1f1dc1c0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fd9c4006b10 tx=0x7fd9c4004730 comp rx=0 tx=0).stop 2023-11-24T06:12:38.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.159+0000 7fd9de77b740 1 -- 172.21.15.92:0/2261852263 shutdown_connections 2023-11-24T06:12:38.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.159+0000 7fd9de77b740 1 --2- 172.21.15.92:0/2261852263 >> [v2:172.21.15.92:6835/1958912045,v1:172.21.15.92:6837/1958912045] conn(0x55ba1f1bedd0 0x55ba1f1bf1b0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:38.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.159+0000 7fd9de77b740 1 --2- 172.21.15.92:0/2261852263 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x55ba1f1c5070 0x55ba1f1e8910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:38.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.159+0000 7fd9de77b740 1 --2- 172.21.15.92:0/2261852263 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ba1f1c47b0 0x55ba1f1dc1c0 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:38.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.159+0000 7fd9de77b740 1 --2- 172.21.15.92:0/2261852263 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ba1f1bfd70 0x55ba1f27d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:38.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.159+0000 7fd9b4ff9640 0 client.5862 ms_handle_reset on v2:172.21.15.92:6835/1958912045 2023-11-24T06:12:38.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.159+0000 7fd9de77b740 1 -- 172.21.15.92:0/2261852263 >> 172.21.15.92:0/2261852263 conn(0x55ba1f186c70 msgr2=0x55ba1f1c3960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:38.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.159+0000 7fd9de77b740 1 -- 172.21.15.92:0/2261852263 shutdown_connections 2023-11-24T06:12:38.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.159+0000 7fd9de77b740 1 -- 172.21.15.92:0/2261852263 wait complete. 2023-11-24T06:12:38.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.159+0000 7fd9de77b740 20 client.5862 trim_cache size 0 max 16384 2023-11-24T06:12:38.167 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-24T06:12:38.167 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_fsid": "2d6dee5f-8aa3-470f-931a-50ade963b1cb", 2023-11-24T06:12:38.167 INFO:teuthology.orchestra.run.smithi092.stdout: "whoami": 0, 2023-11-24T06:12:38.167 INFO:teuthology.orchestra.run.smithi092.stdout: "id": 5709, 2023-11-24T06:12:38.167 INFO:teuthology.orchestra.run.smithi092.stdout: "want_state": "up:active", 2023-11-24T06:12:38.167 INFO:teuthology.orchestra.run.smithi092.stdout: "state": "up:active", 2023-11-24T06:12:38.167 INFO:teuthology.orchestra.run.smithi092.stdout: "fs_name": "cephfs", 2023-11-24T06:12:38.167 INFO:teuthology.orchestra.run.smithi092.stdout: "rank_uptime": 6.6504865909999999, 2023-11-24T06:12:38.168 INFO:teuthology.orchestra.run.smithi092.stdout: "mdsmap_epoch": 31, 2023-11-24T06:12:38.168 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_epoch": 52, 2023-11-24T06:12:38.168 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_epoch_barrier": 52, 2023-11-24T06:12:38.168 INFO:teuthology.orchestra.run.smithi092.stdout: "uptime": 13.448676375 2023-11-24T06:12:38.168 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-24T06:12:38.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.160+0000 7fd9da97c640 1 -- 172.21.15.92:0/1785968076 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fd9b806ae40 msgr2=0x7fd9b806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:38.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.160+0000 7fd9da97c640 1 --2- 172.21.15.92:0/1785968076 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fd9b806ae40 0x7fd9b806d300 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fd9c4036000 tx=0x7fd9c403d040 comp rx=0 tx=0).stop 2023-11-24T06:12:38.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.160+0000 7fd9da97c640 1 -- 172.21.15.92:0/1785968076 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd9d413cfa0 msgr2=0x7fd9d41f6740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:38.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.160+0000 7fd9da97c640 1 --2- 172.21.15.92:0/1785968076 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd9d413cfa0 0x7fd9d41f6740 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fd9d000c410 tx=0x7fd9d000c8d0 comp rx=0 tx=0).stop 2023-11-24T06:12:38.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.160+0000 7fd9da97c640 1 -- 172.21.15.92:0/1785968076 shutdown_connections 2023-11-24T06:12:38.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.160+0000 7fd9da97c640 1 --2- 172.21.15.92:0/1785968076 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fd9b806ae40 0x7fd9b806d300 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:38.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.160+0000 7fd9da97c640 1 --2- 172.21.15.92:0/1785968076 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd9d415afb0 0x7fd9d40ca1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:38.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.160+0000 7fd9da97c640 1 --2- 172.21.15.92:0/1785968076 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd9d413fae0 0x7fd9d41548d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:38.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.160+0000 7fd9da97c640 1 --2- 172.21.15.92:0/1785968076 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd9d413cfa0 0x7fd9d41f6740 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:38.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.160+0000 7fd9da97c640 1 -- 172.21.15.92:0/1785968076 >> 172.21.15.92:0/1785968076 conn(0x7fd9d406a800 msgr2=0x7fd9d406abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:38.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.160+0000 7fd9da97c640 1 -- 172.21.15.92:0/1785968076 shutdown_connections 2023-11-24T06:12:38.170 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.160+0000 7fd9da97c640 1 -- 172.21.15.92:0/1785968076 wait complete. 2023-11-24T06:12:38.180 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-24T06:12:38.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.387+0000 7fe679002640 1 Processor -- start 2023-11-24T06:12:38.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.388+0000 7fe679002640 1 -- start start 2023-11-24T06:12:38.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.389+0000 7fe679002640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fe67413df80 0x7fe67415bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:38.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.389+0000 7fe679002640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe6740cf9a0 0x7fe6740cd840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:38.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.389+0000 7fe679002640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe67406c470 con 0x7fe67413e360 2023-11-24T06:12:38.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.389+0000 7fe679002640 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe67406c750 con 0x7fe67413df80 2023-11-24T06:12:38.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.389+0000 7fe679002640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe67406ca30 con 0x7fe6740cf9a0 2023-11-24T06:12:38.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.389+0000 7fe66bfff640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fe67413e360 0x7fe6740cf290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:49900/0 (socket says 172.21.15.92:49900) 2023-11-24T06:12:38.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.389+0000 7fe66bfff640 1 -- 172.21.15.92:0/2066617770 learned_addr learned my addr 172.21.15.92:0/2066617770 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-24T06:12:38.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.390+0000 7fe672d9d640 1 --2- 172.21.15.92:0/2066617770 >> v2:172.21.15.118:3300/0 conn(0x7fe67413df80 0x7fe67415bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:38.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.390+0000 7fe67359e640 1 --2- 172.21.15.92:0/2066617770 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe6740cf9a0 0x7fe6740cd840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:38.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.390+0000 7fe67259c640 1 -- 172.21.15.92:0/2066617770 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3692636655 0 0) 0x7fe67406c470 con 0x7fe67413e360 2023-11-24T06:12:38.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.390+0000 7fe67259c640 1 -- 172.21.15.92:0/2066617770 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe660003520 con 0x7fe67413e360 2023-11-24T06:12:38.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.390+0000 7fe67259c640 1 -- 172.21.15.92:0/2066617770 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1755513333 0 0) 0x7fe660003520 con 0x7fe67413e360 2023-11-24T06:12:38.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.390+0000 7fe67259c640 1 -- 172.21.15.92:0/2066617770 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe6740cf9a0 msgr2=0x7fe6740cd840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:38.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.390+0000 7fe67259c640 1 --2- 172.21.15.92:0/2066617770 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe6740cf9a0 0x7fe6740cd840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:38.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.390+0000 7fe67259c640 1 -- 172.21.15.92:0/2066617770 >> v2:172.21.15.118:3300/0 conn(0x7fe67413df80 msgr2=0x7fe67415bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:38.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.390+0000 7fe67259c640 1 --2- 172.21.15.92:0/2066617770 >> v2:172.21.15.118:3300/0 conn(0x7fe67413df80 0x7fe67415bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:38.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.390+0000 7fe67259c640 1 -- 172.21.15.92:0/2066617770 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe67406cd10 con 0x7fe67413e360 2023-11-24T06:12:38.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.390+0000 7fe67259c640 1 -- 172.21.15.92:0/2066617770 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2607661350 0 0) 0x7fe65c001670 con 0x7fe67413e360 2023-11-24T06:12:38.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.390+0000 7fe67259c640 1 -- 172.21.15.92:0/2066617770 >> v1:172.21.15.92:6789/0 conn(0x7fe67413e360 legacy=0x7fe6740cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:38.395 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.390+0000 7fe67259c640 1 --2- 172.21.15.92:0/2066617770 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe660003970 0x7fe660003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:38.395 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.390+0000 7fe67259c640 1 --2- 172.21.15.92:0/2066617770 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe6600047c0 0x7fe660004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:38.395 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.390+0000 7fe67259c640 1 --2- 172.21.15.92:0/2066617770 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe6740cf9a0 0x7fe660005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:38.395 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.390+0000 7fe67259c640 1 -- 172.21.15.92:0/2066617770 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe67406c750 con 0x7fe6600047c0 2023-11-24T06:12:38.395 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.390+0000 7fe67259c640 1 -- 172.21.15.92:0/2066617770 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe67406ca30 con 0x7fe660003970 2023-11-24T06:12:38.395 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.390+0000 7fe67259c640 1 -- 172.21.15.92:0/2066617770 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe66000b980 con 0x7fe6740cf9a0 2023-11-24T06:12:38.395 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.390+0000 7fe66bfff640 1 --2- 172.21.15.92:0/2066617770 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe6600047c0 0x7fe660004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:38.395 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.391+0000 7fe672d9d640 1 --2- 172.21.15.92:0/2066617770 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe660003970 0x7fe660003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:38.395 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.391+0000 7fe67359e640 1 --2- 172.21.15.92:0/2066617770 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe6740cf9a0 0x7fe660005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:38.395 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.391+0000 7fe66bfff640 1 -- 172.21.15.92:0/2066617770 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe6740cf9a0 msgr2=0x7fe660005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:38.395 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.391+0000 7fe66bfff640 1 --2- 172.21.15.92:0/2066617770 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe6740cf9a0 0x7fe660005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:38.395 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.391+0000 7fe66bfff640 1 -- 172.21.15.92:0/2066617770 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe660003970 msgr2=0x7fe660003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:38.395 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.391+0000 7fe66bfff640 1 --2- 172.21.15.92:0/2066617770 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe660003970 0x7fe660003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:38.395 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.391+0000 7fe66bfff640 1 -- 172.21.15.92:0/2066617770 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe660003520 con 0x7fe6600047c0 2023-11-24T06:12:38.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.392+0000 7fe66bfff640 1 --2- 172.21.15.92:0/2066617770 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe6600047c0 0x7fe660004c40 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fe6740cc790 tx=0x7fe6600057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63c61123b207cd59 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:38.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.392+0000 7fe67259c640 1 -- 172.21.15.92:0/2066617770 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe65c005ad0 con 0x7fe6600047c0 2023-11-24T06:12:38.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.392+0000 7fe67259c640 1 -- 172.21.15.92:0/2066617770 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe65c005c30 con 0x7fe6600047c0 2023-11-24T06:12:38.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.392+0000 7fe67259c640 1 -- 172.21.15.92:0/2066617770 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe65c02c890 con 0x7fe6600047c0 2023-11-24T06:12:38.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.393+0000 7fe679002640 1 -- 172.21.15.92:0/2066617770 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe6600047c0 msgr2=0x7fe660004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:38.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.393+0000 7fe679002640 1 --2- 172.21.15.92:0/2066617770 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe6600047c0 0x7fe660004c40 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fe6740cc790 tx=0x7fe6600057f0 comp rx=0 tx=0).stop 2023-11-24T06:12:38.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.393+0000 7fe67359e640 1 -- 172.21.15.92:0/2066617770 reap_dead start 2023-11-24T06:12:38.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.393+0000 7fe679002640 1 -- 172.21.15.92:0/2066617770 shutdown_connections 2023-11-24T06:12:38.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.393+0000 7fe679002640 1 -- 172.21.15.92:0/2066617770 >> 172.21.15.92:0/2066617770 conn(0x7fe67406a800 msgr2=0x7fe67406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:38.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.393+0000 7fe679002640 1 -- 172.21.15.92:0/2066617770 shutdown_connections 2023-11-24T06:12:38.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.393+0000 7fe679002640 1 -- 172.21.15.92:0/2066617770 wait complete. 2023-11-24T06:12:38.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.395+0000 7fe679002640 1 Processor -- start 2023-11-24T06:12:38.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.395+0000 7fe679002640 1 -- start start 2023-11-24T06:12:38.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.396+0000 7fe679002640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe6600047c0 0x7fe6741f27b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:38.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.396+0000 7fe679002640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe674070b70 0x7fe6740afab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:38.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.396+0000 7fe679002640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe674070ff0 0x7fe6740afff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:38.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.396+0000 7fe679002640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6741f2cf0 con 0x7fe674070ff0 2023-11-24T06:12:38.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.396+0000 7fe679002640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe674071530 con 0x7fe674070b70 2023-11-24T06:12:38.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.396+0000 7fe679002640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe674071810 con 0x7fe6600047c0 2023-11-24T06:12:38.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.396+0000 7fe67359e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe674070ff0 0x7fe6740afff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:38.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.396+0000 7fe67359e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe674070ff0 0x7fe6740afff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50092/0 (socket says 172.21.15.92:50092) 2023-11-24T06:12:38.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.396+0000 7fe67359e640 1 -- 172.21.15.92:0/1333053147 learned_addr learned my addr 172.21.15.92:0/1333053147 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:12:38.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.397+0000 7fe672d9d640 1 --2- 172.21.15.92:0/1333053147 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe6600047c0 0x7fe6741f27b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:38.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.397+0000 7fe66bfff640 1 --2- 172.21.15.92:0/1333053147 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe674070b70 0x7fe6740afab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:38.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.397+0000 7fe67359e640 1 -- 172.21.15.92:0/1333053147 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe6600047c0 msgr2=0x7fe6741f27b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:38.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.397+0000 7fe67359e640 1 --2- 172.21.15.92:0/1333053147 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe6600047c0 0x7fe6741f27b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:38.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.397+0000 7fe67359e640 1 -- 172.21.15.92:0/1333053147 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe674070b70 msgr2=0x7fe6740afab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:38.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.397+0000 7fe67359e640 1 --2- 172.21.15.92:0/1333053147 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe674070b70 0x7fe6740afab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:38.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.397+0000 7fe67359e640 1 -- 172.21.15.92:0/1333053147 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe674071af0 con 0x7fe674070ff0 2023-11-24T06:12:38.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.397+0000 7fe67359e640 1 --2- 172.21.15.92:0/1333053147 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe674070ff0 0x7fe6740afff0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fe6740cfea0 tx=0x7fe66c00f4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:38.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.397+0000 7fe66b7fe640 1 -- 172.21.15.92:0/1333053147 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe66c017400 con 0x7fe674070ff0 2023-11-24T06:12:38.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.397+0000 7fe66b7fe640 1 -- 172.21.15.92:0/1333053147 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe66c011040 con 0x7fe674070ff0 2023-11-24T06:12:38.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.397+0000 7fe66b7fe640 1 -- 172.21.15.92:0/1333053147 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe66c0177c0 con 0x7fe674070ff0 2023-11-24T06:12:38.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.397+0000 7fe679002640 1 -- 172.21.15.92:0/1333053147 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe674071dd0 con 0x7fe674070ff0 2023-11-24T06:12:38.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.398+0000 7fe679002640 1 -- 172.21.15.92:0/1333053147 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe67413db50 con 0x7fe674070ff0 2023-11-24T06:12:38.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.399+0000 7fe679002640 1 -- 172.21.15.92:0/1333053147 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6740722f0 con 0x7fe674070ff0 2023-11-24T06:12:38.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.400+0000 7fe66b7fe640 1 -- 172.21.15.92:0/1333053147 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe66c017920 con 0x7fe674070ff0 2023-11-24T06:12:38.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.401+0000 7fe66b7fe640 1 --2- 172.21.15.92:0/1333053147 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fe65406ae70 0x7fe65406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:38.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.401+0000 7fe66b7fe640 1 -- 172.21.15.92:0/1333053147 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe66c089ad0 con 0x7fe674070ff0 2023-11-24T06:12:38.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.401+0000 7fe672d9d640 1 --2- 172.21.15.92:0/1333053147 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fe65406ae70 0x7fe65406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:38.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.402+0000 7fe672d9d640 1 --2- 172.21.15.92:0/1333053147 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fe65406ae70 0x7fe65406d330 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fe674151ca0 tx=0x7fe664008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:38.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.405+0000 7fe66b7fe640 1 -- 172.21.15.92:0/1333053147 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe6740722f0 con 0x7fe674070ff0 2023-11-24T06:12:38.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.637+0000 7fe679002640 1 -- 172.21.15.92:0/1333053147 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fe6740722f0 con 0x7fe674070ff0 2023-11-24T06:12:38.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.638+0000 7fe66b7fe640 1 -- 172.21.15.92:0/1333053147 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v31) v1 ==== 53+0+73 (secure 0 0 0) 0x7fe66c01c060 con 0x7fe674070ff0 2023-11-24T06:12:38.640 INFO:teuthology.orchestra.run.smithi092.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-24T06:12:38.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.640+0000 7fe679002640 1 -- 172.21.15.92:0/1333053147 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fe65406ae70 msgr2=0x7fe65406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:38.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.640+0000 7fe679002640 1 --2- 172.21.15.92:0/1333053147 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fe65406ae70 0x7fe65406d330 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fe674151ca0 tx=0x7fe664008040 comp rx=0 tx=0).stop 2023-11-24T06:12:38.644 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.640+0000 7fe679002640 1 -- 172.21.15.92:0/1333053147 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe674070ff0 msgr2=0x7fe6740afff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:38.644 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.640+0000 7fe679002640 1 --2- 172.21.15.92:0/1333053147 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe674070ff0 0x7fe6740afff0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fe6740cfea0 tx=0x7fe66c00f4a0 comp rx=0 tx=0).stop 2023-11-24T06:12:38.644 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.640+0000 7fe679002640 1 -- 172.21.15.92:0/1333053147 shutdown_connections 2023-11-24T06:12:38.644 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.640+0000 7fe679002640 1 --2- 172.21.15.92:0/1333053147 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fe65406ae70 0x7fe65406d330 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:38.644 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.640+0000 7fe679002640 1 --2- 172.21.15.92:0/1333053147 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe674070ff0 0x7fe6740afff0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:38.645 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.640+0000 7fe679002640 1 --2- 172.21.15.92:0/1333053147 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe674070b70 0x7fe6740afab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:38.645 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.640+0000 7fe679002640 1 --2- 172.21.15.92:0/1333053147 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe6600047c0 0x7fe6741f27b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:38.645 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.640+0000 7fe679002640 1 -- 172.21.15.92:0/1333053147 >> 172.21.15.92:0/1333053147 conn(0x7fe67406a800 msgr2=0x7fe674154160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:38.645 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.641+0000 7fe679002640 1 -- 172.21.15.92:0/1333053147 shutdown_connections 2023-11-24T06:12:38.645 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:38.641+0000 7fe679002640 1 -- 172.21.15.92:0/1333053147 wait complete. 2023-11-24T06:12:38.661 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-24T06:12:38.662 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-24T06:12:38.662 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi092.front.sepia.ceph.com 2023-11-24T06:12:38.662 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-24T06:12:38.662 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-24T06:12:38.662 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-24T06:12:38.662 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-24T06:12:38.662 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-24T06:12:38.662 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-24T06:12:38.662 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:12:38.663 DEBUG:teuthology.orchestra.run.smithi092:> ip addr 2023-11-24T06:12:38.712 INFO:teuthology.orchestra.run.smithi092.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-24T06:12:38.712 INFO:teuthology.orchestra.run.smithi092.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-24T06:12:38.712 INFO:teuthology.orchestra.run.smithi092.stdout: inet 127.0.0.1/8 scope host lo 2023-11-24T06:12:38.713 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:12:38.713 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 ::1/128 scope host 2023-11-24T06:12:38.713 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:12:38.713 INFO:teuthology.orchestra.run.smithi092.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T06:12:38.713 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:d8:a1:b2 brd ff:ff:ff:ff:ff:ff 2023-11-24T06:12:38.713 INFO:teuthology.orchestra.run.smithi092.stdout: altname enp7s0f0 2023-11-24T06:12:38.713 INFO:teuthology.orchestra.run.smithi092.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T06:12:38.713 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:d8:a1:b3 brd ff:ff:ff:ff:ff:ff 2023-11-24T06:12:38.713 INFO:teuthology.orchestra.run.smithi092.stdout: altname enp7s0f1 2023-11-24T06:12:38.713 INFO:teuthology.orchestra.run.smithi092.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T06:12:38.713 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:8f:cd:44 brd ff:ff:ff:ff:ff:ff 2023-11-24T06:12:38.713 INFO:teuthology.orchestra.run.smithi092.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-24T06:12:38.713 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:8f:cd:45 brd ff:ff:ff:ff:ff:ff 2023-11-24T06:12:38.714 INFO:teuthology.orchestra.run.smithi092.stdout: inet 172.21.15.92/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-24T06:12:38.714 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft 42083sec preferred_lft 42083sec 2023-11-24T06:12:38.714 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 fe80::ec4:7aff:fe8f:cd45/64 scope link 2023-11-24T06:12:38.714 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:12:38.714 INFO:teuthology.orchestra.run.smithi092.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-24T06:12:38.714 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 7a:4d:f0:06:a7:4e brd ff:ff:ff:ff:ff:ff 2023-11-24T06:12:38.714 INFO:teuthology.orchestra.run.smithi092.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-24T06:12:38.714 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:12:38.714 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 fe80::9055:2fff:fec5:ff24/64 scope link 2023-11-24T06:12:38.714 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:12:38.714 INFO:teuthology.orchestra.run.smithi092.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-24T06:12:38.715 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether fa:d3:03:06:91:4f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-24T06:12:38.715 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 fe80::f8d3:3ff:fe06:914f/64 scope link 2023-11-24T06:12:38.715 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:12:38.715 INFO:teuthology.orchestra.run.smithi092.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-24T06:12:38.715 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 7a:4d:f0:06:a7:4e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-24T06:12:38.715 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 fe80::784d:f0ff:fe06:a74e/64 scope link 2023-11-24T06:12:38.715 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:12:38.715 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:12:38.715 DEBUG:teuthology.orchestra.run.smithi092:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-24T06:12:38.802 INFO:teuthology.orchestra.run.smithi092.stdout:1 2023-11-24T06:12:38.806 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:12:38.806 DEBUG:teuthology.orchestra.run.smithi092:> route 2023-11-24T06:12:38.861 INFO:teuthology.orchestra.run.smithi092.stdout:Kernel IP routing table 2023-11-24T06:12:38.862 INFO:teuthology.orchestra.run.smithi092.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-24T06:12:38.862 INFO:teuthology.orchestra.run.smithi092.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-24T06:12:38.862 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-24T06:12:38.862 INFO:teuthology.orchestra.run.smithi092.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-24T06:12:38.862 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:12:38.862 DEBUG:teuthology.orchestra.run.smithi092:> (cd / && exec bash -c ' 2023-11-24T06:12:38.862 DEBUG:teuthology.orchestra.run.smithi092:> set -e 2023-11-24T06:12:38.863 DEBUG:teuthology.orchestra.run.smithi092:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-24T06:12:38.863 DEBUG:teuthology.orchestra.run.smithi092:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-24T06:12:38.863 DEBUG:teuthology.orchestra.run.smithi092:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-24T06:12:38.863 DEBUG:teuthology.orchestra.run.smithi092:> ') 2023-11-24T06:12:39.045 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:12:39.045 DEBUG:teuthology.orchestra.run.smithi092:> ip netns list 2023-11-24T06:12:39.098 INFO:teuthology.orchestra.run.smithi092.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-24T06:12:39.098 INFO:teuthology.orchestra.run.smithi092.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-24T06:12:39.099 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-24T06:12:39.099 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:12:39.099 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip link set brx.0 up 2023-11-24T06:12:39.196 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-11-24T06:12:39.196 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T06:12:39.196 DEBUG:teuthology.orchestra.run.smithi092:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-11-24T06:12:39.245 INFO:teuthology.orchestra.run.smithi092.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-11-24T06:12:39.245 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T06:12:39.245 DEBUG:teuthology.orchestra.run.smithi092:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-11-24T06:12:39.294 DEBUG:teuthology.orchestra.run.smithi092:> sudo modprobe fuse 2023-11-24T06:12:39.382 DEBUG:teuthology.orchestra.run.smithi092:> cat /proc/self/mounts | awk '{print $2}' 2023-11-24T06:12:39.435 INFO:teuthology.orchestra.run.smithi092.stdout:/proc 2023-11-24T06:12:39.435 INFO:teuthology.orchestra.run.smithi092.stdout:/sys 2023-11-24T06:12:39.435 INFO:teuthology.orchestra.run.smithi092.stdout:/dev 2023-11-24T06:12:39.435 INFO:teuthology.orchestra.run.smithi092.stdout:/sys/kernel/security 2023-11-24T06:12:39.435 INFO:teuthology.orchestra.run.smithi092.stdout:/dev/shm 2023-11-24T06:12:39.436 INFO:teuthology.orchestra.run.smithi092.stdout:/dev/pts 2023-11-24T06:12:39.436 INFO:teuthology.orchestra.run.smithi092.stdout:/run 2023-11-24T06:12:39.436 INFO:teuthology.orchestra.run.smithi092.stdout:/sys/fs/cgroup 2023-11-24T06:12:39.436 INFO:teuthology.orchestra.run.smithi092.stdout:/sys/fs/pstore 2023-11-24T06:12:39.436 INFO:teuthology.orchestra.run.smithi092.stdout:/ 2023-11-24T06:12:39.436 INFO:teuthology.orchestra.run.smithi092.stdout:/sys/fs/selinux 2023-11-24T06:12:39.436 INFO:teuthology.orchestra.run.smithi092.stdout:/proc/sys/fs/binfmt_misc 2023-11-24T06:12:39.436 INFO:teuthology.orchestra.run.smithi092.stdout:/dev/hugepages 2023-11-24T06:12:39.436 INFO:teuthology.orchestra.run.smithi092.stdout:/dev/mqueue 2023-11-24T06:12:39.436 INFO:teuthology.orchestra.run.smithi092.stdout:/sys/kernel/debug 2023-11-24T06:12:39.436 INFO:teuthology.orchestra.run.smithi092.stdout:/sys/kernel/tracing 2023-11-24T06:12:39.436 INFO:teuthology.orchestra.run.smithi092.stdout:/sys/fs/fuse/connections 2023-11-24T06:12:39.437 INFO:teuthology.orchestra.run.smithi092.stdout:/sys/kernel/config 2023-11-24T06:12:39.437 INFO:teuthology.orchestra.run.smithi092.stdout:/run/credentials/systemd-sysctl.service 2023-11-24T06:12:39.437 INFO:teuthology.orchestra.run.smithi092.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-24T06:12:39.437 INFO:teuthology.orchestra.run.smithi092.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-24T06:12:39.437 INFO:teuthology.orchestra.run.smithi092.stdout:/var/lib/nfs/rpc_pipefs 2023-11-24T06:12:39.437 INFO:teuthology.orchestra.run.smithi092.stdout:/run/user/1000 2023-11-24T06:12:39.437 INFO:teuthology.orchestra.run.smithi092.stdout:/var/lib/ceph 2023-11-24T06:12:39.437 INFO:teuthology.orchestra.run.smithi092.stdout:/var/lib/ceph/osd/ceph-0 2023-11-24T06:12:39.437 INFO:teuthology.orchestra.run.smithi092.stdout:/var/lib/ceph/osd/ceph-1 2023-11-24T06:12:39.437 INFO:teuthology.orchestra.run.smithi092.stdout:/var/lib/ceph/osd/ceph-2 2023-11-24T06:12:39.437 INFO:teuthology.orchestra.run.smithi092.stdout:/var/lib/ceph/osd/ceph-3 2023-11-24T06:12:39.437 INFO:teuthology.orchestra.run.smithi092.stdout:/run/netns 2023-11-24T06:12:39.438 INFO:teuthology.orchestra.run.smithi092.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-24T06:12:39.438 INFO:teuthology.orchestra.run.smithi092.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-24T06:12:39.438 INFO:teuthology.orchestra.run.smithi092.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-24T06:12:39.438 INFO:teuthology.orchestra.run.smithi092.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-24T06:12:39.439 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:12:39.439 DEBUG:teuthology.orchestra.run.smithi092:> ls /sys/fs/fuse/connections 2023-11-24T06:12:39.487 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-11-24T06:12:39.487 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-11-24T06:12:39.492 DEBUG:teuthology.orchestra.run.smithi092:> sudo modprobe fuse 2023-11-24T06:12:39.571 DEBUG:teuthology.orchestra.run.smithi092:> cat /proc/self/mounts | awk '{print $2}' 2023-11-24T06:12:39.614 INFO:teuthology.orchestra.run.smithi092.stdout:/proc 2023-11-24T06:12:39.614 INFO:teuthology.orchestra.run.smithi092.stdout:/sys 2023-11-24T06:12:39.614 INFO:teuthology.orchestra.run.smithi092.stdout:/dev 2023-11-24T06:12:39.614 INFO:teuthology.orchestra.run.smithi092.stdout:/sys/kernel/security 2023-11-24T06:12:39.614 INFO:teuthology.orchestra.run.smithi092.stdout:/dev/shm 2023-11-24T06:12:39.614 INFO:teuthology.orchestra.run.smithi092.stdout:/dev/pts 2023-11-24T06:12:39.615 INFO:teuthology.orchestra.run.smithi092.stdout:/run 2023-11-24T06:12:39.615 INFO:teuthology.orchestra.run.smithi092.stdout:/sys/fs/cgroup 2023-11-24T06:12:39.615 INFO:teuthology.orchestra.run.smithi092.stdout:/sys/fs/pstore 2023-11-24T06:12:39.615 INFO:teuthology.orchestra.run.smithi092.stdout:/ 2023-11-24T06:12:39.615 INFO:teuthology.orchestra.run.smithi092.stdout:/sys/fs/selinux 2023-11-24T06:12:39.615 INFO:teuthology.orchestra.run.smithi092.stdout:/proc/sys/fs/binfmt_misc 2023-11-24T06:12:39.615 INFO:teuthology.orchestra.run.smithi092.stdout:/dev/hugepages 2023-11-24T06:12:39.615 INFO:teuthology.orchestra.run.smithi092.stdout:/dev/mqueue 2023-11-24T06:12:39.615 INFO:teuthology.orchestra.run.smithi092.stdout:/sys/kernel/debug 2023-11-24T06:12:39.615 INFO:teuthology.orchestra.run.smithi092.stdout:/sys/kernel/tracing 2023-11-24T06:12:39.615 INFO:teuthology.orchestra.run.smithi092.stdout:/sys/fs/fuse/connections 2023-11-24T06:12:39.615 INFO:teuthology.orchestra.run.smithi092.stdout:/sys/kernel/config 2023-11-24T06:12:39.615 INFO:teuthology.orchestra.run.smithi092.stdout:/run/credentials/systemd-sysctl.service 2023-11-24T06:12:39.616 INFO:teuthology.orchestra.run.smithi092.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-24T06:12:39.616 INFO:teuthology.orchestra.run.smithi092.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-24T06:12:39.616 INFO:teuthology.orchestra.run.smithi092.stdout:/var/lib/nfs/rpc_pipefs 2023-11-24T06:12:39.616 INFO:teuthology.orchestra.run.smithi092.stdout:/run/user/1000 2023-11-24T06:12:39.616 INFO:teuthology.orchestra.run.smithi092.stdout:/var/lib/ceph 2023-11-24T06:12:39.616 INFO:teuthology.orchestra.run.smithi092.stdout:/var/lib/ceph/osd/ceph-0 2023-11-24T06:12:39.616 INFO:teuthology.orchestra.run.smithi092.stdout:/var/lib/ceph/osd/ceph-1 2023-11-24T06:12:39.616 INFO:teuthology.orchestra.run.smithi092.stdout:/var/lib/ceph/osd/ceph-2 2023-11-24T06:12:39.616 INFO:teuthology.orchestra.run.smithi092.stdout:/var/lib/ceph/osd/ceph-3 2023-11-24T06:12:39.616 INFO:teuthology.orchestra.run.smithi092.stdout:/run/netns 2023-11-24T06:12:39.616 INFO:teuthology.orchestra.run.smithi092.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-24T06:12:39.616 INFO:teuthology.orchestra.run.smithi092.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-24T06:12:39.617 INFO:teuthology.orchestra.run.smithi092.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-24T06:12:39.617 INFO:teuthology.orchestra.run.smithi092.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-24T06:12:39.617 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:12:39.618 DEBUG:teuthology.orchestra.run.smithi092:> ls /sys/fs/fuse/connections 2023-11-24T06:12:39.680 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi092.stderr:2023-11-24T06:12:39.677+0000 7f1447447480 -1 init, newargv = 0x55cddd824bb0 newargc=15 2023-11-24T06:12:39.680 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi092.stderr:ceph-fuse[106211]: starting ceph client 2023-11-24T06:12:39.689 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi092.stderr:ceph-fuse[106211]: starting fuse 2023-11-24T06:12:39.704 INFO:teuthology.orchestra.run.smithi092.stdout:40 2023-11-24T06:12:39.704 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [40] 2023-11-24T06:12:39.704 DEBUG:teuthology.orchestra.run.smithi092:> sudo stdin-killer -- python3 -c ' 2023-11-24T06:12:39.704 DEBUG:teuthology.orchestra.run.smithi092:> import glob 2023-11-24T06:12:39.704 DEBUG:teuthology.orchestra.run.smithi092:> import re 2023-11-24T06:12:39.704 DEBUG:teuthology.orchestra.run.smithi092:> import os 2023-11-24T06:12:39.704 DEBUG:teuthology.orchestra.run.smithi092:> import subprocess 2023-11-24T06:12:39.705 DEBUG:teuthology.orchestra.run.smithi092:> 2023-11-24T06:12:39.705 DEBUG:teuthology.orchestra.run.smithi092:> def _find_admin_socket(client_name): 2023-11-24T06:12:39.705 DEBUG:teuthology.orchestra.run.smithi092:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-11-24T06:12:39.705 DEBUG:teuthology.orchestra.run.smithi092:> files = glob.glob(asok_path) 2023-11-24T06:12:39.705 DEBUG:teuthology.orchestra.run.smithi092:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-11-24T06:12:39.705 DEBUG:teuthology.orchestra.run.smithi092:> 2023-11-24T06:12:39.705 DEBUG:teuthology.orchestra.run.smithi092:> # Given a non-glob path, it better be there 2023-11-24T06:12:39.705 DEBUG:teuthology.orchestra.run.smithi092:> if "*" not in asok_path: 2023-11-24T06:12:39.705 DEBUG:teuthology.orchestra.run.smithi092:> assert(len(files) == 1) 2023-11-24T06:12:39.705 DEBUG:teuthology.orchestra.run.smithi092:> return files[0] 2023-11-24T06:12:39.705 DEBUG:teuthology.orchestra.run.smithi092:> 2023-11-24T06:12:39.705 DEBUG:teuthology.orchestra.run.smithi092:> for f in files: 2023-11-24T06:12:39.705 DEBUG:teuthology.orchestra.run.smithi092:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-11-24T06:12:39.705 DEBUG:teuthology.orchestra.run.smithi092:> if os.path.exists("/proc/{0}".format(pid)): 2023-11-24T06:12:39.706 DEBUG:teuthology.orchestra.run.smithi092:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-11-24T06:12:39.706 DEBUG:teuthology.orchestra.run.smithi092:> contents = proc_f.read() 2023-11-24T06:12:39.706 DEBUG:teuthology.orchestra.run.smithi092:> if mountpoint in contents: 2023-11-24T06:12:39.706 DEBUG:teuthology.orchestra.run.smithi092:> return f 2023-11-24T06:12:39.706 DEBUG:teuthology.orchestra.run.smithi092:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-11-24T06:12:39.706 DEBUG:teuthology.orchestra.run.smithi092:> 2023-11-24T06:12:39.706 DEBUG:teuthology.orchestra.run.smithi092:> print(_find_admin_socket("client.0")) 2023-11-24T06:12:39.706 DEBUG:teuthology.orchestra.run.smithi092:> ' 2023-11-24T06:12:39.906 INFO:teuthology.orchestra.run.smithi092.stdout:/var/run/ceph/ceph-client.0.106211.asok 2023-11-24T06:12:39.911 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-24T06:12:39.923 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.106211.asok 2023-11-24T06:12:39.923 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:12:39.923 DEBUG:teuthology.orchestra.run.smithi092:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.106211.asok status 2023-11-24T06:12:40.155 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-24T06:12:40.155 INFO:teuthology.orchestra.run.smithi092.stdout: "metadata": { 2023-11-24T06:12:40.155 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph_sha1": "a13ea5ff00cb4df7a9cc941166ddeb358881144c", 2023-11-24T06:12:40.155 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph_version": "ceph version 18.0.0-7484-ga13ea5ff (a13ea5ff00cb4df7a9cc941166ddeb358881144c) reef (dev)", 2023-11-24T06:12:40.155 INFO:teuthology.orchestra.run.smithi092.stdout: "entity_id": "0", 2023-11-24T06:12:40.156 INFO:teuthology.orchestra.run.smithi092.stdout: "hostname": "smithi092", 2023-11-24T06:12:40.156 INFO:teuthology.orchestra.run.smithi092.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-11-24T06:12:40.156 INFO:teuthology.orchestra.run.smithi092.stdout: "pid": "106211", 2023-11-24T06:12:40.156 INFO:teuthology.orchestra.run.smithi092.stdout: "root": "/" 2023-11-24T06:12:40.156 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-24T06:12:40.156 INFO:teuthology.orchestra.run.smithi092.stdout: "dentry_count": 0, 2023-11-24T06:12:40.157 INFO:teuthology.orchestra.run.smithi092.stdout: "dentry_pinned_count": 0, 2023-11-24T06:12:40.157 INFO:teuthology.orchestra.run.smithi092.stdout: "id": 5871, 2023-11-24T06:12:40.157 INFO:teuthology.orchestra.run.smithi092.stdout: "inst": { 2023-11-24T06:12:40.157 INFO:teuthology.orchestra.run.smithi092.stdout: "name": { 2023-11-24T06:12:40.157 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "client", 2023-11-24T06:12:40.157 INFO:teuthology.orchestra.run.smithi092.stdout: "num": 5871 2023-11-24T06:12:40.158 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-24T06:12:40.158 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": { 2023-11-24T06:12:40.158 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2023-11-24T06:12:40.158 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "192.168.0.1:0", 2023-11-24T06:12:40.158 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 3803210259 2023-11-24T06:12:40.159 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-24T06:12:40.159 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-24T06:12:40.159 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": { 2023-11-24T06:12:40.159 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2023-11-24T06:12:40.159 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "192.168.0.1:0", 2023-11-24T06:12:40.159 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 3803210259 2023-11-24T06:12:40.160 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-24T06:12:40.160 INFO:teuthology.orchestra.run.smithi092.stdout: "inst_str": "client.5871 192.168.0.1:0/3803210259", 2023-11-24T06:12:40.160 INFO:teuthology.orchestra.run.smithi092.stdout: "addr_str": "192.168.0.1:0/3803210259", 2023-11-24T06:12:40.160 INFO:teuthology.orchestra.run.smithi092.stdout: "inode_count": 1, 2023-11-24T06:12:40.160 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_epoch": 31, 2023-11-24T06:12:40.160 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_epoch": 52, 2023-11-24T06:12:40.161 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_epoch_barrier": 0, 2023-11-24T06:12:40.161 INFO:teuthology.orchestra.run.smithi092.stdout: "blocklisted": false, 2023-11-24T06:12:40.161 INFO:teuthology.orchestra.run.smithi092.stdout: "fs_name": "cephfs" 2023-11-24T06:12:40.161 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-24T06:12:40.169 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:12:40.169 DEBUG:teuthology.orchestra.run.smithi092:> stat --file-system '--printf=%T 2023-11-24T06:12:40.170 DEBUG:teuthology.orchestra.run.smithi092:> ' -- /home/ubuntu/cephtest/mnt.0 2023-11-24T06:12:40.225 INFO:teuthology.orchestra.run.smithi092.stdout:fuseblk 2023-11-24T06:12:40.225 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-11-24T06:12:40.226 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:12:40.226 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-24T06:12:40.327 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-24T06:12:40.534 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:40.531+0000 7f3135d7c640 1 Processor -- start 2023-11-24T06:12:40.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:40.531+0000 7f3135d7c640 1 -- start start 2023-11-24T06:12:40.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:40.533+0000 7f3135d7c640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f313013e360 0x7f31300cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:40.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:40.533+0000 7f3135d7c640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f313015f9c0 0x7f31300cf7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:40.537 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:40.533+0000 7f3135d7c640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f313015c5f0 con 0x7f313013df80 2023-11-24T06:12:40.537 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:40.533+0000 7f3135d7c640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f313006c470 con 0x7f313013e360 2023-11-24T06:12:40.537 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:40.533+0000 7f3135d7c640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f313006c750 con 0x7f313015f9c0 2023-11-24T06:12:40.537 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:40.533+0000 7f312f7fe640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f313013df80 0x7f313015bee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:53584/0 (socket says 172.21.15.92:53584) 2023-11-24T06:12:40.537 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:40.533+0000 7f312f7fe640 1 -- 172.21.15.92:0/2550007471 learned_addr learned my addr 172.21.15.92:0/2550007471 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-24T06:12:40.537 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:40.533+0000 7f312effd640 1 --2- 172.21.15.92:0/2550007471 >> v2:172.21.15.92:3300/0 conn(0x7f313013e360 0x7f31300cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:40.538 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:40.534+0000 7f312ffff640 1 --2- 172.21.15.92:0/2550007471 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f313015f9c0 0x7f31300cf7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:40.538 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:40.534+0000 7f312e7fc640 1 -- 172.21.15.92:0/2550007471 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3773978196 0 0) 0x7f313015c5f0 con 0x7f313013df80 2023-11-24T06:12:40.538 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:40.534+0000 7f312e7fc640 1 -- 172.21.15.92:0/2550007471 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3118003520 con 0x7f313013df80 2023-11-24T06:12:40.538 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:40.534+0000 7f312effd640 1 -- 172.21.15.92:0/2550007471 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f313015f9c0 msgr2=0x7f31300cf7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:40.538 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:40.534+0000 7f312effd640 1 --2- 172.21.15.92:0/2550007471 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f313015f9c0 0x7f31300cf7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:40.538 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:40.534+0000 7f312effd640 1 -- 172.21.15.92:0/2550007471 >> v1:172.21.15.92:6789/0 conn(0x7f313013df80 legacy=0x7f313015bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:40.538 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:40.534+0000 7f312effd640 1 -- 172.21.15.92:0/2550007471 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f313006ca30 con 0x7f313013e360 2023-11-24T06:12:40.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:40.535+0000 7f312effd640 1 --2- 172.21.15.92:0/2550007471 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f313013e360 0x7f31300cf290 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f31240097a0 tx=0x7f312402b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95800968bbd5d83f server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:40.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:40.535+0000 7f312e7fc640 1 -- 172.21.15.92:0/2550007471 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f312402d040 con 0x7f313013e360 2023-11-24T06:12:40.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:40.535+0000 7f312e7fc640 1 -- 172.21.15.92:0/2550007471 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3124031810 con 0x7f313013e360 2023-11-24T06:12:40.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:40.535+0000 7f312e7fc640 1 -- 172.21.15.92:0/2550007471 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3124031b80 con 0x7f313013e360 2023-11-24T06:12:40.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:40.536+0000 7f3135d7c640 1 -- 172.21.15.92:0/2550007471 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f313013e360 msgr2=0x7f31300cf290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:40.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:40.536+0000 7f3135d7c640 1 --2- 172.21.15.92:0/2550007471 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f313013e360 0x7f31300cf290 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f31240097a0 tx=0x7f312402b8b0 comp rx=0 tx=0).stop 2023-11-24T06:12:40.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:40.536+0000 7f3135d7c640 1 -- 172.21.15.92:0/2550007471 shutdown_connections 2023-11-24T06:12:40.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:40.536+0000 7f3135d7c640 1 --2- 172.21.15.92:0/2550007471 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f313015f9c0 0x7f31300cf7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:40.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:40.536+0000 7f3135d7c640 1 --2- 172.21.15.92:0/2550007471 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f313013e360 0x7f31300cf290 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:40.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:40.536+0000 7f3135d7c640 1 -- 172.21.15.92:0/2550007471 >> 172.21.15.92:0/2550007471 conn(0x7f313006a800 msgr2=0x7f313006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:40.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:40.536+0000 7f3135d7c640 1 -- 172.21.15.92:0/2550007471 shutdown_connections 2023-11-24T06:12:40.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:40.536+0000 7f3135d7c640 1 -- 172.21.15.92:0/2550007471 wait complete. 2023-11-24T06:12:40.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:40.538+0000 7f3135d7c640 1 Processor -- start 2023-11-24T06:12:40.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:40.538+0000 7f3135d7c640 1 -- start start 2023-11-24T06:12:40.542 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:40.539+0000 7f3135d7c640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f313013df80 0x7f313014a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:40.542 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:40.539+0000 7f3135d7c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f313013e360 0x7f313014d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:40.542 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:40.539+0000 7f3135d7c640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f313015f9c0 0x7f31300b2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:40.543 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:40.539+0000 7f3135d7c640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f313014b2f0 con 0x7f313013e360 2023-11-24T06:12:40.543 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:40.539+0000 7f3135d7c640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f313014be70 con 0x7f313015f9c0 2023-11-24T06:12:40.543 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:40.539+0000 7f3135d7c640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f313014b540 con 0x7f313013df80 2023-11-24T06:12:40.543 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:40.539+0000 7f312effd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f313013e360 0x7f313014d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:40.543 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:40.539+0000 7f312effd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f313013e360 0x7f313014d630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58770/0 (socket says 172.21.15.92:58770) 2023-11-24T06:12:40.543 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:40.539+0000 7f312effd640 1 -- 172.21.15.92:0/886993018 learned_addr learned my addr 172.21.15.92:0/886993018 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:12:40.544 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:40.540+0000 7f312effd640 1 -- 172.21.15.92:0/886993018 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f313013df80 msgr2=0x7f313014a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:40.544 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:40.540+0000 7f312effd640 1 --2- 172.21.15.92:0/886993018 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f313013df80 0x7f313014a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:40.544 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:40.540+0000 7f312effd640 1 -- 172.21.15.92:0/886993018 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f313015f9c0 msgr2=0x7f31300b2cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:40.544 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:40.540+0000 7f312ffff640 1 --2- 172.21.15.92:0/886993018 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f313015f9c0 0x7f31300b2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:40.544 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:40.540+0000 7f312effd640 1 --2- 172.21.15.92:0/886993018 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f313015f9c0 0x7f31300b2cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:40.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:40.540+0000 7f312effd640 1 -- 172.21.15.92:0/886993018 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3124009400 con 0x7f313013e360 2023-11-24T06:12:40.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:40.540+0000 7f312effd640 1 --2- 172.21.15.92:0/886993018 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f313013e360 0x7f313014d630 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f31240098c0 tx=0x7f31240046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:40.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:40.540+0000 7f3117fff640 1 -- 172.21.15.92:0/886993018 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f312403ad60 con 0x7f313013e360 2023-11-24T06:12:40.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:40.540+0000 7f3117fff640 1 -- 172.21.15.92:0/886993018 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31240423f0 con 0x7f313013e360 2023-11-24T06:12:40.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:40.541+0000 7f3135d7c640 1 -- 172.21.15.92:0/886993018 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f313014b820 con 0x7f313013e360 2023-11-24T06:12:40.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:40.541+0000 7f3117fff640 1 -- 172.21.15.92:0/886993018 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3124042780 con 0x7f313013e360 2023-11-24T06:12:40.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:40.541+0000 7f3135d7c640 1 -- 172.21.15.92:0/886993018 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f313013d9c0 con 0x7f313013e360 2023-11-24T06:12:40.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:40.542+0000 7f3135d7c640 1 -- 172.21.15.92:0/886993018 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31300675d0 con 0x7f313013e360 2023-11-24T06:12:40.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:40.543+0000 7f3117fff640 1 -- 172.21.15.92:0/886993018 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f312404b3b0 con 0x7f313013e360 2023-11-24T06:12:40.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:40.544+0000 7f3117fff640 1 --2- 172.21.15.92:0/886993018 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f311c06ae40 0x7f311c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:40.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:40.544+0000 7f3117fff640 1 -- 172.21.15.92:0/886993018 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f31240ab790 con 0x7f313013e360 2023-11-24T06:12:40.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:40.544+0000 7f312f7fe640 1 --2- 172.21.15.92:0/886993018 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f311c06ae40 0x7f311c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:40.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:40.545+0000 7f312f7fe640 1 --2- 172.21.15.92:0/886993018 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f311c06ae40 0x7f311c06d300 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f3120000f60 tx=0x7f31200078f0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:40.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:40.548+0000 7f3117fff640 1 -- 172.21.15.92:0/886993018 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3124076830 con 0x7f313013e360 2023-11-24T06:12:40.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:40.778+0000 7f3135d7c640 1 -- 172.21.15.92:0/886993018 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f313013dd60 con 0x7f313013e360 2023-11-24T06:12:40.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:40.778+0000 7f3117fff640 1 -- 172.21.15.92:0/886993018 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v31) v1 ==== 53+0+73 (secure 0 0 0) 0x7f3124035060 con 0x7f313013e360 2023-11-24T06:12:40.782 INFO:teuthology.orchestra.run.smithi092.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-24T06:12:40.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:40.781+0000 7f3135d7c640 1 -- 172.21.15.92:0/886993018 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f311c06ae40 msgr2=0x7f311c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:40.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:40.781+0000 7f3135d7c640 1 --2- 172.21.15.92:0/886993018 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f311c06ae40 0x7f311c06d300 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f3120000f60 tx=0x7f31200078f0 comp rx=0 tx=0).stop 2023-11-24T06:12:40.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:40.781+0000 7f3135d7c640 1 -- 172.21.15.92:0/886993018 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f313013e360 msgr2=0x7f313014d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:40.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:40.781+0000 7f3135d7c640 1 --2- 172.21.15.92:0/886993018 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f313013e360 0x7f313014d630 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f31240098c0 tx=0x7f31240046c0 comp rx=0 tx=0).stop 2023-11-24T06:12:40.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:40.781+0000 7f3135d7c640 1 -- 172.21.15.92:0/886993018 shutdown_connections 2023-11-24T06:12:40.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:40.781+0000 7f3135d7c640 1 --2- 172.21.15.92:0/886993018 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f311c06ae40 0x7f311c06d300 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:40.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:40.781+0000 7f3135d7c640 1 --2- 172.21.15.92:0/886993018 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f313015f9c0 0x7f31300b2cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:40.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:40.781+0000 7f3135d7c640 1 --2- 172.21.15.92:0/886993018 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f313013e360 0x7f313014d630 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:40.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:40.781+0000 7f3135d7c640 1 --2- 172.21.15.92:0/886993018 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f313013df80 0x7f313014a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:40.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:40.781+0000 7f3135d7c640 1 -- 172.21.15.92:0/886993018 >> 172.21.15.92:0/886993018 conn(0x7f313006a800 msgr2=0x7f31300cdd40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:40.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:40.781+0000 7f3135d7c640 1 -- 172.21.15.92:0/886993018 shutdown_connections 2023-11-24T06:12:40.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:40.782+0000 7f3135d7c640 1 -- 172.21.15.92:0/886993018 wait complete. 2023-11-24T06:12:40.803 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-24T06:12:40.803 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-24T06:12:40.804 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi118.front.sepia.ceph.com 2023-11-24T06:12:40.804 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-11-24T06:12:40.804 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-11-24T06:12:40.804 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-24T06:12:40.804 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-24T06:12:40.804 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-24T06:12:40.804 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-11-24T06:12:40.804 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:12:40.804 DEBUG:teuthology.orchestra.run.smithi118:> ip addr 2023-11-24T06:12:40.865 INFO:teuthology.orchestra.run.smithi118.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-24T06:12:40.865 INFO:teuthology.orchestra.run.smithi118.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-24T06:12:40.865 INFO:teuthology.orchestra.run.smithi118.stdout: inet 127.0.0.1/8 scope host lo 2023-11-24T06:12:40.865 INFO:teuthology.orchestra.run.smithi118.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:12:40.865 INFO:teuthology.orchestra.run.smithi118.stdout: inet6 ::1/128 scope host 2023-11-24T06:12:40.865 INFO:teuthology.orchestra.run.smithi118.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:12:40.865 INFO:teuthology.orchestra.run.smithi118.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T06:12:40.865 INFO:teuthology.orchestra.run.smithi118.stdout: link/ether 0c:c4:7a:d8:a4:cc brd ff:ff:ff:ff:ff:ff 2023-11-24T06:12:40.865 INFO:teuthology.orchestra.run.smithi118.stdout: altname enp7s0f0 2023-11-24T06:12:40.865 INFO:teuthology.orchestra.run.smithi118.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T06:12:40.865 INFO:teuthology.orchestra.run.smithi118.stdout: link/ether 0c:c4:7a:d8:a4:cd brd ff:ff:ff:ff:ff:ff 2023-11-24T06:12:40.865 INFO:teuthology.orchestra.run.smithi118.stdout: altname enp7s0f1 2023-11-24T06:12:40.866 INFO:teuthology.orchestra.run.smithi118.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T06:12:40.866 INFO:teuthology.orchestra.run.smithi118.stdout: link/ether 0c:c4:7a:8f:d7:24 brd ff:ff:ff:ff:ff:ff 2023-11-24T06:12:40.866 INFO:teuthology.orchestra.run.smithi118.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-24T06:12:40.866 INFO:teuthology.orchestra.run.smithi118.stdout: link/ether 0c:c4:7a:8f:d7:25 brd ff:ff:ff:ff:ff:ff 2023-11-24T06:12:40.866 INFO:teuthology.orchestra.run.smithi118.stdout: inet 172.21.15.118/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-24T06:12:40.866 INFO:teuthology.orchestra.run.smithi118.stdout: valid_lft 42087sec preferred_lft 42087sec 2023-11-24T06:12:40.866 INFO:teuthology.orchestra.run.smithi118.stdout: inet6 fe80::ec4:7aff:fe8f:d725/64 scope link 2023-11-24T06:12:40.866 INFO:teuthology.orchestra.run.smithi118.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:12:40.866 INFO:teuthology.orchestra.run.smithi118.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-24T06:12:40.866 INFO:teuthology.orchestra.run.smithi118.stdout: link/ether 16:ef:1e:16:88:fb brd ff:ff:ff:ff:ff:ff 2023-11-24T06:12:40.866 INFO:teuthology.orchestra.run.smithi118.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-24T06:12:40.866 INFO:teuthology.orchestra.run.smithi118.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:12:40.867 INFO:teuthology.orchestra.run.smithi118.stdout: inet6 fe80::50a3:24ff:fea4:c41e/64 scope link 2023-11-24T06:12:40.867 INFO:teuthology.orchestra.run.smithi118.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:12:40.867 INFO:teuthology.orchestra.run.smithi118.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-24T06:12:40.867 INFO:teuthology.orchestra.run.smithi118.stdout: link/ether 16:ef:1e:16:88:fb brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-24T06:12:40.867 INFO:teuthology.orchestra.run.smithi118.stdout: inet6 fe80::14ef:1eff:fe16:88fb/64 scope link 2023-11-24T06:12:40.867 INFO:teuthology.orchestra.run.smithi118.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:12:40.867 INFO:teuthology.orchestra.run.smithi118.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-24T06:12:40.867 INFO:teuthology.orchestra.run.smithi118.stdout: link/ether 6a:b2:50:ae:7a:72 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-24T06:12:40.867 INFO:teuthology.orchestra.run.smithi118.stdout: inet6 fe80::68b2:50ff:feae:7a72/64 scope link 2023-11-24T06:12:40.867 INFO:teuthology.orchestra.run.smithi118.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:12:40.868 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:12:40.868 DEBUG:teuthology.orchestra.run.smithi118:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-24T06:12:40.962 INFO:teuthology.orchestra.run.smithi118.stdout:1 2023-11-24T06:12:40.966 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:12:40.966 DEBUG:teuthology.orchestra.run.smithi118:> route 2023-11-24T06:12:41.017 INFO:teuthology.orchestra.run.smithi118.stdout:Kernel IP routing table 2023-11-24T06:12:41.017 INFO:teuthology.orchestra.run.smithi118.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-24T06:12:41.017 INFO:teuthology.orchestra.run.smithi118.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-24T06:12:41.018 INFO:teuthology.orchestra.run.smithi118.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-24T06:12:41.018 INFO:teuthology.orchestra.run.smithi118.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-24T06:12:41.019 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:12:41.019 DEBUG:teuthology.orchestra.run.smithi118:> (cd / && exec bash -c ' 2023-11-24T06:12:41.019 DEBUG:teuthology.orchestra.run.smithi118:> set -e 2023-11-24T06:12:41.019 DEBUG:teuthology.orchestra.run.smithi118:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-24T06:12:41.019 DEBUG:teuthology.orchestra.run.smithi118:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-24T06:12:41.019 DEBUG:teuthology.orchestra.run.smithi118:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-24T06:12:41.019 DEBUG:teuthology.orchestra.run.smithi118:> ') 2023-11-24T06:12:41.205 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:12:41.206 DEBUG:teuthology.orchestra.run.smithi118:> ip netns list 2023-11-24T06:12:41.259 INFO:teuthology.orchestra.run.smithi118.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-11-24T06:12:41.259 INFO:teuthology.orchestra.run.smithi118.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-11-24T06:12:41.260 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-11-24T06:12:41.260 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:12:41.260 DEBUG:teuthology.orchestra.run.smithi118:> sudo ip link set brx.0 up 2023-11-24T06:12:41.356 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2023-11-24T06:12:41.356 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T06:12:41.356 DEBUG:teuthology.orchestra.run.smithi118:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-11-24T06:12:41.411 INFO:teuthology.orchestra.run.smithi118.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-11-24T06:12:41.411 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T06:12:41.411 DEBUG:teuthology.orchestra.run.smithi118:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-11-24T06:12:41.462 DEBUG:teuthology.orchestra.run.smithi118:> sudo modprobe fuse 2023-11-24T06:12:41.556 DEBUG:teuthology.orchestra.run.smithi118:> cat /proc/self/mounts | awk '{print $2}' 2023-11-24T06:12:41.615 INFO:teuthology.orchestra.run.smithi118.stdout:/proc 2023-11-24T06:12:41.615 INFO:teuthology.orchestra.run.smithi118.stdout:/sys 2023-11-24T06:12:41.615 INFO:teuthology.orchestra.run.smithi118.stdout:/dev 2023-11-24T06:12:41.615 INFO:teuthology.orchestra.run.smithi118.stdout:/sys/kernel/security 2023-11-24T06:12:41.615 INFO:teuthology.orchestra.run.smithi118.stdout:/dev/shm 2023-11-24T06:12:41.615 INFO:teuthology.orchestra.run.smithi118.stdout:/dev/pts 2023-11-24T06:12:41.616 INFO:teuthology.orchestra.run.smithi118.stdout:/run 2023-11-24T06:12:41.616 INFO:teuthology.orchestra.run.smithi118.stdout:/sys/fs/cgroup 2023-11-24T06:12:41.616 INFO:teuthology.orchestra.run.smithi118.stdout:/sys/fs/pstore 2023-11-24T06:12:41.616 INFO:teuthology.orchestra.run.smithi118.stdout:/ 2023-11-24T06:12:41.616 INFO:teuthology.orchestra.run.smithi118.stdout:/sys/fs/selinux 2023-11-24T06:12:41.616 INFO:teuthology.orchestra.run.smithi118.stdout:/proc/sys/fs/binfmt_misc 2023-11-24T06:12:41.616 INFO:teuthology.orchestra.run.smithi118.stdout:/dev/mqueue 2023-11-24T06:12:41.616 INFO:teuthology.orchestra.run.smithi118.stdout:/dev/hugepages 2023-11-24T06:12:41.616 INFO:teuthology.orchestra.run.smithi118.stdout:/sys/kernel/debug 2023-11-24T06:12:41.616 INFO:teuthology.orchestra.run.smithi118.stdout:/sys/kernel/tracing 2023-11-24T06:12:41.616 INFO:teuthology.orchestra.run.smithi118.stdout:/sys/fs/fuse/connections 2023-11-24T06:12:41.616 INFO:teuthology.orchestra.run.smithi118.stdout:/sys/kernel/config 2023-11-24T06:12:41.616 INFO:teuthology.orchestra.run.smithi118.stdout:/run/credentials/systemd-sysctl.service 2023-11-24T06:12:41.617 INFO:teuthology.orchestra.run.smithi118.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-24T06:12:41.617 INFO:teuthology.orchestra.run.smithi118.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-24T06:12:41.617 INFO:teuthology.orchestra.run.smithi118.stdout:/var/lib/nfs/rpc_pipefs 2023-11-24T06:12:41.617 INFO:teuthology.orchestra.run.smithi118.stdout:/run/user/1000 2023-11-24T06:12:41.617 INFO:teuthology.orchestra.run.smithi118.stdout:/var/lib/ceph 2023-11-24T06:12:41.617 INFO:teuthology.orchestra.run.smithi118.stdout:/var/lib/ceph/osd/ceph-4 2023-11-24T06:12:41.617 INFO:teuthology.orchestra.run.smithi118.stdout:/var/lib/ceph/osd/ceph-5 2023-11-24T06:12:41.617 INFO:teuthology.orchestra.run.smithi118.stdout:/var/lib/ceph/osd/ceph-6 2023-11-24T06:12:41.617 INFO:teuthology.orchestra.run.smithi118.stdout:/var/lib/ceph/osd/ceph-7 2023-11-24T06:12:41.617 INFO:teuthology.orchestra.run.smithi118.stdout:/run/netns 2023-11-24T06:12:41.617 INFO:teuthology.orchestra.run.smithi118.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-24T06:12:41.617 INFO:teuthology.orchestra.run.smithi118.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-24T06:12:41.618 INFO:teuthology.orchestra.run.smithi118.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-24T06:12:41.618 INFO:teuthology.orchestra.run.smithi118.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-24T06:12:41.618 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:12:41.619 DEBUG:teuthology.orchestra.run.smithi118:> ls /sys/fs/fuse/connections 2023-11-24T06:12:41.671 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-11-24T06:12:41.672 DEBUG:teuthology.orchestra.run.smithi118:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2023-11-24T06:12:41.677 DEBUG:teuthology.orchestra.run.smithi118:> sudo modprobe fuse 2023-11-24T06:12:41.759 DEBUG:teuthology.orchestra.run.smithi118:> cat /proc/self/mounts | awk '{print $2}' 2023-11-24T06:12:41.804 INFO:teuthology.orchestra.run.smithi118.stdout:/proc 2023-11-24T06:12:41.805 INFO:teuthology.orchestra.run.smithi118.stdout:/sys 2023-11-24T06:12:41.805 INFO:teuthology.orchestra.run.smithi118.stdout:/dev 2023-11-24T06:12:41.805 INFO:teuthology.orchestra.run.smithi118.stdout:/sys/kernel/security 2023-11-24T06:12:41.805 INFO:teuthology.orchestra.run.smithi118.stdout:/dev/shm 2023-11-24T06:12:41.805 INFO:teuthology.orchestra.run.smithi118.stdout:/dev/pts 2023-11-24T06:12:41.805 INFO:teuthology.orchestra.run.smithi118.stdout:/run 2023-11-24T06:12:41.805 INFO:teuthology.orchestra.run.smithi118.stdout:/sys/fs/cgroup 2023-11-24T06:12:41.805 INFO:teuthology.orchestra.run.smithi118.stdout:/sys/fs/pstore 2023-11-24T06:12:41.805 INFO:teuthology.orchestra.run.smithi118.stdout:/ 2023-11-24T06:12:41.805 INFO:teuthology.orchestra.run.smithi118.stdout:/sys/fs/selinux 2023-11-24T06:12:41.805 INFO:teuthology.orchestra.run.smithi118.stdout:/proc/sys/fs/binfmt_misc 2023-11-24T06:12:41.805 INFO:teuthology.orchestra.run.smithi118.stdout:/dev/mqueue 2023-11-24T06:12:41.806 INFO:teuthology.orchestra.run.smithi118.stdout:/dev/hugepages 2023-11-24T06:12:41.806 INFO:teuthology.orchestra.run.smithi118.stdout:/sys/kernel/debug 2023-11-24T06:12:41.806 INFO:teuthology.orchestra.run.smithi118.stdout:/sys/kernel/tracing 2023-11-24T06:12:41.806 INFO:teuthology.orchestra.run.smithi118.stdout:/sys/fs/fuse/connections 2023-11-24T06:12:41.806 INFO:teuthology.orchestra.run.smithi118.stdout:/sys/kernel/config 2023-11-24T06:12:41.806 INFO:teuthology.orchestra.run.smithi118.stdout:/run/credentials/systemd-sysctl.service 2023-11-24T06:12:41.806 INFO:teuthology.orchestra.run.smithi118.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-24T06:12:41.806 INFO:teuthology.orchestra.run.smithi118.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-24T06:12:41.806 INFO:teuthology.orchestra.run.smithi118.stdout:/var/lib/nfs/rpc_pipefs 2023-11-24T06:12:41.806 INFO:teuthology.orchestra.run.smithi118.stdout:/run/user/1000 2023-11-24T06:12:41.806 INFO:teuthology.orchestra.run.smithi118.stdout:/var/lib/ceph 2023-11-24T06:12:41.807 INFO:teuthology.orchestra.run.smithi118.stdout:/var/lib/ceph/osd/ceph-4 2023-11-24T06:12:41.807 INFO:teuthology.orchestra.run.smithi118.stdout:/var/lib/ceph/osd/ceph-5 2023-11-24T06:12:41.807 INFO:teuthology.orchestra.run.smithi118.stdout:/var/lib/ceph/osd/ceph-6 2023-11-24T06:12:41.807 INFO:teuthology.orchestra.run.smithi118.stdout:/var/lib/ceph/osd/ceph-7 2023-11-24T06:12:41.807 INFO:teuthology.orchestra.run.smithi118.stdout:/run/netns 2023-11-24T06:12:41.807 INFO:teuthology.orchestra.run.smithi118.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-24T06:12:41.807 INFO:teuthology.orchestra.run.smithi118.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-24T06:12:41.807 INFO:teuthology.orchestra.run.smithi118.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-24T06:12:41.807 INFO:teuthology.orchestra.run.smithi118.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-24T06:12:41.808 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:12:41.808 DEBUG:teuthology.orchestra.run.smithi118:> ls /sys/fs/fuse/connections 2023-11-24T06:12:41.866 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi118.stderr:2023-11-24T06:12:41.863+0000 7f33f3041480 -1 init, newargv = 0x557b15bbcac0 newargc=15 2023-11-24T06:12:41.866 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi118.stderr:ceph-fuse[94964]: starting ceph client 2023-11-24T06:12:41.878 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi118.stderr:ceph-fuse[94964]: starting fuse 2023-11-24T06:12:41.893 INFO:teuthology.orchestra.run.smithi118.stdout:41 2023-11-24T06:12:41.893 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2023-11-24T06:12:41.894 DEBUG:teuthology.orchestra.run.smithi118:> sudo stdin-killer -- python3 -c ' 2023-11-24T06:12:41.894 DEBUG:teuthology.orchestra.run.smithi118:> import glob 2023-11-24T06:12:41.894 DEBUG:teuthology.orchestra.run.smithi118:> import re 2023-11-24T06:12:41.894 DEBUG:teuthology.orchestra.run.smithi118:> import os 2023-11-24T06:12:41.894 DEBUG:teuthology.orchestra.run.smithi118:> import subprocess 2023-11-24T06:12:41.894 DEBUG:teuthology.orchestra.run.smithi118:> 2023-11-24T06:12:41.894 DEBUG:teuthology.orchestra.run.smithi118:> def _find_admin_socket(client_name): 2023-11-24T06:12:41.894 DEBUG:teuthology.orchestra.run.smithi118:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2023-11-24T06:12:41.894 DEBUG:teuthology.orchestra.run.smithi118:> files = glob.glob(asok_path) 2023-11-24T06:12:41.894 DEBUG:teuthology.orchestra.run.smithi118:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2023-11-24T06:12:41.894 DEBUG:teuthology.orchestra.run.smithi118:> 2023-11-24T06:12:41.895 DEBUG:teuthology.orchestra.run.smithi118:> # Given a non-glob path, it better be there 2023-11-24T06:12:41.895 DEBUG:teuthology.orchestra.run.smithi118:> if "*" not in asok_path: 2023-11-24T06:12:41.895 DEBUG:teuthology.orchestra.run.smithi118:> assert(len(files) == 1) 2023-11-24T06:12:41.895 DEBUG:teuthology.orchestra.run.smithi118:> return files[0] 2023-11-24T06:12:41.895 DEBUG:teuthology.orchestra.run.smithi118:> 2023-11-24T06:12:41.895 DEBUG:teuthology.orchestra.run.smithi118:> for f in files: 2023-11-24T06:12:41.895 DEBUG:teuthology.orchestra.run.smithi118:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-11-24T06:12:41.895 DEBUG:teuthology.orchestra.run.smithi118:> if os.path.exists("/proc/{0}".format(pid)): 2023-11-24T06:12:41.895 DEBUG:teuthology.orchestra.run.smithi118:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-11-24T06:12:41.895 DEBUG:teuthology.orchestra.run.smithi118:> contents = proc_f.read() 2023-11-24T06:12:41.895 DEBUG:teuthology.orchestra.run.smithi118:> if mountpoint in contents: 2023-11-24T06:12:41.895 DEBUG:teuthology.orchestra.run.smithi118:> return f 2023-11-24T06:12:41.895 DEBUG:teuthology.orchestra.run.smithi118:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-11-24T06:12:41.896 DEBUG:teuthology.orchestra.run.smithi118:> 2023-11-24T06:12:41.896 DEBUG:teuthology.orchestra.run.smithi118:> print(_find_admin_socket("client.1")) 2023-11-24T06:12:41.896 DEBUG:teuthology.orchestra.run.smithi118:> ' 2023-11-24T06:12:42.103 INFO:teuthology.orchestra.run.smithi118.stdout:/var/run/ceph/ceph-client.1.94964.asok 2023-11-24T06:12:42.108 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:12:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-24T06:12:42.120 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.94964.asok 2023-11-24T06:12:42.121 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:12:42.121 DEBUG:teuthology.orchestra.run.smithi118:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.94964.asok status 2023-11-24T06:12:42.365 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2023-11-24T06:12:42.365 INFO:teuthology.orchestra.run.smithi118.stdout: "metadata": { 2023-11-24T06:12:42.365 INFO:teuthology.orchestra.run.smithi118.stdout: "ceph_sha1": "a13ea5ff00cb4df7a9cc941166ddeb358881144c", 2023-11-24T06:12:42.365 INFO:teuthology.orchestra.run.smithi118.stdout: "ceph_version": "ceph version 18.0.0-7484-ga13ea5ff (a13ea5ff00cb4df7a9cc941166ddeb358881144c) reef (dev)", 2023-11-24T06:12:42.366 INFO:teuthology.orchestra.run.smithi118.stdout: "entity_id": "1", 2023-11-24T06:12:42.366 INFO:teuthology.orchestra.run.smithi118.stdout: "hostname": "smithi118", 2023-11-24T06:12:42.366 INFO:teuthology.orchestra.run.smithi118.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2023-11-24T06:12:42.366 INFO:teuthology.orchestra.run.smithi118.stdout: "pid": "94964", 2023-11-24T06:12:42.366 INFO:teuthology.orchestra.run.smithi118.stdout: "root": "/" 2023-11-24T06:12:42.366 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-11-24T06:12:42.366 INFO:teuthology.orchestra.run.smithi118.stdout: "dentry_count": 0, 2023-11-24T06:12:42.366 INFO:teuthology.orchestra.run.smithi118.stdout: "dentry_pinned_count": 0, 2023-11-24T06:12:42.366 INFO:teuthology.orchestra.run.smithi118.stdout: "id": 4838, 2023-11-24T06:12:42.366 INFO:teuthology.orchestra.run.smithi118.stdout: "inst": { 2023-11-24T06:12:42.366 INFO:teuthology.orchestra.run.smithi118.stdout: "name": { 2023-11-24T06:12:42.366 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "client", 2023-11-24T06:12:42.367 INFO:teuthology.orchestra.run.smithi118.stdout: "num": 4838 2023-11-24T06:12:42.367 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-11-24T06:12:42.367 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": { 2023-11-24T06:12:42.367 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v1", 2023-11-24T06:12:42.367 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "192.168.0.1:0", 2023-11-24T06:12:42.367 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 4079275662 2023-11-24T06:12:42.367 INFO:teuthology.orchestra.run.smithi118.stdout: } 2023-11-24T06:12:42.367 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-11-24T06:12:42.367 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": { 2023-11-24T06:12:42.367 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v1", 2023-11-24T06:12:42.367 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "192.168.0.1:0", 2023-11-24T06:12:42.368 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 4079275662 2023-11-24T06:12:42.368 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-11-24T06:12:42.368 INFO:teuthology.orchestra.run.smithi118.stdout: "inst_str": "client.4838 192.168.0.1:0/4079275662", 2023-11-24T06:12:42.368 INFO:teuthology.orchestra.run.smithi118.stdout: "addr_str": "192.168.0.1:0/4079275662", 2023-11-24T06:12:42.368 INFO:teuthology.orchestra.run.smithi118.stdout: "inode_count": 1, 2023-11-24T06:12:42.368 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_epoch": 31, 2023-11-24T06:12:42.368 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_epoch": 52, 2023-11-24T06:12:42.368 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_epoch_barrier": 0, 2023-11-24T06:12:42.368 INFO:teuthology.orchestra.run.smithi118.stdout: "blocklisted": false, 2023-11-24T06:12:42.368 INFO:teuthology.orchestra.run.smithi118.stdout: "fs_name": "cephfs" 2023-11-24T06:12:42.368 INFO:teuthology.orchestra.run.smithi118.stdout:} 2023-11-24T06:12:42.380 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:12:42.380 DEBUG:teuthology.orchestra.run.smithi118:> stat --file-system '--printf=%T 2023-11-24T06:12:42.380 DEBUG:teuthology.orchestra.run.smithi118:> ' -- /home/ubuntu/cephtest/mnt.1 2023-11-24T06:12:42.432 INFO:teuthology.orchestra.run.smithi118.stdout:fuseblk 2023-11-24T06:12:42.432 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2023-11-24T06:12:42.432 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:12:42.433 DEBUG:teuthology.orchestra.run.smithi118:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-11-24T06:12:42.520 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2023-11-24T06:12:42.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:42.751+0000 7f6255adf640 1 Processor -- start 2023-11-24T06:12:42.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:42.752+0000 7f6255adf640 1 -- start start 2023-11-24T06:12:42.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:42.753+0000 7f6255adf640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f625013cfa0 0x7f625015afb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:42.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:42.753+0000 7f6255adf640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f62500cd530 0x7f625013f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:42.758 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:42.753+0000 7f6255adf640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f62500cd9b0 0x7f625013fae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:42.758 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:42.753+0000 7f6255adf640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62500cdec0 con 0x7f62500cd9b0 2023-11-24T06:12:42.758 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:42.753+0000 7f6255adf640 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62500ce1a0 con 0x7f625013cfa0 2023-11-24T06:12:42.758 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:42.753+0000 7f6255adf640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62500ce480 con 0x7f62500cd530 2023-11-24T06:12:42.758 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:42.754+0000 7f62552de640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f62500cd9b0 0x7f625013fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:42.758 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:42.754+0000 7f62552de640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f62500cd9b0 0x7f625013fae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58786/0 (socket says 172.21.15.92:58786) 2023-11-24T06:12:42.758 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:42.754+0000 7f62552de640 1 -- 172.21.15.92:0/1907576440 learned_addr learned my addr 172.21.15.92:0/1907576440 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:12:42.758 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:42.754+0000 7f624ffff640 1 --2- 172.21.15.92:0/1907576440 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f62500cd530 0x7f625013f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:42.758 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:42.754+0000 7f6254add640 1 --2- 172.21.15.92:0/1907576440 >> v2:172.21.15.118:3300/0 conn(0x7f625013cfa0 0x7f625015afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:42.758 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:42.754+0000 7f62552de640 1 -- 172.21.15.92:0/1907576440 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f62500cd530 msgr2=0x7f625013f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:42.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:42.754+0000 7f62552de640 1 --2- 172.21.15.92:0/1907576440 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f62500cd530 0x7f625013f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:42.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:42.754+0000 7f62552de640 1 -- 172.21.15.92:0/1907576440 >> v2:172.21.15.118:3300/0 conn(0x7f625013cfa0 msgr2=0x7f625015afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:42.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:42.754+0000 7f62552de640 1 --2- 172.21.15.92:0/1907576440 >> v2:172.21.15.118:3300/0 conn(0x7f625013cfa0 0x7f625015afb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:42.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:42.754+0000 7f62552de640 1 -- 172.21.15.92:0/1907576440 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62500ce760 con 0x7f62500cd9b0 2023-11-24T06:12:42.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:42.754+0000 7f6254add640 1 --2- 172.21.15.92:0/1907576440 >> v2:172.21.15.118:3300/0 conn(0x7f625013cfa0 0x7f625015afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:12:42.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:42.755+0000 7f62552de640 1 --2- 172.21.15.92:0/1907576440 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f62500cd9b0 0x7f625013fae0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f625006cf00 tx=0x7f624802e060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9b27da6c3e83711 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:42.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:42.756+0000 7f624f7fe640 1 -- 172.21.15.92:0/1907576440 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f624802ea00 con 0x7f62500cd9b0 2023-11-24T06:12:42.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:42.756+0000 7f624f7fe640 1 -- 172.21.15.92:0/1907576440 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f624802eb60 con 0x7f62500cd9b0 2023-11-24T06:12:42.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:42.756+0000 7f624f7fe640 1 -- 172.21.15.92:0/1907576440 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6248009600 con 0x7f62500cd9b0 2023-11-24T06:12:42.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:42.756+0000 7f6255adf640 1 -- 172.21.15.92:0/1907576440 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f62500cd9b0 msgr2=0x7f625013fae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:42.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:42.756+0000 7f6255adf640 1 --2- 172.21.15.92:0/1907576440 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f62500cd9b0 0x7f625013fae0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f625006cf00 tx=0x7f624802e060 comp rx=0 tx=0).stop 2023-11-24T06:12:42.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:42.757+0000 7f6255adf640 1 -- 172.21.15.92:0/1907576440 shutdown_connections 2023-11-24T06:12:42.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:42.757+0000 7f6255adf640 1 --2- 172.21.15.92:0/1907576440 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f62500cd9b0 0x7f625013fae0 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:42.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:42.757+0000 7f6255adf640 1 --2- 172.21.15.92:0/1907576440 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f62500cd530 0x7f625013f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:42.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:42.757+0000 7f6255adf640 1 --2- 172.21.15.92:0/1907576440 >> v2:172.21.15.118:3300/0 conn(0x7f625013cfa0 0x7f625015afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:42.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:42.757+0000 7f6255adf640 1 -- 172.21.15.92:0/1907576440 >> 172.21.15.92:0/1907576440 conn(0x7f625006a800 msgr2=0x7f625006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:42.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:42.757+0000 7f6255adf640 1 -- 172.21.15.92:0/1907576440 shutdown_connections 2023-11-24T06:12:42.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:42.757+0000 7f6255adf640 1 -- 172.21.15.92:0/1907576440 wait complete. 2023-11-24T06:12:42.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:42.758+0000 7f6255adf640 1 Processor -- start 2023-11-24T06:12:42.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:42.759+0000 7f6255adf640 1 -- start start 2023-11-24T06:12:42.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:42.759+0000 7f6255adf640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f62500cd530 0x7f62500cb7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:42.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:42.759+0000 7f6255adf640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f62500cd9b0 0x7f625013a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:42.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:42.760+0000 7f6255adf640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f625013cfa0 0x7f62501f2ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:42.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:42.760+0000 7f6255adf640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62500cbce0 con 0x7f625013cfa0 2023-11-24T06:12:42.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:42.760+0000 7f6255adf640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f625013a3c0 con 0x7f62500cd9b0 2023-11-24T06:12:42.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:42.760+0000 7f6255adf640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f625013ad90 con 0x7f62500cd530 2023-11-24T06:12:42.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:42.760+0000 7f62552de640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f625013cfa0 0x7f62501f2ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:42.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:42.760+0000 7f62552de640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f625013cfa0 0x7f62501f2ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58800/0 (socket says 172.21.15.92:58800) 2023-11-24T06:12:42.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:42.760+0000 7f62552de640 1 -- 172.21.15.92:0/1921433412 learned_addr learned my addr 172.21.15.92:0/1921433412 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:12:42.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:42.760+0000 7f62552de640 1 -- 172.21.15.92:0/1921433412 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f62500cd530 msgr2=0x7f62500cb7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:42.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:42.760+0000 7f62552de640 1 --2- 172.21.15.92:0/1921433412 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f62500cd530 0x7f62500cb7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:42.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:42.760+0000 7f62552de640 1 -- 172.21.15.92:0/1921433412 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f62500cd9b0 msgr2=0x7f625013a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:42.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:42.760+0000 7f62552de640 1 --2- 172.21.15.92:0/1921433412 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f62500cd9b0 0x7f625013a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:42.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:42.760+0000 7f62552de640 1 -- 172.21.15.92:0/1921433412 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f624800bd40 con 0x7f625013cfa0 2023-11-24T06:12:42.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:42.760+0000 7f62552de640 1 --2- 172.21.15.92:0/1921433412 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f625013cfa0 0x7f62501f2ec0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f6248038040 tx=0x7f6248004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:42.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:42.761+0000 7f624d7fa640 1 -- 172.21.15.92:0/1921433412 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f624802ea00 con 0x7f625013cfa0 2023-11-24T06:12:42.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:42.761+0000 7f624d7fa640 1 -- 172.21.15.92:0/1921433412 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6248036800 con 0x7f625013cfa0 2023-11-24T06:12:42.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:42.761+0000 7f624d7fa640 1 -- 172.21.15.92:0/1921433412 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6248036bb0 con 0x7f625013cfa0 2023-11-24T06:12:42.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:42.761+0000 7f6255adf640 1 -- 172.21.15.92:0/1921433412 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6250139d90 con 0x7f625013cfa0 2023-11-24T06:12:42.766 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:42.761+0000 7f6255adf640 1 -- 172.21.15.92:0/1921433412 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62501f3d20 con 0x7f625013cfa0 2023-11-24T06:12:42.766 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:42.762+0000 7f6255adf640 1 -- 172.21.15.92:0/1921433412 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6250067450 con 0x7f625013cfa0 2023-11-24T06:12:42.766 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:42.763+0000 7f624d7fa640 1 -- 172.21.15.92:0/1921433412 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6248036d10 con 0x7f625013cfa0 2023-11-24T06:12:42.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:42.764+0000 7f624d7fa640 1 --2- 172.21.15.92:0/1921433412 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f623406ae70 0x7f623406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:42.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:42.765+0000 7f624d7fa640 1 -- 172.21.15.92:0/1921433412 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f62480ac8e0 con 0x7f625013cfa0 2023-11-24T06:12:42.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:42.765+0000 7f6254add640 1 --2- 172.21.15.92:0/1921433412 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f623406ae70 0x7f623406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:42.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:42.765+0000 7f6254add640 1 --2- 172.21.15.92:0/1921433412 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f623406ae70 0x7f623406d330 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f6244006b10 tx=0x7f6244008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:42.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:42.769+0000 7f624d7fa640 1 -- 172.21.15.92:0/1921433412 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6248077ac0 con 0x7f625013cfa0 2023-11-24T06:12:43.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:43.002+0000 7f6255adf640 1 -- 172.21.15.92:0/1921433412 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f625013a500 con 0x7f625013cfa0 2023-11-24T06:12:43.599 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:43.596+0000 7f624d7fa640 1 -- 172.21.15.92:0/1921433412 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v32) v1 ==== 136+0+0 (secure 0 0 0) 0x7f625013a500 con 0x7f625013cfa0 2023-11-24T06:12:43.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:43.599+0000 7f6255adf640 1 -- 172.21.15.92:0/1921433412 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f623406ae70 msgr2=0x7f623406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:43.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:43.599+0000 7f6255adf640 1 --2- 172.21.15.92:0/1921433412 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f623406ae70 0x7f623406d330 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f6244006b10 tx=0x7f6244008040 comp rx=0 tx=0).stop 2023-11-24T06:12:43.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:43.599+0000 7f6255adf640 1 -- 172.21.15.92:0/1921433412 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f625013cfa0 msgr2=0x7f62501f2ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:43.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:43.599+0000 7f6255adf640 1 --2- 172.21.15.92:0/1921433412 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f625013cfa0 0x7f62501f2ec0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f6248038040 tx=0x7f6248004a40 comp rx=0 tx=0).stop 2023-11-24T06:12:43.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:43.599+0000 7f6255adf640 1 -- 172.21.15.92:0/1921433412 shutdown_connections 2023-11-24T06:12:43.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:43.599+0000 7f6255adf640 1 --2- 172.21.15.92:0/1921433412 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f623406ae70 0x7f623406d330 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:43.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:43.599+0000 7f6255adf640 1 --2- 172.21.15.92:0/1921433412 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f625013cfa0 0x7f62501f2ec0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:43.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:43.599+0000 7f6255adf640 1 --2- 172.21.15.92:0/1921433412 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f62500cd9b0 0x7f625013a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:43.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:43.599+0000 7f6255adf640 1 --2- 172.21.15.92:0/1921433412 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f62500cd530 0x7f62500cb7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:43.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:43.599+0000 7f6255adf640 1 -- 172.21.15.92:0/1921433412 >> 172.21.15.92:0/1921433412 conn(0x7f625006a800 msgr2=0x7f625015b5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:43.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:43.599+0000 7f6255adf640 1 -- 172.21.15.92:0/1921433412 shutdown_connections 2023-11-24T06:12:43.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:43.599+0000 7f6255adf640 1 -- 172.21.15.92:0/1921433412 wait complete. 2023-11-24T06:12:43.621 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-11-24T06:12:43.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:43.826+0000 7fb9e06e0640 1 Processor -- start 2023-11-24T06:12:43.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:43.826+0000 7fb9e06e0640 1 -- start start 2023-11-24T06:12:43.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:43.827+0000 7fb9e06e0640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb9d815e3b0 0x7fb9d8164de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:43.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:43.827+0000 7fb9e06e0640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9d80b1ee0 con 0x7fb9d80cf9a0 2023-11-24T06:12:43.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:43.827+0000 7fb9e06e0640 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9d80b2780 con 0x7fb9d8143ae0 2023-11-24T06:12:43.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:43.827+0000 7fb9e06e0640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9d815e990 con 0x7fb9d815e3b0 2023-11-24T06:12:43.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:43.828+0000 7fb9ddc7b640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fb9d80cf9a0 0x7fb9d815dca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:53586/0 (socket says 172.21.15.92:53586) 2023-11-24T06:12:43.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:43.828+0000 7fb9ddc7b640 1 -- 172.21.15.92:0/3488024489 learned_addr learned my addr 172.21.15.92:0/3488024489 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-24T06:12:43.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:43.828+0000 7fb9dec7d640 1 --2- 172.21.15.92:0/3488024489 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb9d815e3b0 0x7fb9d8164de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:43.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:43.828+0000 7fb9dd47a640 1 -- 172.21.15.92:0/3488024489 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 197306678 0 0) 0x7fb9d80b1ee0 con 0x7fb9d80cf9a0 2023-11-24T06:12:43.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:43.828+0000 7fb9dd47a640 1 -- 172.21.15.92:0/3488024489 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb9c4003520 con 0x7fb9d80cf9a0 2023-11-24T06:12:43.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:43.828+0000 7fb9dd47a640 1 -- 172.21.15.92:0/3488024489 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2227849820 0 0) 0x7fb9c4003520 con 0x7fb9d80cf9a0 2023-11-24T06:12:43.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:43.828+0000 7fb9dd47a640 1 -- 172.21.15.92:0/3488024489 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb9d815e3b0 msgr2=0x7fb9d8164de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:43.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:43.828+0000 7fb9dd47a640 1 --2- 172.21.15.92:0/3488024489 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb9d815e3b0 0x7fb9d8164de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:43.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:43.828+0000 7fb9dd47a640 1 -- 172.21.15.92:0/3488024489 >> v1:172.21.15.118:6789/0 conn(0x7fb9d8143ae0 legacy=0x7fb9d80cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:43.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:43.828+0000 7fb9dd47a640 1 -- 172.21.15.92:0/3488024489 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9d80b2130 con 0x7fb9d80cf9a0 2023-11-24T06:12:43.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:43.828+0000 7fb9dd47a640 1 -- 172.21.15.92:0/3488024489 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2607661350 0 0) 0x7fb9c8001670 con 0x7fb9d80cf9a0 2023-11-24T06:12:43.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:43.829+0000 7fb9dd47a640 1 -- 172.21.15.92:0/3488024489 >> v1:172.21.15.92:6789/0 conn(0x7fb9d80cf9a0 legacy=0x7fb9d815dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:43.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:43.829+0000 7fb9dd47a640 1 --2- 172.21.15.92:0/3488024489 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb9c40039a0 0x7fb9c4003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:43.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:43.829+0000 7fb9dd47a640 1 --2- 172.21.15.92:0/3488024489 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb9d815e3b0 0x7fb9c40047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:43.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:43.829+0000 7fb9dd47a640 1 --2- 172.21.15.92:0/3488024489 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb9c4004d00 0x7fb9c4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:43.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:43.829+0000 7fb9dd47a640 1 -- 172.21.15.92:0/3488024489 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9d815e990 con 0x7fb9c40039a0 2023-11-24T06:12:43.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:43.829+0000 7fb9dd47a640 1 -- 172.21.15.92:0/3488024489 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9c400ba60 con 0x7fb9c4004d00 2023-11-24T06:12:43.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:43.829+0000 7fb9dd47a640 1 -- 172.21.15.92:0/3488024489 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9c400de30 con 0x7fb9d815e3b0 2023-11-24T06:12:43.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:43.829+0000 7fb9de47c640 1 --2- 172.21.15.92:0/3488024489 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb9c40039a0 0x7fb9c4003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:43.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:43.829+0000 7fb9dec7d640 1 --2- 172.21.15.92:0/3488024489 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb9c4004d00 0x7fb9c4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:43.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:43.829+0000 7fb9de47c640 1 -- 172.21.15.92:0/3488024489 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb9d815e3b0 msgr2=0x7fb9c40047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:43.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:43.829+0000 7fb9de47c640 1 --2- 172.21.15.92:0/3488024489 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb9d815e3b0 0x7fb9c40047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:43.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:43.829+0000 7fb9de47c640 1 -- 172.21.15.92:0/3488024489 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb9c4004d00 msgr2=0x7fb9c4005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:43.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:43.829+0000 7fb9de47c640 1 --2- 172.21.15.92:0/3488024489 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb9c4004d00 0x7fb9c4005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:43.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:43.829+0000 7fb9de47c640 1 -- 172.21.15.92:0/3488024489 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9c4003520 con 0x7fb9c40039a0 2023-11-24T06:12:43.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:43.830+0000 7fb9de47c640 1 --2- 172.21.15.92:0/3488024489 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb9c40039a0 0x7fb9c4003dc0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fb9d4001e20 tx=0x7fb9c40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75e7b4732652c36d server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:43.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:43.831+0000 7fb9dd47a640 1 -- 172.21.15.92:0/3488024489 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9d4003b70 con 0x7fb9c40039a0 2023-11-24T06:12:43.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:43.831+0000 7fb9dd47a640 1 -- 172.21.15.92:0/3488024489 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9d4003cd0 con 0x7fb9c40039a0 2023-11-24T06:12:43.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:43.831+0000 7fb9dd47a640 1 -- 172.21.15.92:0/3488024489 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9d4005310 con 0x7fb9c40039a0 2023-11-24T06:12:43.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:43.831+0000 7fb9e06e0640 1 -- 172.21.15.92:0/3488024489 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb9c40039a0 msgr2=0x7fb9c4003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:43.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:43.831+0000 7fb9e06e0640 1 --2- 172.21.15.92:0/3488024489 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb9c40039a0 0x7fb9c4003dc0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fb9d4001e20 tx=0x7fb9c40057f0 comp rx=0 tx=0).stop 2023-11-24T06:12:43.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:43.831+0000 7fb9dec7d640 1 -- 172.21.15.92:0/3488024489 reap_dead start 2023-11-24T06:12:43.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:43.831+0000 7fb9e06e0640 1 -- 172.21.15.92:0/3488024489 shutdown_connections 2023-11-24T06:12:43.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:43.831+0000 7fb9e06e0640 1 -- 172.21.15.92:0/3488024489 >> 172.21.15.92:0/3488024489 conn(0x7fb9d806a800 msgr2=0x7fb9d806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:43.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:43.831+0000 7fb9e06e0640 1 -- 172.21.15.92:0/3488024489 shutdown_connections 2023-11-24T06:12:43.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:43.832+0000 7fb9e06e0640 1 -- 172.21.15.92:0/3488024489 wait complete. 2023-11-24T06:12:43.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:43.833+0000 7fb9e06e0640 1 Processor -- start 2023-11-24T06:12:43.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:43.833+0000 7fb9e06e0640 1 -- start start 2023-11-24T06:12:43.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:43.834+0000 7fb9e06e0640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb9c40039a0 0x7fb9d8137e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:43.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:43.834+0000 7fb9e06e0640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb9d8159260 0x7fb9d81396f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:43.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:43.834+0000 7fb9e06e0640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb9d80cd790 0x7fb9d80cdc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:43.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:43.834+0000 7fb9e06e0640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9d80ce180 con 0x7fb9d8159260 2023-11-24T06:12:43.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:43.834+0000 7fb9e06e0640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9d81596e0 con 0x7fb9d80cd790 2023-11-24T06:12:43.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:43.834+0000 7fb9e06e0640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9d81eec90 con 0x7fb9c40039a0 2023-11-24T06:12:43.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:43.835+0000 7fb9ddc7b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb9d8159260 0x7fb9d81396f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:43.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:43.835+0000 7fb9ddc7b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb9d8159260 0x7fb9d81396f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58804/0 (socket says 172.21.15.92:58804) 2023-11-24T06:12:43.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:43.835+0000 7fb9ddc7b640 1 -- 172.21.15.92:0/1172040479 learned_addr learned my addr 172.21.15.92:0/1172040479 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:12:43.839 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:43.835+0000 7fb9de47c640 1 --2- 172.21.15.92:0/1172040479 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb9c40039a0 0x7fb9d8137e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:43.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:43.835+0000 7fb9dec7d640 1 --2- 172.21.15.92:0/1172040479 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb9d80cd790 0x7fb9d80cdc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:43.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:43.835+0000 7fb9ddc7b640 1 -- 172.21.15.92:0/1172040479 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb9c40039a0 msgr2=0x7fb9d8137e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:43.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:43.835+0000 7fb9ddc7b640 1 --2- 172.21.15.92:0/1172040479 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb9c40039a0 0x7fb9d8137e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:43.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:43.835+0000 7fb9ddc7b640 1 -- 172.21.15.92:0/1172040479 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb9d80cd790 msgr2=0x7fb9d80cdc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:43.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:43.835+0000 7fb9ddc7b640 1 --2- 172.21.15.92:0/1172040479 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb9d80cd790 0x7fb9d80cdc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:43.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:43.835+0000 7fb9ddc7b640 1 -- 172.21.15.92:0/1172040479 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9d8139c30 con 0x7fb9d8159260 2023-11-24T06:12:43.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:43.835+0000 7fb9ddc7b640 1 --2- 172.21.15.92:0/1172040479 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb9d8159260 0x7fb9d81396f0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fb9c800c6b0 tx=0x7fb9c800cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:43.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:43.836+0000 7fb9ceffd640 1 -- 172.21.15.92:0/1172040479 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9c80072d0 con 0x7fb9d8159260 2023-11-24T06:12:43.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:43.836+0000 7fb9ceffd640 1 -- 172.21.15.92:0/1172040479 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9c8007430 con 0x7fb9d8159260 2023-11-24T06:12:43.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:43.836+0000 7fb9e06e0640 1 -- 172.21.15.92:0/1172040479 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9d8159b90 con 0x7fb9d8159260 2023-11-24T06:12:43.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:43.836+0000 7fb9ceffd640 1 -- 172.21.15.92:0/1172040479 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9c80047e0 con 0x7fb9d8159260 2023-11-24T06:12:43.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:43.836+0000 7fb9e06e0640 1 -- 172.21.15.92:0/1172040479 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9d813a160 con 0x7fb9d8159260 2023-11-24T06:12:43.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:43.837+0000 7fb9e06e0640 1 -- 172.21.15.92:0/1172040479 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9d806c3f0 con 0x7fb9d8159260 2023-11-24T06:12:43.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:43.838+0000 7fb9ceffd640 1 -- 172.21.15.92:0/1172040479 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb9c8004940 con 0x7fb9d8159260 2023-11-24T06:12:43.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:43.839+0000 7fb9ceffd640 1 --2- 172.21.15.92:0/1172040479 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fb9bc06ae70 0x7fb9bc06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:43.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:43.839+0000 7fb9ceffd640 1 -- 172.21.15.92:0/1172040479 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb9c8088c00 con 0x7fb9d8159260 2023-11-24T06:12:43.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:43.839+0000 7fb9de47c640 1 --2- 172.21.15.92:0/1172040479 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fb9bc06ae70 0x7fb9bc06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:43.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:43.840+0000 7fb9de47c640 1 --2- 172.21.15.92:0/1172040479 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fb9bc06ae70 0x7fb9bc06d330 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fb9d4034000 tx=0x7fb9d4039040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:43.847 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:43.844+0000 7fb9ceffd640 1 -- 172.21.15.92:0/1172040479 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb9c8053c10 con 0x7fb9d8159260 2023-11-24T06:12:44.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:44.058+0000 7fb9e06e0640 1 -- 172.21.15.92:0/1172040479 --> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fb9d80675d0 con 0x7fb9bc06ae70 2023-11-24T06:12:44.064 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:44.062+0000 7fb9ceffd640 1 -- 172.21.15.92:0/1172040479 <== mgr.4101 v2:172.21.15.92:6800/784027401 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7fb9d80675d0 con 0x7fb9bc06ae70 2023-11-24T06:12:44.065 INFO:teuthology.orchestra.run.smithi092.stdout:[ 2023-11-24T06:12:44.065 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-24T06:12:44.065 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "cephfs" 2023-11-24T06:12:44.065 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-24T06:12:44.065 INFO:teuthology.orchestra.run.smithi092.stdout:] 2023-11-24T06:12:44.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:44.064+0000 7fb9e06e0640 1 -- 172.21.15.92:0/1172040479 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fb9bc06ae70 msgr2=0x7fb9bc06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:44.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:44.064+0000 7fb9e06e0640 1 --2- 172.21.15.92:0/1172040479 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fb9bc06ae70 0x7fb9bc06d330 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fb9d4034000 tx=0x7fb9d4039040 comp rx=0 tx=0).stop 2023-11-24T06:12:44.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:44.064+0000 7fb9e06e0640 1 -- 172.21.15.92:0/1172040479 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb9d8159260 msgr2=0x7fb9d81396f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:44.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:44.064+0000 7fb9e06e0640 1 --2- 172.21.15.92:0/1172040479 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb9d8159260 0x7fb9d81396f0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fb9c800c6b0 tx=0x7fb9c800cb70 comp rx=0 tx=0).stop 2023-11-24T06:12:44.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:44.064+0000 7fb9e06e0640 1 -- 172.21.15.92:0/1172040479 shutdown_connections 2023-11-24T06:12:44.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:44.064+0000 7fb9e06e0640 1 --2- 172.21.15.92:0/1172040479 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fb9bc06ae70 0x7fb9bc06d330 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:44.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:44.064+0000 7fb9e06e0640 1 --2- 172.21.15.92:0/1172040479 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb9d80cd790 0x7fb9d80cdc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:44.069 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:44.064+0000 7fb9e06e0640 1 --2- 172.21.15.92:0/1172040479 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb9d8159260 0x7fb9d81396f0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:44.069 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:44.064+0000 7fb9e06e0640 1 --2- 172.21.15.92:0/1172040479 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb9c40039a0 0x7fb9d8137e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:44.069 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:44.064+0000 7fb9e06e0640 1 -- 172.21.15.92:0/1172040479 >> 172.21.15.92:0/1172040479 conn(0x7fb9d806a800 msgr2=0x7fb9d80ce620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:44.069 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:44.065+0000 7fb9e06e0640 1 -- 172.21.15.92:0/1172040479 shutdown_connections 2023-11-24T06:12:44.069 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:44.065+0000 7fb9e06e0640 1 -- 172.21.15.92:0/1172040479 wait complete. 2023-11-24T06:12:44.087 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-24T06:12:44.304 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:44.301+0000 7f0ae4261640 1 Processor -- start 2023-11-24T06:12:44.304 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:44.301+0000 7f0ae4261640 1 -- start start 2023-11-24T06:12:44.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:44.303+0000 7f0ae4261640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f0adc13df80 0x7f0adc15bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:44.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:44.303+0000 7f0ae4261640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0adc0cf9a0 0x7f0adc0cd840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:44.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:44.303+0000 7f0ae4261640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0adc06c470 con 0x7f0adc13e360 2023-11-24T06:12:44.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:44.303+0000 7f0ae4261640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0adc06c750 con 0x7f0adc13df80 2023-11-24T06:12:44.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:44.303+0000 7f0ae4261640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0adc06ca30 con 0x7f0adc0cf9a0 2023-11-24T06:12:44.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:44.303+0000 7f0ae1ffd640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f0adc13df80 0x7f0adc15bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:44.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:44.303+0000 7f0ae17fc640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f0adc13e360 0x7f0adc0cf290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:53590/0 (socket says 172.21.15.92:53590) 2023-11-24T06:12:44.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:44.303+0000 7f0ae17fc640 1 -- 172.21.15.92:0/2747431952 learned_addr learned my addr 172.21.15.92:0/2747431952 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-24T06:12:44.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:44.304+0000 7f0ae1ffd640 1 -- 172.21.15.92:0/2747431952 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0adc0cf9a0 msgr2=0x7f0adc0cd840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:44.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:44.304+0000 7f0ae27fe640 1 --2- 172.21.15.92:0/2747431952 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0adc0cf9a0 0x7f0adc0cd840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:44.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:44.304+0000 7f0ae1ffd640 1 --2- 172.21.15.92:0/2747431952 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0adc0cf9a0 0x7f0adc0cd840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:44.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:44.304+0000 7f0ae1ffd640 1 -- 172.21.15.92:0/2747431952 >> v1:172.21.15.92:6789/0 conn(0x7f0adc13e360 legacy=0x7f0adc0cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:44.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:44.304+0000 7f0ae1ffd640 1 -- 172.21.15.92:0/2747431952 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0adc06cd10 con 0x7f0adc13df80 2023-11-24T06:12:44.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:44.304+0000 7f0ae0ffb640 1 -- 172.21.15.92:0/2747431952 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1869623481 0 0) 0x7f0adc06c470 con 0x7f0adc13e360 2023-11-24T06:12:44.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:44.304+0000 7f0ae27fe640 1 --2- 172.21.15.92:0/2747431952 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0adc0cf9a0 0x7f0adc0cd840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:12:44.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:44.305+0000 7f0ae1ffd640 1 --2- 172.21.15.92:0/2747431952 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0adc13df80 0x7f0adc15bee0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f0ad80097a0 tx=0x7f0ad802b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26275c66aba9d390 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:44.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:44.305+0000 7f0ae0ffb640 1 -- 172.21.15.92:0/2747431952 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ad802d040 con 0x7f0adc13df80 2023-11-24T06:12:44.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:44.305+0000 7f0ae0ffb640 1 -- 172.21.15.92:0/2747431952 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ad8031810 con 0x7f0adc13df80 2023-11-24T06:12:44.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:44.305+0000 7f0ae0ffb640 1 -- 172.21.15.92:0/2747431952 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ad8031ba0 con 0x7f0adc13df80 2023-11-24T06:12:44.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:44.306+0000 7f0ae4261640 1 -- 172.21.15.92:0/2747431952 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0adc13df80 msgr2=0x7f0adc15bee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:44.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:44.306+0000 7f0ae4261640 1 --2- 172.21.15.92:0/2747431952 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0adc13df80 0x7f0adc15bee0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f0ad80097a0 tx=0x7f0ad802b8b0 comp rx=0 tx=0).stop 2023-11-24T06:12:44.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:44.306+0000 7f0ae4261640 1 -- 172.21.15.92:0/2747431952 shutdown_connections 2023-11-24T06:12:44.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:44.306+0000 7f0ae4261640 1 --2- 172.21.15.92:0/2747431952 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0adc0cf9a0 0x7f0adc0cd840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:44.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:44.306+0000 7f0ae4261640 1 --2- 172.21.15.92:0/2747431952 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0adc13df80 0x7f0adc15bee0 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:44.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:44.306+0000 7f0ae4261640 1 -- 172.21.15.92:0/2747431952 >> 172.21.15.92:0/2747431952 conn(0x7f0adc06a800 msgr2=0x7f0adc06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:44.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:44.306+0000 7f0ae4261640 1 -- 172.21.15.92:0/2747431952 shutdown_connections 2023-11-24T06:12:44.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:44.306+0000 7f0ae4261640 1 -- 172.21.15.92:0/2747431952 wait complete. 2023-11-24T06:12:44.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:44.308+0000 7f0ae4261640 1 Processor -- start 2023-11-24T06:12:44.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:44.308+0000 7f0ae4261640 1 -- start start 2023-11-24T06:12:44.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:44.309+0000 7f0ae4261640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0adc0cf9a0 0x7f0adc0b2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:44.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:44.309+0000 7f0ae4261640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0adc13df80 0x7f0adc0b3230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:44.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:44.309+0000 7f0ae4261640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0adc13e360 0x7f0adc1f2060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:44.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:44.309+0000 7f0ae4261640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0adc150df0 con 0x7f0adc13e360 2023-11-24T06:12:44.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:44.309+0000 7f0ae4261640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0adc1510d0 con 0x7f0adc0cf9a0 2023-11-24T06:12:44.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:44.309+0000 7f0ae4261640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0adc1513b0 con 0x7f0adc13df80 2023-11-24T06:12:44.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:44.309+0000 7f0ae27fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0adc13e360 0x7f0adc1f2060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:44.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:44.309+0000 7f0ae27fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0adc13e360 0x7f0adc1f2060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58822/0 (socket says 172.21.15.92:58822) 2023-11-24T06:12:44.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:44.309+0000 7f0ae27fe640 1 -- 172.21.15.92:0/2074059537 learned_addr learned my addr 172.21.15.92:0/2074059537 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:12:44.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:44.309+0000 7f0ae17fc640 1 --2- 172.21.15.92:0/2074059537 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0adc13df80 0x7f0adc0b3230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:44.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:44.309+0000 7f0ae1ffd640 1 --2- 172.21.15.92:0/2074059537 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0adc0cf9a0 0x7f0adc0b2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:44.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:44.310+0000 7f0ae27fe640 1 -- 172.21.15.92:0/2074059537 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0adc13df80 msgr2=0x7f0adc0b3230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:44.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:44.310+0000 7f0ae27fe640 1 --2- 172.21.15.92:0/2074059537 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0adc13df80 0x7f0adc0b3230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:44.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:44.310+0000 7f0ae27fe640 1 -- 172.21.15.92:0/2074059537 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0adc0cf9a0 msgr2=0x7f0adc0b2cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:44.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:44.310+0000 7f0ae27fe640 1 --2- 172.21.15.92:0/2074059537 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0adc0cf9a0 0x7f0adc0b2cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:44.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:44.310+0000 7f0ae27fe640 1 -- 172.21.15.92:0/2074059537 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ad8009400 con 0x7f0adc13e360 2023-11-24T06:12:44.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:44.310+0000 7f0ae1ffd640 1 --2- 172.21.15.92:0/2074059537 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0adc0cf9a0 0x7f0adc0b2cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:12:44.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:44.310+0000 7f0ae27fe640 1 --2- 172.21.15.92:0/2074059537 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0adc13e360 0x7f0adc1f2060 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f0ad400cd80 tx=0x7f0ad400a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:44.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:44.310+0000 7f0ad27fc640 1 -- 172.21.15.92:0/2074059537 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ad400f680 con 0x7f0adc13e360 2023-11-24T06:12:44.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:44.310+0000 7f0ad27fc640 1 -- 172.21.15.92:0/2074059537 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ad400f800 con 0x7f0adc13e360 2023-11-24T06:12:44.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:44.310+0000 7f0ad27fc640 1 -- 172.21.15.92:0/2074059537 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ad400fbb0 con 0x7f0adc13e360 2023-11-24T06:12:44.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:44.310+0000 7f0ae4261640 1 -- 172.21.15.92:0/2074059537 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0adc151690 con 0x7f0adc13e360 2023-11-24T06:12:44.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:44.310+0000 7f0ae4261640 1 -- 172.21.15.92:0/2074059537 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0adc150980 con 0x7f0adc13e360 2023-11-24T06:12:44.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:44.312+0000 7f0ae4261640 1 -- 172.21.15.92:0/2074059537 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0adc151bb0 con 0x7f0adc13e360 2023-11-24T06:12:44.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:44.312+0000 7f0ad27fc640 1 -- 172.21.15.92:0/2074059537 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f0ad400b040 con 0x7f0adc13e360 2023-11-24T06:12:44.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:44.313+0000 7f0ad27fc640 1 --2- 172.21.15.92:0/2074059537 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f0ac006ae40 0x7f0ac006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:44.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:44.313+0000 7f0ad27fc640 1 -- 172.21.15.92:0/2074059537 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0ad4089ff0 con 0x7f0adc13e360 2023-11-24T06:12:44.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:44.314+0000 7f0ae1ffd640 1 --2- 172.21.15.92:0/2074059537 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f0ac006ae40 0x7f0ac006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:44.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:44.314+0000 7f0ae1ffd640 1 --2- 172.21.15.92:0/2074059537 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f0ac006ae40 0x7f0ac006d300 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f0ad8036000 tx=0x7f0ad8008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:44.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:44.318+0000 7f0ad27fc640 1 -- 172.21.15.92:0/2074059537 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0adc151bb0 con 0x7f0adc13e360 2023-11-24T06:12:44.511 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:44.508+0000 7f0ae4261640 1 -- 172.21.15.92:0/2074059537 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f0adc151bb0 con 0x7f0adc13e360 2023-11-24T06:12:44.511 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:44.508+0000 7f0ad27fc640 1 -- 172.21.15.92:0/2074059537 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v12)=0 v12) v1 ==== 122+0+0 (secure 0 0 0) 0x7f0adc151bb0 con 0x7f0adc13e360 2023-11-24T06:12:44.514 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:44.511+0000 7f0ae4261640 1 -- 172.21.15.92:0/2074059537 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f0ac006ae40 msgr2=0x7f0ac006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:44.515 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:44.511+0000 7f0ae4261640 1 --2- 172.21.15.92:0/2074059537 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f0ac006ae40 0x7f0ac006d300 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f0ad8036000 tx=0x7f0ad8008030 comp rx=0 tx=0).stop 2023-11-24T06:12:44.515 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:44.511+0000 7f0ae4261640 1 -- 172.21.15.92:0/2074059537 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0adc13e360 msgr2=0x7f0adc1f2060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:44.515 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:44.511+0000 7f0ae4261640 1 --2- 172.21.15.92:0/2074059537 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0adc13e360 0x7f0adc1f2060 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f0ad400cd80 tx=0x7f0ad400a650 comp rx=0 tx=0).stop 2023-11-24T06:12:44.515 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:44.511+0000 7f0ae4261640 1 -- 172.21.15.92:0/2074059537 shutdown_connections 2023-11-24T06:12:44.515 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:44.511+0000 7f0ae4261640 1 --2- 172.21.15.92:0/2074059537 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f0ac006ae40 0x7f0ac006d300 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:44.515 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:44.511+0000 7f0ae4261640 1 --2- 172.21.15.92:0/2074059537 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0adc13e360 0x7f0adc1f2060 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:44.515 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:44.511+0000 7f0ae4261640 1 --2- 172.21.15.92:0/2074059537 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0adc13df80 0x7f0adc0b3230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:44.515 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:44.511+0000 7f0ae4261640 1 --2- 172.21.15.92:0/2074059537 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0adc0cf9a0 0x7f0adc0b2cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:44.515 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:44.511+0000 7f0ae4261640 1 -- 172.21.15.92:0/2074059537 >> 172.21.15.92:0/2074059537 conn(0x7f0adc06a800 msgr2=0x7f0adc0cf260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:44.515 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:44.511+0000 7f0ae4261640 1 -- 172.21.15.92:0/2074059537 shutdown_connections 2023-11-24T06:12:44.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:44.511+0000 7f0ae4261640 1 -- 172.21.15.92:0/2074059537 wait complete. 2023-11-24T06:12:44.535 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create cephfs subvol_grp7006 2023-11-24T06:12:44.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:44.744+0000 7f7a7027c640 1 Processor -- start 2023-11-24T06:12:44.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:44.744+0000 7f7a7027c640 1 -- start start 2023-11-24T06:12:44.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:44.746+0000 7f7a7027c640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7a6813df80 0x7f7a6815bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:44.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:44.746+0000 7f7a7027c640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f7a680cf9a0 0x7f7a680cd840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:44.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:44.746+0000 7f7a7027c640 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a6806c470 con 0x7f7a6813e360 2023-11-24T06:12:44.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:44.746+0000 7f7a7027c640 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a6806c750 con 0x7f7a680cf9a0 2023-11-24T06:12:44.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:44.746+0000 7f7a7027c640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a6806ca30 con 0x7f7a6813df80 2023-11-24T06:12:44.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:44.747+0000 7f7a6e819640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f7a680cf9a0 0x7f7a680cd840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:44.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:44.747+0000 7f7a6e018640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7a6813df80 0x7f7a6815bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:44.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:44.747+0000 7f7a6e819640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f7a680cf9a0 0x7f7a680cd840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.92:44024/0 (socket says 172.21.15.92:44024) 2023-11-24T06:12:44.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:44.747+0000 7f7a6e819640 1 -- 172.21.15.92:0/1439164039 learned_addr learned my addr 172.21.15.92:0/1439164039 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:12:44.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:44.747+0000 7f7a6e819640 1 -- 172.21.15.92:0/1439164039 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7a6813df80 msgr2=0x7f7a6815bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:44.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:44.747+0000 7f7a6e819640 1 --2- 172.21.15.92:0/1439164039 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7a6813df80 0x7f7a6815bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:44.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:44.747+0000 7f7a6e819640 1 -- 172.21.15.92:0/1439164039 >> v1:172.21.15.118:6789/0 conn(0x7f7a6813e360 legacy=0x7f7a680cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:44.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:44.747+0000 7f7a6e819640 1 -- 172.21.15.92:0/1439164039 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a6806cd10 con 0x7f7a680cf9a0 2023-11-24T06:12:44.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:44.749+0000 7f7a6e819640 1 --2- 172.21.15.92:0/1439164039 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7a680cf9a0 0x7f7a680cd840 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f7a60009870 tx=0x7f7a6002b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=73844f9418f97b84 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:44.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:44.749+0000 7f7a6d016640 1 -- 172.21.15.92:0/1439164039 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a60030840 con 0x7f7a680cf9a0 2023-11-24T06:12:44.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:44.749+0000 7f7a6d016640 1 -- 172.21.15.92:0/1439164039 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a600309a0 con 0x7f7a680cf9a0 2023-11-24T06:12:44.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:44.749+0000 7f7a6d016640 1 -- 172.21.15.92:0/1439164039 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a60030d30 con 0x7f7a680cf9a0 2023-11-24T06:12:44.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:44.750+0000 7f7a7027c640 1 -- 172.21.15.92:0/1439164039 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7a680cf9a0 msgr2=0x7f7a680cd840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:44.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:44.750+0000 7f7a7027c640 1 --2- 172.21.15.92:0/1439164039 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7a680cf9a0 0x7f7a680cd840 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f7a60009870 tx=0x7f7a6002b980 comp rx=0 tx=0).stop 2023-11-24T06:12:44.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:44.750+0000 7f7a7027c640 1 -- 172.21.15.92:0/1439164039 shutdown_connections 2023-11-24T06:12:44.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:44.750+0000 7f7a7027c640 1 --2- 172.21.15.92:0/1439164039 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7a680cf9a0 0x7f7a680cd840 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:44.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:44.750+0000 7f7a7027c640 1 --2- 172.21.15.92:0/1439164039 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7a6813df80 0x7f7a6815bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:44.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:44.750+0000 7f7a7027c640 1 -- 172.21.15.92:0/1439164039 >> 172.21.15.92:0/1439164039 conn(0x7f7a6806a800 msgr2=0x7f7a6806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:44.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:44.750+0000 7f7a7027c640 1 -- 172.21.15.92:0/1439164039 shutdown_connections 2023-11-24T06:12:44.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:44.750+0000 7f7a7027c640 1 -- 172.21.15.92:0/1439164039 wait complete. 2023-11-24T06:12:44.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:44.752+0000 7f7a7027c640 1 Processor -- start 2023-11-24T06:12:44.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:44.752+0000 7f7a7027c640 1 -- start start 2023-11-24T06:12:44.756 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:44.753+0000 7f7a7027c640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7a680cf9a0 0x7f7a681f6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:44.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:44.753+0000 7f7a7027c640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7a6813df80 0x7f7a68150840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:44.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:44.753+0000 7f7a7027c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7a6813e360 0x7f7a680b2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:44.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:44.753+0000 7f7a7027c640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a681533f0 con 0x7f7a6813e360 2023-11-24T06:12:44.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:44.753+0000 7f7a7027c640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a680b4300 con 0x7f7a680cf9a0 2023-11-24T06:12:44.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:44.753+0000 7f7a7027c640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a680b3d40 con 0x7f7a6813df80 2023-11-24T06:12:44.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:44.753+0000 7f7a6e819640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7a6813e360 0x7f7a680b2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:44.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:44.753+0000 7f7a6e819640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7a6813e360 0x7f7a680b2cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58832/0 (socket says 172.21.15.92:58832) 2023-11-24T06:12:44.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:44.753+0000 7f7a6e819640 1 -- 172.21.15.92:0/98828342 learned_addr learned my addr 172.21.15.92:0/98828342 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:12:44.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:44.753+0000 7f7a6e018640 1 --2- 172.21.15.92:0/98828342 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7a680cf9a0 0x7f7a681f6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:44.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:44.753+0000 7f7a6d817640 1 --2- 172.21.15.92:0/98828342 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7a6813df80 0x7f7a68150840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:44.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:44.754+0000 7f7a6e819640 1 -- 172.21.15.92:0/98828342 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7a6813df80 msgr2=0x7f7a68150840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:44.758 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:44.754+0000 7f7a6e819640 1 --2- 172.21.15.92:0/98828342 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7a6813df80 0x7f7a68150840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:44.758 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:44.754+0000 7f7a6e819640 1 -- 172.21.15.92:0/98828342 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7a680cf9a0 msgr2=0x7f7a681f6240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:44.758 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:44.754+0000 7f7a6e819640 1 --2- 172.21.15.92:0/98828342 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7a680cf9a0 0x7f7a681f6240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:44.758 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:44.754+0000 7f7a6e819640 1 -- 172.21.15.92:0/98828342 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a600094d0 con 0x7f7a6813e360 2023-11-24T06:12:44.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:44.754+0000 7f7a6e819640 1 --2- 172.21.15.92:0/98828342 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7a6813e360 0x7f7a680b2cf0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f7a60009870 tx=0x7f7a60004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:44.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:44.754+0000 7f7a5effd640 1 -- 172.21.15.92:0/98828342 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a60039a90 con 0x7f7a6813e360 2023-11-24T06:12:44.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:44.754+0000 7f7a5effd640 1 -- 172.21.15.92:0/98828342 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a60039c10 con 0x7f7a6813e360 2023-11-24T06:12:44.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:44.754+0000 7f7a7027c640 1 -- 172.21.15.92:0/98828342 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a680b4020 con 0x7f7a6813e360 2023-11-24T06:12:44.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:44.754+0000 7f7a5effd640 1 -- 172.21.15.92:0/98828342 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a60043410 con 0x7f7a6813e360 2023-11-24T06:12:44.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:44.754+0000 7f7a7027c640 1 -- 172.21.15.92:0/98828342 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a68130ce0 con 0x7f7a6813e360 2023-11-24T06:12:44.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:44.755+0000 7f7a7027c640 1 -- 172.21.15.92:0/98828342 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a68153560 con 0x7f7a6813e360 2023-11-24T06:12:44.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:44.756+0000 7f7a5effd640 1 -- 172.21.15.92:0/98828342 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f7a6002f390 con 0x7f7a6813e360 2023-11-24T06:12:44.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:44.757+0000 7f7a5effd640 1 --2- 172.21.15.92:0/98828342 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f7a4c06ae40 0x7f7a4c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:44.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:44.757+0000 7f7a5effd640 1 -- 172.21.15.92:0/98828342 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7a600ab620 con 0x7f7a6813e360 2023-11-24T06:12:44.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:44.757+0000 7f7a6e018640 1 --2- 172.21.15.92:0/98828342 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f7a4c06ae40 0x7f7a4c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:44.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:44.758+0000 7f7a6e018640 1 --2- 172.21.15.92:0/98828342 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f7a4c06ae40 0x7f7a4c06d300 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f7a640074a0 tx=0x7f7a6400b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:44.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:44.762+0000 7f7a5effd640 1 -- 172.21.15.92:0/98828342 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7a60076760 con 0x7f7a6813e360 2023-11-24T06:12:44.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:44.978+0000 7f7a7027c640 1 -- 172.21.15.92:0/98828342 --> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "subvol_grp7006", "target": ["mon-mgr", ""]}) v1 -- 0x7f7a6815eac0 con 0x7f7a4c06ae40 2023-11-24T06:12:44.993 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-11-24T06:12:44.990+0000 7ff62b16c640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T06:12:44.993 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-11-24T06:12:44.990+0000 7ff62b16c640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T06:12:44.993 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-11-24T06:12:44.990+0000 7ff62b16c640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T06:12:44.993 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-11-24T06:12:44.990+0000 7ff62b16c640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T06:12:44.993 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-11-24T06:12:44.990+0000 7ff62b16c640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T06:12:45.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:45.014+0000 7f7a5effd640 1 -- 172.21.15.92:0/98828342 <== mgr.4101 v2:172.21.15.92:6800/784027401 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f7a6815eac0 con 0x7f7a4c06ae40 2023-11-24T06:12:45.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:45.017+0000 7f7a7027c640 1 -- 172.21.15.92:0/98828342 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f7a4c06ae40 msgr2=0x7f7a4c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:45.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:45.017+0000 7f7a7027c640 1 --2- 172.21.15.92:0/98828342 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f7a4c06ae40 0x7f7a4c06d300 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f7a640074a0 tx=0x7f7a6400b040 comp rx=0 tx=0).stop 2023-11-24T06:12:45.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:45.017+0000 7f7a7027c640 1 -- 172.21.15.92:0/98828342 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7a6813e360 msgr2=0x7f7a680b2cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:45.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:45.017+0000 7f7a7027c640 1 --2- 172.21.15.92:0/98828342 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7a6813e360 0x7f7a680b2cf0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f7a60009870 tx=0x7f7a60004730 comp rx=0 tx=0).stop 2023-11-24T06:12:45.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:45.017+0000 7f7a7027c640 1 -- 172.21.15.92:0/98828342 shutdown_connections 2023-11-24T06:12:45.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:45.017+0000 7f7a7027c640 1 --2- 172.21.15.92:0/98828342 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f7a4c06ae40 0x7f7a4c06d300 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:45.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:45.017+0000 7f7a7027c640 1 --2- 172.21.15.92:0/98828342 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7a6813e360 0x7f7a680b2cf0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:45.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:45.017+0000 7f7a7027c640 1 --2- 172.21.15.92:0/98828342 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7a6813df80 0x7f7a68150840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:45.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:45.017+0000 7f7a7027c640 1 --2- 172.21.15.92:0/98828342 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7a680cf9a0 0x7f7a681f6240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:45.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:45.017+0000 7f7a7027c640 1 -- 172.21.15.92:0/98828342 >> 172.21.15.92:0/98828342 conn(0x7f7a6806a800 msgr2=0x7f7a68154980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:45.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:45.017+0000 7f7a7027c640 1 -- 172.21.15.92:0/98828342 shutdown_connections 2023-11-24T06:12:45.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:45.017+0000 7f7a7027c640 1 -- 172.21.15.92:0/98828342 wait complete. 2023-11-24T06:12:45.038 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume info cephfs 2023-11-24T06:12:45.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:45.223+0000 7f2875790640 1 Processor -- start 2023-11-24T06:12:45.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:45.223+0000 7f2875790640 1 -- start start 2023-11-24T06:12:45.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:45.225+0000 7f2875790640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f2870143ae0 0x7f28700cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:45.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:45.225+0000 7f2875790640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f287015e3b0 0x7f28700702c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:45.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:45.225+0000 7f2875790640 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28700cfc30 con 0x7f28700cf7d0 2023-11-24T06:12:45.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:45.225+0000 7f2875790640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28700b1ee0 con 0x7f287015e3b0 2023-11-24T06:12:45.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:45.225+0000 7f2875790640 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28700b2780 con 0x7f2870143ae0 2023-11-24T06:12:45.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:45.226+0000 7f286f7fe640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f287015e3b0 0x7f28700702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:45.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:45.226+0000 7f286f7fe640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f287015e3b0 0x7f28700702c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58856/0 (socket says 172.21.15.92:58856) 2023-11-24T06:12:45.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:45.226+0000 7f286f7fe640 1 -- 172.21.15.92:0/3045542401 learned_addr learned my addr 172.21.15.92:0/3045542401 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:12:45.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:45.226+0000 7f286effd640 1 --2- 172.21.15.92:0/3045542401 >> v2:172.21.15.118:3300/0 conn(0x7f2870143ae0 0x7f28700cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:45.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:45.226+0000 7f286f7fe640 1 -- 172.21.15.92:0/3045542401 >> v2:172.21.15.118:3300/0 conn(0x7f2870143ae0 msgr2=0x7f28700cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:45.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:45.226+0000 7f286f7fe640 1 --2- 172.21.15.92:0/3045542401 >> v2:172.21.15.118:3300/0 conn(0x7f2870143ae0 0x7f28700cf290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:45.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:45.226+0000 7f286f7fe640 1 -- 172.21.15.92:0/3045542401 >> v1:172.21.15.118:6789/0 conn(0x7f28700cf7d0 legacy=0x7f287015dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:45.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:45.226+0000 7f286f7fe640 1 -- 172.21.15.92:0/3045542401 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f287015e990 con 0x7f287015e3b0 2023-11-24T06:12:45.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:45.227+0000 7f286f7fe640 1 --2- 172.21.15.92:0/3045542401 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f287015e3b0 0x7f28700702c0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f28700cfea0 tx=0x7f286802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3dcdccc42d7e26c server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:45.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:45.227+0000 7f286e7fc640 1 -- 172.21.15.92:0/3045542401 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f286802ca80 con 0x7f287015e3b0 2023-11-24T06:12:45.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:45.227+0000 7f286e7fc640 1 -- 172.21.15.92:0/3045542401 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f286802cbe0 con 0x7f287015e3b0 2023-11-24T06:12:45.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:45.227+0000 7f286e7fc640 1 -- 172.21.15.92:0/3045542401 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28680343f0 con 0x7f287015e3b0 2023-11-24T06:12:45.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:45.228+0000 7f2875790640 1 -- 172.21.15.92:0/3045542401 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f287015e3b0 msgr2=0x7f28700702c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:45.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:45.228+0000 7f2875790640 1 --2- 172.21.15.92:0/3045542401 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f287015e3b0 0x7f28700702c0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f28700cfea0 tx=0x7f286802bee0 comp rx=0 tx=0).stop 2023-11-24T06:12:45.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:45.228+0000 7f2875790640 1 -- 172.21.15.92:0/3045542401 shutdown_connections 2023-11-24T06:12:45.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:45.228+0000 7f2875790640 1 --2- 172.21.15.92:0/3045542401 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f287015e3b0 0x7f28700702c0 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:45.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:45.228+0000 7f2875790640 1 --2- 172.21.15.92:0/3045542401 >> v2:172.21.15.118:3300/0 conn(0x7f2870143ae0 0x7f28700cf290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:45.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:45.228+0000 7f2875790640 1 -- 172.21.15.92:0/3045542401 >> 172.21.15.92:0/3045542401 conn(0x7f287006a800 msgr2=0x7f287006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:45.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:45.228+0000 7f2875790640 1 -- 172.21.15.92:0/3045542401 shutdown_connections 2023-11-24T06:12:45.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:45.228+0000 7f2875790640 1 -- 172.21.15.92:0/3045542401 wait complete. 2023-11-24T06:12:45.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:45.230+0000 7f2875790640 1 Processor -- start 2023-11-24T06:12:45.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:45.230+0000 7f2875790640 1 -- start start 2023-11-24T06:12:45.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:45.231+0000 7f2875790640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f28700cf7d0 0x7f28700ce030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:45.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:45.231+0000 7f2875790640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2870143ae0 0x7f28700afa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:45.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:45.231+0000 7f2875790640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f287015e3b0 0x7f28700affb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:45.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:45.231+0000 7f2875790640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28701f65a0 con 0x7f287015e3b0 2023-11-24T06:12:45.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:45.231+0000 7f2875790640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28701f6880 con 0x7f28700cf7d0 2023-11-24T06:12:45.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:45.231+0000 7f2875790640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28701f6b60 con 0x7f2870143ae0 2023-11-24T06:12:45.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:45.231+0000 7f286f7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f287015e3b0 0x7f28700affb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:45.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:45.231+0000 7f286f7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f287015e3b0 0x7f28700affb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58860/0 (socket says 172.21.15.92:58860) 2023-11-24T06:12:45.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:45.231+0000 7f286f7fe640 1 -- 172.21.15.92:0/1310366311 learned_addr learned my addr 172.21.15.92:0/1310366311 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:12:45.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:45.232+0000 7f286effd640 1 --2- 172.21.15.92:0/1310366311 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f28700cf7d0 0x7f28700ce030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:45.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:45.232+0000 7f28667fc640 1 --2- 172.21.15.92:0/1310366311 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2870143ae0 0x7f28700afa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:45.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:45.232+0000 7f286f7fe640 1 -- 172.21.15.92:0/1310366311 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2870143ae0 msgr2=0x7f28700afa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:45.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:45.232+0000 7f286f7fe640 1 --2- 172.21.15.92:0/1310366311 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2870143ae0 0x7f28700afa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:45.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:45.232+0000 7f286f7fe640 1 -- 172.21.15.92:0/1310366311 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f28700cf7d0 msgr2=0x7f28700ce030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:45.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:45.232+0000 7f286f7fe640 1 --2- 172.21.15.92:0/1310366311 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f28700cf7d0 0x7f28700ce030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:45.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:45.232+0000 7f286f7fe640 1 -- 172.21.15.92:0/1310366311 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2868009b30 con 0x7f287015e3b0 2023-11-24T06:12:45.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:45.232+0000 7f286f7fe640 1 --2- 172.21.15.92:0/1310366311 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f287015e3b0 0x7f28700affb0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f2868036040 tx=0x7f2868004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:45.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:45.232+0000 7f2867fff640 1 -- 172.21.15.92:0/1310366311 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2868009510 con 0x7f287015e3b0 2023-11-24T06:12:45.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:45.232+0000 7f2867fff640 1 -- 172.21.15.92:0/1310366311 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f286803e450 con 0x7f287015e3b0 2023-11-24T06:12:45.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:45.232+0000 7f2867fff640 1 -- 172.21.15.92:0/1310366311 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f286803e800 con 0x7f287015e3b0 2023-11-24T06:12:45.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:45.232+0000 7f2875790640 1 -- 172.21.15.92:0/1310366311 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28701f6e40 con 0x7f287015e3b0 2023-11-24T06:12:45.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:45.233+0000 7f2875790640 1 -- 172.21.15.92:0/1310366311 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28700ce750 con 0x7f287015e3b0 2023-11-24T06:12:45.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:45.233+0000 7f2875790640 1 -- 172.21.15.92:0/1310366311 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28700675d0 con 0x7f287015e3b0 2023-11-24T06:12:45.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:45.234+0000 7f2867fff640 1 -- 172.21.15.92:0/1310366311 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f286802cd50 con 0x7f287015e3b0 2023-11-24T06:12:45.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:45.235+0000 7f2867fff640 1 --2- 172.21.15.92:0/1310366311 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f285006ae40 0x7f285006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:45.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:45.235+0000 7f2867fff640 1 -- 172.21.15.92:0/1310366311 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f28680ac1a0 con 0x7f287015e3b0 2023-11-24T06:12:45.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:45.235+0000 7f286effd640 1 --2- 172.21.15.92:0/1310366311 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f285006ae40 0x7f285006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:45.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:45.236+0000 7f286effd640 1 --2- 172.21.15.92:0/1310366311 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f285006ae40 0x7f285006d300 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f286000a970 tx=0x7f2860008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:45.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:45.240+0000 7f2867fff640 1 -- 172.21.15.92:0/1310366311 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2868077390 con 0x7f287015e3b0 2023-11-24T06:12:45.456 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:45.453+0000 7f2875790640 1 -- 172.21.15.92:0/1310366311 --> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] -- mgr_command(tid 0: {"prefix": "fs volume info", "vol_name": "cephfs", "target": ["mon-mgr", ""]}) v1 -- 0x7f28700b2410 con 0x7f285006ae40 2023-11-24T06:12:45.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:45.459+0000 7f2867fff640 1 -- 172.21.15.92:0/1310366311 <== mgr.4101 v2:172.21.15.92:6800/784027401 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+530 (secure 0 0 0) 0x7f286000c050 con 0x7f285006ae40 2023-11-24T06:12:45.463 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-24T06:12:45.463 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_addrs": [ 2023-11-24T06:12:45.463 INFO:teuthology.orchestra.run.smithi092.stdout: "172.21.15.92:6789", 2023-11-24T06:12:45.463 INFO:teuthology.orchestra.run.smithi092.stdout: "172.21.15.118:6789", 2023-11-24T06:12:45.463 INFO:teuthology.orchestra.run.smithi092.stdout: "172.21.15.118:6790" 2023-11-24T06:12:45.463 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2023-11-24T06:12:45.463 INFO:teuthology.orchestra.run.smithi092.stdout: "pending_subvolume_deletions": 0, 2023-11-24T06:12:45.463 INFO:teuthology.orchestra.run.smithi092.stdout: "pools": { 2023-11-24T06:12:45.463 INFO:teuthology.orchestra.run.smithi092.stdout: "data": [ 2023-11-24T06:12:45.464 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-24T06:12:45.464 INFO:teuthology.orchestra.run.smithi092.stdout: "avail": 367087058944, 2023-11-24T06:12:45.464 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "cephfs_data", 2023-11-24T06:12:45.464 INFO:teuthology.orchestra.run.smithi092.stdout: "used": 0 2023-11-24T06:12:45.464 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-24T06:12:45.464 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2023-11-24T06:12:45.464 INFO:teuthology.orchestra.run.smithi092.stdout: "metadata": [ 2023-11-24T06:12:45.464 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-24T06:12:45.464 INFO:teuthology.orchestra.run.smithi092.stdout: "avail": 367087058944, 2023-11-24T06:12:45.464 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "cephfs_metadata", 2023-11-24T06:12:45.464 INFO:teuthology.orchestra.run.smithi092.stdout: "used": 65536 2023-11-24T06:12:45.465 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-24T06:12:45.465 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-24T06:12:45.465 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-24T06:12:45.465 INFO:teuthology.orchestra.run.smithi092.stdout: "used_size": 0 2023-11-24T06:12:45.465 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-24T06:12:45.466 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:45.462+0000 7f2875790640 1 -- 172.21.15.92:0/1310366311 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f285006ae40 msgr2=0x7f285006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:45.466 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:45.462+0000 7f2875790640 1 --2- 172.21.15.92:0/1310366311 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f285006ae40 0x7f285006d300 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f286000a970 tx=0x7f2860008040 comp rx=0 tx=0).stop 2023-11-24T06:12:45.466 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:45.462+0000 7f2875790640 1 -- 172.21.15.92:0/1310366311 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f287015e3b0 msgr2=0x7f28700affb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:45.466 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:45.462+0000 7f2875790640 1 --2- 172.21.15.92:0/1310366311 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f287015e3b0 0x7f28700affb0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f2868036040 tx=0x7f2868004a40 comp rx=0 tx=0).stop 2023-11-24T06:12:45.466 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:45.462+0000 7f2875790640 1 -- 172.21.15.92:0/1310366311 shutdown_connections 2023-11-24T06:12:45.466 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:45.462+0000 7f2875790640 1 --2- 172.21.15.92:0/1310366311 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f285006ae40 0x7f285006d300 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:45.466 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:45.463+0000 7f2875790640 1 --2- 172.21.15.92:0/1310366311 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f287015e3b0 0x7f28700affb0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:45.466 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:45.463+0000 7f2875790640 1 --2- 172.21.15.92:0/1310366311 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2870143ae0 0x7f28700afa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:45.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:45.463+0000 7f2875790640 1 --2- 172.21.15.92:0/1310366311 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f28700cf7d0 0x7f28700ce030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:45.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:45.463+0000 7f2875790640 1 -- 172.21.15.92:0/1310366311 >> 172.21.15.92:0/1310366311 conn(0x7f287006a800 msgr2=0x7f287015da10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:45.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:45.463+0000 7f2875790640 1 -- 172.21.15.92:0/1310366311 shutdown_connections 2023-11-24T06:12:45.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:45.463+0000 7f2875790640 1 -- 172.21.15.92:0/1310366311 wait complete. 2023-11-24T06:12:45.483 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-24T06:12:45.483 DEBUG:teuthology.orchestra.run.smithi092:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-24T06:12:45.584 DEBUG:teuthology.orchestra.run.smithi118:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-24T06:12:45.701 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-24T06:12:45.702 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-11-24T06:12:45.702 DEBUG:teuthology.orchestra.run.smithi092:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:12:45.760 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:12:45.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:45.967+0000 7f06d124b640 1 Processor -- start 2023-11-24T06:12:45.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:45.967+0000 7f06d124b640 1 -- start start 2023-11-24T06:12:45.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:45.969+0000 7f06d124b640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f06cc143ae0 0x7f06cc0cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:45.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:45.969+0000 7f06d124b640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06cc0cfc30 con 0x7f06cc0cf7d0 2023-11-24T06:12:45.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:45.969+0000 7f06d124b640 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06cc0b1ee0 con 0x7f06cc15e3b0 2023-11-24T06:12:45.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:45.969+0000 7f06d124b640 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06cc0b2780 con 0x7f06cc143ae0 2023-11-24T06:12:45.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:45.969+0000 7f06c3fff640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f06cc0cf7d0 0x7f06cc15dca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:53594/0 (socket says 172.21.15.92:53594) 2023-11-24T06:12:45.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:45.969+0000 7f06c3fff640 1 -- 172.21.15.92:0/388710581 learned_addr learned my addr 172.21.15.92:0/388710581 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-24T06:12:45.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:45.970+0000 7f06cad9d640 1 --2- 172.21.15.92:0/388710581 >> v2:172.21.15.118:3300/0 conn(0x7f06cc143ae0 0x7f06cc0cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:45.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:45.970+0000 7f06ca59c640 1 -- 172.21.15.92:0/388710581 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2009312506 0 0) 0x7f06cc0cfc30 con 0x7f06cc0cf7d0 2023-11-24T06:12:45.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:45.970+0000 7f06ca59c640 1 -- 172.21.15.92:0/388710581 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f06b8003520 con 0x7f06cc0cf7d0 2023-11-24T06:12:45.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:45.970+0000 7f06ca59c640 1 -- 172.21.15.92:0/388710581 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1656158769 0 0) 0x7f06b8003520 con 0x7f06cc0cf7d0 2023-11-24T06:12:45.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:45.970+0000 7f06ca59c640 1 -- 172.21.15.92:0/388710581 >> v2:172.21.15.118:3300/0 conn(0x7f06cc143ae0 msgr2=0x7f06cc0cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:45.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:45.970+0000 7f06ca59c640 1 --2- 172.21.15.92:0/388710581 >> v2:172.21.15.118:3300/0 conn(0x7f06cc143ae0 0x7f06cc0cf290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:45.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:45.970+0000 7f06ca59c640 1 -- 172.21.15.92:0/388710581 >> v1:172.21.15.118:6789/0 conn(0x7f06cc15e3b0 legacy=0x7f06cc0702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:45.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:45.970+0000 7f06ca59c640 1 -- 172.21.15.92:0/388710581 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06cc15e990 con 0x7f06cc0cf7d0 2023-11-24T06:12:45.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:45.970+0000 7f06ca59c640 1 -- 172.21.15.92:0/388710581 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2607661350 0 0) 0x7f06b4001670 con 0x7f06cc0cf7d0 2023-11-24T06:12:45.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:45.970+0000 7f06ca59c640 1 -- 172.21.15.92:0/388710581 >> v1:172.21.15.92:6789/0 conn(0x7f06cc0cf7d0 legacy=0x7f06cc15dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:45.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:45.970+0000 7f06ca59c640 1 --2- 172.21.15.92:0/388710581 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f06b8003970 0x7f06b8003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:45.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:45.971+0000 7f06ca59c640 1 --2- 172.21.15.92:0/388710581 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f06b80047c0 0x7f06b8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:45.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:45.971+0000 7f06ca59c640 1 --2- 172.21.15.92:0/388710581 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f06b8005180 0x7f06b800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:45.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:45.971+0000 7f06ca59c640 1 -- 172.21.15.92:0/388710581 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06cc0b2780 con 0x7f06b8003970 2023-11-24T06:12:45.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:45.971+0000 7f06ca59c640 1 -- 172.21.15.92:0/388710581 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06b800c020 con 0x7f06b8005180 2023-11-24T06:12:45.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:45.971+0000 7f06ca59c640 1 -- 172.21.15.92:0/388710581 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06b800c1e0 con 0x7f06b80047c0 2023-11-24T06:12:45.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:45.971+0000 7f06cad9d640 1 --2- 172.21.15.92:0/388710581 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f06b8003970 0x7f06b8003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:45.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:45.971+0000 7f06cad9d640 1 -- 172.21.15.92:0/388710581 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f06b80047c0 msgr2=0x7f06b8004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:45.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:45.971+0000 7f06cad9d640 1 --2- 172.21.15.92:0/388710581 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f06b80047c0 0x7f06b8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:45.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:45.971+0000 7f06cad9d640 1 -- 172.21.15.92:0/388710581 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f06b8005180 msgr2=0x7f06b800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:45.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:45.971+0000 7f06cad9d640 1 --2- 172.21.15.92:0/388710581 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f06b8005180 0x7f06b800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:45.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:45.971+0000 7f06cad9d640 1 -- 172.21.15.92:0/388710581 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06b8003520 con 0x7f06b8003970 2023-11-24T06:12:45.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:45.971+0000 7f06cb59e640 1 -- 172.21.15.92:0/388710581 reap_dead start 2023-11-24T06:12:45.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:45.972+0000 7f06cad9d640 1 --2- 172.21.15.92:0/388710581 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f06b8003970 0x7f06b8003d90 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f06bc00bd60 tx=0x7f06b80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cddf5fa15084acd9 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:45.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:45.973+0000 7f06ca59c640 1 -- 172.21.15.92:0/388710581 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06bc02f710 con 0x7f06b8003970 2023-11-24T06:12:45.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:45.973+0000 7f06ca59c640 1 -- 172.21.15.92:0/388710581 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06bc02f870 con 0x7f06b8003970 2023-11-24T06:12:45.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:45.973+0000 7f06ca59c640 1 -- 172.21.15.92:0/388710581 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06bc02fc00 con 0x7f06b8003970 2023-11-24T06:12:45.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:45.973+0000 7f06d124b640 1 -- 172.21.15.92:0/388710581 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f06b8003970 msgr2=0x7f06b8003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:45.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:45.973+0000 7f06d124b640 1 --2- 172.21.15.92:0/388710581 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f06b8003970 0x7f06b8003d90 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f06bc00bd60 tx=0x7f06b80057f0 comp rx=0 tx=0).stop 2023-11-24T06:12:45.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:45.973+0000 7f06d124b640 1 -- 172.21.15.92:0/388710581 shutdown_connections 2023-11-24T06:12:45.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:45.973+0000 7f06d124b640 1 --2- 172.21.15.92:0/388710581 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f06b8003970 0x7f06b8003d90 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:45.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:45.973+0000 7f06d124b640 1 -- 172.21.15.92:0/388710581 >> 172.21.15.92:0/388710581 conn(0x7f06cc06a800 msgr2=0x7f06cc06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:45.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:45.973+0000 7f06d124b640 1 -- 172.21.15.92:0/388710581 shutdown_connections 2023-11-24T06:12:45.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:45.973+0000 7f06d124b640 1 -- 172.21.15.92:0/388710581 wait complete. 2023-11-24T06:12:45.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:45.975+0000 7f06d124b640 1 Processor -- start 2023-11-24T06:12:45.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:45.975+0000 7f06d124b640 1 -- start start 2023-11-24T06:12:45.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:45.976+0000 7f06d124b640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f06b8003970 0x7f06cc0c66f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:45.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:45.976+0000 7f06d124b640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f06cc0c4320 0x7f06cc15cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:45.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:45.976+0000 7f06d124b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f06cc0c47a0 0x7f06cc15d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:45.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:45.976+0000 7f06d124b640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06cc1f3dc0 con 0x7f06cc0c47a0 2023-11-24T06:12:45.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:45.976+0000 7f06d124b640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06cc0c6c30 con 0x7f06b8003970 2023-11-24T06:12:45.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:45.976+0000 7f06d124b640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06cc0c4d70 con 0x7f06cc0c4320 2023-11-24T06:12:45.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:45.977+0000 7f06cb59e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f06cc0c47a0 0x7f06cc15d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:45.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:45.977+0000 7f06cb59e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f06cc0c47a0 0x7f06cc15d1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58890/0 (socket says 172.21.15.92:58890) 2023-11-24T06:12:45.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:45.977+0000 7f06cb59e640 1 -- 172.21.15.92:0/1845723544 learned_addr learned my addr 172.21.15.92:0/1845723544 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:12:45.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:45.977+0000 7f06cb59e640 1 -- 172.21.15.92:0/1845723544 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f06cc0c4320 msgr2=0x7f06cc15cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:45.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:45.977+0000 7f06c3fff640 1 --2- 172.21.15.92:0/1845723544 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f06cc0c4320 0x7f06cc15cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:45.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:45.977+0000 7f06cb59e640 1 --2- 172.21.15.92:0/1845723544 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f06cc0c4320 0x7f06cc15cc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:45.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:45.977+0000 7f06cb59e640 1 -- 172.21.15.92:0/1845723544 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f06b8003970 msgr2=0x7f06cc0c66f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:45.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:45.977+0000 7f06cad9d640 1 --2- 172.21.15.92:0/1845723544 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f06b8003970 0x7f06cc0c66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:45.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:45.977+0000 7f06cb59e640 1 --2- 172.21.15.92:0/1845723544 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f06b8003970 0x7f06cc0c66f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:45.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:45.977+0000 7f06cb59e640 1 -- 172.21.15.92:0/1845723544 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06cc0c5050 con 0x7f06cc0c47a0 2023-11-24T06:12:45.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:45.977+0000 7f06cb59e640 1 --2- 172.21.15.92:0/1845723544 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f06cc0c47a0 0x7f06cc15d1c0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f06cc071f10 tx=0x7f06c400c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:45.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:45.978+0000 7f06c37fe640 1 -- 172.21.15.92:0/1845723544 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06c40070d0 con 0x7f06cc0c47a0 2023-11-24T06:12:45.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:45.978+0000 7f06c37fe640 1 -- 172.21.15.92:0/1845723544 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06c4007230 con 0x7f06cc0c47a0 2023-11-24T06:12:45.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:45.978+0000 7f06c37fe640 1 -- 172.21.15.92:0/1845723544 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06c40075c0 con 0x7f06cc0c47a0 2023-11-24T06:12:45.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:45.978+0000 7f06d124b640 1 -- 172.21.15.92:0/1845723544 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06cc0c5330 con 0x7f06cc0c47a0 2023-11-24T06:12:45.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:45.978+0000 7f06d124b640 1 -- 172.21.15.92:0/1845723544 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06cc0ae560 con 0x7f06cc0c47a0 2023-11-24T06:12:45.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:45.979+0000 7f06d124b640 1 -- 172.21.15.92:0/1845723544 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06cc0c5520 con 0x7f06cc0c47a0 2023-11-24T06:12:45.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:45.980+0000 7f06c37fe640 1 -- 172.21.15.92:0/1845723544 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f06c4007720 con 0x7f06cc0c47a0 2023-11-24T06:12:45.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:45.981+0000 7f06c37fe640 1 --2- 172.21.15.92:0/1845723544 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f06ac06ae70 0x7f06ac06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:45.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:45.981+0000 7f06c37fe640 1 -- 172.21.15.92:0/1845723544 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f06c4088b40 con 0x7f06cc0c47a0 2023-11-24T06:12:45.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:45.981+0000 7f06cad9d640 1 --2- 172.21.15.92:0/1845723544 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f06ac06ae70 0x7f06ac06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:45.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:45.982+0000 7f06cad9d640 1 --2- 172.21.15.92:0/1845723544 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f06ac06ae70 0x7f06ac06d330 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f06bc039000 tx=0x7f06bc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:45.988 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:45.985+0000 7f06c37fe640 1 -- 172.21.15.92:0/1845723544 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f06cc0c5520 con 0x7f06cc0c47a0 2023-11-24T06:12:46.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:46.179+0000 7f06d124b640 1 -- 172.21.15.92:0/1845723544 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f06cc0c5520 con 0x7f06cc0c47a0 2023-11-24T06:12:46.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:46.181+0000 7f06c37fe640 1 -- 172.21.15.92:0/1845723544 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v52) v1 ==== 74+0+16517 (secure 0 0 0) 0x7f06cc0c5520 con 0x7f06cc0c47a0 2023-11-24T06:12:46.185 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-24T06:12:46.185 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":52,"fsid":"2d6dee5f-8aa3-470f-931a-50ade963b1cb","created":"2023-11-24T06:10:04.707204+0000","modified":"2023-11-24T06:12:32.152035+0000","last_up_change":"2023-11-24T06:10:17.598352+0000","last_in_change":"2023-11-24T06:10:09.040793+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:10:17.733098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:12:28.410305+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-11-24T06:12:29.625599+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ff37a9fb-135f-471f-b4b0-f7b5772006c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6803","nonce":20560714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6805","nonce":20560714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6809","nonce":20560714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6807","nonce":20560714}]},"public_addr":"172.21.15.92:6803/20560714","cluster_addr":"172.21.15.92:6805/20560714","heartbeat_back_addr":"172.21.15.92:6809/20560714","heartbeat_front_addr":"172.21.15.92:6807/20560714","state":["exists","up"]},{"osd":1,"uuid":"03cdd353-dcf1-4f38-94fe-a795137c06b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6819","nonce":2451239906}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6821","nonce":2451239906}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6825","nonce":2451239906}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6823","nonce":2451239906}]},"public_addr":"172.21.15.92:6819/2451239906","cluster_addr":"172.21.15.92:6821/2451239906","heartbeat_back_addr":"172.21.15.92:6825/2451239906","heartbeat_front_addr":"172.21.15.92:6823/2451239906","state":["exists","up"]},{"osd":2,"uuid":"a4a20d80-5d57-484b-becd-9aa8070dd3aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6811","nonce":1124979277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6813","nonce":1124979277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6817","nonce":1124979277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6815","nonce":1124979277}]},"public_addr":"172.21.15.92:6811/1124979277","cluster_addr":"172.21.15.92:6813/1124979277","heartbeat_back_addr":"172.21.15.92:6817/1124979277","heartbeat_front_addr":"172.21.15.92:6815/1124979277","state":["exists","up"]},{"osd":3,"uuid":"4648d7ce-9b29-4583-9425-4ba8fd2b1903","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6827","nonce":1734115782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6829","nonce":1734115782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6833","nonce":1734115782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6831","nonce":1734115782}]},"public_addr":"172.21.15.92:6827/1734115782","cluster_addr":"172.21.15.92:6829/1734115782","heartbeat_back_addr":"172.21.15.92:6833/1734115782","heartbeat_front_addr":"172.21.15.92:6831/1734115782","state":["exists","up"]},{"osd":4,"uuid":"3bc9d7eb-c6fa-4fcf-a3ec-ade604e05aac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6819","nonce":3425937229}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6822","nonce":3425937229}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6830","nonce":3425937229}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6826","nonce":3425937229}]},"public_addr":"172.21.15.118:6819/3425937229","cluster_addr":"172.21.15.118:6822/3425937229","heartbeat_back_addr":"172.21.15.118:6830/3425937229","heartbeat_front_addr":"172.21.15.118:6826/3425937229","state":["exists","up"]},{"osd":5,"uuid":"dce9ba6d-59ed-46d1-84fc-a84b13de10c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6801","nonce":827455984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6803","nonce":827455984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6807","nonce":827455984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6805","nonce":827455984}]},"public_addr":"172.21.15.118:6801/827455984","cluster_addr":"172.21.15.118:6803/827455984","heartbeat_back_addr":"172.21.15.118:6807/827455984","heartbeat_front_addr":"172.21.15.118:6805/827455984","state":["exists","up"]},{"osd":6,"uuid":"82aec504-ae58-4709-8d30-46f2b12081b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6809","nonce":2946986141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6811","nonce":2946986141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6815","nonce":2946986141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6813","nonce":2946986141}]},"public_addr":"172.21.15.118:6809/2946986141","cluster_addr":"172.21.15.118:6811/2946986141","heartbeat_back_addr":"172.21.15.118:6815/2946986141","heartbeat_front_addr":"172.21.15.118:6813/2946986141","state":["exists","up"]},{"osd":7,"uuid":"4ed4c624-c88f-4d35-bdb1-bf209b5dffb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6817","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6818","nonce":48122059}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6821","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6823","nonce":48122059}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6829","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6831","nonce":48122059}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6825","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6827","nonce":48122059}]},"public_addr":"172.21.15.118:6818/48122059","cluster_addr":"172.21.15.118:6823/48122059","heartbeat_back_addr":"172.21.15.118:6831/48122059","heartbeat_front_addr":"172.21.15.118:6827/48122059","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.536664+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.368324+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.271201+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.342971+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.621832+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.387110+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.632946+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.416032+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:12:46.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:46.184+0000 7f06d124b640 1 -- 172.21.15.92:0/1845723544 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f06ac06ae70 msgr2=0x7f06ac06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:46.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:46.184+0000 7f06d124b640 1 --2- 172.21.15.92:0/1845723544 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f06ac06ae70 0x7f06ac06d330 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f06bc039000 tx=0x7f06bc008040 comp rx=0 tx=0).stop 2023-11-24T06:12:46.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:46.184+0000 7f06d124b640 1 -- 172.21.15.92:0/1845723544 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f06cc0c47a0 msgr2=0x7f06cc15d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:46.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:46.184+0000 7f06d124b640 1 --2- 172.21.15.92:0/1845723544 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f06cc0c47a0 0x7f06cc15d1c0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f06cc071f10 tx=0x7f06c400c980 comp rx=0 tx=0).stop 2023-11-24T06:12:46.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:46.184+0000 7f06d124b640 1 -- 172.21.15.92:0/1845723544 shutdown_connections 2023-11-24T06:12:46.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:46.184+0000 7f06d124b640 1 --2- 172.21.15.92:0/1845723544 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f06ac06ae70 0x7f06ac06d330 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:46.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:46.184+0000 7f06d124b640 1 --2- 172.21.15.92:0/1845723544 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f06cc0c47a0 0x7f06cc15d1c0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:46.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:46.184+0000 7f06d124b640 1 --2- 172.21.15.92:0/1845723544 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f06cc0c4320 0x7f06cc15cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:46.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:46.184+0000 7f06d124b640 1 --2- 172.21.15.92:0/1845723544 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f06b8003970 0x7f06cc0c66f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:46.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:46.184+0000 7f06d124b640 1 -- 172.21.15.92:0/1845723544 >> 172.21.15.92:0/1845723544 conn(0x7f06cc06a800 msgr2=0x7f06cc15b8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:46.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:46.184+0000 7f06d124b640 1 -- 172.21.15.92:0/1845723544 shutdown_connections 2023-11-24T06:12:46.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:46.185+0000 7f06d124b640 1 -- 172.21.15.92:0/1845723544 wait complete. 2023-11-24T06:12:46.205 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T06:10:17.733098+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T06:12:28.410305+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '51', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T06:12:29.625599+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '52', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T06:12:46.206 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T06:12:46.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:46.418+0000 7f088efef640 1 Processor -- start 2023-11-24T06:12:46.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:46.418+0000 7f088efef640 1 -- start start 2023-11-24T06:12:46.424 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:46.420+0000 7f088efef640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f08880adf20 0x7f0888070a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:46.424 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:46.420+0000 7f088efef640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f088815bf10 0x7f08880cc440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:46.424 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:46.420+0000 7f088efef640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f088815f4e0 con 0x7f08880cc980 2023-11-24T06:12:46.424 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:46.420+0000 7f088efef640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f088815f970 con 0x7f08880adf20 2023-11-24T06:12:46.424 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:46.420+0000 7f088efef640 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08880ccdd0 con 0x7f088815bf10 2023-11-24T06:12:46.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:46.420+0000 7f088dfed640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f08880adf20 0x7f0888070a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:46.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:46.420+0000 7f088e7ee640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f08880cc980 0x7f0888143d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:53606/0 (socket says 172.21.15.92:53606) 2023-11-24T06:12:46.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:46.420+0000 7f088dfed640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f08880adf20 0x7f0888070a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58892/0 (socket says 172.21.15.92:58892) 2023-11-24T06:12:46.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:46.420+0000 7f088e7ee640 1 -- 172.21.15.92:0/2213082773 learned_addr learned my addr 172.21.15.92:0/2213082773 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-24T06:12:46.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:46.420+0000 7f088d7ec640 1 --2- 172.21.15.92:0/2213082773 >> v2:172.21.15.118:3300/0 conn(0x7f088815bf10 0x7f08880cc440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:46.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:46.421+0000 7f088dfed640 1 -- 172.21.15.92:0/2213082773 >> v2:172.21.15.118:3300/0 conn(0x7f088815bf10 msgr2=0x7f08880cc440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:46.426 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:46.421+0000 7f088dfed640 1 --2- 172.21.15.92:0/2213082773 >> v2:172.21.15.118:3300/0 conn(0x7f088815bf10 0x7f08880cc440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:46.426 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:46.421+0000 7f088dfed640 1 -- 172.21.15.92:0/2213082773 >> v1:172.21.15.92:6789/0 conn(0x7f08880cc980 legacy=0x7f0888143d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:46.426 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:46.421+0000 7f088dfed640 1 -- 172.21.15.92:0/2213082773 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0888071130 con 0x7f08880adf20 2023-11-24T06:12:46.426 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:46.421+0000 7f088cfeb640 1 -- 172.21.15.92:0/2213082773 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2130750975 0 0) 0x7f088815f4e0 con 0x7f08880cc980 2023-11-24T06:12:46.427 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:46.422+0000 7f088dfed640 1 --2- 172.21.15.92:0/2213082773 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f08880adf20 0x7f0888070a30 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f088815e9e0 tx=0x7f088402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cffb4f3cb83442d0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:46.427 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:46.422+0000 7f088cfeb640 1 -- 172.21.15.92:0/2213082773 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f088402e070 con 0x7f08880adf20 2023-11-24T06:12:46.427 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:46.422+0000 7f088cfeb640 1 -- 172.21.15.92:0/2213082773 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08840315a0 con 0x7f08880adf20 2023-11-24T06:12:46.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:46.422+0000 7f088cfeb640 1 -- 172.21.15.92:0/2213082773 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0884031930 con 0x7f08880adf20 2023-11-24T06:12:46.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:46.423+0000 7f088efef640 1 -- 172.21.15.92:0/2213082773 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f08880adf20 msgr2=0x7f0888070a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:46.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:46.423+0000 7f088efef640 1 --2- 172.21.15.92:0/2213082773 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f08880adf20 0x7f0888070a30 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f088815e9e0 tx=0x7f088402b840 comp rx=0 tx=0).stop 2023-11-24T06:12:46.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:46.423+0000 7f088efef640 1 -- 172.21.15.92:0/2213082773 shutdown_connections 2023-11-24T06:12:46.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:46.423+0000 7f088efef640 1 --2- 172.21.15.92:0/2213082773 >> v2:172.21.15.118:3300/0 conn(0x7f088815bf10 0x7f08880cc440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:46.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:46.423+0000 7f088efef640 1 --2- 172.21.15.92:0/2213082773 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f08880adf20 0x7f0888070a30 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:46.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:46.423+0000 7f088efef640 1 -- 172.21.15.92:0/2213082773 >> 172.21.15.92:0/2213082773 conn(0x7f088806a7d0 msgr2=0x7f088806abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:46.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:46.423+0000 7f088efef640 1 -- 172.21.15.92:0/2213082773 shutdown_connections 2023-11-24T06:12:46.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:46.423+0000 7f088efef640 1 -- 172.21.15.92:0/2213082773 wait complete. 2023-11-24T06:12:46.431 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:46.425+0000 7f088efef640 1 Processor -- start 2023-11-24T06:12:46.431 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:46.425+0000 7f088efef640 1 -- start start 2023-11-24T06:12:46.431 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:46.426+0000 7f088efef640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f08880adf20 0x7f088815aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:46.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:46.426+0000 7f088efef640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f08880cc980 0x7f08880b28a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:46.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:46.426+0000 7f088efef640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f088815bf10 0x7f088806d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:46.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:46.426+0000 7f088efef640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f088815af40 con 0x7f08880cc980 2023-11-24T06:12:46.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:46.426+0000 7f088efef640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08880b3320 con 0x7f08880adf20 2023-11-24T06:12:46.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:46.426+0000 7f088efef640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08880b3600 con 0x7f088815bf10 2023-11-24T06:12:46.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:46.426+0000 7f088d7ec640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f08880cc980 0x7f08880b28a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:46.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:46.426+0000 7f088d7ec640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f08880cc980 0x7f08880b28a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58902/0 (socket says 172.21.15.92:58902) 2023-11-24T06:12:46.433 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:46.426+0000 7f088d7ec640 1 -- 172.21.15.92:0/2535940592 learned_addr learned my addr 172.21.15.92:0/2535940592 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:12:46.433 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:46.427+0000 7f088dfed640 1 --2- 172.21.15.92:0/2535940592 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f08880adf20 0x7f088815aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:46.433 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:46.427+0000 7f088e7ee640 1 --2- 172.21.15.92:0/2535940592 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f088815bf10 0x7f088806d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:46.433 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:46.427+0000 7f088d7ec640 1 -- 172.21.15.92:0/2535940592 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f088815bf10 msgr2=0x7f088806d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:46.433 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:46.427+0000 7f088d7ec640 1 --2- 172.21.15.92:0/2535940592 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f088815bf10 0x7f088806d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:46.433 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:46.427+0000 7f088d7ec640 1 -- 172.21.15.92:0/2535940592 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f08880adf20 msgr2=0x7f088815aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:46.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:46.427+0000 7f088d7ec640 1 --2- 172.21.15.92:0/2535940592 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f08880adf20 0x7f088815aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:46.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:46.427+0000 7f088d7ec640 1 -- 172.21.15.92:0/2535940592 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0884009400 con 0x7f08880cc980 2023-11-24T06:12:46.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:46.427+0000 7f088e7ee640 1 --2- 172.21.15.92:0/2535940592 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f088815bf10 0x7f088806d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:12:46.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:46.427+0000 7f088d7ec640 1 --2- 172.21.15.92:0/2535940592 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f08880cc980 0x7f08880b28a0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f087800d800 tx=0x7f087800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:46.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:46.427+0000 7f087e7fc640 1 -- 172.21.15.92:0/2535940592 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f087800b6c0 con 0x7f08880cc980 2023-11-24T06:12:46.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:46.427+0000 7f087e7fc640 1 -- 172.21.15.92:0/2535940592 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f087800b820 con 0x7f08880cc980 2023-11-24T06:12:46.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:46.427+0000 7f088efef640 1 -- 172.21.15.92:0/2535940592 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08880c3d00 con 0x7f08880cc980 2023-11-24T06:12:46.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:46.427+0000 7f087e7fc640 1 -- 172.21.15.92:0/2535940592 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f087800bbb0 con 0x7f08880cc980 2023-11-24T06:12:46.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:46.428+0000 7f088efef640 1 -- 172.21.15.92:0/2535940592 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08880c42c0 con 0x7f08880cc980 2023-11-24T06:12:46.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:46.429+0000 7f088efef640 1 -- 172.21.15.92:0/2535940592 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08880c4480 con 0x7f08880cc980 2023-11-24T06:12:46.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:46.430+0000 7f087e7fc640 1 -- 172.21.15.92:0/2535940592 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f087800bd10 con 0x7f08880cc980 2023-11-24T06:12:46.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:46.431+0000 7f087e7fc640 1 --2- 172.21.15.92:0/2535940592 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f086c06ae70 0x7f086c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:46.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:46.431+0000 7f087e7fc640 1 -- 172.21.15.92:0/2535940592 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0878089b90 con 0x7f08880cc980 2023-11-24T06:12:46.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:46.431+0000 7f088dfed640 1 --2- 172.21.15.92:0/2535940592 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f086c06ae70 0x7f086c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:46.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:46.432+0000 7f088dfed640 1 --2- 172.21.15.92:0/2535940592 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f086c06ae70 0x7f086c06d330 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f0884036000 tx=0x7f088403d040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:46.438 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:46.435+0000 7f087e7fc640 1 -- 172.21.15.92:0/2535940592 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0878054c20 con 0x7f08880cc980 2023-11-24T06:12:46.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:46.625+0000 7f088efef640 1 -- 172.21.15.92:0/2535940592 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0888067450 con 0x7f08880cc980 2023-11-24T06:12:46.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:46.636+0000 7f087e7fc640 1 -- 172.21.15.92:0/2535940592 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f0878054ff0 con 0x7f08880cc980 2023-11-24T06:12:46.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:46.639+0000 7f087e7fc640 1 -- 172.21.15.92:0/2535940592 <== mon.0 v2:172.21.15.92:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v52) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0878070400 con 0x7f08880cc980 2023-11-24T06:12:46.642 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 1 2023-11-24T06:12:46.644 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:46.641+0000 7f088efef640 1 -- 172.21.15.92:0/2535940592 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f086c06ae70 msgr2=0x7f086c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:46.644 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:46.641+0000 7f088efef640 1 --2- 172.21.15.92:0/2535940592 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f086c06ae70 0x7f086c06d330 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f0884036000 tx=0x7f088403d040 comp rx=0 tx=0).stop 2023-11-24T06:12:46.645 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:46.641+0000 7f088efef640 1 -- 172.21.15.92:0/2535940592 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f08880cc980 msgr2=0x7f08880b28a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:46.645 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:46.641+0000 7f088efef640 1 --2- 172.21.15.92:0/2535940592 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f08880cc980 0x7f08880b28a0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f087800d800 tx=0x7f087800dcc0 comp rx=0 tx=0).stop 2023-11-24T06:12:46.645 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:46.642+0000 7f088efef640 1 -- 172.21.15.92:0/2535940592 shutdown_connections 2023-11-24T06:12:46.646 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:46.642+0000 7f088efef640 1 --2- 172.21.15.92:0/2535940592 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f086c06ae70 0x7f086c06d330 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:46.646 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:46.642+0000 7f088efef640 1 --2- 172.21.15.92:0/2535940592 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f088815bf10 0x7f088806d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:46.646 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:46.642+0000 7f088efef640 1 --2- 172.21.15.92:0/2535940592 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f08880cc980 0x7f08880b28a0 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:46.646 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:46.642+0000 7f088efef640 1 --2- 172.21.15.92:0/2535940592 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f08880adf20 0x7f088815aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:46.646 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:46.642+0000 7f088efef640 1 -- 172.21.15.92:0/2535940592 >> 172.21.15.92:0/2535940592 conn(0x7f088806a7d0 msgr2=0x7f088815c740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:46.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:46.642+0000 7f088efef640 1 -- 172.21.15.92:0/2535940592 shutdown_connections 2023-11-24T06:12:46.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:46.642+0000 7f088efef640 1 -- 172.21.15.92:0/2535940592 wait complete. 2023-11-24T06:12:46.664 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T06:12:46.873 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:46.870+0000 7f7c1dd89640 1 Processor -- start 2023-11-24T06:12:46.873 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:46.870+0000 7f7c1dd89640 1 -- start start 2023-11-24T06:12:46.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:46.872+0000 7f7c1dd89640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f7c1813df80 0x7f7c1815bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:46.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:46.872+0000 7f7c1dd89640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f7c1813e360 0x7f7c180cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:46.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:46.872+0000 7f7c1dd89640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7c1815f9c0 0x7f7c180cf7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:46.876 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:46.872+0000 7f7c1dd89640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c1806c470 con 0x7f7c1813df80 2023-11-24T06:12:46.876 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:46.872+0000 7f7c1dd89640 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c1806c750 con 0x7f7c1813e360 2023-11-24T06:12:46.876 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:46.872+0000 7f7c1dd89640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c1806ca30 con 0x7f7c1815f9c0 2023-11-24T06:12:46.876 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:46.872+0000 7f7c177fe640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f7c1813df80 0x7f7c1815bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:46.876 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:46.872+0000 7f7c177fe640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f7c1813df80 0x7f7c1815bee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58910/0 (socket says 172.21.15.92:58910) 2023-11-24T06:12:46.876 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:46.872+0000 7f7c177fe640 1 -- 172.21.15.92:0/3532676553 learned_addr learned my addr 172.21.15.92:0/3532676553 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:12:46.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:46.873+0000 7f7c177fe640 1 -- 172.21.15.92:0/3532676553 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7c1815f9c0 msgr2=0x7f7c180cf7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:46.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:46.873+0000 7f7c177fe640 1 --2- 172.21.15.92:0/3532676553 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7c1815f9c0 0x7f7c180cf7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:46.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:46.873+0000 7f7c177fe640 1 -- 172.21.15.92:0/3532676553 >> v2:172.21.15.118:3300/0 conn(0x7f7c1813e360 msgr2=0x7f7c180cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:46.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:46.873+0000 7f7c177fe640 1 --2- 172.21.15.92:0/3532676553 >> v2:172.21.15.118:3300/0 conn(0x7f7c1813e360 0x7f7c180cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:46.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:46.873+0000 7f7c177fe640 1 -- 172.21.15.92:0/3532676553 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c1806cd10 con 0x7f7c1813df80 2023-11-24T06:12:46.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:46.874+0000 7f7c177fe640 1 --2- 172.21.15.92:0/3532676553 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7c1813df80 0x7f7c1815bee0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f7c1806ded0 tx=0x7f7c0802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=894ee2578ea28c2e server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:46.878 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:46.874+0000 7f7c167fc640 1 -- 172.21.15.92:0/3532676553 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c0802ca80 con 0x7f7c1813df80 2023-11-24T06:12:46.878 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:46.874+0000 7f7c167fc640 1 -- 172.21.15.92:0/3532676553 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c0802cbe0 con 0x7f7c1813df80 2023-11-24T06:12:46.878 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:46.874+0000 7f7c167fc640 1 -- 172.21.15.92:0/3532676553 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c080343f0 con 0x7f7c1813df80 2023-11-24T06:12:46.878 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:46.875+0000 7f7c1dd89640 1 -- 172.21.15.92:0/3532676553 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7c1813df80 msgr2=0x7f7c1815bee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:46.879 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:46.875+0000 7f7c1dd89640 1 --2- 172.21.15.92:0/3532676553 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7c1813df80 0x7f7c1815bee0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f7c1806ded0 tx=0x7f7c0802bee0 comp rx=0 tx=0).stop 2023-11-24T06:12:46.879 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:46.875+0000 7f7c1dd89640 1 -- 172.21.15.92:0/3532676553 shutdown_connections 2023-11-24T06:12:46.879 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:46.875+0000 7f7c1dd89640 1 --2- 172.21.15.92:0/3532676553 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7c1815f9c0 0x7f7c180cf7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:46.879 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:46.875+0000 7f7c1dd89640 1 --2- 172.21.15.92:0/3532676553 >> v2:172.21.15.118:3300/0 conn(0x7f7c1813e360 0x7f7c180cf290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:46.879 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:46.875+0000 7f7c1dd89640 1 --2- 172.21.15.92:0/3532676553 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7c1813df80 0x7f7c1815bee0 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:46.879 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:46.875+0000 7f7c1dd89640 1 -- 172.21.15.92:0/3532676553 >> 172.21.15.92:0/3532676553 conn(0x7f7c1806a800 msgr2=0x7f7c1806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:46.879 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:46.875+0000 7f7c1dd89640 1 -- 172.21.15.92:0/3532676553 shutdown_connections 2023-11-24T06:12:46.879 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:46.875+0000 7f7c1dd89640 1 -- 172.21.15.92:0/3532676553 wait complete. 2023-11-24T06:12:46.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:46.877+0000 7f7c1dd89640 1 Processor -- start 2023-11-24T06:12:46.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:46.877+0000 7f7c1dd89640 1 -- start start 2023-11-24T06:12:46.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:46.878+0000 7f7c1dd89640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7c1813df80 0x7f7c18071d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:46.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:46.878+0000 7f7c1dd89640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7c1813e360 0x7f7c181f2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:46.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:46.878+0000 7f7c1dd89640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7c1815f9c0 0x7f7c180702c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:46.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:46.878+0000 7f7c1dd89640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c18071890 con 0x7f7c1813df80 2023-11-24T06:12:46.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:46.878+0000 7f7c1dd89640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c181f3410 con 0x7f7c1815f9c0 2023-11-24T06:12:46.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:46.878+0000 7f7c1dd89640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c18070c80 con 0x7f7c1813e360 2023-11-24T06:12:46.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:46.878+0000 7f7c177fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7c1813df80 0x7f7c18071d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:46.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:46.878+0000 7f7c177fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7c1813df80 0x7f7c18071d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58918/0 (socket says 172.21.15.92:58918) 2023-11-24T06:12:46.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:46.878+0000 7f7c177fe640 1 -- 172.21.15.92:0/4189137627 learned_addr learned my addr 172.21.15.92:0/4189137627 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:12:46.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:46.879+0000 7f7c177fe640 1 -- 172.21.15.92:0/4189137627 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7c1813e360 msgr2=0x7f7c181f2ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:46.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:46.879+0000 7f7c177fe640 1 --2- 172.21.15.92:0/4189137627 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7c1813e360 0x7f7c181f2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:46.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:46.879+0000 7f7c177fe640 1 -- 172.21.15.92:0/4189137627 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7c1815f9c0 msgr2=0x7f7c180702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:46.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:46.879+0000 7f7c177fe640 1 --2- 172.21.15.92:0/4189137627 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7c1815f9c0 0x7f7c180702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:46.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:46.879+0000 7f7c177fe640 1 -- 172.21.15.92:0/4189137627 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c08009b30 con 0x7f7c1813df80 2023-11-24T06:12:46.884 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:46.879+0000 7f7c177fe640 1 --2- 172.21.15.92:0/4189137627 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7c1813df80 0x7f7c18071d20 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f7c08036040 tx=0x7f7c08004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:46.884 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:46.879+0000 7f7bfffff640 1 -- 172.21.15.92:0/4189137627 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c0803e450 con 0x7f7c1813df80 2023-11-24T06:12:46.884 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:46.879+0000 7f7bfffff640 1 -- 172.21.15.92:0/4189137627 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c0803e5d0 con 0x7f7c1813df80 2023-11-24T06:12:46.884 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:46.879+0000 7f7bfffff640 1 -- 172.21.15.92:0/4189137627 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c0803e980 con 0x7f7c1813df80 2023-11-24T06:12:46.884 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:46.879+0000 7f7c1dd89640 1 -- 172.21.15.92:0/4189137627 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c18070f60 con 0x7f7c1813df80 2023-11-24T06:12:46.884 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:46.879+0000 7f7c1dd89640 1 -- 172.21.15.92:0/4189137627 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c18071520 con 0x7f7c1813df80 2023-11-24T06:12:46.884 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:46.880+0000 7f7c1dd89640 1 -- 172.21.15.92:0/4189137627 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c180675d0 con 0x7f7c1813df80 2023-11-24T06:12:46.885 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:46.881+0000 7f7bfffff640 1 -- 172.21.15.92:0/4189137627 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f7c0802cd50 con 0x7f7c1813df80 2023-11-24T06:12:46.885 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:46.882+0000 7f7bfffff640 1 --2- 172.21.15.92:0/4189137627 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f7c0406aed0 0x7f7c0406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:46.885 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:46.882+0000 7f7bfffff640 1 -- 172.21.15.92:0/4189137627 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7c080ac460 con 0x7f7c1813df80 2023-11-24T06:12:46.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:46.883+0000 7f7c16ffd640 1 --2- 172.21.15.92:0/4189137627 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f7c0406aed0 0x7f7c0406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:46.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:46.883+0000 7f7c16ffd640 1 --2- 172.21.15.92:0/4189137627 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f7c0406aed0 0x7f7c0406d390 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f7c181f6a50 tx=0x7f7c0c006c90 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:46.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:46.887+0000 7f7bfffff640 1 -- 172.21.15.92:0/4189137627 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7c08077640 con 0x7f7c1813df80 2023-11-24T06:12:46.899 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T06:12:46.905 DEBUG:teuthology.orchestra.run.smithi118:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T06:12:47.084 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:47.080+0000 7f7c1dd89640 1 -- 172.21.15.92:0/4189137627 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f7c180cfaf0 con 0x7f7c1813df80 2023-11-24T06:12:47.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:47.081+0000 7f7bfffff640 1 -- 172.21.15.92:0/4189137627 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v52) v1 ==== 104+0+11 (secure 0 0 0) 0x7f7c0807ac60 con 0x7f7c1813df80 2023-11-24T06:12:47.085 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 32 2023-11-24T06:12:47.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:47.084+0000 7f7c1dd89640 1 -- 172.21.15.92:0/4189137627 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f7c0406aed0 msgr2=0x7f7c0406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:47.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:47.084+0000 7f7c1dd89640 1 --2- 172.21.15.92:0/4189137627 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f7c0406aed0 0x7f7c0406d390 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f7c181f6a50 tx=0x7f7c0c006c90 comp rx=0 tx=0).stop 2023-11-24T06:12:47.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:47.084+0000 7f7c1dd89640 1 -- 172.21.15.92:0/4189137627 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7c1813df80 msgr2=0x7f7c18071d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:47.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:47.084+0000 7f7c1dd89640 1 --2- 172.21.15.92:0/4189137627 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7c1813df80 0x7f7c18071d20 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f7c08036040 tx=0x7f7c08004a40 comp rx=0 tx=0).stop 2023-11-24T06:12:47.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:47.084+0000 7f7c1dd89640 1 -- 172.21.15.92:0/4189137627 shutdown_connections 2023-11-24T06:12:47.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:47.084+0000 7f7c1dd89640 1 --2- 172.21.15.92:0/4189137627 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f7c0406aed0 0x7f7c0406d390 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:47.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:47.084+0000 7f7c1dd89640 1 --2- 172.21.15.92:0/4189137627 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7c1815f9c0 0x7f7c180702c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:47.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:47.084+0000 7f7c1dd89640 1 --2- 172.21.15.92:0/4189137627 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7c1813e360 0x7f7c181f2ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:47.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:47.084+0000 7f7c1dd89640 1 --2- 172.21.15.92:0/4189137627 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7c1813df80 0x7f7c18071d20 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:47.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:47.084+0000 7f7c1dd89640 1 -- 172.21.15.92:0/4189137627 >> 172.21.15.92:0/4189137627 conn(0x7f7c1806a800 msgr2=0x7f7c1813faa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:47.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:47.084+0000 7f7c1dd89640 1 -- 172.21.15.92:0/4189137627 shutdown_connections 2023-11-24T06:12:47.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:47.084+0000 7f7c1dd89640 1 -- 172.21.15.92:0/4189137627 wait complete. 2023-11-24T06:12:47.107 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T06:12:47.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:47.318+0000 7f8fce5dc640 1 Processor -- start 2023-11-24T06:12:47.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:47.318+0000 7f8fce5dc640 1 -- start start 2023-11-24T06:12:47.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:47.319+0000 7f8fce5dc640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f8fc813df80 0x7f8fc815bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:47.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:47.319+0000 7f8fce5dc640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f8fc813e360 0x7f8fc80cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:47.323 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:47.320+0000 7f8fce5dc640 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8fc806c470 con 0x7f8fc815f9c0 2023-11-24T06:12:47.323 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:47.320+0000 7f8fce5dc640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8fc806c750 con 0x7f8fc813e360 2023-11-24T06:12:47.323 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:47.320+0000 7f8fce5dc640 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8fc806ca30 con 0x7f8fc813df80 2023-11-24T06:12:47.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:47.320+0000 7f8fc77fe640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f8fc813e360 0x7f8fc80cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:47.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:47.320+0000 7f8fc77fe640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f8fc813e360 0x7f8fc80cf290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58928/0 (socket says 172.21.15.92:58928) 2023-11-24T06:12:47.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:47.320+0000 7f8fc77fe640 1 -- 172.21.15.92:0/1279932964 learned_addr learned my addr 172.21.15.92:0/1279932964 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:12:47.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:47.320+0000 7f8fc7fff640 1 --2- 172.21.15.92:0/1279932964 >> v2:172.21.15.118:3300/0 conn(0x7f8fc813df80 0x7f8fc815bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:47.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:47.320+0000 7f8fc77fe640 1 -- 172.21.15.92:0/1279932964 >> v2:172.21.15.118:3300/0 conn(0x7f8fc813df80 msgr2=0x7f8fc815bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:47.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:47.320+0000 7f8fc77fe640 1 --2- 172.21.15.92:0/1279932964 >> v2:172.21.15.118:3300/0 conn(0x7f8fc813df80 0x7f8fc815bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:47.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:47.320+0000 7f8fc77fe640 1 -- 172.21.15.92:0/1279932964 >> v1:172.21.15.118:6789/0 conn(0x7f8fc815f9c0 legacy=0x7f8fc81408d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:47.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:47.320+0000 7f8fc77fe640 1 -- 172.21.15.92:0/1279932964 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fc806cd10 con 0x7f8fc813e360 2023-11-24T06:12:47.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:47.321+0000 7f8fc77fe640 1 --2- 172.21.15.92:0/1279932964 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8fc813e360 0x7f8fc80cf290 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f8fc8072770 tx=0x7f8fb002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a53bb32ccff4470a server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:47.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:47.322+0000 7f8fc6ffd640 1 -- 172.21.15.92:0/1279932964 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fb0031870 con 0x7f8fc813e360 2023-11-24T06:12:47.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:47.322+0000 7f8fc6ffd640 1 -- 172.21.15.92:0/1279932964 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fb00319d0 con 0x7f8fc813e360 2023-11-24T06:12:47.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:47.322+0000 7f8fc6ffd640 1 -- 172.21.15.92:0/1279932964 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fb0031d60 con 0x7f8fc813e360 2023-11-24T06:12:47.326 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:47.323+0000 7f8fce5dc640 1 -- 172.21.15.92:0/1279932964 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8fc813e360 msgr2=0x7f8fc80cf290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:47.326 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:47.323+0000 7f8fce5dc640 1 --2- 172.21.15.92:0/1279932964 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8fc813e360 0x7f8fc80cf290 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f8fc8072770 tx=0x7f8fb002b910 comp rx=0 tx=0).stop 2023-11-24T06:12:47.326 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:47.323+0000 7f8fce5dc640 1 -- 172.21.15.92:0/1279932964 shutdown_connections 2023-11-24T06:12:47.326 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:47.323+0000 7f8fce5dc640 1 --2- 172.21.15.92:0/1279932964 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8fc813e360 0x7f8fc80cf290 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:47.326 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:47.323+0000 7f8fce5dc640 1 --2- 172.21.15.92:0/1279932964 >> v2:172.21.15.118:3300/0 conn(0x7f8fc813df80 0x7f8fc815bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:47.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:47.323+0000 7f8fce5dc640 1 -- 172.21.15.92:0/1279932964 >> 172.21.15.92:0/1279932964 conn(0x7f8fc806a800 msgr2=0x7f8fc806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:47.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:47.323+0000 7f8fce5dc640 1 -- 172.21.15.92:0/1279932964 shutdown_connections 2023-11-24T06:12:47.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:47.323+0000 7f8fce5dc640 1 -- 172.21.15.92:0/1279932964 wait complete. 2023-11-24T06:12:47.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:47.324+0000 7f8fce5dc640 1 Processor -- start 2023-11-24T06:12:47.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:47.325+0000 7f8fce5dc640 1 -- start start 2023-11-24T06:12:47.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:47.325+0000 7f8fce5dc640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8fc813df80 0x7f8fc813d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:47.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:47.326+0000 7f8fce5dc640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8fc813e360 0x7f8fc8070410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:47.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:47.326+0000 7f8fce5dc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8fc815f9c0 0x7f8fc80afb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:47.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:47.326+0000 7f8fce5dc640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fc8070d10 con 0x7f8fc815f9c0 2023-11-24T06:12:47.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:47.326+0000 7f8fce5dc640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fc8070ff0 con 0x7f8fc813e360 2023-11-24T06:12:47.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:47.326+0000 7f8fce5dc640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fc80712d0 con 0x7f8fc813df80 2023-11-24T06:12:47.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:47.326+0000 7f8fccb79640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8fc815f9c0 0x7f8fc80afb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:47.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:47.326+0000 7f8fccb79640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8fc815f9c0 0x7f8fc80afb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58932/0 (socket says 172.21.15.92:58932) 2023-11-24T06:12:47.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:47.326+0000 7f8fccb79640 1 -- 172.21.15.92:0/2400096311 learned_addr learned my addr 172.21.15.92:0/2400096311 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:12:47.331 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:47.326+0000 7f8fc77fe640 1 --2- 172.21.15.92:0/2400096311 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8fc813e360 0x7f8fc8070410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:47.331 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:47.326+0000 7f8fc7fff640 1 --2- 172.21.15.92:0/2400096311 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8fc813df80 0x7f8fc813d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:47.331 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:47.326+0000 7f8fccb79640 1 -- 172.21.15.92:0/2400096311 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8fc813df80 msgr2=0x7f8fc813d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:47.331 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:47.327+0000 7f8fccb79640 1 --2- 172.21.15.92:0/2400096311 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8fc813df80 0x7f8fc813d7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:47.331 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:47.327+0000 7f8fccb79640 1 -- 172.21.15.92:0/2400096311 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8fc813e360 msgr2=0x7f8fc8070410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:47.331 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:47.327+0000 7f8fccb79640 1 --2- 172.21.15.92:0/2400096311 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8fc813e360 0x7f8fc8070410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:47.331 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:47.327+0000 7f8fccb79640 1 -- 172.21.15.92:0/2400096311 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fb00094d0 con 0x7f8fc815f9c0 2023-11-24T06:12:47.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:47.327+0000 7f8fccb79640 1 --2- 172.21.15.92:0/2400096311 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8fc815f9c0 0x7f8fc80afb60 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f8fc000c700 tx=0x7f8fc000cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:47.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:47.327+0000 7f8fc4ff9640 1 -- 172.21.15.92:0/2400096311 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fc000aa50 con 0x7f8fc815f9c0 2023-11-24T06:12:47.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:47.327+0000 7f8fc4ff9640 1 -- 172.21.15.92:0/2400096311 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fc000abb0 con 0x7f8fc815f9c0 2023-11-24T06:12:47.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:47.327+0000 7f8fc4ff9640 1 -- 172.21.15.92:0/2400096311 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fc000ce10 con 0x7f8fc815f9c0 2023-11-24T06:12:47.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:47.327+0000 7f8fce5dc640 1 -- 172.21.15.92:0/2400096311 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fc80715b0 con 0x7f8fc815f9c0 2023-11-24T06:12:47.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:47.328+0000 7f8fce5dc640 1 -- 172.21.15.92:0/2400096311 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fc813dd10 con 0x7f8fc815f9c0 2023-11-24T06:12:47.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:47.329+0000 7f8fce5dc640 1 -- 172.21.15.92:0/2400096311 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8fc8071ad0 con 0x7f8fc815f9c0 2023-11-24T06:12:47.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:47.330+0000 7f8fc4ff9640 1 -- 172.21.15.92:0/2400096311 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f8fc0012420 con 0x7f8fc815f9c0 2023-11-24T06:12:47.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:47.330+0000 7f8fc4ff9640 1 --2- 172.21.15.92:0/2400096311 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f8fa806aed0 0x7f8fa806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:47.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:47.330+0000 7f8fc4ff9640 1 -- 172.21.15.92:0/2400096311 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8fc0089a60 con 0x7f8fc815f9c0 2023-11-24T06:12:47.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:47.331+0000 7f8fc7fff640 1 --2- 172.21.15.92:0/2400096311 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f8fa806aed0 0x7f8fa806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:47.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:47.331+0000 7f8fc7fff640 1 --2- 172.21.15.92:0/2400096311 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f8fa806aed0 0x7f8fa806d390 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f8fb8006fd0 tx=0x7f8fb8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:47.338 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:47.335+0000 7f8fc4ff9640 1 -- 172.21.15.92:0/2400096311 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8fc8071ad0 con 0x7f8fc815f9c0 2023-11-24T06:12:47.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:47.528+0000 7f8fce5dc640 1 -- 172.21.15.92:0/2400096311 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f8fc8071ad0 con 0x7f8fc815f9c0 2023-11-24T06:12:47.532 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:47.529+0000 7f8fc4ff9640 1 -- 172.21.15.92:0/2400096311 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v52) v1 ==== 100+0+11 (secure 0 0 0) 0x7f8fc8071ad0 con 0x7f8fc815f9c0 2023-11-24T06:12:47.532 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 64 2023-11-24T06:12:47.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:47.531+0000 7f8fce5dc640 1 -- 172.21.15.92:0/2400096311 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f8fa806aed0 msgr2=0x7f8fa806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:47.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:47.531+0000 7f8fce5dc640 1 --2- 172.21.15.92:0/2400096311 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f8fa806aed0 0x7f8fa806d390 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f8fb8006fd0 tx=0x7f8fb8008040 comp rx=0 tx=0).stop 2023-11-24T06:12:47.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:47.531+0000 7f8fce5dc640 1 -- 172.21.15.92:0/2400096311 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8fc815f9c0 msgr2=0x7f8fc80afb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:47.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:47.531+0000 7f8fce5dc640 1 --2- 172.21.15.92:0/2400096311 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8fc815f9c0 0x7f8fc80afb60 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f8fc000c700 tx=0x7f8fc000cbc0 comp rx=0 tx=0).stop 2023-11-24T06:12:47.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:47.531+0000 7f8fce5dc640 1 -- 172.21.15.92:0/2400096311 shutdown_connections 2023-11-24T06:12:47.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:47.531+0000 7f8fce5dc640 1 --2- 172.21.15.92:0/2400096311 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f8fa806aed0 0x7f8fa806d390 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:47.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:47.531+0000 7f8fce5dc640 1 --2- 172.21.15.92:0/2400096311 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8fc815f9c0 0x7f8fc80afb60 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:47.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:47.531+0000 7f8fce5dc640 1 --2- 172.21.15.92:0/2400096311 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8fc813e360 0x7f8fc8070410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:47.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:47.531+0000 7f8fce5dc640 1 --2- 172.21.15.92:0/2400096311 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f8fc813df80 0x7f8fc813d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:47.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:47.531+0000 7f8fce5dc640 1 -- 172.21.15.92:0/2400096311 >> 172.21.15.92:0/2400096311 conn(0x7f8fc806a800 msgr2=0x7f8fc815e330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:47.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:47.532+0000 7f8fce5dc640 1 -- 172.21.15.92:0/2400096311 shutdown_connections 2023-11-24T06:12:47.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:47.532+0000 7f8fce5dc640 1 -- 172.21.15.92:0/2400096311 wait complete. 2023-11-24T06:12:47.552 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-24T06:12:47.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:47.767+0000 7f0722dc5640 1 Processor -- start 2023-11-24T06:12:47.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:47.768+0000 7f0722dc5640 1 -- start start 2023-11-24T06:12:47.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:47.769+0000 7f0722dc5640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f071c06c6b0 0x7f071c0b3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:47.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:47.769+0000 7f0722dc5640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f071c06cea0 con 0x7f071c0cd240 2023-11-24T06:12:47.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:47.769+0000 7f0722dc5640 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f071c06cbc0 con 0x7f071c09a170 2023-11-24T06:12:47.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:47.769+0000 7f0722dc5640 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f071c06f350 con 0x7f071c06c6b0 2023-11-24T06:12:47.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:47.769+0000 7f0720b61640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f071c0cd240 0x7f071c099a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:53610/0 (socket says 172.21.15.92:53610) 2023-11-24T06:12:47.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:47.769+0000 7f0720b61640 1 -- 172.21.15.92:0/1967959941 learned_addr learned my addr 172.21.15.92:0/1967959941 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-24T06:12:47.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:47.770+0000 7f0721362640 1 --2- 172.21.15.92:0/1967959941 >> v2:172.21.15.118:3300/0 conn(0x7f071c06c6b0 0x7f071c0b3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:47.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:47.770+0000 7f071b7fe640 1 -- 172.21.15.92:0/1967959941 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4033814550 0 0) 0x7f071c06cea0 con 0x7f071c0cd240 2023-11-24T06:12:47.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:47.770+0000 7f071b7fe640 1 -- 172.21.15.92:0/1967959941 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0708003520 con 0x7f071c0cd240 2023-11-24T06:12:47.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:47.770+0000 7f071b7fe640 1 -- 172.21.15.92:0/1967959941 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1871809142 0 0) 0x7f0708003520 con 0x7f071c0cd240 2023-11-24T06:12:47.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:47.770+0000 7f071b7fe640 1 -- 172.21.15.92:0/1967959941 >> v2:172.21.15.118:3300/0 conn(0x7f071c06c6b0 msgr2=0x7f071c0b3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:47.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:47.770+0000 7f071b7fe640 1 --2- 172.21.15.92:0/1967959941 >> v2:172.21.15.118:3300/0 conn(0x7f071c06c6b0 0x7f071c0b3820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:47.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:47.770+0000 7f071b7fe640 1 -- 172.21.15.92:0/1967959941 >> v1:172.21.15.118:6789/0 conn(0x7f071c09a170 legacy=0x7f071c0cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:47.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:47.770+0000 7f071b7fe640 1 -- 172.21.15.92:0/1967959941 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f071c06f630 con 0x7f071c0cd240 2023-11-24T06:12:47.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:47.770+0000 7f071b7fe640 1 -- 172.21.15.92:0/1967959941 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2607661350 0 0) 0x7f070c0019f0 con 0x7f071c0cd240 2023-11-24T06:12:47.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:47.770+0000 7f0721362640 1 --2- 172.21.15.92:0/1967959941 >> v2:172.21.15.118:3300/0 conn(0x7f071c06c6b0 0x7f071c0b3820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:12:47.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:47.770+0000 7f071b7fe640 1 -- 172.21.15.92:0/1967959941 >> v1:172.21.15.92:6789/0 conn(0x7f071c0cd240 legacy=0x7f071c099a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:47.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:47.770+0000 7f071b7fe640 1 --2- 172.21.15.92:0/1967959941 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0708003970 0x7f0708003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:47.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:47.771+0000 7f071b7fe640 1 --2- 172.21.15.92:0/1967959941 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f07080047c0 0x7f0708004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:47.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:47.771+0000 7f071b7fe640 1 --2- 172.21.15.92:0/1967959941 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0708005180 0x7f070800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:47.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:47.771+0000 7f071b7fe640 1 -- 172.21.15.92:0/1967959941 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f071c06f350 con 0x7f0708003970 2023-11-24T06:12:47.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:47.771+0000 7f071b7fe640 1 -- 172.21.15.92:0/1967959941 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f070800c0d0 con 0x7f07080047c0 2023-11-24T06:12:47.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:47.771+0000 7f071b7fe640 1 -- 172.21.15.92:0/1967959941 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f070800c290 con 0x7f0708005180 2023-11-24T06:12:47.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:47.771+0000 7f0720b61640 1 --2- 172.21.15.92:0/1967959941 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0708003970 0x7f0708003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:47.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:47.771+0000 7f071bfff640 1 --2- 172.21.15.92:0/1967959941 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f07080047c0 0x7f0708004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:47.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:47.771+0000 7f0720b61640 1 -- 172.21.15.92:0/1967959941 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0708005180 msgr2=0x7f070800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:47.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:47.771+0000 7f0720b61640 1 --2- 172.21.15.92:0/1967959941 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0708005180 0x7f070800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:47.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:47.771+0000 7f0720b61640 1 -- 172.21.15.92:0/1967959941 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f07080047c0 msgr2=0x7f0708004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:47.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:47.771+0000 7f0720b61640 1 --2- 172.21.15.92:0/1967959941 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f07080047c0 0x7f0708004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:47.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:47.771+0000 7f0720b61640 1 -- 172.21.15.92:0/1967959941 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0708003520 con 0x7f0708003970 2023-11-24T06:12:47.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:47.771+0000 7f071bfff640 1 --2- 172.21.15.92:0/1967959941 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f07080047c0 0x7f0708004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:12:47.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:47.771+0000 7f0721362640 1 -- 172.21.15.92:0/1967959941 reap_dead start 2023-11-24T06:12:47.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:47.772+0000 7f0720b61640 1 --2- 172.21.15.92:0/1967959941 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0708003970 0x7f0708003d90 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f070c00a830 tx=0x7f07080057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55b7c0784e9caadb server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:47.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:47.772+0000 7f071b7fe640 1 -- 172.21.15.92:0/1967959941 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f070c0072d0 con 0x7f0708003970 2023-11-24T06:12:47.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:47.772+0000 7f071b7fe640 1 -- 172.21.15.92:0/1967959941 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f070c02f070 con 0x7f0708003970 2023-11-24T06:12:47.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:47.772+0000 7f071b7fe640 1 -- 172.21.15.92:0/1967959941 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f070c0076f0 con 0x7f0708003970 2023-11-24T06:12:47.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:47.773+0000 7f0722dc5640 1 -- 172.21.15.92:0/1967959941 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0708003970 msgr2=0x7f0708003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:47.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:47.773+0000 7f0722dc5640 1 --2- 172.21.15.92:0/1967959941 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0708003970 0x7f0708003d90 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f070c00a830 tx=0x7f07080057f0 comp rx=0 tx=0).stop 2023-11-24T06:12:47.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:47.773+0000 7f0722dc5640 1 -- 172.21.15.92:0/1967959941 shutdown_connections 2023-11-24T06:12:47.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:47.773+0000 7f0722dc5640 1 --2- 172.21.15.92:0/1967959941 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0708003970 0x7f0708003d90 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:47.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:47.773+0000 7f0722dc5640 1 -- 172.21.15.92:0/1967959941 >> 172.21.15.92:0/1967959941 conn(0x7f071c0b2d00 msgr2=0x7f071c0b3110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:47.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:47.773+0000 7f0722dc5640 1 -- 172.21.15.92:0/1967959941 shutdown_connections 2023-11-24T06:12:47.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:47.773+0000 7f0722dc5640 1 -- 172.21.15.92:0/1967959941 wait complete. 2023-11-24T06:12:47.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:47.775+0000 7f0722dc5640 1 Processor -- start 2023-11-24T06:12:47.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:47.775+0000 7f0722dc5640 1 -- start start 2023-11-24T06:12:47.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:47.776+0000 7f0722dc5640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0708003970 0x7f071c15db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:47.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:47.776+0000 7f0722dc5640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f071c0b1ec0 0x7f071c0b2340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:47.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:47.776+0000 7f0722dc5640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f071c0c4d90 0x7f071c0c5220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:47.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:47.776+0000 7f0722dc5640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f071c0b4850 con 0x7f071c0b1ec0 2023-11-24T06:12:47.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:47.776+0000 7f0722dc5640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f071c0b2880 con 0x7f0708003970 2023-11-24T06:12:47.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:47.776+0000 7f0722dc5640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f071c0c5da0 con 0x7f071c0c4d90 2023-11-24T06:12:47.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:47.776+0000 7f071bfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f071c0b1ec0 0x7f071c0b2340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:47.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:47.776+0000 7f071bfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f071c0b1ec0 0x7f071c0b2340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58936/0 (socket says 172.21.15.92:58936) 2023-11-24T06:12:47.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:47.776+0000 7f071bfff640 1 -- 172.21.15.92:0/1568028910 learned_addr learned my addr 172.21.15.92:0/1568028910 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:12:47.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:47.777+0000 7f0720b61640 1 --2- 172.21.15.92:0/1568028910 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0708003970 0x7f071c15db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:47.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:47.777+0000 7f0721362640 1 --2- 172.21.15.92:0/1568028910 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f071c0c4d90 0x7f071c0c5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:47.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:47.777+0000 7f071bfff640 1 -- 172.21.15.92:0/1568028910 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f071c0c4d90 msgr2=0x7f071c0c5220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:47.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:47.777+0000 7f071bfff640 1 --2- 172.21.15.92:0/1568028910 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f071c0c4d90 0x7f071c0c5220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:47.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:47.777+0000 7f071bfff640 1 -- 172.21.15.92:0/1568028910 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0708003970 msgr2=0x7f071c15db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:47.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:47.777+0000 7f071bfff640 1 --2- 172.21.15.92:0/1568028910 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0708003970 0x7f071c15db20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:47.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:47.777+0000 7f071bfff640 1 -- 172.21.15.92:0/1568028910 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f071c15e060 con 0x7f071c0b1ec0 2023-11-24T06:12:47.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:47.777+0000 7f0720b61640 1 --2- 172.21.15.92:0/1568028910 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0708003970 0x7f071c15db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:12:47.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:47.777+0000 7f071bfff640 1 --2- 172.21.15.92:0/1568028910 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f071c0b1ec0 0x7f071c0b2340 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f070400d770 tx=0x7f070400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:47.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:47.777+0000 7f07197fa640 1 -- 172.21.15.92:0/1568028910 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0704005270 con 0x7f071c0b1ec0 2023-11-24T06:12:47.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:47.777+0000 7f07197fa640 1 -- 172.21.15.92:0/1568028910 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07040053d0 con 0x7f071c0b1ec0 2023-11-24T06:12:47.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:47.777+0000 7f07197fa640 1 -- 172.21.15.92:0/1568028910 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f070400b8f0 con 0x7f071c0b1ec0 2023-11-24T06:12:47.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:47.777+0000 7f0722dc5640 1 -- 172.21.15.92:0/1568028910 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f071c0c5ac0 con 0x7f071c0b1ec0 2023-11-24T06:12:47.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:47.778+0000 7f0722dc5640 1 -- 172.21.15.92:0/1568028910 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f071c0b4470 con 0x7f071c0b1ec0 2023-11-24T06:12:47.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:47.779+0000 7f0722dc5640 1 -- 172.21.15.92:0/1568028910 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f071c0675b0 con 0x7f071c0b1ec0 2023-11-24T06:12:47.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:47.780+0000 7f07197fa640 1 -- 172.21.15.92:0/1568028910 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f070400ba50 con 0x7f071c0b1ec0 2023-11-24T06:12:47.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:47.780+0000 7f07197fa640 1 --2- 172.21.15.92:0/1568028910 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f06fc06aed0 0x7f06fc06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:47.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:47.781+0000 7f07197fa640 1 -- 172.21.15.92:0/1568028910 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0704089750 con 0x7f071c0b1ec0 2023-11-24T06:12:47.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:47.781+0000 7f0720b61640 1 --2- 172.21.15.92:0/1568028910 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f06fc06aed0 0x7f06fc06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:47.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:47.781+0000 7f0720b61640 1 --2- 172.21.15.92:0/1568028910 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f06fc06aed0 0x7f06fc06d390 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f070c034000 tx=0x7f070c007000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:47.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:47.785+0000 7f07197fa640 1 -- 172.21.15.92:0/1568028910 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0704054690 con 0x7f071c0b1ec0 2023-11-24T06:12:47.991 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:47.988+0000 7f0722dc5640 1 -- 172.21.15.92:0/1568028910 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f071c06d700 con 0x7f071c0b1ec0 2023-11-24T06:12:47.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:47.989+0000 7f07197fa640 1 -- 172.21.15.92:0/1568028910 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v52) v1 ==== 80+0+0 (secure 0 0 0) 0x7f0704019310 con 0x7f071c0b1ec0 2023-11-24T06:12:47.992 INFO:teuthology.orchestra.run.smithi092.stderr:listed 0 entries 2023-11-24T06:12:47.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:47.991+0000 7f0722dc5640 1 -- 172.21.15.92:0/1568028910 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f06fc06aed0 msgr2=0x7f06fc06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:47.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:47.991+0000 7f0722dc5640 1 --2- 172.21.15.92:0/1568028910 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f06fc06aed0 0x7f06fc06d390 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f070c034000 tx=0x7f070c007000 comp rx=0 tx=0).stop 2023-11-24T06:12:47.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:47.991+0000 7f0722dc5640 1 -- 172.21.15.92:0/1568028910 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f071c0b1ec0 msgr2=0x7f071c0b2340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:47.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:47.991+0000 7f0722dc5640 1 --2- 172.21.15.92:0/1568028910 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f071c0b1ec0 0x7f071c0b2340 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f070400d770 tx=0x7f070400dc30 comp rx=0 tx=0).stop 2023-11-24T06:12:47.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:47.991+0000 7f0722dc5640 1 -- 172.21.15.92:0/1568028910 shutdown_connections 2023-11-24T06:12:47.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:47.991+0000 7f0722dc5640 1 --2- 172.21.15.92:0/1568028910 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f06fc06aed0 0x7f06fc06d390 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:47.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:47.991+0000 7f0722dc5640 1 --2- 172.21.15.92:0/1568028910 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f071c0c4d90 0x7f071c0c5220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:47.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:47.991+0000 7f0722dc5640 1 --2- 172.21.15.92:0/1568028910 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f071c0b1ec0 0x7f071c0b2340 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:47.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:47.991+0000 7f0722dc5640 1 --2- 172.21.15.92:0/1568028910 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0708003970 0x7f071c15db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:47.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:47.991+0000 7f0722dc5640 1 -- 172.21.15.92:0/1568028910 >> 172.21.15.92:0/1568028910 conn(0x7f071c0b2d00 msgr2=0x7f071c06e7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:47.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:47.992+0000 7f0722dc5640 1 -- 172.21.15.92:0/1568028910 shutdown_connections 2023-11-24T06:12:47.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:47.992+0000 7f0722dc5640 1 -- 172.21.15.92:0/1568028910 wait complete. 2023-11-24T06:12:48.013 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi092.front.sepia.ceph.com... 2023-11-24T06:12:48.013 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:12:48.013 DEBUG:teuthology.orchestra.run.smithi092:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-11-24T06:12:48.098 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi092.stderr:ceph-fuse[106211]: fuse finished with error 0 and tester_r 0 2023-11-24T06:12:48.103 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi092.front.sepia.ceph.com 2023-11-24T06:12:48.103 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:12:48.103 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-24T06:12:48.279 DEBUG:teuthology.orchestra.run.smithi118:> set -ex 2023-11-24T06:12:48.279 DEBUG:teuthology.orchestra.run.smithi118:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:12:48.330 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:12:48.543 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:48.540+0000 7efea1789640 1 Processor -- start 2023-11-24T06:12:48.543 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:48.540+0000 7efea1789640 1 -- start start 2023-11-24T06:12:48.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:48.542+0000 7efea1789640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7efe9c0cf9a0 0x7efe9c15dca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:48.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:48.542+0000 7efea1789640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7efe9c15e1e0 0x7efe9c0702c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:48.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:48.542+0000 7efea1789640 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe9c0b1ee0 con 0x7efe9c143ae0 2023-11-24T06:12:48.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:48.542+0000 7efea1789640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe9c0b2780 con 0x7efe9c15e1e0 2023-11-24T06:12:48.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:48.542+0000 7efea1789640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe9c15e990 con 0x7efe9c0cf9a0 2023-11-24T06:12:48.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:48.542+0000 7efe9b7fe640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7efe9c15e1e0 0x7efe9c0702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:48.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:48.542+0000 7efe9b7fe640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7efe9c15e1e0 0x7efe9c0702c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58938/0 (socket says 172.21.15.92:58938) 2023-11-24T06:12:48.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:48.542+0000 7efe9b7fe640 1 -- 172.21.15.92:0/4079559936 learned_addr learned my addr 172.21.15.92:0/4079559936 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:12:48.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:48.542+0000 7efe9a7fc640 1 --2- 172.21.15.92:0/4079559936 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7efe9c0cf9a0 0x7efe9c15dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:48.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:48.542+0000 7efe9b7fe640 1 -- 172.21.15.92:0/4079559936 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7efe9c0cf9a0 msgr2=0x7efe9c15dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:48.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:48.542+0000 7efe9b7fe640 1 --2- 172.21.15.92:0/4079559936 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7efe9c0cf9a0 0x7efe9c15dca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:48.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:48.542+0000 7efe9b7fe640 1 -- 172.21.15.92:0/4079559936 >> v1:172.21.15.118:6789/0 conn(0x7efe9c143ae0 legacy=0x7efe9c0cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:48.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:48.543+0000 7efe9b7fe640 1 -- 172.21.15.92:0/4079559936 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe9c0b2130 con 0x7efe9c15e1e0 2023-11-24T06:12:48.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:48.544+0000 7efe9b7fe640 1 --2- 172.21.15.92:0/4079559936 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efe9c15e1e0 0x7efe9c0702c0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7efe94009e50 tx=0x7efe9402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f376b9d02182cb5 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:48.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:48.544+0000 7efe99ffb640 1 -- 172.21.15.92:0/4079559936 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe9402ca70 con 0x7efe9c15e1e0 2023-11-24T06:12:48.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:48.544+0000 7efe99ffb640 1 -- 172.21.15.92:0/4079559936 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe9402cbd0 con 0x7efe9c15e1e0 2023-11-24T06:12:48.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:48.544+0000 7efe99ffb640 1 -- 172.21.15.92:0/4079559936 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe940343f0 con 0x7efe9c15e1e0 2023-11-24T06:12:48.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:48.544+0000 7efea1789640 1 -- 172.21.15.92:0/4079559936 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efe9c15e1e0 msgr2=0x7efe9c0702c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:48.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:48.544+0000 7efea1789640 1 --2- 172.21.15.92:0/4079559936 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efe9c15e1e0 0x7efe9c0702c0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7efe94009e50 tx=0x7efe9402bed0 comp rx=0 tx=0).stop 2023-11-24T06:12:48.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:48.545+0000 7efea1789640 1 -- 172.21.15.92:0/4079559936 shutdown_connections 2023-11-24T06:12:48.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:48.545+0000 7efea1789640 1 --2- 172.21.15.92:0/4079559936 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efe9c15e1e0 0x7efe9c0702c0 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:48.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:48.545+0000 7efea1789640 1 --2- 172.21.15.92:0/4079559936 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7efe9c0cf9a0 0x7efe9c15dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:48.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:48.545+0000 7efea1789640 1 -- 172.21.15.92:0/4079559936 >> 172.21.15.92:0/4079559936 conn(0x7efe9c06a800 msgr2=0x7efe9c06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:48.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:48.545+0000 7efea1789640 1 -- 172.21.15.92:0/4079559936 shutdown_connections 2023-11-24T06:12:48.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:48.545+0000 7efea1789640 1 -- 172.21.15.92:0/4079559936 wait complete. 2023-11-24T06:12:48.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:48.546+0000 7efea1789640 1 Processor -- start 2023-11-24T06:12:48.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:48.547+0000 7efea1789640 1 -- start start 2023-11-24T06:12:48.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:48.547+0000 7efea1789640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efe9c0cf9a0 0x7efe9c1f5ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:48.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:48.548+0000 7efea1789640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7efe9c143ae0 0x7efe9c0c4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:48.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:48.548+0000 7efea1789640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7efe9c15e1e0 0x7efe9c0c50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:48.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:48.548+0000 7efea1789640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe9c1f6220 con 0x7efe9c0cf9a0 2023-11-24T06:12:48.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:48.548+0000 7efea1789640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe9c0ad7c0 con 0x7efe9c143ae0 2023-11-24T06:12:48.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:48.548+0000 7efea1789640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe9c0adaa0 con 0x7efe9c15e1e0 2023-11-24T06:12:48.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:48.548+0000 7efe9affd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efe9c0cf9a0 0x7efe9c1f5ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:48.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:48.548+0000 7efe9affd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efe9c0cf9a0 0x7efe9c1f5ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58948/0 (socket says 172.21.15.92:58948) 2023-11-24T06:12:48.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:48.548+0000 7efe9affd640 1 -- 172.21.15.92:0/3935924879 learned_addr learned my addr 172.21.15.92:0/3935924879 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:12:48.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:48.548+0000 7efe9b7fe640 1 --2- 172.21.15.92:0/3935924879 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7efe9c15e1e0 0x7efe9c0c50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:48.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:48.548+0000 7efe9a7fc640 1 --2- 172.21.15.92:0/3935924879 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7efe9c143ae0 0x7efe9c0c4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:48.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:48.548+0000 7efe9affd640 1 -- 172.21.15.92:0/3935924879 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7efe9c15e1e0 msgr2=0x7efe9c0c50f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:48.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:48.548+0000 7efe9affd640 1 --2- 172.21.15.92:0/3935924879 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7efe9c15e1e0 0x7efe9c0c50f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:48.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:48.548+0000 7efe9affd640 1 -- 172.21.15.92:0/3935924879 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7efe9c143ae0 msgr2=0x7efe9c0c4bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:48.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:48.548+0000 7efe9affd640 1 --2- 172.21.15.92:0/3935924879 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7efe9c143ae0 0x7efe9c0c4bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:48.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:48.548+0000 7efe9affd640 1 -- 172.21.15.92:0/3935924879 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe94009b30 con 0x7efe9c0cf9a0 2023-11-24T06:12:48.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:48.549+0000 7efe9affd640 1 --2- 172.21.15.92:0/3935924879 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efe9c0cf9a0 0x7efe9c1f5ce0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7efe8c00c430 tx=0x7efe8c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:48.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:48.549+0000 7efe837fe640 1 -- 172.21.15.92:0/3935924879 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe8c011790 con 0x7efe9c0cf9a0 2023-11-24T06:12:48.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:48.549+0000 7efe837fe640 1 -- 172.21.15.92:0/3935924879 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe8c0118f0 con 0x7efe9c0cf9a0 2023-11-24T06:12:48.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:48.549+0000 7efe837fe640 1 -- 172.21.15.92:0/3935924879 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe8c011ca0 con 0x7efe9c0cf9a0 2023-11-24T06:12:48.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:48.549+0000 7efea1789640 1 -- 172.21.15.92:0/3935924879 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe9c0add80 con 0x7efe9c0cf9a0 2023-11-24T06:12:48.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:48.549+0000 7efea1789640 1 -- 172.21.15.92:0/3935924879 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe9c0ae340 con 0x7efe9c0cf9a0 2023-11-24T06:12:48.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:48.550+0000 7efea1789640 1 -- 172.21.15.92:0/3935924879 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe9c06ad00 con 0x7efe9c0cf9a0 2023-11-24T06:12:48.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:48.551+0000 7efe837fe640 1 -- 172.21.15.92:0/3935924879 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7efe8c01c460 con 0x7efe9c0cf9a0 2023-11-24T06:12:48.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:48.552+0000 7efe837fe640 1 --2- 172.21.15.92:0/3935924879 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7efe8806aed0 0x7efe8806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:48.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:48.553+0000 7efe837fe640 1 -- 172.21.15.92:0/3935924879 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 5709+0+0 (secure 0 0 0) 0x7efe8c089540 con 0x7efe9c0cf9a0 2023-11-24T06:12:48.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:48.553+0000 7efe9a7fc640 1 --2- 172.21.15.92:0/3935924879 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7efe8806aed0 0x7efe8806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:48.556 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:48.553+0000 7efe9a7fc640 1 --2- 172.21.15.92:0/3935924879 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7efe8806aed0 0x7efe8806d390 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7efe9c1402a0 tx=0x7efe90008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:48.560 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:48.557+0000 7efe837fe640 1 -- 172.21.15.92:0/3935924879 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7efe8c054600 con 0x7efe9c0cf9a0 2023-11-24T06:12:48.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:48.751+0000 7efea1789640 1 -- 172.21.15.92:0/3935924879 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efe9c0675d0 con 0x7efe9c0cf9a0 2023-11-24T06:12:48.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:48.753+0000 7efe837fe640 1 -- 172.21.15.92:0/3935924879 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v52) v1 ==== 74+0+16517 (secure 0 0 0) 0x7efe8c057c20 con 0x7efe9c0cf9a0 2023-11-24T06:12:48.782 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-24T06:12:48.782 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":52,"fsid":"2d6dee5f-8aa3-470f-931a-50ade963b1cb","created":"2023-11-24T06:10:04.707204+0000","modified":"2023-11-24T06:12:32.152035+0000","last_up_change":"2023-11-24T06:10:17.598352+0000","last_in_change":"2023-11-24T06:10:09.040793+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:10:17.733098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:12:28.410305+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-11-24T06:12:29.625599+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ff37a9fb-135f-471f-b4b0-f7b5772006c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6803","nonce":20560714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6805","nonce":20560714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6809","nonce":20560714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6807","nonce":20560714}]},"public_addr":"172.21.15.92:6803/20560714","cluster_addr":"172.21.15.92:6805/20560714","heartbeat_back_addr":"172.21.15.92:6809/20560714","heartbeat_front_addr":"172.21.15.92:6807/20560714","state":["exists","up"]},{"osd":1,"uuid":"03cdd353-dcf1-4f38-94fe-a795137c06b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6819","nonce":2451239906}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6821","nonce":2451239906}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6825","nonce":2451239906}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6823","nonce":2451239906}]},"public_addr":"172.21.15.92:6819/2451239906","cluster_addr":"172.21.15.92:6821/2451239906","heartbeat_back_addr":"172.21.15.92:6825/2451239906","heartbeat_front_addr":"172.21.15.92:6823/2451239906","state":["exists","up"]},{"osd":2,"uuid":"a4a20d80-5d57-484b-becd-9aa8070dd3aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6811","nonce":1124979277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6813","nonce":1124979277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6817","nonce":1124979277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6815","nonce":1124979277}]},"public_addr":"172.21.15.92:6811/1124979277","cluster_addr":"172.21.15.92:6813/1124979277","heartbeat_back_addr":"172.21.15.92:6817/1124979277","heartbeat_front_addr":"172.21.15.92:6815/1124979277","state":["exists","up"]},{"osd":3,"uuid":"4648d7ce-9b29-4583-9425-4ba8fd2b1903","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6827","nonce":1734115782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6829","nonce":1734115782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6833","nonce":1734115782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6831","nonce":1734115782}]},"public_addr":"172.21.15.92:6827/1734115782","cluster_addr":"172.21.15.92:6829/1734115782","heartbeat_back_addr":"172.21.15.92:6833/1734115782","heartbeat_front_addr":"172.21.15.92:6831/1734115782","state":["exists","up"]},{"osd":4,"uuid":"3bc9d7eb-c6fa-4fcf-a3ec-ade604e05aac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6819","nonce":3425937229}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6822","nonce":3425937229}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6830","nonce":3425937229}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6826","nonce":3425937229}]},"public_addr":"172.21.15.118:6819/3425937229","cluster_addr":"172.21.15.118:6822/3425937229","heartbeat_back_addr":"172.21.15.118:6830/3425937229","heartbeat_front_addr":"172.21.15.118:6826/3425937229","state":["exists","up"]},{"osd":5,"uuid":"dce9ba6d-59ed-46d1-84fc-a84b13de10c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6801","nonce":827455984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6803","nonce":827455984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6807","nonce":827455984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6805","nonce":827455984}]},"public_addr":"172.21.15.118:6801/827455984","cluster_addr":"172.21.15.118:6803/827455984","heartbeat_back_addr":"172.21.15.118:6807/827455984","heartbeat_front_addr":"172.21.15.118:6805/827455984","state":["exists","up"]},{"osd":6,"uuid":"82aec504-ae58-4709-8d30-46f2b12081b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6809","nonce":2946986141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6811","nonce":2946986141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6815","nonce":2946986141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6813","nonce":2946986141}]},"public_addr":"172.21.15.118:6809/2946986141","cluster_addr":"172.21.15.118:6811/2946986141","heartbeat_back_addr":"172.21.15.118:6815/2946986141","heartbeat_front_addr":"172.21.15.118:6813/2946986141","state":["exists","up"]},{"osd":7,"uuid":"4ed4c624-c88f-4d35-bdb1-bf209b5dffb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6817","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6818","nonce":48122059}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6821","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6823","nonce":48122059}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6829","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6831","nonce":48122059}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6825","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6827","nonce":48122059}]},"public_addr":"172.21.15.118:6818/48122059","cluster_addr":"172.21.15.118:6823/48122059","heartbeat_back_addr":"172.21.15.118:6831/48122059","heartbeat_front_addr":"172.21.15.118:6827/48122059","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.536664+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.368324+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.271201+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.342971+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.621832+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.387110+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.632946+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.416032+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:12:48.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:48.755+0000 7efea1789640 1 -- 172.21.15.92:0/3935924879 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7efe8806aed0 msgr2=0x7efe8806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:48.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:48.755+0000 7efea1789640 1 --2- 172.21.15.92:0/3935924879 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7efe8806aed0 0x7efe8806d390 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7efe9c1402a0 tx=0x7efe90008040 comp rx=0 tx=0).stop 2023-11-24T06:12:48.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:48.755+0000 7efea1789640 1 -- 172.21.15.92:0/3935924879 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efe9c0cf9a0 msgr2=0x7efe9c1f5ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:48.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:48.755+0000 7efea1789640 1 --2- 172.21.15.92:0/3935924879 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efe9c0cf9a0 0x7efe9c1f5ce0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7efe8c00c430 tx=0x7efe8c00c8f0 comp rx=0 tx=0).stop 2023-11-24T06:12:48.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:48.755+0000 7efea1789640 1 -- 172.21.15.92:0/3935924879 shutdown_connections 2023-11-24T06:12:48.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:48.755+0000 7efea1789640 1 --2- 172.21.15.92:0/3935924879 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7efe8806aed0 0x7efe8806d390 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:48.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:48.755+0000 7efea1789640 1 --2- 172.21.15.92:0/3935924879 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7efe9c15e1e0 0x7efe9c0c50f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:48.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:48.755+0000 7efea1789640 1 --2- 172.21.15.92:0/3935924879 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7efe9c143ae0 0x7efe9c0c4bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:48.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:48.756+0000 7efea1789640 1 --2- 172.21.15.92:0/3935924879 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efe9c0cf9a0 0x7efe9c1f5ce0 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:48.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:48.756+0000 7efea1789640 1 -- 172.21.15.92:0/3935924879 >> 172.21.15.92:0/3935924879 conn(0x7efe9c06a800 msgr2=0x7efe9c15b710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:48.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:48.756+0000 7efea1789640 1 -- 172.21.15.92:0/3935924879 shutdown_connections 2023-11-24T06:12:48.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:48.756+0000 7efea1789640 1 -- 172.21.15.92:0/3935924879 wait complete. 2023-11-24T06:12:48.785 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T06:10:17.733098+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T06:12:28.410305+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '51', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T06:12:29.625599+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '52', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T06:12:48.785 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T06:12:49.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.001+0000 7f1ef861c640 1 Processor -- start 2023-11-24T06:12:49.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.001+0000 7f1ef861c640 1 -- start start 2023-11-24T06:12:49.006 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.003+0000 7f1ef861c640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1ef0143ae0 0x7f1ef00cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:49.006 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.003+0000 7f1ef861c640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f1ef015e3b0 0x7f1ef00702c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:49.006 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.003+0000 7f1ef861c640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ef00cfc30 con 0x7f1ef00cf7d0 2023-11-24T06:12:49.006 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.003+0000 7f1ef861c640 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ef00b1ee0 con 0x7f1ef015e3b0 2023-11-24T06:12:49.006 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.003+0000 7f1ef861c640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ef00b2780 con 0x7f1ef0143ae0 2023-11-24T06:12:49.006 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.003+0000 7f1ef5bb7640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f1ef00cf7d0 0x7f1ef015dca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:53262/0 (socket says 172.21.15.92:53262) 2023-11-24T06:12:49.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.003+0000 7f1ef5bb7640 1 -- 172.21.15.92:0/3961364734 learned_addr learned my addr 172.21.15.92:0/3961364734 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-24T06:12:49.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.003+0000 7f1ef63b8640 1 --2- 172.21.15.92:0/3961364734 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1ef0143ae0 0x7f1ef00cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:49.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.004+0000 7f1ef6bb9640 1 --2- 172.21.15.92:0/3961364734 >> v2:172.21.15.118:3300/0 conn(0x7f1ef015e3b0 0x7f1ef00702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:49.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.004+0000 7f1ef53b6640 1 -- 172.21.15.92:0/3961364734 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3757066034 0 0) 0x7f1ef00cfc30 con 0x7f1ef00cf7d0 2023-11-24T06:12:49.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.004+0000 7f1ef53b6640 1 -- 172.21.15.92:0/3961364734 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1edc003520 con 0x7f1ef00cf7d0 2023-11-24T06:12:49.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.004+0000 7f1ef53b6640 1 -- 172.21.15.92:0/3961364734 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1156743928 0 0) 0x7f1edc003520 con 0x7f1ef00cf7d0 2023-11-24T06:12:49.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.004+0000 7f1ef53b6640 1 -- 172.21.15.92:0/3961364734 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1ef0143ae0 msgr2=0x7f1ef00cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:49.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.004+0000 7f1ef53b6640 1 --2- 172.21.15.92:0/3961364734 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1ef0143ae0 0x7f1ef00cf290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:49.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.004+0000 7f1ef53b6640 1 -- 172.21.15.92:0/3961364734 >> v2:172.21.15.118:3300/0 conn(0x7f1ef015e3b0 msgr2=0x7f1ef00702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:49.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.004+0000 7f1ef53b6640 1 --2- 172.21.15.92:0/3961364734 >> v2:172.21.15.118:3300/0 conn(0x7f1ef015e3b0 0x7f1ef00702c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:49.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.004+0000 7f1ef53b6640 1 -- 172.21.15.92:0/3961364734 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ef015e990 con 0x7f1ef00cf7d0 2023-11-24T06:12:49.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.004+0000 7f1ef53b6640 1 -- 172.21.15.92:0/3961364734 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2607661350 0 0) 0x7f1ee0001670 con 0x7f1ef00cf7d0 2023-11-24T06:12:49.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.004+0000 7f1ef53b6640 1 -- 172.21.15.92:0/3961364734 >> v1:172.21.15.92:6789/0 conn(0x7f1ef00cf7d0 legacy=0x7f1ef015dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:49.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.004+0000 7f1ef53b6640 1 --2- 172.21.15.92:0/3961364734 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1edc003970 0x7f1edc003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:49.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.004+0000 7f1ef53b6640 1 --2- 172.21.15.92:0/3961364734 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1edc0047c0 0x7f1edc004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:49.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.004+0000 7f1ef53b6640 1 --2- 172.21.15.92:0/3961364734 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1ef0143ae0 0x7f1edc005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:49.010 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.004+0000 7f1ef53b6640 1 -- 172.21.15.92:0/3961364734 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ef00b1ee0 con 0x7f1edc003970 2023-11-24T06:12:49.010 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.004+0000 7f1ef53b6640 1 -- 172.21.15.92:0/3961364734 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ef00b2780 con 0x7f1edc0047c0 2023-11-24T06:12:49.010 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.004+0000 7f1ef53b6640 1 -- 172.21.15.92:0/3961364734 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1edc00db90 con 0x7f1ef0143ae0 2023-11-24T06:12:49.010 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.004+0000 7f1ef63b8640 1 --2- 172.21.15.92:0/3961364734 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1edc003970 0x7f1edc003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:49.010 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.005+0000 7f1ef63b8640 1 -- 172.21.15.92:0/3961364734 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1ef0143ae0 msgr2=0x7f1edc005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:49.010 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.005+0000 7f1ef63b8640 1 --2- 172.21.15.92:0/3961364734 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1ef0143ae0 0x7f1edc005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:49.010 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.005+0000 7f1ef5bb7640 1 --2- 172.21.15.92:0/3961364734 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1edc0047c0 0x7f1edc004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:49.010 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.005+0000 7f1ef63b8640 1 -- 172.21.15.92:0/3961364734 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1edc0047c0 msgr2=0x7f1edc004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:49.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.005+0000 7f1ef63b8640 1 --2- 172.21.15.92:0/3961364734 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1edc0047c0 0x7f1edc004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:49.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.005+0000 7f1ef63b8640 1 -- 172.21.15.92:0/3961364734 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1edc003520 con 0x7f1edc003970 2023-11-24T06:12:49.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.005+0000 7f1ef5bb7640 1 --2- 172.21.15.92:0/3961364734 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1edc0047c0 0x7f1edc004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:12:49.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.006+0000 7f1ef63b8640 1 --2- 172.21.15.92:0/3961364734 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1edc003970 0x7f1edc003d90 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f1eec009ac0 tx=0x7f1edc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f24e4fde9d85bc7f server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:49.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.006+0000 7f1ef53b6640 1 -- 172.21.15.92:0/3961364734 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eec02c710 con 0x7f1edc003970 2023-11-24T06:12:49.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.006+0000 7f1ef53b6640 1 -- 172.21.15.92:0/3961364734 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1eec02c870 con 0x7f1edc003970 2023-11-24T06:12:49.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.006+0000 7f1ef53b6640 1 -- 172.21.15.92:0/3961364734 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eec02cc00 con 0x7f1edc003970 2023-11-24T06:12:49.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.007+0000 7f1ef861c640 1 -- 172.21.15.92:0/3961364734 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1edc003970 msgr2=0x7f1edc003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:49.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.007+0000 7f1ef861c640 1 --2- 172.21.15.92:0/3961364734 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1edc003970 0x7f1edc003d90 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f1eec009ac0 tx=0x7f1edc0057f0 comp rx=0 tx=0).stop 2023-11-24T06:12:49.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.007+0000 7f1ef6bb9640 1 -- 172.21.15.92:0/3961364734 reap_dead start 2023-11-24T06:12:49.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.007+0000 7f1ef861c640 1 -- 172.21.15.92:0/3961364734 shutdown_connections 2023-11-24T06:12:49.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.007+0000 7f1ef861c640 1 -- 172.21.15.92:0/3961364734 >> 172.21.15.92:0/3961364734 conn(0x7f1ef006a800 msgr2=0x7f1ef006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:49.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.007+0000 7f1ef861c640 1 -- 172.21.15.92:0/3961364734 shutdown_connections 2023-11-24T06:12:49.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.007+0000 7f1ef861c640 1 -- 172.21.15.92:0/3961364734 wait complete. 2023-11-24T06:12:49.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.009+0000 7f1ef861c640 1 Processor -- start 2023-11-24T06:12:49.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.009+0000 7f1ef861c640 1 -- start start 2023-11-24T06:12:49.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.010+0000 7f1ef861c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1edc003970 0x7f1ef00c4320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:49.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.010+0000 7f1ef861c640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1ef013a9c0 0x7f1ef00b0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:49.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.010+0000 7f1ef861c640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1ef00b0420 0x7f1ef00b1100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:49.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.010+0000 7f1ef861c640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ef00c8160 con 0x7f1edc003970 2023-11-24T06:12:49.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.010+0000 7f1ef861c640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ef00c55a0 con 0x7f1ef00b0420 2023-11-24T06:12:49.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.010+0000 7f1ef861c640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ef0141e70 con 0x7f1ef013a9c0 2023-11-24T06:12:49.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.010+0000 7f1ef63b8640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1edc003970 0x7f1ef00c4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:49.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.010+0000 7f1ef63b8640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1edc003970 0x7f1ef00c4320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:55686/0 (socket says 172.21.15.92:55686) 2023-11-24T06:12:49.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.010+0000 7f1ef63b8640 1 -- 172.21.15.92:0/3409425755 learned_addr learned my addr 172.21.15.92:0/3409425755 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:12:49.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.010+0000 7f1ef5bb7640 1 --2- 172.21.15.92:0/3409425755 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1ef013a9c0 0x7f1ef00b0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:49.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.010+0000 7f1ef63b8640 1 -- 172.21.15.92:0/3409425755 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1ef013a9c0 msgr2=0x7f1ef00b0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:49.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.010+0000 7f1ef6bb9640 1 --2- 172.21.15.92:0/3409425755 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1ef00b0420 0x7f1ef00b1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:49.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.010+0000 7f1ef63b8640 1 --2- 172.21.15.92:0/3409425755 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1ef013a9c0 0x7f1ef00b0bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:49.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.011+0000 7f1ef63b8640 1 -- 172.21.15.92:0/3409425755 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1ef00b0420 msgr2=0x7f1ef00b1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:49.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.011+0000 7f1ef63b8640 1 --2- 172.21.15.92:0/3409425755 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1ef00b0420 0x7f1ef00b1100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:49.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.011+0000 7f1ef63b8640 1 -- 172.21.15.92:0/3409425755 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ef00c4a20 con 0x7f1edc003970 2023-11-24T06:12:49.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.011+0000 7f1ef6bb9640 1 --2- 172.21.15.92:0/3409425755 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1ef00b0420 0x7f1ef00b1100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:12:49.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.011+0000 7f1ef63b8640 1 --2- 172.21.15.92:0/3409425755 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1edc003970 0x7f1ef00c4320 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f1eec02c2b0 tx=0x7f1eec004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:49.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.011+0000 7f1ee6ffd640 1 -- 172.21.15.92:0/3409425755 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eec003810 con 0x7f1edc003970 2023-11-24T06:12:49.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.011+0000 7f1ee6ffd640 1 -- 172.21.15.92:0/3409425755 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1eec02de90 con 0x7f1edc003970 2023-11-24T06:12:49.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.011+0000 7f1ee6ffd640 1 -- 172.21.15.92:0/3409425755 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eec003bb0 con 0x7f1edc003970 2023-11-24T06:12:49.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.011+0000 7f1ef861c640 1 -- 172.21.15.92:0/3409425755 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ef00c4d00 con 0x7f1edc003970 2023-11-24T06:12:49.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.011+0000 7f1ef861c640 1 -- 172.21.15.92:0/3409425755 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ef00c52c0 con 0x7f1edc003970 2023-11-24T06:12:49.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.013+0000 7f1ef861c640 1 -- 172.21.15.92:0/3409425755 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ef00675d0 con 0x7f1edc003970 2023-11-24T06:12:49.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.013+0000 7f1ee6ffd640 1 -- 172.21.15.92:0/3409425755 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f1eec003d10 con 0x7f1edc003970 2023-11-24T06:12:49.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.015+0000 7f1ee6ffd640 1 --2- 172.21.15.92:0/3409425755 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f1ed406af00 0x7f1ed406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:49.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.015+0000 7f1ee6ffd640 1 -- 172.21.15.92:0/3409425755 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1eec0ab4e0 con 0x7f1edc003970 2023-11-24T06:12:49.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.015+0000 7f1ef5bb7640 1 --2- 172.21.15.92:0/3409425755 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f1ed406af00 0x7f1ed406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:49.019 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.016+0000 7f1ef5bb7640 1 --2- 172.21.15.92:0/3409425755 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f1ed406af00 0x7f1ed406d3c0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f1ee0000f60 tx=0x7f1ee0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:49.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.020+0000 7f1ee6ffd640 1 -- 172.21.15.92:0/3409425755 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1eec0765a0 con 0x7f1edc003970 2023-11-24T06:12:49.215 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.211+0000 7f1ef861c640 1 -- 172.21.15.92:0/3409425755 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1ef00c4100 con 0x7f1edc003970 2023-11-24T06:12:49.215 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.212+0000 7f1ee6ffd640 1 -- 172.21.15.92:0/3409425755 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v52) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1eec079bc0 con 0x7f1edc003970 2023-11-24T06:12:49.216 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 1 2023-11-24T06:12:49.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.215+0000 7f1ef861c640 1 -- 172.21.15.92:0/3409425755 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f1ed406af00 msgr2=0x7f1ed406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:49.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.215+0000 7f1ef861c640 1 --2- 172.21.15.92:0/3409425755 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f1ed406af00 0x7f1ed406d3c0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f1ee0000f60 tx=0x7f1ee0009040 comp rx=0 tx=0).stop 2023-11-24T06:12:49.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.215+0000 7f1ef861c640 1 -- 172.21.15.92:0/3409425755 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1edc003970 msgr2=0x7f1ef00c4320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:49.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.215+0000 7f1ef861c640 1 --2- 172.21.15.92:0/3409425755 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1edc003970 0x7f1ef00c4320 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f1eec02c2b0 tx=0x7f1eec004730 comp rx=0 tx=0).stop 2023-11-24T06:12:49.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.215+0000 7f1ef861c640 1 -- 172.21.15.92:0/3409425755 shutdown_connections 2023-11-24T06:12:49.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.215+0000 7f1ef861c640 1 --2- 172.21.15.92:0/3409425755 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f1ed406af00 0x7f1ed406d3c0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:49.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.215+0000 7f1ef861c640 1 --2- 172.21.15.92:0/3409425755 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1ef00b0420 0x7f1ef00b1100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:49.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.215+0000 7f1ef861c640 1 --2- 172.21.15.92:0/3409425755 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1ef013a9c0 0x7f1ef00b0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:49.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.215+0000 7f1ef861c640 1 --2- 172.21.15.92:0/3409425755 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1edc003970 0x7f1ef00c4320 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:49.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.215+0000 7f1ef861c640 1 -- 172.21.15.92:0/3409425755 >> 172.21.15.92:0/3409425755 conn(0x7f1ef006a800 msgr2=0x7f1ef006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:49.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.215+0000 7f1ef861c640 1 -- 172.21.15.92:0/3409425755 shutdown_connections 2023-11-24T06:12:49.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.215+0000 7f1ef861c640 1 -- 172.21.15.92:0/3409425755 wait complete. 2023-11-24T06:12:49.238 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T06:12:49.449 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.446+0000 7f4dd4fec640 1 Processor -- start 2023-11-24T06:12:49.449 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.446+0000 7f4dd4fec640 1 -- start start 2023-11-24T06:12:49.451 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.448+0000 7f4dd4fec640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f4dd013e360 0x7f4dd00cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:49.451 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.448+0000 7f4dd4fec640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4dd015c5f0 con 0x7f4dd013df80 2023-11-24T06:12:49.451 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.448+0000 7f4dd4fec640 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4dd006c470 con 0x7f4dd015f9c0 2023-11-24T06:12:49.451 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.448+0000 7f4dd4fec640 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4dd006c750 con 0x7f4dd013e360 2023-11-24T06:12:49.451 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.448+0000 7f4dce59c640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f4dd013df80 0x7f4dd015bee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:53270/0 (socket says 172.21.15.92:53270) 2023-11-24T06:12:49.451 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.448+0000 7f4dce59c640 1 -- 172.21.15.92:0/847676979 learned_addr learned my addr 172.21.15.92:0/847676979 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-24T06:12:49.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.448+0000 7f4dcdd9b640 1 -- 172.21.15.92:0/847676979 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1781270978 0 0) 0x7f4dd015c5f0 con 0x7f4dd013df80 2023-11-24T06:12:49.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.448+0000 7f4dcdd9b640 1 -- 172.21.15.92:0/847676979 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4dbc003520 con 0x7f4dd013df80 2023-11-24T06:12:49.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.448+0000 7f4dc7fff640 1 --2- 172.21.15.92:0/847676979 >> v2:172.21.15.118:3300/0 conn(0x7f4dd013e360 0x7f4dd00cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:49.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.449+0000 7f4dcdd9b640 1 -- 172.21.15.92:0/847676979 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1144022397 0 0) 0x7f4dbc003520 con 0x7f4dd013df80 2023-11-24T06:12:49.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.449+0000 7f4dcdd9b640 1 -- 172.21.15.92:0/847676979 >> v2:172.21.15.118:3300/0 conn(0x7f4dd013e360 msgr2=0x7f4dd00cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:49.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.449+0000 7f4dcdd9b640 1 --2- 172.21.15.92:0/847676979 >> v2:172.21.15.118:3300/0 conn(0x7f4dd013e360 0x7f4dd00cf290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:49.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.449+0000 7f4dcdd9b640 1 -- 172.21.15.92:0/847676979 >> v1:172.21.15.118:6789/0 conn(0x7f4dd015f9c0 legacy=0x7f4dd00cd840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:49.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.449+0000 7f4dcdd9b640 1 -- 172.21.15.92:0/847676979 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4dd006ca30 con 0x7f4dd013df80 2023-11-24T06:12:49.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.449+0000 7f4dcdd9b640 1 -- 172.21.15.92:0/847676979 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2607661350 0 0) 0x7f4dc0001690 con 0x7f4dd013df80 2023-11-24T06:12:49.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.449+0000 7f4dcdd9b640 1 -- 172.21.15.92:0/847676979 >> v1:172.21.15.92:6789/0 conn(0x7f4dd013df80 legacy=0x7f4dd015bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:49.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.449+0000 7f4dcdd9b640 1 --2- 172.21.15.92:0/847676979 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4dbc0039c0 0x7f4dbc003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:49.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.449+0000 7f4dcdd9b640 1 --2- 172.21.15.92:0/847676979 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4dbc0047c0 0x7f4dbc004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:49.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.449+0000 7f4dcdd9b640 1 --2- 172.21.15.92:0/847676979 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4dbc0051d0 0x7f4dbc00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:49.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.449+0000 7f4dcdd9b640 1 -- 172.21.15.92:0/847676979 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dd006c750 con 0x7f4dbc0051d0 2023-11-24T06:12:49.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.449+0000 7f4dcdd9b640 1 -- 172.21.15.92:0/847676979 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dbc00c100 con 0x7f4dbc0047c0 2023-11-24T06:12:49.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.449+0000 7f4dcdd9b640 1 -- 172.21.15.92:0/847676979 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4dbc00c2c0 con 0x7f4dbc0039c0 2023-11-24T06:12:49.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.449+0000 7f4dced9d640 1 --2- 172.21.15.92:0/847676979 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4dbc0051d0 0x7f4dbc00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:49.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.450+0000 7f4dce59c640 1 --2- 172.21.15.92:0/847676979 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4dbc0039c0 0x7f4dbc003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:49.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.450+0000 7f4dc7fff640 1 --2- 172.21.15.92:0/847676979 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4dbc0047c0 0x7f4dbc004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:49.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.450+0000 7f4dced9d640 1 -- 172.21.15.92:0/847676979 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4dbc0039c0 msgr2=0x7f4dbc003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:49.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.450+0000 7f4dced9d640 1 --2- 172.21.15.92:0/847676979 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4dbc0039c0 0x7f4dbc003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:49.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.450+0000 7f4dced9d640 1 -- 172.21.15.92:0/847676979 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4dbc0047c0 msgr2=0x7f4dbc004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:49.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.450+0000 7f4dced9d640 1 --2- 172.21.15.92:0/847676979 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4dbc0047c0 0x7f4dbc004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:49.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.450+0000 7f4dced9d640 1 -- 172.21.15.92:0/847676979 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4dbc003520 con 0x7f4dbc0051d0 2023-11-24T06:12:49.456 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.451+0000 7f4dced9d640 1 -- 172.21.15.92:0/847676979 reap_dead start 2023-11-24T06:12:49.456 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.451+0000 7f4dced9d640 1 --2- 172.21.15.92:0/847676979 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4dbc0051d0 0x7f4dbc00b980 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f4dd015b6f0 tx=0x7f4dbc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9be2e8a32ca55535 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:49.456 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.451+0000 7f4dcdd9b640 1 -- 172.21.15.92:0/847676979 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dc8031670 con 0x7f4dbc0051d0 2023-11-24T06:12:49.456 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.451+0000 7f4dcdd9b640 1 -- 172.21.15.92:0/847676979 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4dc80317d0 con 0x7f4dbc0051d0 2023-11-24T06:12:49.456 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.451+0000 7f4dcdd9b640 1 -- 172.21.15.92:0/847676979 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dc8031b60 con 0x7f4dbc0051d0 2023-11-24T06:12:49.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.452+0000 7f4dd4fec640 1 -- 172.21.15.92:0/847676979 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4dbc0051d0 msgr2=0x7f4dbc00b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:49.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.452+0000 7f4dd4fec640 1 --2- 172.21.15.92:0/847676979 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4dbc0051d0 0x7f4dbc00b980 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f4dd015b6f0 tx=0x7f4dbc0057f0 comp rx=0 tx=0).stop 2023-11-24T06:12:49.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.452+0000 7f4dd4fec640 1 -- 172.21.15.92:0/847676979 shutdown_connections 2023-11-24T06:12:49.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.452+0000 7f4dd4fec640 1 --2- 172.21.15.92:0/847676979 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4dbc0051d0 0x7f4dbc00b980 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:49.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.452+0000 7f4dd4fec640 1 -- 172.21.15.92:0/847676979 >> 172.21.15.92:0/847676979 conn(0x7f4dd006a800 msgr2=0x7f4dd006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:49.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.452+0000 7f4dd4fec640 1 -- 172.21.15.92:0/847676979 shutdown_connections 2023-11-24T06:12:49.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.452+0000 7f4dd4fec640 1 -- 172.21.15.92:0/847676979 wait complete. 2023-11-24T06:12:49.459 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.454+0000 7f4dd4fec640 1 Processor -- start 2023-11-24T06:12:49.459 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.454+0000 7f4dd4fec640 1 -- start start 2023-11-24T06:12:49.459 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.455+0000 7f4dd4fec640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4dbc0051d0 0x7f4dd014dd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:49.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.455+0000 7f4dd4fec640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4dd013d940 0x7f4dd014b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:49.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.455+0000 7f4dd4fec640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4dd014b730 0x7f4dd0149a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:49.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.455+0000 7f4dd4fec640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dd013ddc0 con 0x7f4dd013d940 2023-11-24T06:12:49.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.455+0000 7f4dd4fec640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dd014bcb0 con 0x7f4dbc0051d0 2023-11-24T06:12:49.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.455+0000 7f4dd4fec640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4dd00703d0 con 0x7f4dd014b730 2023-11-24T06:12:49.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.455+0000 7f4dc7fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4dd013d940 0x7f4dd014b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:49.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.455+0000 7f4dc7fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4dd013d940 0x7f4dd014b1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:55716/0 (socket says 172.21.15.92:55716) 2023-11-24T06:12:49.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.455+0000 7f4dc7fff640 1 -- 172.21.15.92:0/121506353 learned_addr learned my addr 172.21.15.92:0/121506353 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:12:49.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.456+0000 7f4dce59c640 1 --2- 172.21.15.92:0/121506353 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4dbc0051d0 0x7f4dd014dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:49.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.456+0000 7f4dc7fff640 1 -- 172.21.15.92:0/121506353 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4dd014b730 msgr2=0x7f4dd0149a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:49.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.456+0000 7f4dc7fff640 1 --2- 172.21.15.92:0/121506353 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4dd014b730 0x7f4dd0149a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:49.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.456+0000 7f4dc7fff640 1 -- 172.21.15.92:0/121506353 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4dbc0051d0 msgr2=0x7f4dd014dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:49.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.456+0000 7f4dc7fff640 1 --2- 172.21.15.92:0/121506353 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4dbc0051d0 0x7f4dd014dd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:49.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.456+0000 7f4dc7fff640 1 -- 172.21.15.92:0/121506353 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4dd00706b0 con 0x7f4dd013d940 2023-11-24T06:12:49.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.456+0000 7f4dce59c640 1 --2- 172.21.15.92:0/121506353 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4dbc0051d0 0x7f4dd014dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:12:49.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.456+0000 7f4dc7fff640 1 --2- 172.21.15.92:0/121506353 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4dd013d940 0x7f4dd014b1f0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f4db800f4e0 tx=0x7f4db800f9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:49.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.456+0000 7f4dc6ffd640 1 -- 172.21.15.92:0/121506353 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4db8014870 con 0x7f4dd013d940 2023-11-24T06:12:49.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.456+0000 7f4dc6ffd640 1 -- 172.21.15.92:0/121506353 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4db80149d0 con 0x7f4dd013d940 2023-11-24T06:12:49.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.456+0000 7f4dc6ffd640 1 -- 172.21.15.92:0/121506353 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4db8014d40 con 0x7f4dd013d940 2023-11-24T06:12:49.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.456+0000 7f4dd4fec640 1 -- 172.21.15.92:0/121506353 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4dd0070990 con 0x7f4dd013d940 2023-11-24T06:12:49.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.456+0000 7f4dd4fec640 1 -- 172.21.15.92:0/121506353 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4dd014d810 con 0x7f4dd013d940 2023-11-24T06:12:49.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.458+0000 7f4dd4fec640 1 -- 172.21.15.92:0/121506353 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4dd00675d0 con 0x7f4dd013d940 2023-11-24T06:12:49.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.459+0000 7f4dc6ffd640 1 -- 172.21.15.92:0/121506353 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f4db8015410 con 0x7f4dd013d940 2023-11-24T06:12:49.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.459+0000 7f4dc6ffd640 1 --2- 172.21.15.92:0/121506353 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f4db006aed0 0x7f4db006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:49.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.460+0000 7f4dc6ffd640 1 -- 172.21.15.92:0/121506353 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4db8089fe0 con 0x7f4dd013d940 2023-11-24T06:12:49.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.460+0000 7f4dce59c640 1 --2- 172.21.15.92:0/121506353 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f4db006aed0 0x7f4db006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:49.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.460+0000 7f4dce59c640 1 --2- 172.21.15.92:0/121506353 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f4db006aed0 0x7f4db006d390 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f4dd015c780 tx=0x7f4dc0000f90 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:49.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.464+0000 7f4dc6ffd640 1 -- 172.21.15.92:0/121506353 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4db80bce90 con 0x7f4dd013d940 2023-11-24T06:12:49.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.655+0000 7f4dd4fec640 1 -- 172.21.15.92:0/121506353 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f4dd014a5a0 con 0x7f4dd013d940 2023-11-24T06:12:49.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.656+0000 7f4dc6ffd640 1 -- 172.21.15.92:0/121506353 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v52) v1 ==== 104+0+11 (secure 0 0 0) 0x7f4db8058ae0 con 0x7f4dd013d940 2023-11-24T06:12:49.659 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 32 2023-11-24T06:12:49.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.659+0000 7f4dd4fec640 1 -- 172.21.15.92:0/121506353 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f4db006aed0 msgr2=0x7f4db006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:49.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.659+0000 7f4dd4fec640 1 --2- 172.21.15.92:0/121506353 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f4db006aed0 0x7f4db006d390 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f4dd015c780 tx=0x7f4dc0000f90 comp rx=0 tx=0).stop 2023-11-24T06:12:49.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.659+0000 7f4dd4fec640 1 -- 172.21.15.92:0/121506353 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4dd013d940 msgr2=0x7f4dd014b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:49.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.659+0000 7f4dd4fec640 1 --2- 172.21.15.92:0/121506353 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4dd013d940 0x7f4dd014b1f0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f4db800f4e0 tx=0x7f4db800f9a0 comp rx=0 tx=0).stop 2023-11-24T06:12:49.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.659+0000 7f4dd4fec640 1 -- 172.21.15.92:0/121506353 shutdown_connections 2023-11-24T06:12:49.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.659+0000 7f4dd4fec640 1 --2- 172.21.15.92:0/121506353 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f4db006aed0 0x7f4db006d390 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:49.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.659+0000 7f4dd4fec640 1 --2- 172.21.15.92:0/121506353 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4dd014b730 0x7f4dd0149a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:49.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.659+0000 7f4dd4fec640 1 --2- 172.21.15.92:0/121506353 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4dd013d940 0x7f4dd014b1f0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:49.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.659+0000 7f4dd4fec640 1 --2- 172.21.15.92:0/121506353 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4dbc0051d0 0x7f4dd014dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:49.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.659+0000 7f4dd4fec640 1 -- 172.21.15.92:0/121506353 >> 172.21.15.92:0/121506353 conn(0x7f4dd006a800 msgr2=0x7f4dd015c850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:49.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.659+0000 7f4dd4fec640 1 -- 172.21.15.92:0/121506353 shutdown_connections 2023-11-24T06:12:49.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.659+0000 7f4dd4fec640 1 -- 172.21.15.92:0/121506353 wait complete. 2023-11-24T06:12:49.681 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T06:12:49.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.898+0000 7fb4e8148640 1 Processor -- start 2023-11-24T06:12:49.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.898+0000 7fb4e8148640 1 -- start start 2023-11-24T06:12:49.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.900+0000 7fb4e8148640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fb4e00cf9a0 0x7fb4e015dca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:49.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.900+0000 7fb4e8148640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4e00b1ee0 con 0x7fb4e015e1e0 2023-11-24T06:12:49.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.900+0000 7fb4e8148640 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4e00b2780 con 0x7fb4e0143ae0 2023-11-24T06:12:49.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.900+0000 7fb4e8148640 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4e015e990 con 0x7fb4e00cf9a0 2023-11-24T06:12:49.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.900+0000 7fb4e66e5640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fb4e015e1e0 0x7fb4e00702c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:53282/0 (socket says 172.21.15.92:53282) 2023-11-24T06:12:49.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.900+0000 7fb4e66e5640 1 -- 172.21.15.92:0/2960807783 learned_addr learned my addr 172.21.15.92:0/2960807783 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-24T06:12:49.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.901+0000 7fb4e56e3640 1 --2- 172.21.15.92:0/2960807783 >> v2:172.21.15.118:3300/0 conn(0x7fb4e00cf9a0 0x7fb4e015dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:49.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.901+0000 7fb4e4ee2640 1 -- 172.21.15.92:0/2960807783 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3593676362 0 0) 0x7fb4e00b1ee0 con 0x7fb4e015e1e0 2023-11-24T06:12:49.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.901+0000 7fb4e4ee2640 1 -- 172.21.15.92:0/2960807783 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb4cc003520 con 0x7fb4e015e1e0 2023-11-24T06:12:49.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.901+0000 7fb4e4ee2640 1 -- 172.21.15.92:0/2960807783 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 351200515 0 0) 0x7fb4cc003520 con 0x7fb4e015e1e0 2023-11-24T06:12:49.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.901+0000 7fb4e4ee2640 1 -- 172.21.15.92:0/2960807783 >> v2:172.21.15.118:3300/0 conn(0x7fb4e00cf9a0 msgr2=0x7fb4e015dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:49.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.901+0000 7fb4e4ee2640 1 --2- 172.21.15.92:0/2960807783 >> v2:172.21.15.118:3300/0 conn(0x7fb4e00cf9a0 0x7fb4e015dca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:49.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.901+0000 7fb4e4ee2640 1 -- 172.21.15.92:0/2960807783 >> v1:172.21.15.118:6789/0 conn(0x7fb4e0143ae0 legacy=0x7fb4e00cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:49.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.901+0000 7fb4e4ee2640 1 -- 172.21.15.92:0/2960807783 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4e00b2130 con 0x7fb4e015e1e0 2023-11-24T06:12:49.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.901+0000 7fb4e4ee2640 1 -- 172.21.15.92:0/2960807783 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2607661350 0 0) 0x7fb4d80019f0 con 0x7fb4e015e1e0 2023-11-24T06:12:49.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.901+0000 7fb4e4ee2640 1 -- 172.21.15.92:0/2960807783 >> v1:172.21.15.92:6789/0 conn(0x7fb4e015e1e0 legacy=0x7fb4e00702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:49.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.901+0000 7fb4e4ee2640 1 --2- 172.21.15.92:0/2960807783 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb4cc0039c0 0x7fb4cc003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:49.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.901+0000 7fb4e4ee2640 1 --2- 172.21.15.92:0/2960807783 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb4cc0047c0 0x7fb4cc004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:49.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.902+0000 7fb4e4ee2640 1 --2- 172.21.15.92:0/2960807783 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb4cc005180 0x7fb4cc00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:49.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.902+0000 7fb4e4ee2640 1 -- 172.21.15.92:0/2960807783 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4e015e990 con 0x7fb4cc005180 2023-11-24T06:12:49.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.902+0000 7fb4e4ee2640 1 -- 172.21.15.92:0/2960807783 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4cc00c080 con 0x7fb4cc0039c0 2023-11-24T06:12:49.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.902+0000 7fb4e4ee2640 1 -- 172.21.15.92:0/2960807783 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4cc00c240 con 0x7fb4cc0047c0 2023-11-24T06:12:49.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.902+0000 7fb4e66e5640 1 --2- 172.21.15.92:0/2960807783 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb4cc005180 0x7fb4cc00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:49.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.902+0000 7fb4e5ee4640 1 --2- 172.21.15.92:0/2960807783 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb4cc0039c0 0x7fb4cc003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:49.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.902+0000 7fb4e56e3640 1 --2- 172.21.15.92:0/2960807783 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb4cc0047c0 0x7fb4cc004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:49.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.902+0000 7fb4e66e5640 1 -- 172.21.15.92:0/2960807783 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb4cc0047c0 msgr2=0x7fb4cc004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:49.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.902+0000 7fb4e66e5640 1 --2- 172.21.15.92:0/2960807783 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb4cc0047c0 0x7fb4cc004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:49.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.902+0000 7fb4e66e5640 1 -- 172.21.15.92:0/2960807783 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb4cc0039c0 msgr2=0x7fb4cc003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:49.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.902+0000 7fb4e66e5640 1 --2- 172.21.15.92:0/2960807783 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb4cc0039c0 0x7fb4cc003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:49.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.902+0000 7fb4e66e5640 1 -- 172.21.15.92:0/2960807783 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4cc003520 con 0x7fb4cc005180 2023-11-24T06:12:49.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.903+0000 7fb4e66e5640 1 -- 172.21.15.92:0/2960807783 reap_dead start 2023-11-24T06:12:49.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.903+0000 7fb4e66e5640 1 --2- 172.21.15.92:0/2960807783 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb4cc005180 0x7fb4cc00b980 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fb4d800a760 tx=0x7fb4cc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e163cce338b51bf2 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:49.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.904+0000 7fb4e4ee2640 1 -- 172.21.15.92:0/2960807783 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4d8034630 con 0x7fb4cc005180 2023-11-24T06:12:49.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.904+0000 7fb4e4ee2640 1 -- 172.21.15.92:0/2960807783 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4d802f070 con 0x7fb4cc005180 2023-11-24T06:12:49.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.904+0000 7fb4e4ee2640 1 -- 172.21.15.92:0/2960807783 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4d8034a10 con 0x7fb4cc005180 2023-11-24T06:12:49.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.904+0000 7fb4e8148640 1 -- 172.21.15.92:0/2960807783 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb4cc005180 msgr2=0x7fb4cc00b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:49.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.904+0000 7fb4e8148640 1 --2- 172.21.15.92:0/2960807783 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb4cc005180 0x7fb4cc00b980 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fb4d800a760 tx=0x7fb4cc0057f0 comp rx=0 tx=0).stop 2023-11-24T06:12:49.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.904+0000 7fb4e8148640 1 -- 172.21.15.92:0/2960807783 shutdown_connections 2023-11-24T06:12:49.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.904+0000 7fb4e8148640 1 --2- 172.21.15.92:0/2960807783 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb4cc005180 0x7fb4cc00b980 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:49.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.904+0000 7fb4e8148640 1 -- 172.21.15.92:0/2960807783 >> 172.21.15.92:0/2960807783 conn(0x7fb4e006a800 msgr2=0x7fb4e006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:49.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.904+0000 7fb4e8148640 1 -- 172.21.15.92:0/2960807783 shutdown_connections 2023-11-24T06:12:49.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.904+0000 7fb4e8148640 1 -- 172.21.15.92:0/2960807783 wait complete. 2023-11-24T06:12:49.910 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.906+0000 7fb4e8148640 1 Processor -- start 2023-11-24T06:12:49.911 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.906+0000 7fb4e8148640 1 -- start start 2023-11-24T06:12:49.911 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.907+0000 7fb4e8148640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb4cc005180 0x7fb4e01521a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:49.911 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.907+0000 7fb4e8148640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb4e0141750 0x7fb4e014c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:49.911 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.907+0000 7fb4e8148640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb4e00af7e0 0x7fb4e00afc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:49.911 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.907+0000 7fb4e8148640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4e015ac00 con 0x7fb4e00af7e0 2023-11-24T06:12:49.911 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.907+0000 7fb4e8148640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4e00b01d0 con 0x7fb4cc005180 2023-11-24T06:12:49.911 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.907+0000 7fb4e8148640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4e013e510 con 0x7fb4e0141750 2023-11-24T06:12:49.911 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.908+0000 7fb4e66e5640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb4e00af7e0 0x7fb4e00afc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:49.911 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.908+0000 7fb4e66e5640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb4e00af7e0 0x7fb4e00afc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:55740/0 (socket says 172.21.15.92:55740) 2023-11-24T06:12:49.911 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.908+0000 7fb4e66e5640 1 -- 172.21.15.92:0/2786671617 learned_addr learned my addr 172.21.15.92:0/2786671617 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:12:49.912 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.908+0000 7fb4e56e3640 1 --2- 172.21.15.92:0/2786671617 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb4e0141750 0x7fb4e014c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:49.912 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.908+0000 7fb4e5ee4640 1 --2- 172.21.15.92:0/2786671617 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb4cc005180 0x7fb4e01521a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:49.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.908+0000 7fb4e66e5640 1 -- 172.21.15.92:0/2786671617 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb4e0141750 msgr2=0x7fb4e014c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:49.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.908+0000 7fb4e66e5640 1 --2- 172.21.15.92:0/2786671617 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb4e0141750 0x7fb4e014c440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:49.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.908+0000 7fb4e66e5640 1 -- 172.21.15.92:0/2786671617 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb4cc005180 msgr2=0x7fb4e01521a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:49.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.908+0000 7fb4e66e5640 1 --2- 172.21.15.92:0/2786671617 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb4cc005180 0x7fb4e01521a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:49.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.908+0000 7fb4e66e5640 1 -- 172.21.15.92:0/2786671617 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4e013cb10 con 0x7fb4e00af7e0 2023-11-24T06:12:49.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.908+0000 7fb4e66e5640 1 --2- 172.21.15.92:0/2786671617 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb4e00af7e0 0x7fb4e00afc90 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fb4d8002230 tx=0x7fb4d8001cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:49.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.909+0000 7fb4d67fc640 1 -- 172.21.15.92:0/2786671617 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4d8001930 con 0x7fb4e00af7e0 2023-11-24T06:12:49.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.909+0000 7fb4d67fc640 1 -- 172.21.15.92:0/2786671617 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4d8004b60 con 0x7fb4e00af7e0 2023-11-24T06:12:49.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.909+0000 7fb4d67fc640 1 -- 172.21.15.92:0/2786671617 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4d80347b0 con 0x7fb4e00af7e0 2023-11-24T06:12:49.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.909+0000 7fb4e8148640 1 -- 172.21.15.92:0/2786671617 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4e013e7b0 con 0x7fb4e00af7e0 2023-11-24T06:12:49.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.909+0000 7fb4e8148640 1 -- 172.21.15.92:0/2786671617 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4e014c980 con 0x7fb4e00af7e0 2023-11-24T06:12:49.914 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.910+0000 7fb4e8148640 1 -- 172.21.15.92:0/2786671617 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4e013e370 con 0x7fb4e00af7e0 2023-11-24T06:12:49.914 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.911+0000 7fb4d67fc640 1 -- 172.21.15.92:0/2786671617 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7fb4d8031030 con 0x7fb4e00af7e0 2023-11-24T06:12:49.915 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.912+0000 7fb4d67fc640 1 --2- 172.21.15.92:0/2786671617 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fb4c406aec0 0x7fb4c406d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:49.915 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.912+0000 7fb4d67fc640 1 -- 172.21.15.92:0/2786671617 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb4d80aab90 con 0x7fb4e00af7e0 2023-11-24T06:12:49.915 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.912+0000 7fb4e5ee4640 1 --2- 172.21.15.92:0/2786671617 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fb4c406aec0 0x7fb4c406d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:49.916 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.913+0000 7fb4e5ee4640 1 --2- 172.21.15.92:0/2786671617 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fb4c406aec0 0x7fb4c406d380 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fb4dc0014d0 tx=0x7fb4dc009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:49.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:49.916+0000 7fb4d67fc640 1 -- 172.21.15.92:0/2786671617 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb4e013e370 con 0x7fb4e00af7e0 2023-11-24T06:12:50.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:50.105+0000 7fb4e8148640 1 -- 172.21.15.92:0/2786671617 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb4e0073c30 con 0x7fb4e00af7e0 2023-11-24T06:12:50.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:50.106+0000 7fb4d67fc640 1 -- 172.21.15.92:0/2786671617 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v52) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb4e0073c30 con 0x7fb4e00af7e0 2023-11-24T06:12:50.110 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 64 2023-11-24T06:12:50.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:50.109+0000 7fb4e8148640 1 -- 172.21.15.92:0/2786671617 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fb4c406aec0 msgr2=0x7fb4c406d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:50.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:50.109+0000 7fb4e8148640 1 --2- 172.21.15.92:0/2786671617 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fb4c406aec0 0x7fb4c406d380 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fb4dc0014d0 tx=0x7fb4dc009040 comp rx=0 tx=0).stop 2023-11-24T06:12:50.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:50.109+0000 7fb4e8148640 1 -- 172.21.15.92:0/2786671617 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb4e00af7e0 msgr2=0x7fb4e00afc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:50.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:50.109+0000 7fb4e8148640 1 --2- 172.21.15.92:0/2786671617 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb4e00af7e0 0x7fb4e00afc90 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fb4d8002230 tx=0x7fb4d8001cb0 comp rx=0 tx=0).stop 2023-11-24T06:12:50.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:50.109+0000 7fb4e8148640 1 -- 172.21.15.92:0/2786671617 shutdown_connections 2023-11-24T06:12:50.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:50.109+0000 7fb4e8148640 1 --2- 172.21.15.92:0/2786671617 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fb4c406aec0 0x7fb4c406d380 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:50.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:50.109+0000 7fb4e8148640 1 --2- 172.21.15.92:0/2786671617 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb4e00af7e0 0x7fb4e00afc90 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:50.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:50.109+0000 7fb4e8148640 1 --2- 172.21.15.92:0/2786671617 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb4e0141750 0x7fb4e014c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:50.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:50.109+0000 7fb4e8148640 1 --2- 172.21.15.92:0/2786671617 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb4cc005180 0x7fb4e01521a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:50.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:50.109+0000 7fb4e8148640 1 -- 172.21.15.92:0/2786671617 >> 172.21.15.92:0/2786671617 conn(0x7fb4e006a800 msgr2=0x7fb4e00cf260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:50.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:50.110+0000 7fb4e8148640 1 -- 172.21.15.92:0/2786671617 shutdown_connections 2023-11-24T06:12:50.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:50.110+0000 7fb4e8148640 1 -- 172.21.15.92:0/2786671617 wait complete. 2023-11-24T06:12:50.133 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-24T06:12:50.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:50.343+0000 7f923309f640 1 Processor -- start 2023-11-24T06:12:50.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:50.343+0000 7f923309f640 1 -- start start 2023-11-24T06:12:50.348 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:50.345+0000 7f923309f640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f922c15afb0 0x7f922c13f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:50.348 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:50.345+0000 7f923309f640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f922c13fae0 0x7f922c0b2040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:50.348 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:50.345+0000 7f923309f640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f922c13ff30 con 0x7f922c13cfa0 2023-11-24T06:12:50.348 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:50.345+0000 7f923309f640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f922c0cdc40 con 0x7f922c15afb0 2023-11-24T06:12:50.348 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:50.345+0000 7f923309f640 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f922c0cdec0 con 0x7f922c13fae0 2023-11-24T06:12:50.348 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:50.345+0000 7f923189c640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f922c15afb0 0x7f922c13f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:50.348 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:50.345+0000 7f923209d640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f922c13cfa0 0x7f922c0cd530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:53290/0 (socket says 172.21.15.92:53290) 2023-11-24T06:12:50.348 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:50.345+0000 7f923189c640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f922c15afb0 0x7f922c13f5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:55752/0 (socket says 172.21.15.92:55752) 2023-11-24T06:12:50.349 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:50.345+0000 7f923209d640 1 -- 172.21.15.92:0/1979734546 learned_addr learned my addr 172.21.15.92:0/1979734546 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-24T06:12:50.349 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:50.345+0000 7f923289e640 1 --2- 172.21.15.92:0/1979734546 >> v2:172.21.15.118:3300/0 conn(0x7f922c13fae0 0x7f922c0b2040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:50.349 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:50.345+0000 7f923189c640 1 -- 172.21.15.92:0/1979734546 >> v2:172.21.15.118:3300/0 conn(0x7f922c13fae0 msgr2=0x7f922c0b2040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:50.349 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:50.345+0000 7f923189c640 1 --2- 172.21.15.92:0/1979734546 >> v2:172.21.15.118:3300/0 conn(0x7f922c13fae0 0x7f922c0b2040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:50.349 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:50.346+0000 7f923189c640 1 -- 172.21.15.92:0/1979734546 >> v1:172.21.15.92:6789/0 conn(0x7f922c13cfa0 legacy=0x7f922c0cd530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:50.349 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:50.346+0000 7f923189c640 1 -- 172.21.15.92:0/1979734546 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f922c0ce1a0 con 0x7f922c15afb0 2023-11-24T06:12:50.351 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:50.347+0000 7f923189c640 1 --2- 172.21.15.92:0/1979734546 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f922c15afb0 0x7f922c13f5a0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f922c154a90 tx=0x7f921c02da50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96a3928ebf49f066 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:50.351 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:50.347+0000 7f923109b640 1 -- 172.21.15.92:0/1979734546 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f921c031870 con 0x7f922c15afb0 2023-11-24T06:12:50.351 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:50.347+0000 7f923109b640 1 -- 172.21.15.92:0/1979734546 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f921c0319d0 con 0x7f922c15afb0 2023-11-24T06:12:50.351 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:50.347+0000 7f923109b640 1 -- 172.21.15.92:0/1979734546 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f921c031d60 con 0x7f922c15afb0 2023-11-24T06:12:50.351 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:50.348+0000 7f923309f640 1 -- 172.21.15.92:0/1979734546 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f922c15afb0 msgr2=0x7f922c13f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:50.351 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:50.348+0000 7f923309f640 1 --2- 172.21.15.92:0/1979734546 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f922c15afb0 0x7f922c13f5a0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f922c154a90 tx=0x7f921c02da50 comp rx=0 tx=0).stop 2023-11-24T06:12:50.351 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:50.348+0000 7f923309f640 1 -- 172.21.15.92:0/1979734546 shutdown_connections 2023-11-24T06:12:50.351 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:50.348+0000 7f923309f640 1 --2- 172.21.15.92:0/1979734546 >> v2:172.21.15.118:3300/0 conn(0x7f922c13fae0 0x7f922c0b2040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:50.351 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:50.348+0000 7f923309f640 1 --2- 172.21.15.92:0/1979734546 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f922c15afb0 0x7f922c13f5a0 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:50.351 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:50.348+0000 7f923309f640 1 -- 172.21.15.92:0/1979734546 >> 172.21.15.92:0/1979734546 conn(0x7f922c06a800 msgr2=0x7f922c06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:50.352 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:50.348+0000 7f923309f640 1 -- 172.21.15.92:0/1979734546 shutdown_connections 2023-11-24T06:12:50.352 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:50.348+0000 7f923309f640 1 -- 172.21.15.92:0/1979734546 wait complete. 2023-11-24T06:12:50.352 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:50.350+0000 7f923309f640 1 Processor -- start 2023-11-24T06:12:50.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:50.350+0000 7f923309f640 1 -- start start 2023-11-24T06:12:50.354 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:50.351+0000 7f923309f640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f922c13cfa0 0x7f922c1536a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:50.355 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:50.351+0000 7f923309f640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f922c13fae0 0x7f922c156660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:50.355 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:50.351+0000 7f923309f640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f922c15afb0 0x7f922c156ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:50.355 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:50.351+0000 7f923309f640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f922c0c94e0 con 0x7f922c13cfa0 2023-11-24T06:12:50.355 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:50.351+0000 7f923309f640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f922c1570e0 con 0x7f922c13fae0 2023-11-24T06:12:50.355 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:50.351+0000 7f923309f640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f922c0c88d0 con 0x7f922c15afb0 2023-11-24T06:12:50.355 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:50.351+0000 7f923209d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f922c13cfa0 0x7f922c1536a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:50.355 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:50.351+0000 7f923209d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f922c13cfa0 0x7f922c1536a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:55762/0 (socket says 172.21.15.92:55762) 2023-11-24T06:12:50.355 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:50.351+0000 7f923209d640 1 -- 172.21.15.92:0/433213116 learned_addr learned my addr 172.21.15.92:0/433213116 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:12:50.355 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:50.351+0000 7f923189c640 1 --2- 172.21.15.92:0/433213116 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f922c13fae0 0x7f922c156660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:50.355 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:50.351+0000 7f923289e640 1 --2- 172.21.15.92:0/433213116 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f922c15afb0 0x7f922c156ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:50.355 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:50.351+0000 7f923209d640 1 -- 172.21.15.92:0/433213116 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f922c15afb0 msgr2=0x7f922c156ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:50.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:50.351+0000 7f923209d640 1 --2- 172.21.15.92:0/433213116 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f922c15afb0 0x7f922c156ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:50.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:50.351+0000 7f923209d640 1 -- 172.21.15.92:0/433213116 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f922c13fae0 msgr2=0x7f922c156660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:50.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:50.351+0000 7f923209d640 1 --2- 172.21.15.92:0/433213116 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f922c13fae0 0x7f922c156660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:50.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:50.351+0000 7f923209d640 1 -- 172.21.15.92:0/433213116 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f921c009400 con 0x7f922c13cfa0 2023-11-24T06:12:50.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:50.352+0000 7f923209d640 1 --2- 172.21.15.92:0/433213116 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f922c13cfa0 0x7f922c1536a0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f922800c4a0 tx=0x7f922800c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:50.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:50.352+0000 7f9222ffd640 1 -- 172.21.15.92:0/433213116 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9228005ce0 con 0x7f922c13cfa0 2023-11-24T06:12:50.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:50.352+0000 7f9222ffd640 1 -- 172.21.15.92:0/433213116 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9228005e40 con 0x7f922c13cfa0 2023-11-24T06:12:50.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:50.352+0000 7f923309f640 1 -- 172.21.15.92:0/433213116 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f922c0c8bb0 con 0x7f922c13cfa0 2023-11-24T06:12:50.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:50.352+0000 7f9222ffd640 1 -- 172.21.15.92:0/433213116 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9228007270 con 0x7f922c13cfa0 2023-11-24T06:12:50.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:50.352+0000 7f923309f640 1 -- 172.21.15.92:0/433213116 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f922c0c9170 con 0x7f922c13cfa0 2023-11-24T06:12:50.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:50.353+0000 7f923309f640 1 -- 172.21.15.92:0/433213116 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f922c067450 con 0x7f922c13cfa0 2023-11-24T06:12:50.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:50.354+0000 7f9222ffd640 1 -- 172.21.15.92:0/433213116 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f92280073d0 con 0x7f922c13cfa0 2023-11-24T06:12:50.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:50.355+0000 7f9222ffd640 1 --2- 172.21.15.92:0/433213116 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f921006af00 0x7f921006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:50.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:50.355+0000 7f9222ffd640 1 -- 172.21.15.92:0/433213116 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f92280889e0 con 0x7f922c13cfa0 2023-11-24T06:12:50.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:50.355+0000 7f923189c640 1 --2- 172.21.15.92:0/433213116 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f921006af00 0x7f921006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:50.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:50.356+0000 7f923189c640 1 --2- 172.21.15.92:0/433213116 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f921006af00 0x7f921006d3c0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f921c038000 tx=0x7f921c03b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:50.362 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:50.360+0000 7f9222ffd640 1 -- 172.21.15.92:0/433213116 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9228053a50 con 0x7f922c13cfa0 2023-11-24T06:12:50.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:50.562+0000 7f923309f640 1 -- 172.21.15.92:0/433213116 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f922c13ff30 con 0x7f922c13cfa0 2023-11-24T06:12:50.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:50.563+0000 7f9222ffd640 1 -- 172.21.15.92:0/433213116 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v52) v1 ==== 80+0+0 (secure 0 0 0) 0x7f922c13ff30 con 0x7f922c13cfa0 2023-11-24T06:12:50.566 INFO:teuthology.orchestra.run.smithi092.stderr:listed 0 entries 2023-11-24T06:12:50.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:50.566+0000 7f923309f640 1 -- 172.21.15.92:0/433213116 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f921006af00 msgr2=0x7f921006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:50.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:50.566+0000 7f923309f640 1 --2- 172.21.15.92:0/433213116 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f921006af00 0x7f921006d3c0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f921c038000 tx=0x7f921c03b040 comp rx=0 tx=0).stop 2023-11-24T06:12:50.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:50.566+0000 7f923309f640 1 -- 172.21.15.92:0/433213116 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f922c13cfa0 msgr2=0x7f922c1536a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:50.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:50.566+0000 7f923309f640 1 --2- 172.21.15.92:0/433213116 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f922c13cfa0 0x7f922c1536a0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f922800c4a0 tx=0x7f922800c960 comp rx=0 tx=0).stop 2023-11-24T06:12:50.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:50.566+0000 7f923309f640 1 -- 172.21.15.92:0/433213116 shutdown_connections 2023-11-24T06:12:50.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:50.566+0000 7f923309f640 1 --2- 172.21.15.92:0/433213116 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f921006af00 0x7f921006d3c0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:50.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:50.566+0000 7f923309f640 1 --2- 172.21.15.92:0/433213116 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f922c15afb0 0x7f922c156ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:50.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:50.566+0000 7f923309f640 1 --2- 172.21.15.92:0/433213116 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f922c13fae0 0x7f922c156660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:50.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:50.566+0000 7f923309f640 1 --2- 172.21.15.92:0/433213116 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f922c13cfa0 0x7f922c1536a0 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:50.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:50.566+0000 7f923309f640 1 -- 172.21.15.92:0/433213116 >> 172.21.15.92:0/433213116 conn(0x7f922c06a800 msgr2=0x7f922c0b1230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:50.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:50.566+0000 7f923309f640 1 -- 172.21.15.92:0/433213116 shutdown_connections 2023-11-24T06:12:50.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:50.566+0000 7f923309f640 1 -- 172.21.15.92:0/433213116 wait complete. 2023-11-24T06:12:50.588 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi118.front.sepia.ceph.com... 2023-11-24T06:12:50.588 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:12:50.588 DEBUG:teuthology.orchestra.run.smithi118:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2023-11-24T06:12:50.688 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi118.stderr:ceph-fuse[94964]: fuse finished with error 0 and tester_r 0 2023-11-24T06:12:50.695 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi118.front.sepia.ceph.com 2023-11-24T06:12:50.695 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:12:50.695 DEBUG:teuthology.orchestra.run.smithi118:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-11-24T06:12:50.872 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-11-24T06:12:50.872 DEBUG:teuthology.orchestra.run.smithi092:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:12:50.932 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054298,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498824k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249408k,nr_inodes=812352,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498824k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-24T06:12:50.932 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi092.front.sepia.ceph.com 2023-11-24T06:12:50.932 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:12:50.932 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-11-24T06:12:50.987 INFO:teuthology.orchestra.run.smithi092.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-11-24T06:12:50.988 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-24T06:12:50.988 DEBUG:teuthology.orchestra.run.smithi118:> set -ex 2023-11-24T06:12:50.989 DEBUG:teuthology.orchestra.run.smithi118:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:12:51.048 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054284,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249396k,nr_inodes=812349,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-24T06:12:51.048 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi118.front.sepia.ceph.com 2023-11-24T06:12:51.048 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:12:51.048 DEBUG:teuthology.orchestra.run.smithi118:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-11-24T06:12:51.099 INFO:teuthology.orchestra.run.smithi118.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-11-24T06:12:51.100 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-24T06:12:51.101 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T06:12:51.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:51.319+0000 7f97f8083640 1 Processor -- start 2023-11-24T06:12:51.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:51.319+0000 7f97f8083640 1 -- start start 2023-11-24T06:12:51.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:51.321+0000 7f97f8083640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f97f00cf9a0 0x7f97f015dca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:51.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:51.321+0000 7f97f8083640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97f00b1ee0 con 0x7f97f015e1e0 2023-11-24T06:12:51.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:51.321+0000 7f97f8083640 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97f00b2780 con 0x7f97f0143ae0 2023-11-24T06:12:51.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:51.321+0000 7f97f8083640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97f015e990 con 0x7f97f00cf9a0 2023-11-24T06:12:51.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:51.321+0000 7f97f561e640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f97f00cf9a0 0x7f97f015dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:51.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:51.321+0000 7f97f561e640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f97f00cf9a0 0x7f97f015dca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:55774/0 (socket says 172.21.15.92:55774) 2023-11-24T06:12:51.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:51.321+0000 7f97f6620640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f97f015e1e0 0x7f97f00702c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:53298/0 (socket says 172.21.15.92:53298) 2023-11-24T06:12:51.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:51.321+0000 7f97f561e640 1 -- 172.21.15.92:0/1213350703 learned_addr learned my addr 172.21.15.92:0/1213350703 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:12:51.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:51.322+0000 7f97f561e640 1 -- 172.21.15.92:0/1213350703 >> v1:172.21.15.118:6789/0 conn(0x7f97f0143ae0 legacy=0x7f97f00cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:51.326 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:51.322+0000 7f97f561e640 1 -- 172.21.15.92:0/1213350703 >> v1:172.21.15.92:6789/0 conn(0x7f97f015e1e0 legacy=0x7f97f00702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:51.326 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:51.322+0000 7f97f561e640 1 -- 172.21.15.92:0/1213350703 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97f00b2130 con 0x7f97f00cf9a0 2023-11-24T06:12:51.326 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:51.323+0000 7f97f561e640 1 --2- 172.21.15.92:0/1213350703 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f97f00cf9a0 0x7f97f015dca0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f97e0009870 tx=0x7f97e002b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bbe79f51d784ffd server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:51.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:51.323+0000 7f97f4e1d640 1 -- 172.21.15.92:0/1213350703 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97e0030860 con 0x7f97f00cf9a0 2023-11-24T06:12:51.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:51.323+0000 7f97f4e1d640 1 -- 172.21.15.92:0/1213350703 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97e00309c0 con 0x7f97f00cf9a0 2023-11-24T06:12:51.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:51.323+0000 7f97f4e1d640 1 -- 172.21.15.92:0/1213350703 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97e0030d50 con 0x7f97f00cf9a0 2023-11-24T06:12:51.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:51.324+0000 7f97f8083640 1 -- 172.21.15.92:0/1213350703 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f97f00cf9a0 msgr2=0x7f97f015dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:51.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:51.324+0000 7f97f8083640 1 --2- 172.21.15.92:0/1213350703 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f97f00cf9a0 0x7f97f015dca0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f97e0009870 tx=0x7f97e002b9a0 comp rx=0 tx=0).stop 2023-11-24T06:12:51.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:51.324+0000 7f97f8083640 1 -- 172.21.15.92:0/1213350703 shutdown_connections 2023-11-24T06:12:51.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:51.324+0000 7f97f8083640 1 --2- 172.21.15.92:0/1213350703 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f97f00cf9a0 0x7f97f015dca0 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:51.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:51.324+0000 7f97f8083640 1 -- 172.21.15.92:0/1213350703 >> 172.21.15.92:0/1213350703 conn(0x7f97f006a800 msgr2=0x7f97f006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:51.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:51.324+0000 7f97f8083640 1 -- 172.21.15.92:0/1213350703 shutdown_connections 2023-11-24T06:12:51.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:51.324+0000 7f97f8083640 1 -- 172.21.15.92:0/1213350703 wait complete. 2023-11-24T06:12:51.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:51.325+0000 7f97f8083640 1 Processor -- start 2023-11-24T06:12:51.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:51.326+0000 7f97f8083640 1 -- start start 2023-11-24T06:12:51.331 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:51.326+0000 7f97f8083640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f97f00cf9a0 0x7f97f00c3e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:51.331 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:51.326+0000 7f97f8083640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f97f0143ae0 0x7f97f00c4390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:51.331 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:51.327+0000 7f97f8083640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f97f015e1e0 0x7f97f01f6030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:51.331 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:51.327+0000 7f97f8083640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97f00ae780 con 0x7f97f0143ae0 2023-11-24T06:12:51.331 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:51.327+0000 7f97f8083640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97f01f2990 con 0x7f97f00cf9a0 2023-11-24T06:12:51.331 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:51.327+0000 7f97f8083640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97f00adee0 con 0x7f97f015e1e0 2023-11-24T06:12:51.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:51.327+0000 7f97f561e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f97f0143ae0 0x7f97f00c4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:51.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:51.327+0000 7f97f561e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f97f0143ae0 0x7f97f00c4390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:55784/0 (socket says 172.21.15.92:55784) 2023-11-24T06:12:51.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:51.327+0000 7f97f561e640 1 -- 172.21.15.92:0/113850743 learned_addr learned my addr 172.21.15.92:0/113850743 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:12:51.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:51.327+0000 7f97f5e1f640 1 --2- 172.21.15.92:0/113850743 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f97f00cf9a0 0x7f97f00c3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:51.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:51.327+0000 7f97f6620640 1 --2- 172.21.15.92:0/113850743 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f97f015e1e0 0x7f97f01f6030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:51.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:51.327+0000 7f97f561e640 1 -- 172.21.15.92:0/113850743 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f97f015e1e0 msgr2=0x7f97f01f6030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:51.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:51.327+0000 7f97f561e640 1 --2- 172.21.15.92:0/113850743 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f97f015e1e0 0x7f97f01f6030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:51.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:51.327+0000 7f97f561e640 1 -- 172.21.15.92:0/113850743 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f97f00cf9a0 msgr2=0x7f97f00c3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:51.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:51.327+0000 7f97f561e640 1 --2- 172.21.15.92:0/113850743 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f97f00cf9a0 0x7f97f00c3e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:51.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:51.327+0000 7f97f561e640 1 -- 172.21.15.92:0/113850743 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97e00094d0 con 0x7f97f0143ae0 2023-11-24T06:12:51.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:51.327+0000 7f97f5e1f640 1 --2- 172.21.15.92:0/113850743 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f97f00cf9a0 0x7f97f00c3e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:12:51.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:51.327+0000 7f97f6620640 1 --2- 172.21.15.92:0/113850743 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f97f015e1e0 0x7f97f01f6030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:12:51.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:51.328+0000 7f97f561e640 1 --2- 172.21.15.92:0/113850743 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f97f0143ae0 0x7f97f00c4390 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f97e0000c00 tx=0x7f97e0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:51.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:51.328+0000 7f97e67fc640 1 -- 172.21.15.92:0/113850743 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97e0030830 con 0x7f97f0143ae0 2023-11-24T06:12:51.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:51.328+0000 7f97e67fc640 1 -- 172.21.15.92:0/113850743 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97e0043400 con 0x7f97f0143ae0 2023-11-24T06:12:51.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:51.328+0000 7f97e67fc640 1 -- 172.21.15.92:0/113850743 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97e00437d0 con 0x7f97f0143ae0 2023-11-24T06:12:51.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:51.328+0000 7f97f8083640 1 -- 172.21.15.92:0/113850743 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97f00ae1c0 con 0x7f97f0143ae0 2023-11-24T06:12:51.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:51.328+0000 7f97f8083640 1 -- 172.21.15.92:0/113850743 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97f00cc880 con 0x7f97f0143ae0 2023-11-24T06:12:51.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:51.329+0000 7f97f8083640 1 -- 172.21.15.92:0/113850743 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97f00cc6e0 con 0x7f97f0143ae0 2023-11-24T06:12:51.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:51.330+0000 7f97e67fc640 1 -- 172.21.15.92:0/113850743 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f97e0043930 con 0x7f97f0143ae0 2023-11-24T06:12:51.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:51.331+0000 7f97e67fc640 1 --2- 172.21.15.92:0/113850743 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f97d406aed0 0x7f97d406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:51.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:51.331+0000 7f97e67fc640 1 -- 172.21.15.92:0/113850743 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f97e003e070 con 0x7f97f0143ae0 2023-11-24T06:12:51.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:51.331+0000 7f97f5e1f640 1 --2- 172.21.15.92:0/113850743 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f97d406aed0 0x7f97d406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:51.335 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:51.332+0000 7f97f5e1f640 1 --2- 172.21.15.92:0/113850743 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f97d406aed0 0x7f97d406d390 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f97ec003eb0 tx=0x7f97ec009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:51.338 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:51.336+0000 7f97e67fc640 1 -- 172.21.15.92:0/113850743 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f97f00cc6e0 con 0x7f97f0143ae0 2023-11-24T06:12:51.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:51.564+0000 7f97f8083640 1 -- 172.21.15.92:0/113850743 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f97f00cc6e0 con 0x7f97f0143ae0 2023-11-24T06:12:51.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:51.565+0000 7f97e67fc640 1 -- 172.21.15.92:0/113850743 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+4919 (secure 0 0 0) 0x7f97f00cc6e0 con 0x7f97f0143ae0 2023-11-24T06:12:51.568 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-24T06:12:51.568 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":32,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4778,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6835/1184353034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":1184353034},{"type":"v1","addr":"172.21.15.118:6835","nonce":1184353034}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":26},{"gid":4942,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6833/1256969666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":1256969666},{"type":"v1","addr":"172.21.15.118:6833","nonce":1256969666}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":26},{"gid":5706,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6836/3849408561","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":3849408561},{"type":"v1","addr":"172.21.15.92:6836","nonce":3849408561}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":31,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T06:12:30.605447+0000","modified":"2023-11-24T06:12:34.015371+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5709},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5709":{"gid":5709,"name":"c","rank":0,"incarnation":27,"state":"up:active","state_seq":3,"addr":"172.21.15.92:6837/1958912045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":1958912045},{"type":"v1","addr":"172.21.15.92:6837","nonce":1958912045}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-24T06:12:51.569 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 32 2023-11-24T06:12:51.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:51.567+0000 7f97f8083640 1 -- 172.21.15.92:0/113850743 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f97d406aed0 msgr2=0x7f97d406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:51.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:51.567+0000 7f97f8083640 1 --2- 172.21.15.92:0/113850743 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f97d406aed0 0x7f97d406d390 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f97ec003eb0 tx=0x7f97ec009040 comp rx=0 tx=0).stop 2023-11-24T06:12:51.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:51.567+0000 7f97f8083640 1 -- 172.21.15.92:0/113850743 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f97f0143ae0 msgr2=0x7f97f00c4390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:51.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:51.567+0000 7f97f8083640 1 --2- 172.21.15.92:0/113850743 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f97f0143ae0 0x7f97f00c4390 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f97e0000c00 tx=0x7f97e0004730 comp rx=0 tx=0).stop 2023-11-24T06:12:51.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:51.567+0000 7f97f8083640 1 -- 172.21.15.92:0/113850743 shutdown_connections 2023-11-24T06:12:51.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:51.567+0000 7f97f8083640 1 --2- 172.21.15.92:0/113850743 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f97d406aed0 0x7f97d406d390 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:51.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:51.567+0000 7f97f8083640 1 --2- 172.21.15.92:0/113850743 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f97f015e1e0 0x7f97f01f6030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:51.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:51.567+0000 7f97f8083640 1 --2- 172.21.15.92:0/113850743 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f97f0143ae0 0x7f97f00c4390 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:51.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:51.567+0000 7f97f8083640 1 --2- 172.21.15.92:0/113850743 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f97f00cf9a0 0x7f97f00c3e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:51.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:51.567+0000 7f97f8083640 1 -- 172.21.15.92:0/113850743 >> 172.21.15.92:0/113850743 conn(0x7f97f006a800 msgr2=0x7f97f00b0bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:51.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:51.568+0000 7f97f8083640 1 -- 172.21.15.92:0/113850743 shutdown_connections 2023-11-24T06:12:51.572 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:51.568+0000 7f97f8083640 1 -- 172.21.15.92:0/113850743 wait complete. 2023-11-24T06:12:51.589 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:12:51.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:51.800+0000 7ff969bac640 1 Processor -- start 2023-11-24T06:12:51.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:51.800+0000 7ff969bac640 1 -- start start 2023-11-24T06:12:51.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:51.801+0000 7ff969bac640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff96413fcb0 0x7ff964164d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:51.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:51.801+0000 7ff969bac640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9640cdc40 con 0x7ff96415afb0 2023-11-24T06:12:51.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:51.801+0000 7ff969bac640 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9640cdec0 con 0x7ff96413cfa0 2023-11-24T06:12:51.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:51.801+0000 7ff969bac640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9640ce1a0 con 0x7ff96413fcb0 2023-11-24T06:12:51.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:51.802+0000 7ff963fff640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7ff96415afb0 0x7ff96413f5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:53310/0 (socket says 172.21.15.92:53310) 2023-11-24T06:12:51.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:51.802+0000 7ff963fff640 1 -- 172.21.15.92:0/84013352 learned_addr learned my addr 172.21.15.92:0/84013352 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-24T06:12:51.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:51.802+0000 7ff9693ab640 1 --2- 172.21.15.92:0/84013352 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff96413fcb0 0x7ff964164d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:51.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:51.802+0000 7ff9637fe640 1 -- 172.21.15.92:0/84013352 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1449474242 0 0) 0x7ff9640cdc40 con 0x7ff96415afb0 2023-11-24T06:12:51.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:51.802+0000 7ff9637fe640 1 -- 172.21.15.92:0/84013352 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff950003520 con 0x7ff96415afb0 2023-11-24T06:12:51.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:51.802+0000 7ff9637fe640 1 -- 172.21.15.92:0/84013352 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2065757116 0 0) 0x7ff950003520 con 0x7ff96415afb0 2023-11-24T06:12:51.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:51.802+0000 7ff9637fe640 1 -- 172.21.15.92:0/84013352 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff96413fcb0 msgr2=0x7ff964164d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:51.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:51.802+0000 7ff9637fe640 1 --2- 172.21.15.92:0/84013352 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff96413fcb0 0x7ff964164d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:51.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:51.802+0000 7ff9637fe640 1 -- 172.21.15.92:0/84013352 >> v1:172.21.15.118:6789/0 conn(0x7ff96413cfa0 legacy=0x7ff9640cd530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:51.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:51.802+0000 7ff9637fe640 1 -- 172.21.15.92:0/84013352 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9640ce480 con 0x7ff96415afb0 2023-11-24T06:12:51.808 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:51.802+0000 7ff9637fe640 1 -- 172.21.15.92:0/84013352 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2607661350 0 0) 0x7ff94c001670 con 0x7ff96415afb0 2023-11-24T06:12:51.808 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:51.802+0000 7ff9637fe640 1 -- 172.21.15.92:0/84013352 >> v1:172.21.15.92:6789/0 conn(0x7ff96415afb0 legacy=0x7ff96413f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:51.808 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:51.803+0000 7ff9637fe640 1 --2- 172.21.15.92:0/84013352 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff9500039a0 0x7ff950003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:51.808 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:51.803+0000 7ff9637fe640 1 --2- 172.21.15.92:0/84013352 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff96413fcb0 0x7ff9500047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:51.808 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:51.803+0000 7ff9637fe640 1 --2- 172.21.15.92:0/84013352 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff950004d00 0x7ff950005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:51.808 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:51.803+0000 7ff9637fe640 1 -- 172.21.15.92:0/84013352 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9640ce1a0 con 0x7ff950004d00 2023-11-24T06:12:51.808 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:51.803+0000 7ff9637fe640 1 -- 172.21.15.92:0/84013352 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff95000bae0 con 0x7ff9500039a0 2023-11-24T06:12:51.808 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:51.803+0000 7ff9637fe640 1 -- 172.21.15.92:0/84013352 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff95000bca0 con 0x7ff96413fcb0 2023-11-24T06:12:51.808 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:51.803+0000 7ff9693ab640 1 --2- 172.21.15.92:0/84013352 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff950004d00 0x7ff950005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:51.808 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:51.803+0000 7ff963fff640 1 --2- 172.21.15.92:0/84013352 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff96413fcb0 0x7ff9500047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:51.808 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:51.803+0000 7ff9693ab640 1 -- 172.21.15.92:0/84013352 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff96413fcb0 msgr2=0x7ff9500047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:51.808 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:51.803+0000 7ff9693ab640 1 --2- 172.21.15.92:0/84013352 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff96413fcb0 0x7ff9500047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:51.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:51.803+0000 7ff9693ab640 1 -- 172.21.15.92:0/84013352 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff9500039a0 msgr2=0x7ff950003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:51.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:51.803+0000 7ff9693ab640 1 --2- 172.21.15.92:0/84013352 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff9500039a0 0x7ff950003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:51.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:51.803+0000 7ff9693ab640 1 -- 172.21.15.92:0/84013352 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff950003520 con 0x7ff950004d00 2023-11-24T06:12:51.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:51.803+0000 7ff963fff640 1 --2- 172.21.15.92:0/84013352 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff96413fcb0 0x7ff9500047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:12:51.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:51.804+0000 7ff9693ab640 1 --2- 172.21.15.92:0/84013352 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff950004d00 0x7ff950005180 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7ff95c009ac0 tx=0x7ff9500057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d4fbc614674c597 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:51.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:51.805+0000 7ff9637fe640 1 -- 172.21.15.92:0/84013352 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff95c02c6b0 con 0x7ff950004d00 2023-11-24T06:12:51.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:51.805+0000 7ff9637fe640 1 -- 172.21.15.92:0/84013352 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff95c02c810 con 0x7ff950004d00 2023-11-24T06:12:51.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:51.805+0000 7ff9637fe640 1 -- 172.21.15.92:0/84013352 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff95c02cba0 con 0x7ff950004d00 2023-11-24T06:12:51.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:51.805+0000 7ff969bac640 1 -- 172.21.15.92:0/84013352 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff950004d00 msgr2=0x7ff950005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:51.811 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:51.805+0000 7ff969bac640 1 --2- 172.21.15.92:0/84013352 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff950004d00 0x7ff950005180 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7ff95c009ac0 tx=0x7ff9500057f0 comp rx=0 tx=0).stop 2023-11-24T06:12:51.811 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:51.805+0000 7ff9693ab640 1 -- 172.21.15.92:0/84013352 reap_dead start 2023-11-24T06:12:51.811 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:51.805+0000 7ff969bac640 1 -- 172.21.15.92:0/84013352 shutdown_connections 2023-11-24T06:12:51.811 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:51.805+0000 7ff969bac640 1 -- 172.21.15.92:0/84013352 >> 172.21.15.92:0/84013352 conn(0x7ff96406a800 msgr2=0x7ff96406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:51.811 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:51.805+0000 7ff969bac640 1 -- 172.21.15.92:0/84013352 shutdown_connections 2023-11-24T06:12:51.811 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:51.805+0000 7ff969bac640 1 -- 172.21.15.92:0/84013352 wait complete. 2023-11-24T06:12:51.811 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:51.807+0000 7ff969bac640 1 Processor -- start 2023-11-24T06:12:51.811 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:51.807+0000 7ff969bac640 1 -- start start 2023-11-24T06:12:51.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:51.808+0000 7ff969bac640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff950004d00 0x7ff964153af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:51.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:51.808+0000 7ff969bac640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff964135910 0x7ff964135d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:51.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:51.808+0000 7ff969bac640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff964156fa0 0x7ff964157420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:51.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:51.808+0000 7ff969bac640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9641362d0 con 0x7ff950004d00 2023-11-24T06:12:51.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:51.808+0000 7ff969bac640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff96415b5a0 con 0x7ff964156fa0 2023-11-24T06:12:51.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:51.808+0000 7ff969bac640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff964155900 con 0x7ff964135910 2023-11-24T06:12:51.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:51.808+0000 7ff968baa640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff950004d00 0x7ff964153af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:51.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:51.808+0000 7ff968baa640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff950004d00 0x7ff964153af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:55802/0 (socket says 172.21.15.92:55802) 2023-11-24T06:12:51.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:51.808+0000 7ff968baa640 1 -- 172.21.15.92:0/801038914 learned_addr learned my addr 172.21.15.92:0/801038914 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:12:51.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:51.809+0000 7ff968baa640 1 -- 172.21.15.92:0/801038914 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff964135910 msgr2=0x7ff964135d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:51.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:51.809+0000 7ff968baa640 1 --2- 172.21.15.92:0/801038914 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff964135910 0x7ff964135d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:51.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:51.809+0000 7ff968baa640 1 -- 172.21.15.92:0/801038914 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff964156fa0 msgr2=0x7ff964157420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:51.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:51.809+0000 7ff968baa640 1 --2- 172.21.15.92:0/801038914 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff964156fa0 0x7ff964157420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:51.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:51.809+0000 7ff968baa640 1 -- 172.21.15.92:0/801038914 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff964157960 con 0x7ff950004d00 2023-11-24T06:12:51.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:51.809+0000 7ff968baa640 1 --2- 172.21.15.92:0/801038914 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff950004d00 0x7ff964153af0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7ff95400c8a0 tx=0x7ff95400cd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:51.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:51.809+0000 7ff9617fa640 1 -- 172.21.15.92:0/801038914 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff954003b70 con 0x7ff950004d00 2023-11-24T06:12:51.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:51.809+0000 7ff9617fa640 1 -- 172.21.15.92:0/801038914 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff954003cf0 con 0x7ff950004d00 2023-11-24T06:12:51.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:51.809+0000 7ff9617fa640 1 -- 172.21.15.92:0/801038914 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff95400aa10 con 0x7ff950004d00 2023-11-24T06:12:51.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:51.809+0000 7ff969bac640 1 -- 172.21.15.92:0/801038914 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff964154c80 con 0x7ff950004d00 2023-11-24T06:12:51.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:51.810+0000 7ff969bac640 1 -- 172.21.15.92:0/801038914 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff964155240 con 0x7ff950004d00 2023-11-24T06:12:51.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:51.811+0000 7ff969bac640 1 -- 172.21.15.92:0/801038914 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff964067450 con 0x7ff950004d00 2023-11-24T06:12:51.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:51.811+0000 7ff9617fa640 1 -- 172.21.15.92:0/801038914 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7ff95400ab70 con 0x7ff950004d00 2023-11-24T06:12:51.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:51.812+0000 7ff9617fa640 1 --2- 172.21.15.92:0/801038914 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7ff94406aed0 0x7ff94406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:51.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:51.812+0000 7ff9617fa640 1 -- 172.21.15.92:0/801038914 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff954089470 con 0x7ff950004d00 2023-11-24T06:12:51.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:51.813+0000 7ff963fff640 1 --2- 172.21.15.92:0/801038914 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7ff94406aed0 0x7ff94406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:51.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:51.813+0000 7ff963fff640 1 --2- 172.21.15.92:0/801038914 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7ff94406aed0 0x7ff94406d390 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7ff94c0017a0 tx=0x7ff94c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:51.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:51.817+0000 7ff9617fa640 1 -- 172.21.15.92:0/801038914 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff954054530 con 0x7ff950004d00 2023-11-24T06:12:52.010 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:52.007+0000 7ff969bac640 1 -- 172.21.15.92:0/801038914 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff9640cbff0 con 0x7ff950004d00 2023-11-24T06:12:52.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:52.009+0000 7ff9617fa640 1 -- 172.21.15.92:0/801038914 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v52) v1 ==== 74+0+16517 (secure 0 0 0) 0x7ff954003e60 con 0x7ff950004d00 2023-11-24T06:12:52.012 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-24T06:12:52.012 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":52,"fsid":"2d6dee5f-8aa3-470f-931a-50ade963b1cb","created":"2023-11-24T06:10:04.707204+0000","modified":"2023-11-24T06:12:32.152035+0000","last_up_change":"2023-11-24T06:10:17.598352+0000","last_in_change":"2023-11-24T06:10:09.040793+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:10:17.733098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:12:28.410305+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-11-24T06:12:29.625599+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ff37a9fb-135f-471f-b4b0-f7b5772006c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6803","nonce":20560714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6805","nonce":20560714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6809","nonce":20560714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6807","nonce":20560714}]},"public_addr":"172.21.15.92:6803/20560714","cluster_addr":"172.21.15.92:6805/20560714","heartbeat_back_addr":"172.21.15.92:6809/20560714","heartbeat_front_addr":"172.21.15.92:6807/20560714","state":["exists","up"]},{"osd":1,"uuid":"03cdd353-dcf1-4f38-94fe-a795137c06b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6819","nonce":2451239906}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6821","nonce":2451239906}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6825","nonce":2451239906}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6823","nonce":2451239906}]},"public_addr":"172.21.15.92:6819/2451239906","cluster_addr":"172.21.15.92:6821/2451239906","heartbeat_back_addr":"172.21.15.92:6825/2451239906","heartbeat_front_addr":"172.21.15.92:6823/2451239906","state":["exists","up"]},{"osd":2,"uuid":"a4a20d80-5d57-484b-becd-9aa8070dd3aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6811","nonce":1124979277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6813","nonce":1124979277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6817","nonce":1124979277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6815","nonce":1124979277}]},"public_addr":"172.21.15.92:6811/1124979277","cluster_addr":"172.21.15.92:6813/1124979277","heartbeat_back_addr":"172.21.15.92:6817/1124979277","heartbeat_front_addr":"172.21.15.92:6815/1124979277","state":["exists","up"]},{"osd":3,"uuid":"4648d7ce-9b29-4583-9425-4ba8fd2b1903","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6827","nonce":1734115782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6829","nonce":1734115782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6833","nonce":1734115782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6831","nonce":1734115782}]},"public_addr":"172.21.15.92:6827/1734115782","cluster_addr":"172.21.15.92:6829/1734115782","heartbeat_back_addr":"172.21.15.92:6833/1734115782","heartbeat_front_addr":"172.21.15.92:6831/1734115782","state":["exists","up"]},{"osd":4,"uuid":"3bc9d7eb-c6fa-4fcf-a3ec-ade604e05aac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6819","nonce":3425937229}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6822","nonce":3425937229}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6830","nonce":3425937229}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6826","nonce":3425937229}]},"public_addr":"172.21.15.118:6819/3425937229","cluster_addr":"172.21.15.118:6822/3425937229","heartbeat_back_addr":"172.21.15.118:6830/3425937229","heartbeat_front_addr":"172.21.15.118:6826/3425937229","state":["exists","up"]},{"osd":5,"uuid":"dce9ba6d-59ed-46d1-84fc-a84b13de10c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6801","nonce":827455984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6803","nonce":827455984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6807","nonce":827455984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6805","nonce":827455984}]},"public_addr":"172.21.15.118:6801/827455984","cluster_addr":"172.21.15.118:6803/827455984","heartbeat_back_addr":"172.21.15.118:6807/827455984","heartbeat_front_addr":"172.21.15.118:6805/827455984","state":["exists","up"]},{"osd":6,"uuid":"82aec504-ae58-4709-8d30-46f2b12081b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6809","nonce":2946986141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6811","nonce":2946986141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6815","nonce":2946986141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6813","nonce":2946986141}]},"public_addr":"172.21.15.118:6809/2946986141","cluster_addr":"172.21.15.118:6811/2946986141","heartbeat_back_addr":"172.21.15.118:6815/2946986141","heartbeat_front_addr":"172.21.15.118:6813/2946986141","state":["exists","up"]},{"osd":7,"uuid":"4ed4c624-c88f-4d35-bdb1-bf209b5dffb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6817","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6818","nonce":48122059}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6821","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6823","nonce":48122059}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6829","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6831","nonce":48122059}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6825","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6827","nonce":48122059}]},"public_addr":"172.21.15.118:6818/48122059","cluster_addr":"172.21.15.118:6823/48122059","heartbeat_back_addr":"172.21.15.118:6831/48122059","heartbeat_front_addr":"172.21.15.118:6827/48122059","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.536664+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.368324+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.271201+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.342971+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.621832+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.387110+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.632946+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.416032+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:12:52.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:52.011+0000 7ff969bac640 1 -- 172.21.15.92:0/801038914 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7ff94406aed0 msgr2=0x7ff94406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:52.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:52.011+0000 7ff969bac640 1 --2- 172.21.15.92:0/801038914 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7ff94406aed0 0x7ff94406d390 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7ff94c0017a0 tx=0x7ff94c009040 comp rx=0 tx=0).stop 2023-11-24T06:12:52.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:52.011+0000 7ff969bac640 1 -- 172.21.15.92:0/801038914 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff950004d00 msgr2=0x7ff964153af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:52.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:52.011+0000 7ff969bac640 1 --2- 172.21.15.92:0/801038914 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff950004d00 0x7ff964153af0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7ff95400c8a0 tx=0x7ff95400cd60 comp rx=0 tx=0).stop 2023-11-24T06:12:52.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:52.011+0000 7ff969bac640 1 -- 172.21.15.92:0/801038914 shutdown_connections 2023-11-24T06:12:52.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:52.011+0000 7ff969bac640 1 --2- 172.21.15.92:0/801038914 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7ff94406aed0 0x7ff94406d390 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:52.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:52.011+0000 7ff969bac640 1 --2- 172.21.15.92:0/801038914 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff964156fa0 0x7ff964157420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:52.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:52.011+0000 7ff969bac640 1 --2- 172.21.15.92:0/801038914 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff964135910 0x7ff964135d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:52.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:52.011+0000 7ff969bac640 1 --2- 172.21.15.92:0/801038914 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff950004d00 0x7ff964153af0 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:52.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:52.011+0000 7ff969bac640 1 -- 172.21.15.92:0/801038914 >> 172.21.15.92:0/801038914 conn(0x7ff96406a800 msgr2=0x7ff9640aea70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:52.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:52.012+0000 7ff969bac640 1 -- 172.21.15.92:0/801038914 shutdown_connections 2023-11-24T06:12:52.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:52.012+0000 7ff969bac640 1 -- 172.21.15.92:0/801038914 wait complete. 2023-11-24T06:12:52.032 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T06:10:17.733098+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T06:12:28.410305+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '51', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T06:12:29.625599+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '52', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T06:12:52.033 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T06:12:52.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:52.246+0000 7f6fc5200640 1 Processor -- start 2023-11-24T06:12:52.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:52.246+0000 7f6fc5200640 1 -- start start 2023-11-24T06:12:52.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:52.247+0000 7f6fc5200640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f6fc0143ae0 0x7f6fc00cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:52.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:52.247+0000 7f6fc5200640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6fc015e3b0 0x7f6fc00702c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:52.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:52.247+0000 7f6fc5200640 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fc00cfc30 con 0x7f6fc00cf7d0 2023-11-24T06:12:52.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:52.247+0000 7f6fc5200640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6fc00b1ee0 con 0x7f6fc0143ae0 2023-11-24T06:12:52.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:52.247+0000 7f6fc5200640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fc00b2780 con 0x7f6fc015e3b0 2023-11-24T06:12:52.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:52.248+0000 7f6fbed9d640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f6fc0143ae0 0x7f6fc00cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:52.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:52.248+0000 7f6fbed9d640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f6fc0143ae0 0x7f6fc00cf290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:55816/0 (socket says 172.21.15.92:55816) 2023-11-24T06:12:52.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:52.248+0000 7f6fbed9d640 1 -- 172.21.15.92:0/1995108917 learned_addr learned my addr 172.21.15.92:0/1995108917 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:12:52.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:52.248+0000 7f6fbf59e640 1 --2- 172.21.15.92:0/1995108917 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6fc015e3b0 0x7f6fc00702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:52.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:52.248+0000 7f6fbed9d640 1 -- 172.21.15.92:0/1995108917 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6fc015e3b0 msgr2=0x7f6fc00702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:52.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:52.248+0000 7f6fbed9d640 1 --2- 172.21.15.92:0/1995108917 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6fc015e3b0 0x7f6fc00702c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:52.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:52.248+0000 7f6fbed9d640 1 -- 172.21.15.92:0/1995108917 >> v1:172.21.15.118:6789/0 conn(0x7f6fc00cf7d0 legacy=0x7f6fc015dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:52.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:52.248+0000 7f6fbed9d640 1 -- 172.21.15.92:0/1995108917 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fc015e990 con 0x7f6fc0143ae0 2023-11-24T06:12:52.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:52.250+0000 7f6fbed9d640 1 --2- 172.21.15.92:0/1995108917 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6fc0143ae0 0x7f6fc00cf290 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f6fb000bb10 tx=0x7f6fb002e150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a9bfdb863ee13f6 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:52.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:52.250+0000 7f6fbdd9b640 1 -- 172.21.15.92:0/1995108917 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fb0036440 con 0x7f6fc0143ae0 2023-11-24T06:12:52.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:52.250+0000 7f6fbdd9b640 1 -- 172.21.15.92:0/1995108917 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fb00365a0 con 0x7f6fc0143ae0 2023-11-24T06:12:52.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:52.250+0000 7f6fbdd9b640 1 -- 172.21.15.92:0/1995108917 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fb0036930 con 0x7f6fc0143ae0 2023-11-24T06:12:52.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:52.251+0000 7f6fc5200640 1 -- 172.21.15.92:0/1995108917 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6fc0143ae0 msgr2=0x7f6fc00cf290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:52.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:52.251+0000 7f6fc5200640 1 --2- 172.21.15.92:0/1995108917 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6fc0143ae0 0x7f6fc00cf290 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f6fb000bb10 tx=0x7f6fb002e150 comp rx=0 tx=0).stop 2023-11-24T06:12:52.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:52.251+0000 7f6fc5200640 1 -- 172.21.15.92:0/1995108917 shutdown_connections 2023-11-24T06:12:52.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:52.251+0000 7f6fc5200640 1 --2- 172.21.15.92:0/1995108917 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6fc015e3b0 0x7f6fc00702c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:52.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:52.251+0000 7f6fc5200640 1 --2- 172.21.15.92:0/1995108917 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6fc0143ae0 0x7f6fc00cf290 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:52.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:52.251+0000 7f6fc5200640 1 -- 172.21.15.92:0/1995108917 >> 172.21.15.92:0/1995108917 conn(0x7f6fc006a800 msgr2=0x7f6fc006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:52.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:52.251+0000 7f6fc5200640 1 -- 172.21.15.92:0/1995108917 shutdown_connections 2023-11-24T06:12:52.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:52.251+0000 7f6fc5200640 1 -- 172.21.15.92:0/1995108917 wait complete. 2023-11-24T06:12:52.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:52.253+0000 7f6fc5200640 1 Processor -- start 2023-11-24T06:12:52.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:52.253+0000 7f6fc5200640 1 -- start start 2023-11-24T06:12:52.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:52.254+0000 7f6fc5200640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6fc00cf7d0 0x7f6fc013f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:52.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:52.254+0000 7f6fc5200640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6fc0143ae0 0x7f6fc00acb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:52.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:52.254+0000 7f6fc5200640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6fc015e3b0 0x7f6fc0140980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:52.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:52.254+0000 7f6fc5200640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fc00ae200 con 0x7f6fc00cf7d0 2023-11-24T06:12:52.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:52.254+0000 7f6fc5200640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fc00ad5f0 con 0x7f6fc015e3b0 2023-11-24T06:12:52.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:52.254+0000 7f6fc5200640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fc00ad8d0 con 0x7f6fc0143ae0 2023-11-24T06:12:52.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:52.254+0000 7f6fbed9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6fc00cf7d0 0x7f6fc013f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:52.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:52.254+0000 7f6fbed9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6fc00cf7d0 0x7f6fc013f100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:55830/0 (socket says 172.21.15.92:55830) 2023-11-24T06:12:52.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:52.254+0000 7f6fbed9d640 1 -- 172.21.15.92:0/3147125041 learned_addr learned my addr 172.21.15.92:0/3147125041 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:12:52.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:52.255+0000 7f6fbe59c640 1 --2- 172.21.15.92:0/3147125041 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6fc0143ae0 0x7f6fc00acb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:52.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:52.255+0000 7f6fbed9d640 1 -- 172.21.15.92:0/3147125041 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6fc0143ae0 msgr2=0x7f6fc00acb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:52.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:52.255+0000 7f6fbf59e640 1 --2- 172.21.15.92:0/3147125041 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6fc015e3b0 0x7f6fc0140980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:52.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:52.255+0000 7f6fbed9d640 1 --2- 172.21.15.92:0/3147125041 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6fc0143ae0 0x7f6fc00acb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:52.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:52.255+0000 7f6fbed9d640 1 -- 172.21.15.92:0/3147125041 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6fc015e3b0 msgr2=0x7f6fc0140980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:52.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:52.255+0000 7f6fbed9d640 1 --2- 172.21.15.92:0/3147125041 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6fc015e3b0 0x7f6fc0140980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:52.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:52.255+0000 7f6fbed9d640 1 -- 172.21.15.92:0/3147125041 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fb000b770 con 0x7f6fc00cf7d0 2023-11-24T06:12:52.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:52.255+0000 7f6fbe59c640 1 --2- 172.21.15.92:0/3147125041 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6fc0143ae0 0x7f6fc00acb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:12:52.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:52.255+0000 7f6fbed9d640 1 --2- 172.21.15.92:0/3147125041 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6fc00cf7d0 0x7f6fc013f100 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f6fb002e190 tx=0x7f6fb00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:52.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:52.255+0000 7f6fa77fe640 1 -- 172.21.15.92:0/3147125041 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fb0036440 con 0x7f6fc00cf7d0 2023-11-24T06:12:52.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:52.255+0000 7f6fa77fe640 1 -- 172.21.15.92:0/3147125041 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fb0034ba0 con 0x7f6fc00cf7d0 2023-11-24T06:12:52.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:52.255+0000 7f6fa77fe640 1 -- 172.21.15.92:0/3147125041 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fb0041460 con 0x7f6fc00cf7d0 2023-11-24T06:12:52.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:52.255+0000 7f6fc5200640 1 -- 172.21.15.92:0/3147125041 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fc00adbb0 con 0x7f6fc00cf7d0 2023-11-24T06:12:52.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:52.256+0000 7f6fc5200640 1 -- 172.21.15.92:0/3147125041 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fc013f640 con 0x7f6fc00cf7d0 2023-11-24T06:12:52.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:52.257+0000 7f6fc5200640 1 -- 172.21.15.92:0/3147125041 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6fc00675d0 con 0x7f6fc00cf7d0 2023-11-24T06:12:52.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:52.257+0000 7f6fa77fe640 1 -- 172.21.15.92:0/3147125041 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f6fb0049360 con 0x7f6fc00cf7d0 2023-11-24T06:12:52.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:52.258+0000 7f6fa77fe640 1 --2- 172.21.15.92:0/3147125041 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f6fac06af00 0x7f6fac06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:52.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:52.258+0000 7f6fa77fe640 1 -- 172.21.15.92:0/3147125041 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6fb00aba30 con 0x7f6fc00cf7d0 2023-11-24T06:12:52.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:52.259+0000 7f6fbe59c640 1 --2- 172.21.15.92:0/3147125041 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f6fac06af00 0x7f6fac06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:52.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:52.259+0000 7f6fbe59c640 1 --2- 172.21.15.92:0/3147125041 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f6fac06af00 0x7f6fac06d3c0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f6fb4000f60 tx=0x7f6fb4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:52.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:52.263+0000 7f6fa77fe640 1 -- 172.21.15.92:0/3147125041 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6fb0076af0 con 0x7f6fc00cf7d0 2023-11-24T06:12:52.456 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:52.453+0000 7f6fc5200640 1 -- 172.21.15.92:0/3147125041 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f6fc00cf320 con 0x7f6fc00cf7d0 2023-11-24T06:12:52.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:52.454+0000 7f6fa77fe640 1 -- 172.21.15.92:0/3147125041 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v52) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6fb003b020 con 0x7f6fc00cf7d0 2023-11-24T06:12:52.458 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 1 2023-11-24T06:12:52.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:52.457+0000 7f6fc5200640 1 -- 172.21.15.92:0/3147125041 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f6fac06af00 msgr2=0x7f6fac06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:52.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:52.457+0000 7f6fc5200640 1 --2- 172.21.15.92:0/3147125041 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f6fac06af00 0x7f6fac06d3c0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f6fb4000f60 tx=0x7f6fb4009040 comp rx=0 tx=0).stop 2023-11-24T06:12:52.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:52.457+0000 7f6fc5200640 1 -- 172.21.15.92:0/3147125041 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6fc00cf7d0 msgr2=0x7f6fc013f100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:52.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:52.457+0000 7f6fc5200640 1 --2- 172.21.15.92:0/3147125041 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6fc00cf7d0 0x7f6fc013f100 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f6fb002e190 tx=0x7f6fb00046c0 comp rx=0 tx=0).stop 2023-11-24T06:12:52.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:52.457+0000 7f6fc5200640 1 -- 172.21.15.92:0/3147125041 shutdown_connections 2023-11-24T06:12:52.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:52.457+0000 7f6fc5200640 1 --2- 172.21.15.92:0/3147125041 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f6fac06af00 0x7f6fac06d3c0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:52.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:52.457+0000 7f6fc5200640 1 --2- 172.21.15.92:0/3147125041 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6fc015e3b0 0x7f6fc0140980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:52.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:52.457+0000 7f6fc5200640 1 --2- 172.21.15.92:0/3147125041 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6fc0143ae0 0x7f6fc00acb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:52.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:52.457+0000 7f6fc5200640 1 --2- 172.21.15.92:0/3147125041 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6fc00cf7d0 0x7f6fc013f100 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:52.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:52.457+0000 7f6fc5200640 1 -- 172.21.15.92:0/3147125041 >> 172.21.15.92:0/3147125041 conn(0x7f6fc006a800 msgr2=0x7f6fc006abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:52.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:52.457+0000 7f6fc5200640 1 -- 172.21.15.92:0/3147125041 shutdown_connections 2023-11-24T06:12:52.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:52.457+0000 7f6fc5200640 1 -- 172.21.15.92:0/3147125041 wait complete. 2023-11-24T06:12:52.480 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T06:12:52.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:52.693+0000 7f5559b61640 1 Processor -- start 2023-11-24T06:12:52.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:52.694+0000 7f5559b61640 1 -- start start 2023-11-24T06:12:52.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:52.695+0000 7f5559b61640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f55540d1c40 0x7f55540b3fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:52.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:52.695+0000 7f5559b61640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55540b3400 con 0x7f555415d610 2023-11-24T06:12:52.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:52.695+0000 7f5559b61640 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55540b27f0 con 0x7f55540d17e0 2023-11-24T06:12:52.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:52.695+0000 7f5559b61640 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55540d1390 con 0x7f55540d1c40 2023-11-24T06:12:52.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:52.696+0000 7f55537fe640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f555415d610 0x7f55540ce820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:53320/0 (socket says 172.21.15.92:53320) 2023-11-24T06:12:52.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:52.696+0000 7f55537fe640 1 -- 172.21.15.92:0/2224730971 learned_addr learned my addr 172.21.15.92:0/2224730971 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-24T06:12:52.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:52.696+0000 7f55527fc640 1 -- 172.21.15.92:0/2224730971 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 190076209 0 0) 0x7f55540b3400 con 0x7f555415d610 2023-11-24T06:12:52.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:52.696+0000 7f55527fc640 1 -- 172.21.15.92:0/2224730971 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5540003520 con 0x7f555415d610 2023-11-24T06:12:52.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:52.696+0000 7f5553fff640 1 --2- 172.21.15.92:0/2224730971 >> v2:172.21.15.118:3300/0 conn(0x7f55540d1c40 0x7f55540b3fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:52.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:52.696+0000 7f55527fc640 1 -- 172.21.15.92:0/2224730971 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3033111592 0 0) 0x7f5540003520 con 0x7f555415d610 2023-11-24T06:12:52.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:52.696+0000 7f55527fc640 1 -- 172.21.15.92:0/2224730971 >> v2:172.21.15.118:3300/0 conn(0x7f55540d1c40 msgr2=0x7f55540b3fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:52.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:52.696+0000 7f55527fc640 1 --2- 172.21.15.92:0/2224730971 >> v2:172.21.15.118:3300/0 conn(0x7f55540d1c40 0x7f55540b3fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:52.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:52.696+0000 7f55527fc640 1 -- 172.21.15.92:0/2224730971 >> v1:172.21.15.118:6789/0 conn(0x7f55540d17e0 legacy=0x7f55540b3890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:52.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:52.696+0000 7f55527fc640 1 -- 172.21.15.92:0/2224730971 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55540b2ad0 con 0x7f555415d610 2023-11-24T06:12:52.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:52.696+0000 7f55527fc640 1 -- 172.21.15.92:0/2224730971 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2607661350 0 0) 0x7f5544001670 con 0x7f555415d610 2023-11-24T06:12:52.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:52.696+0000 7f55527fc640 1 -- 172.21.15.92:0/2224730971 >> v1:172.21.15.92:6789/0 conn(0x7f555415d610 legacy=0x7f55540ce820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:52.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:52.696+0000 7f55527fc640 1 --2- 172.21.15.92:0/2224730971 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f55400039c0 0x7f5540003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:52.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:52.697+0000 7f55527fc640 1 --2- 172.21.15.92:0/2224730971 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f55400047c0 0x7f5540004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:52.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:52.697+0000 7f55527fc640 1 --2- 172.21.15.92:0/2224730971 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f55400051d0 0x7f554000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:52.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:52.697+0000 7f55527fc640 1 -- 172.21.15.92:0/2224730971 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55540d1390 con 0x7f55400047c0 2023-11-24T06:12:52.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:52.697+0000 7f55527fc640 1 -- 172.21.15.92:0/2224730971 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f554000c100 con 0x7f55400039c0 2023-11-24T06:12:52.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:52.697+0000 7f55527fc640 1 -- 172.21.15.92:0/2224730971 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f554000c2c0 con 0x7f55400051d0 2023-11-24T06:12:52.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:52.697+0000 7f5552ffd640 1 --2- 172.21.15.92:0/2224730971 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f55400047c0 0x7f5540004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:52.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:52.697+0000 7f55537fe640 1 --2- 172.21.15.92:0/2224730971 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f55400039c0 0x7f5540003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:52.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:52.697+0000 7f5553fff640 1 --2- 172.21.15.92:0/2224730971 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f55400051d0 0x7f554000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:52.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:52.697+0000 7f5552ffd640 1 -- 172.21.15.92:0/2224730971 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f55400051d0 msgr2=0x7f554000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:52.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:52.697+0000 7f5552ffd640 1 --2- 172.21.15.92:0/2224730971 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f55400051d0 0x7f554000b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:52.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:52.697+0000 7f5552ffd640 1 -- 172.21.15.92:0/2224730971 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f55400039c0 msgr2=0x7f5540003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:52.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:52.697+0000 7f5552ffd640 1 --2- 172.21.15.92:0/2224730971 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f55400039c0 0x7f5540003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:52.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:52.697+0000 7f5552ffd640 1 -- 172.21.15.92:0/2224730971 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5540003520 con 0x7f55400047c0 2023-11-24T06:12:52.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:52.697+0000 7f5553fff640 1 -- 172.21.15.92:0/2224730971 reap_dead start 2023-11-24T06:12:52.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:52.699+0000 7f5552ffd640 1 --2- 172.21.15.92:0/2224730971 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f55400047c0 0x7f5540004c40 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f553c004480 tx=0x7f55400057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b9109c86ff03473 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:52.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:52.699+0000 7f55527fc640 1 -- 172.21.15.92:0/2224730971 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f553c031650 con 0x7f55400047c0 2023-11-24T06:12:52.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:52.699+0000 7f55527fc640 1 -- 172.21.15.92:0/2224730971 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f553c0317b0 con 0x7f55400047c0 2023-11-24T06:12:52.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:52.699+0000 7f55527fc640 1 -- 172.21.15.92:0/2224730971 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f553c031ba0 con 0x7f55400047c0 2023-11-24T06:12:52.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:52.699+0000 7f5559b61640 1 -- 172.21.15.92:0/2224730971 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f55400047c0 msgr2=0x7f5540004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:52.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:52.699+0000 7f5559b61640 1 --2- 172.21.15.92:0/2224730971 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f55400047c0 0x7f5540004c40 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f553c004480 tx=0x7f55400057f0 comp rx=0 tx=0).stop 2023-11-24T06:12:52.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:52.700+0000 7f5559b61640 1 -- 172.21.15.92:0/2224730971 shutdown_connections 2023-11-24T06:12:52.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:52.700+0000 7f5559b61640 1 --2- 172.21.15.92:0/2224730971 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f55400047c0 0x7f5540004c40 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:52.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:52.700+0000 7f5559b61640 1 -- 172.21.15.92:0/2224730971 >> 172.21.15.92:0/2224730971 conn(0x7f555409c540 msgr2=0x7f55540d3200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:52.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:52.700+0000 7f5559b61640 1 -- 172.21.15.92:0/2224730971 shutdown_connections 2023-11-24T06:12:52.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:52.700+0000 7f5559b61640 1 -- 172.21.15.92:0/2224730971 wait complete. 2023-11-24T06:12:52.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:52.701+0000 7f5559b61640 1 Processor -- start 2023-11-24T06:12:52.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:52.702+0000 7f5559b61640 1 -- start start 2023-11-24T06:12:52.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:52.702+0000 7f5559b61640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f55400047c0 0x7f55541f5e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:52.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:52.702+0000 7f5559b61640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f555415ebc0 0x7f5554153390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:52.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:52.702+0000 7f5559b61640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f555415f040 0x7f55541538d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:52.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:52.702+0000 7f5559b61640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55541601b0 con 0x7f55400047c0 2023-11-24T06:12:52.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:52.702+0000 7f5559b61640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55541f63a0 con 0x7f555415ebc0 2023-11-24T06:12:52.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:52.703+0000 7f5559b61640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f555415f5a0 con 0x7f555415f040 2023-11-24T06:12:52.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:52.703+0000 7f55537fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f55400047c0 0x7f55541f5e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:52.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:52.703+0000 7f55537fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f55400047c0 0x7f55541f5e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:55852/0 (socket says 172.21.15.92:55852) 2023-11-24T06:12:52.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:52.703+0000 7f55537fe640 1 -- 172.21.15.92:0/3400907369 learned_addr learned my addr 172.21.15.92:0/3400907369 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:12:52.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:52.703+0000 7f5553fff640 1 --2- 172.21.15.92:0/3400907369 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f555415f040 0x7f55541538d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:52.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:52.703+0000 7f55537fe640 1 -- 172.21.15.92:0/3400907369 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f555415f040 msgr2=0x7f55541538d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:52.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:52.703+0000 7f55537fe640 1 --2- 172.21.15.92:0/3400907369 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f555415f040 0x7f55541538d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:52.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:52.703+0000 7f55537fe640 1 -- 172.21.15.92:0/3400907369 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f555415ebc0 msgr2=0x7f5554153390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:52.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:52.703+0000 7f55537fe640 1 --2- 172.21.15.92:0/3400907369 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f555415ebc0 0x7f5554153390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:52.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:52.703+0000 7f55537fe640 1 -- 172.21.15.92:0/3400907369 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f555415f880 con 0x7f55400047c0 2023-11-24T06:12:52.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:52.703+0000 7f5553fff640 1 --2- 172.21.15.92:0/3400907369 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f555415f040 0x7f55541538d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:12:52.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:52.703+0000 7f55537fe640 1 --2- 172.21.15.92:0/3400907369 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f55400047c0 0x7f55541f5e60 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f554400e700 tx=0x7f554400ebc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:52.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:52.704+0000 7f554bfff640 1 -- 172.21.15.92:0/3400907369 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f554400c660 con 0x7f55400047c0 2023-11-24T06:12:52.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:52.704+0000 7f554bfff640 1 -- 172.21.15.92:0/3400907369 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f554400c7c0 con 0x7f55400047c0 2023-11-24T06:12:52.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:52.704+0000 7f5559b61640 1 -- 172.21.15.92:0/3400907369 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f555415fb60 con 0x7f55400047c0 2023-11-24T06:12:52.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:52.704+0000 7f554bfff640 1 -- 172.21.15.92:0/3400907369 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f554400cb50 con 0x7f55400047c0 2023-11-24T06:12:52.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:52.704+0000 7f5559b61640 1 -- 172.21.15.92:0/3400907369 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55541554f0 con 0x7f55400047c0 2023-11-24T06:12:52.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:52.705+0000 7f5559b61640 1 -- 172.21.15.92:0/3400907369 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55541556b0 con 0x7f55400047c0 2023-11-24T06:12:52.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:52.706+0000 7f554bfff640 1 -- 172.21.15.92:0/3400907369 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f554400ccb0 con 0x7f55400047c0 2023-11-24T06:12:52.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:52.707+0000 7f554bfff640 1 --2- 172.21.15.92:0/3400907369 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f553406aed0 0x7f553406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:52.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:52.707+0000 7f554bfff640 1 -- 172.21.15.92:0/3400907369 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f55440894b0 con 0x7f55400047c0 2023-11-24T06:12:52.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:52.707+0000 7f5552ffd640 1 --2- 172.21.15.92:0/3400907369 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f553406aed0 0x7f553406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:52.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:52.708+0000 7f5552ffd640 1 --2- 172.21.15.92:0/3400907369 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f553406aed0 0x7f553406d390 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f553c036000 tx=0x7f553c030000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:52.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:52.711+0000 7f554bfff640 1 -- 172.21.15.92:0/3400907369 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5544054500 con 0x7f55400047c0 2023-11-24T06:12:52.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:52.897+0000 7f5559b61640 1 -- 172.21.15.92:0/3400907369 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f555415b890 con 0x7f55400047c0 2023-11-24T06:12:52.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:52.898+0000 7f554bfff640 1 -- 172.21.15.92:0/3400907369 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v52) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5544057b20 con 0x7f55400047c0 2023-11-24T06:12:52.902 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 32 2023-11-24T06:12:52.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:52.901+0000 7f5559b61640 1 -- 172.21.15.92:0/3400907369 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f553406aed0 msgr2=0x7f553406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:52.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:52.901+0000 7f5559b61640 1 --2- 172.21.15.92:0/3400907369 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f553406aed0 0x7f553406d390 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f553c036000 tx=0x7f553c030000 comp rx=0 tx=0).stop 2023-11-24T06:12:52.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:52.901+0000 7f5559b61640 1 -- 172.21.15.92:0/3400907369 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f55400047c0 msgr2=0x7f55541f5e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:52.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:52.901+0000 7f5559b61640 1 --2- 172.21.15.92:0/3400907369 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f55400047c0 0x7f55541f5e60 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f554400e700 tx=0x7f554400ebc0 comp rx=0 tx=0).stop 2023-11-24T06:12:52.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:52.901+0000 7f5559b61640 1 -- 172.21.15.92:0/3400907369 shutdown_connections 2023-11-24T06:12:52.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:52.901+0000 7f5559b61640 1 --2- 172.21.15.92:0/3400907369 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f553406aed0 0x7f553406d390 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:52.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:52.901+0000 7f5559b61640 1 --2- 172.21.15.92:0/3400907369 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f555415f040 0x7f55541538d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:52.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:52.901+0000 7f5559b61640 1 --2- 172.21.15.92:0/3400907369 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f555415ebc0 0x7f5554153390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:52.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:52.901+0000 7f5559b61640 1 --2- 172.21.15.92:0/3400907369 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f55400047c0 0x7f55541f5e60 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:52.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:52.901+0000 7f5559b61640 1 -- 172.21.15.92:0/3400907369 >> 172.21.15.92:0/3400907369 conn(0x7f555409c540 msgr2=0x7f555406c800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:52.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:52.901+0000 7f5559b61640 1 -- 172.21.15.92:0/3400907369 shutdown_connections 2023-11-24T06:12:52.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:52.901+0000 7f5559b61640 1 -- 172.21.15.92:0/3400907369 wait complete. 2023-11-24T06:12:52.923 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T06:12:53.133 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:53.130+0000 7fef5c305640 1 Processor -- start 2023-11-24T06:12:53.133 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:53.130+0000 7fef5c305640 1 -- start start 2023-11-24T06:12:53.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:53.132+0000 7fef5c305640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fef540ae5a0 0x7fef5415e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:53.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:53.132+0000 7fef5c305640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fef540cf290 0x7fef540cf720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:53.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:53.132+0000 7fef5c305640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef54070c40 con 0x7fef540b3ae0 2023-11-24T06:12:53.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:53.132+0000 7fef5c305640 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef54071450 con 0x7fef540cf290 2023-11-24T06:12:53.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:53.132+0000 7fef5c305640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef540cfc60 con 0x7fef540ae5a0 2023-11-24T06:12:53.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:53.132+0000 7fef598a0640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fef540b3ae0 0x7fef5406cbe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:53326/0 (socket says 172.21.15.92:53326) 2023-11-24T06:12:53.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:53.132+0000 7fef598a0640 1 -- 172.21.15.92:0/97158999 learned_addr learned my addr 172.21.15.92:0/97158999 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-24T06:12:53.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:53.133+0000 7fef5a0a1640 1 --2- 172.21.15.92:0/97158999 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fef540ae5a0 0x7fef5415e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:53.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:53.133+0000 7fef5a8a2640 1 --2- 172.21.15.92:0/97158999 >> v2:172.21.15.118:3300/0 conn(0x7fef540cf290 0x7fef540cf720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:53.138 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:53.133+0000 7fef5909f640 1 -- 172.21.15.92:0/97158999 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1331991898 0 0) 0x7fef54070c40 con 0x7fef540b3ae0 2023-11-24T06:12:53.138 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:53.133+0000 7fef5909f640 1 -- 172.21.15.92:0/97158999 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fef40003520 con 0x7fef540b3ae0 2023-11-24T06:12:53.138 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:53.133+0000 7fef5909f640 1 -- 172.21.15.92:0/97158999 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3288604580 0 0) 0x7fef40003520 con 0x7fef540b3ae0 2023-11-24T06:12:53.138 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:53.133+0000 7fef5909f640 1 -- 172.21.15.92:0/97158999 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fef540ae5a0 msgr2=0x7fef5415e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:53.138 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:53.133+0000 7fef5909f640 1 --2- 172.21.15.92:0/97158999 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fef540ae5a0 0x7fef5415e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:53.138 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:53.133+0000 7fef5909f640 1 -- 172.21.15.92:0/97158999 >> v2:172.21.15.118:3300/0 conn(0x7fef540cf290 msgr2=0x7fef540cf720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:53.138 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:53.133+0000 7fef5909f640 1 --2- 172.21.15.92:0/97158999 >> v2:172.21.15.118:3300/0 conn(0x7fef540cf290 0x7fef540cf720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:53.138 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:53.133+0000 7fef5909f640 1 -- 172.21.15.92:0/97158999 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef5406c470 con 0x7fef540b3ae0 2023-11-24T06:12:53.138 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:53.133+0000 7fef5909f640 1 -- 172.21.15.92:0/97158999 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2607661350 0 0) 0x7fef44001670 con 0x7fef540b3ae0 2023-11-24T06:12:53.138 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:53.133+0000 7fef5909f640 1 -- 172.21.15.92:0/97158999 >> v1:172.21.15.92:6789/0 conn(0x7fef540b3ae0 legacy=0x7fef5406cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:53.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:53.133+0000 7fef5909f640 1 --2- 172.21.15.92:0/97158999 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fef40003970 0x7fef40003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:53.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:53.133+0000 7fef5909f640 1 --2- 172.21.15.92:0/97158999 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fef540ae5a0 0x7fef400047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:53.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:53.133+0000 7fef5909f640 1 --2- 172.21.15.92:0/97158999 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fef40004d00 0x7fef40005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:53.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:53.133+0000 7fef5909f640 1 -- 172.21.15.92:0/97158999 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef54071450 con 0x7fef40004d00 2023-11-24T06:12:53.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:53.133+0000 7fef5909f640 1 -- 172.21.15.92:0/97158999 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef540cfc60 con 0x7fef40003970 2023-11-24T06:12:53.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:53.133+0000 7fef5909f640 1 -- 172.21.15.92:0/97158999 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef4000b9b0 con 0x7fef540ae5a0 2023-11-24T06:12:53.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:53.134+0000 7fef5a8a2640 1 --2- 172.21.15.92:0/97158999 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fef40004d00 0x7fef40005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:53.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:53.134+0000 7fef5a0a1640 1 --2- 172.21.15.92:0/97158999 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fef40003970 0x7fef40003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:53.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:53.134+0000 7fef598a0640 1 --2- 172.21.15.92:0/97158999 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fef540ae5a0 0x7fef400047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:53.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:53.134+0000 7fef5a8a2640 1 -- 172.21.15.92:0/97158999 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fef540ae5a0 msgr2=0x7fef400047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:53.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:53.134+0000 7fef5a8a2640 1 --2- 172.21.15.92:0/97158999 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fef540ae5a0 0x7fef400047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:53.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:53.134+0000 7fef5a8a2640 1 -- 172.21.15.92:0/97158999 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fef40003970 msgr2=0x7fef40003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:53.140 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:53.134+0000 7fef5a8a2640 1 --2- 172.21.15.92:0/97158999 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fef40003970 0x7fef40003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:53.140 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:53.134+0000 7fef5a8a2640 1 -- 172.21.15.92:0/97158999 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef40003520 con 0x7fef40004d00 2023-11-24T06:12:53.140 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:53.134+0000 7fef598a0640 1 --2- 172.21.15.92:0/97158999 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fef540ae5a0 0x7fef400047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:12:53.140 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:53.134+0000 7fef5a0a1640 1 --2- 172.21.15.92:0/97158999 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fef40003970 0x7fef40003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:12:53.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:53.135+0000 7fef5a8a2640 1 --2- 172.21.15.92:0/97158999 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fef40004d00 0x7fef40005180 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fef4c009ac0 tx=0x7fef400057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99b80be21d4ac032 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:53.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:53.135+0000 7fef5909f640 1 -- 172.21.15.92:0/97158999 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef4c02c710 con 0x7fef40004d00 2023-11-24T06:12:53.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:53.135+0000 7fef5909f640 1 -- 172.21.15.92:0/97158999 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef4c02c870 con 0x7fef40004d00 2023-11-24T06:12:53.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:53.135+0000 7fef5909f640 1 -- 172.21.15.92:0/97158999 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef4c02cc00 con 0x7fef40004d00 2023-11-24T06:12:53.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:53.136+0000 7fef5c305640 1 -- 172.21.15.92:0/97158999 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fef40004d00 msgr2=0x7fef40005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:53.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:53.136+0000 7fef5c305640 1 --2- 172.21.15.92:0/97158999 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fef40004d00 0x7fef40005180 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fef4c009ac0 tx=0x7fef400057f0 comp rx=0 tx=0).stop 2023-11-24T06:12:53.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:53.136+0000 7fef5a8a2640 1 -- 172.21.15.92:0/97158999 reap_dead start 2023-11-24T06:12:53.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:53.136+0000 7fef5c305640 1 -- 172.21.15.92:0/97158999 shutdown_connections 2023-11-24T06:12:53.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:53.136+0000 7fef5c305640 1 -- 172.21.15.92:0/97158999 >> 172.21.15.92:0/97158999 conn(0x7fef5406a800 msgr2=0x7fef5406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:53.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:53.136+0000 7fef5c305640 1 -- 172.21.15.92:0/97158999 shutdown_connections 2023-11-24T06:12:53.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:53.136+0000 7fef5c305640 1 -- 172.21.15.92:0/97158999 wait complete. 2023-11-24T06:12:53.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:53.138+0000 7fef5c305640 1 Processor -- start 2023-11-24T06:12:53.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:53.138+0000 7fef5c305640 1 -- start start 2023-11-24T06:12:53.144 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:53.139+0000 7fef5c305640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fef40004d00 0x7fef54160500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:53.144 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:53.139+0000 7fef5c305640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fef5415eb70 0x7fef5414ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:53.144 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:53.139+0000 7fef5c305640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fef5415b9e0 0x7fef5415be40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:53.144 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:53.139+0000 7fef5c305640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef54151b80 con 0x7fef5415eb70 2023-11-24T06:12:53.144 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:53.139+0000 7fef5c305640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef5415c380 con 0x7fef40004d00 2023-11-24T06:12:53.144 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:53.139+0000 7fef5c305640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef541504c0 con 0x7fef5415b9e0 2023-11-24T06:12:53.145 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:53.139+0000 7fef598a0640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fef5415eb70 0x7fef5414ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:53.145 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:53.139+0000 7fef598a0640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fef5415eb70 0x7fef5414ed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:55868/0 (socket says 172.21.15.92:55868) 2023-11-24T06:12:53.145 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:53.139+0000 7fef598a0640 1 -- 172.21.15.92:0/354479554 learned_addr learned my addr 172.21.15.92:0/354479554 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:12:53.145 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:53.140+0000 7fef5a8a2640 1 --2- 172.21.15.92:0/354479554 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fef5415b9e0 0x7fef5415be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:53.145 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:53.140+0000 7fef598a0640 1 -- 172.21.15.92:0/354479554 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fef5415b9e0 msgr2=0x7fef5415be40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:53.145 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:53.140+0000 7fef5a0a1640 1 --2- 172.21.15.92:0/354479554 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fef40004d00 0x7fef54160500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:53.145 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:53.140+0000 7fef598a0640 1 --2- 172.21.15.92:0/354479554 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fef5415b9e0 0x7fef5415be40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:53.145 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:53.140+0000 7fef598a0640 1 -- 172.21.15.92:0/354479554 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fef40004d00 msgr2=0x7fef54160500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:53.145 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:53.140+0000 7fef598a0640 1 --2- 172.21.15.92:0/354479554 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fef40004d00 0x7fef54160500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:53.145 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:53.140+0000 7fef598a0640 1 -- 172.21.15.92:0/354479554 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef5415f000 con 0x7fef5415eb70 2023-11-24T06:12:53.145 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:53.140+0000 7fef598a0640 1 --2- 172.21.15.92:0/354479554 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fef5415eb70 0x7fef5414ed90 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fef4400d820 tx=0x7fef4400dce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:53.146 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:53.140+0000 7fef4affd640 1 -- 172.21.15.92:0/354479554 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef440074e0 con 0x7fef5415eb70 2023-11-24T06:12:53.146 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:53.140+0000 7fef4affd640 1 -- 172.21.15.92:0/354479554 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef44007640 con 0x7fef5415eb70 2023-11-24T06:12:53.146 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:53.140+0000 7fef4affd640 1 -- 172.21.15.92:0/354479554 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef440049e0 con 0x7fef5415eb70 2023-11-24T06:12:53.146 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:53.140+0000 7fef5c305640 1 -- 172.21.15.92:0/354479554 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef5415f2e0 con 0x7fef5415eb70 2023-11-24T06:12:53.146 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:53.141+0000 7fef5c305640 1 -- 172.21.15.92:0/354479554 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef5415f8a0 con 0x7fef5415eb70 2023-11-24T06:12:53.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:53.142+0000 7fef5c305640 1 -- 172.21.15.92:0/354479554 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef540675d0 con 0x7fef5415eb70 2023-11-24T06:12:53.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:53.142+0000 7fef4affd640 1 -- 172.21.15.92:0/354479554 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7fef44004b40 con 0x7fef5415eb70 2023-11-24T06:12:53.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:53.143+0000 7fef4affd640 1 --2- 172.21.15.92:0/354479554 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fef3806aed0 0x7fef3806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:53.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:53.143+0000 7fef4affd640 1 -- 172.21.15.92:0/354479554 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fef440890f0 con 0x7fef5415eb70 2023-11-24T06:12:53.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:53.144+0000 7fef5a0a1640 1 --2- 172.21.15.92:0/354479554 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fef3806aed0 0x7fef3806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:53.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:53.144+0000 7fef5a0a1640 1 --2- 172.21.15.92:0/354479554 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fef3806aed0 0x7fef3806d390 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fef50007440 tx=0x7fef5000d040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:53.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:53.148+0000 7fef4affd640 1 -- 172.21.15.92:0/354479554 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fef440540a0 con 0x7fef5415eb70 2023-11-24T06:12:53.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:53.335+0000 7fef5c305640 1 -- 172.21.15.92:0/354479554 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fef540cb620 con 0x7fef5415eb70 2023-11-24T06:12:53.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:53.336+0000 7fef4affd640 1 -- 172.21.15.92:0/354479554 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v52) v1 ==== 100+0+11 (secure 0 0 0) 0x7fef440576c0 con 0x7fef5415eb70 2023-11-24T06:12:53.340 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 64 2023-11-24T06:12:53.342 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:53.339+0000 7fef5c305640 1 -- 172.21.15.92:0/354479554 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fef3806aed0 msgr2=0x7fef3806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:53.342 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:53.339+0000 7fef5c305640 1 --2- 172.21.15.92:0/354479554 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fef3806aed0 0x7fef3806d390 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fef50007440 tx=0x7fef5000d040 comp rx=0 tx=0).stop 2023-11-24T06:12:53.342 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:53.339+0000 7fef5c305640 1 -- 172.21.15.92:0/354479554 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fef5415eb70 msgr2=0x7fef5414ed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:53.342 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:53.339+0000 7fef5c305640 1 --2- 172.21.15.92:0/354479554 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fef5415eb70 0x7fef5414ed90 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fef4400d820 tx=0x7fef4400dce0 comp rx=0 tx=0).stop 2023-11-24T06:12:53.343 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:53.339+0000 7fef5c305640 1 -- 172.21.15.92:0/354479554 shutdown_connections 2023-11-24T06:12:53.343 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:53.339+0000 7fef5c305640 1 --2- 172.21.15.92:0/354479554 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fef3806aed0 0x7fef3806d390 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:53.343 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:53.339+0000 7fef5c305640 1 --2- 172.21.15.92:0/354479554 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fef5415b9e0 0x7fef5415be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:53.343 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:53.339+0000 7fef5c305640 1 --2- 172.21.15.92:0/354479554 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fef5415eb70 0x7fef5414ed90 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:53.343 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:53.339+0000 7fef5c305640 1 --2- 172.21.15.92:0/354479554 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fef40004d00 0x7fef54160500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:53.343 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:53.339+0000 7fef5c305640 1 -- 172.21.15.92:0/354479554 >> 172.21.15.92:0/354479554 conn(0x7fef5406a800 msgr2=0x7fef54160f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:53.343 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:53.339+0000 7fef5c305640 1 -- 172.21.15.92:0/354479554 shutdown_connections 2023-11-24T06:12:53.343 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:53.339+0000 7fef5c305640 1 -- 172.21.15.92:0/354479554 wait complete. 2023-11-24T06:12:53.362 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T06:12:53.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:53.560+0000 7f6186f7d640 1 Processor -- start 2023-11-24T06:12:53.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:53.560+0000 7f6186f7d640 1 -- start start 2023-11-24T06:12:53.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:53.561+0000 7f6186f7d640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f6180143ae0 0x7f61800cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:53.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:53.561+0000 7f6186f7d640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f618015e3b0 0x7f61800702c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:53.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:53.562+0000 7f6186f7d640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61800cfc30 con 0x7f61800cf7d0 2023-11-24T06:12:53.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:53.562+0000 7f6186f7d640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61800b1ee0 con 0x7f6180143ae0 2023-11-24T06:12:53.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:53.562+0000 7f6186f7d640 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61800b2780 con 0x7f618015e3b0 2023-11-24T06:12:53.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:53.562+0000 7f6184d19640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f6180143ae0 0x7f61800cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:53.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:53.562+0000 7f617ffff640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f61800cf7d0 0x7f618015dca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:53334/0 (socket says 172.21.15.92:53334) 2023-11-24T06:12:53.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:53.562+0000 7f6184d19640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f6180143ae0 0x7f61800cf290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:55876/0 (socket says 172.21.15.92:55876) 2023-11-24T06:12:53.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:53.562+0000 7f617ffff640 1 -- 172.21.15.92:0/239322057 learned_addr learned my addr 172.21.15.92:0/239322057 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-24T06:12:53.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:53.562+0000 7f618551a640 1 --2- 172.21.15.92:0/239322057 >> v2:172.21.15.118:3300/0 conn(0x7f618015e3b0 0x7f61800702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:53.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:53.562+0000 7f6184d19640 1 -- 172.21.15.92:0/239322057 >> v2:172.21.15.118:3300/0 conn(0x7f618015e3b0 msgr2=0x7f61800702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:53.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:53.562+0000 7f6184d19640 1 --2- 172.21.15.92:0/239322057 >> v2:172.21.15.118:3300/0 conn(0x7f618015e3b0 0x7f61800702c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:53.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:53.562+0000 7f6184d19640 1 -- 172.21.15.92:0/239322057 >> v1:172.21.15.92:6789/0 conn(0x7f61800cf7d0 legacy=0x7f618015dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:53.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:53.562+0000 7f6184d19640 1 -- 172.21.15.92:0/239322057 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f618015e990 con 0x7f6180143ae0 2023-11-24T06:12:53.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:53.562+0000 7f617f7fe640 1 -- 172.21.15.92:0/239322057 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2453719437 0 0) 0x7f61800cfc30 con 0x7f61800cf7d0 2023-11-24T06:12:53.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:53.563+0000 7f6184d19640 1 --2- 172.21.15.92:0/239322057 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6180143ae0 0x7f61800cf290 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f618006e4f0 tx=0x7f617002e0a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbc7d473497df5d1 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:53.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:53.564+0000 7f617f7fe640 1 -- 172.21.15.92:0/239322057 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f617002ee50 con 0x7f6180143ae0 2023-11-24T06:12:53.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:53.564+0000 7f617f7fe640 1 -- 172.21.15.92:0/239322057 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6170036440 con 0x7f6180143ae0 2023-11-24T06:12:53.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:53.564+0000 7f617f7fe640 1 -- 172.21.15.92:0/239322057 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61700367d0 con 0x7f6180143ae0 2023-11-24T06:12:53.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:53.564+0000 7f6186f7d640 1 -- 172.21.15.92:0/239322057 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6180143ae0 msgr2=0x7f61800cf290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:53.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:53.564+0000 7f6186f7d640 1 --2- 172.21.15.92:0/239322057 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6180143ae0 0x7f61800cf290 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f618006e4f0 tx=0x7f617002e0a0 comp rx=0 tx=0).stop 2023-11-24T06:12:53.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:53.564+0000 7f6186f7d640 1 -- 172.21.15.92:0/239322057 shutdown_connections 2023-11-24T06:12:53.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:53.564+0000 7f6186f7d640 1 --2- 172.21.15.92:0/239322057 >> v2:172.21.15.118:3300/0 conn(0x7f618015e3b0 0x7f61800702c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:53.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:53.564+0000 7f6186f7d640 1 --2- 172.21.15.92:0/239322057 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6180143ae0 0x7f61800cf290 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:53.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:53.564+0000 7f6186f7d640 1 -- 172.21.15.92:0/239322057 >> 172.21.15.92:0/239322057 conn(0x7f618006a800 msgr2=0x7f618006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:53.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:53.565+0000 7f6186f7d640 1 -- 172.21.15.92:0/239322057 shutdown_connections 2023-11-24T06:12:53.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:53.565+0000 7f6186f7d640 1 -- 172.21.15.92:0/239322057 wait complete. 2023-11-24T06:12:53.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:53.566+0000 7f6186f7d640 1 Processor -- start 2023-11-24T06:12:53.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:53.566+0000 7f6186f7d640 1 -- start start 2023-11-24T06:12:53.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:53.567+0000 7f6186f7d640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f61800cf7d0 0x7f618013cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:53.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:53.567+0000 7f6186f7d640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6180143ae0 0x7f61800702c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:53.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:53.567+0000 7f6186f7d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f618015e3b0 0x7f6180070800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:53.572 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:53.567+0000 7f6186f7d640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f618013d810 con 0x7f618015e3b0 2023-11-24T06:12:53.572 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:53.567+0000 7f6186f7d640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f618013daf0 con 0x7f61800cf7d0 2023-11-24T06:12:53.572 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:53.567+0000 7f6186f7d640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f618013ddd0 con 0x7f6180143ae0 2023-11-24T06:12:53.572 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:53.567+0000 7f618551a640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f618015e3b0 0x7f6180070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:53.572 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:53.568+0000 7f618551a640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f618015e3b0 0x7f6180070800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:55880/0 (socket says 172.21.15.92:55880) 2023-11-24T06:12:53.572 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:53.568+0000 7f618551a640 1 -- 172.21.15.92:0/3074788316 learned_addr learned my addr 172.21.15.92:0/3074788316 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:12:53.572 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:53.568+0000 7f618551a640 1 -- 172.21.15.92:0/3074788316 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6180143ae0 msgr2=0x7f61800702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:53.572 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:53.568+0000 7f618551a640 1 --2- 172.21.15.92:0/3074788316 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6180143ae0 0x7f61800702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:53.572 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:53.568+0000 7f6184d19640 1 --2- 172.21.15.92:0/3074788316 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f61800cf7d0 0x7f618013cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:53.572 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:53.568+0000 7f618551a640 1 -- 172.21.15.92:0/3074788316 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f61800cf7d0 msgr2=0x7f618013cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:53.572 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:53.568+0000 7f618551a640 1 --2- 172.21.15.92:0/3074788316 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f61800cf7d0 0x7f618013cfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:53.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:53.568+0000 7f618551a640 1 -- 172.21.15.92:0/3074788316 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f617000b6a0 con 0x7f618015e3b0 2023-11-24T06:12:53.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:53.568+0000 7f618551a640 1 --2- 172.21.15.92:0/3074788316 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f618015e3b0 0x7f6180070800 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f617800c550 tx=0x7f617800ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:53.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:53.568+0000 7f617d7fa640 1 -- 172.21.15.92:0/3074788316 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6178004220 con 0x7f618015e3b0 2023-11-24T06:12:53.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:53.568+0000 7f617d7fa640 1 -- 172.21.15.92:0/3074788316 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6178004380 con 0x7f618015e3b0 2023-11-24T06:12:53.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:53.568+0000 7f617d7fa640 1 -- 172.21.15.92:0/3074788316 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6178003830 con 0x7f618015e3b0 2023-11-24T06:12:53.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:53.568+0000 7f6186f7d640 1 -- 172.21.15.92:0/3074788316 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f618013e0b0 con 0x7f618015e3b0 2023-11-24T06:12:53.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:53.569+0000 7f6186f7d640 1 -- 172.21.15.92:0/3074788316 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f618013d500 con 0x7f618015e3b0 2023-11-24T06:12:53.574 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:53.570+0000 7f6186f7d640 1 -- 172.21.15.92:0/3074788316 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61800aed00 con 0x7f618015e3b0 2023-11-24T06:12:53.574 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:53.570+0000 7f617d7fa640 1 -- 172.21.15.92:0/3074788316 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f6178003990 con 0x7f618015e3b0 2023-11-24T06:12:53.574 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:53.571+0000 7f617d7fa640 1 --2- 172.21.15.92:0/3074788316 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f616006aed0 0x7f616006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:53.575 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:53.571+0000 7f617d7fa640 1 -- 172.21.15.92:0/3074788316 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f61780894b0 con 0x7f618015e3b0 2023-11-24T06:12:53.575 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:53.572+0000 7f6184d19640 1 --2- 172.21.15.92:0/3074788316 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f616006aed0 0x7f616006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:53.575 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:53.572+0000 7f6184d19640 1 --2- 172.21.15.92:0/3074788316 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f616006aed0 0x7f616006d390 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f6170038000 tx=0x7f6170032000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:53.579 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:53.576+0000 7f617d7fa640 1 -- 172.21.15.92:0/3074788316 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f61800aed00 con 0x7f618015e3b0 2023-11-24T06:12:53.794 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:53.791+0000 7f6186f7d640 1 -- 172.21.15.92:0/3074788316 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f61800aed00 con 0x7f618015e3b0 2023-11-24T06:12:53.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:53.792+0000 7f617d7fa640 1 -- 172.21.15.92:0/3074788316 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+4919 (secure 0 0 0) 0x7f61800aed00 con 0x7f618015e3b0 2023-11-24T06:12:53.795 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-24T06:12:53.796 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":32,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4778,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6835/1184353034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":1184353034},{"type":"v1","addr":"172.21.15.118:6835","nonce":1184353034}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":26},{"gid":4942,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6833/1256969666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":1256969666},{"type":"v1","addr":"172.21.15.118:6833","nonce":1256969666}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":26},{"gid":5706,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6836/3849408561","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":3849408561},{"type":"v1","addr":"172.21.15.92:6836","nonce":3849408561}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":31,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T06:12:30.605447+0000","modified":"2023-11-24T06:12:34.015371+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5709},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5709":{"gid":5709,"name":"c","rank":0,"incarnation":27,"state":"up:active","state_seq":3,"addr":"172.21.15.92:6837/1958912045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":1958912045},{"type":"v1","addr":"172.21.15.92:6837","nonce":1958912045}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-24T06:12:53.796 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 32 2023-11-24T06:12:53.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:53.794+0000 7f6186f7d640 1 -- 172.21.15.92:0/3074788316 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f616006aed0 msgr2=0x7f616006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:53.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:53.794+0000 7f6186f7d640 1 --2- 172.21.15.92:0/3074788316 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f616006aed0 0x7f616006d390 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f6170038000 tx=0x7f6170032000 comp rx=0 tx=0).stop 2023-11-24T06:12:53.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:53.794+0000 7f6186f7d640 1 -- 172.21.15.92:0/3074788316 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f618015e3b0 msgr2=0x7f6180070800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:53.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:53.794+0000 7f6186f7d640 1 --2- 172.21.15.92:0/3074788316 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f618015e3b0 0x7f6180070800 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f617800c550 tx=0x7f617800ca10 comp rx=0 tx=0).stop 2023-11-24T06:12:53.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:53.795+0000 7f6186f7d640 1 -- 172.21.15.92:0/3074788316 shutdown_connections 2023-11-24T06:12:53.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:53.795+0000 7f6186f7d640 1 --2- 172.21.15.92:0/3074788316 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f616006aed0 0x7f616006d390 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:53.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:53.795+0000 7f6186f7d640 1 --2- 172.21.15.92:0/3074788316 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f618015e3b0 0x7f6180070800 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:53.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:53.795+0000 7f6186f7d640 1 --2- 172.21.15.92:0/3074788316 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6180143ae0 0x7f61800702c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:53.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:53.795+0000 7f6186f7d640 1 --2- 172.21.15.92:0/3074788316 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f61800cf7d0 0x7f618013cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:53.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:53.795+0000 7f6186f7d640 1 -- 172.21.15.92:0/3074788316 >> 172.21.15.92:0/3074788316 conn(0x7f618006a800 msgr2=0x7f618013fa30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:53.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:53.795+0000 7f6186f7d640 1 -- 172.21.15.92:0/3074788316 shutdown_connections 2023-11-24T06:12:53.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:53.795+0000 7f6186f7d640 1 -- 172.21.15.92:0/3074788316 wait complete. 2023-11-24T06:12:53.816 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:12:54.024 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:54.021+0000 7f895ccf3640 1 Processor -- start 2023-11-24T06:12:54.024 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:54.021+0000 7f895ccf3640 1 -- start start 2023-11-24T06:12:54.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:54.023+0000 7f895ccf3640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f89580cf9a0 0x7f895815e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:54.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:54.023+0000 7f895ccf3640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8958070c40 con 0x7f89580b3ae0 2023-11-24T06:12:54.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:54.023+0000 7f895ccf3640 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f895806c470 con 0x7f89580ae5a0 2023-11-24T06:12:54.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:54.023+0000 7f895ccf3640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8958071450 con 0x7f89580cf9a0 2023-11-24T06:12:54.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:54.023+0000 7f8955d9b640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f89580b3ae0 0x7f89580cf290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:53342/0 (socket says 172.21.15.92:53342) 2023-11-24T06:12:54.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:54.023+0000 7f8955d9b640 1 -- 172.21.15.92:0/115287774 learned_addr learned my addr 172.21.15.92:0/115287774 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-24T06:12:54.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:54.024+0000 7f8956d9d640 1 --2- 172.21.15.92:0/115287774 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f89580cf9a0 0x7f895815e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:54.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:54.024+0000 7f895559a640 1 -- 172.21.15.92:0/115287774 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3393384656 0 0) 0x7f8958070c40 con 0x7f89580b3ae0 2023-11-24T06:12:54.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:54.024+0000 7f895559a640 1 -- 172.21.15.92:0/115287774 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8944003520 con 0x7f89580b3ae0 2023-11-24T06:12:54.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:54.024+0000 7f895559a640 1 -- 172.21.15.92:0/115287774 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3247878715 0 0) 0x7f8944003520 con 0x7f89580b3ae0 2023-11-24T06:12:54.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:54.024+0000 7f895559a640 1 -- 172.21.15.92:0/115287774 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f89580cf9a0 msgr2=0x7f895815e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:54.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:54.024+0000 7f895559a640 1 --2- 172.21.15.92:0/115287774 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f89580cf9a0 0x7f895815e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:54.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:54.024+0000 7f895559a640 1 -- 172.21.15.92:0/115287774 >> v1:172.21.15.118:6789/0 conn(0x7f89580ae5a0 legacy=0x7f895806cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:54.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:54.024+0000 7f895559a640 1 -- 172.21.15.92:0/115287774 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f895806c750 con 0x7f89580b3ae0 2023-11-24T06:12:54.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:54.024+0000 7f895559a640 1 -- 172.21.15.92:0/115287774 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2607661350 0 0) 0x7f8940001670 con 0x7f89580b3ae0 2023-11-24T06:12:54.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:54.024+0000 7f895559a640 1 -- 172.21.15.92:0/115287774 >> v1:172.21.15.92:6789/0 conn(0x7f89580b3ae0 legacy=0x7f89580cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:54.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:54.024+0000 7f895559a640 1 --2- 172.21.15.92:0/115287774 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f89440039a0 0x7f8944003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:54.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:54.024+0000 7f895559a640 1 --2- 172.21.15.92:0/115287774 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f89580cf9a0 0x7f89440047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:54.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:54.024+0000 7f895559a640 1 --2- 172.21.15.92:0/115287774 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8944004d00 0x7f8944005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:54.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:54.024+0000 7f895559a640 1 -- 172.21.15.92:0/115287774 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8958071450 con 0x7f8944004d00 2023-11-24T06:12:54.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:54.024+0000 7f895559a640 1 -- 172.21.15.92:0/115287774 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f894400bba0 con 0x7f89440039a0 2023-11-24T06:12:54.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:54.024+0000 7f895559a640 1 -- 172.21.15.92:0/115287774 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f894400bd60 con 0x7f89580cf9a0 2023-11-24T06:12:54.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:54.025+0000 7f8956d9d640 1 --2- 172.21.15.92:0/115287774 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8944004d00 0x7f8944005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:54.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:54.025+0000 7f8955d9b640 1 --2- 172.21.15.92:0/115287774 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f89580cf9a0 0x7f89440047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:54.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:54.025+0000 7f895659c640 1 --2- 172.21.15.92:0/115287774 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f89440039a0 0x7f8944003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:54.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:54.025+0000 7f8956d9d640 1 -- 172.21.15.92:0/115287774 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f89580cf9a0 msgr2=0x7f89440047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:54.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:54.025+0000 7f8956d9d640 1 --2- 172.21.15.92:0/115287774 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f89580cf9a0 0x7f89440047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:54.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:54.025+0000 7f8956d9d640 1 -- 172.21.15.92:0/115287774 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f89440039a0 msgr2=0x7f8944003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:54.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:54.025+0000 7f8956d9d640 1 --2- 172.21.15.92:0/115287774 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f89440039a0 0x7f8944003dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:54.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:54.025+0000 7f8956d9d640 1 -- 172.21.15.92:0/115287774 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8944003520 con 0x7f8944004d00 2023-11-24T06:12:54.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:54.026+0000 7f8956d9d640 1 --2- 172.21.15.92:0/115287774 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8944004d00 0x7f8944005180 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f895000ba30 tx=0x7f89440057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13b7fdb7da04d511 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:54.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:54.026+0000 7f895559a640 1 -- 172.21.15.92:0/115287774 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8950032640 con 0x7f8944004d00 2023-11-24T06:12:54.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:54.026+0000 7f895559a640 1 -- 172.21.15.92:0/115287774 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89500327a0 con 0x7f8944004d00 2023-11-24T06:12:54.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:54.026+0000 7f895559a640 1 -- 172.21.15.92:0/115287774 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8950032b30 con 0x7f8944004d00 2023-11-24T06:12:54.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:54.027+0000 7f895ccf3640 1 -- 172.21.15.92:0/115287774 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8944004d00 msgr2=0x7f8944005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:54.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:54.027+0000 7f895ccf3640 1 --2- 172.21.15.92:0/115287774 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8944004d00 0x7f8944005180 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f895000ba30 tx=0x7f89440057f0 comp rx=0 tx=0).stop 2023-11-24T06:12:54.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:54.027+0000 7f8956d9d640 1 -- 172.21.15.92:0/115287774 reap_dead start 2023-11-24T06:12:54.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:54.027+0000 7f895ccf3640 1 -- 172.21.15.92:0/115287774 shutdown_connections 2023-11-24T06:12:54.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:54.027+0000 7f895ccf3640 1 -- 172.21.15.92:0/115287774 >> 172.21.15.92:0/115287774 conn(0x7f895806a800 msgr2=0x7f895806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:54.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:54.027+0000 7f895ccf3640 1 -- 172.21.15.92:0/115287774 shutdown_connections 2023-11-24T06:12:54.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:54.027+0000 7f895ccf3640 1 -- 172.21.15.92:0/115287774 wait complete. 2023-11-24T06:12:54.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:54.029+0000 7f895ccf3640 1 Processor -- start 2023-11-24T06:12:54.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:54.029+0000 7f895ccf3640 1 -- start start 2023-11-24T06:12:54.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:54.030+0000 7f895ccf3640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8944004d00 0x7f89580cd280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:54.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:54.030+0000 7f895ccf3640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f89580cbd50 0x7f8958157f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:54.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:54.030+0000 7f895ccf3640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f895815da60 0x7f89580cda50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:54.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:54.030+0000 7f895ccf3640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89580cd7c0 con 0x7f8944004d00 2023-11-24T06:12:54.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:54.030+0000 7f895ccf3640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89580cc1e0 con 0x7f895815da60 2023-11-24T06:12:54.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:54.030+0000 7f895ccf3640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89580cc4c0 con 0x7f89580cbd50 2023-11-24T06:12:54.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:54.030+0000 7f895659c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8944004d00 0x7f89580cd280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:54.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:54.030+0000 7f895659c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8944004d00 0x7f89580cd280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:55900/0 (socket says 172.21.15.92:55900) 2023-11-24T06:12:54.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:54.030+0000 7f895659c640 1 -- 172.21.15.92:0/1708428440 learned_addr learned my addr 172.21.15.92:0/1708428440 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:12:54.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:54.031+0000 7f895659c640 1 -- 172.21.15.92:0/1708428440 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f89580cbd50 msgr2=0x7f8958157f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:54.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:54.031+0000 7f895659c640 1 --2- 172.21.15.92:0/1708428440 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f89580cbd50 0x7f8958157f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:54.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:54.031+0000 7f895659c640 1 -- 172.21.15.92:0/1708428440 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f895815da60 msgr2=0x7f89580cda50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:54.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:54.031+0000 7f895659c640 1 --2- 172.21.15.92:0/1708428440 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f895815da60 0x7f89580cda50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:54.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:54.031+0000 7f895659c640 1 -- 172.21.15.92:0/1708428440 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89580cc7a0 con 0x7f8944004d00 2023-11-24T06:12:54.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:54.031+0000 7f895659c640 1 --2- 172.21.15.92:0/1708428440 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8944004d00 0x7f89580cd280 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f894800d8e0 tx=0x7f894800dda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:54.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:54.031+0000 7f894effd640 1 -- 172.21.15.92:0/1708428440 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8948003da0 con 0x7f8944004d00 2023-11-24T06:12:54.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:54.031+0000 7f894effd640 1 -- 172.21.15.92:0/1708428440 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8948005070 con 0x7f8944004d00 2023-11-24T06:12:54.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:54.031+0000 7f894effd640 1 -- 172.21.15.92:0/1708428440 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8948005420 con 0x7f8944004d00 2023-11-24T06:12:54.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:54.031+0000 7f895ccf3640 1 -- 172.21.15.92:0/1708428440 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89580cca80 con 0x7f8944004d00 2023-11-24T06:12:54.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:54.031+0000 7f895ccf3640 1 -- 172.21.15.92:0/1708428440 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8958158490 con 0x7f8944004d00 2023-11-24T06:12:54.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:54.032+0000 7f895ccf3640 1 -- 172.21.15.92:0/1708428440 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89580675d0 con 0x7f8944004d00 2023-11-24T06:12:54.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:54.033+0000 7f894effd640 1 -- 172.21.15.92:0/1708428440 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f8948005580 con 0x7f8944004d00 2023-11-24T06:12:54.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:54.034+0000 7f894effd640 1 --2- 172.21.15.92:0/1708428440 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f893806aed0 0x7f893806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:54.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:54.034+0000 7f894effd640 1 -- 172.21.15.92:0/1708428440 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8948089a70 con 0x7f8944004d00 2023-11-24T06:12:54.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:54.034+0000 7f8955d9b640 1 --2- 172.21.15.92:0/1708428440 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f893806aed0 0x7f893806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:54.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:54.035+0000 7f8955d9b640 1 --2- 172.21.15.92:0/1708428440 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f893806aed0 0x7f893806d390 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f89580caf50 tx=0x7f8940001d50 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:54.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:54.038+0000 7f894effd640 1 -- 172.21.15.92:0/1708428440 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8948054a20 con 0x7f8944004d00 2023-11-24T06:12:54.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:54.235+0000 7f895ccf3640 1 -- 172.21.15.92:0/1708428440 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f895806b3f0 con 0x7f8944004d00 2023-11-24T06:12:54.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:54.239+0000 7f894effd640 1 -- 172.21.15.92:0/1708428440 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v52) v1 ==== 74+0+16517 (secure 0 0 0) 0x7f8948005830 con 0x7f8944004d00 2023-11-24T06:12:54.242 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-24T06:12:54.243 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":52,"fsid":"2d6dee5f-8aa3-470f-931a-50ade963b1cb","created":"2023-11-24T06:10:04.707204+0000","modified":"2023-11-24T06:12:32.152035+0000","last_up_change":"2023-11-24T06:10:17.598352+0000","last_in_change":"2023-11-24T06:10:09.040793+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:10:17.733098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:12:28.410305+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-11-24T06:12:29.625599+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ff37a9fb-135f-471f-b4b0-f7b5772006c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6803","nonce":20560714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6805","nonce":20560714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6809","nonce":20560714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6807","nonce":20560714}]},"public_addr":"172.21.15.92:6803/20560714","cluster_addr":"172.21.15.92:6805/20560714","heartbeat_back_addr":"172.21.15.92:6809/20560714","heartbeat_front_addr":"172.21.15.92:6807/20560714","state":["exists","up"]},{"osd":1,"uuid":"03cdd353-dcf1-4f38-94fe-a795137c06b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6819","nonce":2451239906}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6821","nonce":2451239906}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6825","nonce":2451239906}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6823","nonce":2451239906}]},"public_addr":"172.21.15.92:6819/2451239906","cluster_addr":"172.21.15.92:6821/2451239906","heartbeat_back_addr":"172.21.15.92:6825/2451239906","heartbeat_front_addr":"172.21.15.92:6823/2451239906","state":["exists","up"]},{"osd":2,"uuid":"a4a20d80-5d57-484b-becd-9aa8070dd3aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6811","nonce":1124979277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6813","nonce":1124979277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6817","nonce":1124979277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6815","nonce":1124979277}]},"public_addr":"172.21.15.92:6811/1124979277","cluster_addr":"172.21.15.92:6813/1124979277","heartbeat_back_addr":"172.21.15.92:6817/1124979277","heartbeat_front_addr":"172.21.15.92:6815/1124979277","state":["exists","up"]},{"osd":3,"uuid":"4648d7ce-9b29-4583-9425-4ba8fd2b1903","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6827","nonce":1734115782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6829","nonce":1734115782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6833","nonce":1734115782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6831","nonce":1734115782}]},"public_addr":"172.21.15.92:6827/1734115782","cluster_addr":"172.21.15.92:6829/1734115782","heartbeat_back_addr":"172.21.15.92:6833/1734115782","heartbeat_front_addr":"172.21.15.92:6831/1734115782","state":["exists","up"]},{"osd":4,"uuid":"3bc9d7eb-c6fa-4fcf-a3ec-ade604e05aac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6819","nonce":3425937229}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6822","nonce":3425937229}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6830","nonce":3425937229}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6826","nonce":3425937229}]},"public_addr":"172.21.15.118:6819/3425937229","cluster_addr":"172.21.15.118:6822/3425937229","heartbeat_back_addr":"172.21.15.118:6830/3425937229","heartbeat_front_addr":"172.21.15.118:6826/3425937229","state":["exists","up"]},{"osd":5,"uuid":"dce9ba6d-59ed-46d1-84fc-a84b13de10c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6801","nonce":827455984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6803","nonce":827455984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6807","nonce":827455984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6805","nonce":827455984}]},"public_addr":"172.21.15.118:6801/827455984","cluster_addr":"172.21.15.118:6803/827455984","heartbeat_back_addr":"172.21.15.118:6807/827455984","heartbeat_front_addr":"172.21.15.118:6805/827455984","state":["exists","up"]},{"osd":6,"uuid":"82aec504-ae58-4709-8d30-46f2b12081b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6809","nonce":2946986141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6811","nonce":2946986141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6815","nonce":2946986141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6813","nonce":2946986141}]},"public_addr":"172.21.15.118:6809/2946986141","cluster_addr":"172.21.15.118:6811/2946986141","heartbeat_back_addr":"172.21.15.118:6815/2946986141","heartbeat_front_addr":"172.21.15.118:6813/2946986141","state":["exists","up"]},{"osd":7,"uuid":"4ed4c624-c88f-4d35-bdb1-bf209b5dffb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6817","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6818","nonce":48122059}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6821","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6823","nonce":48122059}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6829","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6831","nonce":48122059}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6825","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6827","nonce":48122059}]},"public_addr":"172.21.15.118:6818/48122059","cluster_addr":"172.21.15.118:6823/48122059","heartbeat_back_addr":"172.21.15.118:6831/48122059","heartbeat_front_addr":"172.21.15.118:6827/48122059","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.536664+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.368324+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.271201+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.342971+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.621832+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.387110+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.632946+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.416032+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:12:54.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:54.241+0000 7f895ccf3640 1 -- 172.21.15.92:0/1708428440 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f893806aed0 msgr2=0x7f893806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:54.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:54.241+0000 7f895ccf3640 1 --2- 172.21.15.92:0/1708428440 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f893806aed0 0x7f893806d390 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f89580caf50 tx=0x7f8940001d50 comp rx=0 tx=0).stop 2023-11-24T06:12:54.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:54.241+0000 7f895ccf3640 1 -- 172.21.15.92:0/1708428440 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8944004d00 msgr2=0x7f89580cd280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:54.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:54.241+0000 7f895ccf3640 1 --2- 172.21.15.92:0/1708428440 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8944004d00 0x7f89580cd280 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f894800d8e0 tx=0x7f894800dda0 comp rx=0 tx=0).stop 2023-11-24T06:12:54.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:54.241+0000 7f895ccf3640 1 -- 172.21.15.92:0/1708428440 shutdown_connections 2023-11-24T06:12:54.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:54.241+0000 7f895ccf3640 1 --2- 172.21.15.92:0/1708428440 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f893806aed0 0x7f893806d390 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:54.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:54.241+0000 7f895ccf3640 1 --2- 172.21.15.92:0/1708428440 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f895815da60 0x7f89580cda50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:54.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:54.241+0000 7f895ccf3640 1 --2- 172.21.15.92:0/1708428440 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f89580cbd50 0x7f8958157f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:54.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:54.242+0000 7f895ccf3640 1 --2- 172.21.15.92:0/1708428440 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8944004d00 0x7f89580cd280 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:54.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:54.242+0000 7f895ccf3640 1 -- 172.21.15.92:0/1708428440 >> 172.21.15.92:0/1708428440 conn(0x7f895806a800 msgr2=0x7f89580cf260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:54.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:54.242+0000 7f895ccf3640 1 -- 172.21.15.92:0/1708428440 shutdown_connections 2023-11-24T06:12:54.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:54.242+0000 7f895ccf3640 1 -- 172.21.15.92:0/1708428440 wait complete. 2023-11-24T06:12:54.264 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-24T06:12:54.264 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T06:12:54.466 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:54.463+0000 7f0128448640 1 Processor -- start 2023-11-24T06:12:54.466 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:54.463+0000 7f0128448640 1 -- start start 2023-11-24T06:12:54.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:54.464+0000 7f0128448640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f01200cf9a0 0x7f012015dca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:54.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:54.464+0000 7f0128448640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01200b1ee0 con 0x7f012015e1e0 2023-11-24T06:12:54.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:54.464+0000 7f0128448640 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01200b2780 con 0x7f0120143ae0 2023-11-24T06:12:54.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:54.464+0000 7f0128448640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f012015e990 con 0x7f01200cf9a0 2023-11-24T06:12:54.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:54.465+0000 7f01259e3640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f01200cf9a0 0x7f012015dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:54.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:54.465+0000 7f01269e5640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f012015e1e0 0x7f01200702c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:53352/0 (socket says 172.21.15.92:53352) 2023-11-24T06:12:54.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:54.465+0000 7f01269e5640 1 -- 172.21.15.92:0/3850203047 learned_addr learned my addr 172.21.15.92:0/3850203047 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-24T06:12:54.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:54.465+0000 7f01259e3640 1 -- 172.21.15.92:0/3850203047 >> v1:172.21.15.118:6789/0 conn(0x7f0120143ae0 legacy=0x7f01200cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:54.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:54.465+0000 7f01259e3640 1 -- 172.21.15.92:0/3850203047 >> v1:172.21.15.92:6789/0 conn(0x7f012015e1e0 legacy=0x7f01200702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:54.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:54.465+0000 7f01259e3640 1 -- 172.21.15.92:0/3850203047 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01200b2130 con 0x7f01200cf9a0 2023-11-24T06:12:54.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:54.465+0000 7f01251e2640 1 -- 172.21.15.92:0/3850203047 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 978634382 0 0) 0x7f01200b1ee0 con 0x7f012015e1e0 2023-11-24T06:12:54.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:54.466+0000 7f01259e3640 1 --2- 172.21.15.92:0/3850203047 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f01200cf9a0 0x7f012015dca0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f01100097a0 tx=0x7f011002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d9548cb51688381 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:54.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:54.467+0000 7f01251e2640 1 -- 172.21.15.92:0/3850203047 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0110031830 con 0x7f01200cf9a0 2023-11-24T06:12:54.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:54.467+0000 7f01251e2640 1 -- 172.21.15.92:0/3850203047 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0110031990 con 0x7f01200cf9a0 2023-11-24T06:12:54.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:54.467+0000 7f01251e2640 1 -- 172.21.15.92:0/3850203047 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0110031d20 con 0x7f01200cf9a0 2023-11-24T06:12:54.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:54.467+0000 7f0128448640 1 -- 172.21.15.92:0/3850203047 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f01200cf9a0 msgr2=0x7f012015dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:54.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:54.467+0000 7f0128448640 1 --2- 172.21.15.92:0/3850203047 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f01200cf9a0 0x7f012015dca0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f01100097a0 tx=0x7f011002b8d0 comp rx=0 tx=0).stop 2023-11-24T06:12:54.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:54.467+0000 7f0128448640 1 -- 172.21.15.92:0/3850203047 shutdown_connections 2023-11-24T06:12:54.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:54.467+0000 7f0128448640 1 --2- 172.21.15.92:0/3850203047 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f01200cf9a0 0x7f012015dca0 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:54.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:54.467+0000 7f0128448640 1 -- 172.21.15.92:0/3850203047 >> 172.21.15.92:0/3850203047 conn(0x7f012006a800 msgr2=0x7f012006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:54.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:54.467+0000 7f0128448640 1 -- 172.21.15.92:0/3850203047 shutdown_connections 2023-11-24T06:12:54.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:54.467+0000 7f0128448640 1 -- 172.21.15.92:0/3850203047 wait complete. 2023-11-24T06:12:54.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:54.469+0000 7f0128448640 1 Processor -- start 2023-11-24T06:12:54.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:54.469+0000 7f0128448640 1 -- start start 2023-11-24T06:12:54.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:54.470+0000 7f0128448640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f01200cf9a0 0x7f01200adde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:54.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:54.470+0000 7f0128448640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0120143ae0 0x7f0120159750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:54.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:54.470+0000 7f0128448640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f012015e1e0 0x7f0120142910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:54.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:54.470+0000 7f0128448640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f012013b580 con 0x7f012015e1e0 2023-11-24T06:12:54.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:54.470+0000 7f0128448640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01200706a0 con 0x7f01200cf9a0 2023-11-24T06:12:54.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:54.470+0000 7f0128448640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0120070980 con 0x7f0120143ae0 2023-11-24T06:12:54.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:54.470+0000 7f01269e5640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f012015e1e0 0x7f0120142910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:54.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:54.470+0000 7f01269e5640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f012015e1e0 0x7f0120142910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:55916/0 (socket says 172.21.15.92:55916) 2023-11-24T06:12:54.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:54.470+0000 7f01269e5640 1 -- 172.21.15.92:0/1717753033 learned_addr learned my addr 172.21.15.92:0/1717753033 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:12:54.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:54.471+0000 7f01269e5640 1 -- 172.21.15.92:0/1717753033 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0120143ae0 msgr2=0x7f0120159750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:54.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:54.471+0000 7f01269e5640 1 --2- 172.21.15.92:0/1717753033 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0120143ae0 0x7f0120159750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:54.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:54.471+0000 7f01269e5640 1 -- 172.21.15.92:0/1717753033 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f01200cf9a0 msgr2=0x7f01200adde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:54.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:54.471+0000 7f01269e5640 1 --2- 172.21.15.92:0/1717753033 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f01200cf9a0 0x7f01200adde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:54.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:54.471+0000 7f01269e5640 1 -- 172.21.15.92:0/1717753033 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0110009400 con 0x7f012015e1e0 2023-11-24T06:12:54.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:54.471+0000 7f01269e5640 1 --2- 172.21.15.92:0/1717753033 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f012015e1e0 0x7f0120142910 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f011800c780 tx=0x7f011800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:54.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:54.471+0000 7f0116ffd640 1 -- 172.21.15.92:0/1717753033 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f011800ce90 con 0x7f012015e1e0 2023-11-24T06:12:54.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:54.471+0000 7f0116ffd640 1 -- 172.21.15.92:0/1717753033 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f011800a640 con 0x7f012015e1e0 2023-11-24T06:12:54.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:54.471+0000 7f0128448640 1 -- 172.21.15.92:0/1717753033 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0120070c60 con 0x7f012015e1e0 2023-11-24T06:12:54.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:54.471+0000 7f0116ffd640 1 -- 172.21.15.92:0/1717753033 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f011800a9d0 con 0x7f012015e1e0 2023-11-24T06:12:54.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:54.471+0000 7f0128448640 1 -- 172.21.15.92:0/1717753033 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f012013bc00 con 0x7f012015e1e0 2023-11-24T06:12:54.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:54.472+0000 7f0128448640 1 -- 172.21.15.92:0/1717753033 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01200675d0 con 0x7f012015e1e0 2023-11-24T06:12:54.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:54.473+0000 7f0116ffd640 1 -- 172.21.15.92:0/1717753033 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f011800ab30 con 0x7f012015e1e0 2023-11-24T06:12:54.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:54.474+0000 7f0116ffd640 1 --2- 172.21.15.92:0/1717753033 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f010406aed0 0x7f010406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:54.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:54.474+0000 7f0116ffd640 1 -- 172.21.15.92:0/1717753033 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0118088bd0 con 0x7f012015e1e0 2023-11-24T06:12:54.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:54.475+0000 7f01261e4640 1 --2- 172.21.15.92:0/1717753033 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f010406aed0 0x7f010406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:54.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:54.475+0000 7f01261e4640 1 --2- 172.21.15.92:0/1717753033 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f010406aed0 0x7f010406d390 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f011c003cc0 tx=0x7f011c0079b0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:54.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:54.479+0000 7f0116ffd640 1 -- 172.21.15.92:0/1717753033 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0118053c20 con 0x7f012015e1e0 2023-11-24T06:12:54.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:54.702+0000 7f0128448640 1 -- 172.21.15.92:0/1717753033 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f012013def0 con 0x7f012015e1e0 2023-11-24T06:12:54.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:54.703+0000 7f0116ffd640 1 -- 172.21.15.92:0/1717753033 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+4919 (secure 0 0 0) 0x7f0118018070 con 0x7f012015e1e0 2023-11-24T06:12:54.706 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-24T06:12:54.707 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":32,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4778,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6835/1184353034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":1184353034},{"type":"v1","addr":"172.21.15.118:6835","nonce":1184353034}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":26},{"gid":4942,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6833/1256969666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":1256969666},{"type":"v1","addr":"172.21.15.118:6833","nonce":1256969666}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":26},{"gid":5706,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6836/3849408561","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":3849408561},{"type":"v1","addr":"172.21.15.92:6836","nonce":3849408561}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":31,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T06:12:30.605447+0000","modified":"2023-11-24T06:12:34.015371+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5709},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5709":{"gid":5709,"name":"c","rank":0,"incarnation":27,"state":"up:active","state_seq":3,"addr":"172.21.15.92:6837/1958912045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":1958912045},{"type":"v1","addr":"172.21.15.92:6837","nonce":1958912045}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-24T06:12:54.707 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 32 2023-11-24T06:12:54.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:54.705+0000 7f0128448640 1 -- 172.21.15.92:0/1717753033 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f010406aed0 msgr2=0x7f010406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:54.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:54.705+0000 7f0128448640 1 --2- 172.21.15.92:0/1717753033 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f010406aed0 0x7f010406d390 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f011c003cc0 tx=0x7f011c0079b0 comp rx=0 tx=0).stop 2023-11-24T06:12:54.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:54.705+0000 7f0128448640 1 -- 172.21.15.92:0/1717753033 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f012015e1e0 msgr2=0x7f0120142910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:54.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:54.705+0000 7f0128448640 1 --2- 172.21.15.92:0/1717753033 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f012015e1e0 0x7f0120142910 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f011800c780 tx=0x7f011800cc40 comp rx=0 tx=0).stop 2023-11-24T06:12:54.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:54.706+0000 7f0128448640 1 -- 172.21.15.92:0/1717753033 shutdown_connections 2023-11-24T06:12:54.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:54.706+0000 7f0128448640 1 --2- 172.21.15.92:0/1717753033 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f010406aed0 0x7f010406d390 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:54.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:54.706+0000 7f0128448640 1 --2- 172.21.15.92:0/1717753033 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f012015e1e0 0x7f0120142910 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:54.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:54.706+0000 7f0128448640 1 --2- 172.21.15.92:0/1717753033 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0120143ae0 0x7f0120159750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:54.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:54.706+0000 7f0128448640 1 --2- 172.21.15.92:0/1717753033 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f01200cf9a0 0x7f01200adde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:54.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:54.706+0000 7f0128448640 1 -- 172.21.15.92:0/1717753033 >> 172.21.15.92:0/1717753033 conn(0x7f012006a800 msgr2=0x7f012015ac50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:54.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:54.706+0000 7f0128448640 1 -- 172.21.15.92:0/1717753033 shutdown_connections 2023-11-24T06:12:54.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:54.706+0000 7f0128448640 1 -- 172.21.15.92:0/1717753033 wait complete. 2023-11-24T06:12:54.726 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T06:12:54.923 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:54.920+0000 7f4224cf1640 1 Processor -- start 2023-11-24T06:12:54.923 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:54.920+0000 7f4224cf1640 1 -- start start 2023-11-24T06:12:54.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:54.921+0000 7f4224cf1640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f422013cfa0 0x7f422015afb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:54.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:54.921+0000 7f4224cf1640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f422013fcb0 0x7f42200b2040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:54.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:54.921+0000 7f4224cf1640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42200cdec0 con 0x7f42200cd530 2023-11-24T06:12:54.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:54.921+0000 7f4224cf1640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42200ce1a0 con 0x7f422013fcb0 2023-11-24T06:12:54.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:54.921+0000 7f4224cf1640 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42200ce480 con 0x7f422013cfa0 2023-11-24T06:12:54.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:54.922+0000 7f421ffff640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f422013fcb0 0x7f42200b2040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:54.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:54.922+0000 7f421ffff640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f422013fcb0 0x7f42200b2040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:55920/0 (socket says 172.21.15.92:55920) 2023-11-24T06:12:54.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:54.922+0000 7f421ffff640 1 -- 172.21.15.92:0/385268721 learned_addr learned my addr 172.21.15.92:0/385268721 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:12:54.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:54.922+0000 7f421f7fe640 1 --2- 172.21.15.92:0/385268721 >> v2:172.21.15.118:3300/0 conn(0x7f422013cfa0 0x7f422015afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:54.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:54.922+0000 7f421ffff640 1 -- 172.21.15.92:0/385268721 >> v2:172.21.15.118:3300/0 conn(0x7f422013cfa0 msgr2=0x7f422015afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:54.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:54.922+0000 7f421ffff640 1 --2- 172.21.15.92:0/385268721 >> v2:172.21.15.118:3300/0 conn(0x7f422013cfa0 0x7f422015afb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:54.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:54.922+0000 7f421ffff640 1 -- 172.21.15.92:0/385268721 >> v1:172.21.15.92:6789/0 conn(0x7f42200cd530 legacy=0x7f422013f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:54.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:54.922+0000 7f421ffff640 1 -- 172.21.15.92:0/385268721 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42200ce760 con 0x7f422013fcb0 2023-11-24T06:12:54.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:54.923+0000 7f421ffff640 1 --2- 172.21.15.92:0/385268721 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f422013fcb0 0x7f42200b2040 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f422006cf00 tx=0x7f421802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5af21179a8ebdf27 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:54.928 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:54.923+0000 7f421e7fc640 1 -- 172.21.15.92:0/385268721 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f421802ca80 con 0x7f422013fcb0 2023-11-24T06:12:54.928 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:54.924+0000 7f421e7fc640 1 -- 172.21.15.92:0/385268721 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f421802cbe0 con 0x7f422013fcb0 2023-11-24T06:12:54.928 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:54.924+0000 7f421e7fc640 1 -- 172.21.15.92:0/385268721 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42180343f0 con 0x7f422013fcb0 2023-11-24T06:12:54.928 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:54.924+0000 7f4224cf1640 1 -- 172.21.15.92:0/385268721 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f422013fcb0 msgr2=0x7f42200b2040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:54.928 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:54.924+0000 7f4224cf1640 1 --2- 172.21.15.92:0/385268721 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f422013fcb0 0x7f42200b2040 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f422006cf00 tx=0x7f421802bee0 comp rx=0 tx=0).stop 2023-11-24T06:12:54.928 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:54.924+0000 7f4224cf1640 1 -- 172.21.15.92:0/385268721 shutdown_connections 2023-11-24T06:12:54.928 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:54.924+0000 7f4224cf1640 1 --2- 172.21.15.92:0/385268721 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f422013fcb0 0x7f42200b2040 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:54.928 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:54.924+0000 7f4224cf1640 1 --2- 172.21.15.92:0/385268721 >> v2:172.21.15.118:3300/0 conn(0x7f422013cfa0 0x7f422015afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:54.928 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:54.924+0000 7f4224cf1640 1 -- 172.21.15.92:0/385268721 >> 172.21.15.92:0/385268721 conn(0x7f422006a800 msgr2=0x7f422006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:54.928 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:54.924+0000 7f4224cf1640 1 -- 172.21.15.92:0/385268721 shutdown_connections 2023-11-24T06:12:54.928 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:54.924+0000 7f4224cf1640 1 -- 172.21.15.92:0/385268721 wait complete. 2023-11-24T06:12:54.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:54.926+0000 7f4224cf1640 1 Processor -- start 2023-11-24T06:12:54.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:54.926+0000 7f4224cf1640 1 -- start start 2023-11-24T06:12:54.931 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:54.927+0000 7f4224cf1640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f42200cd530 0x7f42201538f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:54.931 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:54.927+0000 7f4224cf1640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f422013cfa0 0x7f4220151230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:54.931 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:54.927+0000 7f4224cf1640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f422013fcb0 0x7f42201f6630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:54.931 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:54.927+0000 7f4224cf1640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4220154c40 con 0x7f422013fcb0 2023-11-24T06:12:54.931 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:54.927+0000 7f4224cf1640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42201f7080 con 0x7f42200cd530 2023-11-24T06:12:54.931 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:54.927+0000 7f4224cf1640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42201f7360 con 0x7f422013cfa0 2023-11-24T06:12:54.932 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:54.927+0000 7f421ffff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f422013fcb0 0x7f42201f6630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:54.932 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:54.927+0000 7f421ffff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f422013fcb0 0x7f42201f6630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:55926/0 (socket says 172.21.15.92:55926) 2023-11-24T06:12:54.932 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:54.927+0000 7f421ffff640 1 -- 172.21.15.92:0/3187722274 learned_addr learned my addr 172.21.15.92:0/3187722274 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:12:54.932 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:54.928+0000 7f421f7fe640 1 --2- 172.21.15.92:0/3187722274 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f42200cd530 0x7f42201538f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:54.932 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:54.928+0000 7f421ffff640 1 -- 172.21.15.92:0/3187722274 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f422013cfa0 msgr2=0x7f4220151230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:54.932 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:54.928+0000 7f421effd640 1 --2- 172.21.15.92:0/3187722274 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f422013cfa0 0x7f4220151230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:54.932 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:54.928+0000 7f421ffff640 1 --2- 172.21.15.92:0/3187722274 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f422013cfa0 0x7f4220151230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:54.932 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:54.928+0000 7f421ffff640 1 -- 172.21.15.92:0/3187722274 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f42200cd530 msgr2=0x7f42201538f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:54.932 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:54.928+0000 7f421ffff640 1 --2- 172.21.15.92:0/3187722274 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f42200cd530 0x7f42201538f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:54.932 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:54.928+0000 7f421ffff640 1 -- 172.21.15.92:0/3187722274 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4218009b30 con 0x7f422013fcb0 2023-11-24T06:12:54.932 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:54.928+0000 7f421f7fe640 1 --2- 172.21.15.92:0/3187722274 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f42200cd530 0x7f42201538f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:12:54.932 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:54.928+0000 7f421ffff640 1 --2- 172.21.15.92:0/3187722274 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f422013fcb0 0x7f42201f6630 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f4218036040 tx=0x7f4218004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:54.933 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:54.928+0000 7f4207fff640 1 -- 172.21.15.92:0/3187722274 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4218009510 con 0x7f422013fcb0 2023-11-24T06:12:54.933 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:54.928+0000 7f4207fff640 1 -- 172.21.15.92:0/3187722274 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f421803e450 con 0x7f422013fcb0 2023-11-24T06:12:54.934 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:54.929+0000 7f4224cf1640 1 -- 172.21.15.92:0/3187722274 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4220154f00 con 0x7f422013fcb0 2023-11-24T06:12:54.934 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:54.929+0000 7f4207fff640 1 -- 172.21.15.92:0/3187722274 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f421803e800 con 0x7f422013fcb0 2023-11-24T06:12:54.934 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:54.929+0000 7f4224cf1640 1 -- 172.21.15.92:0/3187722274 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4220153e30 con 0x7f422013fcb0 2023-11-24T06:12:54.934 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:54.930+0000 7f4224cf1640 1 -- 172.21.15.92:0/3187722274 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4220154aa0 con 0x7f422013fcb0 2023-11-24T06:12:54.934 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:54.930+0000 7f4207fff640 1 -- 172.21.15.92:0/3187722274 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f421802cd50 con 0x7f422013fcb0 2023-11-24T06:12:54.934 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:54.931+0000 7f4207fff640 1 --2- 172.21.15.92:0/3187722274 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f420c06aed0 0x7f420c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:54.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:54.931+0000 7f4207fff640 1 -- 172.21.15.92:0/3187722274 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f42180ac280 con 0x7f422013fcb0 2023-11-24T06:12:54.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:54.932+0000 7f421f7fe640 1 --2- 172.21.15.92:0/3187722274 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f420c06aed0 0x7f420c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:54.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:54.932+0000 7f421f7fe640 1 --2- 172.21.15.92:0/3187722274 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f420c06aed0 0x7f420c06d390 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f4210006fd0 tx=0x7f4210008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:54.939 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:54.936+0000 7f4207fff640 1 -- 172.21.15.92:0/3187722274 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4220154aa0 con 0x7f422013fcb0 2023-11-24T06:12:55.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:55.148+0000 7f4224cf1640 1 -- 172.21.15.92:0/3187722274 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4220154aa0 con 0x7f422013fcb0 2023-11-24T06:12:55.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:55.148+0000 7f4207fff640 1 -- 172.21.15.92:0/3187722274 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+4919 (secure 0 0 0) 0x7f4220154aa0 con 0x7f422013fcb0 2023-11-24T06:12:55.152 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-24T06:12:55.152 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":32,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4778,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6835/1184353034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":1184353034},{"type":"v1","addr":"172.21.15.118:6835","nonce":1184353034}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":26},{"gid":4942,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6833/1256969666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":1256969666},{"type":"v1","addr":"172.21.15.118:6833","nonce":1256969666}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":26},{"gid":5706,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6836/3849408561","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":3849408561},{"type":"v1","addr":"172.21.15.92:6836","nonce":3849408561}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":31,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T06:12:30.605447+0000","modified":"2023-11-24T06:12:34.015371+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5709},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5709":{"gid":5709,"name":"c","rank":0,"incarnation":27,"state":"up:active","state_seq":3,"addr":"172.21.15.92:6837/1958912045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":1958912045},{"type":"v1","addr":"172.21.15.92:6837","nonce":1958912045}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-24T06:12:55.152 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 32 2023-11-24T06:12:55.155 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:55.151+0000 7f4224cf1640 1 -- 172.21.15.92:0/3187722274 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f420c06aed0 msgr2=0x7f420c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:55.155 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:55.151+0000 7f4224cf1640 1 --2- 172.21.15.92:0/3187722274 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f420c06aed0 0x7f420c06d390 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f4210006fd0 tx=0x7f4210008040 comp rx=0 tx=0).stop 2023-11-24T06:12:55.155 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:55.151+0000 7f4224cf1640 1 -- 172.21.15.92:0/3187722274 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f422013fcb0 msgr2=0x7f42201f6630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:55.155 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:55.151+0000 7f4224cf1640 1 --2- 172.21.15.92:0/3187722274 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f422013fcb0 0x7f42201f6630 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f4218036040 tx=0x7f4218004a40 comp rx=0 tx=0).stop 2023-11-24T06:12:55.155 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:55.151+0000 7f4224cf1640 1 -- 172.21.15.92:0/3187722274 shutdown_connections 2023-11-24T06:12:55.155 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:55.151+0000 7f4224cf1640 1 --2- 172.21.15.92:0/3187722274 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f420c06aed0 0x7f420c06d390 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:55.155 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:55.151+0000 7f4224cf1640 1 --2- 172.21.15.92:0/3187722274 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f422013fcb0 0x7f42201f6630 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:55.155 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:55.151+0000 7f4224cf1640 1 --2- 172.21.15.92:0/3187722274 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f422013cfa0 0x7f4220151230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:55.155 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:55.151+0000 7f4224cf1640 1 --2- 172.21.15.92:0/3187722274 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f42200cd530 0x7f42201538f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:55.155 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:55.151+0000 7f4224cf1640 1 -- 172.21.15.92:0/3187722274 >> 172.21.15.92:0/3187722274 conn(0x7f422006a800 msgr2=0x7f42200af3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:55.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:55.152+0000 7f4224cf1640 1 -- 172.21.15.92:0/3187722274 shutdown_connections 2023-11-24T06:12:55.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:55.152+0000 7f4224cf1640 1 -- 172.21.15.92:0/3187722274 wait complete. 2023-11-24T06:12:55.172 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:12:55.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:55.378+0000 7f26b289f640 1 Processor -- start 2023-11-24T06:12:55.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:55.378+0000 7f26b289f640 1 -- start start 2023-11-24T06:12:55.382 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:55.379+0000 7f26b289f640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f26ac0cc990 0x7f26ac06d870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:55.383 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:55.379+0000 7f26b289f640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26ac070e50 con 0x7f26ac143c40 2023-11-24T06:12:55.383 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:55.379+0000 7f26b289f640 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26ac144020 con 0x7f26ac0cc500 2023-11-24T06:12:55.383 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:55.379+0000 7f26b289f640 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26ac0723c0 con 0x7f26ac0cc990 2023-11-24T06:12:55.383 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:55.380+0000 7f26b189d640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f26ac143c40 0x7f26ac071290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:53372/0 (socket says 172.21.15.92:53372) 2023-11-24T06:12:55.383 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:55.380+0000 7f26b189d640 1 -- 172.21.15.92:0/3895000343 learned_addr learned my addr 172.21.15.92:0/3895000343 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-24T06:12:55.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:55.380+0000 7f26b209e640 1 --2- 172.21.15.92:0/3895000343 >> v2:172.21.15.118:3300/0 conn(0x7f26ac0cc990 0x7f26ac06d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:55.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:55.380+0000 7f26b089b640 1 -- 172.21.15.92:0/3895000343 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2022879219 0 0) 0x7f26ac070e50 con 0x7f26ac143c40 2023-11-24T06:12:55.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:55.380+0000 7f26b089b640 1 -- 172.21.15.92:0/3895000343 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2698003520 con 0x7f26ac143c40 2023-11-24T06:12:55.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:55.381+0000 7f26b089b640 1 -- 172.21.15.92:0/3895000343 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 876982193 0 0) 0x7f2698003520 con 0x7f26ac143c40 2023-11-24T06:12:55.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:55.381+0000 7f26b089b640 1 -- 172.21.15.92:0/3895000343 >> v2:172.21.15.118:3300/0 conn(0x7f26ac0cc990 msgr2=0x7f26ac06d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:55.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:55.381+0000 7f26b089b640 1 --2- 172.21.15.92:0/3895000343 >> v2:172.21.15.118:3300/0 conn(0x7f26ac0cc990 0x7f26ac06d870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:55.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:55.381+0000 7f26b089b640 1 -- 172.21.15.92:0/3895000343 >> v1:172.21.15.118:6789/0 conn(0x7f26ac0cc500 legacy=0x7f26ac06d160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:55.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:55.381+0000 7f26b089b640 1 -- 172.21.15.92:0/3895000343 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26ac071e00 con 0x7f26ac143c40 2023-11-24T06:12:55.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:55.381+0000 7f26b089b640 1 -- 172.21.15.92:0/3895000343 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2607661350 0 0) 0x7f26a8001670 con 0x7f26ac143c40 2023-11-24T06:12:55.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:55.381+0000 7f26b089b640 1 -- 172.21.15.92:0/3895000343 >> v1:172.21.15.92:6789/0 conn(0x7f26ac143c40 legacy=0x7f26ac071290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:55.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:55.381+0000 7f26b089b640 1 --2- 172.21.15.92:0/3895000343 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f26980039c0 0x7f2698003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:55.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:55.381+0000 7f26b089b640 1 --2- 172.21.15.92:0/3895000343 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f26980047c0 0x7f2698004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:55.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:55.381+0000 7f26b089b640 1 --2- 172.21.15.92:0/3895000343 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f26980051d0 0x7f269800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:55.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:55.381+0000 7f26b089b640 1 -- 172.21.15.92:0/3895000343 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26ac0723c0 con 0x7f26980051d0 2023-11-24T06:12:55.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:55.381+0000 7f26b089b640 1 -- 172.21.15.92:0/3895000343 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f269800c120 con 0x7f26980047c0 2023-11-24T06:12:55.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:55.381+0000 7f26b089b640 1 -- 172.21.15.92:0/3895000343 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f269800c2e0 con 0x7f26980039c0 2023-11-24T06:12:55.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:55.381+0000 7f26b209e640 1 --2- 172.21.15.92:0/3895000343 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f26980051d0 0x7f269800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:55.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:55.381+0000 7f26b189d640 1 --2- 172.21.15.92:0/3895000343 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f26980039c0 0x7f2698003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:55.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:55.382+0000 7f26b209e640 1 -- 172.21.15.92:0/3895000343 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f26980039c0 msgr2=0x7f2698003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:55.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:55.382+0000 7f26b209e640 1 --2- 172.21.15.92:0/3895000343 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f26980039c0 0x7f2698003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:55.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:55.382+0000 7f26b209e640 1 -- 172.21.15.92:0/3895000343 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f26980047c0 msgr2=0x7f2698004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:55.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:55.382+0000 7f26b109c640 1 --2- 172.21.15.92:0/3895000343 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f26980047c0 0x7f2698004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:55.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:55.382+0000 7f26b209e640 1 --2- 172.21.15.92:0/3895000343 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f26980047c0 0x7f2698004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:55.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:55.382+0000 7f26b209e640 1 -- 172.21.15.92:0/3895000343 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2698003520 con 0x7f26980051d0 2023-11-24T06:12:55.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:55.382+0000 7f26b189d640 1 --2- 172.21.15.92:0/3895000343 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f26980039c0 0x7f2698003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:12:55.388 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:55.383+0000 7f26b209e640 1 -- 172.21.15.92:0/3895000343 reap_dead start 2023-11-24T06:12:55.389 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:55.383+0000 7f26b209e640 1 --2- 172.21.15.92:0/3895000343 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f26980051d0 0x7f269800b980 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f26a4009f90 tx=0x7f26980057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d7f214a74cb1bb8 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:55.389 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:55.383+0000 7f26b089b640 1 -- 172.21.15.92:0/3895000343 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26a402cc20 con 0x7f26980051d0 2023-11-24T06:12:55.389 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:55.383+0000 7f26b089b640 1 -- 172.21.15.92:0/3895000343 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26a400f040 con 0x7f26980051d0 2023-11-24T06:12:55.389 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:55.383+0000 7f26b089b640 1 -- 172.21.15.92:0/3895000343 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26a4003990 con 0x7f26980051d0 2023-11-24T06:12:55.389 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:55.383+0000 7f26b289f640 1 -- 172.21.15.92:0/3895000343 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f26980051d0 msgr2=0x7f269800b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:55.389 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:55.383+0000 7f26b289f640 1 --2- 172.21.15.92:0/3895000343 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f26980051d0 0x7f269800b980 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f26a4009f90 tx=0x7f26980057f0 comp rx=0 tx=0).stop 2023-11-24T06:12:55.389 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:55.384+0000 7f26b289f640 1 -- 172.21.15.92:0/3895000343 shutdown_connections 2023-11-24T06:12:55.389 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:55.384+0000 7f26b289f640 1 --2- 172.21.15.92:0/3895000343 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f26980051d0 0x7f269800b980 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:55.389 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:55.384+0000 7f26b289f640 1 -- 172.21.15.92:0/3895000343 >> 172.21.15.92:0/3895000343 conn(0x7f26ac06a800 msgr2=0x7f26ac06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:55.389 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:55.384+0000 7f26b289f640 1 -- 172.21.15.92:0/3895000343 shutdown_connections 2023-11-24T06:12:55.389 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:55.384+0000 7f26b289f640 1 -- 172.21.15.92:0/3895000343 wait complete. 2023-11-24T06:12:55.389 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:55.385+0000 7f26b289f640 1 Processor -- start 2023-11-24T06:12:55.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:55.386+0000 7f26b289f640 1 -- start start 2023-11-24T06:12:55.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:55.386+0000 7f26b289f640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f26980051d0 0x7f26ac0a8e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:55.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:55.386+0000 7f26b289f640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f26ac0a84c0 0x7f26ac15a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:55.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:55.386+0000 7f26b289f640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f26ac1f3010 0x7f26ac1f3490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:55.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:55.386+0000 7f26b289f640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26ac0a8940 con 0x7f26ac1f3010 2023-11-24T06:12:55.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:55.386+0000 7f26b289f640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26ac13ed50 con 0x7f26980051d0 2023-11-24T06:12:55.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:55.386+0000 7f26b289f640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26ac1e5bc0 con 0x7f26ac0a84c0 2023-11-24T06:12:55.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:55.387+0000 7f26b209e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f26ac1f3010 0x7f26ac1f3490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:55.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:55.387+0000 7f26b209e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f26ac1f3010 0x7f26ac1f3490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:55942/0 (socket says 172.21.15.92:55942) 2023-11-24T06:12:55.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:55.387+0000 7f26b209e640 1 -- 172.21.15.92:0/3840130487 learned_addr learned my addr 172.21.15.92:0/3840130487 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:12:55.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:55.387+0000 7f26b109c640 1 --2- 172.21.15.92:0/3840130487 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f26ac0a84c0 0x7f26ac15a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:55.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:55.387+0000 7f26b189d640 1 --2- 172.21.15.92:0/3840130487 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f26980051d0 0x7f26ac0a8e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:55.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:55.387+0000 7f26b209e640 1 -- 172.21.15.92:0/3840130487 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f26ac0a84c0 msgr2=0x7f26ac15a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:55.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:55.387+0000 7f26b209e640 1 --2- 172.21.15.92:0/3840130487 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f26ac0a84c0 0x7f26ac15a7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:55.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:55.387+0000 7f26b209e640 1 -- 172.21.15.92:0/3840130487 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f26980051d0 msgr2=0x7f26ac0a8e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:55.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:55.387+0000 7f26b209e640 1 --2- 172.21.15.92:0/3840130487 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f26980051d0 0x7f26ac0a8e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:55.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:55.387+0000 7f26b209e640 1 -- 172.21.15.92:0/3840130487 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26ac15ad10 con 0x7f26ac1f3010 2023-11-24T06:12:55.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:55.387+0000 7f26b209e640 1 --2- 172.21.15.92:0/3840130487 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f26ac1f3010 0x7f26ac1f3490 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f26a402c690 tx=0x7f26a4009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:55.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:55.388+0000 7f26a27fc640 1 -- 172.21.15.92:0/3840130487 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26a402cea0 con 0x7f26ac1f3010 2023-11-24T06:12:55.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:55.388+0000 7f26a27fc640 1 -- 172.21.15.92:0/3840130487 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26a4004230 con 0x7f26ac1f3010 2023-11-24T06:12:55.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:55.388+0000 7f26b289f640 1 -- 172.21.15.92:0/3840130487 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26ac1f39d0 con 0x7f26ac1f3010 2023-11-24T06:12:55.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:55.388+0000 7f26a27fc640 1 -- 172.21.15.92:0/3840130487 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26a40045e0 con 0x7f26ac1f3010 2023-11-24T06:12:55.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:55.388+0000 7f26b289f640 1 -- 172.21.15.92:0/3840130487 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26ac1e5570 con 0x7f26ac1f3010 2023-11-24T06:12:55.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:55.389+0000 7f26b289f640 1 -- 172.21.15.92:0/3840130487 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26ac067450 con 0x7f26ac1f3010 2023-11-24T06:12:55.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:55.390+0000 7f26a27fc640 1 -- 172.21.15.92:0/3840130487 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f26a4004740 con 0x7f26ac1f3010 2023-11-24T06:12:55.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:55.391+0000 7f26a27fc640 1 --2- 172.21.15.92:0/3840130487 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f269006aed0 0x7f269006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:55.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:55.391+0000 7f26a27fc640 1 -- 172.21.15.92:0/3840130487 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f26a40abff0 con 0x7f26ac1f3010 2023-11-24T06:12:55.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:55.391+0000 7f26b189d640 1 --2- 172.21.15.92:0/3840130487 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f269006aed0 0x7f269006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:55.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:55.392+0000 7f26b189d640 1 --2- 172.21.15.92:0/3840130487 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f269006aed0 0x7f269006d390 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f26a8000c00 tx=0x7f26a8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:55.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:55.395+0000 7f26a27fc640 1 -- 172.21.15.92:0/3840130487 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f26a40dee90 con 0x7f26ac1f3010 2023-11-24T06:12:55.575 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:55.572+0000 7f26b289f640 1 -- 172.21.15.92:0/3840130487 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f26ac0aebf0 con 0x7f26ac1f3010 2023-11-24T06:12:55.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:55.574+0000 7f26a27fc640 1 -- 172.21.15.92:0/3840130487 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v52) v1 ==== 74+0+16517 (secure 0 0 0) 0x7f26a40771a0 con 0x7f26ac1f3010 2023-11-24T06:12:55.577 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-24T06:12:55.577 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":52,"fsid":"2d6dee5f-8aa3-470f-931a-50ade963b1cb","created":"2023-11-24T06:10:04.707204+0000","modified":"2023-11-24T06:12:32.152035+0000","last_up_change":"2023-11-24T06:10:17.598352+0000","last_in_change":"2023-11-24T06:10:09.040793+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:10:17.733098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:12:28.410305+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-11-24T06:12:29.625599+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ff37a9fb-135f-471f-b4b0-f7b5772006c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6803","nonce":20560714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6805","nonce":20560714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6809","nonce":20560714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6807","nonce":20560714}]},"public_addr":"172.21.15.92:6803/20560714","cluster_addr":"172.21.15.92:6805/20560714","heartbeat_back_addr":"172.21.15.92:6809/20560714","heartbeat_front_addr":"172.21.15.92:6807/20560714","state":["exists","up"]},{"osd":1,"uuid":"03cdd353-dcf1-4f38-94fe-a795137c06b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6819","nonce":2451239906}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6821","nonce":2451239906}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6825","nonce":2451239906}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6823","nonce":2451239906}]},"public_addr":"172.21.15.92:6819/2451239906","cluster_addr":"172.21.15.92:6821/2451239906","heartbeat_back_addr":"172.21.15.92:6825/2451239906","heartbeat_front_addr":"172.21.15.92:6823/2451239906","state":["exists","up"]},{"osd":2,"uuid":"a4a20d80-5d57-484b-becd-9aa8070dd3aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6811","nonce":1124979277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6813","nonce":1124979277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6817","nonce":1124979277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6815","nonce":1124979277}]},"public_addr":"172.21.15.92:6811/1124979277","cluster_addr":"172.21.15.92:6813/1124979277","heartbeat_back_addr":"172.21.15.92:6817/1124979277","heartbeat_front_addr":"172.21.15.92:6815/1124979277","state":["exists","up"]},{"osd":3,"uuid":"4648d7ce-9b29-4583-9425-4ba8fd2b1903","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6827","nonce":1734115782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6829","nonce":1734115782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6833","nonce":1734115782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6831","nonce":1734115782}]},"public_addr":"172.21.15.92:6827/1734115782","cluster_addr":"172.21.15.92:6829/1734115782","heartbeat_back_addr":"172.21.15.92:6833/1734115782","heartbeat_front_addr":"172.21.15.92:6831/1734115782","state":["exists","up"]},{"osd":4,"uuid":"3bc9d7eb-c6fa-4fcf-a3ec-ade604e05aac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6819","nonce":3425937229}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6822","nonce":3425937229}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6830","nonce":3425937229}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6826","nonce":3425937229}]},"public_addr":"172.21.15.118:6819/3425937229","cluster_addr":"172.21.15.118:6822/3425937229","heartbeat_back_addr":"172.21.15.118:6830/3425937229","heartbeat_front_addr":"172.21.15.118:6826/3425937229","state":["exists","up"]},{"osd":5,"uuid":"dce9ba6d-59ed-46d1-84fc-a84b13de10c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6801","nonce":827455984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6803","nonce":827455984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6807","nonce":827455984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6805","nonce":827455984}]},"public_addr":"172.21.15.118:6801/827455984","cluster_addr":"172.21.15.118:6803/827455984","heartbeat_back_addr":"172.21.15.118:6807/827455984","heartbeat_front_addr":"172.21.15.118:6805/827455984","state":["exists","up"]},{"osd":6,"uuid":"82aec504-ae58-4709-8d30-46f2b12081b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6809","nonce":2946986141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6811","nonce":2946986141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6815","nonce":2946986141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6813","nonce":2946986141}]},"public_addr":"172.21.15.118:6809/2946986141","cluster_addr":"172.21.15.118:6811/2946986141","heartbeat_back_addr":"172.21.15.118:6815/2946986141","heartbeat_front_addr":"172.21.15.118:6813/2946986141","state":["exists","up"]},{"osd":7,"uuid":"4ed4c624-c88f-4d35-bdb1-bf209b5dffb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6817","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6818","nonce":48122059}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6821","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6823","nonce":48122059}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6829","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6831","nonce":48122059}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6825","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6827","nonce":48122059}]},"public_addr":"172.21.15.118:6818/48122059","cluster_addr":"172.21.15.118:6823/48122059","heartbeat_back_addr":"172.21.15.118:6831/48122059","heartbeat_front_addr":"172.21.15.118:6827/48122059","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.536664+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.368324+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.271201+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.342971+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.621832+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.387110+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.632946+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.416032+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:12:55.579 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:55.576+0000 7f26b289f640 1 -- 172.21.15.92:0/3840130487 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f269006aed0 msgr2=0x7f269006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:55.579 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:55.576+0000 7f26b289f640 1 --2- 172.21.15.92:0/3840130487 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f269006aed0 0x7f269006d390 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f26a8000c00 tx=0x7f26a8009040 comp rx=0 tx=0).stop 2023-11-24T06:12:55.579 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:55.576+0000 7f26b289f640 1 -- 172.21.15.92:0/3840130487 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f26ac1f3010 msgr2=0x7f26ac1f3490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:55.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:55.576+0000 7f26b289f640 1 --2- 172.21.15.92:0/3840130487 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f26ac1f3010 0x7f26ac1f3490 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f26a402c690 tx=0x7f26a4009750 comp rx=0 tx=0).stop 2023-11-24T06:12:55.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:55.576+0000 7f26b289f640 1 -- 172.21.15.92:0/3840130487 shutdown_connections 2023-11-24T06:12:55.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:55.576+0000 7f26b289f640 1 --2- 172.21.15.92:0/3840130487 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f269006aed0 0x7f269006d390 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:55.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:55.576+0000 7f26b289f640 1 --2- 172.21.15.92:0/3840130487 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f26ac1f3010 0x7f26ac1f3490 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:55.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:55.576+0000 7f26b289f640 1 --2- 172.21.15.92:0/3840130487 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f26ac0a84c0 0x7f26ac15a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:55.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:55.576+0000 7f26b289f640 1 --2- 172.21.15.92:0/3840130487 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f26980051d0 0x7f26ac0a8e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:55.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:55.576+0000 7f26b289f640 1 -- 172.21.15.92:0/3840130487 >> 172.21.15.92:0/3840130487 conn(0x7f26ac06a800 msgr2=0x7f26ac140870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:55.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:55.576+0000 7f26b289f640 1 -- 172.21.15.92:0/3840130487 shutdown_connections 2023-11-24T06:12:55.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:55.576+0000 7f26b289f640 1 -- 172.21.15.92:0/3840130487 wait complete. 2023-11-24T06:12:55.597 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-24T06:12:55.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:55.801+0000 7f601ca51640 1 Processor -- start 2023-11-24T06:12:55.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:55.801+0000 7f601ca51640 1 -- start start 2023-11-24T06:12:55.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:55.802+0000 7f601ca51640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f60180cf9a0 0x7f601815dca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:55.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:55.802+0000 7f601ca51640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f601815e1e0 0x7f60180702c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:55.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:55.802+0000 7f601ca51640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60180b1ee0 con 0x7f6018143ae0 2023-11-24T06:12:55.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:55.802+0000 7f601ca51640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60180b2780 con 0x7f60180cf9a0 2023-11-24T06:12:55.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:55.802+0000 7f601ca51640 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f601815e990 con 0x7f601815e1e0 2023-11-24T06:12:55.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:55.802+0000 7f601659c640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f6018143ae0 0x7f60180cf290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:53376/0 (socket says 172.21.15.92:53376) 2023-11-24T06:12:55.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:55.802+0000 7f600dd9b640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f60180cf9a0 0x7f601815dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:55.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:55.802+0000 7f601659c640 1 -- 172.21.15.92:0/552668686 learned_addr learned my addr 172.21.15.92:0/552668686 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-24T06:12:55.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:55.803+0000 7f600dd9b640 1 -- 172.21.15.92:0/552668686 >> v2:172.21.15.118:3300/0 conn(0x7f601815e1e0 msgr2=0x7f60180702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:55.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:55.803+0000 7f6016d9d640 1 --2- 172.21.15.92:0/552668686 >> v2:172.21.15.118:3300/0 conn(0x7f601815e1e0 0x7f60180702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:55.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:55.803+0000 7f600dd9b640 1 --2- 172.21.15.92:0/552668686 >> v2:172.21.15.118:3300/0 conn(0x7f601815e1e0 0x7f60180702c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:55.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:55.803+0000 7f600dd9b640 1 -- 172.21.15.92:0/552668686 >> v1:172.21.15.92:6789/0 conn(0x7f6018143ae0 legacy=0x7f60180cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:55.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:55.803+0000 7f6015d9b640 1 -- 172.21.15.92:0/552668686 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1910851988 0 0) 0x7f60180b1ee0 con 0x7f6018143ae0 2023-11-24T06:12:55.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:55.803+0000 7f600dd9b640 1 -- 172.21.15.92:0/552668686 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60180b2130 con 0x7f60180cf9a0 2023-11-24T06:12:55.808 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:55.803+0000 7f6016d9d640 1 --2- 172.21.15.92:0/552668686 >> v2:172.21.15.118:3300/0 conn(0x7f601815e1e0 0x7f60180702c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:12:55.808 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:55.804+0000 7f600dd9b640 1 --2- 172.21.15.92:0/552668686 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f60180cf9a0 0x7f601815dca0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f60180ae9d0 tx=0x7f600002bec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4db4f9a392102a11 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:55.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:55.804+0000 7f6015d9b640 1 -- 172.21.15.92:0/552668686 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f600002cc70 con 0x7f60180cf9a0 2023-11-24T06:12:55.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:55.804+0000 7f6015d9b640 1 -- 172.21.15.92:0/552668686 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f600002cdd0 con 0x7f60180cf9a0 2023-11-24T06:12:55.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:55.804+0000 7f6015d9b640 1 -- 172.21.15.92:0/552668686 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6000034660 con 0x7f60180cf9a0 2023-11-24T06:12:55.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:55.805+0000 7f601ca51640 1 -- 172.21.15.92:0/552668686 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f60180cf9a0 msgr2=0x7f601815dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:55.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:55.805+0000 7f601ca51640 1 --2- 172.21.15.92:0/552668686 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f60180cf9a0 0x7f601815dca0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f60180ae9d0 tx=0x7f600002bec0 comp rx=0 tx=0).stop 2023-11-24T06:12:55.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:55.805+0000 7f601ca51640 1 -- 172.21.15.92:0/552668686 shutdown_connections 2023-11-24T06:12:55.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:55.805+0000 7f601ca51640 1 --2- 172.21.15.92:0/552668686 >> v2:172.21.15.118:3300/0 conn(0x7f601815e1e0 0x7f60180702c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:55.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:55.805+0000 7f601ca51640 1 --2- 172.21.15.92:0/552668686 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f60180cf9a0 0x7f601815dca0 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:55.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:55.805+0000 7f601ca51640 1 -- 172.21.15.92:0/552668686 >> 172.21.15.92:0/552668686 conn(0x7f601806a800 msgr2=0x7f601806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:55.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:55.805+0000 7f601ca51640 1 -- 172.21.15.92:0/552668686 shutdown_connections 2023-11-24T06:12:55.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:55.805+0000 7f601ca51640 1 -- 172.21.15.92:0/552668686 wait complete. 2023-11-24T06:12:55.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:55.807+0000 7f601ca51640 1 Processor -- start 2023-11-24T06:12:55.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:55.807+0000 7f601ca51640 1 -- start start 2023-11-24T06:12:55.811 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:55.808+0000 7f601ca51640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f60180cf9a0 0x7f6018137bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:55.811 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:55.808+0000 7f601ca51640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6018143ae0 0x7f6018135570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:55.811 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:55.808+0000 7f601ca51640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f601815e1e0 0x7f60181f21d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:55.811 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:55.808+0000 7f601ca51640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60180ac8d0 con 0x7f6018143ae0 2023-11-24T06:12:55.812 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:55.808+0000 7f601ca51640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6018138110 con 0x7f60180cf9a0 2023-11-24T06:12:55.812 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:55.808+0000 7f601ca51640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60180c4210 con 0x7f601815e1e0 2023-11-24T06:12:55.812 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:55.808+0000 7f600dd9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6018143ae0 0x7f6018135570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:55.812 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:55.808+0000 7f600dd9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6018143ae0 0x7f6018135570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:55962/0 (socket says 172.21.15.92:55962) 2023-11-24T06:12:55.812 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:55.808+0000 7f600dd9b640 1 -- 172.21.15.92:0/3809870151 learned_addr learned my addr 172.21.15.92:0/3809870151 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:12:55.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:55.809+0000 7f601659c640 1 --2- 172.21.15.92:0/3809870151 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f60180cf9a0 0x7f6018137bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:55.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:55.809+0000 7f6016d9d640 1 --2- 172.21.15.92:0/3809870151 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f601815e1e0 0x7f60181f21d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:55.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:55.809+0000 7f600dd9b640 1 -- 172.21.15.92:0/3809870151 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f601815e1e0 msgr2=0x7f60181f21d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:55.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:55.809+0000 7f600dd9b640 1 --2- 172.21.15.92:0/3809870151 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f601815e1e0 0x7f60181f21d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:55.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:55.809+0000 7f600dd9b640 1 -- 172.21.15.92:0/3809870151 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f60180cf9a0 msgr2=0x7f6018137bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:55.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:55.809+0000 7f600dd9b640 1 --2- 172.21.15.92:0/3809870151 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f60180cf9a0 0x7f6018137bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:55.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:55.809+0000 7f600dd9b640 1 -- 172.21.15.92:0/3809870151 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6000009a70 con 0x7f6018143ae0 2023-11-24T06:12:55.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:55.809+0000 7f601659c640 1 --2- 172.21.15.92:0/3809870151 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f60180cf9a0 0x7f6018137bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:12:55.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:55.809+0000 7f600dd9b640 1 --2- 172.21.15.92:0/3809870151 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6018143ae0 0x7f6018135570 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f6000037040 tx=0x7f6000004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:55.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:55.809+0000 7f600f7fe640 1 -- 172.21.15.92:0/3809870151 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f600003e610 con 0x7f6018143ae0 2023-11-24T06:12:55.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:55.809+0000 7f600f7fe640 1 -- 172.21.15.92:0/3809870151 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f600003e770 con 0x7f6018143ae0 2023-11-24T06:12:55.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:55.809+0000 7f600f7fe640 1 -- 172.21.15.92:0/3809870151 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f600003eb20 con 0x7f6018143ae0 2023-11-24T06:12:55.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:55.809+0000 7f601ca51640 1 -- 172.21.15.92:0/3809870151 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60180acab0 con 0x7f6018143ae0 2023-11-24T06:12:55.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:55.810+0000 7f601ca51640 1 -- 172.21.15.92:0/3809870151 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60180ad070 con 0x7f6018143ae0 2023-11-24T06:12:55.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:55.811+0000 7f601ca51640 1 -- 172.21.15.92:0/3809870151 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60180675d0 con 0x7f6018143ae0 2023-11-24T06:12:55.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:55.812+0000 7f600f7fe640 1 -- 172.21.15.92:0/3809870151 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f600003b050 con 0x7f6018143ae0 2023-11-24T06:12:55.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:55.813+0000 7f600f7fe640 1 --2- 172.21.15.92:0/3809870151 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f5ff806aed0 0x7f5ff806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:55.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:55.813+0000 7f600f7fe640 1 -- 172.21.15.92:0/3809870151 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f60000ac040 con 0x7f6018143ae0 2023-11-24T06:12:55.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:55.813+0000 7f601659c640 1 --2- 172.21.15.92:0/3809870151 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f5ff806aed0 0x7f5ff806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:55.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:55.814+0000 7f601659c640 1 --2- 172.21.15.92:0/3809870151 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f5ff806aed0 0x7f5ff806d390 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f6008000f60 tx=0x7f6008009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:55.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:55.818+0000 7f600f7fe640 1 -- 172.21.15.92:0/3809870151 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6000077100 con 0x7f6018143ae0 2023-11-24T06:12:56.052 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:56.049+0000 7f601ca51640 1 -- 172.21.15.92:0/3809870151 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f60180ad230 con 0x7f6018143ae0 2023-11-24T06:12:56.053 INFO:tasks.ceph.mon.a.smithi092.stderr:2023-11-24T06:12:56.050+0000 7f83f3ce4640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-24T06:12:56.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:56.054+0000 7f600f7fe640 1 -- 172.21.15.92:0/3809870151 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v33) v1 ==== 161+0+0 (secure 0 0 0) 0x7f600007a720 con 0x7f6018143ae0 2023-11-24T06:12:56.057 INFO:teuthology.orchestra.run.smithi092.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-24T06:12:56.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:56.057+0000 7f601ca51640 1 -- 172.21.15.92:0/3809870151 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f5ff806aed0 msgr2=0x7f5ff806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:56.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:56.057+0000 7f601ca51640 1 --2- 172.21.15.92:0/3809870151 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f5ff806aed0 0x7f5ff806d390 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f6008000f60 tx=0x7f6008009040 comp rx=0 tx=0).stop 2023-11-24T06:12:56.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:56.057+0000 7f601ca51640 1 -- 172.21.15.92:0/3809870151 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6018143ae0 msgr2=0x7f6018135570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:56.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:56.057+0000 7f601ca51640 1 --2- 172.21.15.92:0/3809870151 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6018143ae0 0x7f6018135570 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f6000037040 tx=0x7f6000004a40 comp rx=0 tx=0).stop 2023-11-24T06:12:56.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:56.057+0000 7f601ca51640 1 -- 172.21.15.92:0/3809870151 shutdown_connections 2023-11-24T06:12:56.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:56.057+0000 7f601ca51640 1 --2- 172.21.15.92:0/3809870151 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f5ff806aed0 0x7f5ff806d390 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:56.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:56.057+0000 7f601ca51640 1 --2- 172.21.15.92:0/3809870151 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f601815e1e0 0x7f60181f21d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:56.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:56.057+0000 7f601ca51640 1 --2- 172.21.15.92:0/3809870151 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6018143ae0 0x7f6018135570 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:56.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:56.057+0000 7f601ca51640 1 --2- 172.21.15.92:0/3809870151 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f60180cf9a0 0x7f6018137bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:56.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:56.057+0000 7f601ca51640 1 -- 172.21.15.92:0/3809870151 >> 172.21.15.92:0/3809870151 conn(0x7f601806a800 msgr2=0x7f60180ce620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:56.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:56.057+0000 7f601ca51640 1 -- 172.21.15.92:0/3809870151 shutdown_connections 2023-11-24T06:12:56.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:56.057+0000 7f601ca51640 1 -- 172.21.15.92:0/3809870151 wait complete. 2023-11-24T06:12:56.079 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-11-24T06:12:56.080 INFO:tasks.ceph.mds.c.smithi092.stderr: -2426> 2023-11-24T06:12:35.522+0000 7f738eab8640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-24T06:12:56.153 INFO:tasks.ceph.mds.c.smithi092.stdout:starting mds.c at 2023-11-24T06:12:56.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:56.257+0000 7f42cd890640 1 Processor -- start 2023-11-24T06:12:56.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:56.257+0000 7f42cd890640 1 -- start start 2023-11-24T06:12:56.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:56.258+0000 7f42cd890640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f42c813e360 0x7f42c80cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:56.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:56.258+0000 7f42cd890640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f42c815f9c0 0x7f42c80cf7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:56.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:56.258+0000 7f42cd890640 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42c815c5f0 con 0x7f42c813df80 2023-11-24T06:12:56.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:56.258+0000 7f42cd890640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42c806c470 con 0x7f42c815f9c0 2023-11-24T06:12:56.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:56.258+0000 7f42cd890640 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42c806c750 con 0x7f42c813e360 2023-11-24T06:12:56.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:56.259+0000 7f42c77fe640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f42c815f9c0 0x7f42c80cf7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:56.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:56.259+0000 7f42c77fe640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f42c815f9c0 0x7f42c80cf7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:55988/0 (socket says 172.21.15.92:55988) 2023-11-24T06:12:56.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:56.259+0000 7f42c77fe640 1 -- 172.21.15.92:0/1063441916 learned_addr learned my addr 172.21.15.92:0/1063441916 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:12:56.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:56.259+0000 7f42c67fc640 1 --2- 172.21.15.92:0/1063441916 >> v2:172.21.15.118:3300/0 conn(0x7f42c813e360 0x7f42c80cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:56.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:56.259+0000 7f42c77fe640 1 -- 172.21.15.92:0/1063441916 >> v2:172.21.15.118:3300/0 conn(0x7f42c813e360 msgr2=0x7f42c80cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:56.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:56.259+0000 7f42c77fe640 1 --2- 172.21.15.92:0/1063441916 >> v2:172.21.15.118:3300/0 conn(0x7f42c813e360 0x7f42c80cf290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:56.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:56.259+0000 7f42c77fe640 1 -- 172.21.15.92:0/1063441916 >> v1:172.21.15.118:6789/0 conn(0x7f42c813df80 legacy=0x7f42c815bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:56.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:56.259+0000 7f42c77fe640 1 -- 172.21.15.92:0/1063441916 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42c806ca30 con 0x7f42c815f9c0 2023-11-24T06:12:56.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:56.260+0000 7f42c77fe640 1 --2- 172.21.15.92:0/1063441916 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f42c815f9c0 0x7f42c80cf7d0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f42c806ded0 tx=0x7f42c002bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64bf75c6989e4e77 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:56.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:56.261+0000 7f42c5ffb640 1 -- 172.21.15.92:0/1063441916 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42c002cb20 con 0x7f42c815f9c0 2023-11-24T06:12:56.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:56.261+0000 7f42c5ffb640 1 -- 172.21.15.92:0/1063441916 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42c002cc80 con 0x7f42c815f9c0 2023-11-24T06:12:56.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:56.261+0000 7f42c5ffb640 1 -- 172.21.15.92:0/1063441916 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42c00343f0 con 0x7f42c815f9c0 2023-11-24T06:12:56.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:56.261+0000 7f42cd890640 1 -- 172.21.15.92:0/1063441916 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f42c815f9c0 msgr2=0x7f42c80cf7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:56.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:56.261+0000 7f42cd890640 1 --2- 172.21.15.92:0/1063441916 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f42c815f9c0 0x7f42c80cf7d0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f42c806ded0 tx=0x7f42c002bf80 comp rx=0 tx=0).stop 2023-11-24T06:12:56.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:56.261+0000 7f42cd890640 1 -- 172.21.15.92:0/1063441916 shutdown_connections 2023-11-24T06:12:56.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:56.261+0000 7f42cd890640 1 --2- 172.21.15.92:0/1063441916 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f42c815f9c0 0x7f42c80cf7d0 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:56.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:56.261+0000 7f42cd890640 1 --2- 172.21.15.92:0/1063441916 >> v2:172.21.15.118:3300/0 conn(0x7f42c813e360 0x7f42c80cf290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:56.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:56.261+0000 7f42cd890640 1 -- 172.21.15.92:0/1063441916 >> 172.21.15.92:0/1063441916 conn(0x7f42c806a800 msgr2=0x7f42c806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:56.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:56.261+0000 7f42cd890640 1 -- 172.21.15.92:0/1063441916 shutdown_connections 2023-11-24T06:12:56.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:56.261+0000 7f42cd890640 1 -- 172.21.15.92:0/1063441916 wait complete. 2023-11-24T06:12:56.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:56.263+0000 7f42cd890640 1 Processor -- start 2023-11-24T06:12:56.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:56.263+0000 7f42cd890640 1 -- start start 2023-11-24T06:12:56.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:56.264+0000 7f42cd890640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f42c813df80 0x7f42c814ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:56.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:56.264+0000 7f42cd890640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f42c813e360 0x7f42c8148f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:56.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:56.264+0000 7f42cd890640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f42c815f9c0 0x7f42c81494c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:56.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:56.264+0000 7f42cd890640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42c8149a00 con 0x7f42c813e360 2023-11-24T06:12:56.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:56.264+0000 7f42cd890640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42c814e240 con 0x7f42c815f9c0 2023-11-24T06:12:56.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:56.264+0000 7f42cd890640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42c814d630 con 0x7f42c813df80 2023-11-24T06:12:56.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:56.264+0000 7f42c67fc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f42c813e360 0x7f42c8148f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:56.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:56.264+0000 7f42c67fc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f42c813e360 0x7f42c8148f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:56002/0 (socket says 172.21.15.92:56002) 2023-11-24T06:12:56.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:56.264+0000 7f42c67fc640 1 -- 172.21.15.92:0/2160912847 learned_addr learned my addr 172.21.15.92:0/2160912847 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:12:56.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:56.265+0000 7f42c67fc640 1 -- 172.21.15.92:0/2160912847 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f42c813df80 msgr2=0x7f42c814ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:56.269 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:56.265+0000 7f42c67fc640 1 --2- 172.21.15.92:0/2160912847 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f42c813df80 0x7f42c814ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:56.269 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:56.265+0000 7f42c67fc640 1 -- 172.21.15.92:0/2160912847 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f42c815f9c0 msgr2=0x7f42c81494c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:56.269 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:56.265+0000 7f42c67fc640 1 --2- 172.21.15.92:0/2160912847 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f42c815f9c0 0x7f42c81494c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:56.269 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:56.265+0000 7f42c67fc640 1 -- 172.21.15.92:0/2160912847 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42c0009b30 con 0x7f42c813e360 2023-11-24T06:12:56.269 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:56.265+0000 7f42c67fc640 1 --2- 172.21.15.92:0/2160912847 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f42c813e360 0x7f42c8148f80 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f42bc00d7d0 tx=0x7f42bc00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:56.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:56.265+0000 7f42affff640 1 -- 172.21.15.92:0/2160912847 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42bc00b6c0 con 0x7f42c813e360 2023-11-24T06:12:56.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:56.265+0000 7f42affff640 1 -- 172.21.15.92:0/2160912847 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42bc00b820 con 0x7f42c813e360 2023-11-24T06:12:56.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:56.265+0000 7f42cd890640 1 -- 172.21.15.92:0/2160912847 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42c814d910 con 0x7f42c813e360 2023-11-24T06:12:56.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:56.265+0000 7f42affff640 1 -- 172.21.15.92:0/2160912847 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42bc00bbb0 con 0x7f42c813e360 2023-11-24T06:12:56.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:56.265+0000 7f42cd890640 1 -- 172.21.15.92:0/2160912847 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42c814ded0 con 0x7f42c813e360 2023-11-24T06:12:56.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:56.266+0000 7f42cd890640 1 -- 172.21.15.92:0/2160912847 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42c80675d0 con 0x7f42c813e360 2023-11-24T06:12:56.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:56.267+0000 7f42affff640 1 -- 172.21.15.92:0/2160912847 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f42bc00bd10 con 0x7f42c813e360 2023-11-24T06:12:56.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:56.268+0000 7f42affff640 1 --2- 172.21.15.92:0/2160912847 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f42b406af00 0x7f42b406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:56.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:56.268+0000 7f42affff640 1 -- 172.21.15.92:0/2160912847 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f42bc089c10 con 0x7f42c813e360 2023-11-24T06:12:56.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:56.268+0000 7f42c6ffd640 1 --2- 172.21.15.92:0/2160912847 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f42b406af00 0x7f42b406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:56.272 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:56.269+0000 7f42c6ffd640 1 --2- 172.21.15.92:0/2160912847 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f42b406af00 0x7f42b406d3c0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f42c815c780 tx=0x7f42b8007950 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:56.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:56.273+0000 7f42affff640 1 -- 172.21.15.92:0/2160912847 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f42bc054c70 con 0x7f42c813e360 2023-11-24T06:12:56.488 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:56.485+0000 7f42cd890640 1 -- 172.21.15.92:0/2160912847 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f42c806df00 con 0x7f42c813e360 2023-11-24T06:12:57.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:57.057+0000 7f42affff640 1 -- 172.21.15.92:0/2160912847 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v34) v1 ==== 104+0+0 (secure 0 0 0) 0x7f42bc058290 con 0x7f42c813e360 2023-11-24T06:12:57.064 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:57.061+0000 7f42cd890640 1 -- 172.21.15.92:0/2160912847 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f42b406af00 msgr2=0x7f42b406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:57.064 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:57.061+0000 7f42cd890640 1 --2- 172.21.15.92:0/2160912847 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f42b406af00 0x7f42b406d3c0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f42c815c780 tx=0x7f42b8007950 comp rx=0 tx=0).stop 2023-11-24T06:12:57.064 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:57.061+0000 7f42cd890640 1 -- 172.21.15.92:0/2160912847 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f42c813e360 msgr2=0x7f42c8148f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:57.064 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:57.061+0000 7f42cd890640 1 --2- 172.21.15.92:0/2160912847 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f42c813e360 0x7f42c8148f80 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f42bc00d7d0 tx=0x7f42bc00dc90 comp rx=0 tx=0).stop 2023-11-24T06:12:57.064 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:57.061+0000 7f42cd890640 1 -- 172.21.15.92:0/2160912847 shutdown_connections 2023-11-24T06:12:57.064 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:57.061+0000 7f42cd890640 1 --2- 172.21.15.92:0/2160912847 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f42b406af00 0x7f42b406d3c0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:57.064 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:57.061+0000 7f42cd890640 1 --2- 172.21.15.92:0/2160912847 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f42c815f9c0 0x7f42c81494c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:57.065 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:57.061+0000 7f42cd890640 1 --2- 172.21.15.92:0/2160912847 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f42c813e360 0x7f42c8148f80 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:57.065 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:57.061+0000 7f42cd890640 1 --2- 172.21.15.92:0/2160912847 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f42c813df80 0x7f42c814ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:57.065 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:57.061+0000 7f42cd890640 1 -- 172.21.15.92:0/2160912847 >> 172.21.15.92:0/2160912847 conn(0x7f42c806a800 msgr2=0x7f42c80b3ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:57.065 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:57.061+0000 7f42cd890640 1 -- 172.21.15.92:0/2160912847 shutdown_connections 2023-11-24T06:12:57.065 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:57.061+0000 7f42cd890640 1 -- 172.21.15.92:0/2160912847 wait complete. 2023-11-24T06:12:57.085 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-11-24T06:12:57.290 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:57.286+0000 7faa942ec640 1 Processor -- start 2023-11-24T06:12:57.290 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:57.287+0000 7faa942ec640 1 -- start start 2023-11-24T06:12:57.292 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:57.288+0000 7faa942ec640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7faa8c15d5e0 0x7faa8c0b36d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:57.292 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:57.288+0000 7faa942ec640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7faa8c071b00 0x7faa8c06a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:57.292 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:57.288+0000 7faa942ec640 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa8c0b2f60 con 0x7faa8c0b3c10 2023-11-24T06:12:57.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:57.288+0000 7faa942ec640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa8c0cfd80 con 0x7faa8c15d5e0 2023-11-24T06:12:57.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:57.288+0000 7faa942ec640 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa8c06fd00 con 0x7faa8c071b00 2023-11-24T06:12:57.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:57.289+0000 7faa92088640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7faa8c15d5e0 0x7faa8c0b36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:57.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:57.289+0000 7faa92088640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7faa8c15d5e0 0x7faa8c0b36d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:56018/0 (socket says 172.21.15.92:56018) 2023-11-24T06:12:57.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:57.289+0000 7faa92088640 1 -- 172.21.15.92:0/2952287072 learned_addr learned my addr 172.21.15.92:0/2952287072 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:12:57.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:57.289+0000 7faa92889640 1 --2- 172.21.15.92:0/2952287072 >> v2:172.21.15.118:3300/0 conn(0x7faa8c071b00 0x7faa8c06a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:57.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:57.289+0000 7faa92088640 1 -- 172.21.15.92:0/2952287072 >> v2:172.21.15.118:3300/0 conn(0x7faa8c071b00 msgr2=0x7faa8c06a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:57.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:57.289+0000 7faa92088640 1 --2- 172.21.15.92:0/2952287072 >> v2:172.21.15.118:3300/0 conn(0x7faa8c071b00 0x7faa8c06a3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:57.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:57.289+0000 7faa92088640 1 -- 172.21.15.92:0/2952287072 >> v1:172.21.15.118:6789/0 conn(0x7faa8c0b3c10 legacy=0x7faa8c0713f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:57.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:57.289+0000 7faa92088640 1 -- 172.21.15.92:0/2952287072 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa8c06fa20 con 0x7faa8c15d5e0 2023-11-24T06:12:57.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:57.290+0000 7faa92088640 1 --2- 172.21.15.92:0/2952287072 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faa8c15d5e0 0x7faa8c0b36d0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7faa8c071380 tx=0x7faa7c02bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa1db401dfb5ca44 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:57.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:57.290+0000 7faa91086640 1 -- 172.21.15.92:0/2952287072 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa7c02ca80 con 0x7faa8c15d5e0 2023-11-24T06:12:57.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:57.290+0000 7faa91086640 1 -- 172.21.15.92:0/2952287072 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa7c02cbe0 con 0x7faa8c15d5e0 2023-11-24T06:12:57.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:57.290+0000 7faa91086640 1 -- 172.21.15.92:0/2952287072 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa7c0343f0 con 0x7faa8c15d5e0 2023-11-24T06:12:57.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:57.291+0000 7faa942ec640 1 -- 172.21.15.92:0/2952287072 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faa8c15d5e0 msgr2=0x7faa8c0b36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:57.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:57.291+0000 7faa942ec640 1 --2- 172.21.15.92:0/2952287072 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faa8c15d5e0 0x7faa8c0b36d0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7faa8c071380 tx=0x7faa7c02bee0 comp rx=0 tx=0).stop 2023-11-24T06:12:57.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:57.291+0000 7faa942ec640 1 -- 172.21.15.92:0/2952287072 shutdown_connections 2023-11-24T06:12:57.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:57.291+0000 7faa942ec640 1 --2- 172.21.15.92:0/2952287072 >> v2:172.21.15.118:3300/0 conn(0x7faa8c071b00 0x7faa8c06a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:57.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:57.291+0000 7faa942ec640 1 --2- 172.21.15.92:0/2952287072 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faa8c15d5e0 0x7faa8c0b36d0 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:57.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:57.291+0000 7faa942ec640 1 -- 172.21.15.92:0/2952287072 >> 172.21.15.92:0/2952287072 conn(0x7faa8c0b1c60 msgr2=0x7faa8c0b2070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:57.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:57.291+0000 7faa942ec640 1 -- 172.21.15.92:0/2952287072 shutdown_connections 2023-11-24T06:12:57.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:57.291+0000 7faa942ec640 1 -- 172.21.15.92:0/2952287072 wait complete. 2023-11-24T06:12:57.299 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:57.293+0000 7faa942ec640 1 Processor -- start 2023-11-24T06:12:57.299 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:57.293+0000 7faa942ec640 1 -- start start 2023-11-24T06:12:57.299 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:57.294+0000 7faa942ec640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7faa8c15d5e0 0x7faa8c152d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:57.299 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:57.294+0000 7faa942ec640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7faa8c156330 0x7faa8c0ceb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:57.299 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:57.294+0000 7faa942ec640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faa8c0cf0a0 0x7faa8c15fb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:57.300 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:57.294+0000 7faa942ec640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa8c153ea0 con 0x7faa8c0cf0a0 2023-11-24T06:12:57.300 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:57.294+0000 7faa942ec640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa8c0cf840 con 0x7faa8c15d5e0 2023-11-24T06:12:57.300 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:57.294+0000 7faa942ec640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa8c153820 con 0x7faa8c156330 2023-11-24T06:12:57.300 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:57.294+0000 7faa92889640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faa8c0cf0a0 0x7faa8c15fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:57.300 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:57.294+0000 7faa92889640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faa8c0cf0a0 0x7faa8c15fb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:56026/0 (socket says 172.21.15.92:56026) 2023-11-24T06:12:57.300 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:57.294+0000 7faa92889640 1 -- 172.21.15.92:0/1152473556 learned_addr learned my addr 172.21.15.92:0/1152473556 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:12:57.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:57.294+0000 7faa92088640 1 --2- 172.21.15.92:0/1152473556 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7faa8c15d5e0 0x7faa8c152d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:57.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:57.295+0000 7faa91887640 1 --2- 172.21.15.92:0/1152473556 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7faa8c156330 0x7faa8c0ceb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:57.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:57.295+0000 7faa92889640 1 -- 172.21.15.92:0/1152473556 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7faa8c156330 msgr2=0x7faa8c0ceb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:57.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:57.295+0000 7faa92889640 1 --2- 172.21.15.92:0/1152473556 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7faa8c156330 0x7faa8c0ceb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:57.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:57.295+0000 7faa92889640 1 -- 172.21.15.92:0/1152473556 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7faa8c15d5e0 msgr2=0x7faa8c152d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:57.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:57.295+0000 7faa92889640 1 --2- 172.21.15.92:0/1152473556 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7faa8c15d5e0 0x7faa8c152d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:57.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:57.295+0000 7faa92889640 1 -- 172.21.15.92:0/1152473556 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa7c009b30 con 0x7faa8c0cf0a0 2023-11-24T06:12:57.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:57.295+0000 7faa91887640 1 --2- 172.21.15.92:0/1152473556 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7faa8c156330 0x7faa8c0ceb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:12:57.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:57.295+0000 7faa92889640 1 --2- 172.21.15.92:0/1152473556 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faa8c0cf0a0 0x7faa8c15fb90 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7faa8400c7c0 tx=0x7faa8400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:57.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:57.295+0000 7faa8affd640 1 -- 172.21.15.92:0/1152473556 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa84004220 con 0x7faa8c0cf0a0 2023-11-24T06:12:57.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:57.295+0000 7faa8affd640 1 -- 172.21.15.92:0/1152473556 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa84004380 con 0x7faa8c0cf0a0 2023-11-24T06:12:57.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:57.295+0000 7faa8affd640 1 -- 172.21.15.92:0/1152473556 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa84007840 con 0x7faa8c0cf0a0 2023-11-24T06:12:57.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:57.295+0000 7faa942ec640 1 -- 172.21.15.92:0/1152473556 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa8c153290 con 0x7faa8c0cf0a0 2023-11-24T06:12:57.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:57.295+0000 7faa942ec640 1 -- 172.21.15.92:0/1152473556 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa8c153ac0 con 0x7faa8c0cf0a0 2023-11-24T06:12:57.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:57.296+0000 7faa942ec640 1 -- 172.21.15.92:0/1152473556 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa8c0b44e0 con 0x7faa8c0cf0a0 2023-11-24T06:12:57.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:57.297+0000 7faa8affd640 1 -- 172.21.15.92:0/1152473556 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7faa840079a0 con 0x7faa8c0cf0a0 2023-11-24T06:12:57.304 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:57.298+0000 7faa8affd640 1 --2- 172.21.15.92:0/1152473556 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7faa7806aed0 0x7faa7806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:57.304 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:57.298+0000 7faa8affd640 1 -- 172.21.15.92:0/1152473556 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 5795+0+0 (secure 0 0 0) 0x7faa84089560 con 0x7faa8c0cf0a0 2023-11-24T06:12:57.304 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:57.298+0000 7faa92088640 1 --2- 172.21.15.92:0/1152473556 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7faa7806aed0 0x7faa7806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:57.304 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:57.299+0000 7faa92088640 1 --2- 172.21.15.92:0/1152473556 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7faa7806aed0 0x7faa7806d390 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7faa7c035000 tx=0x7faa7c030000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:57.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:57.303+0000 7faa8affd640 1 -- 172.21.15.92:0/1152473556 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7faa84054500 con 0x7faa8c0cf0a0 2023-11-24T06:12:57.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:57.494+0000 7faa942ec640 1 -- 172.21.15.92:0/1152473556 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7faa8c153680 con 0x7faa8c0cf0a0 2023-11-24T06:12:58.065 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:58.061+0000 7faa8affd640 1 -- 172.21.15.92:0/1152473556 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v54) v1 ==== 181+0+0 (secure 0 0 0) 0x7faa8c153680 con 0x7faa8c0cf0a0 2023-11-24T06:12:58.065 INFO:teuthology.orchestra.run.smithi092.stderr:pool 'cephfs_metadata' removed 2023-11-24T06:12:58.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:58.064+0000 7faa942ec640 1 -- 172.21.15.92:0/1152473556 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7faa7806aed0 msgr2=0x7faa7806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:58.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:58.064+0000 7faa942ec640 1 --2- 172.21.15.92:0/1152473556 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7faa7806aed0 0x7faa7806d390 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7faa7c035000 tx=0x7faa7c030000 comp rx=0 tx=0).stop 2023-11-24T06:12:58.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:58.064+0000 7faa942ec640 1 -- 172.21.15.92:0/1152473556 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faa8c0cf0a0 msgr2=0x7faa8c15fb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:58.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:58.064+0000 7faa942ec640 1 --2- 172.21.15.92:0/1152473556 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faa8c0cf0a0 0x7faa8c15fb90 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7faa8400c7c0 tx=0x7faa8400cc80 comp rx=0 tx=0).stop 2023-11-24T06:12:58.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:58.065+0000 7faa942ec640 1 -- 172.21.15.92:0/1152473556 shutdown_connections 2023-11-24T06:12:58.069 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:58.065+0000 7faa942ec640 1 --2- 172.21.15.92:0/1152473556 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7faa7806aed0 0x7faa7806d390 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:58.069 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:58.065+0000 7faa942ec640 1 --2- 172.21.15.92:0/1152473556 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faa8c0cf0a0 0x7faa8c15fb90 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:58.069 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:58.065+0000 7faa942ec640 1 --2- 172.21.15.92:0/1152473556 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7faa8c156330 0x7faa8c0ceb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:58.069 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:58.065+0000 7faa942ec640 1 --2- 172.21.15.92:0/1152473556 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7faa8c15d5e0 0x7faa8c152d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:58.069 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:58.065+0000 7faa942ec640 1 -- 172.21.15.92:0/1152473556 >> 172.21.15.92:0/1152473556 conn(0x7faa8c0b1c60 msgr2=0x7faa8c0b2040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:58.069 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:58.065+0000 7faa942ec640 1 -- 172.21.15.92:0/1152473556 shutdown_connections 2023-11-24T06:12:58.069 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:58.065+0000 7faa942ec640 1 -- 172.21.15.92:0/1152473556 wait complete. 2023-11-24T06:12:58.090 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-11-24T06:12:58.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:58.300+0000 7f140ffff640 1 Processor -- start 2023-11-24T06:12:58.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:58.300+0000 7f140ffff640 1 -- start start 2023-11-24T06:12:58.304 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:58.301+0000 7f140ffff640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f141013cfa0 0x7f141015afb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:58.304 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:58.301+0000 7f140ffff640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f14100cd530 0x7f141013f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:58.304 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:58.301+0000 7f140ffff640 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14100cdec0 con 0x7f14100cd9b0 2023-11-24T06:12:58.305 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:58.301+0000 7f140ffff640 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14100ce1a0 con 0x7f141013cfa0 2023-11-24T06:12:58.305 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:58.301+0000 7f140ffff640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14100ce480 con 0x7f14100cd530 2023-11-24T06:12:58.305 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:58.302+0000 7f140effd640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f141013cfa0 0x7f141015afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:58.305 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:58.302+0000 7f1407fff640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f14100cd530 0x7f141013f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:58.305 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:58.302+0000 7f1407fff640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f14100cd530 0x7f141013f5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3301/0 says I am v2:172.21.15.92:47128/0 (socket says 172.21.15.92:47128) 2023-11-24T06:12:58.305 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:58.302+0000 7f140effd640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f141013cfa0 0x7f141015afb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.92:41566/0 (socket says 172.21.15.92:41566) 2023-11-24T06:12:58.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:58.302+0000 7f1407fff640 1 -- 172.21.15.92:0/1230680064 learned_addr learned my addr 172.21.15.92:0/1230680064 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:12:58.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:58.303+0000 7f1407fff640 1 -- 172.21.15.92:0/1230680064 >> v2:172.21.15.118:3300/0 conn(0x7f141013cfa0 msgr2=0x7f141015afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:58.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:58.303+0000 7f1407fff640 1 --2- 172.21.15.92:0/1230680064 >> v2:172.21.15.118:3300/0 conn(0x7f141013cfa0 0x7f141015afb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:58.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:58.303+0000 7f1407fff640 1 -- 172.21.15.92:0/1230680064 >> v1:172.21.15.118:6789/0 conn(0x7f14100cd9b0 legacy=0x7f141015d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:58.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:58.303+0000 7f1407fff640 1 -- 172.21.15.92:0/1230680064 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14100ce760 con 0x7f14100cd530 2023-11-24T06:12:58.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:58.304+0000 7f1407fff640 1 --2- 172.21.15.92:0/1230680064 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f14100cd530 0x7f141013f5a0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f13f8009870 tx=0x7f13f802b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c6f6f56762fa0159 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:58.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:58.305+0000 7f140e7fc640 1 -- 172.21.15.92:0/1230680064 <== mon.2 v2:172.21.15.118:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13f8031640 con 0x7f14100cd530 2023-11-24T06:12:58.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:58.305+0000 7f140e7fc640 1 -- 172.21.15.92:0/1230680064 <== mon.2 v2:172.21.15.118:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13f80317a0 con 0x7f14100cd530 2023-11-24T06:12:58.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:58.305+0000 7f140e7fc640 1 -- 172.21.15.92:0/1230680064 <== mon.2 v2:172.21.15.118:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13f8031b30 con 0x7f14100cd530 2023-11-24T06:12:58.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:58.306+0000 7f140ffff640 1 -- 172.21.15.92:0/1230680064 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f14100cd530 msgr2=0x7f141013f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:58.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:58.306+0000 7f140ffff640 1 --2- 172.21.15.92:0/1230680064 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f14100cd530 0x7f141013f5a0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f13f8009870 tx=0x7f13f802b8f0 comp rx=0 tx=0).stop 2023-11-24T06:12:58.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:58.306+0000 7f140ffff640 1 -- 172.21.15.92:0/1230680064 shutdown_connections 2023-11-24T06:12:58.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:58.306+0000 7f140ffff640 1 --2- 172.21.15.92:0/1230680064 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f14100cd530 0x7f141013f5a0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:58.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:58.306+0000 7f140ffff640 1 --2- 172.21.15.92:0/1230680064 >> v2:172.21.15.118:3300/0 conn(0x7f141013cfa0 0x7f141015afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:58.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:58.306+0000 7f140ffff640 1 -- 172.21.15.92:0/1230680064 >> 172.21.15.92:0/1230680064 conn(0x7f141006a800 msgr2=0x7f141006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:58.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:58.306+0000 7f140ffff640 1 -- 172.21.15.92:0/1230680064 shutdown_connections 2023-11-24T06:12:58.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:58.306+0000 7f140ffff640 1 -- 172.21.15.92:0/1230680064 wait complete. 2023-11-24T06:12:58.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:58.308+0000 7f140ffff640 1 Processor -- start 2023-11-24T06:12:58.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:58.308+0000 7f140ffff640 1 -- start start 2023-11-24T06:12:58.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:58.309+0000 7f140ffff640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f14100cd530 0x7f1410158d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:58.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:58.309+0000 7f140ffff640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f14100cd9b0 0x7f14101f6a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:58.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:58.309+0000 7f140ffff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f141013cfa0 0x7f14101f6fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:58.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:58.309+0000 7f140ffff640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14101517a0 con 0x7f141013cfa0 2023-11-24T06:12:58.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:58.309+0000 7f140ffff640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1410159260 con 0x7f14100cd530 2023-11-24T06:12:58.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:58.309+0000 7f140ffff640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14101e56b0 con 0x7f14100cd9b0 2023-11-24T06:12:58.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:58.309+0000 7f140f7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f141013cfa0 0x7f14101f6fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:58.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:58.309+0000 7f140f7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f141013cfa0 0x7f14101f6fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:56028/0 (socket says 172.21.15.92:56028) 2023-11-24T06:12:58.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:58.309+0000 7f140f7fe640 1 -- 172.21.15.92:0/3745409441 learned_addr learned my addr 172.21.15.92:0/3745409441 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:12:58.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:58.309+0000 7f1407fff640 1 --2- 172.21.15.92:0/3745409441 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f14100cd9b0 0x7f14101f6a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:58.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:58.310+0000 7f140effd640 1 --2- 172.21.15.92:0/3745409441 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f14100cd530 0x7f1410158d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:58.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:58.310+0000 7f140f7fe640 1 -- 172.21.15.92:0/3745409441 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f14100cd9b0 msgr2=0x7f14101f6a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:58.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:58.310+0000 7f140f7fe640 1 --2- 172.21.15.92:0/3745409441 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f14100cd9b0 0x7f14101f6a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:58.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:58.310+0000 7f140f7fe640 1 -- 172.21.15.92:0/3745409441 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f14100cd530 msgr2=0x7f1410158d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:58.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:58.310+0000 7f140f7fe640 1 --2- 172.21.15.92:0/3745409441 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f14100cd530 0x7f1410158d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:58.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:58.310+0000 7f140f7fe640 1 -- 172.21.15.92:0/3745409441 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13f80094d0 con 0x7f141013cfa0 2023-11-24T06:12:58.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:58.310+0000 7f140effd640 1 --2- 172.21.15.92:0/3745409441 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f14100cd530 0x7f1410158d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:12:58.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:58.310+0000 7f140f7fe640 1 --2- 172.21.15.92:0/3745409441 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f141013cfa0 0x7f14101f6fd0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f140800c7a0 tx=0x7f140800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:58.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:58.310+0000 7f140cff9640 1 -- 172.21.15.92:0/3745409441 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1408003b70 con 0x7f141013cfa0 2023-11-24T06:12:58.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:58.310+0000 7f140cff9640 1 -- 172.21.15.92:0/3745409441 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1408003cf0 con 0x7f141013cfa0 2023-11-24T06:12:58.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:58.310+0000 7f140cff9640 1 -- 172.21.15.92:0/3745409441 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1408003ea0 con 0x7f141013cfa0 2023-11-24T06:12:58.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:58.310+0000 7f140ffff640 1 -- 172.21.15.92:0/3745409441 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14101e5990 con 0x7f141013cfa0 2023-11-24T06:12:58.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:58.310+0000 7f140ffff640 1 -- 172.21.15.92:0/3745409441 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14101e5f50 con 0x7f141013cfa0 2023-11-24T06:12:58.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:58.311+0000 7f140ffff640 1 -- 172.21.15.92:0/3745409441 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1410151950 con 0x7f141013cfa0 2023-11-24T06:12:58.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:58.312+0000 7f140cff9640 1 -- 172.21.15.92:0/3745409441 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f140800aa90 con 0x7f141013cfa0 2023-11-24T06:12:58.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:58.313+0000 7f140cff9640 1 --2- 172.21.15.92:0/3745409441 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f13f006aed0 0x7f13f006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:58.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:58.313+0000 7f140cff9640 1 -- 172.21.15.92:0/3745409441 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 5357+0+0 (secure 0 0 0) 0x7f14080893b0 con 0x7f141013cfa0 2023-11-24T06:12:58.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:58.313+0000 7f140effd640 1 --2- 172.21.15.92:0/3745409441 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f13f006aed0 0x7f13f006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:58.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:58.314+0000 7f140effd640 1 --2- 172.21.15.92:0/3745409441 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f13f006aed0 0x7f13f006d390 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f1400009a20 tx=0x7f1400008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:58.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:58.318+0000 7f140cff9640 1 -- 172.21.15.92:0/3745409441 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1410151950 con 0x7f141013cfa0 2023-11-24T06:12:58.509 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:58.506+0000 7f140ffff640 1 -- 172.21.15.92:0/3745409441 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f1410151950 con 0x7f141013cfa0 2023-11-24T06:12:59.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:59.064+0000 7f140cff9640 1 -- 172.21.15.92:0/3745409441 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v55) v1 ==== 169+0+0 (secure 0 0 0) 0x7f1410151950 con 0x7f141013cfa0 2023-11-24T06:12:59.068 INFO:teuthology.orchestra.run.smithi092.stderr:pool 'cephfs_data' removed 2023-11-24T06:12:59.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:59.068+0000 7f140ffff640 1 -- 172.21.15.92:0/3745409441 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f13f006aed0 msgr2=0x7f13f006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:59.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:59.068+0000 7f140ffff640 1 --2- 172.21.15.92:0/3745409441 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f13f006aed0 0x7f13f006d390 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f1400009a20 tx=0x7f1400008040 comp rx=0 tx=0).stop 2023-11-24T06:12:59.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:59.068+0000 7f140ffff640 1 -- 172.21.15.92:0/3745409441 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f141013cfa0 msgr2=0x7f14101f6fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:59.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:59.068+0000 7f140ffff640 1 --2- 172.21.15.92:0/3745409441 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f141013cfa0 0x7f14101f6fd0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f140800c7a0 tx=0x7f140800cc60 comp rx=0 tx=0).stop 2023-11-24T06:12:59.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:59.068+0000 7f140ffff640 1 -- 172.21.15.92:0/3745409441 shutdown_connections 2023-11-24T06:12:59.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:59.068+0000 7f140ffff640 1 --2- 172.21.15.92:0/3745409441 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f13f006aed0 0x7f13f006d390 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:59.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:59.068+0000 7f140ffff640 1 --2- 172.21.15.92:0/3745409441 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f141013cfa0 0x7f14101f6fd0 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:59.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:59.068+0000 7f140ffff640 1 --2- 172.21.15.92:0/3745409441 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f14100cd9b0 0x7f14101f6a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:59.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:59.068+0000 7f140ffff640 1 --2- 172.21.15.92:0/3745409441 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f14100cd530 0x7f1410158d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:59.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:59.068+0000 7f140ffff640 1 -- 172.21.15.92:0/3745409441 >> 172.21.15.92:0/3745409441 conn(0x7f141006a800 msgr2=0x7f1410155060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:59.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:59.069+0000 7f140ffff640 1 -- 172.21.15.92:0/3745409441 shutdown_connections 2023-11-24T06:12:59.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:59.069+0000 7f140ffff640 1 -- 172.21.15.92:0/3745409441 wait complete. 2023-11-24T06:12:59.092 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-11-24T06:12:59.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:59.308+0000 7f4ff21b0640 1 Processor -- start 2023-11-24T06:12:59.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:59.309+0000 7f4ff21b0640 1 -- start start 2023-11-24T06:12:59.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:59.310+0000 7f4ff21b0640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f4fec13e360 0x7f4fec0cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:59.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:59.310+0000 7f4ff21b0640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f4fec15f9c0 0x7f4fec0cf7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:59.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:59.310+0000 7f4ff21b0640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fec15c5f0 con 0x7f4fec13df80 2023-11-24T06:12:59.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:59.310+0000 7f4ff21b0640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4fec06c470 con 0x7f4fec15f9c0 2023-11-24T06:12:59.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:59.310+0000 7f4ff21b0640 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4fec06c750 con 0x7f4fec13e360 2023-11-24T06:12:59.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:59.310+0000 7f4febfff640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f4fec15f9c0 0x7f4fec0cf7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:59.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:59.310+0000 7f4febfff640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f4fec15f9c0 0x7f4fec0cf7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:54444/0 (socket says 172.21.15.92:54444) 2023-11-24T06:12:59.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:59.310+0000 7f4febfff640 1 -- 172.21.15.92:0/3836968564 learned_addr learned my addr 172.21.15.92:0/3836968564 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:12:59.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:59.311+0000 7f4feaffd640 1 --2- 172.21.15.92:0/3836968564 >> v2:172.21.15.118:3300/0 conn(0x7f4fec13e360 0x7f4fec0cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:59.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:59.311+0000 7f4febfff640 1 -- 172.21.15.92:0/3836968564 >> v2:172.21.15.118:3300/0 conn(0x7f4fec13e360 msgr2=0x7f4fec0cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:59.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:59.311+0000 7f4febfff640 1 --2- 172.21.15.92:0/3836968564 >> v2:172.21.15.118:3300/0 conn(0x7f4fec13e360 0x7f4fec0cf290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:59.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:59.311+0000 7f4febfff640 1 -- 172.21.15.92:0/3836968564 >> v1:172.21.15.92:6789/0 conn(0x7f4fec13df80 legacy=0x7f4fec15bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:59.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:59.311+0000 7f4febfff640 1 -- 172.21.15.92:0/3836968564 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fec06ca30 con 0x7f4fec15f9c0 2023-11-24T06:12:59.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:59.311+0000 7f4feaffd640 1 --2- 172.21.15.92:0/3836968564 >> v2:172.21.15.118:3300/0 conn(0x7f4fec13e360 0x7f4fec0cf290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:12:59.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:59.311+0000 7f4fea7fc640 1 -- 172.21.15.92:0/3836968564 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2117614878 0 0) 0x7f4fec15c5f0 con 0x7f4fec13df80 2023-11-24T06:12:59.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:59.312+0000 7f4febfff640 1 --2- 172.21.15.92:0/3836968564 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4fec15f9c0 0x7f4fec0cf7d0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f4fec06ded0 tx=0x7f4fe402b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89661eb9f940683f server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:59.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:59.312+0000 7f4fea7fc640 1 -- 172.21.15.92:0/3836968564 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fe4030860 con 0x7f4fec15f9c0 2023-11-24T06:12:59.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:59.312+0000 7f4fea7fc640 1 -- 172.21.15.92:0/3836968564 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fe40309c0 con 0x7f4fec15f9c0 2023-11-24T06:12:59.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:59.312+0000 7f4fea7fc640 1 -- 172.21.15.92:0/3836968564 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fe4030d50 con 0x7f4fec15f9c0 2023-11-24T06:12:59.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:59.313+0000 7f4ff21b0640 1 -- 172.21.15.92:0/3836968564 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4fec15f9c0 msgr2=0x7f4fec0cf7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:59.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:59.313+0000 7f4ff21b0640 1 --2- 172.21.15.92:0/3836968564 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4fec15f9c0 0x7f4fec0cf7d0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f4fec06ded0 tx=0x7f4fe402b9a0 comp rx=0 tx=0).stop 2023-11-24T06:12:59.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:59.313+0000 7f4ff21b0640 1 -- 172.21.15.92:0/3836968564 shutdown_connections 2023-11-24T06:12:59.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:59.313+0000 7f4ff21b0640 1 --2- 172.21.15.92:0/3836968564 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4fec15f9c0 0x7f4fec0cf7d0 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:59.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:59.313+0000 7f4ff21b0640 1 --2- 172.21.15.92:0/3836968564 >> v2:172.21.15.118:3300/0 conn(0x7f4fec13e360 0x7f4fec0cf290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:59.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:59.313+0000 7f4ff21b0640 1 -- 172.21.15.92:0/3836968564 >> 172.21.15.92:0/3836968564 conn(0x7f4fec06a800 msgr2=0x7f4fec06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:59.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:59.313+0000 7f4ff21b0640 1 -- 172.21.15.92:0/3836968564 shutdown_connections 2023-11-24T06:12:59.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:59.313+0000 7f4ff21b0640 1 -- 172.21.15.92:0/3836968564 wait complete. 2023-11-24T06:12:59.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:59.315+0000 7f4ff21b0640 1 Processor -- start 2023-11-24T06:12:59.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:59.315+0000 7f4ff21b0640 1 -- start start 2023-11-24T06:12:59.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:59.316+0000 7f4ff21b0640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4fec13df80 0x7f4fec150c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:59.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:59.316+0000 7f4ff21b0640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4fec13e360 0x7f4fec0b2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:59.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:59.316+0000 7f4ff21b0640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4fec15f9c0 0x7f4fec0b3230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:59.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:59.316+0000 7f4ff21b0640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fec14b2a0 con 0x7f4fec13df80 2023-11-24T06:12:59.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:59.316+0000 7f4ff21b0640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fec151180 con 0x7f4fec15f9c0 2023-11-24T06:12:59.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:59.316+0000 7f4ff21b0640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fec14bd00 con 0x7f4fec13e360 2023-11-24T06:12:59.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:59.316+0000 7f4feb7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4fec13df80 0x7f4fec150c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:59.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:59.316+0000 7f4feb7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4fec13df80 0x7f4fec150c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:54450/0 (socket says 172.21.15.92:54450) 2023-11-24T06:12:59.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:59.316+0000 7f4feb7fe640 1 -- 172.21.15.92:0/2214082875 learned_addr learned my addr 172.21.15.92:0/2214082875 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:12:59.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:59.316+0000 7f4febfff640 1 --2- 172.21.15.92:0/2214082875 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4fec15f9c0 0x7f4fec0b3230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:59.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:59.317+0000 7f4feaffd640 1 --2- 172.21.15.92:0/2214082875 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4fec13e360 0x7f4fec0b2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:59.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:59.317+0000 7f4feb7fe640 1 -- 172.21.15.92:0/2214082875 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4fec13e360 msgr2=0x7f4fec0b2cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:59.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:59.317+0000 7f4feb7fe640 1 --2- 172.21.15.92:0/2214082875 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4fec13e360 0x7f4fec0b2cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:59.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:59.317+0000 7f4feb7fe640 1 -- 172.21.15.92:0/2214082875 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4fec15f9c0 msgr2=0x7f4fec0b3230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:59.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:59.317+0000 7f4feb7fe640 1 --2- 172.21.15.92:0/2214082875 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4fec15f9c0 0x7f4fec0b3230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:59.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:59.317+0000 7f4feb7fe640 1 -- 172.21.15.92:0/2214082875 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fe40094d0 con 0x7f4fec13df80 2023-11-24T06:12:59.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:59.317+0000 7f4feaffd640 1 --2- 172.21.15.92:0/2214082875 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4fec13e360 0x7f4fec0b2cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:12:59.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:59.317+0000 7f4feb7fe640 1 --2- 172.21.15.92:0/2214082875 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4fec13df80 0x7f4fec150c40 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f4fdc00c760 tx=0x7f4fdc00cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:59.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:59.317+0000 7f4fe3fff640 1 -- 172.21.15.92:0/2214082875 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fdc003b90 con 0x7f4fec13df80 2023-11-24T06:12:59.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:59.317+0000 7f4fe3fff640 1 -- 172.21.15.92:0/2214082875 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fdc003cf0 con 0x7f4fec13df80 2023-11-24T06:12:59.323 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:59.317+0000 7f4fe3fff640 1 -- 172.21.15.92:0/2214082875 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fdc003ea0 con 0x7f4fec13df80 2023-11-24T06:12:59.323 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:59.317+0000 7f4ff21b0640 1 -- 172.21.15.92:0/2214082875 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fec14b4f0 con 0x7f4fec13df80 2023-11-24T06:12:59.323 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:59.317+0000 7f4ff21b0640 1 -- 172.21.15.92:0/2214082875 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fec14bfc0 con 0x7f4fec13df80 2023-11-24T06:12:59.323 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:59.318+0000 7f4ff21b0640 1 -- 172.21.15.92:0/2214082875 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fec0675d0 con 0x7f4fec13df80 2023-11-24T06:12:59.323 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:59.319+0000 7f4fe3fff640 1 -- 172.21.15.92:0/2214082875 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f4fdc00aa90 con 0x7f4fec13df80 2023-11-24T06:12:59.323 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:59.320+0000 7f4fe3fff640 1 --2- 172.21.15.92:0/2214082875 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f4fcc06aed0 0x7f4fcc06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:59.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:59.320+0000 7f4fe3fff640 1 -- 172.21.15.92:0/2214082875 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f4fdc089300 con 0x7f4fec13df80 2023-11-24T06:12:59.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:59.321+0000 7f4feaffd640 1 --2- 172.21.15.92:0/2214082875 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f4fcc06aed0 0x7f4fcc06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:59.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:59.321+0000 7f4feaffd640 1 --2- 172.21.15.92:0/2214082875 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f4fcc06aed0 0x7f4fcc06d390 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f4fec1536d0 tx=0x7f4fd4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:59.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:59.325+0000 7f4fe3fff640 1 -- 172.21.15.92:0/2214082875 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4fdc054530 con 0x7f4fec13df80 2023-11-24T06:12:59.519 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:59.516+0000 7f4ff21b0640 1 -- 172.21.15.92:0/2214082875 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f4fec06df00 con 0x7f4fec13df80 2023-11-24T06:12:59.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:59.523+0000 7f4fe3fff640 1 -- 172.21.15.92:0/2214082875 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v13) v1 ==== 104+0+0 (secure 0 0 0) 0x7f4fdc057b50 con 0x7f4fec13df80 2023-11-24T06:12:59.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:59.526+0000 7f4ff21b0640 1 -- 172.21.15.92:0/2214082875 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f4fcc06aed0 msgr2=0x7f4fcc06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:59.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:59.526+0000 7f4ff21b0640 1 --2- 172.21.15.92:0/2214082875 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f4fcc06aed0 0x7f4fcc06d390 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f4fec1536d0 tx=0x7f4fd4008040 comp rx=0 tx=0).stop 2023-11-24T06:12:59.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:59.526+0000 7f4ff21b0640 1 -- 172.21.15.92:0/2214082875 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4fec13df80 msgr2=0x7f4fec150c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:59.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:59.526+0000 7f4ff21b0640 1 --2- 172.21.15.92:0/2214082875 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4fec13df80 0x7f4fec150c40 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f4fdc00c760 tx=0x7f4fdc00cc20 comp rx=0 tx=0).stop 2023-11-24T06:12:59.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:59.526+0000 7f4ff21b0640 1 -- 172.21.15.92:0/2214082875 shutdown_connections 2023-11-24T06:12:59.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:59.526+0000 7f4ff21b0640 1 --2- 172.21.15.92:0/2214082875 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f4fcc06aed0 0x7f4fcc06d390 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:59.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:59.526+0000 7f4ff21b0640 1 --2- 172.21.15.92:0/2214082875 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4fec15f9c0 0x7f4fec0b3230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:59.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:59.526+0000 7f4ff21b0640 1 --2- 172.21.15.92:0/2214082875 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4fec13e360 0x7f4fec0b2cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:59.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:59.526+0000 7f4ff21b0640 1 --2- 172.21.15.92:0/2214082875 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4fec13df80 0x7f4fec150c40 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:59.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:59.526+0000 7f4ff21b0640 1 -- 172.21.15.92:0/2214082875 >> 172.21.15.92:0/2214082875 conn(0x7f4fec06a800 msgr2=0x7f4fec153ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:59.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:59.526+0000 7f4ff21b0640 1 -- 172.21.15.92:0/2214082875 shutdown_connections 2023-11-24T06:12:59.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:59.526+0000 7f4ff21b0640 1 -- 172.21.15.92:0/2214082875 wait complete. 2023-11-24T06:12:59.548 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestVolumes.test_volume_info' 2023-11-24T06:12:59.758 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:59.755+0000 7f58dcbce640 1 Processor -- start 2023-11-24T06:12:59.758 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:59.755+0000 7f58dcbce640 1 -- start start 2023-11-24T06:12:59.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:59.757+0000 7f58dcbce640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f58d813df80 0x7f58d815bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:59.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:59.757+0000 7f58dcbce640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f58d80cf9a0 0x7f58d80cd840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:59.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:59.757+0000 7f58dcbce640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58d806c470 con 0x7f58d813e360 2023-11-24T06:12:59.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:59.757+0000 7f58dcbce640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58d806c750 con 0x7f58d80cf9a0 2023-11-24T06:12:59.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:59.757+0000 7f58dcbce640 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58d806ca30 con 0x7f58d813df80 2023-11-24T06:12:59.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:59.757+0000 7f58d6d9d640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f58d80cf9a0 0x7f58d80cd840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:59.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:59.757+0000 7f58cffff640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f58d813e360 0x7f58d80cf290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:47856/0 (socket says 172.21.15.92:47856) 2023-11-24T06:12:59.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:59.757+0000 7f58cffff640 1 -- 172.21.15.92:0/3145673647 learned_addr learned my addr 172.21.15.92:0/3145673647 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-24T06:12:59.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:59.758+0000 7f58d5d9b640 1 -- 172.21.15.92:0/3145673647 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1807737501 0 0) 0x7f58d806c470 con 0x7f58d813e360 2023-11-24T06:12:59.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:59.758+0000 7f58d5d9b640 1 -- 172.21.15.92:0/3145673647 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f58c4003520 con 0x7f58d813e360 2023-11-24T06:12:59.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:59.758+0000 7f58d659c640 1 --2- 172.21.15.92:0/3145673647 >> v2:172.21.15.118:3300/0 conn(0x7f58d813df80 0x7f58d815bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:59.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:59.758+0000 7f58d6d9d640 1 -- 172.21.15.92:0/3145673647 >> v2:172.21.15.118:3300/0 conn(0x7f58d813df80 msgr2=0x7f58d815bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:59.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:59.758+0000 7f58d6d9d640 1 --2- 172.21.15.92:0/3145673647 >> v2:172.21.15.118:3300/0 conn(0x7f58d813df80 0x7f58d815bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:59.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:59.758+0000 7f58d6d9d640 1 -- 172.21.15.92:0/3145673647 >> v1:172.21.15.92:6789/0 conn(0x7f58d813e360 legacy=0x7f58d80cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:59.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:59.758+0000 7f58d6d9d640 1 -- 172.21.15.92:0/3145673647 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58d806cd10 con 0x7f58d80cf9a0 2023-11-24T06:12:59.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:59.758+0000 7f58d659c640 1 --2- 172.21.15.92:0/3145673647 >> v2:172.21.15.118:3300/0 conn(0x7f58d813df80 0x7f58d815bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:12:59.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:59.759+0000 7f58d6d9d640 1 --2- 172.21.15.92:0/3145673647 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f58d80cf9a0 0x7f58d80cd840 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f58d8072770 tx=0x7f58d002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a53619de70ec063 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:59.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:59.759+0000 7f58d5d9b640 1 -- 172.21.15.92:0/3145673647 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58d002cc80 con 0x7f58d80cf9a0 2023-11-24T06:12:59.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:59.759+0000 7f58d5d9b640 1 -- 172.21.15.92:0/3145673647 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58d002cde0 con 0x7f58d80cf9a0 2023-11-24T06:12:59.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:59.759+0000 7f58d5d9b640 1 -- 172.21.15.92:0/3145673647 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58d0034660 con 0x7f58d80cf9a0 2023-11-24T06:12:59.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:59.760+0000 7f58dcbce640 1 -- 172.21.15.92:0/3145673647 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f58d80cf9a0 msgr2=0x7f58d80cd840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:59.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:59.760+0000 7f58dcbce640 1 --2- 172.21.15.92:0/3145673647 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f58d80cf9a0 0x7f58d80cd840 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f58d8072770 tx=0x7f58d002bed0 comp rx=0 tx=0).stop 2023-11-24T06:12:59.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:59.760+0000 7f58dcbce640 1 -- 172.21.15.92:0/3145673647 shutdown_connections 2023-11-24T06:12:59.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:59.760+0000 7f58dcbce640 1 --2- 172.21.15.92:0/3145673647 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f58d80cf9a0 0x7f58d80cd840 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:59.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:59.760+0000 7f58dcbce640 1 --2- 172.21.15.92:0/3145673647 >> v2:172.21.15.118:3300/0 conn(0x7f58d813df80 0x7f58d815bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:59.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:59.760+0000 7f58dcbce640 1 -- 172.21.15.92:0/3145673647 >> 172.21.15.92:0/3145673647 conn(0x7f58d806a800 msgr2=0x7f58d806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:59.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:59.760+0000 7f58dcbce640 1 -- 172.21.15.92:0/3145673647 shutdown_connections 2023-11-24T06:12:59.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:59.760+0000 7f58dcbce640 1 -- 172.21.15.92:0/3145673647 wait complete. 2023-11-24T06:12:59.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:59.762+0000 7f58dcbce640 1 Processor -- start 2023-11-24T06:12:59.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:59.762+0000 7f58dcbce640 1 -- start start 2023-11-24T06:12:59.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:59.763+0000 7f58dcbce640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f58d80cf9a0 0x7f58d813d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:59.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:59.763+0000 7f58dcbce640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f58d813df80 0x7f58d81f2f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:59.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:59.763+0000 7f58dcbce640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f58d813e360 0x7f58d81f3440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:59.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:59.763+0000 7f58dcbce640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58d813cc10 con 0x7f58d813e360 2023-11-24T06:12:59.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:59.763+0000 7f58dcbce640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58d81f3a40 con 0x7f58d813df80 2023-11-24T06:12:59.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:59.763+0000 7f58dcbce640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58d81f3d20 con 0x7f58d80cf9a0 2023-11-24T06:12:59.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:59.763+0000 7f58d6d9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f58d813e360 0x7f58d81f3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:59.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:59.763+0000 7f58d6d9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f58d813e360 0x7f58d81f3440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:54464/0 (socket says 172.21.15.92:54464) 2023-11-24T06:12:59.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:59.763+0000 7f58d6d9d640 1 -- 172.21.15.92:0/3161471875 learned_addr learned my addr 172.21.15.92:0/3161471875 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:12:59.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:59.764+0000 7f58d659c640 1 --2- 172.21.15.92:0/3161471875 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f58d80cf9a0 0x7f58d813d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:59.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:59.764+0000 7f58cffff640 1 --2- 172.21.15.92:0/3161471875 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f58d813df80 0x7f58d81f2f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:59.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:59.764+0000 7f58d6d9d640 1 -- 172.21.15.92:0/3161471875 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f58d80cf9a0 msgr2=0x7f58d813d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:59.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:59.764+0000 7f58d6d9d640 1 --2- 172.21.15.92:0/3161471875 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f58d80cf9a0 0x7f58d813d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:59.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:59.764+0000 7f58d6d9d640 1 -- 172.21.15.92:0/3161471875 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f58d813df80 msgr2=0x7f58d81f2f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:59.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:59.764+0000 7f58d6d9d640 1 --2- 172.21.15.92:0/3161471875 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f58d813df80 0x7f58d81f2f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:59.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:59.764+0000 7f58d6d9d640 1 -- 172.21.15.92:0/3161471875 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58d0009a70 con 0x7f58d813e360 2023-11-24T06:12:59.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:59.764+0000 7f58d6d9d640 1 --2- 172.21.15.92:0/3161471875 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f58d813e360 0x7f58d81f3440 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f58d002c4e0 tx=0x7f58d0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:59.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:59.765+0000 7f58ceffd640 1 -- 172.21.15.92:0/3161471875 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58d003e610 con 0x7f58d813e360 2023-11-24T06:12:59.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:59.765+0000 7f58ceffd640 1 -- 172.21.15.92:0/3161471875 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58d003e770 con 0x7f58d813e360 2023-11-24T06:12:59.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:59.765+0000 7f58ceffd640 1 -- 172.21.15.92:0/3161471875 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58d003eb60 con 0x7f58d813e360 2023-11-24T06:12:59.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:59.765+0000 7f58dcbce640 1 -- 172.21.15.92:0/3161471875 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58d813ce60 con 0x7f58d813e360 2023-11-24T06:12:59.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:59.765+0000 7f58dcbce640 1 -- 172.21.15.92:0/3161471875 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58d80aa090 con 0x7f58d813e360 2023-11-24T06:12:59.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:59.766+0000 7f58dcbce640 1 -- 172.21.15.92:0/3161471875 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58d813d380 con 0x7f58d813e360 2023-11-24T06:12:59.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:59.767+0000 7f58ceffd640 1 -- 172.21.15.92:0/3161471875 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f58d003b050 con 0x7f58d813e360 2023-11-24T06:12:59.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:59.768+0000 7f58ceffd640 1 --2- 172.21.15.92:0/3161471875 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f58b806aed0 0x7f58b806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:59.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:59.769+0000 7f58ceffd640 1 -- 172.21.15.92:0/3161471875 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f58d00ab560 con 0x7f58d813e360 2023-11-24T06:12:59.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:59.769+0000 7f58d659c640 1 --2- 172.21.15.92:0/3161471875 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f58b806aed0 0x7f58b806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:59.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:59.769+0000 7f58d659c640 1 --2- 172.21.15.92:0/3161471875 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f58b806aed0 0x7f58b806d390 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f58c800a990 tx=0x7f58c8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:59.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:12:59.774+0000 7f58ceffd640 1 -- 172.21.15.92:0/3161471875 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f58d00768a0 con 0x7f58d813e360 2023-11-24T06:13:00.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:00.032+0000 7f58dcbce640 1 -- 172.21.15.92:0/3161471875 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestVolumes.test_volume_info"]} v 0) v1 -- 0x7f58d80675d0 con 0x7f58d813e360 2023-11-24T06:13:00.071 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:00.068+0000 7f58ceffd640 1 -- 172.21.15.92:0/3161471875 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestVolumes.test_volume_info"]}]=0 v166) v1 ==== 133+0+0 (secure 0 0 0) 0x7f58d0009450 con 0x7f58d813e360 2023-11-24T06:13:00.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:00.070+0000 7f58dcbce640 1 -- 172.21.15.92:0/3161471875 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f58b806aed0 msgr2=0x7f58b806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:00.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:00.071+0000 7f58dcbce640 1 --2- 172.21.15.92:0/3161471875 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f58b806aed0 0x7f58b806d390 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f58c800a990 tx=0x7f58c8008040 comp rx=0 tx=0).stop 2023-11-24T06:13:00.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:00.071+0000 7f58dcbce640 1 -- 172.21.15.92:0/3161471875 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f58d813e360 msgr2=0x7f58d81f3440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:00.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:00.071+0000 7f58dcbce640 1 --2- 172.21.15.92:0/3161471875 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f58d813e360 0x7f58d81f3440 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f58d002c4e0 tx=0x7f58d0004a40 comp rx=0 tx=0).stop 2023-11-24T06:13:00.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:00.071+0000 7f58dcbce640 1 -- 172.21.15.92:0/3161471875 shutdown_connections 2023-11-24T06:13:00.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:00.071+0000 7f58dcbce640 1 --2- 172.21.15.92:0/3161471875 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f58b806aed0 0x7f58b806d390 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:00.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:00.071+0000 7f58dcbce640 1 --2- 172.21.15.92:0/3161471875 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f58d813e360 0x7f58d81f3440 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:00.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:00.071+0000 7f58dcbce640 1 --2- 172.21.15.92:0/3161471875 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f58d813df80 0x7f58d81f2f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:00.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:00.071+0000 7f58dcbce640 1 --2- 172.21.15.92:0/3161471875 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f58d80cf9a0 0x7f58d813d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:00.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:00.071+0000 7f58dcbce640 1 -- 172.21.15.92:0/3161471875 >> 172.21.15.92:0/3161471875 conn(0x7f58d806a800 msgr2=0x7f58d80b0060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:13:00.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:00.071+0000 7f58dcbce640 1 -- 172.21.15.92:0/3161471875 shutdown_connections 2023-11-24T06:13:00.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:00.071+0000 7f58dcbce640 1 -- 172.21.15.92:0/3161471875 wait complete. 2023-11-24T06:13:00.093 INFO:tasks.cephfs_test_runner:Tests the 'fs volume info' command ... ok 2023-11-24T06:13:00.093 INFO:tasks.cephfs_test_runner:Starting test: test_volume_info_pending_subvol_deletions (tasks.cephfs.test_volumes.TestVolumes) Tests the pending_subvolume_deletions in 'fs volume info' command 2023-11-24T06:13:00.093 INFO:tasks.cephfs_test_runner:test_volume_info_pending_subvol_deletions (tasks.cephfs.test_volumes.TestVolumes) 2023-11-24T06:13:00.094 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:13:00.305 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:00.302+0000 7fcbdb4ff640 1 Processor -- start 2023-11-24T06:13:00.305 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:00.302+0000 7fcbdb4ff640 1 -- start start 2023-11-24T06:13:00.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:00.304+0000 7fcbdb4ff640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fcbd4143c40 0x7fcbd4071290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:00.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:00.304+0000 7fcbdb4ff640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fcbd40717d0 0x7fcbd40cc500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:00.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:00.304+0000 7fcbdb4ff640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fcbd40cca40 0x7fcbd415eab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:00.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:00.304+0000 7fcbdb4ff640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcbd40723c0 con 0x7fcbd40cca40 2023-11-24T06:13:00.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:00.304+0000 7fcbdb4ff640 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcbd4070e50 con 0x7fcbd4143c40 2023-11-24T06:13:00.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:00.304+0000 7fcbdb4ff640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbd40cce70 con 0x7fcbd40717d0 2023-11-24T06:13:00.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:00.305+0000 7fcbda4fd640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fcbd4143c40 0x7fcbd4071290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:00.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:00.306+0000 7fcbda4fd640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fcbd4143c40 0x7fcbd4071290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.92:46022/0 (socket says 172.21.15.92:46022) 2023-11-24T06:13:00.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:00.306+0000 7fcbda4fd640 1 -- 172.21.15.92:0/3067532428 learned_addr learned my addr 172.21.15.92:0/3067532428 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:13:00.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:00.306+0000 7fcbda4fd640 1 -- 172.21.15.92:0/3067532428 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fcbd40717d0 msgr2=0x7fcbd40cc500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:13:00.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:00.306+0000 7fcbd9cfc640 1 --2- 172.21.15.92:0/3067532428 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fcbd40717d0 0x7fcbd40cc500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:00.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:00.307+0000 7fcbda4fd640 1 --2- 172.21.15.92:0/3067532428 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fcbd40717d0 0x7fcbd40cc500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:00.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:00.307+0000 7fcbda4fd640 1 -- 172.21.15.92:0/3067532428 >> v2:172.21.15.92:3300/0 conn(0x7fcbd40cca40 msgr2=0x7fcbd415eab0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-24T06:13:00.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:00.307+0000 7fcbda4fd640 1 --2- 172.21.15.92:0/3067532428 >> v2:172.21.15.92:3300/0 conn(0x7fcbd40cca40 0x7fcbd415eab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:00.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:00.307+0000 7fcbda4fd640 1 -- 172.21.15.92:0/3067532428 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbd4144020 con 0x7fcbd4143c40 2023-11-24T06:13:00.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:00.307+0000 7fcbd9cfc640 1 --2- 172.21.15.92:0/3067532428 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fcbd40717d0 0x7fcbd40cc500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:13:00.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:00.310+0000 7fcbda4fd640 1 --2- 172.21.15.92:0/3067532428 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fcbd4143c40 0x7fcbd4071290 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fcbd406e410 tx=0x7fcbc802be70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5b54662616ac9ee9 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:00.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:00.310+0000 7fcbd94fb640 1 -- 172.21.15.92:0/3067532428 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbc802ca10 con 0x7fcbd4143c40 2023-11-24T06:13:00.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:00.311+0000 7fcbd94fb640 1 -- 172.21.15.92:0/3067532428 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbc802cb70 con 0x7fcbd4143c40 2023-11-24T06:13:00.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:00.311+0000 7fcbd94fb640 1 -- 172.21.15.92:0/3067532428 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbc80343f0 con 0x7fcbd4143c40 2023-11-24T06:13:00.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:00.312+0000 7fcbdb4ff640 1 -- 172.21.15.92:0/3067532428 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fcbd4143c40 msgr2=0x7fcbd4071290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:00.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:00.312+0000 7fcbdb4ff640 1 --2- 172.21.15.92:0/3067532428 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fcbd4143c40 0x7fcbd4071290 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fcbd406e410 tx=0x7fcbc802be70 comp rx=0 tx=0).stop 2023-11-24T06:13:00.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:00.312+0000 7fcbdb4ff640 1 -- 172.21.15.92:0/3067532428 shutdown_connections 2023-11-24T06:13:00.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:00.312+0000 7fcbdb4ff640 1 --2- 172.21.15.92:0/3067532428 >> v2:172.21.15.92:3300/0 conn(0x7fcbd40cca40 0x7fcbd415eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:00.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:00.312+0000 7fcbdb4ff640 1 --2- 172.21.15.92:0/3067532428 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fcbd40717d0 0x7fcbd40cc500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:00.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:00.312+0000 7fcbdb4ff640 1 --2- 172.21.15.92:0/3067532428 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fcbd4143c40 0x7fcbd4071290 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:00.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:00.312+0000 7fcbdb4ff640 1 -- 172.21.15.92:0/3067532428 >> 172.21.15.92:0/3067532428 conn(0x7fcbd406a800 msgr2=0x7fcbd406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:13:00.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:00.313+0000 7fcbdb4ff640 1 -- 172.21.15.92:0/3067532428 shutdown_connections 2023-11-24T06:13:00.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:00.313+0000 7fcbdb4ff640 1 -- 172.21.15.92:0/3067532428 wait complete. 2023-11-24T06:13:00.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:00.316+0000 7fcbdb4ff640 1 Processor -- start 2023-11-24T06:13:00.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:00.316+0000 7fcbdb4ff640 1 -- start start 2023-11-24T06:13:00.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:00.317+0000 7fcbdb4ff640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fcbd40717d0 0x7fcbd415a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:00.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:00.317+0000 7fcbdb4ff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcbd40cca40 0x7fcbd40ac8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:00.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:00.317+0000 7fcbdb4ff640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fcbd4143c40 0x7fcbd40ace10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:00.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:00.317+0000 7fcbdb4ff640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbd415ad10 con 0x7fcbd40cca40 2023-11-24T06:13:00.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:00.317+0000 7fcbdb4ff640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbd40cae00 con 0x7fcbd40717d0 2023-11-24T06:13:00.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:00.317+0000 7fcbdb4ff640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbd40cb0e0 con 0x7fcbd4143c40 2023-11-24T06:13:00.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:00.317+0000 7fcbd9cfc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcbd40cca40 0x7fcbd40ac8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:00.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:00.317+0000 7fcbd9cfc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcbd40cca40 0x7fcbd40ac8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:54470/0 (socket says 172.21.15.92:54470) 2023-11-24T06:13:00.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:00.317+0000 7fcbd9cfc640 1 -- 172.21.15.92:0/196370031 learned_addr learned my addr 172.21.15.92:0/196370031 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:13:00.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:00.317+0000 7fcbda4fd640 1 --2- 172.21.15.92:0/196370031 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fcbd40717d0 0x7fcbd415a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:00.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:00.317+0000 7fcbdacfe640 1 --2- 172.21.15.92:0/196370031 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fcbd4143c40 0x7fcbd40ace10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:00.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:00.317+0000 7fcbd9cfc640 1 -- 172.21.15.92:0/196370031 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fcbd4143c40 msgr2=0x7fcbd40ace10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:00.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:00.317+0000 7fcbd9cfc640 1 --2- 172.21.15.92:0/196370031 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fcbd4143c40 0x7fcbd40ace10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:00.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:00.317+0000 7fcbd9cfc640 1 -- 172.21.15.92:0/196370031 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fcbd40717d0 msgr2=0x7fcbd415a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:00.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:00.317+0000 7fcbd9cfc640 1 --2- 172.21.15.92:0/196370031 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fcbd40717d0 0x7fcbd415a7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:00.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:00.317+0000 7fcbd9cfc640 1 -- 172.21.15.92:0/196370031 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbc8009b30 con 0x7fcbd40cca40 2023-11-24T06:13:00.323 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:00.317+0000 7fcbda4fd640 1 --2- 172.21.15.92:0/196370031 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fcbd40717d0 0x7fcbd415a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:13:00.323 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:00.317+0000 7fcbdacfe640 1 --2- 172.21.15.92:0/196370031 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fcbd4143c40 0x7fcbd40ace10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:13:00.323 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:00.318+0000 7fcbd9cfc640 1 --2- 172.21.15.92:0/196370031 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcbd40cca40 0x7fcbd40ac8d0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fcbc000c7f0 tx=0x7fcbc000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:00.323 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:00.318+0000 7fcbd2ffd640 1 -- 172.21.15.92:0/196370031 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbc0007820 con 0x7fcbd40cca40 2023-11-24T06:13:00.323 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:00.318+0000 7fcbd2ffd640 1 -- 172.21.15.92:0/196370031 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbc0007980 con 0x7fcbd40cca40 2023-11-24T06:13:00.323 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:00.318+0000 7fcbd2ffd640 1 -- 172.21.15.92:0/196370031 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbc0007cf0 con 0x7fcbd40cca40 2023-11-24T06:13:00.323 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:00.319+0000 7fcbdb4ff640 1 -- 172.21.15.92:0/196370031 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbd40cb3c0 con 0x7fcbd40cca40 2023-11-24T06:13:00.323 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:00.319+0000 7fcbdb4ff640 1 -- 172.21.15.92:0/196370031 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbd40c4050 con 0x7fcbd40cca40 2023-11-24T06:13:00.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:00.320+0000 7fcbdb4ff640 1 -- 172.21.15.92:0/196370031 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcbd40c4210 con 0x7fcbd40cca40 2023-11-24T06:13:00.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:00.321+0000 7fcbd2ffd640 1 -- 172.21.15.92:0/196370031 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7fcbc00113f0 con 0x7fcbd40cca40 2023-11-24T06:13:00.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:00.322+0000 7fcbd2ffd640 1 --2- 172.21.15.92:0/196370031 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fcbb806aed0 0x7fcbb806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:00.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:00.322+0000 7fcbd2ffd640 1 -- 172.21.15.92:0/196370031 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fcbc0089760 con 0x7fcbd40cca40 2023-11-24T06:13:00.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:00.322+0000 7fcbda4fd640 1 --2- 172.21.15.92:0/196370031 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fcbb806aed0 0x7fcbb806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:00.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:00.323+0000 7fcbda4fd640 1 --2- 172.21.15.92:0/196370031 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fcbb806aed0 0x7fcbb806d390 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fcbc8035000 tx=0x7fcbc8030000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:00.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:00.326+0000 7fcbd2ffd640 1 -- 172.21.15.92:0/196370031 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fcbc0054a10 con 0x7fcbd40cca40 2023-11-24T06:13:00.519 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:00.516+0000 7fcbdb4ff640 1 -- 172.21.15.92:0/196370031 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcbd40cd970 con 0x7fcbd40cca40 2023-11-24T06:13:00.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:00.517+0000 7fcbd2ffd640 1 -- 172.21.15.92:0/196370031 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v55) v1 ==== 74+0+12938 (secure 0 0 0) 0x7fcbc0058030 con 0x7fcbd40cca40 2023-11-24T06:13:00.520 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-24T06:13:00.521 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":55,"fsid":"2d6dee5f-8aa3-470f-931a-50ade963b1cb","created":"2023-11-24T06:10:04.707204+0000","modified":"2023-11-24T06:12:59.062203+0000","last_up_change":"2023-11-24T06:10:17.598352+0000","last_in_change":"2023-11-24T06:10:09.040793+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:10:17.733098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ff37a9fb-135f-471f-b4b0-f7b5772006c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6803","nonce":20560714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6805","nonce":20560714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6809","nonce":20560714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6807","nonce":20560714}]},"public_addr":"172.21.15.92:6803/20560714","cluster_addr":"172.21.15.92:6805/20560714","heartbeat_back_addr":"172.21.15.92:6809/20560714","heartbeat_front_addr":"172.21.15.92:6807/20560714","state":["exists","up"]},{"osd":1,"uuid":"03cdd353-dcf1-4f38-94fe-a795137c06b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6819","nonce":2451239906}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6821","nonce":2451239906}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6825","nonce":2451239906}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6823","nonce":2451239906}]},"public_addr":"172.21.15.92:6819/2451239906","cluster_addr":"172.21.15.92:6821/2451239906","heartbeat_back_addr":"172.21.15.92:6825/2451239906","heartbeat_front_addr":"172.21.15.92:6823/2451239906","state":["exists","up"]},{"osd":2,"uuid":"a4a20d80-5d57-484b-becd-9aa8070dd3aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6811","nonce":1124979277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6813","nonce":1124979277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6817","nonce":1124979277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6815","nonce":1124979277}]},"public_addr":"172.21.15.92:6811/1124979277","cluster_addr":"172.21.15.92:6813/1124979277","heartbeat_back_addr":"172.21.15.92:6817/1124979277","heartbeat_front_addr":"172.21.15.92:6815/1124979277","state":["exists","up"]},{"osd":3,"uuid":"4648d7ce-9b29-4583-9425-4ba8fd2b1903","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6827","nonce":1734115782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6829","nonce":1734115782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6833","nonce":1734115782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6831","nonce":1734115782}]},"public_addr":"172.21.15.92:6827/1734115782","cluster_addr":"172.21.15.92:6829/1734115782","heartbeat_back_addr":"172.21.15.92:6833/1734115782","heartbeat_front_addr":"172.21.15.92:6831/1734115782","state":["exists","up"]},{"osd":4,"uuid":"3bc9d7eb-c6fa-4fcf-a3ec-ade604e05aac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6819","nonce":3425937229}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6822","nonce":3425937229}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6830","nonce":3425937229}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6826","nonce":3425937229}]},"public_addr":"172.21.15.118:6819/3425937229","cluster_addr":"172.21.15.118:6822/3425937229","heartbeat_back_addr":"172.21.15.118:6830/3425937229","heartbeat_front_addr":"172.21.15.118:6826/3425937229","state":["exists","up"]},{"osd":5,"uuid":"dce9ba6d-59ed-46d1-84fc-a84b13de10c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6801","nonce":827455984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6803","nonce":827455984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6807","nonce":827455984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6805","nonce":827455984}]},"public_addr":"172.21.15.118:6801/827455984","cluster_addr":"172.21.15.118:6803/827455984","heartbeat_back_addr":"172.21.15.118:6807/827455984","heartbeat_front_addr":"172.21.15.118:6805/827455984","state":["exists","up"]},{"osd":6,"uuid":"82aec504-ae58-4709-8d30-46f2b12081b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6809","nonce":2946986141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6811","nonce":2946986141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6815","nonce":2946986141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6813","nonce":2946986141}]},"public_addr":"172.21.15.118:6809/2946986141","cluster_addr":"172.21.15.118:6811/2946986141","heartbeat_back_addr":"172.21.15.118:6815/2946986141","heartbeat_front_addr":"172.21.15.118:6813/2946986141","state":["exists","up"]},{"osd":7,"uuid":"4ed4c624-c88f-4d35-bdb1-bf209b5dffb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6817","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6818","nonce":48122059}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6821","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6823","nonce":48122059}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6829","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6831","nonce":48122059}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6825","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6827","nonce":48122059}]},"public_addr":"172.21.15.118:6818/48122059","cluster_addr":"172.21.15.118:6823/48122059","heartbeat_back_addr":"172.21.15.118:6831/48122059","heartbeat_front_addr":"172.21.15.118:6827/48122059","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.536664+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.368324+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.271201+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.342971+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.621832+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.387110+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.632946+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.416032+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.92:6837/1958912045":"2023-11-25T06:12:56.051028+0000","172.21.15.92:6835/1958912045":"2023-11-25T06:12:56.051028+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:13:00.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:00.520+0000 7fcbdb4ff640 1 -- 172.21.15.92:0/196370031 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fcbb806aed0 msgr2=0x7fcbb806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:00.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:00.520+0000 7fcbdb4ff640 1 --2- 172.21.15.92:0/196370031 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fcbb806aed0 0x7fcbb806d390 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fcbc8035000 tx=0x7fcbc8030000 comp rx=0 tx=0).stop 2023-11-24T06:13:00.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:00.520+0000 7fcbdb4ff640 1 -- 172.21.15.92:0/196370031 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcbd40cca40 msgr2=0x7fcbd40ac8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:00.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:00.520+0000 7fcbdb4ff640 1 --2- 172.21.15.92:0/196370031 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcbd40cca40 0x7fcbd40ac8d0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fcbc000c7f0 tx=0x7fcbc000ccb0 comp rx=0 tx=0).stop 2023-11-24T06:13:00.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:00.520+0000 7fcbdb4ff640 1 -- 172.21.15.92:0/196370031 shutdown_connections 2023-11-24T06:13:00.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:00.520+0000 7fcbdb4ff640 1 --2- 172.21.15.92:0/196370031 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fcbb806aed0 0x7fcbb806d390 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:00.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:00.520+0000 7fcbdb4ff640 1 --2- 172.21.15.92:0/196370031 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fcbd4143c40 0x7fcbd40ace10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:00.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:00.520+0000 7fcbdb4ff640 1 --2- 172.21.15.92:0/196370031 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcbd40cca40 0x7fcbd40ac8d0 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:00.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:00.520+0000 7fcbdb4ff640 1 --2- 172.21.15.92:0/196370031 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fcbd40717d0 0x7fcbd415a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:00.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:00.520+0000 7fcbdb4ff640 1 -- 172.21.15.92:0/196370031 >> 172.21.15.92:0/196370031 conn(0x7fcbd406a800 msgr2=0x7fcbd413f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:13:00.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:00.521+0000 7fcbdb4ff640 1 -- 172.21.15.92:0/196370031 shutdown_connections 2023-11-24T06:13:00.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:00.521+0000 7fcbdb4ff640 1 -- 172.21.15.92:0/196370031 wait complete. 2023-11-24T06:13:00.543 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T06:10:17.733098+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T06:13:00.543 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T06:13:00.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:00.741+0000 7fcf965ac640 1 Processor -- start 2023-11-24T06:13:00.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:00.741+0000 7fcf965ac640 1 -- start start 2023-11-24T06:13:00.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:00.742+0000 7fcf965ac640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fcf90143810 0x7fcf90071e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:00.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:00.742+0000 7fcf965ac640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fcf900ae170 0x7fcf9015e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:00.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:00.743+0000 7fcf965ac640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fcf9015d770 0x7fcf9015dc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:00.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:00.743+0000 7fcf965ac640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf90143bf0 con 0x7fcf90143810 2023-11-24T06:13:00.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:00.743+0000 7fcf965ac640 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf900ae620 con 0x7fcf900ae170 2023-11-24T06:13:00.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:00.743+0000 7fcf965ac640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf9015e140 con 0x7fcf9015d770 2023-11-24T06:13:00.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:00.743+0000 7fcf955aa640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fcf90143810 0x7fcf90071e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:00.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:00.743+0000 7fcf955aa640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fcf90143810 0x7fcf90071e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:54472/0 (socket says 172.21.15.92:54472) 2023-11-24T06:13:00.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:00.743+0000 7fcf955aa640 1 -- 172.21.15.92:0/1527346229 learned_addr learned my addr 172.21.15.92:0/1527346229 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:13:00.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:00.743+0000 7fcf955aa640 1 -- 172.21.15.92:0/1527346229 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fcf9015d770 msgr2=0x7fcf9015dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:13:00.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:00.743+0000 7fcf95dab640 1 --2- 172.21.15.92:0/1527346229 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fcf9015d770 0x7fcf9015dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:00.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:00.743+0000 7fcf94da9640 1 --2- 172.21.15.92:0/1527346229 >> v2:172.21.15.118:3300/0 conn(0x7fcf900ae170 0x7fcf9015e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:00.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:00.743+0000 7fcf955aa640 1 --2- 172.21.15.92:0/1527346229 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fcf9015d770 0x7fcf9015dc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:00.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:00.743+0000 7fcf955aa640 1 -- 172.21.15.92:0/1527346229 >> v2:172.21.15.118:3300/0 conn(0x7fcf900ae170 msgr2=0x7fcf9015e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:13:00.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:00.743+0000 7fcf955aa640 1 --2- 172.21.15.92:0/1527346229 >> v2:172.21.15.118:3300/0 conn(0x7fcf900ae170 0x7fcf9015e8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:00.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:00.743+0000 7fcf955aa640 1 -- 172.21.15.92:0/1527346229 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf9006c730 con 0x7fcf90143810 2023-11-24T06:13:00.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:00.743+0000 7fcf95dab640 1 --2- 172.21.15.92:0/1527346229 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fcf9015d770 0x7fcf9015dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:13:00.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:00.744+0000 7fcf955aa640 1 --2- 172.21.15.92:0/1527346229 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcf90143810 0x7fcf90071e40 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fcf900b0a40 tx=0x7fcf8c02b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16b40e9333820a14 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:00.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:00.745+0000 7fcf87fff640 1 -- 172.21.15.92:0/1527346229 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf8c0317d0 con 0x7fcf90143810 2023-11-24T06:13:00.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:00.745+0000 7fcf87fff640 1 -- 172.21.15.92:0/1527346229 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf8c031930 con 0x7fcf90143810 2023-11-24T06:13:00.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:00.745+0000 7fcf87fff640 1 -- 172.21.15.92:0/1527346229 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf8c031cc0 con 0x7fcf90143810 2023-11-24T06:13:00.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:00.745+0000 7fcf965ac640 1 -- 172.21.15.92:0/1527346229 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcf90143810 msgr2=0x7fcf90071e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:00.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:00.745+0000 7fcf965ac640 1 --2- 172.21.15.92:0/1527346229 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcf90143810 0x7fcf90071e40 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fcf900b0a40 tx=0x7fcf8c02b870 comp rx=0 tx=0).stop 2023-11-24T06:13:00.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:00.745+0000 7fcf965ac640 1 -- 172.21.15.92:0/1527346229 shutdown_connections 2023-11-24T06:13:00.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:00.745+0000 7fcf965ac640 1 --2- 172.21.15.92:0/1527346229 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fcf9015d770 0x7fcf9015dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:00.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:00.745+0000 7fcf965ac640 1 --2- 172.21.15.92:0/1527346229 >> v2:172.21.15.118:3300/0 conn(0x7fcf900ae170 0x7fcf9015e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:00.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:00.745+0000 7fcf965ac640 1 --2- 172.21.15.92:0/1527346229 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcf90143810 0x7fcf90071e40 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:00.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:00.745+0000 7fcf965ac640 1 -- 172.21.15.92:0/1527346229 >> 172.21.15.92:0/1527346229 conn(0x7fcf9006a7d0 msgr2=0x7fcf9006abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:13:00.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:00.746+0000 7fcf965ac640 1 -- 172.21.15.92:0/1527346229 shutdown_connections 2023-11-24T06:13:00.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:00.746+0000 7fcf965ac640 1 -- 172.21.15.92:0/1527346229 wait complete. 2023-11-24T06:13:00.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:00.747+0000 7fcf965ac640 1 Processor -- start 2023-11-24T06:13:00.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:00.747+0000 7fcf965ac640 1 -- start start 2023-11-24T06:13:00.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:00.748+0000 7fcf965ac640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fcf900ae170 0x7fcf9013b590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:00.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:00.748+0000 7fcf965ac640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcf90143810 0x7fcf901f2d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:00.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:00.748+0000 7fcf965ac640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fcf9015d770 0x7fcf9013cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:00.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:00.748+0000 7fcf965ac640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf901f3d70 con 0x7fcf90143810 2023-11-24T06:13:00.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:00.748+0000 7fcf965ac640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf901f4430 con 0x7fcf900ae170 2023-11-24T06:13:00.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:00.748+0000 7fcf965ac640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf901f37b0 con 0x7fcf9015d770 2023-11-24T06:13:00.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:00.749+0000 7fcf94da9640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcf90143810 0x7fcf901f2d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:00.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:00.749+0000 7fcf94da9640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcf90143810 0x7fcf901f2d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:54480/0 (socket says 172.21.15.92:54480) 2023-11-24T06:13:00.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:00.749+0000 7fcf94da9640 1 -- 172.21.15.92:0/285259208 learned_addr learned my addr 172.21.15.92:0/285259208 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:13:00.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:00.749+0000 7fcf95dab640 1 --2- 172.21.15.92:0/285259208 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fcf9015d770 0x7fcf9013cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:00.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:00.749+0000 7fcf94da9640 1 -- 172.21.15.92:0/285259208 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fcf9015d770 msgr2=0x7fcf9013cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:00.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:00.749+0000 7fcf955aa640 1 --2- 172.21.15.92:0/285259208 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fcf900ae170 0x7fcf9013b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:00.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:00.749+0000 7fcf94da9640 1 --2- 172.21.15.92:0/285259208 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fcf9015d770 0x7fcf9013cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:00.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:00.749+0000 7fcf94da9640 1 -- 172.21.15.92:0/285259208 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fcf900ae170 msgr2=0x7fcf9013b590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:00.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:00.749+0000 7fcf94da9640 1 --2- 172.21.15.92:0/285259208 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fcf900ae170 0x7fcf9013b590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:00.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:00.749+0000 7fcf94da9640 1 -- 172.21.15.92:0/285259208 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf8c0094d0 con 0x7fcf90143810 2023-11-24T06:13:00.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:00.749+0000 7fcf95dab640 1 --2- 172.21.15.92:0/285259208 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fcf9015d770 0x7fcf9013cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:13:00.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:00.749+0000 7fcf94da9640 1 --2- 172.21.15.92:0/285259208 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcf90143810 0x7fcf901f2d60 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fcf8000c7f0 tx=0x7fcf8000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:00.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:00.750+0000 7fcf85ffb640 1 -- 172.21.15.92:0/285259208 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf80004220 con 0x7fcf90143810 2023-11-24T06:13:00.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:00.750+0000 7fcf85ffb640 1 -- 172.21.15.92:0/285259208 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf80004380 con 0x7fcf90143810 2023-11-24T06:13:00.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:00.750+0000 7fcf965ac640 1 -- 172.21.15.92:0/285259208 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf901f3a90 con 0x7fcf90143810 2023-11-24T06:13:00.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:00.750+0000 7fcf85ffb640 1 -- 172.21.15.92:0/285259208 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf80007860 con 0x7fcf90143810 2023-11-24T06:13:00.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:00.750+0000 7fcf965ac640 1 -- 172.21.15.92:0/285259208 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf9013bad0 con 0x7fcf90143810 2023-11-24T06:13:00.756 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:00.751+0000 7fcf965ac640 1 -- 172.21.15.92:0/285259208 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf901f3f20 con 0x7fcf90143810 2023-11-24T06:13:00.756 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:00.752+0000 7fcf85ffb640 1 -- 172.21.15.92:0/285259208 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7fcf80007bd0 con 0x7fcf90143810 2023-11-24T06:13:00.756 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:00.753+0000 7fcf85ffb640 1 --2- 172.21.15.92:0/285259208 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fcf7406af00 0x7fcf7406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:00.756 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:00.753+0000 7fcf85ffb640 1 -- 172.21.15.92:0/285259208 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fcf80089510 con 0x7fcf90143810 2023-11-24T06:13:00.756 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:00.753+0000 7fcf955aa640 1 --2- 172.21.15.92:0/285259208 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fcf7406af00 0x7fcf7406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:00.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:00.754+0000 7fcf955aa640 1 --2- 172.21.15.92:0/285259208 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fcf7406af00 0x7fcf7406d3c0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fcf8c036000 tx=0x7fcf8c03d040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:00.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:00.757+0000 7fcf85ffb640 1 -- 172.21.15.92:0/285259208 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fcf901f3f20 con 0x7fcf90143810 2023-11-24T06:13:00.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:00.947+0000 7fcf965ac640 1 -- 172.21.15.92:0/285259208 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fcf901f3f20 con 0x7fcf90143810 2023-11-24T06:13:00.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:00.948+0000 7fcf85ffb640 1 -- 172.21.15.92:0/285259208 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v55) v1 ==== 93+0+10 (secure 0 0 0) 0x7fcf901f3f20 con 0x7fcf90143810 2023-11-24T06:13:00.951 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 1 2023-11-24T06:13:00.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:00.950+0000 7fcf965ac640 1 -- 172.21.15.92:0/285259208 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fcf7406af00 msgr2=0x7fcf7406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:00.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:00.950+0000 7fcf965ac640 1 --2- 172.21.15.92:0/285259208 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fcf7406af00 0x7fcf7406d3c0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fcf8c036000 tx=0x7fcf8c03d040 comp rx=0 tx=0).stop 2023-11-24T06:13:00.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:00.950+0000 7fcf965ac640 1 -- 172.21.15.92:0/285259208 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcf90143810 msgr2=0x7fcf901f2d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:00.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:00.950+0000 7fcf965ac640 1 --2- 172.21.15.92:0/285259208 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcf90143810 0x7fcf901f2d60 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fcf8000c7f0 tx=0x7fcf8000ccb0 comp rx=0 tx=0).stop 2023-11-24T06:13:00.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:00.950+0000 7fcf965ac640 1 -- 172.21.15.92:0/285259208 shutdown_connections 2023-11-24T06:13:00.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:00.950+0000 7fcf965ac640 1 --2- 172.21.15.92:0/285259208 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fcf7406af00 0x7fcf7406d3c0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:00.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:00.950+0000 7fcf965ac640 1 --2- 172.21.15.92:0/285259208 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fcf9015d770 0x7fcf9013cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:00.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:00.950+0000 7fcf965ac640 1 --2- 172.21.15.92:0/285259208 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcf90143810 0x7fcf901f2d60 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:00.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:00.950+0000 7fcf965ac640 1 --2- 172.21.15.92:0/285259208 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fcf900ae170 0x7fcf9013b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:00.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:00.950+0000 7fcf965ac640 1 -- 172.21.15.92:0/285259208 >> 172.21.15.92:0/285259208 conn(0x7fcf9006a7d0 msgr2=0x7fcf90160580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:13:00.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:00.951+0000 7fcf965ac640 1 -- 172.21.15.92:0/285259208 shutdown_connections 2023-11-24T06:13:00.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:00.951+0000 7fcf965ac640 1 -- 172.21.15.92:0/285259208 wait complete. 2023-11-24T06:13:00.972 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_info_pending_subvol_deletions' 2023-11-24T06:13:01.180 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:01.177+0000 7f30513ea640 1 Processor -- start 2023-11-24T06:13:01.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:01.177+0000 7f30513ea640 1 -- start start 2023-11-24T06:13:01.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:01.179+0000 7f30513ea640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f304c13df80 0x7f304c15bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:01.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:01.179+0000 7f30513ea640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f304c0cf9a0 0x7f304c0cd840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:01.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:01.179+0000 7f30513ea640 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f304c06c470 con 0x7f304c13e360 2023-11-24T06:13:01.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:01.179+0000 7f30513ea640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f304c06c750 con 0x7f304c0cf9a0 2023-11-24T06:13:01.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:01.179+0000 7f30513ea640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f304c06ca30 con 0x7f304c13df80 2023-11-24T06:13:01.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:01.179+0000 7f304b7fe640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f304c0cf9a0 0x7f304c0cd840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:01.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:01.179+0000 7f304b7fe640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f304c0cf9a0 0x7f304c0cd840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:54494/0 (socket says 172.21.15.92:54494) 2023-11-24T06:13:01.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:01.179+0000 7f304b7fe640 1 -- 172.21.15.92:0/2916422261 learned_addr learned my addr 172.21.15.92:0/2916422261 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:13:01.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:01.180+0000 7f304affd640 1 --2- 172.21.15.92:0/2916422261 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f304c13df80 0x7f304c15bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:01.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:01.180+0000 7f304b7fe640 1 -- 172.21.15.92:0/2916422261 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f304c13df80 msgr2=0x7f304c15bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:13:01.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:01.180+0000 7f304b7fe640 1 --2- 172.21.15.92:0/2916422261 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f304c13df80 0x7f304c15bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:01.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:01.180+0000 7f304b7fe640 1 -- 172.21.15.92:0/2916422261 >> v1:172.21.15.118:6789/0 conn(0x7f304c13e360 legacy=0x7f304c0cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:13:01.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:01.180+0000 7f304b7fe640 1 -- 172.21.15.92:0/2916422261 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f304c06cd10 con 0x7f304c0cf9a0 2023-11-24T06:13:01.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:01.180+0000 7f304affd640 1 --2- 172.21.15.92:0/2916422261 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f304c13df80 0x7f304c15bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:13:01.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:01.181+0000 7f304b7fe640 1 --2- 172.21.15.92:0/2916422261 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f304c0cf9a0 0x7f304c0cd840 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f3044009870 tx=0x7f304402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af0c0de6449bb8e0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:01.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:01.181+0000 7f3049ffb640 1 -- 172.21.15.92:0/2916422261 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3044031640 con 0x7f304c0cf9a0 2023-11-24T06:13:01.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:01.181+0000 7f3049ffb640 1 -- 172.21.15.92:0/2916422261 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30440317a0 con 0x7f304c0cf9a0 2023-11-24T06:13:01.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:01.181+0000 7f3049ffb640 1 -- 172.21.15.92:0/2916422261 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3044031b30 con 0x7f304c0cf9a0 2023-11-24T06:13:01.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:01.182+0000 7f30513ea640 1 -- 172.21.15.92:0/2916422261 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f304c0cf9a0 msgr2=0x7f304c0cd840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:01.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:01.182+0000 7f30513ea640 1 --2- 172.21.15.92:0/2916422261 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f304c0cf9a0 0x7f304c0cd840 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f3044009870 tx=0x7f304402b8f0 comp rx=0 tx=0).stop 2023-11-24T06:13:01.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:01.182+0000 7f30513ea640 1 -- 172.21.15.92:0/2916422261 shutdown_connections 2023-11-24T06:13:01.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:01.182+0000 7f30513ea640 1 --2- 172.21.15.92:0/2916422261 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f304c0cf9a0 0x7f304c0cd840 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:01.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:01.182+0000 7f30513ea640 1 --2- 172.21.15.92:0/2916422261 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f304c13df80 0x7f304c15bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:01.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:01.182+0000 7f30513ea640 1 -- 172.21.15.92:0/2916422261 >> 172.21.15.92:0/2916422261 conn(0x7f304c06a800 msgr2=0x7f304c06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:13:01.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:01.182+0000 7f30513ea640 1 -- 172.21.15.92:0/2916422261 shutdown_connections 2023-11-24T06:13:01.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:01.182+0000 7f30513ea640 1 -- 172.21.15.92:0/2916422261 wait complete. 2023-11-24T06:13:01.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:01.184+0000 7f30513ea640 1 Processor -- start 2023-11-24T06:13:01.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:01.184+0000 7f30513ea640 1 -- start start 2023-11-24T06:13:01.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:01.185+0000 7f30513ea640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f304c0cf9a0 0x7f304c13af00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:01.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:01.185+0000 7f30513ea640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f304c13df80 0x7f304c137900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:01.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:01.185+0000 7f30513ea640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f304c13e360 0x7f304c1392f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:01.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:01.185+0000 7f30513ea640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f304c137e40 con 0x7f304c13df80 2023-11-24T06:13:01.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:01.185+0000 7f30513ea640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f304c13aa70 con 0x7f304c0cf9a0 2023-11-24T06:13:01.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:01.185+0000 7f30513ea640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f304c153310 con 0x7f304c13e360 2023-11-24T06:13:01.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:01.186+0000 7f304a7fc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f304c13df80 0x7f304c137900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:01.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:01.186+0000 7f304a7fc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f304c13df80 0x7f304c137900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:54508/0 (socket says 172.21.15.92:54508) 2023-11-24T06:13:01.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:01.186+0000 7f304a7fc640 1 -- 172.21.15.92:0/1815895971 learned_addr learned my addr 172.21.15.92:0/1815895971 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:13:01.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:01.186+0000 7f304affd640 1 --2- 172.21.15.92:0/1815895971 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f304c0cf9a0 0x7f304c13af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:01.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:01.186+0000 7f304b7fe640 1 --2- 172.21.15.92:0/1815895971 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f304c13e360 0x7f304c1392f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:01.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:01.186+0000 7f304a7fc640 1 -- 172.21.15.92:0/1815895971 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f304c13e360 msgr2=0x7f304c1392f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:01.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:01.186+0000 7f304a7fc640 1 --2- 172.21.15.92:0/1815895971 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f304c13e360 0x7f304c1392f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:01.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:01.186+0000 7f304a7fc640 1 -- 172.21.15.92:0/1815895971 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f304c0cf9a0 msgr2=0x7f304c13af00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:01.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:01.186+0000 7f304a7fc640 1 --2- 172.21.15.92:0/1815895971 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f304c0cf9a0 0x7f304c13af00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:01.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:01.186+0000 7f304a7fc640 1 -- 172.21.15.92:0/1815895971 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30440094d0 con 0x7f304c13df80 2023-11-24T06:13:01.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:01.186+0000 7f304a7fc640 1 --2- 172.21.15.92:0/1815895971 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f304c13df80 0x7f304c137900 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f303c00c7f0 tx=0x7f303c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:01.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:01.187+0000 7f30437fe640 1 -- 172.21.15.92:0/1815895971 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f303c00aa50 con 0x7f304c13df80 2023-11-24T06:13:01.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:01.187+0000 7f30437fe640 1 -- 172.21.15.92:0/1815895971 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f303c00abb0 con 0x7f304c13df80 2023-11-24T06:13:01.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:01.187+0000 7f30437fe640 1 -- 172.21.15.92:0/1815895971 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f303c012420 con 0x7f304c13df80 2023-11-24T06:13:01.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:01.187+0000 7f30513ea640 1 -- 172.21.15.92:0/1815895971 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f304c139e60 con 0x7f304c13df80 2023-11-24T06:13:01.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:01.187+0000 7f30513ea640 1 -- 172.21.15.92:0/1815895971 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f304c13a420 con 0x7f304c13df80 2023-11-24T06:13:01.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:01.188+0000 7f30513ea640 1 -- 172.21.15.92:0/1815895971 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f304c0675d0 con 0x7f304c13df80 2023-11-24T06:13:01.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:01.189+0000 7f30437fe640 1 -- 172.21.15.92:0/1815895971 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f303c012580 con 0x7f304c13df80 2023-11-24T06:13:01.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:01.190+0000 7f30437fe640 1 --2- 172.21.15.92:0/1815895971 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f302c06af00 0x7f302c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:01.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:01.190+0000 7f30437fe640 1 -- 172.21.15.92:0/1815895971 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f303c089610 con 0x7f304c13df80 2023-11-24T06:13:01.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:01.190+0000 7f304affd640 1 --2- 172.21.15.92:0/1815895971 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f302c06af00 0x7f302c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:01.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:01.191+0000 7f304affd640 1 --2- 172.21.15.92:0/1815895971 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f302c06af00 0x7f302c06d3c0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f303400a990 tx=0x7f3034008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:01.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:01.195+0000 7f30437fe640 1 -- 172.21.15.92:0/1815895971 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f303c054960 con 0x7f304c13df80 2023-11-24T06:13:01.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:01.452+0000 7f30513ea640 1 -- 172.21.15.92:0/1815895971 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_info_pending_subvol_deletions"]} v 0) v1 -- 0x7f304c13abb0 con 0x7f304c13df80 2023-11-24T06:13:02.084 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:02.080+0000 7f30437fe640 1 -- 172.21.15.92:0/1815895971 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_info_pending_subvol_deletions"]}]=0 v168) v1 ==== 161+0+0 (secure 0 0 0) 0x7f303c057f80 con 0x7f304c13df80 2023-11-24T06:13:02.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:02.084+0000 7f30513ea640 1 -- 172.21.15.92:0/1815895971 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f302c06af00 msgr2=0x7f302c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:02.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:02.084+0000 7f30513ea640 1 --2- 172.21.15.92:0/1815895971 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f302c06af00 0x7f302c06d3c0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f303400a990 tx=0x7f3034008040 comp rx=0 tx=0).stop 2023-11-24T06:13:02.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:02.084+0000 7f30513ea640 1 -- 172.21.15.92:0/1815895971 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f304c13df80 msgr2=0x7f304c137900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:02.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:02.084+0000 7f30513ea640 1 --2- 172.21.15.92:0/1815895971 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f304c13df80 0x7f304c137900 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f303c00c7f0 tx=0x7f303c00ccb0 comp rx=0 tx=0).stop 2023-11-24T06:13:02.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:02.084+0000 7f30513ea640 1 -- 172.21.15.92:0/1815895971 shutdown_connections 2023-11-24T06:13:02.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:02.084+0000 7f30513ea640 1 --2- 172.21.15.92:0/1815895971 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f302c06af00 0x7f302c06d3c0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:02.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:02.084+0000 7f30513ea640 1 --2- 172.21.15.92:0/1815895971 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f304c13e360 0x7f304c1392f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:02.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:02.084+0000 7f30513ea640 1 --2- 172.21.15.92:0/1815895971 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f304c13df80 0x7f304c137900 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:02.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:02.084+0000 7f30513ea640 1 --2- 172.21.15.92:0/1815895971 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f304c0cf9a0 0x7f304c13af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:02.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:02.084+0000 7f30513ea640 1 -- 172.21.15.92:0/1815895971 >> 172.21.15.92:0/1815895971 conn(0x7f304c06a800 msgr2=0x7f304c15d0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:13:02.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:02.084+0000 7f30513ea640 1 -- 172.21.15.92:0/1815895971 shutdown_connections 2023-11-24T06:13:02.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:02.084+0000 7f30513ea640 1 -- 172.21.15.92:0/1815895971 wait complete. 2023-11-24T06:13:02.108 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-24T06:13:02.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:02.320+0000 7fed7fc1c640 1 Processor -- start 2023-11-24T06:13:02.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:02.321+0000 7fed7fc1c640 1 -- start start 2023-11-24T06:13:02.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:02.322+0000 7fed7fc1c640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fed7813df80 0x7fed7815bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:02.326 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:02.322+0000 7fed7fc1c640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fed780cf9a0 0x7fed780cd840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:02.326 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:02.322+0000 7fed7fc1c640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed7806c470 con 0x7fed7813e360 2023-11-24T06:13:02.326 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:02.322+0000 7fed7fc1c640 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed7806c750 con 0x7fed780cf9a0 2023-11-24T06:13:02.326 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:02.322+0000 7fed7fc1c640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed7806ca30 con 0x7fed7813df80 2023-11-24T06:13:02.326 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:02.323+0000 7fed7d1b7640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fed7813e360 0x7fed780cf290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:47858/0 (socket says 172.21.15.92:47858) 2023-11-24T06:13:02.326 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:02.323+0000 7fed7d1b7640 1 -- 172.21.15.92:0/142728166 learned_addr learned my addr 172.21.15.92:0/142728166 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-24T06:13:02.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:02.323+0000 7fed7d9b8640 1 --2- 172.21.15.92:0/142728166 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fed7813df80 0x7fed7815bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:02.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:02.323+0000 7fed7c9b6640 1 -- 172.21.15.92:0/142728166 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3911756529 0 0) 0x7fed7806c470 con 0x7fed7813e360 2023-11-24T06:13:02.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:02.323+0000 7fed7c9b6640 1 -- 172.21.15.92:0/142728166 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fed64003520 con 0x7fed7813e360 2023-11-24T06:13:02.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:02.323+0000 7fed7e1b9640 1 --2- 172.21.15.92:0/142728166 >> v2:172.21.15.118:3300/0 conn(0x7fed780cf9a0 0x7fed780cd840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:02.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:02.323+0000 7fed7c9b6640 1 -- 172.21.15.92:0/142728166 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3927724639 0 0) 0x7fed64003520 con 0x7fed7813e360 2023-11-24T06:13:02.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:02.324+0000 7fed7c9b6640 1 -- 172.21.15.92:0/142728166 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fed7813df80 msgr2=0x7fed7815bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:13:02.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:02.324+0000 7fed7c9b6640 1 --2- 172.21.15.92:0/142728166 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fed7813df80 0x7fed7815bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:02.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:02.324+0000 7fed7c9b6640 1 -- 172.21.15.92:0/142728166 >> v2:172.21.15.118:3300/0 conn(0x7fed780cf9a0 msgr2=0x7fed780cd840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:13:02.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:02.324+0000 7fed7c9b6640 1 --2- 172.21.15.92:0/142728166 >> v2:172.21.15.118:3300/0 conn(0x7fed780cf9a0 0x7fed780cd840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:02.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:02.324+0000 7fed7c9b6640 1 -- 172.21.15.92:0/142728166 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed7806cd10 con 0x7fed7813e360 2023-11-24T06:13:02.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:02.324+0000 7fed7c9b6640 1 -- 172.21.15.92:0/142728166 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2607661350 0 0) 0x7fed68001670 con 0x7fed7813e360 2023-11-24T06:13:02.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:02.324+0000 7fed7c9b6640 1 -- 172.21.15.92:0/142728166 >> v1:172.21.15.92:6789/0 conn(0x7fed7813e360 legacy=0x7fed780cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:02.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:02.324+0000 7fed7c9b6640 1 --2- 172.21.15.92:0/142728166 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fed64003970 0x7fed64003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:02.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:02.324+0000 7fed7c9b6640 1 --2- 172.21.15.92:0/142728166 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fed640047c0 0x7fed64004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:02.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:02.324+0000 7fed7c9b6640 1 --2- 172.21.15.92:0/142728166 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fed7813df80 0x7fed64005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:02.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:02.324+0000 7fed7c9b6640 1 -- 172.21.15.92:0/142728166 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed7806c750 con 0x7fed64003970 2023-11-24T06:13:02.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:02.324+0000 7fed7c9b6640 1 -- 172.21.15.92:0/142728166 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed7806ca30 con 0x7fed640047c0 2023-11-24T06:13:02.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:02.324+0000 7fed7c9b6640 1 -- 172.21.15.92:0/142728166 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed6400b980 con 0x7fed7813df80 2023-11-24T06:13:02.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:02.324+0000 7fed7d9b8640 1 --2- 172.21.15.92:0/142728166 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fed64003970 0x7fed64003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:02.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:02.324+0000 7fed7d1b7640 1 --2- 172.21.15.92:0/142728166 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fed640047c0 0x7fed64004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:02.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:02.324+0000 7fed7e1b9640 1 --2- 172.21.15.92:0/142728166 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fed7813df80 0x7fed64005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:02.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:02.324+0000 7fed7d9b8640 1 -- 172.21.15.92:0/142728166 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fed7813df80 msgr2=0x7fed64005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:13:02.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:02.324+0000 7fed7d9b8640 1 --2- 172.21.15.92:0/142728166 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fed7813df80 0x7fed64005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:02.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:02.324+0000 7fed7d9b8640 1 -- 172.21.15.92:0/142728166 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fed640047c0 msgr2=0x7fed64004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:13:02.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:02.324+0000 7fed7d9b8640 1 --2- 172.21.15.92:0/142728166 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fed640047c0 0x7fed64004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:02.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:02.324+0000 7fed7d9b8640 1 -- 172.21.15.92:0/142728166 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed64003520 con 0x7fed64003970 2023-11-24T06:13:02.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:02.325+0000 7fed7d1b7640 1 --2- 172.21.15.92:0/142728166 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fed640047c0 0x7fed64004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:13:02.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:02.325+0000 7fed7e1b9640 1 --2- 172.21.15.92:0/142728166 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fed7813df80 0x7fed64005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:13:02.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:02.326+0000 7fed7d9b8640 1 --2- 172.21.15.92:0/142728166 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fed64003970 0x7fed64003d90 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fed74009ac0 tx=0x7fed640057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12368f552f1e41e7 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:02.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:02.326+0000 7fed7c9b6640 1 -- 172.21.15.92:0/142728166 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed7402c6b0 con 0x7fed64003970 2023-11-24T06:13:02.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:02.326+0000 7fed7c9b6640 1 -- 172.21.15.92:0/142728166 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed7402c810 con 0x7fed64003970 2023-11-24T06:13:02.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:02.326+0000 7fed7c9b6640 1 -- 172.21.15.92:0/142728166 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed7402cba0 con 0x7fed64003970 2023-11-24T06:13:02.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:02.326+0000 7fed7fc1c640 1 -- 172.21.15.92:0/142728166 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fed64003970 msgr2=0x7fed64003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:02.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:02.326+0000 7fed7fc1c640 1 --2- 172.21.15.92:0/142728166 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fed64003970 0x7fed64003d90 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fed74009ac0 tx=0x7fed640057f0 comp rx=0 tx=0).stop 2023-11-24T06:13:02.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:02.326+0000 7fed7e1b9640 1 -- 172.21.15.92:0/142728166 reap_dead start 2023-11-24T06:13:02.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:02.327+0000 7fed7fc1c640 1 -- 172.21.15.92:0/142728166 shutdown_connections 2023-11-24T06:13:02.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:02.327+0000 7fed7fc1c640 1 -- 172.21.15.92:0/142728166 >> 172.21.15.92:0/142728166 conn(0x7fed7806a800 msgr2=0x7fed7806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:13:02.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:02.327+0000 7fed7fc1c640 1 -- 172.21.15.92:0/142728166 shutdown_connections 2023-11-24T06:13:02.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:02.327+0000 7fed7fc1c640 1 -- 172.21.15.92:0/142728166 wait complete. 2023-11-24T06:13:02.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:02.328+0000 7fed7fc1c640 1 Processor -- start 2023-11-24T06:13:02.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:02.328+0000 7fed7fc1c640 1 -- start start 2023-11-24T06:13:02.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:02.329+0000 7fed7fc1c640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fed64003970 0x7fed7813d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:02.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:02.329+0000 7fed7fc1c640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fed7815a280 0x7fed78071490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:02.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:02.329+0000 7fed7fc1c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fed7813cb70 0x7fed780afa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:02.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:02.329+0000 7fed7fc1c640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed7815a700 con 0x7fed7813cb70 2023-11-24T06:13:02.335 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:02.329+0000 7fed7fc1c640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed7813dc80 con 0x7fed64003970 2023-11-24T06:13:02.335 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:02.329+0000 7fed7fc1c640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed780703f0 con 0x7fed7815a280 2023-11-24T06:13:02.335 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:02.330+0000 7fed7e1b9640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fed7813cb70 0x7fed780afa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:02.335 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:02.330+0000 7fed7e1b9640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fed7813cb70 0x7fed780afa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:54534/0 (socket says 172.21.15.92:54534) 2023-11-24T06:13:02.335 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:02.330+0000 7fed7e1b9640 1 -- 172.21.15.92:0/1663765991 learned_addr learned my addr 172.21.15.92:0/1663765991 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:13:02.335 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:02.330+0000 7fed7d9b8640 1 --2- 172.21.15.92:0/1663765991 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fed64003970 0x7fed7813d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:02.335 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:02.330+0000 7fed7d1b7640 1 --2- 172.21.15.92:0/1663765991 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fed7815a280 0x7fed78071490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:02.335 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:02.330+0000 7fed7e1b9640 1 -- 172.21.15.92:0/1663765991 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fed7815a280 msgr2=0x7fed78071490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:02.335 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:02.330+0000 7fed7e1b9640 1 --2- 172.21.15.92:0/1663765991 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fed7815a280 0x7fed78071490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:02.335 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:02.330+0000 7fed7e1b9640 1 -- 172.21.15.92:0/1663765991 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fed64003970 msgr2=0x7fed7813d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:02.335 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:02.330+0000 7fed7e1b9640 1 --2- 172.21.15.92:0/1663765991 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fed64003970 0x7fed7813d400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:02.335 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:02.330+0000 7fed7e1b9640 1 -- 172.21.15.92:0/1663765991 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed780706d0 con 0x7fed7813cb70 2023-11-24T06:13:02.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:02.330+0000 7fed7e1b9640 1 --2- 172.21.15.92:0/1663765991 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fed7813cb70 0x7fed780afa30 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7fed780cc770 tx=0x7fed7000f4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:02.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:02.331+0000 7fed6e7fc640 1 -- 172.21.15.92:0/1663765991 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed70017400 con 0x7fed7813cb70 2023-11-24T06:13:02.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:02.331+0000 7fed6e7fc640 1 -- 172.21.15.92:0/1663765991 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed70011040 con 0x7fed7813cb70 2023-11-24T06:13:02.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:02.331+0000 7fed6e7fc640 1 -- 172.21.15.92:0/1663765991 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed70017780 con 0x7fed7813cb70 2023-11-24T06:13:02.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:02.331+0000 7fed7fc1c640 1 -- 172.21.15.92:0/1663765991 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed780709b0 con 0x7fed7813cb70 2023-11-24T06:13:02.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:02.331+0000 7fed7fc1c640 1 -- 172.21.15.92:0/1663765991 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed78070f70 con 0x7fed7813cb70 2023-11-24T06:13:02.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:02.332+0000 7fed7fc1c640 1 -- 172.21.15.92:0/1663765991 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed78071190 con 0x7fed7813cb70 2023-11-24T06:13:02.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:02.333+0000 7fed6e7fc640 1 -- 172.21.15.92:0/1663765991 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7fed700203b0 con 0x7fed7813cb70 2023-11-24T06:13:02.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:02.334+0000 7fed6e7fc640 1 --2- 172.21.15.92:0/1663765991 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fed5c06af50 0x7fed5c06d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:02.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:02.334+0000 7fed6e7fc640 1 -- 172.21.15.92:0/1663765991 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fed70089660 con 0x7fed7813cb70 2023-11-24T06:13:02.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:02.335+0000 7fed7d9b8640 1 --2- 172.21.15.92:0/1663765991 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fed5c06af50 0x7fed5c06d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:02.338 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:02.335+0000 7fed7d9b8640 1 --2- 172.21.15.92:0/1663765991 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fed5c06af50 0x7fed5c06d410 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fed74036000 tx=0x7fed74030000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:02.342 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:02.339+0000 7fed6e7fc640 1 -- 172.21.15.92:0/1663765991 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fed70054920 con 0x7fed7813cb70 2023-11-24T06:13:02.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:02.523+0000 7fed7fc1c640 1 -- 172.21.15.92:0/1663765991 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fed780675d0 con 0x7fed7813cb70 2023-11-24T06:13:02.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:02.530+0000 7fed6e7fc640 1 -- 172.21.15.92:0/1663765991 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v14)=0 v14) v1 ==== 122+0+0 (secure 0 0 0) 0x7fed70057f40 con 0x7fed7813cb70 2023-11-24T06:13:02.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:02.532+0000 7fed7fc1c640 1 -- 172.21.15.92:0/1663765991 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fed5c06af50 msgr2=0x7fed5c06d410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:02.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:02.532+0000 7fed7fc1c640 1 --2- 172.21.15.92:0/1663765991 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fed5c06af50 0x7fed5c06d410 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fed74036000 tx=0x7fed74030000 comp rx=0 tx=0).stop 2023-11-24T06:13:02.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:02.532+0000 7fed7fc1c640 1 -- 172.21.15.92:0/1663765991 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fed7813cb70 msgr2=0x7fed780afa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:02.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:02.532+0000 7fed7fc1c640 1 --2- 172.21.15.92:0/1663765991 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fed7813cb70 0x7fed780afa30 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7fed780cc770 tx=0x7fed7000f4a0 comp rx=0 tx=0).stop 2023-11-24T06:13:02.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:02.533+0000 7fed7fc1c640 1 -- 172.21.15.92:0/1663765991 shutdown_connections 2023-11-24T06:13:02.537 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:02.533+0000 7fed7fc1c640 1 --2- 172.21.15.92:0/1663765991 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fed5c06af50 0x7fed5c06d410 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:02.537 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:02.533+0000 7fed7fc1c640 1 --2- 172.21.15.92:0/1663765991 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fed7813cb70 0x7fed780afa30 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:02.537 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:02.533+0000 7fed7fc1c640 1 --2- 172.21.15.92:0/1663765991 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fed7815a280 0x7fed78071490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:02.537 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:02.533+0000 7fed7fc1c640 1 --2- 172.21.15.92:0/1663765991 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fed64003970 0x7fed7813d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:02.537 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:02.533+0000 7fed7fc1c640 1 -- 172.21.15.92:0/1663765991 >> 172.21.15.92:0/1663765991 conn(0x7fed7806a800 msgr2=0x7fed780aeb30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:13:02.537 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:02.533+0000 7fed7fc1c640 1 -- 172.21.15.92:0/1663765991 shutdown_connections 2023-11-24T06:13:02.537 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:02.533+0000 7fed7fc1c640 1 -- 172.21.15.92:0/1663765991 wait complete. 2023-11-24T06:13:02.554 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-24T06:13:02.554 DEBUG:teuthology.orchestra.run.smithi092:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-24T06:13:02.661 DEBUG:teuthology.orchestra.run.smithi118:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-24T06:13:02.790 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-24T06:13:02.790 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-11-24T06:13:02.791 DEBUG:teuthology.orchestra.run.smithi092:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:13:02.851 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054298,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498824k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249408k,nr_inodes=812352,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498824k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-24T06:13:02.851 DEBUG:teuthology.orchestra.run.smithi118:> set -ex 2023-11-24T06:13:02.851 DEBUG:teuthology.orchestra.run.smithi118:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:13:02.911 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054284,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249396k,nr_inodes=812349,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-24T06:13:02.911 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-11-24T06:13:02.911 DEBUG:teuthology.orchestra.run.smithi092:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:13:02.971 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054298,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498824k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249408k,nr_inodes=812352,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498824k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-24T06:13:02.971 DEBUG:teuthology.orchestra.run.smithi118:> set -ex 2023-11-24T06:13:02.971 DEBUG:teuthology.orchestra.run.smithi118:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:13:03.030 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054284,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249396k,nr_inodes=812349,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-24T06:13:03.030 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-11-24T06:13:03.031 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T06:13:03.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:03.250+0000 7f2f817fb640 1 Processor -- start 2023-11-24T06:13:03.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:03.251+0000 7f2f817fb640 1 -- start start 2023-11-24T06:13:03.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:03.252+0000 7f2f817fb640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f2f7c13df80 0x7f2f7c15bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:03.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:03.252+0000 7f2f817fb640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f2f7c0cf9a0 0x7f2f7c0cd840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:03.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:03.252+0000 7f2f817fb640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f7c06c470 con 0x7f2f7c13e360 2023-11-24T06:13:03.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:03.252+0000 7f2f817fb640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f7c06c750 con 0x7f2f7c0cf9a0 2023-11-24T06:13:03.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:03.252+0000 7f2f817fb640 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f7c06ca30 con 0x7f2f7c13df80 2023-11-24T06:13:03.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:03.253+0000 7f2f7b7fe640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f2f7c0cf9a0 0x7f2f7c0cd840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:03.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:03.253+0000 7f2f7b7fe640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f2f7c0cf9a0 0x7f2f7c0cd840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:54544/0 (socket says 172.21.15.92:54544) 2023-11-24T06:13:03.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:03.253+0000 7f2f7b7fe640 1 -- 172.21.15.92:0/2160018461 learned_addr learned my addr 172.21.15.92:0/2160018461 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:13:03.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:03.253+0000 7f2f7a7fc640 1 --1- 172.21.15.92:0/2160018461 >> v1:172.21.15.92:6789/0 conn(0x7f2f7c13e360 0x7f2f7c0cf290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:47866/0 (socket says 172.21.15.92:47866) 2023-11-24T06:13:03.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:03.253+0000 7f2f7affd640 1 --2- 172.21.15.92:0/2160018461 >> v2:172.21.15.118:3300/0 conn(0x7f2f7c13df80 0x7f2f7c15bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:03.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:03.253+0000 7f2f7b7fe640 1 -- 172.21.15.92:0/2160018461 >> v2:172.21.15.118:3300/0 conn(0x7f2f7c13df80 msgr2=0x7f2f7c15bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:13:03.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:03.253+0000 7f2f7b7fe640 1 --2- 172.21.15.92:0/2160018461 >> v2:172.21.15.118:3300/0 conn(0x7f2f7c13df80 0x7f2f7c15bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:03.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:03.253+0000 7f2f7b7fe640 1 -- 172.21.15.92:0/2160018461 >> v1:172.21.15.92:6789/0 conn(0x7f2f7c13e360 legacy=0x7f2f7c0cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:03.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:03.253+0000 7f2f7b7fe640 1 -- 172.21.15.92:0/2160018461 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f7c06cd10 con 0x7f2f7c0cf9a0 2023-11-24T06:13:03.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:03.254+0000 7f2f7b7fe640 1 --2- 172.21.15.92:0/2160018461 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2f7c0cf9a0 0x7f2f7c0cd840 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f2f7c072770 tx=0x7f2f7402bf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d793f2b7061e1bc server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:03.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:03.255+0000 7f2f79ffb640 1 -- 172.21.15.92:0/2160018461 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f7402cb10 con 0x7f2f7c0cf9a0 2023-11-24T06:13:03.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:03.255+0000 7f2f79ffb640 1 -- 172.21.15.92:0/2160018461 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f7402cc70 con 0x7f2f7c0cf9a0 2023-11-24T06:13:03.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:03.255+0000 7f2f79ffb640 1 -- 172.21.15.92:0/2160018461 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f740343f0 con 0x7f2f7c0cf9a0 2023-11-24T06:13:03.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:03.255+0000 7f2f817fb640 1 -- 172.21.15.92:0/2160018461 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2f7c0cf9a0 msgr2=0x7f2f7c0cd840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:03.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:03.255+0000 7f2f817fb640 1 --2- 172.21.15.92:0/2160018461 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2f7c0cf9a0 0x7f2f7c0cd840 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f2f7c072770 tx=0x7f2f7402bf70 comp rx=0 tx=0).stop 2023-11-24T06:13:03.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:03.255+0000 7f2f817fb640 1 -- 172.21.15.92:0/2160018461 shutdown_connections 2023-11-24T06:13:03.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:03.255+0000 7f2f817fb640 1 --2- 172.21.15.92:0/2160018461 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2f7c0cf9a0 0x7f2f7c0cd840 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:03.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:03.255+0000 7f2f817fb640 1 --2- 172.21.15.92:0/2160018461 >> v2:172.21.15.118:3300/0 conn(0x7f2f7c13df80 0x7f2f7c15bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:03.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:03.255+0000 7f2f817fb640 1 -- 172.21.15.92:0/2160018461 >> 172.21.15.92:0/2160018461 conn(0x7f2f7c06a800 msgr2=0x7f2f7c06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:13:03.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:03.255+0000 7f2f817fb640 1 -- 172.21.15.92:0/2160018461 shutdown_connections 2023-11-24T06:13:03.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:03.255+0000 7f2f817fb640 1 -- 172.21.15.92:0/2160018461 wait complete. 2023-11-24T06:13:03.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:03.257+0000 7f2f817fb640 1 Processor -- start 2023-11-24T06:13:03.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:03.257+0000 7f2f817fb640 1 -- start start 2023-11-24T06:13:03.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:03.258+0000 7f2f817fb640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2f7c0cf9a0 0x7f2f7c152c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:03.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:03.258+0000 7f2f817fb640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2f7c13df80 0x7f2f7c1531a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:03.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:03.258+0000 7f2f817fb640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2f7c13e360 0x7f2f7c14c450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:03.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:03.258+0000 7f2f817fb640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f7c13dbc0 con 0x7f2f7c0cf9a0 2023-11-24T06:13:03.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:03.258+0000 7f2f817fb640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f7c14afe0 con 0x7f2f7c13e360 2023-11-24T06:13:03.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:03.258+0000 7f2f817fb640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f7c14b2c0 con 0x7f2f7c13df80 2023-11-24T06:13:03.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:03.258+0000 7f2f7affd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2f7c0cf9a0 0x7f2f7c152c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:03.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:03.258+0000 7f2f7affd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2f7c0cf9a0 0x7f2f7c152c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:54550/0 (socket says 172.21.15.92:54550) 2023-11-24T06:13:03.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:03.258+0000 7f2f7affd640 1 -- 172.21.15.92:0/3516489185 learned_addr learned my addr 172.21.15.92:0/3516489185 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:13:03.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:03.259+0000 7f2f7affd640 1 -- 172.21.15.92:0/3516489185 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2f7c13df80 msgr2=0x7f2f7c1531a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:03.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:03.259+0000 7f2f7affd640 1 --2- 172.21.15.92:0/3516489185 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2f7c13df80 0x7f2f7c1531a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:03.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:03.259+0000 7f2f7affd640 1 -- 172.21.15.92:0/3516489185 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2f7c13e360 msgr2=0x7f2f7c14c450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:03.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:03.259+0000 7f2f7b7fe640 1 --2- 172.21.15.92:0/3516489185 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2f7c13e360 0x7f2f7c14c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:03.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:03.259+0000 7f2f7affd640 1 --2- 172.21.15.92:0/3516489185 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2f7c13e360 0x7f2f7c14c450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:03.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:03.259+0000 7f2f7affd640 1 -- 172.21.15.92:0/3516489185 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f74009b30 con 0x7f2f7c0cf9a0 2023-11-24T06:13:03.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:03.259+0000 7f2f7affd640 1 --2- 172.21.15.92:0/3516489185 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2f7c0cf9a0 0x7f2f7c152c60 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f2f6c00d800 tx=0x7f2f6c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:03.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:03.259+0000 7f2f737fe640 1 -- 172.21.15.92:0/3516489185 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f6c00b6c0 con 0x7f2f7c0cf9a0 2023-11-24T06:13:03.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:03.260+0000 7f2f737fe640 1 -- 172.21.15.92:0/3516489185 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f6c00b820 con 0x7f2f7c0cf9a0 2023-11-24T06:13:03.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:03.260+0000 7f2f737fe640 1 -- 172.21.15.92:0/3516489185 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f6c00bbb0 con 0x7f2f7c0cf9a0 2023-11-24T06:13:03.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:03.260+0000 7f2f817fb640 1 -- 172.21.15.92:0/3516489185 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f7c14b5a0 con 0x7f2f7c0cf9a0 2023-11-24T06:13:03.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:03.260+0000 7f2f817fb640 1 -- 172.21.15.92:0/3516489185 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f7c14bb60 con 0x7f2f7c0cf9a0 2023-11-24T06:13:03.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:03.261+0000 7f2f817fb640 1 -- 172.21.15.92:0/3516489185 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f7c0675d0 con 0x7f2f7c0cf9a0 2023-11-24T06:13:03.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:03.262+0000 7f2f737fe640 1 -- 172.21.15.92:0/3516489185 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f2f6c00bd10 con 0x7f2f7c0cf9a0 2023-11-24T06:13:03.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:03.263+0000 7f2f737fe640 1 --2- 172.21.15.92:0/3516489185 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f2f5c06aed0 0x7f2f5c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:03.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:03.263+0000 7f2f737fe640 1 -- 172.21.15.92:0/3516489185 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f2f6c089a70 con 0x7f2f7c0cf9a0 2023-11-24T06:13:03.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:03.263+0000 7f2f7a7fc640 1 --2- 172.21.15.92:0/3516489185 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f2f5c06aed0 0x7f2f5c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:03.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:03.264+0000 7f2f7a7fc640 1 --2- 172.21.15.92:0/3516489185 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f2f5c06aed0 0x7f2f5c06d390 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f2f7c13d7b0 tx=0x7f2f640079c0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:03.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:03.268+0000 7f2f737fe640 1 -- 172.21.15.92:0/3516489185 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2f6c054ca0 con 0x7f2f7c0cf9a0 2023-11-24T06:13:03.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:03.498+0000 7f2f817fb640 1 -- 172.21.15.92:0/3516489185 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2f7c06df00 con 0x7f2f7c0cf9a0 2023-11-24T06:13:03.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:03.499+0000 7f2f737fe640 1 -- 172.21.15.92:0/3516489185 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+3705 (secure 0 0 0) 0x7f2f6c0582c0 con 0x7f2f7c0cf9a0 2023-11-24T06:13:03.503 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-24T06:13:03.503 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":34,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4778,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6835/1184353034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":1184353034},{"type":"v1","addr":"172.21.15.118:6835","nonce":1184353034}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":26},{"gid":4942,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6833/1256969666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":1256969666},{"type":"v1","addr":"172.21.15.118:6833","nonce":1256969666}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":26},{"gid":5706,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6836/3849408561","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":3849408561},{"type":"v1","addr":"172.21.15.92:6836","nonce":3849408561}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":26},{"gid":6069,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/2737152236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":2737152236},{"type":"v1","addr":"172.21.15.92:6837","nonce":2737152236}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":34}],"filesystems":[]} 2023-11-24T06:13:03.503 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 34 2023-11-24T06:13:03.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:03.502+0000 7f2f817fb640 1 -- 172.21.15.92:0/3516489185 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f2f5c06aed0 msgr2=0x7f2f5c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:03.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:03.502+0000 7f2f817fb640 1 --2- 172.21.15.92:0/3516489185 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f2f5c06aed0 0x7f2f5c06d390 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f2f7c13d7b0 tx=0x7f2f640079c0 comp rx=0 tx=0).stop 2023-11-24T06:13:03.506 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:03.502+0000 7f2f817fb640 1 -- 172.21.15.92:0/3516489185 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2f7c0cf9a0 msgr2=0x7f2f7c152c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:03.506 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:03.502+0000 7f2f817fb640 1 --2- 172.21.15.92:0/3516489185 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2f7c0cf9a0 0x7f2f7c152c60 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f2f6c00d800 tx=0x7f2f6c00dcc0 comp rx=0 tx=0).stop 2023-11-24T06:13:03.506 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:03.502+0000 7f2f817fb640 1 -- 172.21.15.92:0/3516489185 shutdown_connections 2023-11-24T06:13:03.506 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:03.502+0000 7f2f817fb640 1 --2- 172.21.15.92:0/3516489185 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f2f5c06aed0 0x7f2f5c06d390 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:03.506 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:03.502+0000 7f2f817fb640 1 --2- 172.21.15.92:0/3516489185 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2f7c13e360 0x7f2f7c14c450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:03.506 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:03.502+0000 7f2f817fb640 1 --2- 172.21.15.92:0/3516489185 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2f7c13df80 0x7f2f7c1531a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:03.506 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:03.502+0000 7f2f817fb640 1 --2- 172.21.15.92:0/3516489185 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2f7c0cf9a0 0x7f2f7c152c60 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:03.506 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:03.502+0000 7f2f817fb640 1 -- 172.21.15.92:0/3516489185 >> 172.21.15.92:0/3516489185 conn(0x7f2f7c06a800 msgr2=0x7f2f7c15d1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:13:03.506 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:03.502+0000 7f2f817fb640 1 -- 172.21.15.92:0/3516489185 shutdown_connections 2023-11-24T06:13:03.506 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:03.502+0000 7f2f817fb640 1 -- 172.21.15.92:0/3516489185 wait complete. 2023-11-24T06:13:03.523 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-24T06:13:03.523 INFO:tasks.ceph.mds.a:Stopping old one... 2023-11-24T06:13:03.523 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-24T06:13:03.523 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T06:13:03.524 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-24T06:13:03.524 INFO:tasks.ceph.mds.c:Stopping old one... 2023-11-24T06:13:03.524 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-11-24T06:13:03.524 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T06:13:03.524 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-24T06:13:03.524 INFO:tasks.ceph.mds.b:Stopping old one... 2023-11-24T06:13:03.525 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-24T06:13:03.525 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T06:13:03.525 INFO:tasks.ceph.mds.d:Restarting daemon 2023-11-24T06:13:03.525 INFO:tasks.ceph.mds.d:Stopping old one... 2023-11-24T06:13:03.525 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-11-24T06:13:03.525 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T06:13:03.545 INFO:tasks.ceph.mds.c:Stopped 2023-11-24T06:13:03.545 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-24T06:13:03.547 INFO:tasks.ceph.mds.a:Stopped 2023-11-24T06:13:03.547 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-24T06:13:03.552 INFO:tasks.ceph.mds.b:Stopped 2023-11-24T06:13:03.552 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-24T06:13:03.556 INFO:tasks.ceph.mds.d:Stopped 2023-11-24T06:13:03.556 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-11-24T06:13:03.592 INFO:tasks.ceph.mds.c:Started 2023-11-24T06:13:03.592 DEBUG:teuthology.parallel:result is None 2023-11-24T06:13:03.596 INFO:tasks.ceph.mds.a:Started 2023-11-24T06:13:03.596 DEBUG:teuthology.parallel:result is None 2023-11-24T06:13:03.604 INFO:tasks.ceph.mds.b:Started 2023-11-24T06:13:03.604 DEBUG:teuthology.parallel:result is None 2023-11-24T06:13:03.611 INFO:tasks.ceph.mds.d:Started 2023-11-24T06:13:03.612 DEBUG:teuthology.parallel:result is None 2023-11-24T06:13:03.612 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-11-24T06:13:03.749 INFO:tasks.ceph.mds.c.smithi092.stdout:starting mds.c at 2023-11-24T06:13:03.749 INFO:tasks.ceph.mds.a.smithi092.stdout:starting mds.a at 2023-11-24T06:13:03.776 INFO:tasks.ceph.mds.b.smithi118.stdout:starting mds.b at 2023-11-24T06:13:03.783 INFO:tasks.ceph.mds.d.smithi118.stdout:starting mds.d at 2023-11-24T06:13:03.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:03.795+0000 7ff55d547640 1 Processor -- start 2023-11-24T06:13:03.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:03.795+0000 7ff55d547640 1 -- start start 2023-11-24T06:13:03.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:03.797+0000 7ff55d547640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7ff558160920 0x7ff5580b3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:03.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:03.797+0000 7ff55d547640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ff5580cf9f0 0x7ff5580cfea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:03.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:03.797+0000 7ff55d547640 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5580d0a10 con 0x7ff558161010 2023-11-24T06:13:03.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:03.797+0000 7ff55d547640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5580d03e0 con 0x7ff5580cf9f0 2023-11-24T06:13:03.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:03.797+0000 7ff55d547640 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5580d1f50 con 0x7ff558160920 2023-11-24T06:13:03.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:03.797+0000 7ff54ffff640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ff5580cf9f0 0x7ff5580cfea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:03.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:03.797+0000 7ff54ffff640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ff5580cf9f0 0x7ff5580cfea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:54576/0 (socket says 172.21.15.92:54576) 2023-11-24T06:13:03.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:03.797+0000 7ff54ffff640 1 -- 172.21.15.92:0/1191929834 learned_addr learned my addr 172.21.15.92:0/1191929834 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:13:03.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:03.797+0000 7ff557fff640 1 --2- 172.21.15.92:0/1191929834 >> v2:172.21.15.118:3300/0 conn(0x7ff558160920 0x7ff5580b3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:03.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:03.797+0000 7ff54ffff640 1 -- 172.21.15.92:0/1191929834 >> v2:172.21.15.118:3300/0 conn(0x7ff558160920 msgr2=0x7ff5580b3f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:13:03.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:03.797+0000 7ff54ffff640 1 --2- 172.21.15.92:0/1191929834 >> v2:172.21.15.118:3300/0 conn(0x7ff558160920 0x7ff5580b3f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:03.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:03.797+0000 7ff54ffff640 1 -- 172.21.15.92:0/1191929834 >> v1:172.21.15.118:6789/0 conn(0x7ff558161010 legacy=0x7ff558161480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:03.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:03.797+0000 7ff54ffff640 1 -- 172.21.15.92:0/1191929834 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5580cf080 con 0x7ff5580cf9f0 2023-11-24T06:13:03.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:03.798+0000 7ff54ffff640 1 --2- 172.21.15.92:0/1191929834 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff5580cf9f0 0x7ff5580cfea0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7ff55808c060 tx=0x7ff54002b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97b164331f8f76e2 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:03.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:03.799+0000 7ff5577fe640 1 -- 172.21.15.92:0/1191929834 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff540030860 con 0x7ff5580cf9f0 2023-11-24T06:13:03.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:03.799+0000 7ff5577fe640 1 -- 172.21.15.92:0/1191929834 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5400309c0 con 0x7ff5580cf9f0 2023-11-24T06:13:03.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:03.799+0000 7ff5577fe640 1 -- 172.21.15.92:0/1191929834 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff540030d50 con 0x7ff5580cf9f0 2023-11-24T06:13:03.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:03.799+0000 7ff55d547640 1 -- 172.21.15.92:0/1191929834 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff5580cf9f0 msgr2=0x7ff5580cfea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:03.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:03.799+0000 7ff55d547640 1 --2- 172.21.15.92:0/1191929834 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff5580cf9f0 0x7ff5580cfea0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7ff55808c060 tx=0x7ff54002b9a0 comp rx=0 tx=0).stop 2023-11-24T06:13:03.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:03.799+0000 7ff55d547640 1 -- 172.21.15.92:0/1191929834 shutdown_connections 2023-11-24T06:13:03.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:03.799+0000 7ff55d547640 1 --2- 172.21.15.92:0/1191929834 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff5580cf9f0 0x7ff5580cfea0 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:03.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:03.799+0000 7ff55d547640 1 --2- 172.21.15.92:0/1191929834 >> v2:172.21.15.118:3300/0 conn(0x7ff558160920 0x7ff5580b3f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:03.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:03.799+0000 7ff55d547640 1 -- 172.21.15.92:0/1191929834 >> 172.21.15.92:0/1191929834 conn(0x7ff5580d3090 msgr2=0x7ff5580d34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:13:03.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:03.800+0000 7ff55d547640 1 -- 172.21.15.92:0/1191929834 shutdown_connections 2023-11-24T06:13:03.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:03.800+0000 7ff55d547640 1 -- 172.21.15.92:0/1191929834 wait complete. 2023-11-24T06:13:03.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:03.801+0000 7ff55d547640 1 Processor -- start 2023-11-24T06:13:03.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:03.801+0000 7ff55d547640 1 -- start start 2023-11-24T06:13:03.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:03.802+0000 7ff55d547640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff5580cf9f0 0x7ff5580ccf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:03.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:03.802+0000 7ff55d547640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff558160920 0x7ff558149620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:03.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:03.802+0000 7ff55d547640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff558161010 0x7ff55815df10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:03.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:03.802+0000 7ff55d547640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5581f3e90 con 0x7ff558161010 2023-11-24T06:13:03.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:03.802+0000 7ff55d547640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5580cbef0 con 0x7ff5580cf9f0 2023-11-24T06:13:03.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:03.802+0000 7ff55d547640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5580cc190 con 0x7ff558160920 2023-11-24T06:13:03.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:03.803+0000 7ff55cd46640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff558161010 0x7ff55815df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:03.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:03.803+0000 7ff557fff640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff5580cf9f0 0x7ff5580ccf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:03.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:03.803+0000 7ff54ffff640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff558160920 0x7ff558149620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:03.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:03.803+0000 7ff557fff640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff5580cf9f0 0x7ff5580ccf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.92:46136/0 (socket says 172.21.15.92:46136) 2023-11-24T06:13:03.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:03.803+0000 7ff55cd46640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff558161010 0x7ff55815df10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:54584/0 (socket says 172.21.15.92:54584) 2023-11-24T06:13:03.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:03.803+0000 7ff557fff640 1 -- 172.21.15.92:0/3168431438 learned_addr learned my addr 172.21.15.92:0/3168431438 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:13:03.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:03.803+0000 7ff557fff640 1 -- 172.21.15.92:0/3168431438 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff558160920 msgr2=0x7ff558149620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:03.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:03.803+0000 7ff557fff640 1 --2- 172.21.15.92:0/3168431438 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff558160920 0x7ff558149620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:03.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:03.803+0000 7ff557fff640 1 -- 172.21.15.92:0/3168431438 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff558161010 msgr2=0x7ff55815df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:03.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:03.803+0000 7ff557fff640 1 --2- 172.21.15.92:0/3168431438 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff558161010 0x7ff55815df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:03.808 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:03.803+0000 7ff557fff640 1 -- 172.21.15.92:0/3168431438 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5400094d0 con 0x7ff5580cf9f0 2023-11-24T06:13:03.808 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:03.803+0000 7ff55cd46640 1 --2- 172.21.15.92:0/3168431438 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff558161010 0x7ff55815df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:13:03.808 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:03.804+0000 7ff557fff640 1 --2- 172.21.15.92:0/3168431438 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff5580cf9f0 0x7ff5580ccf50 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7ff54800d830 tx=0x7ff54800dcf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:03.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:03.804+0000 7ff5557fa640 1 -- 172.21.15.92:0/3168431438 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff54800b6c0 con 0x7ff5580cf9f0 2023-11-24T06:13:03.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:03.804+0000 7ff5557fa640 1 -- 172.21.15.92:0/3168431438 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff54800b820 con 0x7ff5580cf9f0 2023-11-24T06:13:03.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:03.804+0000 7ff55d547640 1 -- 172.21.15.92:0/3168431438 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5580cc470 con 0x7ff5580cf9f0 2023-11-24T06:13:03.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:03.804+0000 7ff5557fa640 1 -- 172.21.15.92:0/3168431438 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff54800bbb0 con 0x7ff5580cf9f0 2023-11-24T06:13:03.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:03.804+0000 7ff55d547640 1 -- 172.21.15.92:0/3168431438 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5580cca30 con 0x7ff5580cf9f0 2023-11-24T06:13:03.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:03.805+0000 7ff55d547640 1 -- 172.21.15.92:0/3168431438 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff558067450 con 0x7ff5580cf9f0 2023-11-24T06:13:03.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:03.806+0000 7ff5557fa640 1 -- 172.21.15.92:0/3168431438 <== mon.1 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7ff54801e460 con 0x7ff5580cf9f0 2023-11-24T06:13:03.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:03.807+0000 7ff5557fa640 1 --2- 172.21.15.92:0/3168431438 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7ff53806aed0 0x7ff53806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:03.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:03.807+0000 7ff5557fa640 1 -- 172.21.15.92:0/3168431438 <== mon.1 v2:172.21.15.118:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 4943+0+0 (secure 0 0 0) 0x7ff548089a80 con 0x7ff5580cf9f0 2023-11-24T06:13:03.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:03.807+0000 7ff54ffff640 1 --2- 172.21.15.92:0/3168431438 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7ff53806aed0 0x7ff53806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:03.811 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:03.808+0000 7ff54ffff640 1 --2- 172.21.15.92:0/3168431438 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7ff53806aed0 0x7ff53806d390 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7ff540036000 tx=0x7ff540008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:03.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:03.814+0000 7ff5557fa640 1 -- 172.21.15.92:0/3168431438 <== mon.1 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff548054df0 con 0x7ff5580cf9f0 2023-11-24T06:13:04.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:04.012+0000 7ff55d547640 1 -- 172.21.15.92:0/3168431438 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7ff55808c7e0 con 0x7ff5580cf9f0 2023-11-24T06:13:04.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:04.109+0000 7ff5557fa640 1 -- 172.21.15.92:0/3168431438 <== mon.1 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v57) v1 ==== 97+0+0 (secure 0 0 0) 0x7ff548058410 con 0x7ff5580cf9f0 2023-11-24T06:13:04.113 INFO:teuthology.orchestra.run.smithi092.stderr: removed all blocklist entries 2023-11-24T06:13:04.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:04.112+0000 7ff55d547640 1 -- 172.21.15.92:0/3168431438 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7ff53806aed0 msgr2=0x7ff53806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:04.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:04.112+0000 7ff55d547640 1 --2- 172.21.15.92:0/3168431438 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7ff53806aed0 0x7ff53806d390 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7ff540036000 tx=0x7ff540008030 comp rx=0 tx=0).stop 2023-11-24T06:13:04.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:04.112+0000 7ff55d547640 1 -- 172.21.15.92:0/3168431438 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff5580cf9f0 msgr2=0x7ff5580ccf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:04.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:04.112+0000 7ff55d547640 1 --2- 172.21.15.92:0/3168431438 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff5580cf9f0 0x7ff5580ccf50 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7ff54800d830 tx=0x7ff54800dcf0 comp rx=0 tx=0).stop 2023-11-24T06:13:04.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:04.112+0000 7ff55d547640 1 -- 172.21.15.92:0/3168431438 shutdown_connections 2023-11-24T06:13:04.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:04.112+0000 7ff55d547640 1 --2- 172.21.15.92:0/3168431438 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7ff53806aed0 0x7ff53806d390 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:04.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:04.112+0000 7ff55d547640 1 --2- 172.21.15.92:0/3168431438 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff558161010 0x7ff55815df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:04.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:04.112+0000 7ff55d547640 1 --2- 172.21.15.92:0/3168431438 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff558160920 0x7ff558149620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:04.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:04.112+0000 7ff55d547640 1 --2- 172.21.15.92:0/3168431438 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff5580cf9f0 0x7ff5580ccf50 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:04.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:04.112+0000 7ff55d547640 1 -- 172.21.15.92:0/3168431438 >> 172.21.15.92:0/3168431438 conn(0x7ff5580d3090 msgr2=0x7ff55806c3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:13:04.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:04.113+0000 7ff55d547640 1 -- 172.21.15.92:0/3168431438 shutdown_connections 2023-11-24T06:13:04.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:04.113+0000 7ff55d547640 1 -- 172.21.15.92:0/3168431438 wait complete. 2023-11-24T06:13:04.136 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-11-24T06:13:04.343 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:04.340+0000 7f1174ddb640 1 Processor -- start 2023-11-24T06:13:04.343 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:04.340+0000 7f1174ddb640 1 -- start start 2023-11-24T06:13:04.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:04.341+0000 7f1174ddb640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f11700cccf0 0x7f117015c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:04.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:04.342+0000 7f1174ddb640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f117006ca10 con 0x7f117006c470 2023-11-24T06:13:04.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:04.342+0000 7f1174ddb640 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f117006ccf0 con 0x7f11701447f0 2023-11-24T06:13:04.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:04.342+0000 7f1174ddb640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f117006d690 con 0x7f11700cccf0 2023-11-24T06:13:04.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:04.342+0000 7f116ffff640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f11700cccf0 0x7f117015c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:04.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:04.342+0000 7f116effd640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f117006c470 0x7f11700cc5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:47896/0 (socket says 172.21.15.92:47896) 2023-11-24T06:13:04.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:04.342+0000 7f116ffff640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f11700cccf0 0x7f117015c7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:54590/0 (socket says 172.21.15.92:54590) 2023-11-24T06:13:04.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:04.342+0000 7f116effd640 1 -- 172.21.15.92:0/1261606723 learned_addr learned my addr 172.21.15.92:0/1261606723 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-24T06:13:04.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:04.342+0000 7f116ffff640 1 -- 172.21.15.92:0/1261606723 >> v1:172.21.15.118:6789/0 conn(0x7f11701447f0 legacy=0x7f117015c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:04.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:04.342+0000 7f116ffff640 1 -- 172.21.15.92:0/1261606723 >> v1:172.21.15.92:6789/0 conn(0x7f117006c470 legacy=0x7f11700cc5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:04.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:04.342+0000 7f116ffff640 1 -- 172.21.15.92:0/1261606723 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f117006cfd0 con 0x7f11700cccf0 2023-11-24T06:13:04.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:04.342+0000 7f116e7fc640 1 -- 172.21.15.92:0/1261606723 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3087650639 0 0) 0x7f117006ca10 con 0x7f117006c470 2023-11-24T06:13:04.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:04.343+0000 7f116ffff640 1 --2- 172.21.15.92:0/1261606723 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f11700cccf0 0x7f117015c7c0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f11680097a0 tx=0x7f116802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65fcec31e7fb4baa server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:04.348 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:04.344+0000 7f116e7fc640 1 -- 172.21.15.92:0/1261606723 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1168031830 con 0x7f11700cccf0 2023-11-24T06:13:04.348 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:04.344+0000 7f116e7fc640 1 -- 172.21.15.92:0/1261606723 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1168031990 con 0x7f11700cccf0 2023-11-24T06:13:04.348 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:04.344+0000 7f116e7fc640 1 -- 172.21.15.92:0/1261606723 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1168031d20 con 0x7f11700cccf0 2023-11-24T06:13:04.348 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:04.344+0000 7f1174ddb640 1 -- 172.21.15.92:0/1261606723 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f11700cccf0 msgr2=0x7f117015c7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:04.348 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:04.344+0000 7f1174ddb640 1 --2- 172.21.15.92:0/1261606723 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f11700cccf0 0x7f117015c7c0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f11680097a0 tx=0x7f116802b8d0 comp rx=0 tx=0).stop 2023-11-24T06:13:04.348 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:04.344+0000 7f1174ddb640 1 -- 172.21.15.92:0/1261606723 shutdown_connections 2023-11-24T06:13:04.348 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:04.344+0000 7f1174ddb640 1 --2- 172.21.15.92:0/1261606723 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f11700cccf0 0x7f117015c7c0 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:04.348 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:04.344+0000 7f1174ddb640 1 -- 172.21.15.92:0/1261606723 >> 172.21.15.92:0/1261606723 conn(0x7f117006a3e0 msgr2=0x7f117006a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:13:04.348 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:04.345+0000 7f1174ddb640 1 -- 172.21.15.92:0/1261606723 shutdown_connections 2023-11-24T06:13:04.348 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:04.345+0000 7f1174ddb640 1 -- 172.21.15.92:0/1261606723 wait complete. 2023-11-24T06:13:04.349 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:04.346+0000 7f1174ddb640 1 Processor -- start 2023-11-24T06:13:04.349 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:04.346+0000 7f1174ddb640 1 -- start start 2023-11-24T06:13:04.351 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:04.347+0000 7f1174ddb640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f117006c470 0x7f11700b10b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:04.351 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:04.347+0000 7f1174ddb640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f11700cccf0 0x7f1170157e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:04.351 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:04.347+0000 7f1174ddb640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f11701447f0 0x7f11701f4150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:04.351 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:04.347+0000 7f1174ddb640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11700b15f0 con 0x7f11701447f0 2023-11-24T06:13:04.351 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:04.347+0000 7f1174ddb640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11700ae8d0 con 0x7f11700cccf0 2023-11-24T06:13:04.351 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:04.347+0000 7f1174ddb640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11700aebb0 con 0x7f117006c470 2023-11-24T06:13:04.351 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:04.348+0000 7f116ffff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f11701447f0 0x7f11701f4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:04.351 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:04.348+0000 7f116ffff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f11701447f0 0x7f11701f4150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:54596/0 (socket says 172.21.15.92:54596) 2023-11-24T06:13:04.351 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:04.348+0000 7f116ffff640 1 -- 172.21.15.92:0/3993851046 learned_addr learned my addr 172.21.15.92:0/3993851046 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:13:04.352 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:04.348+0000 7f116effd640 1 --2- 172.21.15.92:0/3993851046 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f11700cccf0 0x7f1170157e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:04.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:04.348+0000 7f116ffff640 1 -- 172.21.15.92:0/3993851046 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f117006c470 msgr2=0x7f11700b10b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:04.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:04.348+0000 7f116f7fe640 1 --2- 172.21.15.92:0/3993851046 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f117006c470 0x7f11700b10b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:04.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:04.348+0000 7f116ffff640 1 --2- 172.21.15.92:0/3993851046 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f117006c470 0x7f11700b10b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:04.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:04.348+0000 7f116ffff640 1 -- 172.21.15.92:0/3993851046 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f11700cccf0 msgr2=0x7f1170157e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:04.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:04.348+0000 7f116ffff640 1 --2- 172.21.15.92:0/3993851046 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f11700cccf0 0x7f1170157e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:04.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:04.348+0000 7f116ffff640 1 -- 172.21.15.92:0/3993851046 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1168009400 con 0x7f11701447f0 2023-11-24T06:13:04.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:04.348+0000 7f116effd640 1 --2- 172.21.15.92:0/3993851046 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f11700cccf0 0x7f1170157e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:13:04.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:04.349+0000 7f116ffff640 1 --2- 172.21.15.92:0/3993851046 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f11701447f0 0x7f11701f4150 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f11680097a0 tx=0x7f11680046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:04.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:04.349+0000 7f1157fff640 1 -- 172.21.15.92:0/3993851046 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f116803ca80 con 0x7f11701447f0 2023-11-24T06:13:04.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:04.349+0000 7f1157fff640 1 -- 172.21.15.92:0/3993851046 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f116803cc20 con 0x7f11701447f0 2023-11-24T06:13:04.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:04.349+0000 7f1157fff640 1 -- 172.21.15.92:0/3993851046 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1168004030 con 0x7f11701447f0 2023-11-24T06:13:04.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:04.349+0000 7f1174ddb640 1 -- 172.21.15.92:0/3993851046 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11700aee90 con 0x7f11701447f0 2023-11-24T06:13:04.354 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:04.349+0000 7f1174ddb640 1 -- 172.21.15.92:0/3993851046 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11701583c0 con 0x7f11701447f0 2023-11-24T06:13:04.354 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:04.350+0000 7f1174ddb640 1 -- 172.21.15.92:0/3993851046 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11700af3b0 con 0x7f11701447f0 2023-11-24T06:13:04.354 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:04.351+0000 7f1157fff640 1 -- 172.21.15.92:0/3993851046 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f1168004190 con 0x7f11701447f0 2023-11-24T06:13:04.355 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:04.352+0000 7f1157fff640 1 --2- 172.21.15.92:0/3993851046 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f115c06aed0 0x7f115c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:04.355 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:04.352+0000 7f1157fff640 1 -- 172.21.15.92:0/3993851046 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f11680abce0 con 0x7f11701447f0 2023-11-24T06:13:04.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:04.353+0000 7f116f7fe640 1 --2- 172.21.15.92:0/3993851046 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f115c06aed0 0x7f115c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:04.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:04.353+0000 7f116f7fe640 1 --2- 172.21.15.92:0/3993851046 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f115c06aed0 0x7f115c06d390 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f1160000f60 tx=0x7f1160009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:04.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:04.357+0000 7f1157fff640 1 -- 172.21.15.92:0/3993851046 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f11700af3b0 con 0x7f11701447f0 2023-11-24T06:13:04.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:04.617+0000 7f1174ddb640 1 -- 172.21.15.92:0/3993851046 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f11700af3b0 con 0x7f11701447f0 2023-11-24T06:13:04.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:04.618+0000 7f1157fff640 1 -- 172.21.15.92:0/3993851046 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v10) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f11700af3b0 con 0x7f11701447f0 2023-11-24T06:13:04.621 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-24T06:13:04.622 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-24T06:13:04.622 INFO:teuthology.orchestra.run.smithi092.stdout: "auth_dump": [ 2023-11-24T06:13:04.622 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-24T06:13:04.622 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "mds.a", 2023-11-24T06:13:04.622 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQAQPmBl501xBxAAEtXTFSRjPeIoKJpcPILQwA==", 2023-11-24T06:13:04.622 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-11-24T06:13:04.622 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-11-24T06:13:04.622 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow *", 2023-11-24T06:13:04.622 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow *", 2023-11-24T06:13:04.622 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-11-24T06:13:04.622 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-24T06:13:04.623 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-24T06:13:04.623 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-24T06:13:04.623 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "mds.b", 2023-11-24T06:13:04.623 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQAQPmBlaY/AKhAArTolS+ALdzAB8M0lQ3/Mjg==", 2023-11-24T06:13:04.623 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-11-24T06:13:04.623 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-11-24T06:13:04.623 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow *", 2023-11-24T06:13:04.623 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow *", 2023-11-24T06:13:04.623 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-11-24T06:13:04.623 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-24T06:13:04.623 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-24T06:13:04.623 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-24T06:13:04.623 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "mds.c", 2023-11-24T06:13:04.624 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQAQPmBl4VbbGBAAkJwrHXAC4d5S2hdPIcE/Cg==", 2023-11-24T06:13:04.624 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-11-24T06:13:04.624 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-11-24T06:13:04.624 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow *", 2023-11-24T06:13:04.624 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow *", 2023-11-24T06:13:04.624 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-11-24T06:13:04.624 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-24T06:13:04.624 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-24T06:13:04.624 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-24T06:13:04.624 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "mds.d", 2023-11-24T06:13:04.624 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQARPmBl00MmABAAvqEvZ0bL8D1oHg4gcKXEyA==", 2023-11-24T06:13:04.624 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-11-24T06:13:04.625 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-11-24T06:13:04.625 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow *", 2023-11-24T06:13:04.625 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow *", 2023-11-24T06:13:04.625 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-11-24T06:13:04.625 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-24T06:13:04.625 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-24T06:13:04.625 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-24T06:13:04.625 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.0", 2023-11-24T06:13:04.625 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQAVPmBlr+4mLRAAgsX9kvv5frsFc+E1VoO8Fw==", 2023-11-24T06:13:04.625 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-11-24T06:13:04.625 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-11-24T06:13:04.625 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-11-24T06:13:04.626 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-11-24T06:13:04.626 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-24T06:13:04.626 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-24T06:13:04.626 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-24T06:13:04.626 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.1", 2023-11-24T06:13:04.626 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQAZPmBl7IUIChAA/DL645CltqHdzRS8r4Lksg==", 2023-11-24T06:13:04.626 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-11-24T06:13:04.626 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-11-24T06:13:04.626 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-11-24T06:13:04.626 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-11-24T06:13:04.626 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-24T06:13:04.626 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-24T06:13:04.627 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-24T06:13:04.627 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.2", 2023-11-24T06:13:04.627 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQAcPmBlJ5VbIhAAEnc+QX3whpwM0Ok9WD8dcA==", 2023-11-24T06:13:04.627 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-11-24T06:13:04.627 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-11-24T06:13:04.627 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-11-24T06:13:04.627 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-11-24T06:13:04.627 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-24T06:13:04.627 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-24T06:13:04.627 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-24T06:13:04.627 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.3", 2023-11-24T06:13:04.627 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQAfPmBlKdS9KxAAbdaBTd1CEf/A6RyKxc9t6A==", 2023-11-24T06:13:04.628 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-11-24T06:13:04.628 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-11-24T06:13:04.628 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-11-24T06:13:04.628 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-11-24T06:13:04.628 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-24T06:13:04.628 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-24T06:13:04.628 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-24T06:13:04.628 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.4", 2023-11-24T06:13:04.628 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQAkPmBlnOJ6KxAAuLTyFpOBStxC5FevOc4pKw==", 2023-11-24T06:13:04.628 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-11-24T06:13:04.628 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-11-24T06:13:04.628 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-11-24T06:13:04.628 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-11-24T06:13:04.629 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-24T06:13:04.629 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-24T06:13:04.629 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-24T06:13:04.629 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.5", 2023-11-24T06:13:04.629 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQAoPmBlrkDXCBAAwv3u79yJeeTjreYWfr/yvQ==", 2023-11-24T06:13:04.629 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-11-24T06:13:04.629 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-11-24T06:13:04.629 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-11-24T06:13:04.629 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-11-24T06:13:04.629 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-24T06:13:04.629 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-24T06:13:04.629 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-24T06:13:04.630 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.6", 2023-11-24T06:13:04.630 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQArPmBlP1AoIRAA8D8nUCwXZRHZmB9d73fYbQ==", 2023-11-24T06:13:04.630 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-11-24T06:13:04.630 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-11-24T06:13:04.630 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-11-24T06:13:04.630 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-11-24T06:13:04.630 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-24T06:13:04.630 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-24T06:13:04.630 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-24T06:13:04.630 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.7", 2023-11-24T06:13:04.630 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQAuPmBlUBKMKRAA3obYjr9s13SuYnbFt+Y0bQ==", 2023-11-24T06:13:04.630 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-11-24T06:13:04.631 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-11-24T06:13:04.631 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-11-24T06:13:04.631 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-11-24T06:13:04.631 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-24T06:13:04.631 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-24T06:13:04.631 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-24T06:13:04.631 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "client.0", 2023-11-24T06:13:04.631 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQARPmBlGMUsDhAAlgrJvA/TT5hpUYd5o9OzIQ==", 2023-11-24T06:13:04.631 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-11-24T06:13:04.631 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-11-24T06:13:04.631 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow r", 2023-11-24T06:13:04.631 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-11-24T06:13:04.632 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-24T06:13:04.632 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-24T06:13:04.632 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-24T06:13:04.632 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "client.1", 2023-11-24T06:13:04.632 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQARPmBlOeviIxAA/egv9DYEDJ+1yKDFbsY1pw==", 2023-11-24T06:13:04.632 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-11-24T06:13:04.632 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-11-24T06:13:04.632 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow r", 2023-11-24T06:13:04.632 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow rw", 2023-11-24T06:13:04.632 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow rwx" 2023-11-24T06:13:04.632 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-24T06:13:04.632 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-24T06:13:04.633 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-24T06:13:04.633 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "client.2", 2023-11-24T06:13:04.633 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQARPmBlIqJ9GBAA46UmMl1hl/522waZSndXhw==", 2023-11-24T06:13:04.633 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-11-24T06:13:04.633 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-11-24T06:13:04.633 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow r", 2023-11-24T06:13:04.633 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow rw", 2023-11-24T06:13:04.633 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow rwx" 2023-11-24T06:13:04.633 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-24T06:13:04.633 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-24T06:13:04.633 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-24T06:13:04.633 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "client.3", 2023-11-24T06:13:04.634 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQARPmBlQRCDLhAA80Oc0yKHPMD5EjVxqurfPw==", 2023-11-24T06:13:04.634 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-11-24T06:13:04.634 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-11-24T06:13:04.634 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow r", 2023-11-24T06:13:04.634 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow rw", 2023-11-24T06:13:04.634 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow rwx" 2023-11-24T06:13:04.634 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-24T06:13:04.634 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-24T06:13:04.634 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-24T06:13:04.634 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "client.admin", 2023-11-24T06:13:04.634 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQAOPmBl/8h5MxAAs//puGUeKqNyBdlTaWY/1w==", 2023-11-24T06:13:04.634 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-11-24T06:13:04.635 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow *", 2023-11-24T06:13:04.635 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow *", 2023-11-24T06:13:04.635 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow *", 2023-11-24T06:13:04.635 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-11-24T06:13:04.635 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-24T06:13:04.635 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-24T06:13:04.635 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-24T06:13:04.635 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "mgr.x", 2023-11-24T06:13:04.635 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQAPPmBl1LwlNxAAsk8YjJh9jW0OCcGdfnBA1w==", 2023-11-24T06:13:04.635 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-11-24T06:13:04.635 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow *", 2023-11-24T06:13:04.635 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile mgr", 2023-11-24T06:13:04.636 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-11-24T06:13:04.636 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-24T06:13:04.636 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-24T06:13:04.636 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-24T06:13:04.636 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "mgr.y", 2023-11-24T06:13:04.636 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQAPPmBlyGxUKhAAWOTBKusZ4JOcJ1SaHJ6ZmQ==", 2023-11-24T06:13:04.636 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-11-24T06:13:04.636 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow *", 2023-11-24T06:13:04.636 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile mgr", 2023-11-24T06:13:04.636 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-11-24T06:13:04.636 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-24T06:13:04.636 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-24T06:13:04.637 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-24T06:13:04.637 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-24T06:13:04.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:04.621+0000 7f1174ddb640 1 -- 172.21.15.92:0/3993851046 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f115c06aed0 msgr2=0x7f115c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:04.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:04.621+0000 7f1174ddb640 1 --2- 172.21.15.92:0/3993851046 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f115c06aed0 0x7f115c06d390 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f1160000f60 tx=0x7f1160009040 comp rx=0 tx=0).stop 2023-11-24T06:13:04.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:04.621+0000 7f1174ddb640 1 -- 172.21.15.92:0/3993851046 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f11701447f0 msgr2=0x7f11701f4150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:04.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:04.621+0000 7f1174ddb640 1 --2- 172.21.15.92:0/3993851046 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f11701447f0 0x7f11701f4150 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f11680097a0 tx=0x7f11680046c0 comp rx=0 tx=0).stop 2023-11-24T06:13:04.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:04.621+0000 7f1174ddb640 1 -- 172.21.15.92:0/3993851046 shutdown_connections 2023-11-24T06:13:04.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:04.621+0000 7f1174ddb640 1 --2- 172.21.15.92:0/3993851046 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f115c06aed0 0x7f115c06d390 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:04.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:04.621+0000 7f1174ddb640 1 --2- 172.21.15.92:0/3993851046 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f11701447f0 0x7f11701f4150 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:04.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:04.621+0000 7f1174ddb640 1 --2- 172.21.15.92:0/3993851046 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f11700cccf0 0x7f1170157e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:04.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:04.621+0000 7f1174ddb640 1 --2- 172.21.15.92:0/3993851046 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f117006c470 0x7f11700b10b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:04.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:04.621+0000 7f1174ddb640 1 -- 172.21.15.92:0/3993851046 >> 172.21.15.92:0/3993851046 conn(0x7f117006a3e0 msgr2=0x7f117015e130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:13:04.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:04.621+0000 7f1174ddb640 1 -- 172.21.15.92:0/3993851046 shutdown_connections 2023-11-24T06:13:04.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:04.621+0000 7f1174ddb640 1 -- 172.21.15.92:0/3993851046 wait complete. 2023-11-24T06:13:04.643 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:13:04.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:04.845+0000 7fce559d5640 1 Processor -- start 2023-11-24T06:13:04.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:04.845+0000 7fce559d5640 1 -- start start 2023-11-24T06:13:04.850 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:04.847+0000 7fce559d5640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fce500cf9a0 0x7fce5015e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:04.850 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:04.847+0000 7fce559d5640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce50070c40 con 0x7fce500ae5a0 2023-11-24T06:13:04.850 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:04.847+0000 7fce559d5640 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce5006c470 con 0x7fce500b3ae0 2023-11-24T06:13:04.850 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:04.847+0000 7fce559d5640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce50071450 con 0x7fce500cf9a0 2023-11-24T06:13:04.850 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:04.847+0000 7fce4effd640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fce500ae5a0 0x7fce5006cbe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:47900/0 (socket says 172.21.15.92:47900) 2023-11-24T06:13:04.850 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:04.847+0000 7fce4effd640 1 -- 172.21.15.92:0/1094194147 learned_addr learned my addr 172.21.15.92:0/1094194147 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-24T06:13:04.850 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:04.847+0000 7fce4f7fe640 1 --2- 172.21.15.92:0/1094194147 >> v2:172.21.15.92:3300/0 conn(0x7fce500cf9a0 0x7fce5015e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:04.851 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:04.847+0000 7fce4e7fc640 1 -- 172.21.15.92:0/1094194147 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1786908692 0 0) 0x7fce50070c40 con 0x7fce500ae5a0 2023-11-24T06:13:04.851 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:04.847+0000 7fce4e7fc640 1 -- 172.21.15.92:0/1094194147 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fce3c003520 con 0x7fce500ae5a0 2023-11-24T06:13:04.851 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:04.847+0000 7fce4f7fe640 1 -- 172.21.15.92:0/1094194147 >> v1:172.21.15.118:6789/0 conn(0x7fce500b3ae0 legacy=0x7fce500cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:13:04.851 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:04.847+0000 7fce4f7fe640 1 -- 172.21.15.92:0/1094194147 >> v1:172.21.15.92:6789/0 conn(0x7fce500ae5a0 legacy=0x7fce5006cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:04.852 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:04.847+0000 7fce4f7fe640 1 -- 172.21.15.92:0/1094194147 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce5006c750 con 0x7fce500cf9a0 2023-11-24T06:13:04.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:04.849+0000 7fce4f7fe640 1 --2- 172.21.15.92:0/1094194147 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fce500cf9a0 0x7fce5015e430 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fce48009d90 tx=0x7fce4802bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5563c684bd6ecfa2 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:04.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:04.849+0000 7fce4e7fc640 1 -- 172.21.15.92:0/1094194147 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce4802cd80 con 0x7fce500cf9a0 2023-11-24T06:13:04.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:04.849+0000 7fce4e7fc640 1 -- 172.21.15.92:0/1094194147 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce48034440 con 0x7fce500cf9a0 2023-11-24T06:13:04.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:04.849+0000 7fce4e7fc640 1 -- 172.21.15.92:0/1094194147 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce480347d0 con 0x7fce500cf9a0 2023-11-24T06:13:04.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:04.849+0000 7fce559d5640 1 -- 172.21.15.92:0/1094194147 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fce500cf9a0 msgr2=0x7fce5015e430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:04.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:04.849+0000 7fce559d5640 1 --2- 172.21.15.92:0/1094194147 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fce500cf9a0 0x7fce5015e430 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fce48009d90 tx=0x7fce4802bfd0 comp rx=0 tx=0).stop 2023-11-24T06:13:04.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:04.850+0000 7fce559d5640 1 -- 172.21.15.92:0/1094194147 shutdown_connections 2023-11-24T06:13:04.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:04.850+0000 7fce559d5640 1 --2- 172.21.15.92:0/1094194147 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fce500cf9a0 0x7fce5015e430 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:04.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:04.850+0000 7fce559d5640 1 -- 172.21.15.92:0/1094194147 >> 172.21.15.92:0/1094194147 conn(0x7fce5006a800 msgr2=0x7fce5006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:13:04.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:04.850+0000 7fce559d5640 1 -- 172.21.15.92:0/1094194147 shutdown_connections 2023-11-24T06:13:04.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:04.850+0000 7fce559d5640 1 -- 172.21.15.92:0/1094194147 wait complete. 2023-11-24T06:13:04.854 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:04.851+0000 7fce559d5640 1 Processor -- start 2023-11-24T06:13:04.854 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:04.852+0000 7fce559d5640 1 -- start start 2023-11-24T06:13:04.856 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:04.852+0000 7fce559d5640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fce500ae5a0 0x7fce50155e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:04.856 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:04.852+0000 7fce559d5640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fce500b3ae0 0x7fce501f6d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:04.856 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:04.853+0000 7fce559d5640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fce500cf9a0 0x7fce500cc280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:04.856 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:04.853+0000 7fce559d5640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce50156340 con 0x7fce500ae5a0 2023-11-24T06:13:04.856 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:04.853+0000 7fce559d5640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce501f72d0 con 0x7fce500cf9a0 2023-11-24T06:13:04.856 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:04.853+0000 7fce559d5640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce500cbfe0 con 0x7fce500b3ae0 2023-11-24T06:13:04.856 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:04.853+0000 7fce4effd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fce500ae5a0 0x7fce50155e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:04.856 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:04.853+0000 7fce4effd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fce500ae5a0 0x7fce50155e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:54610/0 (socket says 172.21.15.92:54610) 2023-11-24T06:13:04.856 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:04.853+0000 7fce4effd640 1 -- 172.21.15.92:0/3551454175 learned_addr learned my addr 172.21.15.92:0/3551454175 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:13:04.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:04.853+0000 7fce4f7fe640 1 --2- 172.21.15.92:0/3551454175 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fce500cf9a0 0x7fce500cc280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:04.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:04.853+0000 7fce467fc640 1 --2- 172.21.15.92:0/3551454175 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fce500b3ae0 0x7fce501f6d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:04.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:04.853+0000 7fce4effd640 1 -- 172.21.15.92:0/3551454175 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fce500b3ae0 msgr2=0x7fce501f6d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:04.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:04.853+0000 7fce4effd640 1 --2- 172.21.15.92:0/3551454175 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fce500b3ae0 0x7fce501f6d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:04.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:04.853+0000 7fce4effd640 1 -- 172.21.15.92:0/3551454175 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fce500cf9a0 msgr2=0x7fce500cc280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:04.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:04.853+0000 7fce4effd640 1 --2- 172.21.15.92:0/3551454175 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fce500cf9a0 0x7fce500cc280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:04.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:04.853+0000 7fce4effd640 1 -- 172.21.15.92:0/3551454175 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce48009a70 con 0x7fce500ae5a0 2023-11-24T06:13:04.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:04.853+0000 7fce4f7fe640 1 --2- 172.21.15.92:0/3551454175 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fce500cf9a0 0x7fce500cc280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:13:04.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:04.854+0000 7fce4effd640 1 --2- 172.21.15.92:0/3551454175 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fce500ae5a0 0x7fce50155e00 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fce4000c560 tx=0x7fce4000ca20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:04.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:04.854+0000 7fce4cff9640 1 -- 172.21.15.92:0/3551454175 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce40005ce0 con 0x7fce500ae5a0 2023-11-24T06:13:04.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:04.854+0000 7fce4cff9640 1 -- 172.21.15.92:0/3551454175 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce40005e40 con 0x7fce500ae5a0 2023-11-24T06:13:04.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:04.854+0000 7fce559d5640 1 -- 172.21.15.92:0/3551454175 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce500cba20 con 0x7fce500ae5a0 2023-11-24T06:13:04.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:04.854+0000 7fce4cff9640 1 -- 172.21.15.92:0/3551454175 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce400072b0 con 0x7fce500ae5a0 2023-11-24T06:13:04.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:04.854+0000 7fce559d5640 1 -- 172.21.15.92:0/3551454175 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce501f6590 con 0x7fce500ae5a0 2023-11-24T06:13:04.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:04.855+0000 7fce559d5640 1 -- 172.21.15.92:0/3551454175 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce501f63f0 con 0x7fce500ae5a0 2023-11-24T06:13:04.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:04.856+0000 7fce4cff9640 1 -- 172.21.15.92:0/3551454175 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7fce40007410 con 0x7fce500ae5a0 2023-11-24T06:13:04.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:04.857+0000 7fce4cff9640 1 --2- 172.21.15.92:0/3551454175 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fce3006aed0 0x7fce3006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:04.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:04.857+0000 7fce4cff9640 1 -- 172.21.15.92:0/3551454175 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fce40088930 con 0x7fce500ae5a0 2023-11-24T06:13:04.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:04.857+0000 7fce467fc640 1 --2- 172.21.15.92:0/3551454175 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fce3006aed0 0x7fce3006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:04.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:04.858+0000 7fce467fc640 1 --2- 172.21.15.92:0/3551454175 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fce3006aed0 0x7fce3006d390 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fce501609f0 tx=0x7fce38009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:04.865 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:04.862+0000 7fce4cff9640 1 -- 172.21.15.92:0/3551454175 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fce501f63f0 con 0x7fce500ae5a0 2023-11-24T06:13:05.056 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:05.053+0000 7fce559d5640 1 -- 172.21.15.92:0/3551454175 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fce501f63f0 con 0x7fce500ae5a0 2023-11-24T06:13:05.058 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:05.057+0000 7fce4cff9640 1 -- 172.21.15.92:0/3551454175 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v59) v1 ==== 74+0+12809 (secure 0 0 0) 0x7fce501f63f0 con 0x7fce500ae5a0 2023-11-24T06:13:05.059 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-24T06:13:05.059 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":59,"fsid":"2d6dee5f-8aa3-470f-931a-50ade963b1cb","created":"2023-11-24T06:10:04.707204+0000","modified":"2023-11-24T06:13:04.119795+0000","last_up_change":"2023-11-24T06:10:17.598352+0000","last_in_change":"2023-11-24T06:10:09.040793+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:10:17.733098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ff37a9fb-135f-471f-b4b0-f7b5772006c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6803","nonce":20560714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6805","nonce":20560714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6809","nonce":20560714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6807","nonce":20560714}]},"public_addr":"172.21.15.92:6803/20560714","cluster_addr":"172.21.15.92:6805/20560714","heartbeat_back_addr":"172.21.15.92:6809/20560714","heartbeat_front_addr":"172.21.15.92:6807/20560714","state":["exists","up"]},{"osd":1,"uuid":"03cdd353-dcf1-4f38-94fe-a795137c06b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6819","nonce":2451239906}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6821","nonce":2451239906}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6825","nonce":2451239906}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6823","nonce":2451239906}]},"public_addr":"172.21.15.92:6819/2451239906","cluster_addr":"172.21.15.92:6821/2451239906","heartbeat_back_addr":"172.21.15.92:6825/2451239906","heartbeat_front_addr":"172.21.15.92:6823/2451239906","state":["exists","up"]},{"osd":2,"uuid":"a4a20d80-5d57-484b-becd-9aa8070dd3aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6811","nonce":1124979277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6813","nonce":1124979277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6817","nonce":1124979277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6815","nonce":1124979277}]},"public_addr":"172.21.15.92:6811/1124979277","cluster_addr":"172.21.15.92:6813/1124979277","heartbeat_back_addr":"172.21.15.92:6817/1124979277","heartbeat_front_addr":"172.21.15.92:6815/1124979277","state":["exists","up"]},{"osd":3,"uuid":"4648d7ce-9b29-4583-9425-4ba8fd2b1903","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6827","nonce":1734115782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6829","nonce":1734115782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6833","nonce":1734115782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6831","nonce":1734115782}]},"public_addr":"172.21.15.92:6827/1734115782","cluster_addr":"172.21.15.92:6829/1734115782","heartbeat_back_addr":"172.21.15.92:6833/1734115782","heartbeat_front_addr":"172.21.15.92:6831/1734115782","state":["exists","up"]},{"osd":4,"uuid":"3bc9d7eb-c6fa-4fcf-a3ec-ade604e05aac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6819","nonce":3425937229}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6822","nonce":3425937229}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6830","nonce":3425937229}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6826","nonce":3425937229}]},"public_addr":"172.21.15.118:6819/3425937229","cluster_addr":"172.21.15.118:6822/3425937229","heartbeat_back_addr":"172.21.15.118:6830/3425937229","heartbeat_front_addr":"172.21.15.118:6826/3425937229","state":["exists","up"]},{"osd":5,"uuid":"dce9ba6d-59ed-46d1-84fc-a84b13de10c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6801","nonce":827455984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6803","nonce":827455984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6807","nonce":827455984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6805","nonce":827455984}]},"public_addr":"172.21.15.118:6801/827455984","cluster_addr":"172.21.15.118:6803/827455984","heartbeat_back_addr":"172.21.15.118:6807/827455984","heartbeat_front_addr":"172.21.15.118:6805/827455984","state":["exists","up"]},{"osd":6,"uuid":"82aec504-ae58-4709-8d30-46f2b12081b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6809","nonce":2946986141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6811","nonce":2946986141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6815","nonce":2946986141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6813","nonce":2946986141}]},"public_addr":"172.21.15.118:6809/2946986141","cluster_addr":"172.21.15.118:6811/2946986141","heartbeat_back_addr":"172.21.15.118:6815/2946986141","heartbeat_front_addr":"172.21.15.118:6813/2946986141","state":["exists","up"]},{"osd":7,"uuid":"4ed4c624-c88f-4d35-bdb1-bf209b5dffb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6817","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6818","nonce":48122059}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6821","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6823","nonce":48122059}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6829","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6831","nonce":48122059}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6825","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6827","nonce":48122059}]},"public_addr":"172.21.15.118:6818/48122059","cluster_addr":"172.21.15.118:6823/48122059","heartbeat_back_addr":"172.21.15.118:6831/48122059","heartbeat_front_addr":"172.21.15.118:6827/48122059","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.536664+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.368324+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.271201+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.342971+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.621832+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.387110+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.632946+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.416032+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:13:05.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:05.058+0000 7fce559d5640 1 -- 172.21.15.92:0/3551454175 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fce3006aed0 msgr2=0x7fce3006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:05.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:05.058+0000 7fce559d5640 1 --2- 172.21.15.92:0/3551454175 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fce3006aed0 0x7fce3006d390 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fce501609f0 tx=0x7fce38009040 comp rx=0 tx=0).stop 2023-11-24T06:13:05.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:05.058+0000 7fce559d5640 1 -- 172.21.15.92:0/3551454175 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fce500ae5a0 msgr2=0x7fce50155e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:05.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:05.058+0000 7fce559d5640 1 --2- 172.21.15.92:0/3551454175 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fce500ae5a0 0x7fce50155e00 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fce4000c560 tx=0x7fce4000ca20 comp rx=0 tx=0).stop 2023-11-24T06:13:05.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:05.059+0000 7fce559d5640 1 -- 172.21.15.92:0/3551454175 shutdown_connections 2023-11-24T06:13:05.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:05.059+0000 7fce559d5640 1 --2- 172.21.15.92:0/3551454175 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fce3006aed0 0x7fce3006d390 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:05.063 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:05.059+0000 7fce559d5640 1 --2- 172.21.15.92:0/3551454175 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fce500cf9a0 0x7fce500cc280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:05.063 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:05.059+0000 7fce559d5640 1 --2- 172.21.15.92:0/3551454175 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fce500b3ae0 0x7fce501f6d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:05.063 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:05.059+0000 7fce559d5640 1 --2- 172.21.15.92:0/3551454175 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fce500ae5a0 0x7fce50155e00 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:05.063 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:05.059+0000 7fce559d5640 1 -- 172.21.15.92:0/3551454175 >> 172.21.15.92:0/3551454175 conn(0x7fce5006a800 msgr2=0x7fce500cf260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:13:05.063 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:05.059+0000 7fce559d5640 1 -- 172.21.15.92:0/3551454175 shutdown_connections 2023-11-24T06:13:05.063 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:05.059+0000 7fce559d5640 1 -- 172.21.15.92:0/3551454175 wait complete. 2023-11-24T06:13:05.081 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T06:10:17.733098+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T06:13:05.081 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T06:13:05.291 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:05.289+0000 7fb04ac07640 1 Processor -- start 2023-11-24T06:13:05.292 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:05.289+0000 7fb04ac07640 1 -- start start 2023-11-24T06:13:05.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:05.291+0000 7fb04ac07640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb0441540b0 0x7fb044159f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:05.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:05.291+0000 7fb04ac07640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fb04406b690 0x7fb04415dd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:05.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:05.291+0000 7fb04ac07640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb044154490 con 0x7fb04415e270 2023-11-24T06:13:05.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:05.291+0000 7fb04ac07640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb04415d550 con 0x7fb0441540b0 2023-11-24T06:13:05.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:05.291+0000 7fb04ac07640 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb04415a7b0 con 0x7fb04406b690 2023-11-24T06:13:05.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:05.291+0000 7fb0489a3640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb0441540b0 0x7fb044159f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:05.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:05.291+0000 7fb0491a4640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fb04415e270 0x7fb044070af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:47914/0 (socket says 172.21.15.92:47914) 2023-11-24T06:13:05.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:05.291+0000 7fb0491a4640 1 -- 172.21.15.92:0/364746042 learned_addr learned my addr 172.21.15.92:0/364746042 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-24T06:13:05.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:05.292+0000 7fb043fff640 1 --2- 172.21.15.92:0/364746042 >> v2:172.21.15.118:3300/0 conn(0x7fb04406b690 0x7fb04415dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:05.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:05.292+0000 7fb0489a3640 1 -- 172.21.15.92:0/364746042 >> v2:172.21.15.118:3300/0 conn(0x7fb04406b690 msgr2=0x7fb04415dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:13:05.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:05.292+0000 7fb0489a3640 1 --2- 172.21.15.92:0/364746042 >> v2:172.21.15.118:3300/0 conn(0x7fb04406b690 0x7fb04415dd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:05.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:05.292+0000 7fb0489a3640 1 -- 172.21.15.92:0/364746042 >> v1:172.21.15.92:6789/0 conn(0x7fb04415e270 legacy=0x7fb044070af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:05.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:05.292+0000 7fb0437fe640 1 -- 172.21.15.92:0/364746042 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4007458725 0 0) 0x7fb044154490 con 0x7fb04415e270 2023-11-24T06:13:05.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:05.292+0000 7fb0489a3640 1 -- 172.21.15.92:0/364746042 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb04415aa90 con 0x7fb0441540b0 2023-11-24T06:13:05.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:05.293+0000 7fb0489a3640 1 --2- 172.21.15.92:0/364746042 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb0441540b0 0x7fb044159f60 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fb04406eb80 tx=0x7fb03402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a605aea0e61bb00 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:05.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:05.293+0000 7fb0437fe640 1 -- 172.21.15.92:0/364746042 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb034031830 con 0x7fb0441540b0 2023-11-24T06:13:05.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:05.293+0000 7fb0437fe640 1 -- 172.21.15.92:0/364746042 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb034031990 con 0x7fb0441540b0 2023-11-24T06:13:05.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:05.293+0000 7fb0437fe640 1 -- 172.21.15.92:0/364746042 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb034031d20 con 0x7fb0441540b0 2023-11-24T06:13:05.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:05.294+0000 7fb04ac07640 1 -- 172.21.15.92:0/364746042 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb0441540b0 msgr2=0x7fb044159f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:05.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:05.294+0000 7fb04ac07640 1 --2- 172.21.15.92:0/364746042 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb0441540b0 0x7fb044159f60 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fb04406eb80 tx=0x7fb03402b8d0 comp rx=0 tx=0).stop 2023-11-24T06:13:05.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:05.294+0000 7fb04ac07640 1 -- 172.21.15.92:0/364746042 shutdown_connections 2023-11-24T06:13:05.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:05.294+0000 7fb04ac07640 1 --2- 172.21.15.92:0/364746042 >> v2:172.21.15.118:3300/0 conn(0x7fb04406b690 0x7fb04415dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:05.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:05.294+0000 7fb04ac07640 1 --2- 172.21.15.92:0/364746042 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb0441540b0 0x7fb044159f60 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:05.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:05.294+0000 7fb04ac07640 1 -- 172.21.15.92:0/364746042 >> 172.21.15.92:0/364746042 conn(0x7fb0440cf700 msgr2=0x7fb0440cfae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:13:05.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:05.294+0000 7fb04ac07640 1 -- 172.21.15.92:0/364746042 shutdown_connections 2023-11-24T06:13:05.299 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:05.294+0000 7fb04ac07640 1 -- 172.21.15.92:0/364746042 wait complete. 2023-11-24T06:13:05.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:05.296+0000 7fb04ac07640 1 Processor -- start 2023-11-24T06:13:05.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:05.296+0000 7fb04ac07640 1 -- start start 2023-11-24T06:13:05.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:05.297+0000 7fb04ac07640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb04406b690 0x7fb04414b620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:05.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:05.297+0000 7fb04ac07640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb0441540b0 0x7fb044150e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:05.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:05.297+0000 7fb04ac07640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb044151380 0x7fb04415c520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:05.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:05.297+0000 7fb04ac07640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0440b0a80 con 0x7fb04406b690 2023-11-24T06:13:05.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:05.297+0000 7fb04ac07640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0440aff00 con 0x7fb0441540b0 2023-11-24T06:13:05.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:05.297+0000 7fb04ac07640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0440b01e0 con 0x7fb044151380 2023-11-24T06:13:05.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:05.297+0000 7fb0489a3640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb04406b690 0x7fb04414b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:05.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:05.297+0000 7fb0489a3640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb04406b690 0x7fb04414b620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:54624/0 (socket says 172.21.15.92:54624) 2023-11-24T06:13:05.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:05.297+0000 7fb0489a3640 1 -- 172.21.15.92:0/2257308523 learned_addr learned my addr 172.21.15.92:0/2257308523 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:13:05.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:05.298+0000 7fb0489a3640 1 -- 172.21.15.92:0/2257308523 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb044151380 msgr2=0x7fb04415c520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:05.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:05.298+0000 7fb0489a3640 1 --2- 172.21.15.92:0/2257308523 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb044151380 0x7fb04415c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:05.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:05.298+0000 7fb0489a3640 1 -- 172.21.15.92:0/2257308523 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb0441540b0 msgr2=0x7fb044150e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:05.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:05.298+0000 7fb0489a3640 1 --2- 172.21.15.92:0/2257308523 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb0441540b0 0x7fb044150e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:05.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:05.298+0000 7fb0489a3640 1 -- 172.21.15.92:0/2257308523 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb034009400 con 0x7fb04406b690 2023-11-24T06:13:05.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:05.298+0000 7fb0489a3640 1 --2- 172.21.15.92:0/2257308523 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb04406b690 0x7fb04414b620 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fb034009890 tx=0x7fb0340046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:05.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:05.298+0000 7fb0417fa640 1 -- 172.21.15.92:0/2257308523 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0340317d0 con 0x7fb04406b690 2023-11-24T06:13:05.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:05.298+0000 7fb0417fa640 1 -- 172.21.15.92:0/2257308523 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb03403ba70 con 0x7fb04406b690 2023-11-24T06:13:05.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:05.298+0000 7fb0417fa640 1 -- 172.21.15.92:0/2257308523 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb03403be20 con 0x7fb04406b690 2023-11-24T06:13:05.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:05.298+0000 7fb04ac07640 1 -- 172.21.15.92:0/2257308523 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0440b04c0 con 0x7fb04406b690 2023-11-24T06:13:05.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:05.299+0000 7fb04ac07640 1 -- 172.21.15.92:0/2257308523 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb04414bb60 con 0x7fb04406b690 2023-11-24T06:13:05.304 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:05.300+0000 7fb04ac07640 1 -- 172.21.15.92:0/2257308523 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0440cd600 con 0x7fb04406b690 2023-11-24T06:13:05.304 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:05.301+0000 7fb0417fa640 1 -- 172.21.15.92:0/2257308523 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7fb0340453f0 con 0x7fb04406b690 2023-11-24T06:13:05.304 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:05.302+0000 7fb0417fa640 1 --2- 172.21.15.92:0/2257308523 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fb03006aed0 0x7fb03006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:05.304 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:05.302+0000 7fb0417fa640 1 -- 172.21.15.92:0/2257308523 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fb0340ab7c0 con 0x7fb04406b690 2023-11-24T06:13:05.304 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:05.302+0000 7fb043fff640 1 --2- 172.21.15.92:0/2257308523 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fb03006aed0 0x7fb03006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:05.305 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:05.303+0000 7fb043fff640 1 --2- 172.21.15.92:0/2257308523 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fb03006aed0 0x7fb03006d390 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fb04406afa0 tx=0x7fb038008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:05.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:05.306+0000 7fb0417fa640 1 -- 172.21.15.92:0/2257308523 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb034076b60 con 0x7fb04406b690 2023-11-24T06:13:05.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:05.494+0000 7fb04ac07640 1 -- 172.21.15.92:0/2257308523 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb0440711d0 con 0x7fb04406b690 2023-11-24T06:13:05.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:05.495+0000 7fb0417fa640 1 -- 172.21.15.92:0/2257308523 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v59) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb0340350a0 con 0x7fb04406b690 2023-11-24T06:13:05.497 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 1 2023-11-24T06:13:05.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:05.497+0000 7fb04ac07640 1 -- 172.21.15.92:0/2257308523 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fb03006aed0 msgr2=0x7fb03006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:05.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:05.497+0000 7fb04ac07640 1 --2- 172.21.15.92:0/2257308523 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fb03006aed0 0x7fb03006d390 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fb04406afa0 tx=0x7fb038008040 comp rx=0 tx=0).stop 2023-11-24T06:13:05.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:05.497+0000 7fb04ac07640 1 -- 172.21.15.92:0/2257308523 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb04406b690 msgr2=0x7fb04414b620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:05.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:05.497+0000 7fb04ac07640 1 --2- 172.21.15.92:0/2257308523 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb04406b690 0x7fb04414b620 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fb034009890 tx=0x7fb0340046c0 comp rx=0 tx=0).stop 2023-11-24T06:13:05.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:05.497+0000 7fb04ac07640 1 -- 172.21.15.92:0/2257308523 shutdown_connections 2023-11-24T06:13:05.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:05.497+0000 7fb04ac07640 1 --2- 172.21.15.92:0/2257308523 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fb03006aed0 0x7fb03006d390 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:05.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:05.497+0000 7fb04ac07640 1 --2- 172.21.15.92:0/2257308523 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb044151380 0x7fb04415c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:05.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:05.497+0000 7fb04ac07640 1 --2- 172.21.15.92:0/2257308523 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb0441540b0 0x7fb044150e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:05.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:05.497+0000 7fb04ac07640 1 --2- 172.21.15.92:0/2257308523 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb04406b690 0x7fb04414b620 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:05.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:05.497+0000 7fb04ac07640 1 -- 172.21.15.92:0/2257308523 >> 172.21.15.92:0/2257308523 conn(0x7fb0440cf700 msgr2=0x7fb044066270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:13:05.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:05.498+0000 7fb04ac07640 1 -- 172.21.15.92:0/2257308523 shutdown_connections 2023-11-24T06:13:05.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:05.498+0000 7fb04ac07640 1 -- 172.21.15.92:0/2257308523 wait complete. 2023-11-24T06:13:05.519 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-11-24T06:13:05.720 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:05.718+0000 7f410e9fb640 1 Processor -- start 2023-11-24T06:13:05.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:05.719+0000 7f410e9fb640 1 -- start start 2023-11-24T06:13:05.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:05.720+0000 7f410e9fb640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f41080cf9a0 0x7f4108166030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:05.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:05.720+0000 7f410e9fb640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f410815c5f0 con 0x7f410813e360 2023-11-24T06:13:05.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:05.720+0000 7f410e9fb640 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f410806c470 con 0x7f410813df80 2023-11-24T06:13:05.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:05.720+0000 7f410e9fb640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f410806c750 con 0x7f41080cf9a0 2023-11-24T06:13:05.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:05.721+0000 7f410cf98640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f41080cf9a0 0x7f4108166030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:05.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:05.721+0000 7f410cf98640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f41080cf9a0 0x7f4108166030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:54626/0 (socket says 172.21.15.92:54626) 2023-11-24T06:13:05.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:05.721+0000 7f410cf98640 1 -- 172.21.15.92:0/1033409173 learned_addr learned my addr 172.21.15.92:0/1033409173 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:13:05.724 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:05.721+0000 7f410cf98640 1 -- 172.21.15.92:0/1033409173 >> v1:172.21.15.118:6789/0 conn(0x7f410813df80 legacy=0x7f410815bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:13:05.724 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:05.721+0000 7f410cf98640 1 -- 172.21.15.92:0/1033409173 >> v1:172.21.15.92:6789/0 conn(0x7f410813e360 legacy=0x7f41080cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:05.724 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:05.721+0000 7f410cf98640 1 -- 172.21.15.92:0/1033409173 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f410806ca30 con 0x7f41080cf9a0 2023-11-24T06:13:05.724 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:05.722+0000 7f410cf98640 1 --2- 172.21.15.92:0/1033409173 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f41080cf9a0 0x7f4108166030 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f4100009870 tx=0x7f410002b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d87632d90e332c40 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:05.725 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:05.722+0000 7f4106ffd640 1 -- 172.21.15.92:0/1033409173 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4100030850 con 0x7f41080cf9a0 2023-11-24T06:13:05.725 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:05.722+0000 7f4106ffd640 1 -- 172.21.15.92:0/1033409173 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41000309b0 con 0x7f41080cf9a0 2023-11-24T06:13:05.725 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:05.723+0000 7f4106ffd640 1 -- 172.21.15.92:0/1033409173 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4100030d40 con 0x7f41080cf9a0 2023-11-24T06:13:05.725 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:05.723+0000 7f410e9fb640 1 -- 172.21.15.92:0/1033409173 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f41080cf9a0 msgr2=0x7f4108166030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:05.726 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:05.723+0000 7f410e9fb640 1 --2- 172.21.15.92:0/1033409173 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f41080cf9a0 0x7f4108166030 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f4100009870 tx=0x7f410002b990 comp rx=0 tx=0).stop 2023-11-24T06:13:05.726 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:05.723+0000 7f410e9fb640 1 -- 172.21.15.92:0/1033409173 shutdown_connections 2023-11-24T06:13:05.726 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:05.723+0000 7f410e9fb640 1 --2- 172.21.15.92:0/1033409173 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f41080cf9a0 0x7f4108166030 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:05.726 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:05.723+0000 7f410e9fb640 1 -- 172.21.15.92:0/1033409173 >> 172.21.15.92:0/1033409173 conn(0x7f410806a800 msgr2=0x7f410806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:13:05.726 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:05.723+0000 7f410e9fb640 1 -- 172.21.15.92:0/1033409173 shutdown_connections 2023-11-24T06:13:05.726 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:05.723+0000 7f410e9fb640 1 -- 172.21.15.92:0/1033409173 wait complete. 2023-11-24T06:13:05.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:05.726+0000 7f410e9fb640 1 Processor -- start 2023-11-24T06:13:05.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:05.726+0000 7f410e9fb640 1 -- start start 2023-11-24T06:13:05.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:05.727+0000 7f410e9fb640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f41080cf9a0 0x7f4108153170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:05.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:05.727+0000 7f410e9fb640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f410813df80 0x7f410815d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:05.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:05.727+0000 7f410e9fb640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f410813e360 0x7f410815dde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:05.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:05.727+0000 7f410e9fb640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41081e5590 con 0x7f410813df80 2023-11-24T06:13:05.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:05.727+0000 7f410e9fb640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41081e4c60 con 0x7f41080cf9a0 2023-11-24T06:13:05.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:05.727+0000 7f410e9fb640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41081e4f40 con 0x7f410813e360 2023-11-24T06:13:05.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:05.727+0000 7f41077fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f410813df80 0x7f410815d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:05.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:05.728+0000 7f41077fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f410813df80 0x7f410815d8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:54630/0 (socket says 172.21.15.92:54630) 2023-11-24T06:13:05.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:05.728+0000 7f41077fe640 1 -- 172.21.15.92:0/3367070999 learned_addr learned my addr 172.21.15.92:0/3367070999 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:13:05.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:05.728+0000 7f4107fff640 1 --2- 172.21.15.92:0/3367070999 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f41080cf9a0 0x7f4108153170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:05.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:05.728+0000 7f410cf98640 1 --2- 172.21.15.92:0/3367070999 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f410813e360 0x7f410815dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:05.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:05.728+0000 7f41077fe640 1 -- 172.21.15.92:0/3367070999 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f410813e360 msgr2=0x7f410815dde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:05.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:05.728+0000 7f41077fe640 1 --2- 172.21.15.92:0/3367070999 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f410813e360 0x7f410815dde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:05.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:05.728+0000 7f41077fe640 1 -- 172.21.15.92:0/3367070999 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f41080cf9a0 msgr2=0x7f4108153170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:05.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:05.728+0000 7f41077fe640 1 --2- 172.21.15.92:0/3367070999 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f41080cf9a0 0x7f4108153170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:05.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:05.728+0000 7f41077fe640 1 -- 172.21.15.92:0/3367070999 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41000094d0 con 0x7f410813df80 2023-11-24T06:13:05.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:05.728+0000 7f410cf98640 1 --2- 172.21.15.92:0/3367070999 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f410813e360 0x7f410815dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:13:05.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:05.728+0000 7f41077fe640 1 --2- 172.21.15.92:0/3367070999 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f410813df80 0x7f410815d8a0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f40f000c400 tx=0x7f40f000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:05.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:05.729+0000 7f4104ff9640 1 -- 172.21.15.92:0/3367070999 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40f00052d0 con 0x7f410813df80 2023-11-24T06:13:05.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:05.729+0000 7f4104ff9640 1 -- 172.21.15.92:0/3367070999 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40f0005430 con 0x7f410813df80 2023-11-24T06:13:05.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:05.729+0000 7f4104ff9640 1 -- 172.21.15.92:0/3367070999 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40f0003830 con 0x7f410813df80 2023-11-24T06:13:05.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:05.729+0000 7f410e9fb640 1 -- 172.21.15.92:0/3367070999 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41081e5220 con 0x7f410813df80 2023-11-24T06:13:05.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:05.729+0000 7f410e9fb640 1 -- 172.21.15.92:0/3367070999 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41081e5a20 con 0x7f410813df80 2023-11-24T06:13:05.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:05.730+0000 7f410e9fb640 1 -- 172.21.15.92:0/3367070999 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41081e5700 con 0x7f410813df80 2023-11-24T06:13:05.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:05.731+0000 7f4104ff9640 1 -- 172.21.15.92:0/3367070999 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f40f0003990 con 0x7f410813df80 2023-11-24T06:13:05.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:05.732+0000 7f4104ff9640 1 --2- 172.21.15.92:0/3367070999 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f40e806af00 0x7f40e806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:05.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:05.732+0000 7f4104ff9640 1 -- 172.21.15.92:0/3367070999 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f40f00888a0 con 0x7f410813df80 2023-11-24T06:13:05.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:05.732+0000 7f4107fff640 1 --2- 172.21.15.92:0/3367070999 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f40e806af00 0x7f40e806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:05.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:05.733+0000 7f4107fff640 1 --2- 172.21.15.92:0/3367070999 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f40e806af00 0x7f40e806d3c0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f41080724a0 tx=0x7f40f80072f0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:05.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:05.736+0000 7f4104ff9640 1 -- 172.21.15.92:0/3367070999 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f40f0053cb0 con 0x7f410813df80 2023-11-24T06:13:05.920 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:05.918+0000 7f410e9fb640 1 -- 172.21.15.92:0/3367070999 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f41080675d0 con 0x7f410813df80 2023-11-24T06:13:05.920 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:05.918+0000 7f4104ff9640 1 -- 172.21.15.92:0/3367070999 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v59) v1 ==== 84+0+65 (secure 0 0 0) 0x7f40f0017060 con 0x7f410813df80 2023-11-24T06:13:05.921 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-24T06:13:05.921 INFO:teuthology.orchestra.run.smithi092.stdout:[ 2023-11-24T06:13:05.921 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-24T06:13:05.921 INFO:teuthology.orchestra.run.smithi092.stdout: "poolnum": 1, 2023-11-24T06:13:05.921 INFO:teuthology.orchestra.run.smithi092.stdout: "poolname": ".mgr" 2023-11-24T06:13:05.921 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-24T06:13:05.921 INFO:teuthology.orchestra.run.smithi092.stdout:] 2023-11-24T06:13:05.923 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:05.920+0000 7f410e9fb640 1 -- 172.21.15.92:0/3367070999 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f40e806af00 msgr2=0x7f40e806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:05.923 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:05.921+0000 7f410e9fb640 1 --2- 172.21.15.92:0/3367070999 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f40e806af00 0x7f40e806d3c0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f41080724a0 tx=0x7f40f80072f0 comp rx=0 tx=0).stop 2023-11-24T06:13:05.923 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:05.921+0000 7f410e9fb640 1 -- 172.21.15.92:0/3367070999 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f410813df80 msgr2=0x7f410815d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:05.923 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:05.921+0000 7f410e9fb640 1 --2- 172.21.15.92:0/3367070999 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f410813df80 0x7f410815d8a0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f40f000c400 tx=0x7f40f000c8c0 comp rx=0 tx=0).stop 2023-11-24T06:13:05.923 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:05.921+0000 7f410e9fb640 1 -- 172.21.15.92:0/3367070999 shutdown_connections 2023-11-24T06:13:05.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:05.921+0000 7f410e9fb640 1 --2- 172.21.15.92:0/3367070999 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f40e806af00 0x7f40e806d3c0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:05.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:05.921+0000 7f410e9fb640 1 --2- 172.21.15.92:0/3367070999 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f410813e360 0x7f410815dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:05.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:05.921+0000 7f410e9fb640 1 --2- 172.21.15.92:0/3367070999 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f410813df80 0x7f410815d8a0 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:05.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:05.921+0000 7f410e9fb640 1 --2- 172.21.15.92:0/3367070999 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f41080cf9a0 0x7f4108153170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:05.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:05.921+0000 7f410e9fb640 1 -- 172.21.15.92:0/3367070999 >> 172.21.15.92:0/3367070999 conn(0x7f410806a800 msgr2=0x7f41080afae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:13:05.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:05.921+0000 7f410e9fb640 1 -- 172.21.15.92:0/3367070999 shutdown_connections 2023-11-24T06:13:05.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:05.921+0000 7f410e9fb640 1 -- 172.21.15.92:0/3367070999 wait complete. 2023-11-24T06:13:05.940 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-11-24T06:13:05.941 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-11-24T06:13:06.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:06.151+0000 7f89d46e7640 1 Processor -- start 2023-11-24T06:13:06.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:06.151+0000 7f89d46e7640 1 -- start start 2023-11-24T06:13:06.155 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:06.152+0000 7f89d46e7640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f89cc13df80 0x7f89cc15bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:06.155 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:06.152+0000 7f89d46e7640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f89cc13e360 0x7f89cc0cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:06.155 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:06.153+0000 7f89d46e7640 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89cc06c470 con 0x7f89cc15f9c0 2023-11-24T06:13:06.155 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:06.153+0000 7f89d46e7640 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89cc06c750 con 0x7f89cc13df80 2023-11-24T06:13:06.155 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:06.153+0000 7f89d46e7640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89cc06ca30 con 0x7f89cc13e360 2023-11-24T06:13:06.155 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:06.153+0000 7f89d2483640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f89cc13df80 0x7f89cc15bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:06.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:06.153+0000 7f89d1c82640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f89cc13e360 0x7f89cc0cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:06.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:06.153+0000 7f89d2483640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f89cc13df80 0x7f89cc15bee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.92:46180/0 (socket says 172.21.15.92:46180) 2023-11-24T06:13:06.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:06.153+0000 7f89d2483640 1 -- 172.21.15.92:0/665953960 learned_addr learned my addr 172.21.15.92:0/665953960 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:13:06.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:06.154+0000 7f89d2483640 1 -- 172.21.15.92:0/665953960 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f89cc13e360 msgr2=0x7f89cc0cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:13:06.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:06.154+0000 7f89d2483640 1 --2- 172.21.15.92:0/665953960 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f89cc13e360 0x7f89cc0cf290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:06.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:06.154+0000 7f89d2483640 1 -- 172.21.15.92:0/665953960 >> v1:172.21.15.118:6789/0 conn(0x7f89cc15f9c0 legacy=0x7f89cc1408d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:06.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:06.154+0000 7f89d2483640 1 -- 172.21.15.92:0/665953960 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89cc06cd10 con 0x7f89cc13df80 2023-11-24T06:13:06.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:06.156+0000 7f89d2483640 1 --2- 172.21.15.92:0/665953960 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f89cc13df80 0x7f89cc15bee0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f89c8009870 tx=0x7f89c802b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=27a58784aa32a5c4 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:06.158 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:06.156+0000 7f89d1481640 1 -- 172.21.15.92:0/665953960 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89c8030840 con 0x7f89cc13df80 2023-11-24T06:13:06.158 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:06.156+0000 7f89d1481640 1 -- 172.21.15.92:0/665953960 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89c80309a0 con 0x7f89cc13df80 2023-11-24T06:13:06.158 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:06.156+0000 7f89d1481640 1 -- 172.21.15.92:0/665953960 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89c8030d30 con 0x7f89cc13df80 2023-11-24T06:13:06.159 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:06.157+0000 7f89d46e7640 1 -- 172.21.15.92:0/665953960 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f89cc13df80 msgr2=0x7f89cc15bee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:06.159 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:06.157+0000 7f89d46e7640 1 --2- 172.21.15.92:0/665953960 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f89cc13df80 0x7f89cc15bee0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f89c8009870 tx=0x7f89c802b980 comp rx=0 tx=0).stop 2023-11-24T06:13:06.159 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:06.157+0000 7f89d46e7640 1 -- 172.21.15.92:0/665953960 shutdown_connections 2023-11-24T06:13:06.159 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:06.157+0000 7f89d46e7640 1 --2- 172.21.15.92:0/665953960 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f89cc13e360 0x7f89cc0cf290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:06.159 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:06.157+0000 7f89d46e7640 1 --2- 172.21.15.92:0/665953960 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f89cc13df80 0x7f89cc15bee0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:06.159 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:06.157+0000 7f89d46e7640 1 -- 172.21.15.92:0/665953960 >> 172.21.15.92:0/665953960 conn(0x7f89cc06a800 msgr2=0x7f89cc06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:13:06.159 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:06.157+0000 7f89d46e7640 1 -- 172.21.15.92:0/665953960 shutdown_connections 2023-11-24T06:13:06.159 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:06.157+0000 7f89d46e7640 1 -- 172.21.15.92:0/665953960 wait complete. 2023-11-24T06:13:06.161 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:06.159+0000 7f89d46e7640 1 Processor -- start 2023-11-24T06:13:06.161 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:06.159+0000 7f89d46e7640 1 -- start start 2023-11-24T06:13:06.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:06.160+0000 7f89d46e7640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f89cc13df80 0x7f89cc153270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:06.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:06.160+0000 7f89d46e7640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f89cc13e360 0x7f89cc0aa840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:06.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:06.161+0000 7f89d46e7640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f89cc15f9c0 0x7f89cc0aad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:06.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:06.161+0000 7f89d46e7640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89cc1537b0 con 0x7f89cc13e360 2023-11-24T06:13:06.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:06.161+0000 7f89d46e7640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89cc0b2350 con 0x7f89cc15f9c0 2023-11-24T06:13:06.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:06.161+0000 7f89d46e7640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89cc0b1d40 con 0x7f89cc13df80 2023-11-24T06:13:06.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:06.161+0000 7f89d1c82640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f89cc13e360 0x7f89cc0aa840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:06.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:06.161+0000 7f89d1c82640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f89cc13e360 0x7f89cc0aa840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:54642/0 (socket says 172.21.15.92:54642) 2023-11-24T06:13:06.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:06.161+0000 7f89d1c82640 1 -- 172.21.15.92:0/2545244599 learned_addr learned my addr 172.21.15.92:0/2545244599 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:13:06.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:06.161+0000 7f89d2483640 1 --2- 172.21.15.92:0/2545244599 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f89cc13df80 0x7f89cc153270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:06.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:06.161+0000 7f89d2c84640 1 --2- 172.21.15.92:0/2545244599 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f89cc15f9c0 0x7f89cc0aad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:06.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:06.161+0000 7f89d1c82640 1 -- 172.21.15.92:0/2545244599 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f89cc13df80 msgr2=0x7f89cc153270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:06.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:06.161+0000 7f89d1c82640 1 --2- 172.21.15.92:0/2545244599 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f89cc13df80 0x7f89cc153270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:06.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:06.161+0000 7f89d1c82640 1 -- 172.21.15.92:0/2545244599 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f89cc15f9c0 msgr2=0x7f89cc0aad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:06.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:06.161+0000 7f89d1c82640 1 --2- 172.21.15.92:0/2545244599 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f89cc15f9c0 0x7f89cc0aad80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:06.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:06.161+0000 7f89d1c82640 1 -- 172.21.15.92:0/2545244599 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89c80094d0 con 0x7f89cc13e360 2023-11-24T06:13:06.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:06.162+0000 7f89d1c82640 1 --2- 172.21.15.92:0/2545244599 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f89cc13e360 0x7f89cc0aa840 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f89bc00e9b0 tx=0x7f89bc00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:06.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:06.162+0000 7f89c2ffd640 1 -- 172.21.15.92:0/2545244599 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89bc009670 con 0x7f89cc13e360 2023-11-24T06:13:06.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:06.162+0000 7f89c2ffd640 1 -- 172.21.15.92:0/2545244599 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89bc00c9a0 con 0x7f89cc13e360 2023-11-24T06:13:06.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:06.162+0000 7f89d46e7640 1 -- 172.21.15.92:0/2545244599 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89cc13d270 con 0x7f89cc13e360 2023-11-24T06:13:06.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:06.162+0000 7f89c2ffd640 1 -- 172.21.15.92:0/2545244599 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89bc00cd50 con 0x7f89cc13e360 2023-11-24T06:13:06.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:06.162+0000 7f89d46e7640 1 -- 172.21.15.92:0/2545244599 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89cc0b1fe0 con 0x7f89cc13e360 2023-11-24T06:13:06.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:06.163+0000 7f89d46e7640 1 -- 172.21.15.92:0/2545244599 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89cc0675d0 con 0x7f89cc13e360 2023-11-24T06:13:06.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:06.164+0000 7f89c2ffd640 1 -- 172.21.15.92:0/2545244599 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f89bc013470 con 0x7f89cc13e360 2023-11-24T06:13:06.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:06.165+0000 7f89c2ffd640 1 --2- 172.21.15.92:0/2545244599 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f89b006af00 0x7f89b006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:06.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:06.166+0000 7f89c2ffd640 1 -- 172.21.15.92:0/2545244599 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f89bc089670 con 0x7f89cc13e360 2023-11-24T06:13:06.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:06.166+0000 7f89d2483640 1 --2- 172.21.15.92:0/2545244599 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f89b006af00 0x7f89b006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:06.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:06.166+0000 7f89d2483640 1 --2- 172.21.15.92:0/2545244599 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f89b006af00 0x7f89b006d3c0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f89c8036000 tx=0x7f89c803b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:06.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:06.170+0000 7f89c2ffd640 1 -- 172.21.15.92:0/2545244599 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f89bc054940 con 0x7f89cc13e360 2023-11-24T06:13:06.362 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:06.360+0000 7f89d46e7640 1 -- 172.21.15.92:0/2545244599 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f89cc0b2490 con 0x7f89cc13e360 2023-11-24T06:13:07.132 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:07.129+0000 7f89c2ffd640 1 -- 172.21.15.92:0/2545244599 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v60) v1 ==== 138+0+0 (secure 0 0 0) 0x7f89bc057f60 con 0x7f89cc13e360 2023-11-24T06:13:07.132 INFO:teuthology.orchestra.run.smithi092.stderr:pool 'cephfs_metadata' created 2023-11-24T06:13:07.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:07.133+0000 7f89d46e7640 1 -- 172.21.15.92:0/2545244599 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f89b006af00 msgr2=0x7f89b006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:07.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:07.133+0000 7f89d46e7640 1 --2- 172.21.15.92:0/2545244599 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f89b006af00 0x7f89b006d3c0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f89c8036000 tx=0x7f89c803b040 comp rx=0 tx=0).stop 2023-11-24T06:13:07.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:07.133+0000 7f89d46e7640 1 -- 172.21.15.92:0/2545244599 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f89cc13e360 msgr2=0x7f89cc0aa840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:07.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:07.133+0000 7f89d46e7640 1 --2- 172.21.15.92:0/2545244599 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f89cc13e360 0x7f89cc0aa840 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f89bc00e9b0 tx=0x7f89bc00ee70 comp rx=0 tx=0).stop 2023-11-24T06:13:07.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:07.133+0000 7f89d46e7640 1 -- 172.21.15.92:0/2545244599 shutdown_connections 2023-11-24T06:13:07.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:07.133+0000 7f89d46e7640 1 --2- 172.21.15.92:0/2545244599 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f89b006af00 0x7f89b006d3c0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:07.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:07.133+0000 7f89d46e7640 1 --2- 172.21.15.92:0/2545244599 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f89cc15f9c0 0x7f89cc0aad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:07.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:07.133+0000 7f89d46e7640 1 --2- 172.21.15.92:0/2545244599 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f89cc13e360 0x7f89cc0aa840 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:07.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:07.133+0000 7f89d46e7640 1 --2- 172.21.15.92:0/2545244599 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f89cc13df80 0x7f89cc153270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:07.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:07.133+0000 7f89d46e7640 1 -- 172.21.15.92:0/2545244599 >> 172.21.15.92:0/2545244599 conn(0x7f89cc06a800 msgr2=0x7f89cc06abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:13:07.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:07.133+0000 7f89d46e7640 1 -- 172.21.15.92:0/2545244599 shutdown_connections 2023-11-24T06:13:07.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:07.133+0000 7f89d46e7640 1 -- 172.21.15.92:0/2545244599 wait complete. 2023-11-24T06:13:07.163 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-11-24T06:13:07.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:07.362+0000 7f502bfff640 1 Processor -- start 2023-11-24T06:13:07.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:07.363+0000 7f502bfff640 1 -- start start 2023-11-24T06:13:07.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:07.364+0000 7f502bfff640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f502c160920 0x7f502c0b3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:07.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:07.364+0000 7f502bfff640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f502c0d0a10 con 0x7f502c0cfe70 2023-11-24T06:13:07.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:07.364+0000 7f502bfff640 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f502c161e30 con 0x7f502c0cf9f0 2023-11-24T06:13:07.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:07.364+0000 7f502bfff640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f502c0d1f50 con 0x7f502c160920 2023-11-24T06:13:07.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:07.365+0000 7f502b7fe640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f502c0cfe70 0x7f502c161720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:47936/0 (socket says 172.21.15.92:47936) 2023-11-24T06:13:07.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:07.365+0000 7f502b7fe640 1 -- 172.21.15.92:0/2569028781 learned_addr learned my addr 172.21.15.92:0/2569028781 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-24T06:13:07.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:07.365+0000 7f502affd640 1 --2- 172.21.15.92:0/2569028781 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f502c160920 0x7f502c0b3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:07.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:07.365+0000 7f5029ffb640 1 -- 172.21.15.92:0/2569028781 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3154121624 0 0) 0x7f502c0d0a10 con 0x7f502c0cfe70 2023-11-24T06:13:07.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:07.366+0000 7f5029ffb640 1 -- 172.21.15.92:0/2569028781 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5018003520 con 0x7f502c0cfe70 2023-11-24T06:13:07.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:07.366+0000 7f5029ffb640 1 -- 172.21.15.92:0/2569028781 <== mon.1 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3983971386 0 0) 0x7f502c161e30 con 0x7f502c0cf9f0 2023-11-24T06:13:07.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:07.366+0000 7f5029ffb640 1 -- 172.21.15.92:0/2569028781 --> v1:172.21.15.118:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f502c0d0a10 con 0x7f502c0cf9f0 2023-11-24T06:13:07.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:07.366+0000 7f502affd640 1 -- 172.21.15.92:0/2569028781 >> v1:172.21.15.118:6789/0 conn(0x7f502c0cf9f0 legacy=0x7f502c161010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:07.369 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:07.366+0000 7f502affd640 1 -- 172.21.15.92:0/2569028781 >> v1:172.21.15.92:6789/0 conn(0x7f502c0cfe70 legacy=0x7f502c161720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:07.369 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:07.366+0000 7f5029ffb640 1 -- 172.21.15.92:0/2569028781 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4220802670 0 0) 0x7f5018003520 con 0x7f502c0cfe70 2023-11-24T06:13:07.369 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:07.366+0000 7f502affd640 1 -- 172.21.15.92:0/2569028781 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f502c0cf080 con 0x7f502c160920 2023-11-24T06:13:07.369 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:07.367+0000 7f502affd640 1 --2- 172.21.15.92:0/2569028781 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f502c160920 0x7f502c0b3f30 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f501c009dc0 tx=0x7f501c02bde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f0c717837a985c6 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:07.370 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:07.368+0000 7f5029ffb640 1 -- 172.21.15.92:0/2569028781 <== mon.2 v2:172.21.15.118:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5024001670 con 0x7f502c160920 2023-11-24T06:13:07.370 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:07.368+0000 7f5029ffb640 1 -- 172.21.15.92:0/2569028781 <== mon.2 v2:172.21.15.118:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f501c02c910 con 0x7f502c160920 2023-11-24T06:13:07.370 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:07.368+0000 7f5029ffb640 1 -- 172.21.15.92:0/2569028781 <== mon.2 v2:172.21.15.118:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f501c02cc80 con 0x7f502c160920 2023-11-24T06:13:07.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:07.368+0000 7f502bfff640 1 -- 172.21.15.92:0/2569028781 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f502c160920 msgr2=0x7f502c0b3f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:07.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:07.368+0000 7f502bfff640 1 --2- 172.21.15.92:0/2569028781 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f502c160920 0x7f502c0b3f30 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f501c009dc0 tx=0x7f501c02bde0 comp rx=0 tx=0).stop 2023-11-24T06:13:07.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:07.368+0000 7f502bfff640 1 -- 172.21.15.92:0/2569028781 shutdown_connections 2023-11-24T06:13:07.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:07.368+0000 7f502bfff640 1 --2- 172.21.15.92:0/2569028781 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f502c160920 0x7f502c0b3f30 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:07.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:07.368+0000 7f502bfff640 1 -- 172.21.15.92:0/2569028781 >> 172.21.15.92:0/2569028781 conn(0x7f502c0d3090 msgr2=0x7f502c0d34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:13:07.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:07.368+0000 7f502bfff640 1 -- 172.21.15.92:0/2569028781 shutdown_connections 2023-11-24T06:13:07.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:07.368+0000 7f502bfff640 1 -- 172.21.15.92:0/2569028781 wait complete. 2023-11-24T06:13:07.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:07.370+0000 7f502bfff640 1 Processor -- start 2023-11-24T06:13:07.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:07.370+0000 7f502bfff640 1 -- start start 2023-11-24T06:13:07.374 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:07.371+0000 7f502bfff640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f502c0cf9f0 0x7f502c153480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:07.374 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:07.371+0000 7f502bfff640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f502c0cfe70 0x7f502c1514a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:07.374 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:07.371+0000 7f502bfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f502c160920 0x7f502c0c8510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:07.374 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:07.371+0000 7f502bfff640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f502c1f3350 con 0x7f502c160920 2023-11-24T06:13:07.375 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:07.371+0000 7f502bfff640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f502c1f7420 con 0x7f502c0cf9f0 2023-11-24T06:13:07.375 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:07.371+0000 7f502bfff640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f502c1f7140 con 0x7f502c0cfe70 2023-11-24T06:13:07.375 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:07.372+0000 7f502b7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f502c160920 0x7f502c0c8510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:07.375 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:07.372+0000 7f502b7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f502c160920 0x7f502c0c8510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:54656/0 (socket says 172.21.15.92:54656) 2023-11-24T06:13:07.375 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:07.372+0000 7f502b7fe640 1 -- 172.21.15.92:0/466742573 learned_addr learned my addr 172.21.15.92:0/466742573 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:13:07.375 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:07.372+0000 7f502affd640 1 --2- 172.21.15.92:0/466742573 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f502c0cf9f0 0x7f502c153480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:07.375 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:07.372+0000 7f502a7fc640 1 --2- 172.21.15.92:0/466742573 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f502c0cfe70 0x7f502c1514a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:07.375 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:07.372+0000 7f502b7fe640 1 -- 172.21.15.92:0/466742573 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f502c0cfe70 msgr2=0x7f502c1514a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:07.375 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:07.372+0000 7f502b7fe640 1 --2- 172.21.15.92:0/466742573 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f502c0cfe70 0x7f502c1514a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:07.375 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:07.372+0000 7f502b7fe640 1 -- 172.21.15.92:0/466742573 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f502c0cf9f0 msgr2=0x7f502c153480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:07.375 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:07.372+0000 7f502b7fe640 1 --2- 172.21.15.92:0/466742573 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f502c0cf9f0 0x7f502c153480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:07.376 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:07.372+0000 7f502b7fe640 1 -- 172.21.15.92:0/466742573 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f501c009a70 con 0x7f502c160920 2023-11-24T06:13:07.376 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:07.372+0000 7f502b7fe640 1 --2- 172.21.15.92:0/466742573 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f502c160920 0x7f502c0c8510 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f502400c780 tx=0x7f502400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:07.376 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:07.373+0000 7f50237fe640 1 -- 172.21.15.92:0/466742573 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5024005ad0 con 0x7f502c160920 2023-11-24T06:13:07.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:07.373+0000 7f50237fe640 1 -- 172.21.15.92:0/466742573 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5024005c30 con 0x7f502c160920 2023-11-24T06:13:07.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:07.373+0000 7f502bfff640 1 -- 172.21.15.92:0/466742573 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f502c1529a0 con 0x7f502c160920 2023-11-24T06:13:07.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:07.373+0000 7f50237fe640 1 -- 172.21.15.92:0/466742573 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5024007070 con 0x7f502c160920 2023-11-24T06:13:07.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:07.373+0000 7f502bfff640 1 -- 172.21.15.92:0/466742573 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f502c152f60 con 0x7f502c160920 2023-11-24T06:13:07.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:07.374+0000 7f502bfff640 1 -- 172.21.15.92:0/466742573 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f502c067450 con 0x7f502c160920 2023-11-24T06:13:07.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:07.375+0000 7f50237fe640 1 -- 172.21.15.92:0/466742573 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f50240071d0 con 0x7f502c160920 2023-11-24T06:13:07.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:07.376+0000 7f50237fe640 1 --2- 172.21.15.92:0/466742573 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f500c06aed0 0x7f500c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:07.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:07.376+0000 7f50237fe640 1 -- 172.21.15.92:0/466742573 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f5024089bd0 con 0x7f502c160920 2023-11-24T06:13:07.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:07.377+0000 7f502affd640 1 --2- 172.21.15.92:0/466742573 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f500c06aed0 0x7f500c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:07.379 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:07.377+0000 7f502affd640 1 --2- 172.21.15.92:0/466742573 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f500c06aed0 0x7f500c06d390 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f501c036000 tx=0x7f501c030000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:07.382 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:07.381+0000 7f50237fe640 1 -- 172.21.15.92:0/466742573 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5024054f80 con 0x7f502c160920 2023-11-24T06:13:07.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:07.569+0000 7f502bfff640 1 -- 172.21.15.92:0/466742573 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f502c08c7e0 con 0x7f502c160920 2023-11-24T06:13:08.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:08.146+0000 7f50237fe640 1 -- 172.21.15.92:0/466742573 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v61) v1 ==== 170+0+0 (secure 0 0 0) 0x7f50240585a0 con 0x7f502c160920 2023-11-24T06:13:08.149 INFO:teuthology.orchestra.run.smithi092.stderr:pool 'cephfs_data' created 2023-11-24T06:13:08.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:08.150+0000 7f502bfff640 1 -- 172.21.15.92:0/466742573 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f500c06aed0 msgr2=0x7f500c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:08.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:08.150+0000 7f502bfff640 1 --2- 172.21.15.92:0/466742573 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f500c06aed0 0x7f500c06d390 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f501c036000 tx=0x7f501c030000 comp rx=0 tx=0).stop 2023-11-24T06:13:08.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:08.150+0000 7f502bfff640 1 -- 172.21.15.92:0/466742573 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f502c160920 msgr2=0x7f502c0c8510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:08.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:08.150+0000 7f502bfff640 1 --2- 172.21.15.92:0/466742573 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f502c160920 0x7f502c0c8510 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f502400c780 tx=0x7f502400cc40 comp rx=0 tx=0).stop 2023-11-24T06:13:08.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:08.151+0000 7f502bfff640 1 -- 172.21.15.92:0/466742573 shutdown_connections 2023-11-24T06:13:08.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:08.151+0000 7f502bfff640 1 --2- 172.21.15.92:0/466742573 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f500c06aed0 0x7f500c06d390 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:08.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:08.151+0000 7f502bfff640 1 --2- 172.21.15.92:0/466742573 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f502c160920 0x7f502c0c8510 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:08.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:08.151+0000 7f502bfff640 1 --2- 172.21.15.92:0/466742573 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f502c0cfe70 0x7f502c1514a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:08.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:08.151+0000 7f502bfff640 1 --2- 172.21.15.92:0/466742573 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f502c0cf9f0 0x7f502c153480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:08.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:08.151+0000 7f502bfff640 1 -- 172.21.15.92:0/466742573 >> 172.21.15.92:0/466742573 conn(0x7f502c0d3090 msgr2=0x7f502c160d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:13:08.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:08.151+0000 7f502bfff640 1 -- 172.21.15.92:0/466742573 shutdown_connections 2023-11-24T06:13:08.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:08.151+0000 7f502bfff640 1 -- 172.21.15.92:0/466742573 wait complete. 2023-11-24T06:13:08.191 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-11-24T06:13:08.376 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:08.374+0000 7f661f31d640 1 Processor -- start 2023-11-24T06:13:08.376 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:08.374+0000 7f661f31d640 1 -- start start 2023-11-24T06:13:08.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:08.375+0000 7f661f31d640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f661813df80 0x7f661815bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:08.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:08.376+0000 7f661f31d640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f661806c470 con 0x7f661813e360 2023-11-24T06:13:08.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:08.376+0000 7f661f31d640 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f661806c750 con 0x7f66180cf9a0 2023-11-24T06:13:08.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:08.376+0000 7f661f31d640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f661806ca30 con 0x7f661813df80 2023-11-24T06:13:08.379 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:08.376+0000 7f661c8b8640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f661813e360 0x7f66180cf290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:47938/0 (socket says 172.21.15.92:47938) 2023-11-24T06:13:08.379 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:08.376+0000 7f661c8b8640 1 -- 172.21.15.92:0/2761285832 learned_addr learned my addr 172.21.15.92:0/2761285832 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-24T06:13:08.379 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:08.376+0000 7f661d0b9640 1 --2- 172.21.15.92:0/2761285832 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f661813df80 0x7f661815bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:08.379 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:08.376+0000 7f660ffff640 1 -- 172.21.15.92:0/2761285832 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2658097277 0 0) 0x7f661806c470 con 0x7f661813e360 2023-11-24T06:13:08.379 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:08.376+0000 7f660ffff640 1 -- 172.21.15.92:0/2761285832 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6604003520 con 0x7f661813e360 2023-11-24T06:13:08.380 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:08.377+0000 7f660ffff640 1 -- 172.21.15.92:0/2761285832 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2509832375 0 0) 0x7f6604003520 con 0x7f661813e360 2023-11-24T06:13:08.380 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:08.377+0000 7f660ffff640 1 -- 172.21.15.92:0/2761285832 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f661813df80 msgr2=0x7f661815bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:13:08.380 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:08.377+0000 7f660ffff640 1 --2- 172.21.15.92:0/2761285832 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f661813df80 0x7f661815bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:08.380 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:08.377+0000 7f660ffff640 1 -- 172.21.15.92:0/2761285832 >> v1:172.21.15.118:6789/0 conn(0x7f66180cf9a0 legacy=0x7f66180cd840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:08.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:08.377+0000 7f660ffff640 1 -- 172.21.15.92:0/2761285832 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f661806cd10 con 0x7f661813e360 2023-11-24T06:13:08.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:08.377+0000 7f660ffff640 1 -- 172.21.15.92:0/2761285832 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2607661350 0 0) 0x7f6608001670 con 0x7f661813e360 2023-11-24T06:13:08.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:08.377+0000 7f660ffff640 1 -- 172.21.15.92:0/2761285832 >> v1:172.21.15.92:6789/0 conn(0x7f661813e360 legacy=0x7f66180cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:08.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:08.377+0000 7f660ffff640 1 --2- 172.21.15.92:0/2761285832 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f66040039a0 0x7f6604003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:08.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:08.377+0000 7f660ffff640 1 --2- 172.21.15.92:0/2761285832 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f661813df80 0x7f66040047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:08.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:08.377+0000 7f660ffff640 1 --2- 172.21.15.92:0/2761285832 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6604004d00 0x7f6604005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:08.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:08.377+0000 7f660ffff640 1 -- 172.21.15.92:0/2761285832 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f661806ca30 con 0x7f66040039a0 2023-11-24T06:13:08.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:08.377+0000 7f660ffff640 1 -- 172.21.15.92:0/2761285832 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f660400ba30 con 0x7f6604004d00 2023-11-24T06:13:08.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:08.377+0000 7f660ffff640 1 -- 172.21.15.92:0/2761285832 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f660400bbf0 con 0x7f661813df80 2023-11-24T06:13:08.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:08.377+0000 7f661d0b9640 1 --2- 172.21.15.92:0/2761285832 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f66040039a0 0x7f6604003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:08.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:08.378+0000 7f661d8ba640 1 --2- 172.21.15.92:0/2761285832 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6604004d00 0x7f6604005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:08.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:08.378+0000 7f661c8b8640 1 --2- 172.21.15.92:0/2761285832 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f661813df80 0x7f66040047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:08.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:08.378+0000 7f661d0b9640 1 -- 172.21.15.92:0/2761285832 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f661813df80 msgr2=0x7f66040047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:13:08.382 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:08.378+0000 7f661d0b9640 1 --2- 172.21.15.92:0/2761285832 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f661813df80 0x7f66040047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:08.382 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:08.378+0000 7f661d0b9640 1 -- 172.21.15.92:0/2761285832 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6604004d00 msgr2=0x7f6604005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:13:08.382 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:08.378+0000 7f661d0b9640 1 --2- 172.21.15.92:0/2761285832 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6604004d00 0x7f6604005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:08.382 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:08.378+0000 7f661d0b9640 1 -- 172.21.15.92:0/2761285832 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6604003520 con 0x7f66040039a0 2023-11-24T06:13:08.383 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:08.379+0000 7f661d0b9640 1 --2- 172.21.15.92:0/2761285832 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f66040039a0 0x7f6604003dc0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f6614009a90 tx=0x7f66040057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=388a731fb0ef648c server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:08.383 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:08.379+0000 7f660ffff640 1 -- 172.21.15.92:0/2761285832 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f661402c680 con 0x7f66040039a0 2023-11-24T06:13:08.383 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:08.379+0000 7f660ffff640 1 -- 172.21.15.92:0/2761285832 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f661402c7e0 con 0x7f66040039a0 2023-11-24T06:13:08.383 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:08.379+0000 7f660ffff640 1 -- 172.21.15.92:0/2761285832 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f661402cb70 con 0x7f66040039a0 2023-11-24T06:13:08.383 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:08.380+0000 7f661f31d640 1 -- 172.21.15.92:0/2761285832 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f66040039a0 msgr2=0x7f6604003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:08.383 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:08.380+0000 7f661f31d640 1 --2- 172.21.15.92:0/2761285832 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f66040039a0 0x7f6604003dc0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f6614009a90 tx=0x7f66040057f0 comp rx=0 tx=0).stop 2023-11-24T06:13:08.383 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:08.380+0000 7f661d8ba640 1 -- 172.21.15.92:0/2761285832 reap_dead start 2023-11-24T06:13:08.383 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:08.380+0000 7f661f31d640 1 -- 172.21.15.92:0/2761285832 shutdown_connections 2023-11-24T06:13:08.383 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:08.380+0000 7f661f31d640 1 -- 172.21.15.92:0/2761285832 >> 172.21.15.92:0/2761285832 conn(0x7f661806a800 msgr2=0x7f661806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:13:08.384 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:08.380+0000 7f661f31d640 1 -- 172.21.15.92:0/2761285832 shutdown_connections 2023-11-24T06:13:08.384 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:08.380+0000 7f661f31d640 1 -- 172.21.15.92:0/2761285832 wait complete. 2023-11-24T06:13:08.384 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:08.382+0000 7f661f31d640 1 Processor -- start 2023-11-24T06:13:08.384 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:08.382+0000 7f661f31d640 1 -- start start 2023-11-24T06:13:08.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:08.383+0000 7f661f31d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f66040039a0 0x7f66181599b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:08.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:08.383+0000 7f661f31d640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6618148f10 0x7f6618157da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:08.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:08.383+0000 7f661f31d640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f66181582e0 0x7f66180b2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:08.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:08.383+0000 7f661f31d640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6618159520 con 0x7f66040039a0 2023-11-24T06:13:08.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:08.383+0000 7f661f31d640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6618158910 con 0x7f66181582e0 2023-11-24T06:13:08.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:08.383+0000 7f661f31d640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6618158bf0 con 0x7f6618148f10 2023-11-24T06:13:08.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:08.383+0000 7f661d0b9640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f66040039a0 0x7f66181599b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:08.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:08.383+0000 7f661d0b9640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f66040039a0 0x7f66181599b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:54670/0 (socket says 172.21.15.92:54670) 2023-11-24T06:13:08.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:08.383+0000 7f661d0b9640 1 -- 172.21.15.92:0/2539326155 learned_addr learned my addr 172.21.15.92:0/2539326155 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:13:08.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:08.383+0000 7f661d8ba640 1 --2- 172.21.15.92:0/2539326155 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f66181582e0 0x7f66180b2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:08.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:08.383+0000 7f661c8b8640 1 --2- 172.21.15.92:0/2539326155 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6618148f10 0x7f6618157da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:08.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:08.383+0000 7f661d0b9640 1 -- 172.21.15.92:0/2539326155 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6618148f10 msgr2=0x7f6618157da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:08.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:08.383+0000 7f661d0b9640 1 --2- 172.21.15.92:0/2539326155 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6618148f10 0x7f6618157da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:08.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:08.383+0000 7f661d0b9640 1 -- 172.21.15.92:0/2539326155 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f66181582e0 msgr2=0x7f66180b2cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:08.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:08.383+0000 7f661d0b9640 1 --2- 172.21.15.92:0/2539326155 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f66181582e0 0x7f66180b2cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:08.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:08.383+0000 7f661d0b9640 1 -- 172.21.15.92:0/2539326155 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6618158ed0 con 0x7f66040039a0 2023-11-24T06:13:08.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:08.384+0000 7f661d0b9640 1 --2- 172.21.15.92:0/2539326155 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f66040039a0 0x7f66181599b0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f661402c220 tx=0x7f6614004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:08.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:08.384+0000 7f660dffb640 1 -- 172.21.15.92:0/2539326155 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6614003810 con 0x7f66040039a0 2023-11-24T06:13:08.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:08.384+0000 7f660dffb640 1 -- 172.21.15.92:0/2539326155 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6614003970 con 0x7f66040039a0 2023-11-24T06:13:08.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:08.384+0000 7f660dffb640 1 -- 172.21.15.92:0/2539326155 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6614003ce0 con 0x7f66040039a0 2023-11-24T06:13:08.388 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:08.384+0000 7f661f31d640 1 -- 172.21.15.92:0/2539326155 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66181591b0 con 0x7f66040039a0 2023-11-24T06:13:08.388 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:08.384+0000 7f661f31d640 1 -- 172.21.15.92:0/2539326155 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6618149390 con 0x7f66040039a0 2023-11-24T06:13:08.388 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:08.385+0000 7f661f31d640 1 -- 172.21.15.92:0/2539326155 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66181586c0 con 0x7f66040039a0 2023-11-24T06:13:08.388 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:08.386+0000 7f660dffb640 1 -- 172.21.15.92:0/2539326155 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f6614034440 con 0x7f66040039a0 2023-11-24T06:13:08.389 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:08.387+0000 7f660dffb640 1 --2- 172.21.15.92:0/2539326155 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f65fc06af00 0x7f65fc06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:08.389 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:08.387+0000 7f660dffb640 1 -- 172.21.15.92:0/2539326155 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f66140ab570 con 0x7f66040039a0 2023-11-24T06:13:08.389 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:08.388+0000 7f661c8b8640 1 --2- 172.21.15.92:0/2539326155 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f65fc06af00 0x7f65fc06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:08.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:08.388+0000 7f661c8b8640 1 --2- 172.21.15.92:0/2539326155 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f65fc06af00 0x7f65fc06d3c0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f6608000f60 tx=0x7f6608009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:08.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:08.392+0000 7f660dffb640 1 -- 172.21.15.92:0/2539326155 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f66140765b0 con 0x7f66040039a0 2023-11-24T06:13:08.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:08.623+0000 7f661f31d640 1 -- 172.21.15.92:0/2539326155 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f66180675d0 con 0x7f66040039a0 2023-11-24T06:13:09.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:09.159+0000 7f660dffb640 1 -- 172.21.15.92:0/2539326155 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 10 and data pool 11 v37) v1 ==== 174+0+0 (secure 0 0 0) 0x7f66140044e0 con 0x7f66040039a0 2023-11-24T06:13:09.164 INFO:teuthology.orchestra.run.smithi092.stderr:new fs with metadata pool 10 and data pool 11 2023-11-24T06:13:09.175 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:09.173+0000 7f661f31d640 1 -- 172.21.15.92:0/2539326155 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f65fc06af00 msgr2=0x7f65fc06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:09.175 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:09.173+0000 7f661f31d640 1 --2- 172.21.15.92:0/2539326155 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f65fc06af00 0x7f65fc06d3c0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f6608000f60 tx=0x7f6608009040 comp rx=0 tx=0).stop 2023-11-24T06:13:09.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:09.173+0000 7f661f31d640 1 -- 172.21.15.92:0/2539326155 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f66040039a0 msgr2=0x7f66181599b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:09.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:09.173+0000 7f661f31d640 1 --2- 172.21.15.92:0/2539326155 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f66040039a0 0x7f66181599b0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f661402c220 tx=0x7f6614004730 comp rx=0 tx=0).stop 2023-11-24T06:13:09.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:09.173+0000 7f661f31d640 1 -- 172.21.15.92:0/2539326155 shutdown_connections 2023-11-24T06:13:09.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:09.173+0000 7f661f31d640 1 --2- 172.21.15.92:0/2539326155 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f65fc06af00 0x7f65fc06d3c0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:09.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:09.173+0000 7f661f31d640 1 --2- 172.21.15.92:0/2539326155 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f66181582e0 0x7f66180b2cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:09.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:09.173+0000 7f661f31d640 1 --2- 172.21.15.92:0/2539326155 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6618148f10 0x7f6618157da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:09.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:09.173+0000 7f661f31d640 1 --2- 172.21.15.92:0/2539326155 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f66040039a0 0x7f66181599b0 secure :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f661402c220 tx=0x7f6614004730 comp rx=0 tx=0).stop 2023-11-24T06:13:09.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:09.173+0000 7f661f31d640 1 -- 172.21.15.92:0/2539326155 >> 172.21.15.92:0/2539326155 conn(0x7f661806a800 msgr2=0x7f66180af230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:13:09.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:09.175+0000 7f661f31d640 1 -- 172.21.15.92:0/2539326155 shutdown_connections 2023-11-24T06:13:09.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:09.176+0000 7f661f31d640 1 -- 172.21.15.92:0/2539326155 wait complete. 2023-11-24T06:13:09.205 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:13:09.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:09.385+0000 7f4c23fff640 1 Processor -- start 2023-11-24T06:13:09.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:09.385+0000 7f4c23fff640 1 -- start start 2023-11-24T06:13:09.389 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:09.386+0000 7f4c23fff640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4c2415e740 0x7f4c240b2830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:09.389 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:09.386+0000 7f4c23fff640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c2415dba0 con 0x7f4c240726f0 2023-11-24T06:13:09.389 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:09.387+0000 7f4c23fff640 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c240cd130 con 0x7f4c24146650 2023-11-24T06:13:09.389 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:09.387+0000 7f4c23fff640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c2406f800 con 0x7f4c2415e740 2023-11-24T06:13:09.389 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:09.387+0000 7f4c227fc640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f4c240726f0 0x7f4c2415e030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:59280/0 (socket says 172.21.15.92:59280) 2023-11-24T06:13:09.389 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:09.387+0000 7f4c227fc640 1 -- 172.21.15.92:0/1940351231 learned_addr learned my addr 172.21.15.92:0/1940351231 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-24T06:13:09.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:09.387+0000 7f4c237fe640 1 --2- 172.21.15.92:0/1940351231 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4c2415e740 0x7f4c240b2830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:09.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:09.387+0000 7f4c21ffb640 1 -- 172.21.15.92:0/1940351231 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3533323230 0 0) 0x7f4c2415dba0 con 0x7f4c240726f0 2023-11-24T06:13:09.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:09.387+0000 7f4c21ffb640 1 -- 172.21.15.92:0/1940351231 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4c10003520 con 0x7f4c240726f0 2023-11-24T06:13:09.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:09.388+0000 7f4c21ffb640 1 -- 172.21.15.92:0/1940351231 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 137256158 0 0) 0x7f4c10003520 con 0x7f4c240726f0 2023-11-24T06:13:09.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:09.388+0000 7f4c21ffb640 1 -- 172.21.15.92:0/1940351231 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4c2415e740 msgr2=0x7f4c240b2830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:13:09.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:09.388+0000 7f4c21ffb640 1 --2- 172.21.15.92:0/1940351231 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4c2415e740 0x7f4c240b2830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:09.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:09.388+0000 7f4c21ffb640 1 -- 172.21.15.92:0/1940351231 >> v1:172.21.15.118:6789/0 conn(0x7f4c24146650 legacy=0x7f4c240cca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:09.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:09.388+0000 7f4c21ffb640 1 -- 172.21.15.92:0/1940351231 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c2406f240 con 0x7f4c240726f0 2023-11-24T06:13:09.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:09.388+0000 7f4c21ffb640 1 -- 172.21.15.92:0/1940351231 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2607661350 0 0) 0x7f4c0c001670 con 0x7f4c240726f0 2023-11-24T06:13:09.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:09.388+0000 7f4c21ffb640 1 -- 172.21.15.92:0/1940351231 >> v1:172.21.15.92:6789/0 conn(0x7f4c240726f0 legacy=0x7f4c2415e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:09.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:09.388+0000 7f4c21ffb640 1 --2- 172.21.15.92:0/1940351231 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4c100039a0 0x7f4c10003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:09.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:09.388+0000 7f4c21ffb640 1 --2- 172.21.15.92:0/1940351231 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4c100047c0 0x7f4c10004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:09.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:09.388+0000 7f4c21ffb640 1 --2- 172.21.15.92:0/1940351231 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4c2415e740 0x7f4c10005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:09.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:09.388+0000 7f4c21ffb640 1 -- 172.21.15.92:0/1940351231 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c2406f800 con 0x7f4c100047c0 2023-11-24T06:13:09.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:09.388+0000 7f4c21ffb640 1 -- 172.21.15.92:0/1940351231 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c1000ba60 con 0x7f4c100039a0 2023-11-24T06:13:09.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:09.388+0000 7f4c21ffb640 1 -- 172.21.15.92:0/1940351231 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c1000bc20 con 0x7f4c2415e740 2023-11-24T06:13:09.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:09.388+0000 7f4c227fc640 1 --2- 172.21.15.92:0/1940351231 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4c100047c0 0x7f4c10004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:09.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:09.388+0000 7f4c22ffd640 1 --2- 172.21.15.92:0/1940351231 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4c100039a0 0x7f4c10003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:09.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:09.389+0000 7f4c237fe640 1 --2- 172.21.15.92:0/1940351231 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4c2415e740 0x7f4c10005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:09.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:09.389+0000 7f4c227fc640 1 -- 172.21.15.92:0/1940351231 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4c2415e740 msgr2=0x7f4c10005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:13:09.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:09.389+0000 7f4c227fc640 1 --2- 172.21.15.92:0/1940351231 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4c2415e740 0x7f4c10005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:09.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:09.389+0000 7f4c227fc640 1 -- 172.21.15.92:0/1940351231 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4c100039a0 msgr2=0x7f4c10003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:13:09.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:09.389+0000 7f4c227fc640 1 --2- 172.21.15.92:0/1940351231 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4c100039a0 0x7f4c10003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:09.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:09.389+0000 7f4c227fc640 1 -- 172.21.15.92:0/1940351231 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c10003520 con 0x7f4c100047c0 2023-11-24T06:13:09.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:09.389+0000 7f4c237fe640 1 --2- 172.21.15.92:0/1940351231 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4c2415e740 0x7f4c10005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:13:09.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:09.390+0000 7f4c227fc640 1 --2- 172.21.15.92:0/1940351231 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4c100047c0 0x7f4c10004c40 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f4c24072af0 tx=0x7f4c100057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7481cf4e1ad8d9b server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:09.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:09.390+0000 7f4c21ffb640 1 -- 172.21.15.92:0/1940351231 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c0c005ad0 con 0x7f4c100047c0 2023-11-24T06:13:09.395 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:09.390+0000 7f4c21ffb640 1 -- 172.21.15.92:0/1940351231 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c0c005c30 con 0x7f4c100047c0 2023-11-24T06:13:09.395 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:09.390+0000 7f4c21ffb640 1 -- 172.21.15.92:0/1940351231 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c0c02c850 con 0x7f4c100047c0 2023-11-24T06:13:09.395 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:09.390+0000 7f4c23fff640 1 -- 172.21.15.92:0/1940351231 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4c100047c0 msgr2=0x7f4c10004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:09.395 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:09.390+0000 7f4c23fff640 1 --2- 172.21.15.92:0/1940351231 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4c100047c0 0x7f4c10004c40 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f4c24072af0 tx=0x7f4c100057f0 comp rx=0 tx=0).stop 2023-11-24T06:13:09.395 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:09.391+0000 7f4c237fe640 1 -- 172.21.15.92:0/1940351231 reap_dead start 2023-11-24T06:13:09.395 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:09.391+0000 7f4c23fff640 1 -- 172.21.15.92:0/1940351231 shutdown_connections 2023-11-24T06:13:09.395 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:09.391+0000 7f4c23fff640 1 -- 172.21.15.92:0/1940351231 >> 172.21.15.92:0/1940351231 conn(0x7f4c240b1d40 msgr2=0x7f4c240b2120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:13:09.395 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:09.391+0000 7f4c23fff640 1 -- 172.21.15.92:0/1940351231 shutdown_connections 2023-11-24T06:13:09.395 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:09.391+0000 7f4c23fff640 1 -- 172.21.15.92:0/1940351231 wait complete. 2023-11-24T06:13:09.395 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:09.392+0000 7f4c23fff640 1 Processor -- start 2023-11-24T06:13:09.395 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:09.393+0000 7f4c23fff640 1 -- start start 2023-11-24T06:13:09.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:09.393+0000 7f4c23fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4c100047c0 0x7f4c2415c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:09.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:09.394+0000 7f4c23fff640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4c2413f500 0x7f4c24141100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:09.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:09.394+0000 7f4c23fff640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4c2413f960 0x7f4c24141640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:09.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:09.394+0000 7f4c23fff640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c240acd60 con 0x7f4c100047c0 2023-11-24T06:13:09.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:09.394+0000 7f4c23fff640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c2415ced0 con 0x7f4c2413f500 2023-11-24T06:13:09.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:09.394+0000 7f4c23fff640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c24141f10 con 0x7f4c2413f960 2023-11-24T06:13:09.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:09.394+0000 7f4c22ffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4c100047c0 0x7f4c2415c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:09.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:09.394+0000 7f4c22ffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4c100047c0 0x7f4c2415c990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52686/0 (socket says 172.21.15.92:52686) 2023-11-24T06:13:09.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:09.394+0000 7f4c22ffd640 1 -- 172.21.15.92:0/3302083424 learned_addr learned my addr 172.21.15.92:0/3302083424 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:13:09.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:09.394+0000 7f4c237fe640 1 --2- 172.21.15.92:0/3302083424 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4c2413f960 0x7f4c24141640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:09.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:09.394+0000 7f4c227fc640 1 --2- 172.21.15.92:0/3302083424 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4c2413f500 0x7f4c24141100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:09.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:09.394+0000 7f4c22ffd640 1 -- 172.21.15.92:0/3302083424 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4c2413f960 msgr2=0x7f4c24141640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:09.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:09.394+0000 7f4c22ffd640 1 --2- 172.21.15.92:0/3302083424 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4c2413f960 0x7f4c24141640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:09.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:09.394+0000 7f4c22ffd640 1 -- 172.21.15.92:0/3302083424 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4c2413f500 msgr2=0x7f4c24141100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:09.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:09.394+0000 7f4c22ffd640 1 --2- 172.21.15.92:0/3302083424 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4c2413f500 0x7f4c24141100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:09.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:09.394+0000 7f4c22ffd640 1 -- 172.21.15.92:0/3302083424 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c241421f0 con 0x7f4c100047c0 2023-11-24T06:13:09.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:09.394+0000 7f4c227fc640 1 --2- 172.21.15.92:0/3302083424 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4c2413f500 0x7f4c24141100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:13:09.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:09.395+0000 7f4c22ffd640 1 --2- 172.21.15.92:0/3302083424 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4c100047c0 0x7f4c2415c990 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f4c1400e8c0 tx=0x7f4c1400ed80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:09.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:09.395+0000 7f4c1bfff640 1 -- 172.21.15.92:0/3302083424 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c14003b80 con 0x7f4c100047c0 2023-11-24T06:13:09.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:09.395+0000 7f4c1bfff640 1 -- 172.21.15.92:0/3302083424 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c14003ce0 con 0x7f4c100047c0 2023-11-24T06:13:09.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:09.395+0000 7f4c1bfff640 1 -- 172.21.15.92:0/3302083424 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c140052f0 con 0x7f4c100047c0 2023-11-24T06:13:09.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:09.395+0000 7f4c23fff640 1 -- 172.21.15.92:0/3302083424 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c241424d0 con 0x7f4c100047c0 2023-11-24T06:13:09.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:09.395+0000 7f4c23fff640 1 -- 172.21.15.92:0/3302083424 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c2413f030 con 0x7f4c100047c0 2023-11-24T06:13:09.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:09.397+0000 7f4c23fff640 1 -- 172.21.15.92:0/3302083424 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c240acea0 con 0x7f4c100047c0 2023-11-24T06:13:09.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:09.398+0000 7f4c1bfff640 1 -- 172.21.15.92:0/3302083424 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f4c14005450 con 0x7f4c100047c0 2023-11-24T06:13:09.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:09.399+0000 7f4c1bfff640 1 --2- 172.21.15.92:0/3302083424 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f4c0406aed0 0x7f4c0406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:09.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:09.399+0000 7f4c1bfff640 1 -- 172.21.15.92:0/3302083424 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4c14089940 con 0x7f4c100047c0 2023-11-24T06:13:09.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:09.399+0000 7f4c227fc640 1 --2- 172.21.15.92:0/3302083424 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f4c0406aed0 0x7f4c0406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:09.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:09.400+0000 7f4c227fc640 1 --2- 172.21.15.92:0/3302083424 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f4c0406aed0 0x7f4c0406d390 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f4c0c034000 tx=0x7f4c0c009030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:09.406 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:09.404+0000 7f4c1bfff640 1 -- 172.21.15.92:0/3302083424 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4c240acea0 con 0x7f4c100047c0 2023-11-24T06:13:09.593 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:09.591+0000 7f4c23fff640 1 -- 172.21.15.92:0/3302083424 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4c240acea0 con 0x7f4c100047c0 2023-11-24T06:13:09.594 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:09.592+0000 7f4c1bfff640 1 -- 172.21.15.92:0/3302083424 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v62) v1 ==== 74+0+16533 (secure 0 0 0) 0x7f4c240acea0 con 0x7f4c100047c0 2023-11-24T06:13:09.595 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-24T06:13:09.595 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":62,"fsid":"2d6dee5f-8aa3-470f-931a-50ade963b1cb","created":"2023-11-24T06:10:04.707204+0000","modified":"2023-11-24T06:13:08.624749+0000","last_up_change":"2023-11-24T06:10:17.598352+0000","last_in_change":"2023-11-24T06:10:09.040793+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:10:17.733098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:13:06.409727+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-11-24T06:13:07.623678+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ff37a9fb-135f-471f-b4b0-f7b5772006c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6803","nonce":20560714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6805","nonce":20560714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6809","nonce":20560714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6807","nonce":20560714}]},"public_addr":"172.21.15.92:6803/20560714","cluster_addr":"172.21.15.92:6805/20560714","heartbeat_back_addr":"172.21.15.92:6809/20560714","heartbeat_front_addr":"172.21.15.92:6807/20560714","state":["exists","up"]},{"osd":1,"uuid":"03cdd353-dcf1-4f38-94fe-a795137c06b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6819","nonce":2451239906}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6821","nonce":2451239906}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6825","nonce":2451239906}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6823","nonce":2451239906}]},"public_addr":"172.21.15.92:6819/2451239906","cluster_addr":"172.21.15.92:6821/2451239906","heartbeat_back_addr":"172.21.15.92:6825/2451239906","heartbeat_front_addr":"172.21.15.92:6823/2451239906","state":["exists","up"]},{"osd":2,"uuid":"a4a20d80-5d57-484b-becd-9aa8070dd3aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6811","nonce":1124979277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6813","nonce":1124979277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6817","nonce":1124979277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6815","nonce":1124979277}]},"public_addr":"172.21.15.92:6811/1124979277","cluster_addr":"172.21.15.92:6813/1124979277","heartbeat_back_addr":"172.21.15.92:6817/1124979277","heartbeat_front_addr":"172.21.15.92:6815/1124979277","state":["exists","up"]},{"osd":3,"uuid":"4648d7ce-9b29-4583-9425-4ba8fd2b1903","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6827","nonce":1734115782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6829","nonce":1734115782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6833","nonce":1734115782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6831","nonce":1734115782}]},"public_addr":"172.21.15.92:6827/1734115782","cluster_addr":"172.21.15.92:6829/1734115782","heartbeat_back_addr":"172.21.15.92:6833/1734115782","heartbeat_front_addr":"172.21.15.92:6831/1734115782","state":["exists","up"]},{"osd":4,"uuid":"3bc9d7eb-c6fa-4fcf-a3ec-ade604e05aac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6819","nonce":3425937229}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6822","nonce":3425937229}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6830","nonce":3425937229}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6826","nonce":3425937229}]},"public_addr":"172.21.15.118:6819/3425937229","cluster_addr":"172.21.15.118:6822/3425937229","heartbeat_back_addr":"172.21.15.118:6830/3425937229","heartbeat_front_addr":"172.21.15.118:6826/3425937229","state":["exists","up"]},{"osd":5,"uuid":"dce9ba6d-59ed-46d1-84fc-a84b13de10c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6801","nonce":827455984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6803","nonce":827455984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6807","nonce":827455984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6805","nonce":827455984}]},"public_addr":"172.21.15.118:6801/827455984","cluster_addr":"172.21.15.118:6803/827455984","heartbeat_back_addr":"172.21.15.118:6807/827455984","heartbeat_front_addr":"172.21.15.118:6805/827455984","state":["exists","up"]},{"osd":6,"uuid":"82aec504-ae58-4709-8d30-46f2b12081b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6809","nonce":2946986141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6811","nonce":2946986141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6815","nonce":2946986141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6813","nonce":2946986141}]},"public_addr":"172.21.15.118:6809/2946986141","cluster_addr":"172.21.15.118:6811/2946986141","heartbeat_back_addr":"172.21.15.118:6815/2946986141","heartbeat_front_addr":"172.21.15.118:6813/2946986141","state":["exists","up"]},{"osd":7,"uuid":"4ed4c624-c88f-4d35-bdb1-bf209b5dffb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6817","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6818","nonce":48122059}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6821","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6823","nonce":48122059}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6829","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6831","nonce":48122059}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6825","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6827","nonce":48122059}]},"public_addr":"172.21.15.118:6818/48122059","cluster_addr":"172.21.15.118:6823/48122059","heartbeat_back_addr":"172.21.15.118:6831/48122059","heartbeat_front_addr":"172.21.15.118:6827/48122059","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.536664+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.368324+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.271201+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.342971+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.621832+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.387110+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.632946+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.416032+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:13:09.597 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:09.594+0000 7f4c23fff640 1 -- 172.21.15.92:0/3302083424 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f4c0406aed0 msgr2=0x7f4c0406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:09.597 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:09.594+0000 7f4c23fff640 1 --2- 172.21.15.92:0/3302083424 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f4c0406aed0 0x7f4c0406d390 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f4c0c034000 tx=0x7f4c0c009030 comp rx=0 tx=0).stop 2023-11-24T06:13:09.597 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:09.594+0000 7f4c23fff640 1 -- 172.21.15.92:0/3302083424 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4c100047c0 msgr2=0x7f4c2415c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:09.597 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:09.594+0000 7f4c23fff640 1 --2- 172.21.15.92:0/3302083424 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4c100047c0 0x7f4c2415c990 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f4c1400e8c0 tx=0x7f4c1400ed80 comp rx=0 tx=0).stop 2023-11-24T06:13:09.597 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:09.595+0000 7f4c23fff640 1 -- 172.21.15.92:0/3302083424 shutdown_connections 2023-11-24T06:13:09.597 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:09.595+0000 7f4c23fff640 1 --2- 172.21.15.92:0/3302083424 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f4c0406aed0 0x7f4c0406d390 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:09.597 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:09.595+0000 7f4c23fff640 1 --2- 172.21.15.92:0/3302083424 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4c2413f960 0x7f4c24141640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:09.598 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:09.595+0000 7f4c23fff640 1 --2- 172.21.15.92:0/3302083424 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4c2413f500 0x7f4c24141100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:09.598 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:09.595+0000 7f4c23fff640 1 --2- 172.21.15.92:0/3302083424 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4c100047c0 0x7f4c2415c990 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:09.598 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:09.595+0000 7f4c23fff640 1 -- 172.21.15.92:0/3302083424 >> 172.21.15.92:0/3302083424 conn(0x7f4c240b1d40 msgr2=0x7f4c240b1270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:13:09.598 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:09.595+0000 7f4c23fff640 1 -- 172.21.15.92:0/3302083424 shutdown_connections 2023-11-24T06:13:09.598 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:09.595+0000 7f4c23fff640 1 -- 172.21.15.92:0/3302083424 wait complete. 2023-11-24T06:13:09.614 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:13:09.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:09.811+0000 7f3d653c5640 1 Processor -- start 2023-11-24T06:13:09.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:09.811+0000 7f3d653c5640 1 -- start start 2023-11-24T06:13:09.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:09.813+0000 7f3d653c5640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f3d6006c6b0 0x7f3d600b3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:09.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:09.813+0000 7f3d653c5640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d6006cea0 con 0x7f3d600cd240 2023-11-24T06:13:09.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:09.813+0000 7f3d653c5640 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d6006cbc0 con 0x7f3d6009a170 2023-11-24T06:13:09.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:09.813+0000 7f3d653c5640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d6006f350 con 0x7f3d6006c6b0 2023-11-24T06:13:09.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:09.813+0000 7f3d5effd640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f3d600cd240 0x7f3d60099a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:59292/0 (socket says 172.21.15.92:59292) 2023-11-24T06:13:09.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:09.813+0000 7f3d5effd640 1 -- 172.21.15.92:0/1680549105 learned_addr learned my addr 172.21.15.92:0/1680549105 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-24T06:13:09.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:09.813+0000 7f3d5f7fe640 1 --2- 172.21.15.92:0/1680549105 >> v2:172.21.15.92:3300/0 conn(0x7f3d6006c6b0 0x7f3d600b3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:09.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:09.813+0000 7f3d5e7fc640 1 -- 172.21.15.92:0/1680549105 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4034092392 0 0) 0x7f3d6006cea0 con 0x7f3d600cd240 2023-11-24T06:13:09.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:09.813+0000 7f3d5e7fc640 1 -- 172.21.15.92:0/1680549105 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3d4c003520 con 0x7f3d600cd240 2023-11-24T06:13:09.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:09.813+0000 7f3d5f7fe640 1 -- 172.21.15.92:0/1680549105 >> v1:172.21.15.118:6789/0 conn(0x7f3d6009a170 legacy=0x7f3d600cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:13:09.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:09.814+0000 7f3d5f7fe640 1 -- 172.21.15.92:0/1680549105 >> v1:172.21.15.92:6789/0 conn(0x7f3d600cd240 legacy=0x7f3d60099a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:09.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:09.814+0000 7f3d5f7fe640 1 -- 172.21.15.92:0/1680549105 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d6006f630 con 0x7f3d6006c6b0 2023-11-24T06:13:09.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:09.815+0000 7f3d5f7fe640 1 --2- 172.21.15.92:0/1680549105 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3d6006c6b0 0x7f3d600b3820 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f3d58009d90 tx=0x7f3d5802beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffccb6d5b2c03462 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:09.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:09.815+0000 7f3d5e7fc640 1 -- 172.21.15.92:0/1680549105 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d5802cc60 con 0x7f3d6006c6b0 2023-11-24T06:13:09.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:09.815+0000 7f3d5e7fc640 1 -- 172.21.15.92:0/1680549105 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d5802cdc0 con 0x7f3d6006c6b0 2023-11-24T06:13:09.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:09.815+0000 7f3d5e7fc640 1 -- 172.21.15.92:0/1680549105 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d58034660 con 0x7f3d6006c6b0 2023-11-24T06:13:09.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:09.815+0000 7f3d653c5640 1 -- 172.21.15.92:0/1680549105 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3d6006c6b0 msgr2=0x7f3d600b3820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:09.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:09.815+0000 7f3d653c5640 1 --2- 172.21.15.92:0/1680549105 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3d6006c6b0 0x7f3d600b3820 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f3d58009d90 tx=0x7f3d5802beb0 comp rx=0 tx=0).stop 2023-11-24T06:13:09.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:09.815+0000 7f3d653c5640 1 -- 172.21.15.92:0/1680549105 shutdown_connections 2023-11-24T06:13:09.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:09.816+0000 7f3d653c5640 1 --2- 172.21.15.92:0/1680549105 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3d6006c6b0 0x7f3d600b3820 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:09.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:09.816+0000 7f3d653c5640 1 -- 172.21.15.92:0/1680549105 >> 172.21.15.92:0/1680549105 conn(0x7f3d600b2d00 msgr2=0x7f3d600b3110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:13:09.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:09.816+0000 7f3d653c5640 1 -- 172.21.15.92:0/1680549105 shutdown_connections 2023-11-24T06:13:09.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:09.816+0000 7f3d653c5640 1 -- 172.21.15.92:0/1680549105 wait complete. 2023-11-24T06:13:09.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:09.817+0000 7f3d653c5640 1 Processor -- start 2023-11-24T06:13:09.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:09.817+0000 7f3d653c5640 1 -- start start 2023-11-24T06:13:09.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:09.818+0000 7f3d653c5640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3d6006c6b0 0x7f3d600c8cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:09.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:09.818+0000 7f3d653c5640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3d6009a170 0x7f3d601f22f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:09.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:09.818+0000 7f3d653c5640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3d600cd240 0x7f3d6015e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:09.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:09.818+0000 7f3d653c5640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d600c53e0 con 0x7f3d600cd240 2023-11-24T06:13:09.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:09.818+0000 7f3d653c5640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d6015adc0 con 0x7f3d6006c6b0 2023-11-24T06:13:09.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:09.818+0000 7f3d653c5640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d600ca1f0 con 0x7f3d6009a170 2023-11-24T06:13:09.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:09.818+0000 7f3d5f7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3d600cd240 0x7f3d6015e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:09.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:09.819+0000 7f3d5f7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3d600cd240 0x7f3d6015e9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52698/0 (socket says 172.21.15.92:52698) 2023-11-24T06:13:09.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:09.819+0000 7f3d5f7fe640 1 -- 172.21.15.92:0/841913461 learned_addr learned my addr 172.21.15.92:0/841913461 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:13:09.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:09.819+0000 7f3d5f7fe640 1 -- 172.21.15.92:0/841913461 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3d6009a170 msgr2=0x7f3d601f22f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:09.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:09.819+0000 7f3d5f7fe640 1 --2- 172.21.15.92:0/841913461 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3d6009a170 0x7f3d601f22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:09.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:09.819+0000 7f3d5f7fe640 1 -- 172.21.15.92:0/841913461 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3d6006c6b0 msgr2=0x7f3d600c8cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:09.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:09.819+0000 7f3d5f7fe640 1 --2- 172.21.15.92:0/841913461 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3d6006c6b0 0x7f3d600c8cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:09.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:09.819+0000 7f3d5f7fe640 1 -- 172.21.15.92:0/841913461 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d58009a70 con 0x7f3d600cd240 2023-11-24T06:13:09.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:09.819+0000 7f3d5f7fe640 1 --2- 172.21.15.92:0/841913461 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3d600cd240 0x7f3d6015e9a0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f3d58009d90 tx=0x7f3d58004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:09.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:09.819+0000 7f3d577fe640 1 -- 172.21.15.92:0/841913461 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d58040650 con 0x7f3d600cd240 2023-11-24T06:13:09.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:09.819+0000 7f3d577fe640 1 -- 172.21.15.92:0/841913461 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d580407b0 con 0x7f3d600cd240 2023-11-24T06:13:09.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:09.819+0000 7f3d653c5640 1 -- 172.21.15.92:0/841913461 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d600ca4d0 con 0x7f3d600cd240 2023-11-24T06:13:09.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:09.820+0000 7f3d577fe640 1 -- 172.21.15.92:0/841913461 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d58040b60 con 0x7f3d600cd240 2023-11-24T06:13:09.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:09.820+0000 7f3d653c5640 1 -- 172.21.15.92:0/841913461 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d600c5000 con 0x7f3d600cd240 2023-11-24T06:13:09.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:09.821+0000 7f3d653c5640 1 -- 172.21.15.92:0/841913461 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d60073dc0 con 0x7f3d600cd240 2023-11-24T06:13:09.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:09.822+0000 7f3d577fe640 1 -- 172.21.15.92:0/841913461 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f3d58040cc0 con 0x7f3d600cd240 2023-11-24T06:13:09.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:09.822+0000 7f3d577fe640 1 --2- 172.21.15.92:0/841913461 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f3d4006aed0 0x7f3d4006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:09.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:09.822+0000 7f3d577fe640 1 -- 172.21.15.92:0/841913461 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3d580ac520 con 0x7f3d600cd240 2023-11-24T06:13:09.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:09.823+0000 7f3d5effd640 1 --2- 172.21.15.92:0/841913461 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f3d4006aed0 0x7f3d4006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:09.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:09.823+0000 7f3d5effd640 1 --2- 172.21.15.92:0/841913461 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f3d4006aed0 0x7f3d4006d390 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f3d50001610 tx=0x7f3d50007930 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:09.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:09.827+0000 7f3d577fe640 1 -- 172.21.15.92:0/841913461 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3d60073dc0 con 0x7f3d600cd240 2023-11-24T06:13:10.020 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:10.018+0000 7f3d653c5640 1 -- 172.21.15.92:0/841913461 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3d60073dc0 con 0x7f3d600cd240 2023-11-24T06:13:10.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:10.020+0000 7f3d577fe640 1 -- 172.21.15.92:0/841913461 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v62) v1 ==== 74+0+16533 (secure 0 0 0) 0x7f3d60073dc0 con 0x7f3d600cd240 2023-11-24T06:13:10.023 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-24T06:13:10.023 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":62,"fsid":"2d6dee5f-8aa3-470f-931a-50ade963b1cb","created":"2023-11-24T06:10:04.707204+0000","modified":"2023-11-24T06:13:08.624749+0000","last_up_change":"2023-11-24T06:10:17.598352+0000","last_in_change":"2023-11-24T06:10:09.040793+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:10:17.733098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:13:06.409727+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-11-24T06:13:07.623678+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ff37a9fb-135f-471f-b4b0-f7b5772006c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6803","nonce":20560714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6805","nonce":20560714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6809","nonce":20560714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6807","nonce":20560714}]},"public_addr":"172.21.15.92:6803/20560714","cluster_addr":"172.21.15.92:6805/20560714","heartbeat_back_addr":"172.21.15.92:6809/20560714","heartbeat_front_addr":"172.21.15.92:6807/20560714","state":["exists","up"]},{"osd":1,"uuid":"03cdd353-dcf1-4f38-94fe-a795137c06b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6819","nonce":2451239906}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6821","nonce":2451239906}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6825","nonce":2451239906}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6823","nonce":2451239906}]},"public_addr":"172.21.15.92:6819/2451239906","cluster_addr":"172.21.15.92:6821/2451239906","heartbeat_back_addr":"172.21.15.92:6825/2451239906","heartbeat_front_addr":"172.21.15.92:6823/2451239906","state":["exists","up"]},{"osd":2,"uuid":"a4a20d80-5d57-484b-becd-9aa8070dd3aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6811","nonce":1124979277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6813","nonce":1124979277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6817","nonce":1124979277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6815","nonce":1124979277}]},"public_addr":"172.21.15.92:6811/1124979277","cluster_addr":"172.21.15.92:6813/1124979277","heartbeat_back_addr":"172.21.15.92:6817/1124979277","heartbeat_front_addr":"172.21.15.92:6815/1124979277","state":["exists","up"]},{"osd":3,"uuid":"4648d7ce-9b29-4583-9425-4ba8fd2b1903","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6827","nonce":1734115782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6829","nonce":1734115782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6833","nonce":1734115782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6831","nonce":1734115782}]},"public_addr":"172.21.15.92:6827/1734115782","cluster_addr":"172.21.15.92:6829/1734115782","heartbeat_back_addr":"172.21.15.92:6833/1734115782","heartbeat_front_addr":"172.21.15.92:6831/1734115782","state":["exists","up"]},{"osd":4,"uuid":"3bc9d7eb-c6fa-4fcf-a3ec-ade604e05aac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6819","nonce":3425937229}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6822","nonce":3425937229}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6830","nonce":3425937229}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6826","nonce":3425937229}]},"public_addr":"172.21.15.118:6819/3425937229","cluster_addr":"172.21.15.118:6822/3425937229","heartbeat_back_addr":"172.21.15.118:6830/3425937229","heartbeat_front_addr":"172.21.15.118:6826/3425937229","state":["exists","up"]},{"osd":5,"uuid":"dce9ba6d-59ed-46d1-84fc-a84b13de10c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6801","nonce":827455984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6803","nonce":827455984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6807","nonce":827455984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6805","nonce":827455984}]},"public_addr":"172.21.15.118:6801/827455984","cluster_addr":"172.21.15.118:6803/827455984","heartbeat_back_addr":"172.21.15.118:6807/827455984","heartbeat_front_addr":"172.21.15.118:6805/827455984","state":["exists","up"]},{"osd":6,"uuid":"82aec504-ae58-4709-8d30-46f2b12081b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6809","nonce":2946986141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6811","nonce":2946986141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6815","nonce":2946986141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6813","nonce":2946986141}]},"public_addr":"172.21.15.118:6809/2946986141","cluster_addr":"172.21.15.118:6811/2946986141","heartbeat_back_addr":"172.21.15.118:6815/2946986141","heartbeat_front_addr":"172.21.15.118:6813/2946986141","state":["exists","up"]},{"osd":7,"uuid":"4ed4c624-c88f-4d35-bdb1-bf209b5dffb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6817","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6818","nonce":48122059}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6821","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6823","nonce":48122059}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6829","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6831","nonce":48122059}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6825","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6827","nonce":48122059}]},"public_addr":"172.21.15.118:6818/48122059","cluster_addr":"172.21.15.118:6823/48122059","heartbeat_back_addr":"172.21.15.118:6831/48122059","heartbeat_front_addr":"172.21.15.118:6827/48122059","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.536664+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.368324+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.271201+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.342971+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.621832+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.387110+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.632946+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.416032+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:13:10.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:10.023+0000 7f3d653c5640 1 -- 172.21.15.92:0/841913461 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f3d4006aed0 msgr2=0x7f3d4006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:10.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:10.023+0000 7f3d653c5640 1 --2- 172.21.15.92:0/841913461 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f3d4006aed0 0x7f3d4006d390 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f3d50001610 tx=0x7f3d50007930 comp rx=0 tx=0).stop 2023-11-24T06:13:10.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:10.023+0000 7f3d653c5640 1 -- 172.21.15.92:0/841913461 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3d600cd240 msgr2=0x7f3d6015e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:10.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:10.023+0000 7f3d653c5640 1 --2- 172.21.15.92:0/841913461 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3d600cd240 0x7f3d6015e9a0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f3d58009d90 tx=0x7f3d58004a40 comp rx=0 tx=0).stop 2023-11-24T06:13:10.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:10.024+0000 7f3d653c5640 1 -- 172.21.15.92:0/841913461 shutdown_connections 2023-11-24T06:13:10.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:10.024+0000 7f3d653c5640 1 --2- 172.21.15.92:0/841913461 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f3d4006aed0 0x7f3d4006d390 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:10.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:10.024+0000 7f3d653c5640 1 --2- 172.21.15.92:0/841913461 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3d600cd240 0x7f3d6015e9a0 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:10.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:10.024+0000 7f3d653c5640 1 --2- 172.21.15.92:0/841913461 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3d6009a170 0x7f3d601f22f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:10.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:10.024+0000 7f3d653c5640 1 --2- 172.21.15.92:0/841913461 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3d6006c6b0 0x7f3d600c8cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:10.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:10.024+0000 7f3d653c5640 1 -- 172.21.15.92:0/841913461 >> 172.21.15.92:0/841913461 conn(0x7f3d600b2d00 msgr2=0x7f3d60160850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:13:10.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:10.024+0000 7f3d653c5640 1 -- 172.21.15.92:0/841913461 shutdown_connections 2023-11-24T06:13:10.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:10.024+0000 7f3d653c5640 1 -- 172.21.15.92:0/841913461 wait complete. 2023-11-24T06:13:10.045 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-11-24T06:13:10.116 INFO:tasks.ceph.mds.a.smithi092.stderr:2023-11-24T06:13:10.114+0000 7fd3afc08640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-24T06:13:10.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:10.249+0000 7fd6dcd7c640 1 Processor -- start 2023-11-24T06:13:10.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:10.249+0000 7fd6dcd7c640 1 -- start start 2023-11-24T06:13:10.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:10.251+0000 7fd6dcd7c640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fd6d8143ae0 0x7fd6d80cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:10.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:10.251+0000 7fd6dcd7c640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6d80cfc30 con 0x7fd6d815e3b0 2023-11-24T06:13:10.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:10.251+0000 7fd6dcd7c640 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6d80b1ee0 con 0x7fd6d80cf7d0 2023-11-24T06:13:10.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:10.251+0000 7fd6dcd7c640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6d80b2780 con 0x7fd6d8143ae0 2023-11-24T06:13:10.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:10.251+0000 7fd6d659c640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fd6d8143ae0 0x7fd6d80cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:10.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:10.251+0000 7fd6d659c640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fd6d8143ae0 0x7fd6d80cf290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52712/0 (socket says 172.21.15.92:52712) 2023-11-24T06:13:10.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:10.251+0000 7fd6d6d9d640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fd6d815e3b0 0x7fd6d80702c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:59294/0 (socket says 172.21.15.92:59294) 2023-11-24T06:13:10.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:10.251+0000 7fd6d659c640 1 -- 172.21.15.92:0/19245609 learned_addr learned my addr 172.21.15.92:0/19245609 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:13:10.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:10.251+0000 7fd6d659c640 1 -- 172.21.15.92:0/19245609 >> v1:172.21.15.118:6789/0 conn(0x7fd6d80cf7d0 legacy=0x7fd6d815dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:13:10.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:10.251+0000 7fd6d659c640 1 -- 172.21.15.92:0/19245609 >> v1:172.21.15.92:6789/0 conn(0x7fd6d815e3b0 legacy=0x7fd6d80702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:10.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:10.251+0000 7fd6d659c640 1 -- 172.21.15.92:0/19245609 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6d815e990 con 0x7fd6d8143ae0 2023-11-24T06:13:10.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:10.252+0000 7fd6d659c640 1 --2- 172.21.15.92:0/19245609 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd6d8143ae0 0x7fd6d80cf290 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fd6c8009870 tx=0x7fd6c802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b276259e0b6c91c server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:10.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:10.253+0000 7fd6d559a640 1 -- 172.21.15.92:0/19245609 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6c8030860 con 0x7fd6d8143ae0 2023-11-24T06:13:10.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:10.253+0000 7fd6d559a640 1 -- 172.21.15.92:0/19245609 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6c80309c0 con 0x7fd6d8143ae0 2023-11-24T06:13:10.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:10.253+0000 7fd6d559a640 1 -- 172.21.15.92:0/19245609 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6c8030d50 con 0x7fd6d8143ae0 2023-11-24T06:13:10.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:10.253+0000 7fd6dcd7c640 1 -- 172.21.15.92:0/19245609 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd6d8143ae0 msgr2=0x7fd6d80cf290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:10.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:10.253+0000 7fd6dcd7c640 1 --2- 172.21.15.92:0/19245609 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd6d8143ae0 0x7fd6d80cf290 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fd6c8009870 tx=0x7fd6c802b9a0 comp rx=0 tx=0).stop 2023-11-24T06:13:10.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:10.253+0000 7fd6dcd7c640 1 -- 172.21.15.92:0/19245609 shutdown_connections 2023-11-24T06:13:10.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:10.253+0000 7fd6dcd7c640 1 --2- 172.21.15.92:0/19245609 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd6d8143ae0 0x7fd6d80cf290 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:10.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:10.253+0000 7fd6dcd7c640 1 -- 172.21.15.92:0/19245609 >> 172.21.15.92:0/19245609 conn(0x7fd6d806a800 msgr2=0x7fd6d806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:13:10.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:10.253+0000 7fd6dcd7c640 1 -- 172.21.15.92:0/19245609 shutdown_connections 2023-11-24T06:13:10.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:10.254+0000 7fd6dcd7c640 1 -- 172.21.15.92:0/19245609 wait complete. 2023-11-24T06:13:10.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:10.255+0000 7fd6dcd7c640 1 Processor -- start 2023-11-24T06:13:10.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:10.255+0000 7fd6dcd7c640 1 -- start start 2023-11-24T06:13:10.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:10.256+0000 7fd6dcd7c640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd6d80cf7d0 0x7fd6d80ce260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:10.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:10.256+0000 7fd6dcd7c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd6d8143ae0 0x7fd6d80ac8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:10.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:10.256+0000 7fd6dcd7c640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd6d815e3b0 0x7fd6d81327d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:10.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:10.256+0000 7fd6dcd7c640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6d80ace10 con 0x7fd6d8143ae0 2023-11-24T06:13:10.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:10.256+0000 7fd6dcd7c640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6d80ad0b0 con 0x7fd6d80cf7d0 2023-11-24T06:13:10.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:10.256+0000 7fd6dcd7c640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6d80ad390 con 0x7fd6d815e3b0 2023-11-24T06:13:10.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:10.256+0000 7fd6d5d9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd6d8143ae0 0x7fd6d80ac8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:10.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:10.256+0000 7fd6d5d9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd6d8143ae0 0x7fd6d80ac8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52726/0 (socket says 172.21.15.92:52726) 2023-11-24T06:13:10.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:10.256+0000 7fd6d5d9b640 1 -- 172.21.15.92:0/1790312694 learned_addr learned my addr 172.21.15.92:0/1790312694 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:13:10.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:10.257+0000 7fd6d5d9b640 1 -- 172.21.15.92:0/1790312694 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd6d815e3b0 msgr2=0x7fd6d81327d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:10.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:10.257+0000 7fd6d5d9b640 1 --2- 172.21.15.92:0/1790312694 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd6d815e3b0 0x7fd6d81327d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:10.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:10.257+0000 7fd6d5d9b640 1 -- 172.21.15.92:0/1790312694 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd6d80cf7d0 msgr2=0x7fd6d80ce260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:10.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:10.257+0000 7fd6d5d9b640 1 --2- 172.21.15.92:0/1790312694 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd6d80cf7d0 0x7fd6d80ce260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:10.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:10.257+0000 7fd6d5d9b640 1 -- 172.21.15.92:0/1790312694 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6c80094d0 con 0x7fd6d8143ae0 2023-11-24T06:13:10.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:10.257+0000 7fd6d5d9b640 1 --2- 172.21.15.92:0/1790312694 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd6d8143ae0 0x7fd6d80ac8d0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fd6cc00c410 tx=0x7fd6cc00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:10.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:10.257+0000 7fd6c6ffd640 1 -- 172.21.15.92:0/1790312694 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6cc0035f0 con 0x7fd6d8143ae0 2023-11-24T06:13:10.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:10.257+0000 7fd6c6ffd640 1 -- 172.21.15.92:0/1790312694 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6cc003750 con 0x7fd6d8143ae0 2023-11-24T06:13:10.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:10.257+0000 7fd6c6ffd640 1 -- 172.21.15.92:0/1790312694 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6cc003b40 con 0x7fd6d8143ae0 2023-11-24T06:13:10.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:10.257+0000 7fd6dcd7c640 1 -- 172.21.15.92:0/1790312694 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6d80ad670 con 0x7fd6d8143ae0 2023-11-24T06:13:10.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:10.257+0000 7fd6dcd7c640 1 -- 172.21.15.92:0/1790312694 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6d80adc30 con 0x7fd6d8143ae0 2023-11-24T06:13:10.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:10.259+0000 7fd6dcd7c640 1 -- 172.21.15.92:0/1790312694 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6d80675d0 con 0x7fd6d8143ae0 2023-11-24T06:13:10.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:10.259+0000 7fd6c6ffd640 1 -- 172.21.15.92:0/1790312694 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7fd6cc003ca0 con 0x7fd6d8143ae0 2023-11-24T06:13:10.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:10.260+0000 7fd6c6ffd640 1 --2- 172.21.15.92:0/1790312694 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fd6b806aed0 0x7fd6b806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:10.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:10.260+0000 7fd6c6ffd640 1 -- 172.21.15.92:0/1790312694 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd6cc0888c0 con 0x7fd6d8143ae0 2023-11-24T06:13:10.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:10.261+0000 7fd6d659c640 1 --2- 172.21.15.92:0/1790312694 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fd6b806aed0 0x7fd6b806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:10.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:10.261+0000 7fd6d659c640 1 --2- 172.21.15.92:0/1790312694 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fd6b806aed0 0x7fd6b806d390 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fd6c8036000 tx=0x7fd6c803b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:10.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:10.265+0000 7fd6c6ffd640 1 -- 172.21.15.92:0/1790312694 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd6cc053910 con 0x7fd6d8143ae0 2023-11-24T06:13:10.493 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:10.491+0000 7fd6dcd7c640 1 -- 172.21.15.92:0/1790312694 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fd6d80addf0 con 0x7fd6d8143ae0 2023-11-24T06:13:11.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:11.172+0000 7fd6c6ffd640 1 -- 172.21.15.92:0/1790312694 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v39) v1 ==== 118+0+0 (secure 0 0 0) 0x7fd6cc056f30 con 0x7fd6d8143ae0 2023-11-24T06:13:11.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:11.175+0000 7fd6dcd7c640 1 -- 172.21.15.92:0/1790312694 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fd6b806aed0 msgr2=0x7fd6b806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:11.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:11.175+0000 7fd6dcd7c640 1 --2- 172.21.15.92:0/1790312694 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fd6b806aed0 0x7fd6b806d390 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fd6c8036000 tx=0x7fd6c803b040 comp rx=0 tx=0).stop 2023-11-24T06:13:11.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:11.175+0000 7fd6dcd7c640 1 -- 172.21.15.92:0/1790312694 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd6d8143ae0 msgr2=0x7fd6d80ac8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:11.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:11.175+0000 7fd6dcd7c640 1 --2- 172.21.15.92:0/1790312694 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd6d8143ae0 0x7fd6d80ac8d0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fd6cc00c410 tx=0x7fd6cc00c8d0 comp rx=0 tx=0).stop 2023-11-24T06:13:11.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:11.175+0000 7fd6dcd7c640 1 -- 172.21.15.92:0/1790312694 shutdown_connections 2023-11-24T06:13:11.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:11.175+0000 7fd6dcd7c640 1 --2- 172.21.15.92:0/1790312694 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fd6b806aed0 0x7fd6b806d390 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:11.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:11.175+0000 7fd6dcd7c640 1 --2- 172.21.15.92:0/1790312694 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd6d815e3b0 0x7fd6d81327d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:11.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:11.175+0000 7fd6dcd7c640 1 --2- 172.21.15.92:0/1790312694 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd6d8143ae0 0x7fd6d80ac8d0 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:11.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:11.175+0000 7fd6dcd7c640 1 --2- 172.21.15.92:0/1790312694 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd6d80cf7d0 0x7fd6d80ce260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:11.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:11.175+0000 7fd6dcd7c640 1 -- 172.21.15.92:0/1790312694 >> 172.21.15.92:0/1790312694 conn(0x7fd6d806a800 msgr2=0x7fd6d815ad80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:13:11.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:11.176+0000 7fd6dcd7c640 1 -- 172.21.15.92:0/1790312694 shutdown_connections 2023-11-24T06:13:11.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:11.176+0000 7fd6dcd7c640 1 -- 172.21.15.92:0/1790312694 wait complete. 2023-11-24T06:13:11.198 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-11-24T06:13:11.199 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-11-24T06:13:11.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:11.413+0000 7f089b0cf640 1 Processor -- start 2023-11-24T06:13:11.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:11.413+0000 7f089b0cf640 1 -- start start 2023-11-24T06:13:11.417 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:11.415+0000 7f089b0cf640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f089413df80 0x7f089415bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:11.417 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:11.415+0000 7f089b0cf640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f089413e360 0x7f08940cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:11.417 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:11.415+0000 7f089b0cf640 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f089406c470 con 0x7f089415f9c0 2023-11-24T06:13:11.417 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:11.415+0000 7f089b0cf640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f089406c750 con 0x7f089413e360 2023-11-24T06:13:11.417 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:11.415+0000 7f089b0cf640 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f089406ca30 con 0x7f089413df80 2023-11-24T06:13:11.418 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:11.415+0000 7f0893fff640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f089413e360 0x7f08940cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:11.418 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:11.415+0000 7f0893fff640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f089413e360 0x7f08940cf290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52740/0 (socket says 172.21.15.92:52740) 2023-11-24T06:13:11.418 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:11.415+0000 7f0893fff640 1 -- 172.21.15.92:0/3534352158 learned_addr learned my addr 172.21.15.92:0/3534352158 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:13:11.418 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:11.416+0000 7f0893fff640 1 -- 172.21.15.92:0/3534352158 >> v2:172.21.15.118:3300/0 conn(0x7f089413df80 msgr2=0x7f089415bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:13:11.418 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:11.416+0000 7f0893fff640 1 --2- 172.21.15.92:0/3534352158 >> v2:172.21.15.118:3300/0 conn(0x7f089413df80 0x7f089415bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:11.418 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:11.416+0000 7f0893fff640 1 -- 172.21.15.92:0/3534352158 >> v1:172.21.15.118:6789/0 conn(0x7f089415f9c0 legacy=0x7f08941408d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:13:11.418 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:11.416+0000 7f0893fff640 1 -- 172.21.15.92:0/3534352158 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f089406cd10 con 0x7f089413e360 2023-11-24T06:13:11.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:11.417+0000 7f0893fff640 1 --2- 172.21.15.92:0/3534352158 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f089413e360 0x7f08940cf290 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f0894072770 tx=0x7f088002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13211556f57c056c server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:11.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:11.417+0000 7f08937fe640 1 -- 172.21.15.92:0/3534352158 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0880031870 con 0x7f089413e360 2023-11-24T06:13:11.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:11.417+0000 7f08937fe640 1 -- 172.21.15.92:0/3534352158 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08800319d0 con 0x7f089413e360 2023-11-24T06:13:11.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:11.417+0000 7f08937fe640 1 -- 172.21.15.92:0/3534352158 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0880031d60 con 0x7f089413e360 2023-11-24T06:13:11.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:11.418+0000 7f089b0cf640 1 -- 172.21.15.92:0/3534352158 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f089413e360 msgr2=0x7f08940cf290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:11.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:11.418+0000 7f089b0cf640 1 --2- 172.21.15.92:0/3534352158 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f089413e360 0x7f08940cf290 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f0894072770 tx=0x7f088002b910 comp rx=0 tx=0).stop 2023-11-24T06:13:11.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:11.418+0000 7f089b0cf640 1 -- 172.21.15.92:0/3534352158 shutdown_connections 2023-11-24T06:13:11.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:11.418+0000 7f089b0cf640 1 --2- 172.21.15.92:0/3534352158 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f089413e360 0x7f08940cf290 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:11.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:11.418+0000 7f089b0cf640 1 --2- 172.21.15.92:0/3534352158 >> v2:172.21.15.118:3300/0 conn(0x7f089413df80 0x7f089415bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:11.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:11.418+0000 7f089b0cf640 1 -- 172.21.15.92:0/3534352158 >> 172.21.15.92:0/3534352158 conn(0x7f089406a800 msgr2=0x7f089406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:13:11.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:11.418+0000 7f089b0cf640 1 -- 172.21.15.92:0/3534352158 shutdown_connections 2023-11-24T06:13:11.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:11.418+0000 7f089b0cf640 1 -- 172.21.15.92:0/3534352158 wait complete. 2023-11-24T06:13:11.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:11.420+0000 7f089b0cf640 1 Processor -- start 2023-11-24T06:13:11.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:11.420+0000 7f089b0cf640 1 -- start start 2023-11-24T06:13:11.423 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:11.421+0000 7f089b0cf640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f089413df80 0x7f089413ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:11.423 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:11.421+0000 7f089b0cf640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f089413e360 0x7f089413cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:11.423 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:11.421+0000 7f089b0cf640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f089415f9c0 0x7f089413c230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:11.423 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:11.421+0000 7f089b0cf640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f089413d450 con 0x7f089413e360 2023-11-24T06:13:11.423 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:11.421+0000 7f089b0cf640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f089413b190 con 0x7f089415f9c0 2023-11-24T06:13:11.423 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:11.421+0000 7f089b0cf640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f089413b470 con 0x7f089413df80 2023-11-24T06:13:11.424 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:11.421+0000 7f0893fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f089413e360 0x7f089413cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:11.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:11.422+0000 7f0893fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f089413e360 0x7f089413cf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52744/0 (socket says 172.21.15.92:52744) 2023-11-24T06:13:11.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:11.422+0000 7f0893fff640 1 -- 172.21.15.92:0/1685006821 learned_addr learned my addr 172.21.15.92:0/1685006821 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:13:11.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:11.422+0000 7f0893fff640 1 -- 172.21.15.92:0/1685006821 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f089413df80 msgr2=0x7f089413ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:11.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:11.422+0000 7f0893fff640 1 --2- 172.21.15.92:0/1685006821 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f089413df80 0x7f089413ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:11.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:11.422+0000 7f0893fff640 1 -- 172.21.15.92:0/1685006821 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f089415f9c0 msgr2=0x7f089413c230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:11.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:11.422+0000 7f0893fff640 1 --2- 172.21.15.92:0/1685006821 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f089415f9c0 0x7f089413c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:11.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:11.422+0000 7f0893fff640 1 -- 172.21.15.92:0/1685006821 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08800094d0 con 0x7f089413e360 2023-11-24T06:13:11.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:11.422+0000 7f0893fff640 1 --2- 172.21.15.92:0/1685006821 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f089413e360 0x7f089413cf10 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f0880000c00 tx=0x7f08800046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:11.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:11.422+0000 7f08917fa640 1 -- 172.21.15.92:0/1685006821 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f088003ad10 con 0x7f089413e360 2023-11-24T06:13:11.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:11.422+0000 7f08917fa640 1 -- 172.21.15.92:0/1685006821 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f088003ae70 con 0x7f089413e360 2023-11-24T06:13:11.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:11.422+0000 7f08917fa640 1 -- 172.21.15.92:0/1685006821 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0880044610 con 0x7f089413e360 2023-11-24T06:13:11.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:11.422+0000 7f089b0cf640 1 -- 172.21.15.92:0/1685006821 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f089413b750 con 0x7f089413e360 2023-11-24T06:13:11.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:11.423+0000 7f089b0cf640 1 -- 172.21.15.92:0/1685006821 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f089413bd10 con 0x7f089413e360 2023-11-24T06:13:11.426 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:11.424+0000 7f089b0cf640 1 -- 172.21.15.92:0/1685006821 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f089413bf00 con 0x7f089413e360 2023-11-24T06:13:11.427 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:11.425+0000 7f08917fa640 1 -- 172.21.15.92:0/1685006821 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f088004d450 con 0x7f089413e360 2023-11-24T06:13:11.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:11.426+0000 7f08917fa640 1 --2- 172.21.15.92:0/1685006821 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f087806aed0 0x7f087806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:11.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:11.426+0000 7f08917fa640 1 -- 172.21.15.92:0/1685006821 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f088002d040 con 0x7f089413e360 2023-11-24T06:13:11.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:11.426+0000 7f0898e6b640 1 --2- 172.21.15.92:0/1685006821 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f087806aed0 0x7f087806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:11.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:11.427+0000 7f0898e6b640 1 --2- 172.21.15.92:0/1685006821 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f087806aed0 0x7f087806d390 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f089415aed0 tx=0x7f0888006bf0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:11.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:11.430+0000 7f08917fa640 1 -- 172.21.15.92:0/1685006821 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f08800b0070 con 0x7f089413e360 2023-11-24T06:13:11.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:11.657+0000 7f089b0cf640 1 -- 172.21.15.92:0/1685006821 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f08940675d0 con 0x7f089413e360 2023-11-24T06:13:12.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:12.184+0000 7f08917fa640 1 -- 172.21.15.92:0/1685006821 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v40) v1 ==== 122+0+0 (secure 0 0 0) 0x7f0880076e10 con 0x7f089413e360 2023-11-24T06:13:12.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:12.187+0000 7f089b0cf640 1 -- 172.21.15.92:0/1685006821 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f087806aed0 msgr2=0x7f087806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:12.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:12.187+0000 7f089b0cf640 1 --2- 172.21.15.92:0/1685006821 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f087806aed0 0x7f087806d390 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f089415aed0 tx=0x7f0888006bf0 comp rx=0 tx=0).stop 2023-11-24T06:13:12.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:12.187+0000 7f089b0cf640 1 -- 172.21.15.92:0/1685006821 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f089413e360 msgr2=0x7f089413cf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:12.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:12.187+0000 7f089b0cf640 1 --2- 172.21.15.92:0/1685006821 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f089413e360 0x7f089413cf10 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f0880000c00 tx=0x7f08800046c0 comp rx=0 tx=0).stop 2023-11-24T06:13:12.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:12.187+0000 7f089b0cf640 1 -- 172.21.15.92:0/1685006821 shutdown_connections 2023-11-24T06:13:12.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:12.187+0000 7f089b0cf640 1 --2- 172.21.15.92:0/1685006821 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f087806aed0 0x7f087806d390 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:12.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:12.187+0000 7f089b0cf640 1 --2- 172.21.15.92:0/1685006821 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f089415f9c0 0x7f089413c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:12.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:12.187+0000 7f089b0cf640 1 --2- 172.21.15.92:0/1685006821 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f089413e360 0x7f089413cf10 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:12.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:12.187+0000 7f089b0cf640 1 --2- 172.21.15.92:0/1685006821 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f089413df80 0x7f089413ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:12.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:12.187+0000 7f089b0cf640 1 -- 172.21.15.92:0/1685006821 >> 172.21.15.92:0/1685006821 conn(0x7f089406a800 msgr2=0x7f08940afc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:13:12.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:12.187+0000 7f089b0cf640 1 -- 172.21.15.92:0/1685006821 shutdown_connections 2023-11-24T06:13:12.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:12.187+0000 7f089b0cf640 1 -- 172.21.15.92:0/1685006821 wait complete. 2023-11-24T06:13:12.210 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-11-24T06:13:12.210 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T06:13:12.417 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:12.415+0000 7f928de07640 1 Processor -- start 2023-11-24T06:13:12.417 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:12.415+0000 7f928de07640 1 -- start start 2023-11-24T06:13:12.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:12.417+0000 7f928de07640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f92880ae5a0 0x7f928815e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:12.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:12.417+0000 7f928de07640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9288070c40 con 0x7f92880b3ae0 2023-11-24T06:13:12.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:12.417+0000 7f928de07640 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f928806c470 con 0x7f92880cf290 2023-11-24T06:13:12.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:12.417+0000 7f928de07640 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9288071450 con 0x7f92880ae5a0 2023-11-24T06:13:12.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:12.417+0000 7f9286ffd640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f92880b3ae0 0x7f928806cbe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:59304/0 (socket says 172.21.15.92:59304) 2023-11-24T06:13:12.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:12.417+0000 7f9286ffd640 1 -- 172.21.15.92:0/1475866889 learned_addr learned my addr 172.21.15.92:0/1475866889 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-24T06:13:12.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:12.417+0000 7f92877fe640 1 --2- 172.21.15.92:0/1475866889 >> v2:172.21.15.118:3300/0 conn(0x7f92880ae5a0 0x7f928815e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:12.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:12.417+0000 7f92867fc640 1 -- 172.21.15.92:0/1475866889 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1236668331 0 0) 0x7f9288070c40 con 0x7f92880b3ae0 2023-11-24T06:13:12.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:12.417+0000 7f92867fc640 1 -- 172.21.15.92:0/1475866889 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9270003520 con 0x7f92880b3ae0 2023-11-24T06:13:12.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:12.418+0000 7f92867fc640 1 -- 172.21.15.92:0/1475866889 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2499523371 0 0) 0x7f9270003520 con 0x7f92880b3ae0 2023-11-24T06:13:12.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:12.418+0000 7f92867fc640 1 -- 172.21.15.92:0/1475866889 >> v2:172.21.15.118:3300/0 conn(0x7f92880ae5a0 msgr2=0x7f928815e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:13:12.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:12.418+0000 7f92867fc640 1 --2- 172.21.15.92:0/1475866889 >> v2:172.21.15.118:3300/0 conn(0x7f92880ae5a0 0x7f928815e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:12.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:12.418+0000 7f92867fc640 1 -- 172.21.15.92:0/1475866889 >> v1:172.21.15.118:6789/0 conn(0x7f92880cf290 legacy=0x7f92880b1fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:12.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:12.418+0000 7f92867fc640 1 -- 172.21.15.92:0/1475866889 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f928806c750 con 0x7f92880b3ae0 2023-11-24T06:13:12.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:12.418+0000 7f92867fc640 1 -- 172.21.15.92:0/1475866889 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2607661350 0 0) 0x7f927c001670 con 0x7f92880b3ae0 2023-11-24T06:13:12.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:12.418+0000 7f92867fc640 1 -- 172.21.15.92:0/1475866889 >> v1:172.21.15.92:6789/0 conn(0x7f92880b3ae0 legacy=0x7f928806cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:12.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:12.418+0000 7f92867fc640 1 --2- 172.21.15.92:0/1475866889 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9270003970 0x7f9270003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:12.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:12.418+0000 7f92867fc640 1 --2- 172.21.15.92:0/1475866889 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f92700047c0 0x7f9270004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:12.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:12.418+0000 7f92867fc640 1 --2- 172.21.15.92:0/1475866889 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f9270005180 0x7f927000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:12.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:12.418+0000 7f92867fc640 1 -- 172.21.15.92:0/1475866889 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9288071450 con 0x7f9270003970 2023-11-24T06:13:12.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:12.418+0000 7f92867fc640 1 -- 172.21.15.92:0/1475866889 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f927000c0b0 con 0x7f92700047c0 2023-11-24T06:13:12.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:12.418+0000 7f92867fc640 1 -- 172.21.15.92:0/1475866889 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f927000c270 con 0x7f9270005180 2023-11-24T06:13:12.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:12.418+0000 7f92877fe640 1 --2- 172.21.15.92:0/1475866889 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9270003970 0x7f9270003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:12.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:12.419+0000 7f9286ffd640 1 --2- 172.21.15.92:0/1475866889 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f92700047c0 0x7f9270004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:12.423 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:12.419+0000 7f9287fff640 1 --2- 172.21.15.92:0/1475866889 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f9270005180 0x7f927000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:12.423 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:12.419+0000 7f92877fe640 1 -- 172.21.15.92:0/1475866889 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f9270005180 msgr2=0x7f927000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:13:12.423 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:12.419+0000 7f92877fe640 1 --2- 172.21.15.92:0/1475866889 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f9270005180 0x7f927000b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:12.423 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:12.419+0000 7f92877fe640 1 -- 172.21.15.92:0/1475866889 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f92700047c0 msgr2=0x7f9270004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:13:12.423 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:12.419+0000 7f92877fe640 1 --2- 172.21.15.92:0/1475866889 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f92700047c0 0x7f9270004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:12.423 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:12.419+0000 7f92877fe640 1 -- 172.21.15.92:0/1475866889 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9270003520 con 0x7f9270003970 2023-11-24T06:13:12.423 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:12.419+0000 7f9287fff640 1 --2- 172.21.15.92:0/1475866889 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f9270005180 0x7f927000b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:13:12.423 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:12.419+0000 7f9286ffd640 1 --2- 172.21.15.92:0/1475866889 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f92700047c0 0x7f9270004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:13:12.423 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:12.419+0000 7f9287fff640 1 -- 172.21.15.92:0/1475866889 reap_dead start 2023-11-24T06:13:12.424 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:12.420+0000 7f92877fe640 1 --2- 172.21.15.92:0/1475866889 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9270003970 0x7f9270003d90 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f927800ba30 tx=0x7f92700057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5931a4782f84966c server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:12.424 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:12.420+0000 7f92867fc640 1 -- 172.21.15.92:0/1475866889 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9278032630 con 0x7f9270003970 2023-11-24T06:13:12.424 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:12.420+0000 7f92867fc640 1 -- 172.21.15.92:0/1475866889 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9278032790 con 0x7f9270003970 2023-11-24T06:13:12.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:12.420+0000 7f92867fc640 1 -- 172.21.15.92:0/1475866889 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9278032b20 con 0x7f9270003970 2023-11-24T06:13:12.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:12.421+0000 7f928de07640 1 -- 172.21.15.92:0/1475866889 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9270003970 msgr2=0x7f9270003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:12.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:12.421+0000 7f928de07640 1 --2- 172.21.15.92:0/1475866889 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9270003970 0x7f9270003d90 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f927800ba30 tx=0x7f92700057f0 comp rx=0 tx=0).stop 2023-11-24T06:13:12.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:12.421+0000 7f928de07640 1 -- 172.21.15.92:0/1475866889 shutdown_connections 2023-11-24T06:13:12.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:12.421+0000 7f928de07640 1 --2- 172.21.15.92:0/1475866889 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9270003970 0x7f9270003d90 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:12.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:12.421+0000 7f928de07640 1 -- 172.21.15.92:0/1475866889 >> 172.21.15.92:0/1475866889 conn(0x7f928806a800 msgr2=0x7f928806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:13:12.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:12.421+0000 7f928de07640 1 -- 172.21.15.92:0/1475866889 shutdown_connections 2023-11-24T06:13:12.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:12.421+0000 7f928de07640 1 -- 172.21.15.92:0/1475866889 wait complete. 2023-11-24T06:13:12.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:12.422+0000 7f928de07640 1 Processor -- start 2023-11-24T06:13:12.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:12.423+0000 7f928de07640 1 -- start start 2023-11-24T06:13:12.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:12.423+0000 7f928de07640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9270003970 0x7f9288151cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:12.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:12.423+0000 7f928de07640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9288150560 0x7f9288159c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:12.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:12.423+0000 7f928de07640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f92880aef30 0x7f92880af3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:12.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:12.423+0000 7f928de07640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92880af8f0 con 0x7f9270003970 2023-11-24T06:13:12.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:12.423+0000 7f928de07640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f928815a1b0 con 0x7f9288150560 2023-11-24T06:13:12.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:12.423+0000 7f928de07640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9288153ae0 con 0x7f92880aef30 2023-11-24T06:13:12.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:12.424+0000 7f92877fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9270003970 0x7f9288151cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:12.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:12.424+0000 7f92877fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9270003970 0x7f9288151cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52750/0 (socket says 172.21.15.92:52750) 2023-11-24T06:13:12.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:12.424+0000 7f92877fe640 1 -- 172.21.15.92:0/3583324934 learned_addr learned my addr 172.21.15.92:0/3583324934 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:13:12.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:12.424+0000 7f92877fe640 1 -- 172.21.15.92:0/3583324934 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f92880aef30 msgr2=0x7f92880af3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:12.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:12.424+0000 7f92877fe640 1 --2- 172.21.15.92:0/3583324934 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f92880aef30 0x7f92880af3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:12.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:12.424+0000 7f92877fe640 1 -- 172.21.15.92:0/3583324934 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9288150560 msgr2=0x7f9288159c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:12.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:12.424+0000 7f92877fe640 1 --2- 172.21.15.92:0/3583324934 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9288150560 0x7f9288159c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:12.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:12.424+0000 7f92877fe640 1 -- 172.21.15.92:0/3583324934 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9288154af0 con 0x7f9270003970 2023-11-24T06:13:12.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:12.424+0000 7f92877fe640 1 --2- 172.21.15.92:0/3583324934 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9270003970 0x7f9288151cd0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f927802beb0 tx=0x7f9278004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:12.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:12.425+0000 7f926ffff640 1 -- 172.21.15.92:0/3583324934 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f927803ba70 con 0x7f9270003970 2023-11-24T06:13:12.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:12.425+0000 7f926ffff640 1 -- 172.21.15.92:0/3583324934 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f927803bbd0 con 0x7f9270003970 2023-11-24T06:13:12.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:12.425+0000 7f926ffff640 1 -- 172.21.15.92:0/3583324934 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9278030400 con 0x7f9270003970 2023-11-24T06:13:12.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:12.425+0000 7f928de07640 1 -- 172.21.15.92:0/3583324934 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92881509e0 con 0x7f9270003970 2023-11-24T06:13:12.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:12.425+0000 7f928de07640 1 -- 172.21.15.92:0/3583324934 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9288154010 con 0x7f9270003970 2023-11-24T06:13:12.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:12.426+0000 7f928de07640 1 -- 172.21.15.92:0/3583324934 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92880675d0 con 0x7f9270003970 2023-11-24T06:13:12.430 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:12.427+0000 7f926ffff640 1 -- 172.21.15.92:0/3583324934 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f9278030560 con 0x7f9270003970 2023-11-24T06:13:12.430 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:12.428+0000 7f926ffff640 1 --2- 172.21.15.92:0/3583324934 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f927406aed0 0x7f927406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:12.430 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:12.428+0000 7f926ffff640 1 -- 172.21.15.92:0/3583324934 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f92780abea0 con 0x7f9270003970 2023-11-24T06:13:12.430 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:12.428+0000 7f9286ffd640 1 --2- 172.21.15.92:0/3583324934 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f927406aed0 0x7f927406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:12.431 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:12.429+0000 7f9286ffd640 1 --2- 172.21.15.92:0/3583324934 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f927406aed0 0x7f927406d390 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f9288150f20 tx=0x7f927c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:12.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:12.432+0000 7f926ffff640 1 -- 172.21.15.92:0/3583324934 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9278076de0 con 0x7f9270003970 2023-11-24T06:13:12.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:12.658+0000 7f928de07640 1 -- 172.21.15.92:0/3583324934 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9288071700 con 0x7f9270003970 2023-11-24T06:13:12.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:12.659+0000 7f926ffff640 1 -- 172.21.15.92:0/3583324934 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) v1 ==== 94+0+4921 (secure 0 0 0) 0x7f9278036020 con 0x7f9270003970 2023-11-24T06:13:12.661 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-24T06:13:12.662 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":40,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5143,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6833/2094733752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":2094733752},{"type":"v1","addr":"172.21.15.118:6833","nonce":2094733752}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":36},{"gid":5146,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6835/2873546779","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":2873546779},{"type":"v1","addr":"172.21.15.118:6835","nonce":2873546779}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":36},{"gid":6141,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6836/3335501795","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":3335501795},{"type":"v1","addr":"172.21.15.92:6836","nonce":3335501795}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":35}],"filesystems":[{"mdsmap":{"epoch":40,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T06:13:08.625588+0000","modified":"2023-11-24T06:13:11.658333+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6144},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6144":{"gid":6144,"name":"a","rank":0,"incarnation":37,"state":"up:active","state_seq":3,"addr":"172.21.15.92:6837/3913457833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":3913457833},{"type":"v1","addr":"172.21.15.92:6837","nonce":3913457833}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-11-24T06:13:12.662 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 40 2023-11-24T06:13:12.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:12.662+0000 7f928de07640 1 -- 172.21.15.92:0/3583324934 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f927406aed0 msgr2=0x7f927406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:12.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:12.662+0000 7f928de07640 1 --2- 172.21.15.92:0/3583324934 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f927406aed0 0x7f927406d390 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f9288150f20 tx=0x7f927c009040 comp rx=0 tx=0).stop 2023-11-24T06:13:12.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:12.662+0000 7f928de07640 1 -- 172.21.15.92:0/3583324934 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9270003970 msgr2=0x7f9288151cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:12.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:12.662+0000 7f928de07640 1 --2- 172.21.15.92:0/3583324934 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9270003970 0x7f9288151cd0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f927802beb0 tx=0x7f9278004730 comp rx=0 tx=0).stop 2023-11-24T06:13:12.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:12.662+0000 7f928de07640 1 -- 172.21.15.92:0/3583324934 shutdown_connections 2023-11-24T06:13:12.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:12.662+0000 7f928de07640 1 --2- 172.21.15.92:0/3583324934 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f927406aed0 0x7f927406d390 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:12.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:12.662+0000 7f928de07640 1 --2- 172.21.15.92:0/3583324934 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f92880aef30 0x7f92880af3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:12.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:12.662+0000 7f928de07640 1 --2- 172.21.15.92:0/3583324934 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9288150560 0x7f9288159c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:12.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:12.662+0000 7f928de07640 1 --2- 172.21.15.92:0/3583324934 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9270003970 0x7f9288151cd0 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:12.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:12.662+0000 7f928de07640 1 -- 172.21.15.92:0/3583324934 >> 172.21.15.92:0/3583324934 conn(0x7f928806a800 msgr2=0x7f928806cae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:13:12.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:12.662+0000 7f928de07640 1 -- 172.21.15.92:0/3583324934 shutdown_connections 2023-11-24T06:13:12.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:12.662+0000 7f928de07640 1 -- 172.21.15.92:0/3583324934 wait complete. 2023-11-24T06:13:12.683 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:13:12.891 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:12.888+0000 7fe4d167e640 1 Processor -- start 2023-11-24T06:13:12.891 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:12.888+0000 7fe4d167e640 1 -- start start 2023-11-24T06:13:12.892 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:12.890+0000 7fe4d167e640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe4cc15d3c0 0x7fe4cc0d1f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:12.892 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:12.890+0000 7fe4d167e640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fe4cc0cfa40 0x7fe4cc0cfe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:12.892 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:12.890+0000 7fe4d167e640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4cc0b2c70 con 0x7fe4cc0cde90 2023-11-24T06:13:12.892 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:12.890+0000 7fe4d167e640 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4cc0d03c0 con 0x7fe4cc0cfa40 2023-11-24T06:13:12.892 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:12.890+0000 7fe4d167e640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4cc0cf6d0 con 0x7fe4cc15d3c0 2023-11-24T06:13:12.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:12.890+0000 7fe4d0e7d640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fe4cc0cde90 0x7fe4cc0ce300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:59318/0 (socket says 172.21.15.92:59318) 2023-11-24T06:13:12.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:12.890+0000 7fe4d0e7d640 1 -- 172.21.15.92:0/2908781650 learned_addr learned my addr 172.21.15.92:0/2908781650 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-24T06:13:12.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:12.891+0000 7fe4cbfff640 1 --2- 172.21.15.92:0/2908781650 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe4cc15d3c0 0x7fe4cc0d1f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:12.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:12.891+0000 7fe4c3fff640 1 --2- 172.21.15.92:0/2908781650 >> v2:172.21.15.118:3300/0 conn(0x7fe4cc0cfa40 0x7fe4cc0cfe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:12.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:12.891+0000 7fe4cb7fe640 1 -- 172.21.15.92:0/2908781650 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2866055632 0 0) 0x7fe4cc0b2c70 con 0x7fe4cc0cde90 2023-11-24T06:13:12.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:12.891+0000 7fe4cb7fe640 1 -- 172.21.15.92:0/2908781650 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe4b8003520 con 0x7fe4cc0cde90 2023-11-24T06:13:12.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:12.891+0000 7fe4cb7fe640 1 -- 172.21.15.92:0/2908781650 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2550696155 0 0) 0x7fe4b8003520 con 0x7fe4cc0cde90 2023-11-24T06:13:12.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:12.891+0000 7fe4cb7fe640 1 -- 172.21.15.92:0/2908781650 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe4cc15d3c0 msgr2=0x7fe4cc0d1f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:13:12.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:12.891+0000 7fe4cb7fe640 1 --2- 172.21.15.92:0/2908781650 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe4cc15d3c0 0x7fe4cc0d1f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:12.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:12.891+0000 7fe4cb7fe640 1 -- 172.21.15.92:0/2908781650 >> v2:172.21.15.118:3300/0 conn(0x7fe4cc0cfa40 msgr2=0x7fe4cc0cfe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:13:12.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:12.891+0000 7fe4cb7fe640 1 --2- 172.21.15.92:0/2908781650 >> v2:172.21.15.118:3300/0 conn(0x7fe4cc0cfa40 0x7fe4cc0cfe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:12.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:12.891+0000 7fe4cb7fe640 1 -- 172.21.15.92:0/2908781650 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4cc0cf3f0 con 0x7fe4cc0cde90 2023-11-24T06:13:12.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:12.891+0000 7fe4cb7fe640 1 -- 172.21.15.92:0/2908781650 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2607661350 0 0) 0x7fe4c40019f0 con 0x7fe4cc0cde90 2023-11-24T06:13:12.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:12.891+0000 7fe4c3fff640 1 --2- 172.21.15.92:0/2908781650 >> v2:172.21.15.118:3300/0 conn(0x7fe4cc0cfa40 0x7fe4cc0cfe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:13:12.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:12.891+0000 7fe4cb7fe640 1 -- 172.21.15.92:0/2908781650 >> v1:172.21.15.92:6789/0 conn(0x7fe4cc0cde90 legacy=0x7fe4cc0ce300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:12.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:12.891+0000 7fe4cb7fe640 1 --2- 172.21.15.92:0/2908781650 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe4b8003970 0x7fe4b8003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:12.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:12.891+0000 7fe4cb7fe640 1 --2- 172.21.15.92:0/2908781650 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe4b80047c0 0x7fe4b8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:12.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:12.891+0000 7fe4cb7fe640 1 --2- 172.21.15.92:0/2908781650 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe4cc15d3c0 0x7fe4b8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:12.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:12.891+0000 7fe4cb7fe640 1 -- 172.21.15.92:0/2908781650 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4cc0d03c0 con 0x7fe4b8003970 2023-11-24T06:13:12.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:12.891+0000 7fe4cb7fe640 1 -- 172.21.15.92:0/2908781650 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4cc0cf6d0 con 0x7fe4b80047c0 2023-11-24T06:13:12.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:12.891+0000 7fe4cb7fe640 1 -- 172.21.15.92:0/2908781650 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4b800b9d0 con 0x7fe4cc15d3c0 2023-11-24T06:13:12.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:12.892+0000 7fe4cbfff640 1 --2- 172.21.15.92:0/2908781650 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe4b8003970 0x7fe4b8003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:12.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:12.892+0000 7fe4cbfff640 1 -- 172.21.15.92:0/2908781650 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe4cc15d3c0 msgr2=0x7fe4b8005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:13:12.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:12.892+0000 7fe4cbfff640 1 --2- 172.21.15.92:0/2908781650 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe4cc15d3c0 0x7fe4b8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:12.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:12.892+0000 7fe4cbfff640 1 -- 172.21.15.92:0/2908781650 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe4b80047c0 msgr2=0x7fe4b8004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:13:12.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:12.892+0000 7fe4cbfff640 1 --2- 172.21.15.92:0/2908781650 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe4b80047c0 0x7fe4b8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:12.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:12.892+0000 7fe4cbfff640 1 -- 172.21.15.92:0/2908781650 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4b8003520 con 0x7fe4b8003970 2023-11-24T06:13:12.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:12.893+0000 7fe4cbfff640 1 --2- 172.21.15.92:0/2908781650 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe4b8003970 0x7fe4b8003d90 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fe4bc009ac0 tx=0x7fe4b80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e484360235de5f1d server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:12.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:12.893+0000 7fe4cb7fe640 1 -- 172.21.15.92:0/2908781650 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4bc02c710 con 0x7fe4b8003970 2023-11-24T06:13:12.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:12.893+0000 7fe4cb7fe640 1 -- 172.21.15.92:0/2908781650 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4bc02c870 con 0x7fe4b8003970 2023-11-24T06:13:12.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:12.893+0000 7fe4cb7fe640 1 -- 172.21.15.92:0/2908781650 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4bc02cc00 con 0x7fe4b8003970 2023-11-24T06:13:12.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:12.894+0000 7fe4d167e640 1 -- 172.21.15.92:0/2908781650 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe4b8003970 msgr2=0x7fe4b8003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:12.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:12.894+0000 7fe4d167e640 1 --2- 172.21.15.92:0/2908781650 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe4b8003970 0x7fe4b8003d90 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fe4bc009ac0 tx=0x7fe4b80057f0 comp rx=0 tx=0).stop 2023-11-24T06:13:12.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:12.894+0000 7fe4d0e7d640 1 -- 172.21.15.92:0/2908781650 reap_dead start 2023-11-24T06:13:12.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:12.894+0000 7fe4d167e640 1 -- 172.21.15.92:0/2908781650 shutdown_connections 2023-11-24T06:13:12.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:12.894+0000 7fe4d167e640 1 -- 172.21.15.92:0/2908781650 >> 172.21.15.92:0/2908781650 conn(0x7fe4cc0d3050 msgr2=0x7fe4cc0d3460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:13:12.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:12.894+0000 7fe4d167e640 1 -- 172.21.15.92:0/2908781650 shutdown_connections 2023-11-24T06:13:12.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:12.894+0000 7fe4d167e640 1 -- 172.21.15.92:0/2908781650 wait complete. 2023-11-24T06:13:12.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:12.896+0000 7fe4d167e640 1 Processor -- start 2023-11-24T06:13:12.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:12.896+0000 7fe4d167e640 1 -- start start 2023-11-24T06:13:12.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:12.897+0000 7fe4d167e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe4b8003970 0x7fe4cc1f6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:12.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:12.897+0000 7fe4d167e640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe4cc06c330 0x7fe4cc0b8560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:12.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:12.897+0000 7fe4d167e640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe4cc0b5c30 0x7fe4cc0b60b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:12.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:12.897+0000 7fe4d167e640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4cc0b75b0 con 0x7fe4b8003970 2023-11-24T06:13:12.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:12.897+0000 7fe4d167e640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4cc0b65f0 con 0x7fe4cc06c330 2023-11-24T06:13:12.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:12.897+0000 7fe4d167e640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4cc06bea0 con 0x7fe4cc0b5c30 2023-11-24T06:13:12.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:12.897+0000 7fe4cbfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe4b8003970 0x7fe4cc1f6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:12.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:12.897+0000 7fe4cbfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe4b8003970 0x7fe4cc1f6240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52756/0 (socket says 172.21.15.92:52756) 2023-11-24T06:13:12.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:12.897+0000 7fe4cbfff640 1 -- 172.21.15.92:0/1093435417 learned_addr learned my addr 172.21.15.92:0/1093435417 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:13:12.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:12.897+0000 7fe4cbfff640 1 -- 172.21.15.92:0/1093435417 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe4cc0b5c30 msgr2=0x7fe4cc0b60b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:12.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:12.897+0000 7fe4cbfff640 1 --2- 172.21.15.92:0/1093435417 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe4cc0b5c30 0x7fe4cc0b60b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:12.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:12.897+0000 7fe4cbfff640 1 -- 172.21.15.92:0/1093435417 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe4cc06c330 msgr2=0x7fe4cc0b8560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:12.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:12.897+0000 7fe4cbfff640 1 --2- 172.21.15.92:0/1093435417 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe4cc06c330 0x7fe4cc0b8560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:12.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:12.897+0000 7fe4cbfff640 1 -- 172.21.15.92:0/1093435417 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4cc156b00 con 0x7fe4b8003970 2023-11-24T06:13:12.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:12.898+0000 7fe4cbfff640 1 --2- 172.21.15.92:0/1093435417 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe4b8003970 0x7fe4cc1f6240 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fe4bc037040 tx=0x7fe4bc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:12.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:12.898+0000 7fe4c97fa640 1 -- 172.21.15.92:0/1093435417 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4bc003810 con 0x7fe4b8003970 2023-11-24T06:13:12.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:12.898+0000 7fe4c97fa640 1 -- 172.21.15.92:0/1093435417 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4bc02de90 con 0x7fe4b8003970 2023-11-24T06:13:12.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:12.898+0000 7fe4c97fa640 1 -- 172.21.15.92:0/1093435417 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4bc003b70 con 0x7fe4b8003970 2023-11-24T06:13:12.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:12.898+0000 7fe4d167e640 1 -- 172.21.15.92:0/1093435417 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4cc06b290 con 0x7fe4b8003970 2023-11-24T06:13:12.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:12.898+0000 7fe4d167e640 1 -- 172.21.15.92:0/1093435417 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4cc06b850 con 0x7fe4b8003970 2023-11-24T06:13:12.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:12.899+0000 7fe4d167e640 1 -- 172.21.15.92:0/1093435417 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4cc0cda10 con 0x7fe4b8003970 2023-11-24T06:13:12.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:12.900+0000 7fe4c97fa640 1 -- 172.21.15.92:0/1093435417 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7fe4bc003cd0 con 0x7fe4b8003970 2023-11-24T06:13:12.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:12.901+0000 7fe4c97fa640 1 --2- 172.21.15.92:0/1093435417 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fe4ac06af00 0x7fe4ac06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:12.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:12.901+0000 7fe4c97fa640 1 -- 172.21.15.92:0/1093435417 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe4bc0ab2f0 con 0x7fe4b8003970 2023-11-24T06:13:12.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:12.901+0000 7fe4c3fff640 1 --2- 172.21.15.92:0/1093435417 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fe4ac06af00 0x7fe4ac06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:12.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:12.902+0000 7fe4c3fff640 1 --2- 172.21.15.92:0/1093435417 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fe4ac06af00 0x7fe4ac06d3c0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fe4b4006fd0 tx=0x7fe4b4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:12.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:12.905+0000 7fe4c97fa640 1 -- 172.21.15.92:0/1093435417 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe4bc0762b0 con 0x7fe4b8003970 2023-11-24T06:13:13.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:13.096+0000 7fe4d167e640 1 -- 172.21.15.92:0/1093435417 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe4cc0b47c0 con 0x7fe4b8003970 2023-11-24T06:13:13.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:13.099+0000 7fe4c97fa640 1 -- 172.21.15.92:0/1093435417 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v63) v1 ==== 74+0+16520 (secure 0 0 0) 0x7fe4bc0798d0 con 0x7fe4b8003970 2023-11-24T06:13:13.101 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-24T06:13:13.101 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":63,"fsid":"2d6dee5f-8aa3-470f-931a-50ade963b1cb","created":"2023-11-24T06:10:04.707204+0000","modified":"2023-11-24T06:13:10.158258+0000","last_up_change":"2023-11-24T06:10:17.598352+0000","last_in_change":"2023-11-24T06:10:09.040793+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:10:17.733098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:13:06.409727+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-11-24T06:13:07.623678+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ff37a9fb-135f-471f-b4b0-f7b5772006c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6803","nonce":20560714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6805","nonce":20560714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6809","nonce":20560714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6807","nonce":20560714}]},"public_addr":"172.21.15.92:6803/20560714","cluster_addr":"172.21.15.92:6805/20560714","heartbeat_back_addr":"172.21.15.92:6809/20560714","heartbeat_front_addr":"172.21.15.92:6807/20560714","state":["exists","up"]},{"osd":1,"uuid":"03cdd353-dcf1-4f38-94fe-a795137c06b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6819","nonce":2451239906}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6821","nonce":2451239906}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6825","nonce":2451239906}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6823","nonce":2451239906}]},"public_addr":"172.21.15.92:6819/2451239906","cluster_addr":"172.21.15.92:6821/2451239906","heartbeat_back_addr":"172.21.15.92:6825/2451239906","heartbeat_front_addr":"172.21.15.92:6823/2451239906","state":["exists","up"]},{"osd":2,"uuid":"a4a20d80-5d57-484b-becd-9aa8070dd3aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6811","nonce":1124979277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6813","nonce":1124979277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6817","nonce":1124979277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6815","nonce":1124979277}]},"public_addr":"172.21.15.92:6811/1124979277","cluster_addr":"172.21.15.92:6813/1124979277","heartbeat_back_addr":"172.21.15.92:6817/1124979277","heartbeat_front_addr":"172.21.15.92:6815/1124979277","state":["exists","up"]},{"osd":3,"uuid":"4648d7ce-9b29-4583-9425-4ba8fd2b1903","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6827","nonce":1734115782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6829","nonce":1734115782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6833","nonce":1734115782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6831","nonce":1734115782}]},"public_addr":"172.21.15.92:6827/1734115782","cluster_addr":"172.21.15.92:6829/1734115782","heartbeat_back_addr":"172.21.15.92:6833/1734115782","heartbeat_front_addr":"172.21.15.92:6831/1734115782","state":["exists","up"]},{"osd":4,"uuid":"3bc9d7eb-c6fa-4fcf-a3ec-ade604e05aac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6819","nonce":3425937229}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6822","nonce":3425937229}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6830","nonce":3425937229}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6826","nonce":3425937229}]},"public_addr":"172.21.15.118:6819/3425937229","cluster_addr":"172.21.15.118:6822/3425937229","heartbeat_back_addr":"172.21.15.118:6830/3425937229","heartbeat_front_addr":"172.21.15.118:6826/3425937229","state":["exists","up"]},{"osd":5,"uuid":"dce9ba6d-59ed-46d1-84fc-a84b13de10c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6801","nonce":827455984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6803","nonce":827455984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6807","nonce":827455984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6805","nonce":827455984}]},"public_addr":"172.21.15.118:6801/827455984","cluster_addr":"172.21.15.118:6803/827455984","heartbeat_back_addr":"172.21.15.118:6807/827455984","heartbeat_front_addr":"172.21.15.118:6805/827455984","state":["exists","up"]},{"osd":6,"uuid":"82aec504-ae58-4709-8d30-46f2b12081b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6809","nonce":2946986141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6811","nonce":2946986141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6815","nonce":2946986141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6813","nonce":2946986141}]},"public_addr":"172.21.15.118:6809/2946986141","cluster_addr":"172.21.15.118:6811/2946986141","heartbeat_back_addr":"172.21.15.118:6815/2946986141","heartbeat_front_addr":"172.21.15.118:6813/2946986141","state":["exists","up"]},{"osd":7,"uuid":"4ed4c624-c88f-4d35-bdb1-bf209b5dffb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6817","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6818","nonce":48122059}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6821","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6823","nonce":48122059}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6829","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6831","nonce":48122059}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6825","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6827","nonce":48122059}]},"public_addr":"172.21.15.118:6818/48122059","cluster_addr":"172.21.15.118:6823/48122059","heartbeat_back_addr":"172.21.15.118:6831/48122059","heartbeat_front_addr":"172.21.15.118:6827/48122059","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.536664+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.368324+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.271201+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.342971+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.621832+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.387110+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.632946+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.416032+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:13:13.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:13.101+0000 7fe4d167e640 1 -- 172.21.15.92:0/1093435417 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fe4ac06af00 msgr2=0x7fe4ac06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:13.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:13.101+0000 7fe4d167e640 1 --2- 172.21.15.92:0/1093435417 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fe4ac06af00 0x7fe4ac06d3c0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fe4b4006fd0 tx=0x7fe4b4008040 comp rx=0 tx=0).stop 2023-11-24T06:13:13.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:13.101+0000 7fe4d167e640 1 -- 172.21.15.92:0/1093435417 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe4b8003970 msgr2=0x7fe4cc1f6240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:13.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:13.101+0000 7fe4d167e640 1 --2- 172.21.15.92:0/1093435417 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe4b8003970 0x7fe4cc1f6240 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fe4bc037040 tx=0x7fe4bc004730 comp rx=0 tx=0).stop 2023-11-24T06:13:13.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:13.101+0000 7fe4d167e640 1 -- 172.21.15.92:0/1093435417 shutdown_connections 2023-11-24T06:13:13.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:13.101+0000 7fe4d167e640 1 --2- 172.21.15.92:0/1093435417 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fe4ac06af00 0x7fe4ac06d3c0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:13.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:13.101+0000 7fe4d167e640 1 --2- 172.21.15.92:0/1093435417 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe4cc0b5c30 0x7fe4cc0b60b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:13.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:13.101+0000 7fe4d167e640 1 --2- 172.21.15.92:0/1093435417 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe4cc06c330 0x7fe4cc0b8560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:13.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:13.101+0000 7fe4d167e640 1 --2- 172.21.15.92:0/1093435417 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe4b8003970 0x7fe4cc1f6240 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:13.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:13.101+0000 7fe4d167e640 1 -- 172.21.15.92:0/1093435417 >> 172.21.15.92:0/1093435417 conn(0x7fe4cc0d3050 msgr2=0x7fe4cc0b4b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:13:13.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:13.102+0000 7fe4d167e640 1 -- 172.21.15.92:0/1093435417 shutdown_connections 2023-11-24T06:13:13.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:13.102+0000 7fe4d167e640 1 -- 172.21.15.92:0/1093435417 wait complete. 2023-11-24T06:13:13.123 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-11-24T06:13:13.123 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-24T06:13:13.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:13.332+0000 7f0b1289b640 1 Processor -- start 2023-11-24T06:13:13.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:13.332+0000 7f0b1289b640 1 -- start start 2023-11-24T06:13:13.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:13.334+0000 7f0b1289b640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f0b0c15e3b0 0x7f0b0c164de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:13.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:13.334+0000 7f0b1289b640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b0c0b1ee0 con 0x7f0b0c0cf9a0 2023-11-24T06:13:13.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:13.334+0000 7f0b1289b640 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b0c0b2780 con 0x7f0b0c143ae0 2023-11-24T06:13:13.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:13.334+0000 7f0b1289b640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b0c15e990 con 0x7f0b0c15e3b0 2023-11-24T06:13:13.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:13.334+0000 7f0b0b7fe640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f0b0c0cf9a0 0x7f0b0c15dca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:59334/0 (socket says 172.21.15.92:59334) 2023-11-24T06:13:13.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:13.334+0000 7f0b0b7fe640 1 -- 172.21.15.92:0/2190765277 learned_addr learned my addr 172.21.15.92:0/2190765277 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-24T06:13:13.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:13.334+0000 7f0b10e38640 1 --2- 172.21.15.92:0/2190765277 >> v2:172.21.15.92:3300/0 conn(0x7f0b0c15e3b0 0x7f0b0c164de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:13.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:13.334+0000 7f0b0affd640 1 -- 172.21.15.92:0/2190765277 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 747267848 0 0) 0x7f0b0c0b1ee0 con 0x7f0b0c0cf9a0 2023-11-24T06:13:13.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:13.334+0000 7f0b0affd640 1 -- 172.21.15.92:0/2190765277 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0af4003520 con 0x7f0b0c0cf9a0 2023-11-24T06:13:13.338 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:13.335+0000 7f0b10e38640 1 -- 172.21.15.92:0/2190765277 >> v1:172.21.15.118:6789/0 conn(0x7f0b0c143ae0 legacy=0x7f0b0c0cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:13:13.338 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:13.335+0000 7f0b10e38640 1 -- 172.21.15.92:0/2190765277 >> v1:172.21.15.92:6789/0 conn(0x7f0b0c0cf9a0 legacy=0x7f0b0c15dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:13.338 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:13.335+0000 7f0b10e38640 1 -- 172.21.15.92:0/2190765277 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b0c0b2130 con 0x7f0b0c15e3b0 2023-11-24T06:13:13.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:13.336+0000 7f0b10e38640 1 --2- 172.21.15.92:0/2190765277 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0b0c15e3b0 0x7f0b0c164de0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f0b04009d90 tx=0x7f0b0402beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1dea4505870a246d server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:13.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:13.336+0000 7f0b0affd640 1 -- 172.21.15.92:0/2190765277 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b0402cc60 con 0x7f0b0c15e3b0 2023-11-24T06:13:13.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:13.336+0000 7f0b0affd640 1 -- 172.21.15.92:0/2190765277 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b0402cdc0 con 0x7f0b0c15e3b0 2023-11-24T06:13:13.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:13.336+0000 7f0b0affd640 1 -- 172.21.15.92:0/2190765277 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b04034660 con 0x7f0b0c15e3b0 2023-11-24T06:13:13.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:13.336+0000 7f0b1289b640 1 -- 172.21.15.92:0/2190765277 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0b0c15e3b0 msgr2=0x7f0b0c164de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:13.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:13.336+0000 7f0b1289b640 1 --2- 172.21.15.92:0/2190765277 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0b0c15e3b0 0x7f0b0c164de0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f0b04009d90 tx=0x7f0b0402beb0 comp rx=0 tx=0).stop 2023-11-24T06:13:13.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:13.337+0000 7f0b1289b640 1 -- 172.21.15.92:0/2190765277 shutdown_connections 2023-11-24T06:13:13.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:13.337+0000 7f0b1289b640 1 --2- 172.21.15.92:0/2190765277 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0b0c15e3b0 0x7f0b0c164de0 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:13.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:13.337+0000 7f0b1289b640 1 -- 172.21.15.92:0/2190765277 >> 172.21.15.92:0/2190765277 conn(0x7f0b0c06a800 msgr2=0x7f0b0c06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:13:13.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:13.337+0000 7f0b1289b640 1 -- 172.21.15.92:0/2190765277 shutdown_connections 2023-11-24T06:13:13.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:13.337+0000 7f0b1289b640 1 -- 172.21.15.92:0/2190765277 wait complete. 2023-11-24T06:13:13.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:13.338+0000 7f0b1289b640 1 Processor -- start 2023-11-24T06:13:13.341 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:13.339+0000 7f0b1289b640 1 -- start start 2023-11-24T06:13:13.343 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:13.339+0000 7f0b1289b640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0b0c0cf9a0 0x7f0b0c1410b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:13.343 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:13.340+0000 7f0b1289b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0b0c143ae0 0x7f0b0c159420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:13.343 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:13.340+0000 7f0b1289b640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0b0c15e3b0 0x7f0b0c0af540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:13.343 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:13.340+0000 7f0b1289b640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b0c0acbe0 con 0x7f0b0c143ae0 2023-11-24T06:13:13.343 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:13.340+0000 7f0b1289b640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b0c1415f0 con 0x7f0b0c15e3b0 2023-11-24T06:13:13.344 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:13.340+0000 7f0b1289b640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b0c1418a0 con 0x7f0b0c0cf9a0 2023-11-24T06:13:13.344 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:13.340+0000 7f0b0b7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0b0c143ae0 0x7f0b0c159420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:13.344 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:13.340+0000 7f0b0b7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0b0c143ae0 0x7f0b0c159420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52780/0 (socket says 172.21.15.92:52780) 2023-11-24T06:13:13.344 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:13.340+0000 7f0b0b7fe640 1 -- 172.21.15.92:0/1389843650 learned_addr learned my addr 172.21.15.92:0/1389843650 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:13:13.344 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:13.340+0000 7f0b0bfff640 1 --2- 172.21.15.92:0/1389843650 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0b0c0cf9a0 0x7f0b0c1410b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:13.344 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:13.340+0000 7f0b0b7fe640 1 -- 172.21.15.92:0/1389843650 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0b0c0cf9a0 msgr2=0x7f0b0c1410b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:13.344 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:13.340+0000 7f0b0b7fe640 1 --2- 172.21.15.92:0/1389843650 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0b0c0cf9a0 0x7f0b0c1410b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:13.344 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:13.340+0000 7f0b0b7fe640 1 -- 172.21.15.92:0/1389843650 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0b0c15e3b0 msgr2=0x7f0b0c0af540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:13.344 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:13.340+0000 7f0b0b7fe640 1 --2- 172.21.15.92:0/1389843650 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0b0c15e3b0 0x7f0b0c0af540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:13.344 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:13.340+0000 7f0b0b7fe640 1 -- 172.21.15.92:0/1389843650 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b04009a70 con 0x7f0b0c143ae0 2023-11-24T06:13:13.344 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:13.340+0000 7f0b0bfff640 1 --2- 172.21.15.92:0/1389843650 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0b0c0cf9a0 0x7f0b0c1410b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:13:13.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:13.341+0000 7f0b0b7fe640 1 --2- 172.21.15.92:0/1389843650 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0b0c143ae0 0x7f0b0c159420 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f0b0000c500 tx=0x7f0b0000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:13.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:13.341+0000 7f0b08ff9640 1 -- 172.21.15.92:0/1389843650 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b00005ce0 con 0x7f0b0c143ae0 2023-11-24T06:13:13.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:13.341+0000 7f0b08ff9640 1 -- 172.21.15.92:0/1389843650 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b00005e40 con 0x7f0b0c143ae0 2023-11-24T06:13:13.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:13.341+0000 7f0b1289b640 1 -- 172.21.15.92:0/1389843650 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b0c141b80 con 0x7f0b0c143ae0 2023-11-24T06:13:13.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:13.341+0000 7f0b08ff9640 1 -- 172.21.15.92:0/1389843650 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b00007330 con 0x7f0b0c143ae0 2023-11-24T06:13:13.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:13.341+0000 7f0b1289b640 1 -- 172.21.15.92:0/1389843650 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b0c0ace30 con 0x7f0b0c143ae0 2023-11-24T06:13:13.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:13.342+0000 7f0b1289b640 1 -- 172.21.15.92:0/1389843650 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b0c0ad020 con 0x7f0b0c143ae0 2023-11-24T06:13:13.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:13.343+0000 7f0b08ff9640 1 -- 172.21.15.92:0/1389843650 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f0b00007550 con 0x7f0b0c143ae0 2023-11-24T06:13:13.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:13.344+0000 7f0b08ff9640 1 --2- 172.21.15.92:0/1389843650 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f0af806aed0 0x7f0af806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:13.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:13.345+0000 7f0b08ff9640 1 -- 172.21.15.92:0/1389843650 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0b00088c50 con 0x7f0b0c143ae0 2023-11-24T06:13:13.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:13.345+0000 7f0b0bfff640 1 --2- 172.21.15.92:0/1389843650 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f0af806aed0 0x7f0af806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:13.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:13.345+0000 7f0b0bfff640 1 --2- 172.21.15.92:0/1389843650 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f0af806aed0 0x7f0af806d390 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f0b0c1f6b70 tx=0x7f0afc009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:13.351 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:13.349+0000 7f0b08ff9640 1 -- 172.21.15.92:0/1389843650 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0b00053e60 con 0x7f0b0c143ae0 2023-11-24T06:13:13.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:13.539+0000 7f0b1289b640 1 -- 172.21.15.92:0/1389843650 --> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0b0c13ff80 con 0x7f0af806aed0 2023-11-24T06:13:13.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:13.562+0000 7f0b08ff9640 1 -- 172.21.15.92:0/1389843650 <== mgr.4101 v2:172.21.15.92:6800/784027401 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+275376 (secure 0 0 0) 0x7f0b0c13ff80 con 0x7f0af806aed0 2023-11-24T06:13:13.568 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-24T06:13:13.569 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2023-11-24T06:13:13.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:13.568+0000 7f0b1289b640 1 -- 172.21.15.92:0/1389843650 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f0af806aed0 msgr2=0x7f0af806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:13.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:13.568+0000 7f0b1289b640 1 --2- 172.21.15.92:0/1389843650 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f0af806aed0 0x7f0af806d390 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f0b0c1f6b70 tx=0x7f0afc009040 comp rx=0 tx=0).stop 2023-11-24T06:13:13.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:13.568+0000 7f0b1289b640 1 -- 172.21.15.92:0/1389843650 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0b0c143ae0 msgr2=0x7f0b0c159420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:13.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:13.568+0000 7f0b1289b640 1 --2- 172.21.15.92:0/1389843650 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0b0c143ae0 0x7f0b0c159420 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f0b0000c500 tx=0x7f0b0000c9c0 comp rx=0 tx=0).stop 2023-11-24T06:13:13.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:13.568+0000 7f0b1289b640 1 -- 172.21.15.92:0/1389843650 shutdown_connections 2023-11-24T06:13:13.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:13.568+0000 7f0b1289b640 1 --2- 172.21.15.92:0/1389843650 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f0af806aed0 0x7f0af806d390 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:13.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:13.568+0000 7f0b1289b640 1 --2- 172.21.15.92:0/1389843650 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0b0c15e3b0 0x7f0b0c0af540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:13.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:13.568+0000 7f0b1289b640 1 --2- 172.21.15.92:0/1389843650 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0b0c143ae0 0x7f0b0c159420 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:13.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:13.568+0000 7f0b1289b640 1 --2- 172.21.15.92:0/1389843650 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0b0c0cf9a0 0x7f0b0c1410b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:13.574 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:13.568+0000 7f0b1289b640 1 -- 172.21.15.92:0/1389843650 >> 172.21.15.92:0/1389843650 conn(0x7f0b0c06a800 msgr2=0x7f0b0c06abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:13:13.574 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:13.568+0000 7f0b1289b640 1 -- 172.21.15.92:0/1389843650 shutdown_connections 2023-11-24T06:13:13.574 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:13.568+0000 7f0b1289b640 1 -- 172.21.15.92:0/1389843650 wait complete. 2023-11-24T06:13:13.590 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":143,"stamp":"2023-11-24T06:13:11.716381+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":8,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":243800,"kb_used_data":6824,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754730920,"statfs":{"total":773094113280,"available":772844462080,"internally_reserved":0,"allocated":6987776,"data_stored":3639050,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":242535985},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":23,"apply_latency_ms":23,"commit_latency_ns":23000000,"apply_latency_ns":23000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.584243"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:13:08.130949+0000","last_change":"2023-11-24T06:13:08.130949+0000","last_active":"2023-11-24T06:13:08.130949+0000","last_peered":"2023-11-24T06:13:08.130949+0000","last_clean":"2023-11-24T06:13:08.130949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:13:08.130949+0000","last_undegraded":"2023-11-24T06:13:08.130949+0000","last_fullsized":"2023-11-24T06:13:08.130949+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_clean_scrub_stamp":"2023-11-24T06:13:08.130949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:13:08.130949+0000","last_change":"2023-11-24T06:13:08.130949+0000","last_active":"2023-11-24T06:13:08.130949+0000","last_peered":"2023-11-24T06:13:08.130949+0000","last_clean":"2023-11-24T06:13:08.130949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:13:08.130949+0000","last_undegraded":"2023-11-24T06:13:08.130949+0000","last_fullsized":"2023-11-24T06:13:08.130949+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_clean_scrub_stamp":"2023-11-24T06:13:08.130949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:13:08.130949+0000","last_change":"2023-11-24T06:13:08.130949+0000","last_active":"2023-11-24T06:13:08.130949+0000","last_peered":"2023-11-24T06:13:08.130949+0000","last_clean":"2023-11-24T06:13:08.130949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:13:08.130949+0000","last_undegraded":"2023-11-24T06:13:08.130949+0000","last_fullsized":"2023-11-24T06:13:08.130949+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_clean_scrub_stamp":"2023-11-24T06:13:08.130949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:13:08.130949+0000","last_change":"2023-11-24T06:13:08.130949+0000","last_active":"2023-11-24T06:13:08.130949+0000","last_peered":"2023-11-24T06:13:08.130949+0000","last_clean":"2023-11-24T06:13:08.130949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:13:08.130949+0000","last_undegraded":"2023-11-24T06:13:08.130949+0000","last_fullsized":"2023-11-24T06:13:08.130949+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_clean_scrub_stamp":"2023-11-24T06:13:08.130949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:13:08.130949+0000","last_change":"2023-11-24T06:13:08.130949+0000","last_active":"2023-11-24T06:13:08.130949+0000","last_peered":"2023-11-24T06:13:08.130949+0000","last_clean":"2023-11-24T06:13:08.130949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:13:08.130949+0000","last_undegraded":"2023-11-24T06:13:08.130949+0000","last_fullsized":"2023-11-24T06:13:08.130949+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_clean_scrub_stamp":"2023-11-24T06:13:08.130949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:13:08.130949+0000","last_change":"2023-11-24T06:13:08.130949+0000","last_active":"2023-11-24T06:13:08.130949+0000","last_peered":"2023-11-24T06:13:08.130949+0000","last_clean":"2023-11-24T06:13:08.130949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:13:08.130949+0000","last_undegraded":"2023-11-24T06:13:08.130949+0000","last_fullsized":"2023-11-24T06:13:08.130949+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_clean_scrub_stamp":"2023-11-24T06:13:08.130949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:13:08.130949+0000","last_change":"2023-11-24T06:13:08.130949+0000","last_active":"2023-11-24T06:13:08.130949+0000","last_peered":"2023-11-24T06:13:08.130949+0000","last_clean":"2023-11-24T06:13:08.130949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:13:08.130949+0000","last_undegraded":"2023-11-24T06:13:08.130949+0000","last_fullsized":"2023-11-24T06:13:08.130949+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_clean_scrub_stamp":"2023-11-24T06:13:08.130949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:13:08.130949+0000","last_change":"2023-11-24T06:13:08.130949+0000","last_active":"2023-11-24T06:13:08.130949+0000","last_peered":"2023-11-24T06:13:08.130949+0000","last_clean":"2023-11-24T06:13:08.130949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:13:08.130949+0000","last_undegraded":"2023-11-24T06:13:08.130949+0000","last_fullsized":"2023-11-24T06:13:08.130949+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_clean_scrub_stamp":"2023-11-24T06:13:08.130949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:13:08.130949+0000","last_change":"2023-11-24T06:13:08.130949+0000","last_active":"2023-11-24T06:13:08.130949+0000","last_peered":"2023-11-24T06:13:08.130949+0000","last_clean":"2023-11-24T06:13:08.130949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:13:08.130949+0000","last_undegraded":"2023-11-24T06:13:08.130949+0000","last_fullsized":"2023-11-24T06:13:08.130949+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_clean_scrub_stamp":"2023-11-24T06:13:08.130949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:13:08.130949+0000","last_change":"2023-11-24T06:13:08.130949+0000","last_active":"2023-11-24T06:13:08.130949+0000","last_peered":"2023-11-24T06:13:08.130949+0000","last_clean":"2023-11-24T06:13:08.130949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:13:08.130949+0000","last_undegraded":"2023-11-24T06:13:08.130949+0000","last_fullsized":"2023-11-24T06:13:08.130949+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_clean_scrub_stamp":"2023-11-24T06:13:08.130949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:13:08.130949+0000","last_change":"2023-11-24T06:13:08.130949+0000","last_active":"2023-11-24T06:13:08.130949+0000","last_peered":"2023-11-24T06:13:08.130949+0000","last_clean":"2023-11-24T06:13:08.130949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:13:08.130949+0000","last_undegraded":"2023-11-24T06:13:08.130949+0000","last_fullsized":"2023-11-24T06:13:08.130949+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_clean_scrub_stamp":"2023-11-24T06:13:08.130949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:13:08.130949+0000","last_change":"2023-11-24T06:13:08.130949+0000","last_active":"2023-11-24T06:13:08.130949+0000","last_peered":"2023-11-24T06:13:08.130949+0000","last_clean":"2023-11-24T06:13:08.130949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:13:08.130949+0000","last_undegraded":"2023-11-24T06:13:08.130949+0000","last_fullsized":"2023-11-24T06:13:08.130949+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_clean_scrub_stamp":"2023-11-24T06:13:08.130949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:13:08.130949+0000","last_change":"2023-11-24T06:13:08.130949+0000","last_active":"2023-11-24T06:13:08.130949+0000","last_peered":"2023-11-24T06:13:08.130949+0000","last_clean":"2023-11-24T06:13:08.130949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:13:08.130949+0000","last_undegraded":"2023-11-24T06:13:08.130949+0000","last_fullsized":"2023-11-24T06:13:08.130949+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_clean_scrub_stamp":"2023-11-24T06:13:08.130949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:13:08.130949+0000","last_change":"2023-11-24T06:13:08.130949+0000","last_active":"2023-11-24T06:13:08.130949+0000","last_peered":"2023-11-24T06:13:08.130949+0000","last_clean":"2023-11-24T06:13:08.130949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:13:08.130949+0000","last_undegraded":"2023-11-24T06:13:08.130949+0000","last_fullsized":"2023-11-24T06:13:08.130949+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_clean_scrub_stamp":"2023-11-24T06:13:08.130949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:13:08.130949+0000","last_change":"2023-11-24T06:13:08.130949+0000","last_active":"2023-11-24T06:13:08.130949+0000","last_peered":"2023-11-24T06:13:08.130949+0000","last_clean":"2023-11-24T06:13:08.130949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:13:08.130949+0000","last_undegraded":"2023-11-24T06:13:08.130949+0000","last_fullsized":"2023-11-24T06:13:08.130949+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_clean_scrub_stamp":"2023-11-24T06:13:08.130949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:13:08.130949+0000","last_change":"2023-11-24T06:13:08.130949+0000","last_active":"2023-11-24T06:13:08.130949+0000","last_peered":"2023-11-24T06:13:08.130949+0000","last_clean":"2023-11-24T06:13:08.130949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:13:08.130949+0000","last_undegraded":"2023-11-24T06:13:08.130949+0000","last_fullsized":"2023-11-24T06:13:08.130949+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_clean_scrub_stamp":"2023-11-24T06:13:08.130949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:13:08.130949+0000","last_change":"2023-11-24T06:13:08.130949+0000","last_active":"2023-11-24T06:13:08.130949+0000","last_peered":"2023-11-24T06:13:08.130949+0000","last_clean":"2023-11-24T06:13:08.130949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:13:08.130949+0000","last_undegraded":"2023-11-24T06:13:08.130949+0000","last_fullsized":"2023-11-24T06:13:08.130949+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_clean_scrub_stamp":"2023-11-24T06:13:08.130949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:13:08.130949+0000","last_change":"2023-11-24T06:13:08.130949+0000","last_active":"2023-11-24T06:13:08.130949+0000","last_peered":"2023-11-24T06:13:08.130949+0000","last_clean":"2023-11-24T06:13:08.130949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:13:08.130949+0000","last_undegraded":"2023-11-24T06:13:08.130949+0000","last_fullsized":"2023-11-24T06:13:08.130949+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_clean_scrub_stamp":"2023-11-24T06:13:08.130949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:13:08.130949+0000","last_change":"2023-11-24T06:13:08.130949+0000","last_active":"2023-11-24T06:13:08.130949+0000","last_peered":"2023-11-24T06:13:08.130949+0000","last_clean":"2023-11-24T06:13:08.130949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:13:08.130949+0000","last_undegraded":"2023-11-24T06:13:08.130949+0000","last_fullsized":"2023-11-24T06:13:08.130949+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_clean_scrub_stamp":"2023-11-24T06:13:08.130949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:13:08.130949+0000","last_change":"2023-11-24T06:13:08.130949+0000","last_active":"2023-11-24T06:13:08.130949+0000","last_peered":"2023-11-24T06:13:08.130949+0000","last_clean":"2023-11-24T06:13:08.130949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:13:08.130949+0000","last_undegraded":"2023-11-24T06:13:08.130949+0000","last_fullsized":"2023-11-24T06:13:08.130949+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_clean_scrub_stamp":"2023-11-24T06:13:08.130949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:13:08.130949+0000","last_change":"2023-11-24T06:13:08.130949+0000","last_active":"2023-11-24T06:13:08.130949+0000","last_peered":"2023-11-24T06:13:08.130949+0000","last_clean":"2023-11-24T06:13:08.130949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:13:08.130949+0000","last_undegraded":"2023-11-24T06:13:08.130949+0000","last_fullsized":"2023-11-24T06:13:08.130949+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_clean_scrub_stamp":"2023-11-24T06:13:08.130949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:13:08.130949+0000","last_change":"2023-11-24T06:13:08.130949+0000","last_active":"2023-11-24T06:13:08.130949+0000","last_peered":"2023-11-24T06:13:08.130949+0000","last_clean":"2023-11-24T06:13:08.130949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:13:08.130949+0000","last_undegraded":"2023-11-24T06:13:08.130949+0000","last_fullsized":"2023-11-24T06:13:08.130949+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_clean_scrub_stamp":"2023-11-24T06:13:08.130949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:13:08.130949+0000","last_change":"2023-11-24T06:13:08.130949+0000","last_active":"2023-11-24T06:13:08.130949+0000","last_peered":"2023-11-24T06:13:08.130949+0000","last_clean":"2023-11-24T06:13:08.130949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:13:08.130949+0000","last_undegraded":"2023-11-24T06:13:08.130949+0000","last_fullsized":"2023-11-24T06:13:08.130949+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_clean_scrub_stamp":"2023-11-24T06:13:08.130949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:13:08.130949+0000","last_change":"2023-11-24T06:13:08.130949+0000","last_active":"2023-11-24T06:13:08.130949+0000","last_peered":"2023-11-24T06:13:08.130949+0000","last_clean":"2023-11-24T06:13:08.130949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:13:08.130949+0000","last_undegraded":"2023-11-24T06:13:08.130949+0000","last_fullsized":"2023-11-24T06:13:08.130949+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_clean_scrub_stamp":"2023-11-24T06:13:08.130949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:13:08.130949+0000","last_change":"2023-11-24T06:13:08.130949+0000","last_active":"2023-11-24T06:13:08.130949+0000","last_peered":"2023-11-24T06:13:08.130949+0000","last_clean":"2023-11-24T06:13:08.130949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:13:08.130949+0000","last_undegraded":"2023-11-24T06:13:08.130949+0000","last_fullsized":"2023-11-24T06:13:08.130949+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_clean_scrub_stamp":"2023-11-24T06:13:08.130949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:13:08.130949+0000","last_change":"2023-11-24T06:13:08.130949+0000","last_active":"2023-11-24T06:13:08.130949+0000","last_peered":"2023-11-24T06:13:08.130949+0000","last_clean":"2023-11-24T06:13:08.130949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:13:08.130949+0000","last_undegraded":"2023-11-24T06:13:08.130949+0000","last_fullsized":"2023-11-24T06:13:08.130949+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_clean_scrub_stamp":"2023-11-24T06:13:08.130949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:13:08.130949+0000","last_change":"2023-11-24T06:13:08.130949+0000","last_active":"2023-11-24T06:13:08.130949+0000","last_peered":"2023-11-24T06:13:08.130949+0000","last_clean":"2023-11-24T06:13:08.130949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:13:08.130949+0000","last_undegraded":"2023-11-24T06:13:08.130949+0000","last_fullsized":"2023-11-24T06:13:08.130949+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_clean_scrub_stamp":"2023-11-24T06:13:08.130949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:13:08.130949+0000","last_change":"2023-11-24T06:13:08.130949+0000","last_active":"2023-11-24T06:13:08.130949+0000","last_peered":"2023-11-24T06:13:08.130949+0000","last_clean":"2023-11-24T06:13:08.130949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:13:08.130949+0000","last_undegraded":"2023-11-24T06:13:08.130949+0000","last_fullsized":"2023-11-24T06:13:08.130949+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_clean_scrub_stamp":"2023-11-24T06:13:08.130949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:13:08.130949+0000","last_change":"2023-11-24T06:13:08.130949+0000","last_active":"2023-11-24T06:13:08.130949+0000","last_peered":"2023-11-24T06:13:08.130949+0000","last_clean":"2023-11-24T06:13:08.130949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:13:08.130949+0000","last_undegraded":"2023-11-24T06:13:08.130949+0000","last_fullsized":"2023-11-24T06:13:08.130949+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_clean_scrub_stamp":"2023-11-24T06:13:08.130949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:13:08.130949+0000","last_change":"2023-11-24T06:13:08.130949+0000","last_active":"2023-11-24T06:13:08.130949+0000","last_peered":"2023-11-24T06:13:08.130949+0000","last_clean":"2023-11-24T06:13:08.130949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:13:08.130949+0000","last_undegraded":"2023-11-24T06:13:08.130949+0000","last_fullsized":"2023-11-24T06:13:08.130949+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_clean_scrub_stamp":"2023-11-24T06:13:08.130949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:13:08.130949+0000","last_change":"2023-11-24T06:13:08.130949+0000","last_active":"2023-11-24T06:13:08.130949+0000","last_peered":"2023-11-24T06:13:08.130949+0000","last_clean":"2023-11-24T06:13:08.130949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:13:08.130949+0000","last_undegraded":"2023-11-24T06:13:08.130949+0000","last_fullsized":"2023-11-24T06:13:08.130949+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_clean_scrub_stamp":"2023-11-24T06:13:08.130949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:13:08.130949+0000","last_change":"2023-11-24T06:13:08.130949+0000","last_active":"2023-11-24T06:13:08.130949+0000","last_peered":"2023-11-24T06:13:08.130949+0000","last_clean":"2023-11-24T06:13:08.130949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:13:08.130949+0000","last_undegraded":"2023-11-24T06:13:08.130949+0000","last_fullsized":"2023-11-24T06:13:08.130949+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_clean_scrub_stamp":"2023-11-24T06:13:08.130949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:13:08.130949+0000","last_change":"2023-11-24T06:13:08.130949+0000","last_active":"2023-11-24T06:13:08.130949+0000","last_peered":"2023-11-24T06:13:08.130949+0000","last_clean":"2023-11-24T06:13:08.130949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:13:08.130949+0000","last_undegraded":"2023-11-24T06:13:08.130949+0000","last_fullsized":"2023-11-24T06:13:08.130949+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_clean_scrub_stamp":"2023-11-24T06:13:08.130949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-24T06:13:07.148100+0000","last_change":"2023-11-24T06:13:07.148100+0000","last_active":"2023-11-24T06:13:07.116698+0000","last_peered":"2023-11-24T06:13:07.116698+0000","last_clean":"2023-11-24T06:13:07.116698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:13:07.148100+0000","last_undegraded":"2023-11-24T06:13:07.148100+0000","last_fullsized":"2023-11-24T06:13:07.148100+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:13:07.116698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:13:07.116698+0000","last_clean_scrub_stamp":"2023-11-24T06:13:07.116698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:13:08.130949+0000","last_change":"2023-11-24T06:13:08.130949+0000","last_active":"2023-11-24T06:13:08.130949+0000","last_peered":"2023-11-24T06:13:08.130949+0000","last_clean":"2023-11-24T06:13:08.130949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:13:08.130949+0000","last_undegraded":"2023-11-24T06:13:08.130949+0000","last_fullsized":"2023-11-24T06:13:08.130949+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_clean_scrub_stamp":"2023-11-24T06:13:08.130949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-24T06:13:07.143194+0000","last_change":"2023-11-24T06:13:07.143194+0000","last_active":"2023-11-24T06:13:07.116698+0000","last_peered":"2023-11-24T06:13:07.116698+0000","last_clean":"2023-11-24T06:13:07.116698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:13:07.143194+0000","last_undegraded":"2023-11-24T06:13:07.143194+0000","last_fullsized":"2023-11-24T06:13:07.143194+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:13:07.116698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:13:07.116698+0000","last_clean_scrub_stamp":"2023-11-24T06:13:07.116698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:13:08.130949+0000","last_change":"2023-11-24T06:13:08.130949+0000","last_active":"2023-11-24T06:13:08.130949+0000","last_peered":"2023-11-24T06:13:08.130949+0000","last_clean":"2023-11-24T06:13:08.130949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:13:08.130949+0000","last_undegraded":"2023-11-24T06:13:08.130949+0000","last_fullsized":"2023-11-24T06:13:08.130949+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_clean_scrub_stamp":"2023-11-24T06:13:08.130949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-24T06:13:07.138619+0000","last_change":"2023-11-24T06:13:07.138619+0000","last_active":"2023-11-24T06:13:07.116698+0000","last_peered":"2023-11-24T06:13:07.116698+0000","last_clean":"2023-11-24T06:13:07.116698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:13:07.138619+0000","last_undegraded":"2023-11-24T06:13:07.138619+0000","last_fullsized":"2023-11-24T06:13:07.138619+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:13:07.116698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:13:07.116698+0000","last_clean_scrub_stamp":"2023-11-24T06:13:07.116698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:13:08.130949+0000","last_change":"2023-11-24T06:13:08.130949+0000","last_active":"2023-11-24T06:13:08.130949+0000","last_peered":"2023-11-24T06:13:08.130949+0000","last_clean":"2023-11-24T06:13:08.130949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:13:08.130949+0000","last_undegraded":"2023-11-24T06:13:08.130949+0000","last_fullsized":"2023-11-24T06:13:08.130949+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_clean_scrub_stamp":"2023-11-24T06:13:08.130949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-24T06:13:07.137986+0000","last_change":"2023-11-24T06:13:07.137986+0000","last_active":"2023-11-24T06:13:07.116698+0000","last_peered":"2023-11-24T06:13:07.116698+0000","last_clean":"2023-11-24T06:13:07.116698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:13:07.137986+0000","last_undegraded":"2023-11-24T06:13:07.137986+0000","last_fullsized":"2023-11-24T06:13:07.137986+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:13:07.116698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:13:07.116698+0000","last_clean_scrub_stamp":"2023-11-24T06:13:07.116698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:13:08.130949+0000","last_change":"2023-11-24T06:13:08.130949+0000","last_active":"2023-11-24T06:13:08.130949+0000","last_peered":"2023-11-24T06:13:08.130949+0000","last_clean":"2023-11-24T06:13:08.130949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:13:08.130949+0000","last_undegraded":"2023-11-24T06:13:08.130949+0000","last_fullsized":"2023-11-24T06:13:08.130949+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_clean_scrub_stamp":"2023-11-24T06:13:08.130949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-24T06:13:07.143233+0000","last_change":"2023-11-24T06:13:07.143233+0000","last_active":"2023-11-24T06:13:07.116698+0000","last_peered":"2023-11-24T06:13:07.116698+0000","last_clean":"2023-11-24T06:13:07.116698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:13:07.143233+0000","last_undegraded":"2023-11-24T06:13:07.143233+0000","last_fullsized":"2023-11-24T06:13:07.143233+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:13:07.116698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:13:07.116698+0000","last_clean_scrub_stamp":"2023-11-24T06:13:07.116698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:13:08.130949+0000","last_change":"2023-11-24T06:13:08.130949+0000","last_active":"2023-11-24T06:13:08.130949+0000","last_peered":"2023-11-24T06:13:08.130949+0000","last_clean":"2023-11-24T06:13:08.130949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:13:08.130949+0000","last_undegraded":"2023-11-24T06:13:08.130949+0000","last_fullsized":"2023-11-24T06:13:08.130949+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_clean_scrub_stamp":"2023-11-24T06:13:08.130949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-24T06:13:07.141017+0000","last_change":"2023-11-24T06:13:07.141017+0000","last_active":"2023-11-24T06:13:07.116698+0000","last_peered":"2023-11-24T06:13:07.116698+0000","last_clean":"2023-11-24T06:13:07.116698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:13:07.141017+0000","last_undegraded":"2023-11-24T06:13:07.141017+0000","last_fullsized":"2023-11-24T06:13:07.141017+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:13:07.116698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:13:07.116698+0000","last_clean_scrub_stamp":"2023-11-24T06:13:07.116698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:13:08.130949+0000","last_change":"2023-11-24T06:13:08.130949+0000","last_active":"2023-11-24T06:13:08.130949+0000","last_peered":"2023-11-24T06:13:08.130949+0000","last_clean":"2023-11-24T06:13:08.130949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:13:08.130949+0000","last_undegraded":"2023-11-24T06:13:08.130949+0000","last_fullsized":"2023-11-24T06:13:08.130949+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_clean_scrub_stamp":"2023-11-24T06:13:08.130949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-24T06:13:07.144780+0000","last_change":"2023-11-24T06:13:07.144780+0000","last_active":"2023-11-24T06:13:07.116698+0000","last_peered":"2023-11-24T06:13:07.116698+0000","last_clean":"2023-11-24T06:13:07.116698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:13:07.144780+0000","last_undegraded":"2023-11-24T06:13:07.144780+0000","last_fullsized":"2023-11-24T06:13:07.144780+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:13:07.116698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:13:07.116698+0000","last_clean_scrub_stamp":"2023-11-24T06:13:07.116698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:13:08.130949+0000","last_change":"2023-11-24T06:13:08.130949+0000","last_active":"2023-11-24T06:13:08.130949+0000","last_peered":"2023-11-24T06:13:08.130949+0000","last_clean":"2023-11-24T06:13:08.130949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:13:08.130949+0000","last_undegraded":"2023-11-24T06:13:08.130949+0000","last_fullsized":"2023-11-24T06:13:08.130949+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_clean_scrub_stamp":"2023-11-24T06:13:08.130949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-24T06:13:07.146455+0000","last_change":"2023-11-24T06:13:07.146455+0000","last_active":"2023-11-24T06:13:07.116698+0000","last_peered":"2023-11-24T06:13:07.116698+0000","last_clean":"2023-11-24T06:13:07.116698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:13:07.146455+0000","last_undegraded":"2023-11-24T06:13:07.146455+0000","last_fullsized":"2023-11-24T06:13:07.146455+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:13:07.116698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:13:07.116698+0000","last_clean_scrub_stamp":"2023-11-24T06:13:07.116698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:13:08.130949+0000","last_change":"2023-11-24T06:13:08.130949+0000","last_active":"2023-11-24T06:13:08.130949+0000","last_peered":"2023-11-24T06:13:08.130949+0000","last_clean":"2023-11-24T06:13:08.130949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:13:08.130949+0000","last_undegraded":"2023-11-24T06:13:08.130949+0000","last_fullsized":"2023-11-24T06:13:08.130949+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_clean_scrub_stamp":"2023-11-24T06:13:08.130949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-24T06:13:07.139092+0000","last_change":"2023-11-24T06:13:07.139092+0000","last_active":"2023-11-24T06:13:07.116698+0000","last_peered":"2023-11-24T06:13:07.116698+0000","last_clean":"2023-11-24T06:13:07.116698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:13:07.139092+0000","last_undegraded":"2023-11-24T06:13:07.139092+0000","last_fullsized":"2023-11-24T06:13:07.139092+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:13:07.116698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:13:07.116698+0000","last_clean_scrub_stamp":"2023-11-24T06:13:07.116698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:13:08.130949+0000","last_change":"2023-11-24T06:13:08.130949+0000","last_active":"2023-11-24T06:13:08.130949+0000","last_peered":"2023-11-24T06:13:08.130949+0000","last_clean":"2023-11-24T06:13:08.130949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:13:08.130949+0000","last_undegraded":"2023-11-24T06:13:08.130949+0000","last_fullsized":"2023-11-24T06:13:08.130949+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_clean_scrub_stamp":"2023-11-24T06:13:08.130949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-24T06:13:07.139232+0000","last_change":"2023-11-24T06:13:07.139232+0000","last_active":"2023-11-24T06:13:07.116698+0000","last_peered":"2023-11-24T06:13:07.116698+0000","last_clean":"2023-11-24T06:13:07.116698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:13:07.139232+0000","last_undegraded":"2023-11-24T06:13:07.139232+0000","last_fullsized":"2023-11-24T06:13:07.139232+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:13:07.116698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:13:07.116698+0000","last_clean_scrub_stamp":"2023-11-24T06:13:07.116698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:13:08.130949+0000","last_change":"2023-11-24T06:13:08.130949+0000","last_active":"2023-11-24T06:13:08.130949+0000","last_peered":"2023-11-24T06:13:08.130949+0000","last_clean":"2023-11-24T06:13:08.130949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:13:08.130949+0000","last_undegraded":"2023-11-24T06:13:08.130949+0000","last_fullsized":"2023-11-24T06:13:08.130949+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_clean_scrub_stamp":"2023-11-24T06:13:08.130949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-24T06:13:07.143209+0000","last_change":"2023-11-24T06:13:07.143209+0000","last_active":"2023-11-24T06:13:07.116698+0000","last_peered":"2023-11-24T06:13:07.116698+0000","last_clean":"2023-11-24T06:13:07.116698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:13:07.143209+0000","last_undegraded":"2023-11-24T06:13:07.143209+0000","last_fullsized":"2023-11-24T06:13:07.143209+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:13:07.116698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:13:07.116698+0000","last_clean_scrub_stamp":"2023-11-24T06:13:07.116698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:13:08.130949+0000","last_change":"2023-11-24T06:13:08.130949+0000","last_active":"2023-11-24T06:13:08.130949+0000","last_peered":"2023-11-24T06:13:08.130949+0000","last_clean":"2023-11-24T06:13:08.130949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:13:08.130949+0000","last_undegraded":"2023-11-24T06:13:08.130949+0000","last_fullsized":"2023-11-24T06:13:08.130949+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_clean_scrub_stamp":"2023-11-24T06:13:08.130949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-24T06:13:07.144864+0000","last_change":"2023-11-24T06:13:07.144864+0000","last_active":"2023-11-24T06:13:07.116698+0000","last_peered":"2023-11-24T06:13:07.116698+0000","last_clean":"2023-11-24T06:13:07.116698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:13:07.144864+0000","last_undegraded":"2023-11-24T06:13:07.144864+0000","last_fullsized":"2023-11-24T06:13:07.144864+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:13:07.116698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:13:07.116698+0000","last_clean_scrub_stamp":"2023-11-24T06:13:07.116698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:13:08.130949+0000","last_change":"2023-11-24T06:13:08.130949+0000","last_active":"2023-11-24T06:13:08.130949+0000","last_peered":"2023-11-24T06:13:08.130949+0000","last_clean":"2023-11-24T06:13:08.130949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:13:08.130949+0000","last_undegraded":"2023-11-24T06:13:08.130949+0000","last_fullsized":"2023-11-24T06:13:08.130949+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_clean_scrub_stamp":"2023-11-24T06:13:08.130949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-24T06:13:07.145363+0000","last_change":"2023-11-24T06:13:07.145363+0000","last_active":"2023-11-24T06:13:07.116698+0000","last_peered":"2023-11-24T06:13:07.116698+0000","last_clean":"2023-11-24T06:13:07.116698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:13:07.145363+0000","last_undegraded":"2023-11-24T06:13:07.145363+0000","last_fullsized":"2023-11-24T06:13:07.145363+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:13:07.116698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:13:07.116698+0000","last_clean_scrub_stamp":"2023-11-24T06:13:07.116698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:13:08.130949+0000","last_change":"2023-11-24T06:13:08.130949+0000","last_active":"2023-11-24T06:13:08.130949+0000","last_peered":"2023-11-24T06:13:08.130949+0000","last_clean":"2023-11-24T06:13:08.130949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:13:08.130949+0000","last_undegraded":"2023-11-24T06:13:08.130949+0000","last_fullsized":"2023-11-24T06:13:08.130949+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_clean_scrub_stamp":"2023-11-24T06:13:08.130949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-24T06:13:07.139895+0000","last_change":"2023-11-24T06:13:07.139895+0000","last_active":"2023-11-24T06:13:07.116698+0000","last_peered":"2023-11-24T06:13:07.116698+0000","last_clean":"2023-11-24T06:13:07.116698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:13:07.139895+0000","last_undegraded":"2023-11-24T06:13:07.139895+0000","last_fullsized":"2023-11-24T06:13:07.139895+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:13:07.116698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:13:07.116698+0000","last_clean_scrub_stamp":"2023-11-24T06:13:07.116698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:13:08.130949+0000","last_change":"2023-11-24T06:13:08.130949+0000","last_active":"2023-11-24T06:13:08.130949+0000","last_peered":"2023-11-24T06:13:08.130949+0000","last_clean":"2023-11-24T06:13:08.130949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:13:08.130949+0000","last_undegraded":"2023-11-24T06:13:08.130949+0000","last_fullsized":"2023-11-24T06:13:08.130949+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_clean_scrub_stamp":"2023-11-24T06:13:08.130949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-24T06:13:07.140897+0000","last_change":"2023-11-24T06:13:07.140897+0000","last_active":"2023-11-24T06:13:07.116698+0000","last_peered":"2023-11-24T06:13:07.116698+0000","last_clean":"2023-11-24T06:13:07.116698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:13:07.140897+0000","last_undegraded":"2023-11-24T06:13:07.140897+0000","last_fullsized":"2023-11-24T06:13:07.140897+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:13:07.116698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:13:07.116698+0000","last_clean_scrub_stamp":"2023-11-24T06:13:07.116698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:13:08.130949+0000","last_change":"2023-11-24T06:13:08.130949+0000","last_active":"2023-11-24T06:13:08.130949+0000","last_peered":"2023-11-24T06:13:08.130949+0000","last_clean":"2023-11-24T06:13:08.130949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:13:08.130949+0000","last_undegraded":"2023-11-24T06:13:08.130949+0000","last_fullsized":"2023-11-24T06:13:08.130949+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_clean_scrub_stamp":"2023-11-24T06:13:08.130949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-24T06:13:07.138303+0000","last_change":"2023-11-24T06:13:07.138303+0000","last_active":"2023-11-24T06:13:07.116698+0000","last_peered":"2023-11-24T06:13:07.116698+0000","last_clean":"2023-11-24T06:13:07.116698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:13:07.138303+0000","last_undegraded":"2023-11-24T06:13:07.138303+0000","last_fullsized":"2023-11-24T06:13:07.138303+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:13:07.116698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:13:07.116698+0000","last_clean_scrub_stamp":"2023-11-24T06:13:07.116698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:13:08.130949+0000","last_change":"2023-11-24T06:13:08.130949+0000","last_active":"2023-11-24T06:13:08.130949+0000","last_peered":"2023-11-24T06:13:08.130949+0000","last_clean":"2023-11-24T06:13:08.130949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:13:08.130949+0000","last_undegraded":"2023-11-24T06:13:08.130949+0000","last_fullsized":"2023-11-24T06:13:08.130949+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_clean_scrub_stamp":"2023-11-24T06:13:08.130949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-24T06:13:07.141652+0000","last_change":"2023-11-24T06:13:07.141652+0000","last_active":"2023-11-24T06:13:07.116698+0000","last_peered":"2023-11-24T06:13:07.116698+0000","last_clean":"2023-11-24T06:13:07.116698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:13:07.141652+0000","last_undegraded":"2023-11-24T06:13:07.141652+0000","last_fullsized":"2023-11-24T06:13:07.141652+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:13:07.116698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:13:07.116698+0000","last_clean_scrub_stamp":"2023-11-24T06:13:07.116698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:13:08.130949+0000","last_change":"2023-11-24T06:13:08.130949+0000","last_active":"2023-11-24T06:13:08.130949+0000","last_peered":"2023-11-24T06:13:08.130949+0000","last_clean":"2023-11-24T06:13:08.130949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:13:08.130949+0000","last_undegraded":"2023-11-24T06:13:08.130949+0000","last_fullsized":"2023-11-24T06:13:08.130949+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_clean_scrub_stamp":"2023-11-24T06:13:08.130949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-24T06:13:07.138753+0000","last_change":"2023-11-24T06:13:07.138753+0000","last_active":"2023-11-24T06:13:07.116698+0000","last_peered":"2023-11-24T06:13:07.116698+0000","last_clean":"2023-11-24T06:13:07.116698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:13:07.138753+0000","last_undegraded":"2023-11-24T06:13:07.138753+0000","last_fullsized":"2023-11-24T06:13:07.138753+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:13:07.116698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:13:07.116698+0000","last_clean_scrub_stamp":"2023-11-24T06:13:07.116698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:13:08.130949+0000","last_change":"2023-11-24T06:13:08.130949+0000","last_active":"2023-11-24T06:13:08.130949+0000","last_peered":"2023-11-24T06:13:08.130949+0000","last_clean":"2023-11-24T06:13:08.130949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:13:08.130949+0000","last_undegraded":"2023-11-24T06:13:08.130949+0000","last_fullsized":"2023-11-24T06:13:08.130949+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_clean_scrub_stamp":"2023-11-24T06:13:08.130949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-24T06:13:07.141218+0000","last_change":"2023-11-24T06:13:07.141218+0000","last_active":"2023-11-24T06:13:07.116698+0000","last_peered":"2023-11-24T06:13:07.116698+0000","last_clean":"2023-11-24T06:13:07.116698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:13:07.141218+0000","last_undegraded":"2023-11-24T06:13:07.141218+0000","last_fullsized":"2023-11-24T06:13:07.141218+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:13:07.116698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:13:07.116698+0000","last_clean_scrub_stamp":"2023-11-24T06:13:07.116698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:13:08.130949+0000","last_change":"2023-11-24T06:13:08.130949+0000","last_active":"2023-11-24T06:13:08.130949+0000","last_peered":"2023-11-24T06:13:08.130949+0000","last_clean":"2023-11-24T06:13:08.130949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:13:08.130949+0000","last_undegraded":"2023-11-24T06:13:08.130949+0000","last_fullsized":"2023-11-24T06:13:08.130949+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_clean_scrub_stamp":"2023-11-24T06:13:08.130949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-24T06:13:07.150032+0000","last_change":"2023-11-24T06:13:07.150032+0000","last_active":"2023-11-24T06:13:07.116698+0000","last_peered":"2023-11-24T06:13:07.116698+0000","last_clean":"2023-11-24T06:13:07.116698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:13:07.150032+0000","last_undegraded":"2023-11-24T06:13:07.150032+0000","last_fullsized":"2023-11-24T06:13:07.150032+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:13:07.116698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:13:07.116698+0000","last_clean_scrub_stamp":"2023-11-24T06:13:07.116698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:13:08.130949+0000","last_change":"2023-11-24T06:13:08.130949+0000","last_active":"2023-11-24T06:13:08.130949+0000","last_peered":"2023-11-24T06:13:08.130949+0000","last_clean":"2023-11-24T06:13:08.130949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:13:08.130949+0000","last_undegraded":"2023-11-24T06:13:08.130949+0000","last_fullsized":"2023-11-24T06:13:08.130949+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_clean_scrub_stamp":"2023-11-24T06:13:08.130949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-24T06:13:07.137658+0000","last_change":"2023-11-24T06:13:07.137658+0000","last_active":"2023-11-24T06:13:07.116698+0000","last_peered":"2023-11-24T06:13:07.116698+0000","last_clean":"2023-11-24T06:13:07.116698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:13:07.137658+0000","last_undegraded":"2023-11-24T06:13:07.137658+0000","last_fullsized":"2023-11-24T06:13:07.137658+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:13:07.116698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:13:07.116698+0000","last_clean_scrub_stamp":"2023-11-24T06:13:07.116698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:13:08.130949+0000","last_change":"2023-11-24T06:13:08.130949+0000","last_active":"2023-11-24T06:13:08.130949+0000","last_peered":"2023-11-24T06:13:08.130949+0000","last_clean":"2023-11-24T06:13:08.130949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:13:08.130949+0000","last_undegraded":"2023-11-24T06:13:08.130949+0000","last_fullsized":"2023-11-24T06:13:08.130949+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_clean_scrub_stamp":"2023-11-24T06:13:08.130949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-24T06:13:07.139609+0000","last_change":"2023-11-24T06:13:07.139609+0000","last_active":"2023-11-24T06:13:07.116698+0000","last_peered":"2023-11-24T06:13:07.116698+0000","last_clean":"2023-11-24T06:13:07.116698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:13:07.139609+0000","last_undegraded":"2023-11-24T06:13:07.139609+0000","last_fullsized":"2023-11-24T06:13:07.139609+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:13:07.116698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:13:07.116698+0000","last_clean_scrub_stamp":"2023-11-24T06:13:07.116698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:13:08.130949+0000","last_change":"2023-11-24T06:13:08.130949+0000","last_active":"2023-11-24T06:13:08.130949+0000","last_peered":"2023-11-24T06:13:08.130949+0000","last_clean":"2023-11-24T06:13:08.130949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:13:08.130949+0000","last_undegraded":"2023-11-24T06:13:08.130949+0000","last_fullsized":"2023-11-24T06:13:08.130949+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_clean_scrub_stamp":"2023-11-24T06:13:08.130949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-24T06:13:07.142692+0000","last_change":"2023-11-24T06:13:07.142692+0000","last_active":"2023-11-24T06:13:07.116698+0000","last_peered":"2023-11-24T06:13:07.116698+0000","last_clean":"2023-11-24T06:13:07.116698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:13:07.142692+0000","last_undegraded":"2023-11-24T06:13:07.142692+0000","last_fullsized":"2023-11-24T06:13:07.142692+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:13:07.116698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:13:07.116698+0000","last_clean_scrub_stamp":"2023-11-24T06:13:07.116698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:13:08.130949+0000","last_change":"2023-11-24T06:13:08.130949+0000","last_active":"2023-11-24T06:13:08.130949+0000","last_peered":"2023-11-24T06:13:08.130949+0000","last_clean":"2023-11-24T06:13:08.130949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:13:08.130949+0000","last_undegraded":"2023-11-24T06:13:08.130949+0000","last_fullsized":"2023-11-24T06:13:08.130949+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_clean_scrub_stamp":"2023-11-24T06:13:08.130949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-24T06:13:07.140392+0000","last_change":"2023-11-24T06:13:07.140392+0000","last_active":"2023-11-24T06:13:07.116698+0000","last_peered":"2023-11-24T06:13:07.116698+0000","last_clean":"2023-11-24T06:13:07.116698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:13:07.140392+0000","last_undegraded":"2023-11-24T06:13:07.140392+0000","last_fullsized":"2023-11-24T06:13:07.140392+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:13:07.116698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:13:07.116698+0000","last_clean_scrub_stamp":"2023-11-24T06:13:07.116698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:13:08.130949+0000","last_change":"2023-11-24T06:13:08.130949+0000","last_active":"2023-11-24T06:13:08.130949+0000","last_peered":"2023-11-24T06:13:08.130949+0000","last_clean":"2023-11-24T06:13:08.130949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:13:08.130949+0000","last_undegraded":"2023-11-24T06:13:08.130949+0000","last_fullsized":"2023-11-24T06:13:08.130949+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_clean_scrub_stamp":"2023-11-24T06:13:08.130949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-24T06:13:07.142828+0000","last_change":"2023-11-24T06:13:07.142828+0000","last_active":"2023-11-24T06:13:07.116698+0000","last_peered":"2023-11-24T06:13:07.116698+0000","last_clean":"2023-11-24T06:13:07.116698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:13:07.142828+0000","last_undegraded":"2023-11-24T06:13:07.142828+0000","last_fullsized":"2023-11-24T06:13:07.142828+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:13:07.116698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:13:07.116698+0000","last_clean_scrub_stamp":"2023-11-24T06:13:07.116698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:13:08.130949+0000","last_change":"2023-11-24T06:13:08.130949+0000","last_active":"2023-11-24T06:13:08.130949+0000","last_peered":"2023-11-24T06:13:08.130949+0000","last_clean":"2023-11-24T06:13:08.130949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:13:08.130949+0000","last_undegraded":"2023-11-24T06:13:08.130949+0000","last_fullsized":"2023-11-24T06:13:08.130949+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_clean_scrub_stamp":"2023-11-24T06:13:08.130949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-24T06:13:07.146669+0000","last_change":"2023-11-24T06:13:07.146669+0000","last_active":"2023-11-24T06:13:07.116698+0000","last_peered":"2023-11-24T06:13:07.116698+0000","last_clean":"2023-11-24T06:13:07.116698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:13:07.146669+0000","last_undegraded":"2023-11-24T06:13:07.146669+0000","last_fullsized":"2023-11-24T06:13:07.146669+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:13:07.116698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:13:07.116698+0000","last_clean_scrub_stamp":"2023-11-24T06:13:07.116698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:13:08.130949+0000","last_change":"2023-11-24T06:13:08.130949+0000","last_active":"2023-11-24T06:13:08.130949+0000","last_peered":"2023-11-24T06:13:08.130949+0000","last_clean":"2023-11-24T06:13:08.130949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:13:08.130949+0000","last_undegraded":"2023-11-24T06:13:08.130949+0000","last_fullsized":"2023-11-24T06:13:08.130949+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_clean_scrub_stamp":"2023-11-24T06:13:08.130949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-24T06:13:07.144961+0000","last_change":"2023-11-24T06:13:07.144961+0000","last_active":"2023-11-24T06:13:07.116698+0000","last_peered":"2023-11-24T06:13:07.116698+0000","last_clean":"2023-11-24T06:13:07.116698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:13:07.144961+0000","last_undegraded":"2023-11-24T06:13:07.144961+0000","last_fullsized":"2023-11-24T06:13:07.144961+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:13:07.116698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:13:07.116698+0000","last_clean_scrub_stamp":"2023-11-24T06:13:07.116698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:13:08.130949+0000","last_change":"2023-11-24T06:13:08.130949+0000","last_active":"2023-11-24T06:13:08.130949+0000","last_peered":"2023-11-24T06:13:08.130949+0000","last_clean":"2023-11-24T06:13:08.130949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:13:08.130949+0000","last_undegraded":"2023-11-24T06:13:08.130949+0000","last_fullsized":"2023-11-24T06:13:08.130949+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_clean_scrub_stamp":"2023-11-24T06:13:08.130949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-24T06:13:07.146808+0000","last_change":"2023-11-24T06:13:07.146808+0000","last_active":"2023-11-24T06:13:07.116698+0000","last_peered":"2023-11-24T06:13:07.116698+0000","last_clean":"2023-11-24T06:13:07.116698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:13:07.146808+0000","last_undegraded":"2023-11-24T06:13:07.146808+0000","last_fullsized":"2023-11-24T06:13:07.146808+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:13:07.116698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:13:07.116698+0000","last_clean_scrub_stamp":"2023-11-24T06:13:07.116698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:13:08.130949+0000","last_change":"2023-11-24T06:13:08.130949+0000","last_active":"2023-11-24T06:13:08.130949+0000","last_peered":"2023-11-24T06:13:08.130949+0000","last_clean":"2023-11-24T06:13:08.130949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:13:08.130949+0000","last_undegraded":"2023-11-24T06:13:08.130949+0000","last_fullsized":"2023-11-24T06:13:08.130949+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_clean_scrub_stamp":"2023-11-24T06:13:08.130949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-24T06:13:07.145662+0000","last_change":"2023-11-24T06:13:07.145662+0000","last_active":"2023-11-24T06:13:07.116698+0000","last_peered":"2023-11-24T06:13:07.116698+0000","last_clean":"2023-11-24T06:13:07.116698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:13:07.145662+0000","last_undegraded":"2023-11-24T06:13:07.145662+0000","last_fullsized":"2023-11-24T06:13:07.145662+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:13:07.116698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:13:07.116698+0000","last_clean_scrub_stamp":"2023-11-24T06:13:07.116698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:13:08.130949+0000","last_change":"2023-11-24T06:13:08.130949+0000","last_active":"2023-11-24T06:13:08.130949+0000","last_peered":"2023-11-24T06:13:08.130949+0000","last_clean":"2023-11-24T06:13:08.130949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:13:08.130949+0000","last_undegraded":"2023-11-24T06:13:08.130949+0000","last_fullsized":"2023-11-24T06:13:08.130949+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_clean_scrub_stamp":"2023-11-24T06:13:08.130949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-24T06:13:07.138318+0000","last_change":"2023-11-24T06:13:07.138318+0000","last_active":"2023-11-24T06:13:07.116698+0000","last_peered":"2023-11-24T06:13:07.116698+0000","last_clean":"2023-11-24T06:13:07.116698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:13:07.138318+0000","last_undegraded":"2023-11-24T06:13:07.138318+0000","last_fullsized":"2023-11-24T06:13:07.138318+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:13:07.116698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:13:07.116698+0000","last_clean_scrub_stamp":"2023-11-24T06:13:07.116698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:13:08.130949+0000","last_change":"2023-11-24T06:13:08.130949+0000","last_active":"2023-11-24T06:13:08.130949+0000","last_peered":"2023-11-24T06:13:08.130949+0000","last_clean":"2023-11-24T06:13:08.130949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:13:08.130949+0000","last_undegraded":"2023-11-24T06:13:08.130949+0000","last_fullsized":"2023-11-24T06:13:08.130949+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_clean_scrub_stamp":"2023-11-24T06:13:08.130949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-24T06:13:07.137663+0000","last_change":"2023-11-24T06:13:07.137663+0000","last_active":"2023-11-24T06:13:07.116698+0000","last_peered":"2023-11-24T06:13:07.116698+0000","last_clean":"2023-11-24T06:13:07.116698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:13:07.137663+0000","last_undegraded":"2023-11-24T06:13:07.137663+0000","last_fullsized":"2023-11-24T06:13:07.137663+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:13:07.116698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:13:07.116698+0000","last_clean_scrub_stamp":"2023-11-24T06:13:07.116698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:13:08.130949+0000","last_change":"2023-11-24T06:13:08.130949+0000","last_active":"2023-11-24T06:13:08.130949+0000","last_peered":"2023-11-24T06:13:08.130949+0000","last_clean":"2023-11-24T06:13:08.130949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:13:08.130949+0000","last_undegraded":"2023-11-24T06:13:08.130949+0000","last_fullsized":"2023-11-24T06:13:08.130949+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_clean_scrub_stamp":"2023-11-24T06:13:08.130949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-24T06:13:07.141267+0000","last_change":"2023-11-24T06:13:07.141267+0000","last_active":"2023-11-24T06:13:07.116698+0000","last_peered":"2023-11-24T06:13:07.116698+0000","last_clean":"2023-11-24T06:13:07.116698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:13:07.141267+0000","last_undegraded":"2023-11-24T06:13:07.141267+0000","last_fullsized":"2023-11-24T06:13:07.141267+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:13:07.116698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:13:07.116698+0000","last_clean_scrub_stamp":"2023-11-24T06:13:07.116698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":199,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:13:04.553316+0000","last_change":"2023-11-24T06:10:20.082677+0000","last_active":"2023-11-24T06:13:04.553316+0000","last_peered":"2023-11-24T06:13:04.553316+0000","last_clean":"2023-11-24T06:13:04.553316+0000","last_became_active":"2023-11-24T06:10:20.080551+0000","last_became_peered":"2023-11-24T06:10:20.080551+0000","last_unstale":"2023-11-24T06:13:04.553316+0000","last_undegraded":"2023-11-24T06:13:04.553316+0000","last_fullsized":"2023-11-24T06:13:04.553316+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:18.602209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:18.602209+0000","last_clean_scrub_stamp":"2023-11-24T06:10:18.602209+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:47:24.216235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574884,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32472,"kb_used_data":1292,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339368,"statfs":{"total":96636764160,"available":96603512832,"internally_reserved":0,"allocated":1323008,"data_stored":899265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:12:18 2023","interfaces":[{"interface":"back","average":{"1min":1.016,"5min":1.091,"15min":1.103},"min":{"1min":0.75600000000000001,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":1.5580000000000001,"5min":1.9630000000000001,"15min":1.9630000000000001},"last":1.581},{"interface":"front","average":{"1min":1.0529999999999999,"5min":1.056,"15min":1.056},"min":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":1.605,"5min":1.621,"15min":1.621},"last":1.2130000000000001}]},{"osd":1,"last update":"Fri Nov 24 06:12:18 2023","interfaces":[{"interface":"back","average":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"min":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":1.657,"5min":2.0259999999999998,"15min":2.0259999999999998},"last":1.1120000000000001},{"interface":"front","average":{"1min":1.226,"5min":1.149,"15min":1.1359999999999999},"min":{"1min":0.70699999999999996,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.786,"5min":2.1480000000000001,"15min":2.1480000000000001},"last":1.5329999999999999}]},{"osd":2,"last update":"Fri Nov 24 06:12:18 2023","interfaces":[{"interface":"back","average":{"1min":1.1000000000000001,"5min":1.1339999999999999,"15min":1.1399999999999999},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.488,"5min":1.7809999999999999,"15min":1.7809999999999999},"last":0.77100000000000002},{"interface":"front","average":{"1min":1.2370000000000001,"5min":1.1850000000000001,"15min":1.1759999999999999},"min":{"1min":0.90900000000000003,"5min":0.86599999999999999,"15min":0.86599999999999999},"max":{"1min":2.0619999999999998,"5min":2.0619999999999998,"15min":2.0619999999999998},"last":1.272}]},{"osd":3,"last update":"Fri Nov 24 06:12:18 2023","interfaces":[{"interface":"back","average":{"1min":1.1970000000000001,"5min":1.204,"15min":1.2050000000000001},"min":{"1min":0.89600000000000002,"5min":0.73999999999999999,"15min":0.73999999999999999},"max":{"1min":1.615,"5min":1.615,"15min":1.615},"last":1.0820000000000001},{"interface":"front","average":{"1min":1.2909999999999999,"5min":1.3069999999999999,"15min":1.3089999999999999},"min":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"max":{"1min":1.9570000000000001,"5min":1.9570000000000001,"15min":1.9570000000000001},"last":1.417}]},{"osd":4,"last update":"Fri Nov 24 06:12:18 2023","interfaces":[{"interface":"back","average":{"1min":1.3660000000000001,"5min":1.304,"15min":1.294},"min":{"1min":0.748,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.9570000000000001,"5min":1.9570000000000001,"15min":1.9570000000000001},"last":1.3560000000000001},{"interface":"front","average":{"1min":1.3320000000000001,"5min":1.329,"15min":1.329},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.8340000000000001,"5min":2.1680000000000001,"15min":2.1680000000000001},"last":1.1519999999999999}]},{"osd":5,"last update":"Fri Nov 24 06:12:18 2023","interfaces":[{"interface":"back","average":{"1min":1.546,"5min":1.466,"15min":1.452},"min":{"1min":0.75900000000000001,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":2.0379999999999998,"5min":2.0379999999999998,"15min":2.0379999999999998},"last":1.446},{"interface":"front","average":{"1min":1.5720000000000001,"5min":1.4770000000000001,"15min":1.4610000000000001},"min":{"1min":1.077,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.923,"5min":2.0539999999999998,"15min":2.0539999999999998},"last":1.4870000000000001}]},{"osd":6,"last update":"Fri Nov 24 06:12:18 2023","interfaces":[{"interface":"back","average":{"1min":1.6559999999999999,"5min":1.548,"15min":1.53},"min":{"1min":0.94499999999999995,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":2.0899999999999999,"5min":2.0899999999999999,"15min":2.0899999999999999},"last":1.665},{"interface":"front","average":{"1min":1.6719999999999999,"5min":1.5580000000000001,"15min":1.5389999999999999},"min":{"1min":1.127,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":2.0070000000000001,"5min":2.0680000000000001,"15min":2.0680000000000001},"last":1.631}]}]},{"osd":6,"up_from":13,"seq":55834574885,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31892,"kb_used_data":712,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339948,"statfs":{"total":96636764160,"available":96604106752,"internally_reserved":0,"allocated":729088,"data_stored":308897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:12:21 2023","interfaces":[{"interface":"back","average":{"1min":1.097,"5min":1.194,"15min":1.21},"min":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":2.032,"5min":2.032,"15min":2.032},"last":1.0189999999999999},{"interface":"front","average":{"1min":1.1639999999999999,"5min":1.212,"15min":1.22},"min":{"1min":0.81599999999999995,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.9199999999999999,"5min":2.0230000000000001,"15min":2.0230000000000001},"last":0.78800000000000003}]},{"osd":1,"last update":"Fri Nov 24 06:12:21 2023","interfaces":[{"interface":"back","average":{"1min":1.149,"5min":1.163,"15min":1.165},"min":{"1min":0.72199999999999998,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":2.1499999999999999,"5min":2.1499999999999999,"15min":2.1499999999999999},"last":0.88800000000000001},{"interface":"front","average":{"1min":1.1970000000000001,"5min":1.2270000000000001,"15min":1.232},"min":{"1min":0.78700000000000003,"5min":0.77500000000000002,"15min":0.77500000000000002},"max":{"1min":1.645,"5min":1.8899999999999999,"15min":1.8899999999999999},"last":1.165}]},{"osd":2,"last update":"Fri Nov 24 06:12:21 2023","interfaces":[{"interface":"back","average":{"1min":1.4179999999999999,"5min":1.2050000000000001,"15min":1.169},"min":{"1min":0.88,"5min":0.82299999999999995,"15min":0.82299999999999995},"max":{"1min":4.7359999999999998,"5min":4.7359999999999998,"15min":4.7359999999999998},"last":0.95699999999999996},{"interface":"front","average":{"1min":1.4970000000000001,"5min":1.45,"15min":1.4419999999999999},"min":{"1min":0.93799999999999994,"5min":0.85899999999999999,"15min":0.85899999999999999},"max":{"1min":3.9700000000000002,"5min":3.9700000000000002,"15min":3.9700000000000002},"last":1.274}]},{"osd":3,"last update":"Fri Nov 24 06:12:21 2023","interfaces":[{"interface":"back","average":{"1min":1.2809999999999999,"5min":1.3069999999999999,"15min":1.3109999999999999},"min":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"max":{"1min":1.798,"5min":1.919,"15min":1.919},"last":1.3480000000000001},{"interface":"front","average":{"1min":1.28,"5min":1.292,"15min":1.294},"min":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":2.3929999999999998,"5min":2.3929999999999998,"15min":2.3929999999999998},"last":1.0800000000000001}]},{"osd":4,"last update":"Fri Nov 24 06:12:21 2023","interfaces":[{"interface":"back","average":{"1min":1.4470000000000001,"5min":1.4550000000000001,"15min":1.4570000000000001},"min":{"1min":0.873,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":2.4039999999999999,"5min":2.4039999999999999,"15min":2.4039999999999999},"last":1.4770000000000001},{"interface":"front","average":{"1min":1.5620000000000001,"5min":1.496,"15min":1.4850000000000001},"min":{"1min":0.88400000000000001,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":2.5609999999999999,"5min":2.5609999999999999,"15min":2.5609999999999999},"last":1.5369999999999999}]},{"osd":5,"last update":"Fri Nov 24 06:12:21 2023","interfaces":[{"interface":"back","average":{"1min":1.4610000000000001,"5min":1.4730000000000001,"15min":1.4750000000000001},"min":{"1min":0.90000000000000002,"5min":0.79100000000000004,"15min":0.79100000000000004},"max":{"1min":2.8290000000000002,"5min":2.8290000000000002,"15min":2.8290000000000002},"last":1.3819999999999999},{"interface":"front","average":{"1min":1.639,"5min":1.653,"15min":1.655},"min":{"1min":1.2250000000000001,"5min":0.93400000000000005,"15min":0.93400000000000005},"max":{"1min":2.9420000000000002,"5min":2.9420000000000002,"15min":2.9420000000000002},"last":1.5700000000000001}]},{"osd":7,"last update":"Fri Nov 24 06:12:21 2023","interfaces":[{"interface":"back","average":{"1min":1.6160000000000001,"5min":1.5960000000000001,"15min":1.5920000000000001},"min":{"1min":1.1299999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":2.1859999999999999,"5min":2.5139999999999998,"15min":2.5139999999999998},"last":1.6559999999999999},{"interface":"front","average":{"1min":1.7509999999999999,"5min":1.5880000000000001,"15min":1.5609999999999999},"min":{"1min":1.0249999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":4.7859999999999996,"5min":4.7859999999999996,"15min":4.7859999999999996},"last":1.706}]}]},{"osd":5,"up_from":13,"seq":55834574885,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31888,"kb_used_data":708,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339952,"statfs":{"total":96636764160,"available":96604110848,"internally_reserved":0,"allocated":724992,"data_stored":308283,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:12:21 2023","interfaces":[{"interface":"back","average":{"1min":0.94099999999999995,"5min":1.1890000000000001,"15min":1.23},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.4750000000000001,"5min":3.0030000000000001,"15min":3.0030000000000001},"last":1.1439999999999999},{"interface":"front","average":{"1min":0.99199999999999999,"5min":1.1799999999999999,"15min":1.2110000000000001},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.5369999999999999,"5min":2.0979999999999999,"15min":2.0979999999999999},"last":0.90000000000000002}]},{"osd":1,"last update":"Fri Nov 24 06:12:21 2023","interfaces":[{"interface":"back","average":{"1min":1.1599999999999999,"5min":1.252,"15min":1.268},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.728,"5min":5.0499999999999998,"15min":5.0499999999999998},"last":1.24},{"interface":"front","average":{"1min":1.0529999999999999,"5min":1.3939999999999999,"15min":1.4510000000000001},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.5509999999999999,"5min":3.6930000000000001,"15min":3.6930000000000001},"last":0.999}]},{"osd":2,"last update":"Fri Nov 24 06:12:21 2023","interfaces":[{"interface":"back","average":{"1min":1.161,"5min":1.3080000000000001,"15min":1.3320000000000001},"min":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"max":{"1min":1.5389999999999999,"5min":3.085,"15min":3.085},"last":1.4139999999999999},{"interface":"front","average":{"1min":1.1879999999999999,"5min":1.53,"15min":1.587},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.8340000000000001,"5min":4.7359999999999998,"15min":4.7359999999999998},"last":1.4450000000000001}]},{"osd":3,"last update":"Fri Nov 24 06:12:21 2023","interfaces":[{"interface":"back","average":{"1min":1.2450000000000001,"5min":1.385,"15min":1.409},"min":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"max":{"1min":1.7629999999999999,"5min":3.2229999999999999,"15min":3.2229999999999999},"last":1.472},{"interface":"front","average":{"1min":1.2929999999999999,"5min":1.4139999999999999,"15min":1.4339999999999999},"min":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"max":{"1min":1.8069999999999999,"5min":2.3260000000000001,"15min":2.3260000000000001},"last":1.327}]},{"osd":4,"last update":"Fri Nov 24 06:12:21 2023","interfaces":[{"interface":"back","average":{"1min":1.3759999999999999,"5min":1.528,"15min":1.554},"min":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"max":{"1min":1.756,"5min":3.1230000000000002,"15min":3.1230000000000002},"last":1.5569999999999999},{"interface":"front","average":{"1min":1.375,"5min":1.522,"15min":1.546},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":2.044,"5min":2.4809999999999999,"15min":2.4809999999999999},"last":1.1830000000000001}]},{"osd":6,"last update":"Fri Nov 24 06:12:21 2023","interfaces":[{"interface":"back","average":{"1min":1.427,"5min":1.6910000000000001,"15min":1.7350000000000001},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.913,"5min":4.6529999999999996,"15min":4.6529999999999996},"last":1.5069999999999999},{"interface":"front","average":{"1min":1.4890000000000001,"5min":1.6970000000000001,"15min":1.732},"min":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"max":{"1min":2.266,"5min":4.3529999999999998,"15min":4.3529999999999998},"last":1.3600000000000001}]},{"osd":7,"last update":"Fri Nov 24 06:12:21 2023","interfaces":[{"interface":"back","average":{"1min":1.496,"5min":1.6679999999999999,"15min":1.696},"min":{"1min":0.874,"5min":0.874,"15min":0.874},"max":{"1min":1.9830000000000001,"5min":2.8839999999999999,"15min":2.8839999999999999},"last":1.3009999999999999},{"interface":"front","average":{"1min":1.575,"5min":1.8080000000000001,"15min":1.847},"min":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":2.0699999999999998,"5min":4.8739999999999997,"15min":4.8739999999999997},"last":1.597}]}]},{"osd":4,"up_from":13,"seq":55834574885,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31892,"kb_used_data":712,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339948,"statfs":{"total":96636764160,"available":96604106752,"internally_reserved":0,"allocated":729088,"data_stored":308897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:12:21 2023","interfaces":[{"interface":"back","average":{"1min":1.127,"5min":1.0589999999999999,"15min":1.0469999999999999},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.585,"5min":1.585,"15min":1.585},"last":0.90500000000000003},{"interface":"front","average":{"1min":1.0429999999999999,"5min":1.1819999999999999,"15min":1.2050000000000001},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.744,"5min":1.8220000000000001,"15min":1.8220000000000001},"last":1.407}]},{"osd":1,"last update":"Fri Nov 24 06:12:21 2023","interfaces":[{"interface":"back","average":{"1min":1.1499999999999999,"5min":1.1259999999999999,"15min":1.1220000000000001},"min":{"1min":0.73499999999999999,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":1.97,"5min":1.97,"15min":1.97},"last":1.597},{"interface":"front","average":{"1min":1.1819999999999999,"5min":1.1779999999999999,"15min":1.1779999999999999},"min":{"1min":0.79800000000000004,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":1.917,"5min":1.917,"15min":1.917},"last":1.077}]},{"osd":2,"last update":"Fri Nov 24 06:12:21 2023","interfaces":[{"interface":"back","average":{"1min":1.0980000000000001,"5min":1.22,"15min":1.24},"min":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":1.6970000000000001,"5min":2.319,"15min":2.319},"last":1.131},{"interface":"front","average":{"1min":1.3320000000000001,"5min":1.3129999999999999,"15min":1.3100000000000001},"min":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"max":{"1min":2.077,"5min":2.0990000000000002,"15min":2.0990000000000002},"last":1.669}]},{"osd":3,"last update":"Fri Nov 24 06:12:21 2023","interfaces":[{"interface":"back","average":{"1min":1.206,"5min":1.3700000000000001,"15min":1.3979999999999999},"min":{"1min":0.752,"5min":0.752,"15min":0.752},"max":{"1min":2.048,"5min":2.048,"15min":2.048},"last":1.1890000000000001},{"interface":"front","average":{"1min":1.3340000000000001,"5min":1.4199999999999999,"15min":1.4339999999999999},"min":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"max":{"1min":1.8149999999999999,"5min":2.1720000000000002,"15min":2.1720000000000002},"last":1.2350000000000001}]},{"osd":5,"last update":"Fri Nov 24 06:12:21 2023","interfaces":[{"interface":"back","average":{"1min":1.2829999999999999,"5min":1.327,"15min":1.3340000000000001},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.835,"5min":1.895,"15min":1.895},"last":1.0309999999999999},{"interface":"front","average":{"1min":1.4359999999999999,"5min":1.528,"15min":1.544},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":2.044,"5min":2.5059999999999998,"15min":2.5059999999999998},"last":1.476}]},{"osd":6,"last update":"Fri Nov 24 06:12:21 2023","interfaces":[{"interface":"back","average":{"1min":1.423,"5min":1.522,"15min":1.538},"min":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"max":{"1min":2.1509999999999998,"5min":2.2069999999999999,"15min":2.2069999999999999},"last":1.494},{"interface":"front","average":{"1min":1.587,"5min":1.665,"15min":1.6779999999999999},"min":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":2.0139999999999998,"5min":2.4510000000000001,"15min":2.4510000000000001},"last":1.5309999999999999}]},{"osd":7,"last update":"Fri Nov 24 06:12:21 2023","interfaces":[{"interface":"back","average":{"1min":1.5640000000000001,"5min":1.6279999999999999,"15min":1.6379999999999999},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":2.214,"5min":2.4350000000000001,"15min":2.4350000000000001},"last":1.2869999999999999},{"interface":"front","average":{"1min":1.554,"5min":1.617,"15min":1.627},"min":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":2.1499999999999999,"5min":3.1120000000000001,"15min":3.1120000000000001},"last":1.4359999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574885,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27724,"kb_used_data":700,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344116,"statfs":{"total":96636764160,"available":96608374784,"internally_reserved":0,"allocated":716800,"data_stored":303080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:12:26 2023","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.98599999999999999,"15min":1},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":1.8360000000000001,"5min":2.766,"15min":2.766},"last":0.41799999999999998},{"interface":"front","average":{"1min":1.0800000000000001,"5min":0.94799999999999995,"15min":0.92600000000000005},"min":{"1min":0.44600000000000001,"5min":0.441,"15min":0.441},"max":{"1min":1.873,"5min":1.873,"15min":1.873},"last":0.67800000000000005}]},{"osd":1,"last update":"Fri Nov 24 06:12:26 2023","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.876,"15min":0.88200000000000001},"min":{"1min":0.46999999999999997,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.2569999999999999,"5min":2.5779999999999998,"15min":2.5779999999999998},"last":0.72499999999999998},{"interface":"front","average":{"1min":1.02,"5min":1.151,"15min":1.173},"min":{"1min":0.50700000000000001,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.7350000000000001,"5min":3.6899999999999999,"15min":3.6899999999999999},"last":0.629}]},{"osd":2,"last update":"Fri Nov 24 06:12:26 2023","interfaces":[{"interface":"back","average":{"1min":0.996,"5min":1.1399999999999999,"15min":1.1639999999999999},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.637,"5min":4.0620000000000003,"15min":4.0620000000000003},"last":0.79800000000000004},{"interface":"front","average":{"1min":1.113,"5min":1.204,"15min":1.2190000000000001},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.5309999999999999,"5min":2.9529999999999998,"15min":2.9529999999999998},"last":0.58799999999999997}]},{"osd":4,"last update":"Fri Nov 24 06:12:26 2023","interfaces":[{"interface":"back","average":{"1min":1.26,"5min":1.3939999999999999,"15min":1.4159999999999999},"min":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"max":{"1min":1.6819999999999999,"5min":4.125,"15min":4.125},"last":1.2330000000000001},{"interface":"front","average":{"1min":1.351,"5min":1.427,"15min":1.4399999999999999},"min":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"max":{"1min":1.7689999999999999,"5min":3.9409999999999998,"15min":3.9409999999999998},"last":1.0820000000000001}]},{"osd":5,"last update":"Fri Nov 24 06:12:26 2023","interfaces":[{"interface":"back","average":{"1min":1.319,"5min":1.421,"15min":1.4379999999999999},"min":{"1min":0.749,"5min":0.749,"15min":0.749},"max":{"1min":1.6679999999999999,"5min":3.3359999999999999,"15min":3.3359999999999999},"last":1.117},{"interface":"front","average":{"1min":1.214,"5min":1.4099999999999999,"15min":1.4430000000000001},"min":{"1min":0.86499999999999999,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.542,"5min":4.1669999999999998,"15min":4.1669999999999998},"last":0.91500000000000004}]},{"osd":6,"last update":"Fri Nov 24 06:12:26 2023","interfaces":[{"interface":"back","average":{"1min":1.2230000000000001,"5min":1.3939999999999999,"15min":1.4219999999999999},"min":{"1min":0.871,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":1.571,"5min":4.0190000000000001,"15min":4.0190000000000001},"last":1.194},{"interface":"front","average":{"1min":1.429,"5min":1.512,"15min":1.526},"min":{"1min":0.95999999999999996,"5min":0.92300000000000004,"15min":0.92300000000000004},"max":{"1min":1.966,"5min":3.2480000000000002,"15min":3.2480000000000002},"last":1.0149999999999999}]},{"osd":7,"last update":"Fri Nov 24 06:12:26 2023","interfaces":[{"interface":"back","average":{"1min":1.3400000000000001,"5min":1.5580000000000001,"15min":1.5940000000000001},"min":{"1min":0.79500000000000004,"5min":0.76600000000000001,"15min":0.76600000000000001},"max":{"1min":1.8049999999999999,"5min":4.202,"15min":4.202},"last":1.151},{"interface":"front","average":{"1min":1.3480000000000001,"5min":1.5669999999999999,"15min":1.603},"min":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"max":{"1min":1.9199999999999999,"5min":3.9849999999999999,"15min":3.9849999999999999},"last":1.2669999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574885,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27724,"kb_used_data":696,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344116,"statfs":{"total":96636764160,"available":96608374784,"internally_reserved":0,"allocated":712704,"data_stored":302466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:12:22 2023","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.88,"15min":0.88600000000000001},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":1.3149999999999999,"5min":1.6259999999999999,"15min":1.6259999999999999},"last":1},{"interface":"front","average":{"1min":0.96799999999999997,"5min":1.0740000000000001,"15min":1.0920000000000001},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.619,"5min":1.7669999999999999,"15min":1.7669999999999999},"last":1.0289999999999999}]},{"osd":1,"last update":"Fri Nov 24 06:12:22 2023","interfaces":[{"interface":"back","average":{"1min":0.99399999999999999,"5min":0.94999999999999996,"15min":0.94199999999999995},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.635,"5min":1.635,"15min":1.635},"last":0.88800000000000001},{"interface":"front","average":{"1min":0.95499999999999996,"5min":0.97599999999999998,"15min":0.97999999999999998},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.498,"5min":1.8140000000000001,"15min":1.8140000000000001},"last":1.579}]},{"osd":3,"last update":"Fri Nov 24 06:12:22 2023","interfaces":[{"interface":"back","average":{"1min":0.94499999999999995,"5min":1.0529999999999999,"15min":1.071},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.476,"5min":2.3559999999999999,"15min":2.3559999999999999},"last":0.95899999999999996},{"interface":"front","average":{"1min":1.1719999999999999,"5min":1.1319999999999999,"15min":1.1259999999999999},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.643,"5min":2.1840000000000002,"15min":2.1840000000000002},"last":1.339}]},{"osd":4,"last update":"Fri Nov 24 06:12:22 2023","interfaces":[{"interface":"back","average":{"1min":1.411,"5min":1.417,"15min":1.4179999999999999},"min":{"1min":0.72099999999999997,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.9350000000000001,"5min":3.411,"15min":3.411},"last":1.169},{"interface":"front","average":{"1min":1.333,"5min":1.4710000000000001,"15min":1.494},"min":{"1min":0.88400000000000001,"5min":0.82699999999999996,"15min":0.82699999999999996},"max":{"1min":1.964,"5min":3.7109999999999999,"15min":3.7109999999999999},"last":1.2}]},{"osd":5,"last update":"Fri Nov 24 06:12:22 2023","interfaces":[{"interface":"back","average":{"1min":1.246,"5min":1.3089999999999999,"15min":1.319},"min":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"max":{"1min":1.5620000000000001,"5min":2.5259999999999998,"15min":2.5259999999999998},"last":1.498},{"interface":"front","average":{"1min":1.444,"5min":1.3300000000000001,"15min":1.3109999999999999},"min":{"1min":0.96799999999999997,"5min":0.90400000000000003,"15min":0.90400000000000003},"max":{"1min":1.8839999999999999,"5min":2.1640000000000001,"15min":2.1640000000000001},"last":1.524}]},{"osd":6,"last update":"Fri Nov 24 06:12:22 2023","interfaces":[{"interface":"back","average":{"1min":1.244,"5min":1.4139999999999999,"15min":1.4419999999999999},"min":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"max":{"1min":1.8400000000000001,"5min":2.3039999999999998,"15min":2.3039999999999998},"last":1.6950000000000001},{"interface":"front","average":{"1min":1.347,"5min":1.444,"15min":1.46},"min":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"max":{"1min":1.6850000000000001,"5min":2.605,"15min":2.605},"last":1.2709999999999999}]},{"osd":7,"last update":"Fri Nov 24 06:12:22 2023","interfaces":[{"interface":"back","average":{"1min":1.365,"5min":1.5489999999999999,"15min":1.579},"min":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"max":{"1min":1.9079999999999999,"5min":2.3740000000000001,"15min":2.3740000000000001},"last":1.407},{"interface":"front","average":{"1min":1.3009999999999999,"5min":1.4330000000000001,"15min":1.4550000000000001},"min":{"1min":0.877,"5min":0.877,"15min":0.877},"max":{"1min":1.575,"5min":2.5699999999999998,"15min":2.5699999999999998},"last":1.6299999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574884,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27736,"kb_used_data":712,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344104,"statfs":{"total":96636764160,"available":96608362496,"internally_reserved":0,"allocated":729088,"data_stored":308897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:12:21 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.878,"15min":0.90800000000000003},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":1.3740000000000001,"5min":1.9299999999999999,"15min":1.9299999999999999},"last":1.034},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.90700000000000003,"15min":0.92500000000000004},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":1.258,"5min":1.7669999999999999,"15min":1.7669999999999999},"last":0.63100000000000001}]},{"osd":2,"last update":"Fri Nov 24 06:12:21 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.96499999999999997,"15min":1.0009999999999999},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.0840000000000001,"5min":2.5329999999999999,"15min":2.5329999999999999},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.99099999999999999,"15min":1.0169999999999999},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.159,"5min":1.657,"15min":1.657},"last":1.1140000000000001}]},{"osd":3,"last update":"Fri Nov 24 06:12:21 2023","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.98899999999999999,"15min":1.0209999999999999},"min":{"1min":0.49399999999999999,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.2729999999999999,"5min":1.841,"15min":1.841},"last":1.325},{"interface":"front","average":{"1min":0.98399999999999999,"5min":1.121,"15min":1.1439999999999999},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.5249999999999999,"5min":2.8940000000000001,"15min":2.8940000000000001},"last":1.0880000000000001}]},{"osd":4,"last update":"Fri Nov 24 06:12:21 2023","interfaces":[{"interface":"back","average":{"1min":1.145,"5min":1.254,"15min":1.272},"min":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":1.454,"5min":2.1930000000000001,"15min":2.1930000000000001},"last":1.5900000000000001},{"interface":"front","average":{"1min":1.1200000000000001,"5min":1.1839999999999999,"15min":1.194},"min":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":1.569,"5min":1.784,"15min":1.784},"last":1.5189999999999999}]},{"osd":5,"last update":"Fri Nov 24 06:12:21 2023","interfaces":[{"interface":"back","average":{"1min":1.1499999999999999,"5min":1.454,"15min":1.504},"min":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":1.5820000000000001,"5min":5.5449999999999999,"15min":5.5449999999999999},"last":1.4339999999999999},{"interface":"front","average":{"1min":1.167,"5min":1.484,"15min":1.5369999999999999},"min":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":1.538,"5min":6.0030000000000001,"15min":6.0030000000000001},"last":1.3740000000000001}]},{"osd":6,"last update":"Fri Nov 24 06:12:21 2023","interfaces":[{"interface":"back","average":{"1min":1.2030000000000001,"5min":1.4950000000000001,"15min":1.544},"min":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":1.5189999999999999,"5min":5.2809999999999997,"15min":5.2809999999999997},"last":1.22},{"interface":"front","average":{"1min":1.2849999999999999,"5min":1.4370000000000001,"15min":1.462},"min":{"1min":0.79300000000000004,"5min":0.749,"15min":0.749},"max":{"1min":1.7969999999999999,"5min":4.327,"15min":4.327},"last":1.4830000000000001}]},{"osd":7,"last update":"Fri Nov 24 06:12:21 2023","interfaces":[{"interface":"back","average":{"1min":1.173,"5min":1.321,"15min":1.3460000000000001},"min":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":1.8540000000000001,"5min":2.3069999999999999,"15min":2.3069999999999999},"last":1.681},{"interface":"front","average":{"1min":1.252,"5min":1.528,"15min":1.5740000000000001},"min":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":1.6819999999999999,"5min":5.4500000000000002,"15min":5.4500000000000002},"last":1.548}]}]},{"osd":0,"up_from":13,"seq":55834574884,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32472,"kb_used_data":1292,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339368,"statfs":{"total":96636764160,"available":96603512832,"internally_reserved":0,"allocated":1323008,"data_stored":899265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Nov 24 06:12:20 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.90500000000000003,"15min":0.92700000000000005},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":1.675,"5min":2.3999999999999999,"15min":2.3999999999999999},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.79900000000000004,"5min":1.1100000000000001,"15min":1.1619999999999999},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.403,"5min":4.8289999999999997,"15min":4.8289999999999997},"last":0.871}]},{"osd":2,"last update":"Fri Nov 24 06:12:20 2023","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":1.0249999999999999,"15min":1.05},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.5960000000000001,"5min":2.5139999999999998,"15min":2.5139999999999998},"last":1.111},{"interface":"front","average":{"1min":0.98199999999999998,"5min":1.0860000000000001,"15min":1.103},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.4390000000000001,"5min":2.3599999999999999,"15min":2.3599999999999999},"last":1.1930000000000001}]},{"osd":3,"last update":"Fri Nov 24 06:12:20 2023","interfaces":[{"interface":"back","average":{"1min":1.018,"5min":1.1040000000000001,"15min":1.1180000000000001},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.514,"5min":2.5840000000000001,"15min":2.5840000000000001},"last":1.331},{"interface":"front","average":{"1min":1.071,"5min":1.3460000000000001,"15min":1.3919999999999999},"min":{"1min":0.503,"5min":0.495,"15min":0.495},"max":{"1min":1.526,"5min":5.1230000000000002,"15min":5.1230000000000002},"last":1.3700000000000001}]},{"osd":4,"last update":"Fri Nov 24 06:12:20 2023","interfaces":[{"interface":"back","average":{"1min":1.0209999999999999,"5min":1.23,"15min":1.2649999999999999},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.3919999999999999,"5min":2.887,"15min":2.887},"last":1.238},{"interface":"front","average":{"1min":1.0760000000000001,"5min":1.284,"15min":1.3180000000000001},"min":{"1min":0.71599999999999997,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":1.6819999999999999,"5min":2.8530000000000002,"15min":2.8530000000000002},"last":1.1479999999999999}]},{"osd":5,"last update":"Fri Nov 24 06:12:20 2023","interfaces":[{"interface":"back","average":{"1min":1.2370000000000001,"5min":1.5069999999999999,"15min":1.552},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.804,"5min":4.6299999999999999,"15min":4.6299999999999999},"last":1.2949999999999999},{"interface":"front","average":{"1min":1.056,"5min":1.2769999999999999,"15min":1.3140000000000001},"min":{"1min":0.72699999999999998,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.3560000000000001,"5min":2.794,"15min":2.794},"last":1.0129999999999999}]},{"osd":6,"last update":"Fri Nov 24 06:12:20 2023","interfaces":[{"interface":"back","average":{"1min":1.2829999999999999,"5min":1.5940000000000001,"15min":1.6459999999999999},"min":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.6779999999999999,"5min":4.798,"15min":4.798},"last":1.458},{"interface":"front","average":{"1min":1.375,"5min":1.5840000000000001,"15min":1.619},"min":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"max":{"1min":1.9350000000000001,"5min":4.7149999999999999,"15min":4.7149999999999999},"last":1.4950000000000001}]},{"osd":7,"last update":"Fri Nov 24 06:12:20 2023","interfaces":[{"interface":"back","average":{"1min":1.3109999999999999,"5min":1.635,"15min":1.6890000000000001},"min":{"1min":0.73799999999999999,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.9750000000000001,"5min":4.7649999999999997,"15min":4.7649999999999997},"last":1.409},{"interface":"front","average":{"1min":1.421,"5min":1.6399999999999999,"15min":1.6759999999999999},"min":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"max":{"1min":2.0430000000000001,"5min":4.6820000000000004,"15min":4.6820000000000004},"last":1.4359999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-24T06:13:13.596 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-24T06:13:13.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:13.851+0000 7fd46c6d4640 1 Processor -- start 2023-11-24T06:13:13.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:13.851+0000 7fd46c6d4640 1 -- start start 2023-11-24T06:13:13.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:13.853+0000 7fd46c6d4640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fd4640b3ae0 0x7fd46415e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:13.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:13.853+0000 7fd46c6d4640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fd4640cf290 0x7fd4640cf720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:13.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:13.853+0000 7fd46c6d4640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd464070c40 con 0x7fd4640ae5a0 2023-11-24T06:13:13.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:13.853+0000 7fd46c6d4640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd464071450 con 0x7fd4640cf290 2023-11-24T06:13:13.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:13.853+0000 7fd46c6d4640 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4640cfc60 con 0x7fd4640b3ae0 2023-11-24T06:13:13.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:13.853+0000 7fd46ac71640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fd4640cf290 0x7fd4640cf720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:13.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:13.853+0000 7fd46a470640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fd4640ae5a0 0x7fd46406cbe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:59336/0 (socket says 172.21.15.92:59336) 2023-11-24T06:13:13.856 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:13.853+0000 7fd46ac71640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fd4640cf290 0x7fd4640cf720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52782/0 (socket says 172.21.15.92:52782) 2023-11-24T06:13:13.856 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:13.853+0000 7fd46a470640 1 -- 172.21.15.92:0/2488083593 learned_addr learned my addr 172.21.15.92:0/2488083593 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-24T06:13:13.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:13.853+0000 7fd469c6f640 1 --2- 172.21.15.92:0/2488083593 >> v2:172.21.15.118:3300/0 conn(0x7fd4640b3ae0 0x7fd46415e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:13.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:13.853+0000 7fd46ac71640 1 -- 172.21.15.92:0/2488083593 >> v2:172.21.15.118:3300/0 conn(0x7fd4640b3ae0 msgr2=0x7fd46415e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:13:13.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:13.853+0000 7fd46ac71640 1 --2- 172.21.15.92:0/2488083593 >> v2:172.21.15.118:3300/0 conn(0x7fd4640b3ae0 0x7fd46415e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:13.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:13.853+0000 7fd46ac71640 1 -- 172.21.15.92:0/2488083593 >> v1:172.21.15.92:6789/0 conn(0x7fd4640ae5a0 legacy=0x7fd46406cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:13.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:13.854+0000 7fd46ac71640 1 -- 172.21.15.92:0/2488083593 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd46406c470 con 0x7fd4640cf290 2023-11-24T06:13:13.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:13.854+0000 7fd46946e640 1 -- 172.21.15.92:0/2488083593 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3876739513 0 0) 0x7fd464070c40 con 0x7fd4640ae5a0 2023-11-24T06:13:13.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:13.855+0000 7fd46ac71640 1 --2- 172.21.15.92:0/2488083593 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd4640cf290 0x7fd4640cf720 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fd4640cfea0 tx=0x7fd45c02bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abd2c88d0790dd32 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:13.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:13.855+0000 7fd46946e640 1 -- 172.21.15.92:0/2488083593 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd45c02cd00 con 0x7fd4640cf290 2023-11-24T06:13:13.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:13.855+0000 7fd46946e640 1 -- 172.21.15.92:0/2488083593 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd45c02ce60 con 0x7fd4640cf290 2023-11-24T06:13:13.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:13.855+0000 7fd46946e640 1 -- 172.21.15.92:0/2488083593 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd45c034660 con 0x7fd4640cf290 2023-11-24T06:13:13.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:13.855+0000 7fd46c6d4640 1 -- 172.21.15.92:0/2488083593 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd4640cf290 msgr2=0x7fd4640cf720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:13.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:13.855+0000 7fd46c6d4640 1 --2- 172.21.15.92:0/2488083593 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd4640cf290 0x7fd4640cf720 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fd4640cfea0 tx=0x7fd45c02bf50 comp rx=0 tx=0).stop 2023-11-24T06:13:13.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:13.856+0000 7fd46c6d4640 1 -- 172.21.15.92:0/2488083593 shutdown_connections 2023-11-24T06:13:13.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:13.856+0000 7fd46c6d4640 1 --2- 172.21.15.92:0/2488083593 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd4640cf290 0x7fd4640cf720 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:13.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:13.856+0000 7fd46c6d4640 1 --2- 172.21.15.92:0/2488083593 >> v2:172.21.15.118:3300/0 conn(0x7fd4640b3ae0 0x7fd46415e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:13.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:13.856+0000 7fd46c6d4640 1 -- 172.21.15.92:0/2488083593 >> 172.21.15.92:0/2488083593 conn(0x7fd46406a800 msgr2=0x7fd46406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:13:13.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:13.856+0000 7fd46c6d4640 1 -- 172.21.15.92:0/2488083593 shutdown_connections 2023-11-24T06:13:13.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:13.856+0000 7fd46c6d4640 1 -- 172.21.15.92:0/2488083593 wait complete. 2023-11-24T06:13:13.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:13.857+0000 7fd46c6d4640 1 Processor -- start 2023-11-24T06:13:13.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:13.858+0000 7fd46c6d4640 1 -- start start 2023-11-24T06:13:13.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:13.859+0000 7fd46c6d4640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd4640ae5a0 0x7fd46414ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:13.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:13.859+0000 7fd46c6d4640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd4640b3ae0 0x7fd4640b1ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:13.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:13.859+0000 7fd46c6d4640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd4640cf290 0x7fd4640b21e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:13.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:13.859+0000 7fd46c6d4640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4641570f0 con 0x7fd4640ae5a0 2023-11-24T06:13:13.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:13.859+0000 7fd46c6d4640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd464152450 con 0x7fd4640cf290 2023-11-24T06:13:13.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:13.859+0000 7fd46c6d4640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd464152730 con 0x7fd4640b3ae0 2023-11-24T06:13:13.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:13.859+0000 7fd46a470640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd4640ae5a0 0x7fd46414ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:13.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:13.859+0000 7fd46a470640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd4640ae5a0 0x7fd46414ec90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52796/0 (socket says 172.21.15.92:52796) 2023-11-24T06:13:13.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:13.859+0000 7fd46a470640 1 -- 172.21.15.92:0/2965433755 learned_addr learned my addr 172.21.15.92:0/2965433755 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:13:13.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:13.859+0000 7fd469c6f640 1 --2- 172.21.15.92:0/2965433755 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd4640b3ae0 0x7fd4640b1ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:13.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:13.859+0000 7fd46a470640 1 -- 172.21.15.92:0/2965433755 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd4640b3ae0 msgr2=0x7fd4640b1ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:13.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:13.859+0000 7fd46a470640 1 --2- 172.21.15.92:0/2965433755 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd4640b3ae0 0x7fd4640b1ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:13.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:13.859+0000 7fd46ac71640 1 --2- 172.21.15.92:0/2965433755 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd4640cf290 0x7fd4640b21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:13.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:13.859+0000 7fd46a470640 1 -- 172.21.15.92:0/2965433755 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd4640cf290 msgr2=0x7fd4640b21e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:13.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:13.859+0000 7fd46a470640 1 --2- 172.21.15.92:0/2965433755 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd4640cf290 0x7fd4640b21e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:13.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:13.859+0000 7fd46a470640 1 -- 172.21.15.92:0/2965433755 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd45c009a70 con 0x7fd4640ae5a0 2023-11-24T06:13:13.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:13.860+0000 7fd46a470640 1 --2- 172.21.15.92:0/2965433755 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd4640ae5a0 0x7fd46414ec90 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7fd46000c4d0 tx=0x7fd46000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:13.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:13.860+0000 7fd45affd640 1 -- 172.21.15.92:0/2965433755 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4600074e0 con 0x7fd4640ae5a0 2023-11-24T06:13:13.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:13.860+0000 7fd45affd640 1 -- 172.21.15.92:0/2965433755 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd460007660 con 0x7fd4640ae5a0 2023-11-24T06:13:13.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:13.860+0000 7fd46c6d4640 1 -- 172.21.15.92:0/2965433755 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd464152a10 con 0x7fd4640ae5a0 2023-11-24T06:13:13.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:13.860+0000 7fd45affd640 1 -- 172.21.15.92:0/2965433755 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd460004a20 con 0x7fd4640ae5a0 2023-11-24T06:13:13.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:13.860+0000 7fd46c6d4640 1 -- 172.21.15.92:0/2965433755 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd464152fd0 con 0x7fd4640ae5a0 2023-11-24T06:13:13.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:13.861+0000 7fd46c6d4640 1 -- 172.21.15.92:0/2965433755 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4640675d0 con 0x7fd4640ae5a0 2023-11-24T06:13:13.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:13.862+0000 7fd45affd640 1 -- 172.21.15.92:0/2965433755 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7fd460004b80 con 0x7fd4640ae5a0 2023-11-24T06:13:13.865 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:13.863+0000 7fd45affd640 1 --2- 172.21.15.92:0/2965433755 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fd44806aed0 0x7fd44806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:13.865 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:13.863+0000 7fd45affd640 1 -- 172.21.15.92:0/2965433755 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd460088b70 con 0x7fd4640ae5a0 2023-11-24T06:13:13.865 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:13.863+0000 7fd469c6f640 1 --2- 172.21.15.92:0/2965433755 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fd44806aed0 0x7fd44806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:13.866 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:13.864+0000 7fd469c6f640 1 --2- 172.21.15.92:0/2965433755 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fd44806aed0 0x7fd44806d390 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fd454006fd0 tx=0x7fd454008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:13.869 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:13.868+0000 7fd45affd640 1 -- 172.21.15.92:0/2965433755 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd460053c30 con 0x7fd4640ae5a0 2023-11-24T06:13:14.056 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:14.054+0000 7fd46c6d4640 1 -- 172.21.15.92:0/2965433755 --> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd464071170 con 0x7fd44806aed0 2023-11-24T06:13:14.065 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:14.063+0000 7fd45affd640 1 -- 172.21.15.92:0/2965433755 <== mgr.4101 v2:172.21.15.92:6800/784027401 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+287306 (secure 0 0 0) 0x7fd454009d60 con 0x7fd44806aed0 2023-11-24T06:13:14.069 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-24T06:13:14.071 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2023-11-24T06:13:14.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:14.069+0000 7fd46c6d4640 1 -- 172.21.15.92:0/2965433755 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fd44806aed0 msgr2=0x7fd44806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:14.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:14.069+0000 7fd46c6d4640 1 --2- 172.21.15.92:0/2965433755 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fd44806aed0 0x7fd44806d390 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fd454006fd0 tx=0x7fd454008040 comp rx=0 tx=0).stop 2023-11-24T06:13:14.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:14.069+0000 7fd46c6d4640 1 -- 172.21.15.92:0/2965433755 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd4640ae5a0 msgr2=0x7fd46414ec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:14.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:14.069+0000 7fd46c6d4640 1 --2- 172.21.15.92:0/2965433755 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd4640ae5a0 0x7fd46414ec90 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7fd46000c4d0 tx=0x7fd46000c990 comp rx=0 tx=0).stop 2023-11-24T06:13:14.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:14.069+0000 7fd46c6d4640 1 -- 172.21.15.92:0/2965433755 shutdown_connections 2023-11-24T06:13:14.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:14.069+0000 7fd46c6d4640 1 --2- 172.21.15.92:0/2965433755 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fd44806aed0 0x7fd44806d390 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:14.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:14.069+0000 7fd46c6d4640 1 --2- 172.21.15.92:0/2965433755 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd4640cf290 0x7fd4640b21e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:14.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:14.069+0000 7fd46c6d4640 1 --2- 172.21.15.92:0/2965433755 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd4640b3ae0 0x7fd4640b1ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:14.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:14.069+0000 7fd46c6d4640 1 --2- 172.21.15.92:0/2965433755 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd4640ae5a0 0x7fd46414ec90 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:14.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:14.069+0000 7fd46c6d4640 1 -- 172.21.15.92:0/2965433755 >> 172.21.15.92:0/2965433755 conn(0x7fd46406a800 msgr2=0x7fd4640b29f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:13:14.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:14.069+0000 7fd46c6d4640 1 -- 172.21.15.92:0/2965433755 shutdown_connections 2023-11-24T06:13:14.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:14.070+0000 7fd46c6d4640 1 -- 172.21.15.92:0/2965433755 wait complete. 2023-11-24T06:13:14.092 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":144,"stamp":"2023-11-24T06:13:13.716938+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":170,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":244184,"kb_used_data":7200,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754730536,"statfs":{"total":773094113280,"available":772844068864,"internally_reserved":0,"allocated":7372800,"data_stored":3810304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":242535985},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":41,"apply_latency_ms":41,"commit_latency_ns":41000000,"apply_latency_ns":41000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001917"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-24T06:13:10.170940+0000","last_change":"2023-11-24T06:13:09.170059+0000","last_active":"2023-11-24T06:13:10.170940+0000","last_peered":"2023-11-24T06:13:10.170940+0000","last_clean":"2023-11-24T06:13:10.170940+0000","last_became_active":"2023-11-24T06:13:09.169330+0000","last_became_peered":"2023-11-24T06:13:09.169330+0000","last_unstale":"2023-11-24T06:13:10.170940+0000","last_undegraded":"2023-11-24T06:13:10.170940+0000","last_fullsized":"2023-11-24T06:13:10.170940+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_clean_scrub_stamp":"2023-11-24T06:13:08.130949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:11:17.683821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-24T06:13:10.173065+0000","last_change":"2023-11-24T06:13:09.177337+0000","last_active":"2023-11-24T06:13:10.173065+0000","last_peered":"2023-11-24T06:13:10.173065+0000","last_clean":"2023-11-24T06:13:10.173065+0000","last_became_active":"2023-11-24T06:13:09.175034+0000","last_became_peered":"2023-11-24T06:13:09.175034+0000","last_unstale":"2023-11-24T06:13:10.173065+0000","last_undegraded":"2023-11-24T06:13:10.173065+0000","last_fullsized":"2023-11-24T06:13:10.173065+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_clean_scrub_stamp":"2023-11-24T06:13:08.130949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:25:13.466132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-24T06:13:10.622257+0000","last_change":"2023-11-24T06:13:09.174821+0000","last_active":"2023-11-24T06:13:10.622257+0000","last_peered":"2023-11-24T06:13:10.622257+0000","last_clean":"2023-11-24T06:13:10.622257+0000","last_became_active":"2023-11-24T06:13:09.174512+0000","last_became_peered":"2023-11-24T06:13:09.174512+0000","last_unstale":"2023-11-24T06:13:10.622257+0000","last_undegraded":"2023-11-24T06:13:10.622257+0000","last_fullsized":"2023-11-24T06:13:10.622257+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_clean_scrub_stamp":"2023-11-24T06:13:08.130949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:04:50.089652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-24T06:13:10.808385+0000","last_change":"2023-11-24T06:13:09.183014+0000","last_active":"2023-11-24T06:13:10.808385+0000","last_peered":"2023-11-24T06:13:10.808385+0000","last_clean":"2023-11-24T06:13:10.808385+0000","last_became_active":"2023-11-24T06:13:09.182715+0000","last_became_peered":"2023-11-24T06:13:09.182715+0000","last_unstale":"2023-11-24T06:13:10.808385+0000","last_undegraded":"2023-11-24T06:13:10.808385+0000","last_fullsized":"2023-11-24T06:13:10.808385+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_clean_scrub_stamp":"2023-11-24T06:13:08.130949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:02:33.773714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-24T06:13:11.383067+0000","last_change":"2023-11-24T06:13:09.176388+0000","last_active":"2023-11-24T06:13:11.383067+0000","last_peered":"2023-11-24T06:13:11.383067+0000","last_clean":"2023-11-24T06:13:11.383067+0000","last_became_active":"2023-11-24T06:13:09.176121+0000","last_became_peered":"2023-11-24T06:13:09.176121+0000","last_unstale":"2023-11-24T06:13:11.383067+0000","last_undegraded":"2023-11-24T06:13:11.383067+0000","last_fullsized":"2023-11-24T06:13:11.383067+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_clean_scrub_stamp":"2023-11-24T06:13:08.130949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:35:10.782571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-24T06:13:10.171605+0000","last_change":"2023-11-24T06:13:09.176367+0000","last_active":"2023-11-24T06:13:10.171605+0000","last_peered":"2023-11-24T06:13:10.171605+0000","last_clean":"2023-11-24T06:13:10.171605+0000","last_became_active":"2023-11-24T06:13:09.176054+0000","last_became_peered":"2023-11-24T06:13:09.176054+0000","last_unstale":"2023-11-24T06:13:10.171605+0000","last_undegraded":"2023-11-24T06:13:10.171605+0000","last_fullsized":"2023-11-24T06:13:10.171605+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_clean_scrub_stamp":"2023-11-24T06:13:08.130949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:25:45.781079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-24T06:13:10.167450+0000","last_change":"2023-11-24T06:13:09.167026+0000","last_active":"2023-11-24T06:13:10.167450+0000","last_peered":"2023-11-24T06:13:10.167450+0000","last_clean":"2023-11-24T06:13:10.167450+0000","last_became_active":"2023-11-24T06:13:09.166754+0000","last_became_peered":"2023-11-24T06:13:09.166754+0000","last_unstale":"2023-11-24T06:13:10.167450+0000","last_undegraded":"2023-11-24T06:13:10.167450+0000","last_fullsized":"2023-11-24T06:13:10.167450+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_clean_scrub_stamp":"2023-11-24T06:13:08.130949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:32:54.676733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-24T06:13:10.170259+0000","last_change":"2023-11-24T06:13:09.170603+0000","last_active":"2023-11-24T06:13:10.170259+0000","last_peered":"2023-11-24T06:13:10.170259+0000","last_clean":"2023-11-24T06:13:10.170259+0000","last_became_active":"2023-11-24T06:13:09.170286+0000","last_became_peered":"2023-11-24T06:13:09.170286+0000","last_unstale":"2023-11-24T06:13:10.170259+0000","last_undegraded":"2023-11-24T06:13:10.170259+0000","last_fullsized":"2023-11-24T06:13:10.170259+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_clean_scrub_stamp":"2023-11-24T06:13:08.130949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:32:45.712807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-24T06:13:11.383583+0000","last_change":"2023-11-24T06:13:09.177670+0000","last_active":"2023-11-24T06:13:11.383583+0000","last_peered":"2023-11-24T06:13:11.383583+0000","last_clean":"2023-11-24T06:13:11.383583+0000","last_became_active":"2023-11-24T06:13:09.177413+0000","last_became_peered":"2023-11-24T06:13:09.177413+0000","last_unstale":"2023-11-24T06:13:11.383583+0000","last_undegraded":"2023-11-24T06:13:11.383583+0000","last_fullsized":"2023-11-24T06:13:11.383583+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_clean_scrub_stamp":"2023-11-24T06:13:08.130949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:21:21.330034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-24T06:13:11.383892+0000","last_change":"2023-11-24T06:13:09.180255+0000","last_active":"2023-11-24T06:13:11.383892+0000","last_peered":"2023-11-24T06:13:11.383892+0000","last_clean":"2023-11-24T06:13:11.383892+0000","last_became_active":"2023-11-24T06:13:09.179676+0000","last_became_peered":"2023-11-24T06:13:09.179676+0000","last_unstale":"2023-11-24T06:13:11.383892+0000","last_undegraded":"2023-11-24T06:13:11.383892+0000","last_fullsized":"2023-11-24T06:13:11.383892+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_clean_scrub_stamp":"2023-11-24T06:13:08.130949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:12:07.070925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-24T06:13:10.171418+0000","last_change":"2023-11-24T06:13:09.178014+0000","last_active":"2023-11-24T06:13:10.171418+0000","last_peered":"2023-11-24T06:13:10.171418+0000","last_clean":"2023-11-24T06:13:10.171418+0000","last_became_active":"2023-11-24T06:13:09.177543+0000","last_became_peered":"2023-11-24T06:13:09.177543+0000","last_unstale":"2023-11-24T06:13:10.171418+0000","last_undegraded":"2023-11-24T06:13:10.171418+0000","last_fullsized":"2023-11-24T06:13:10.171418+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_clean_scrub_stamp":"2023-11-24T06:13:08.130949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:46:04.603891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-24T06:13:10.808505+0000","last_change":"2023-11-24T06:13:09.173327+0000","last_active":"2023-11-24T06:13:10.808505+0000","last_peered":"2023-11-24T06:13:10.808505+0000","last_clean":"2023-11-24T06:13:10.808505+0000","last_became_active":"2023-11-24T06:13:09.173004+0000","last_became_peered":"2023-11-24T06:13:09.173004+0000","last_unstale":"2023-11-24T06:13:10.808505+0000","last_undegraded":"2023-11-24T06:13:10.808505+0000","last_fullsized":"2023-11-24T06:13:10.808505+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_clean_scrub_stamp":"2023-11-24T06:13:08.130949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:46:42.614245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-24T06:13:11.384736+0000","last_change":"2023-11-24T06:13:09.174064+0000","last_active":"2023-11-24T06:13:11.384736+0000","last_peered":"2023-11-24T06:13:11.384736+0000","last_clean":"2023-11-24T06:13:11.384736+0000","last_became_active":"2023-11-24T06:13:09.173805+0000","last_became_peered":"2023-11-24T06:13:09.173805+0000","last_unstale":"2023-11-24T06:13:11.384736+0000","last_undegraded":"2023-11-24T06:13:11.384736+0000","last_fullsized":"2023-11-24T06:13:11.384736+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_clean_scrub_stamp":"2023-11-24T06:13:08.130949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:11:02.449120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-24T06:13:11.382995+0000","last_change":"2023-11-24T06:13:09.173084+0000","last_active":"2023-11-24T06:13:11.382995+0000","last_peered":"2023-11-24T06:13:11.382995+0000","last_clean":"2023-11-24T06:13:11.382995+0000","last_became_active":"2023-11-24T06:13:09.166799+0000","last_became_peered":"2023-11-24T06:13:09.166799+0000","last_unstale":"2023-11-24T06:13:11.382995+0000","last_undegraded":"2023-11-24T06:13:11.382995+0000","last_fullsized":"2023-11-24T06:13:11.382995+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_clean_scrub_stamp":"2023-11-24T06:13:08.130949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:41:38.528990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-24T06:13:11.383108+0000","last_change":"2023-11-24T06:13:09.180093+0000","last_active":"2023-11-24T06:13:11.383108+0000","last_peered":"2023-11-24T06:13:11.383108+0000","last_clean":"2023-11-24T06:13:11.383108+0000","last_became_active":"2023-11-24T06:13:09.179736+0000","last_became_peered":"2023-11-24T06:13:09.179736+0000","last_unstale":"2023-11-24T06:13:11.383108+0000","last_undegraded":"2023-11-24T06:13:11.383108+0000","last_fullsized":"2023-11-24T06:13:11.383108+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_clean_scrub_stamp":"2023-11-24T06:13:08.130949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:10:28.958299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-24T06:13:11.382756+0000","last_change":"2023-11-24T06:13:09.180094+0000","last_active":"2023-11-24T06:13:11.382756+0000","last_peered":"2023-11-24T06:13:11.382756+0000","last_clean":"2023-11-24T06:13:11.382756+0000","last_became_active":"2023-11-24T06:13:09.179320+0000","last_became_peered":"2023-11-24T06:13:09.179320+0000","last_unstale":"2023-11-24T06:13:11.382756+0000","last_undegraded":"2023-11-24T06:13:11.382756+0000","last_fullsized":"2023-11-24T06:13:11.382756+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_clean_scrub_stamp":"2023-11-24T06:13:08.130949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:20:27.097587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-24T06:13:11.384044+0000","last_change":"2023-11-24T06:13:09.177601+0000","last_active":"2023-11-24T06:13:11.384044+0000","last_peered":"2023-11-24T06:13:11.384044+0000","last_clean":"2023-11-24T06:13:11.384044+0000","last_became_active":"2023-11-24T06:13:09.176914+0000","last_became_peered":"2023-11-24T06:13:09.176914+0000","last_unstale":"2023-11-24T06:13:11.384044+0000","last_undegraded":"2023-11-24T06:13:11.384044+0000","last_fullsized":"2023-11-24T06:13:11.384044+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_clean_scrub_stamp":"2023-11-24T06:13:08.130949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:15:34.858677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-24T06:13:10.808739+0000","last_change":"2023-11-24T06:13:09.173954+0000","last_active":"2023-11-24T06:13:10.808739+0000","last_peered":"2023-11-24T06:13:10.808739+0000","last_clean":"2023-11-24T06:13:10.808739+0000","last_became_active":"2023-11-24T06:13:09.173635+0000","last_became_peered":"2023-11-24T06:13:09.173635+0000","last_unstale":"2023-11-24T06:13:10.808739+0000","last_undegraded":"2023-11-24T06:13:10.808739+0000","last_fullsized":"2023-11-24T06:13:10.808739+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_clean_scrub_stamp":"2023-11-24T06:13:08.130949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:56:46.215502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-24T06:13:10.171868+0000","last_change":"2023-11-24T06:13:09.176391+0000","last_active":"2023-11-24T06:13:10.171868+0000","last_peered":"2023-11-24T06:13:10.171868+0000","last_clean":"2023-11-24T06:13:10.171868+0000","last_became_active":"2023-11-24T06:13:09.176040+0000","last_became_peered":"2023-11-24T06:13:09.176040+0000","last_unstale":"2023-11-24T06:13:10.171868+0000","last_undegraded":"2023-11-24T06:13:10.171868+0000","last_fullsized":"2023-11-24T06:13:10.171868+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_clean_scrub_stamp":"2023-11-24T06:13:08.130949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:40:58.911893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-24T06:13:11.384767+0000","last_change":"2023-11-24T06:13:09.165897+0000","last_active":"2023-11-24T06:13:11.384767+0000","last_peered":"2023-11-24T06:13:11.384767+0000","last_clean":"2023-11-24T06:13:11.384767+0000","last_became_active":"2023-11-24T06:13:09.165508+0000","last_became_peered":"2023-11-24T06:13:09.165508+0000","last_unstale":"2023-11-24T06:13:11.384767+0000","last_undegraded":"2023-11-24T06:13:11.384767+0000","last_fullsized":"2023-11-24T06:13:11.384767+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_clean_scrub_stamp":"2023-11-24T06:13:08.130949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:41:23.231120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-24T06:13:11.383191+0000","last_change":"2023-11-24T06:13:09.168339+0000","last_active":"2023-11-24T06:13:11.383191+0000","last_peered":"2023-11-24T06:13:11.383191+0000","last_clean":"2023-11-24T06:13:11.383191+0000","last_became_active":"2023-11-24T06:13:09.167374+0000","last_became_peered":"2023-11-24T06:13:09.167374+0000","last_unstale":"2023-11-24T06:13:11.383191+0000","last_undegraded":"2023-11-24T06:13:11.383191+0000","last_fullsized":"2023-11-24T06:13:11.383191+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_clean_scrub_stamp":"2023-11-24T06:13:08.130949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:26:17.968789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-24T06:13:11.383166+0000","last_change":"2023-11-24T06:13:09.183136+0000","last_active":"2023-11-24T06:13:11.383166+0000","last_peered":"2023-11-24T06:13:11.383166+0000","last_clean":"2023-11-24T06:13:11.383166+0000","last_became_active":"2023-11-24T06:13:09.182859+0000","last_became_peered":"2023-11-24T06:13:09.182859+0000","last_unstale":"2023-11-24T06:13:11.383166+0000","last_undegraded":"2023-11-24T06:13:11.383166+0000","last_fullsized":"2023-11-24T06:13:11.383166+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_clean_scrub_stamp":"2023-11-24T06:13:08.130949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:56:46.215502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-24T06:13:10.170846+0000","last_change":"2023-11-24T06:13:09.165340+0000","last_active":"2023-11-24T06:13:10.170846+0000","last_peered":"2023-11-24T06:13:10.170846+0000","last_clean":"2023-11-24T06:13:10.170846+0000","last_became_active":"2023-11-24T06:13:09.165011+0000","last_became_peered":"2023-11-24T06:13:09.165011+0000","last_unstale":"2023-11-24T06:13:10.170846+0000","last_undegraded":"2023-11-24T06:13:10.170846+0000","last_fullsized":"2023-11-24T06:13:10.170846+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_clean_scrub_stamp":"2023-11-24T06:13:08.130949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:40:30.107261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-24T06:13:10.169981+0000","last_change":"2023-11-24T06:13:09.174338+0000","last_active":"2023-11-24T06:13:10.169981+0000","last_peered":"2023-11-24T06:13:10.169981+0000","last_clean":"2023-11-24T06:13:10.169981+0000","last_became_active":"2023-11-24T06:13:09.169772+0000","last_became_peered":"2023-11-24T06:13:09.169772+0000","last_unstale":"2023-11-24T06:13:10.169981+0000","last_undegraded":"2023-11-24T06:13:10.169981+0000","last_fullsized":"2023-11-24T06:13:10.169981+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_clean_scrub_stamp":"2023-11-24T06:13:08.130949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:03:58.717203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-24T06:13:11.383628+0000","last_change":"2023-11-24T06:13:09.181617+0000","last_active":"2023-11-24T06:13:11.383628+0000","last_peered":"2023-11-24T06:13:11.383628+0000","last_clean":"2023-11-24T06:13:11.383628+0000","last_became_active":"2023-11-24T06:13:09.181331+0000","last_became_peered":"2023-11-24T06:13:09.181331+0000","last_unstale":"2023-11-24T06:13:11.383628+0000","last_undegraded":"2023-11-24T06:13:11.383628+0000","last_fullsized":"2023-11-24T06:13:11.383628+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_clean_scrub_stamp":"2023-11-24T06:13:08.130949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:13:26.587683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-24T06:13:10.167605+0000","last_change":"2023-11-24T06:13:09.170539+0000","last_active":"2023-11-24T06:13:10.167605+0000","last_peered":"2023-11-24T06:13:10.167605+0000","last_clean":"2023-11-24T06:13:10.167605+0000","last_became_active":"2023-11-24T06:13:09.170106+0000","last_became_peered":"2023-11-24T06:13:09.170106+0000","last_unstale":"2023-11-24T06:13:10.167605+0000","last_undegraded":"2023-11-24T06:13:10.167605+0000","last_fullsized":"2023-11-24T06:13:10.167605+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_clean_scrub_stamp":"2023-11-24T06:13:08.130949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:23:34.210151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-24T06:13:10.622467+0000","last_change":"2023-11-24T06:13:09.175903+0000","last_active":"2023-11-24T06:13:10.622467+0000","last_peered":"2023-11-24T06:13:10.622467+0000","last_clean":"2023-11-24T06:13:10.622467+0000","last_became_active":"2023-11-24T06:13:09.173897+0000","last_became_peered":"2023-11-24T06:13:09.173897+0000","last_unstale":"2023-11-24T06:13:10.622467+0000","last_undegraded":"2023-11-24T06:13:10.622467+0000","last_fullsized":"2023-11-24T06:13:10.622467+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_clean_scrub_stamp":"2023-11-24T06:13:08.130949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:13:46.341162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-24T06:13:11.383472+0000","last_change":"2023-11-24T06:13:09.180694+0000","last_active":"2023-11-24T06:13:11.383472+0000","last_peered":"2023-11-24T06:13:11.383472+0000","last_clean":"2023-11-24T06:13:11.383472+0000","last_became_active":"2023-11-24T06:13:09.179929+0000","last_became_peered":"2023-11-24T06:13:09.179929+0000","last_unstale":"2023-11-24T06:13:11.383472+0000","last_undegraded":"2023-11-24T06:13:11.383472+0000","last_fullsized":"2023-11-24T06:13:11.383472+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_clean_scrub_stamp":"2023-11-24T06:13:08.130949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:10:20.400239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-24T06:13:10.621989+0000","last_change":"2023-11-24T06:13:09.176369+0000","last_active":"2023-11-24T06:13:10.621989+0000","last_peered":"2023-11-24T06:13:10.621989+0000","last_clean":"2023-11-24T06:13:10.621989+0000","last_became_active":"2023-11-24T06:13:09.174320+0000","last_became_peered":"2023-11-24T06:13:09.174320+0000","last_unstale":"2023-11-24T06:13:10.621989+0000","last_undegraded":"2023-11-24T06:13:10.621989+0000","last_fullsized":"2023-11-24T06:13:10.621989+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_clean_scrub_stamp":"2023-11-24T06:13:08.130949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:56:15.626987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-24T06:13:10.167142+0000","last_change":"2023-11-24T06:13:09.164259+0000","last_active":"2023-11-24T06:13:10.167142+0000","last_peered":"2023-11-24T06:13:10.167142+0000","last_clean":"2023-11-24T06:13:10.167142+0000","last_became_active":"2023-11-24T06:13:09.163987+0000","last_became_peered":"2023-11-24T06:13:09.163987+0000","last_unstale":"2023-11-24T06:13:10.167142+0000","last_undegraded":"2023-11-24T06:13:10.167142+0000","last_fullsized":"2023-11-24T06:13:10.167142+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_clean_scrub_stamp":"2023-11-24T06:13:08.130949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:46:17.818773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-24T06:13:10.168035+0000","last_change":"2023-11-24T06:13:09.175229+0000","last_active":"2023-11-24T06:13:10.168035+0000","last_peered":"2023-11-24T06:13:10.168035+0000","last_clean":"2023-11-24T06:13:10.168035+0000","last_became_active":"2023-11-24T06:13:09.174953+0000","last_became_peered":"2023-11-24T06:13:09.174953+0000","last_unstale":"2023-11-24T06:13:10.168035+0000","last_undegraded":"2023-11-24T06:13:10.168035+0000","last_fullsized":"2023-11-24T06:13:10.168035+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_clean_scrub_stamp":"2023-11-24T06:13:08.130949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:13:27.518884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-24T06:13:10.170624+0000","last_change":"2023-11-24T06:13:09.178297+0000","last_active":"2023-11-24T06:13:10.170624+0000","last_peered":"2023-11-24T06:13:10.170624+0000","last_clean":"2023-11-24T06:13:10.170624+0000","last_became_active":"2023-11-24T06:13:09.177819+0000","last_became_peered":"2023-11-24T06:13:09.177819+0000","last_unstale":"2023-11-24T06:13:10.170624+0000","last_undegraded":"2023-11-24T06:13:10.170624+0000","last_fullsized":"2023-11-24T06:13:10.170624+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_clean_scrub_stamp":"2023-11-24T06:13:08.130949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:42:40.180587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-24T06:13:10.170780+0000","last_change":"2023-11-24T06:13:09.179158+0000","last_active":"2023-11-24T06:13:10.170780+0000","last_peered":"2023-11-24T06:13:10.170780+0000","last_clean":"2023-11-24T06:13:10.170780+0000","last_became_active":"2023-11-24T06:13:09.178863+0000","last_became_peered":"2023-11-24T06:13:09.178863+0000","last_unstale":"2023-11-24T06:13:10.170780+0000","last_undegraded":"2023-11-24T06:13:10.170780+0000","last_fullsized":"2023-11-24T06:13:10.170780+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_clean_scrub_stamp":"2023-11-24T06:13:08.130949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:02:08.062623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"62'4","reported_seq":16,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-24T06:13:09.194350+0000","last_change":"2023-11-24T06:13:08.187895+0000","last_active":"2023-11-24T06:13:09.194350+0000","last_peered":"2023-11-24T06:13:09.194350+0000","last_clean":"2023-11-24T06:13:09.194350+0000","last_became_active":"2023-11-24T06:13:08.187548+0000","last_became_peered":"2023-11-24T06:13:08.187548+0000","last_unstale":"2023-11-24T06:13:09.194350+0000","last_undegraded":"2023-11-24T06:13:09.194350+0000","last_fullsized":"2023-11-24T06:13:09.194350+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:13:07.116698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:13:07.116698+0000","last_clean_scrub_stamp":"2023-11-24T06:13:07.116698+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:12:41.405587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-24T06:13:11.384354+0000","last_change":"2023-11-24T06:13:09.187107+0000","last_active":"2023-11-24T06:13:11.384354+0000","last_peered":"2023-11-24T06:13:11.384354+0000","last_clean":"2023-11-24T06:13:11.384354+0000","last_became_active":"2023-11-24T06:13:09.186790+0000","last_became_peered":"2023-11-24T06:13:09.186790+0000","last_unstale":"2023-11-24T06:13:11.384354+0000","last_undegraded":"2023-11-24T06:13:11.384354+0000","last_fullsized":"2023-11-24T06:13:11.384354+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_clean_scrub_stamp":"2023-11-24T06:13:08.130949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:35:31.662234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"62'1","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-24T06:13:09.188743+0000","last_change":"2023-11-24T06:13:08.177698+0000","last_active":"2023-11-24T06:13:09.188743+0000","last_peered":"2023-11-24T06:13:09.188743+0000","last_clean":"2023-11-24T06:13:09.188743+0000","last_became_active":"2023-11-24T06:13:08.176542+0000","last_became_peered":"2023-11-24T06:13:08.176542+0000","last_unstale":"2023-11-24T06:13:09.188743+0000","last_undegraded":"2023-11-24T06:13:09.188743+0000","last_fullsized":"2023-11-24T06:13:09.188743+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:13:07.116698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:13:07.116698+0000","last_clean_scrub_stamp":"2023-11-24T06:13:07.116698+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:35:39.617906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-24T06:13:10.172224+0000","last_change":"2023-11-24T06:13:09.173733+0000","last_active":"2023-11-24T06:13:10.172224+0000","last_peered":"2023-11-24T06:13:10.172224+0000","last_clean":"2023-11-24T06:13:10.172224+0000","last_became_active":"2023-11-24T06:13:09.172987+0000","last_became_peered":"2023-11-24T06:13:09.172987+0000","last_unstale":"2023-11-24T06:13:10.172224+0000","last_undegraded":"2023-11-24T06:13:10.172224+0000","last_fullsized":"2023-11-24T06:13:10.172224+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_clean_scrub_stamp":"2023-11-24T06:13:08.130949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:28:12.508807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"62'2","reported_seq":14,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-24T06:13:09.191649+0000","last_change":"2023-11-24T06:13:08.180995+0000","last_active":"2023-11-24T06:13:09.191649+0000","last_peered":"2023-11-24T06:13:09.191649+0000","last_clean":"2023-11-24T06:13:09.191649+0000","last_became_active":"2023-11-24T06:13:08.180693+0000","last_became_peered":"2023-11-24T06:13:08.180693+0000","last_unstale":"2023-11-24T06:13:09.191649+0000","last_undegraded":"2023-11-24T06:13:09.191649+0000","last_fullsized":"2023-11-24T06:13:09.191649+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:13:07.116698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:13:07.116698+0000","last_clean_scrub_stamp":"2023-11-24T06:13:07.116698+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:28:11.494557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-24T06:13:10.171771+0000","last_change":"2023-11-24T06:13:09.169536+0000","last_active":"2023-11-24T06:13:10.171771+0000","last_peered":"2023-11-24T06:13:10.171771+0000","last_clean":"2023-11-24T06:13:10.171771+0000","last_became_active":"2023-11-24T06:13:09.169239+0000","last_became_peered":"2023-11-24T06:13:09.169239+0000","last_unstale":"2023-11-24T06:13:10.171771+0000","last_undegraded":"2023-11-24T06:13:10.171771+0000","last_fullsized":"2023-11-24T06:13:10.171771+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_clean_scrub_stamp":"2023-11-24T06:13:08.130949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:30:10.574156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"62'2","reported_seq":14,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-24T06:13:09.193249+0000","last_change":"2023-11-24T06:13:08.185564+0000","last_active":"2023-11-24T06:13:09.193249+0000","last_peered":"2023-11-24T06:13:09.193249+0000","last_clean":"2023-11-24T06:13:09.193249+0000","last_became_active":"2023-11-24T06:13:08.185232+0000","last_became_peered":"2023-11-24T06:13:08.185232+0000","last_unstale":"2023-11-24T06:13:09.193249+0000","last_undegraded":"2023-11-24T06:13:09.193249+0000","last_fullsized":"2023-11-24T06:13:09.193249+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:13:07.116698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:13:07.116698+0000","last_clean_scrub_stamp":"2023-11-24T06:13:07.116698+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:53:39.958849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-24T06:13:10.167125+0000","last_change":"2023-11-24T06:13:09.169373+0000","last_active":"2023-11-24T06:13:10.167125+0000","last_peered":"2023-11-24T06:13:10.167125+0000","last_clean":"2023-11-24T06:13:10.167125+0000","last_became_active":"2023-11-24T06:13:09.168836+0000","last_became_peered":"2023-11-24T06:13:09.168836+0000","last_unstale":"2023-11-24T06:13:10.167125+0000","last_undegraded":"2023-11-24T06:13:10.167125+0000","last_fullsized":"2023-11-24T06:13:10.167125+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_clean_scrub_stamp":"2023-11-24T06:13:08.130949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:12:07.070925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"62'1","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-24T06:13:09.187919+0000","last_change":"2023-11-24T06:13:08.173289+0000","last_active":"2023-11-24T06:13:09.187919+0000","last_peered":"2023-11-24T06:13:09.187919+0000","last_clean":"2023-11-24T06:13:09.187919+0000","last_became_active":"2023-11-24T06:13:08.172952+0000","last_became_peered":"2023-11-24T06:13:08.172952+0000","last_unstale":"2023-11-24T06:13:09.187919+0000","last_undegraded":"2023-11-24T06:13:09.187919+0000","last_fullsized":"2023-11-24T06:13:09.187919+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:13:07.116698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:13:07.116698+0000","last_clean_scrub_stamp":"2023-11-24T06:13:07.116698+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:08:28.938840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-24T06:13:11.383649+0000","last_change":"2023-11-24T06:13:09.172971+0000","last_active":"2023-11-24T06:13:11.383649+0000","last_peered":"2023-11-24T06:13:11.383649+0000","last_clean":"2023-11-24T06:13:11.383649+0000","last_became_active":"2023-11-24T06:13:09.172681+0000","last_became_peered":"2023-11-24T06:13:09.172681+0000","last_unstale":"2023-11-24T06:13:11.383649+0000","last_undegraded":"2023-11-24T06:13:11.383649+0000","last_fullsized":"2023-11-24T06:13:11.383649+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_clean_scrub_stamp":"2023-11-24T06:13:08.130949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:49:38.760024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"62'1","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-24T06:13:09.189696+0000","last_change":"2023-11-24T06:13:08.185077+0000","last_active":"2023-11-24T06:13:09.189696+0000","last_peered":"2023-11-24T06:13:09.189696+0000","last_clean":"2023-11-24T06:13:09.189696+0000","last_became_active":"2023-11-24T06:13:08.184786+0000","last_became_peered":"2023-11-24T06:13:08.184786+0000","last_unstale":"2023-11-24T06:13:09.189696+0000","last_undegraded":"2023-11-24T06:13:09.189696+0000","last_fullsized":"2023-11-24T06:13:09.189696+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:13:07.116698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:13:07.116698+0000","last_clean_scrub_stamp":"2023-11-24T06:13:07.116698+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:09:46.175447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-24T06:13:11.382407+0000","last_change":"2023-11-24T06:13:09.167517+0000","last_active":"2023-11-24T06:13:11.382407+0000","last_peered":"2023-11-24T06:13:11.382407+0000","last_clean":"2023-11-24T06:13:11.382407+0000","last_became_active":"2023-11-24T06:13:09.167237+0000","last_became_peered":"2023-11-24T06:13:09.167237+0000","last_unstale":"2023-11-24T06:13:11.382407+0000","last_undegraded":"2023-11-24T06:13:11.382407+0000","last_fullsized":"2023-11-24T06:13:11.382407+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_clean_scrub_stamp":"2023-11-24T06:13:08.130949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:57:07.655601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-24T06:13:09.164249+0000","last_change":"2023-11-24T06:13:08.172790+0000","last_active":"2023-11-24T06:13:09.164249+0000","last_peered":"2023-11-24T06:13:09.164249+0000","last_clean":"2023-11-24T06:13:09.164249+0000","last_became_active":"2023-11-24T06:13:08.172505+0000","last_became_peered":"2023-11-24T06:13:08.172505+0000","last_unstale":"2023-11-24T06:13:09.164249+0000","last_undegraded":"2023-11-24T06:13:09.164249+0000","last_fullsized":"2023-11-24T06:13:09.164249+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:13:07.116698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:13:07.116698+0000","last_clean_scrub_stamp":"2023-11-24T06:13:07.116698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:32:47.183642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-24T06:13:11.384354+0000","last_change":"2023-11-24T06:13:09.172260+0000","last_active":"2023-11-24T06:13:11.384354+0000","last_peered":"2023-11-24T06:13:11.384354+0000","last_clean":"2023-11-24T06:13:11.384354+0000","last_became_active":"2023-11-24T06:13:09.171961+0000","last_became_peered":"2023-11-24T06:13:09.171961+0000","last_unstale":"2023-11-24T06:13:11.384354+0000","last_undegraded":"2023-11-24T06:13:11.384354+0000","last_fullsized":"2023-11-24T06:13:11.384354+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_clean_scrub_stamp":"2023-11-24T06:13:08.130949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:30:42.992079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-24T06:13:09.157715+0000","last_change":"2023-11-24T06:13:08.178812+0000","last_active":"2023-11-24T06:13:09.157715+0000","last_peered":"2023-11-24T06:13:09.157715+0000","last_clean":"2023-11-24T06:13:09.157715+0000","last_became_active":"2023-11-24T06:13:08.178536+0000","last_became_peered":"2023-11-24T06:13:08.178536+0000","last_unstale":"2023-11-24T06:13:09.157715+0000","last_undegraded":"2023-11-24T06:13:09.157715+0000","last_fullsized":"2023-11-24T06:13:09.157715+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:13:07.116698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:13:07.116698+0000","last_clean_scrub_stamp":"2023-11-24T06:13:07.116698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:53:44.449347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-24T06:13:10.167949+0000","last_change":"2023-11-24T06:13:09.171551+0000","last_active":"2023-11-24T06:13:10.167949+0000","last_peered":"2023-11-24T06:13:10.167949+0000","last_clean":"2023-11-24T06:13:10.167949+0000","last_became_active":"2023-11-24T06:13:09.171258+0000","last_became_peered":"2023-11-24T06:13:09.171258+0000","last_unstale":"2023-11-24T06:13:10.167949+0000","last_undegraded":"2023-11-24T06:13:10.167949+0000","last_fullsized":"2023-11-24T06:13:10.167949+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_clean_scrub_stamp":"2023-11-24T06:13:08.130949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:57:59.107199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-24T06:13:09.167131+0000","last_change":"2023-11-24T06:13:08.168838+0000","last_active":"2023-11-24T06:13:09.167131+0000","last_peered":"2023-11-24T06:13:09.167131+0000","last_clean":"2023-11-24T06:13:09.167131+0000","last_became_active":"2023-11-24T06:13:08.168070+0000","last_became_peered":"2023-11-24T06:13:08.168070+0000","last_unstale":"2023-11-24T06:13:09.167131+0000","last_undegraded":"2023-11-24T06:13:09.167131+0000","last_fullsized":"2023-11-24T06:13:09.167131+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:13:07.116698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:13:07.116698+0000","last_clean_scrub_stamp":"2023-11-24T06:13:07.116698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:20:23.437716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-24T06:13:10.171055+0000","last_change":"2023-11-24T06:13:09.183094+0000","last_active":"2023-11-24T06:13:10.171055+0000","last_peered":"2023-11-24T06:13:10.171055+0000","last_clean":"2023-11-24T06:13:10.171055+0000","last_became_active":"2023-11-24T06:13:09.182814+0000","last_became_peered":"2023-11-24T06:13:09.182814+0000","last_unstale":"2023-11-24T06:13:10.171055+0000","last_undegraded":"2023-11-24T06:13:10.171055+0000","last_fullsized":"2023-11-24T06:13:10.171055+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_clean_scrub_stamp":"2023-11-24T06:13:08.130949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:45:46.147177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-24T06:13:09.172542+0000","last_change":"2023-11-24T06:13:08.179198+0000","last_active":"2023-11-24T06:13:09.172542+0000","last_peered":"2023-11-24T06:13:09.172542+0000","last_clean":"2023-11-24T06:13:09.172542+0000","last_became_active":"2023-11-24T06:13:08.178921+0000","last_became_peered":"2023-11-24T06:13:08.178921+0000","last_unstale":"2023-11-24T06:13:09.172542+0000","last_undegraded":"2023-11-24T06:13:09.172542+0000","last_fullsized":"2023-11-24T06:13:09.172542+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:13:07.116698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:13:07.116698+0000","last_clean_scrub_stamp":"2023-11-24T06:13:07.116698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:10:42.046934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-24T06:13:10.172579+0000","last_change":"2023-11-24T06:13:09.171935+0000","last_active":"2023-11-24T06:13:10.172579+0000","last_peered":"2023-11-24T06:13:10.172579+0000","last_clean":"2023-11-24T06:13:10.172579+0000","last_became_active":"2023-11-24T06:13:09.171663+0000","last_became_peered":"2023-11-24T06:13:09.171663+0000","last_unstale":"2023-11-24T06:13:10.172579+0000","last_undegraded":"2023-11-24T06:13:10.172579+0000","last_fullsized":"2023-11-24T06:13:10.172579+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_clean_scrub_stamp":"2023-11-24T06:13:08.130949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:32:48.197892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"62'3","reported_seq":15,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-24T06:13:09.189368+0000","last_change":"2023-11-24T06:13:08.178982+0000","last_active":"2023-11-24T06:13:09.189368+0000","last_peered":"2023-11-24T06:13:09.189368+0000","last_clean":"2023-11-24T06:13:09.189368+0000","last_became_active":"2023-11-24T06:13:08.178285+0000","last_became_peered":"2023-11-24T06:13:08.178285+0000","last_unstale":"2023-11-24T06:13:09.189368+0000","last_undegraded":"2023-11-24T06:13:09.189368+0000","last_fullsized":"2023-11-24T06:13:09.189368+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:13:07.116698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:13:07.116698+0000","last_clean_scrub_stamp":"2023-11-24T06:13:07.116698+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:11:16.669570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-24T06:13:11.383596+0000","last_change":"2023-11-24T06:13:09.175181+0000","last_active":"2023-11-24T06:13:11.383596+0000","last_peered":"2023-11-24T06:13:11.383596+0000","last_clean":"2023-11-24T06:13:11.383596+0000","last_became_active":"2023-11-24T06:13:09.174817+0000","last_became_peered":"2023-11-24T06:13:09.174817+0000","last_unstale":"2023-11-24T06:13:11.383596+0000","last_undegraded":"2023-11-24T06:13:11.383596+0000","last_fullsized":"2023-11-24T06:13:11.383596+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_clean_scrub_stamp":"2023-11-24T06:13:08.130949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:33:17.885209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"62'6","reported_seq":18,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-24T06:13:09.196603+0000","last_change":"2023-11-24T06:13:08.176659+0000","last_active":"2023-11-24T06:13:09.196603+0000","last_peered":"2023-11-24T06:13:09.196603+0000","last_clean":"2023-11-24T06:13:09.196603+0000","last_became_active":"2023-11-24T06:13:08.176329+0000","last_became_peered":"2023-11-24T06:13:08.176329+0000","last_unstale":"2023-11-24T06:13:09.196603+0000","last_undegraded":"2023-11-24T06:13:09.196603+0000","last_fullsized":"2023-11-24T06:13:09.196603+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:13:07.116698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:13:07.116698+0000","last_clean_scrub_stamp":"2023-11-24T06:13:07.116698+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:25:12.451881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-24T06:13:10.171233+0000","last_change":"2023-11-24T06:13:09.169731+0000","last_active":"2023-11-24T06:13:10.171233+0000","last_peered":"2023-11-24T06:13:10.171233+0000","last_clean":"2023-11-24T06:13:10.171233+0000","last_became_active":"2023-11-24T06:13:09.168200+0000","last_became_peered":"2023-11-24T06:13:09.168200+0000","last_unstale":"2023-11-24T06:13:10.171233+0000","last_undegraded":"2023-11-24T06:13:10.171233+0000","last_fullsized":"2023-11-24T06:13:10.171233+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_clean_scrub_stamp":"2023-11-24T06:13:08.130949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:53:39.945102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"62'2","reported_seq":14,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-24T06:13:09.193961+0000","last_change":"2023-11-24T06:13:08.179928+0000","last_active":"2023-11-24T06:13:09.193961+0000","last_peered":"2023-11-24T06:13:09.193961+0000","last_clean":"2023-11-24T06:13:09.193961+0000","last_became_active":"2023-11-24T06:13:08.179680+0000","last_became_peered":"2023-11-24T06:13:08.179680+0000","last_unstale":"2023-11-24T06:13:09.193961+0000","last_undegraded":"2023-11-24T06:13:09.193961+0000","last_fullsized":"2023-11-24T06:13:09.193961+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:13:07.116698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:13:07.116698+0000","last_clean_scrub_stamp":"2023-11-24T06:13:07.116698+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:02:48.920596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-24T06:13:10.167301+0000","last_change":"2023-11-24T06:13:09.174167+0000","last_active":"2023-11-24T06:13:10.167301+0000","last_peered":"2023-11-24T06:13:10.167301+0000","last_clean":"2023-11-24T06:13:10.167301+0000","last_became_active":"2023-11-24T06:13:09.173898+0000","last_became_peered":"2023-11-24T06:13:09.173898+0000","last_unstale":"2023-11-24T06:13:10.167301+0000","last_undegraded":"2023-11-24T06:13:10.167301+0000","last_fullsized":"2023-11-24T06:13:10.167301+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_clean_scrub_stamp":"2023-11-24T06:13:08.130949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:35:40.632156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-24T06:13:09.164466+0000","last_change":"2023-11-24T06:13:08.181547+0000","last_active":"2023-11-24T06:13:09.164466+0000","last_peered":"2023-11-24T06:13:09.164466+0000","last_clean":"2023-11-24T06:13:09.164466+0000","last_became_active":"2023-11-24T06:13:08.181231+0000","last_became_peered":"2023-11-24T06:13:08.181231+0000","last_unstale":"2023-11-24T06:13:09.164466+0000","last_undegraded":"2023-11-24T06:13:09.164466+0000","last_fullsized":"2023-11-24T06:13:09.164466+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:13:07.116698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:13:07.116698+0000","last_clean_scrub_stamp":"2023-11-24T06:13:07.116698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:47:53.944009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-24T06:13:11.382931+0000","last_change":"2023-11-24T06:13:09.180186+0000","last_active":"2023-11-24T06:13:11.382931+0000","last_peered":"2023-11-24T06:13:11.382931+0000","last_clean":"2023-11-24T06:13:11.382931+0000","last_became_active":"2023-11-24T06:13:09.179887+0000","last_became_peered":"2023-11-24T06:13:09.179887+0000","last_unstale":"2023-11-24T06:13:11.382931+0000","last_undegraded":"2023-11-24T06:13:11.382931+0000","last_fullsized":"2023-11-24T06:13:11.382931+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_clean_scrub_stamp":"2023-11-24T06:13:08.130949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:13:27.518884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-24T06:13:09.144680+0000","last_change":"2023-11-24T06:13:08.185975+0000","last_active":"2023-11-24T06:13:09.144680+0000","last_peered":"2023-11-24T06:13:09.144680+0000","last_clean":"2023-11-24T06:13:09.144680+0000","last_became_active":"2023-11-24T06:13:08.185655+0000","last_became_peered":"2023-11-24T06:13:08.185655+0000","last_unstale":"2023-11-24T06:13:09.144680+0000","last_undegraded":"2023-11-24T06:13:09.144680+0000","last_fullsized":"2023-11-24T06:13:09.144680+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:13:07.116698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:13:07.116698+0000","last_clean_scrub_stamp":"2023-11-24T06:13:07.116698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:20:23.437716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-24T06:13:11.383376+0000","last_change":"2023-11-24T06:13:09.177553+0000","last_active":"2023-11-24T06:13:11.383376+0000","last_peered":"2023-11-24T06:13:11.383376+0000","last_clean":"2023-11-24T06:13:11.383376+0000","last_became_active":"2023-11-24T06:13:09.177224+0000","last_became_peered":"2023-11-24T06:13:09.177224+0000","last_unstale":"2023-11-24T06:13:11.383376+0000","last_undegraded":"2023-11-24T06:13:11.383376+0000","last_fullsized":"2023-11-24T06:13:11.383376+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_clean_scrub_stamp":"2023-11-24T06:13:08.130949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:11:45.907786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"62'8","reported_seq":20,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-24T06:13:09.193083+0000","last_change":"2023-11-24T06:13:08.179177+0000","last_active":"2023-11-24T06:13:09.193083+0000","last_peered":"2023-11-24T06:13:09.193083+0000","last_clean":"2023-11-24T06:13:09.193083+0000","last_became_active":"2023-11-24T06:13:08.178866+0000","last_became_peered":"2023-11-24T06:13:08.178866+0000","last_unstale":"2023-11-24T06:13:09.193083+0000","last_undegraded":"2023-11-24T06:13:09.193083+0000","last_fullsized":"2023-11-24T06:13:09.193083+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:13:07.116698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:13:07.116698+0000","last_clean_scrub_stamp":"2023-11-24T06:13:07.116698+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:40:57.897642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-24T06:13:11.383837+0000","last_change":"2023-11-24T06:13:09.174579+0000","last_active":"2023-11-24T06:13:11.383837+0000","last_peered":"2023-11-24T06:13:11.383837+0000","last_clean":"2023-11-24T06:13:11.383837+0000","last_became_active":"2023-11-24T06:13:09.174302+0000","last_became_peered":"2023-11-24T06:13:09.174302+0000","last_unstale":"2023-11-24T06:13:11.383837+0000","last_undegraded":"2023-11-24T06:13:11.383837+0000","last_fullsized":"2023-11-24T06:13:11.383837+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_clean_scrub_stamp":"2023-11-24T06:13:08.130949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:00:53.804845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"62'1","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-24T06:13:09.191519+0000","last_change":"2023-11-24T06:13:08.182817+0000","last_active":"2023-11-24T06:13:09.191519+0000","last_peered":"2023-11-24T06:13:09.191519+0000","last_clean":"2023-11-24T06:13:09.191519+0000","last_became_active":"2023-11-24T06:13:08.182179+0000","last_became_peered":"2023-11-24T06:13:08.182179+0000","last_unstale":"2023-11-24T06:13:09.191519+0000","last_undegraded":"2023-11-24T06:13:09.191519+0000","last_fullsized":"2023-11-24T06:13:09.191519+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:13:07.116698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:13:07.116698+0000","last_clean_scrub_stamp":"2023-11-24T06:13:07.116698+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:00:01.864867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-24T06:13:10.171297+0000","last_change":"2023-11-24T06:13:09.175618+0000","last_active":"2023-11-24T06:13:10.171297+0000","last_peered":"2023-11-24T06:13:10.171297+0000","last_clean":"2023-11-24T06:13:10.171297+0000","last_became_active":"2023-11-24T06:13:09.175326+0000","last_became_peered":"2023-11-24T06:13:09.175326+0000","last_unstale":"2023-11-24T06:13:10.171297+0000","last_undegraded":"2023-11-24T06:13:10.171297+0000","last_fullsized":"2023-11-24T06:13:10.171297+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_clean_scrub_stamp":"2023-11-24T06:13:08.130949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:08:29.953091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"62'2","reported_seq":14,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-24T06:13:09.190743+0000","last_change":"2023-11-24T06:13:08.183605+0000","last_active":"2023-11-24T06:13:09.190743+0000","last_peered":"2023-11-24T06:13:09.190743+0000","last_clean":"2023-11-24T06:13:09.190743+0000","last_became_active":"2023-11-24T06:13:08.183306+0000","last_became_peered":"2023-11-24T06:13:08.183306+0000","last_unstale":"2023-11-24T06:13:09.190743+0000","last_undegraded":"2023-11-24T06:13:09.190743+0000","last_fullsized":"2023-11-24T06:13:09.190743+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:13:07.116698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:13:07.116698+0000","last_clean_scrub_stamp":"2023-11-24T06:13:07.116698+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:02:32.759463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-24T06:13:10.808739+0000","last_change":"2023-11-24T06:13:09.173297+0000","last_active":"2023-11-24T06:13:10.808739+0000","last_peered":"2023-11-24T06:13:10.808739+0000","last_clean":"2023-11-24T06:13:10.808739+0000","last_became_active":"2023-11-24T06:13:09.172964+0000","last_became_peered":"2023-11-24T06:13:09.172964+0000","last_unstale":"2023-11-24T06:13:10.808739+0000","last_undegraded":"2023-11-24T06:13:10.808739+0000","last_fullsized":"2023-11-24T06:13:10.808739+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_clean_scrub_stamp":"2023-11-24T06:13:08.130949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:20:27.097587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"62'1","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-24T06:13:09.189365+0000","last_change":"2023-11-24T06:13:08.183249+0000","last_active":"2023-11-24T06:13:09.189365+0000","last_peered":"2023-11-24T06:13:09.189365+0000","last_clean":"2023-11-24T06:13:09.189365+0000","last_became_active":"2023-11-24T06:13:08.182896+0000","last_became_peered":"2023-11-24T06:13:08.182896+0000","last_unstale":"2023-11-24T06:13:09.189365+0000","last_undegraded":"2023-11-24T06:13:09.189365+0000","last_fullsized":"2023-11-24T06:13:09.189365+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:13:07.116698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:13:07.116698+0000","last_clean_scrub_stamp":"2023-11-24T06:13:07.116698+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:35:25.591721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-24T06:13:11.384165+0000","last_change":"2023-11-24T06:13:09.177444+0000","last_active":"2023-11-24T06:13:11.384165+0000","last_peered":"2023-11-24T06:13:11.384165+0000","last_clean":"2023-11-24T06:13:11.384165+0000","last_became_active":"2023-11-24T06:13:09.176779+0000","last_became_peered":"2023-11-24T06:13:09.176779+0000","last_unstale":"2023-11-24T06:13:11.384165+0000","last_undegraded":"2023-11-24T06:13:11.384165+0000","last_fullsized":"2023-11-24T06:13:11.384165+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_clean_scrub_stamp":"2023-11-24T06:13:08.130949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:04:57.416969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-24T06:13:09.155203+0000","last_change":"2023-11-24T06:13:08.186499+0000","last_active":"2023-11-24T06:13:09.155203+0000","last_peered":"2023-11-24T06:13:09.155203+0000","last_clean":"2023-11-24T06:13:09.155203+0000","last_became_active":"2023-11-24T06:13:08.186190+0000","last_became_peered":"2023-11-24T06:13:08.186190+0000","last_unstale":"2023-11-24T06:13:09.155203+0000","last_undegraded":"2023-11-24T06:13:09.155203+0000","last_fullsized":"2023-11-24T06:13:09.155203+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:13:07.116698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:13:07.116698+0000","last_clean_scrub_stamp":"2023-11-24T06:13:07.116698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:30:39.016728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-24T06:13:10.622226+0000","last_change":"2023-11-24T06:13:09.172536+0000","last_active":"2023-11-24T06:13:10.622226+0000","last_peered":"2023-11-24T06:13:10.622226+0000","last_clean":"2023-11-24T06:13:10.622226+0000","last_became_active":"2023-11-24T06:13:09.172161+0000","last_became_peered":"2023-11-24T06:13:09.172161+0000","last_unstale":"2023-11-24T06:13:10.622226+0000","last_undegraded":"2023-11-24T06:13:10.622226+0000","last_fullsized":"2023-11-24T06:13:10.622226+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_clean_scrub_stamp":"2023-11-24T06:13:08.130949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:18:48.128215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-24T06:13:09.164661+0000","last_change":"2023-11-24T06:13:08.184883+0000","last_active":"2023-11-24T06:13:09.164661+0000","last_peered":"2023-11-24T06:13:09.164661+0000","last_clean":"2023-11-24T06:13:09.164661+0000","last_became_active":"2023-11-24T06:13:08.184347+0000","last_became_peered":"2023-11-24T06:13:08.184347+0000","last_unstale":"2023-11-24T06:13:09.164661+0000","last_undegraded":"2023-11-24T06:13:09.164661+0000","last_fullsized":"2023-11-24T06:13:09.164661+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:13:07.116698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:13:07.116698+0000","last_clean_scrub_stamp":"2023-11-24T06:13:07.116698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:50:23.508096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-24T06:13:10.621992+0000","last_change":"2023-11-24T06:13:09.170018+0000","last_active":"2023-11-24T06:13:10.621992+0000","last_peered":"2023-11-24T06:13:10.621992+0000","last_clean":"2023-11-24T06:13:10.621992+0000","last_became_active":"2023-11-24T06:13:09.169764+0000","last_became_peered":"2023-11-24T06:13:09.169764+0000","last_unstale":"2023-11-24T06:13:10.621992+0000","last_undegraded":"2023-11-24T06:13:10.621992+0000","last_fullsized":"2023-11-24T06:13:10.621992+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_clean_scrub_stamp":"2023-11-24T06:13:08.130949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:30:10.574156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-24T06:13:09.146079+0000","last_change":"2023-11-24T06:13:08.177276+0000","last_active":"2023-11-24T06:13:09.146079+0000","last_peered":"2023-11-24T06:13:09.146079+0000","last_clean":"2023-11-24T06:13:09.146079+0000","last_became_active":"2023-11-24T06:13:08.176553+0000","last_became_peered":"2023-11-24T06:13:08.176553+0000","last_unstale":"2023-11-24T06:13:09.146079+0000","last_undegraded":"2023-11-24T06:13:09.146079+0000","last_fullsized":"2023-11-24T06:13:09.146079+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:13:07.116698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:13:07.116698+0000","last_clean_scrub_stamp":"2023-11-24T06:13:07.116698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:56:45.201251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-24T06:13:10.622045+0000","last_change":"2023-11-24T06:13:09.168736+0000","last_active":"2023-11-24T06:13:10.622045+0000","last_peered":"2023-11-24T06:13:10.622045+0000","last_clean":"2023-11-24T06:13:10.622045+0000","last_became_active":"2023-11-24T06:13:09.168447+0000","last_became_peered":"2023-11-24T06:13:09.168447+0000","last_unstale":"2023-11-24T06:13:10.622045+0000","last_undegraded":"2023-11-24T06:13:10.622045+0000","last_fullsized":"2023-11-24T06:13:10.622045+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_clean_scrub_stamp":"2023-11-24T06:13:08.130949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:45:03.543887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-24T06:13:09.156605+0000","last_change":"2023-11-24T06:13:08.170518+0000","last_active":"2023-11-24T06:13:09.156605+0000","last_peered":"2023-11-24T06:13:09.156605+0000","last_clean":"2023-11-24T06:13:09.156605+0000","last_became_active":"2023-11-24T06:13:08.170186+0000","last_became_peered":"2023-11-24T06:13:08.170186+0000","last_unstale":"2023-11-24T06:13:09.156605+0000","last_undegraded":"2023-11-24T06:13:09.156605+0000","last_fullsized":"2023-11-24T06:13:09.156605+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:13:07.116698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:13:07.116698+0000","last_clean_scrub_stamp":"2023-11-24T06:13:07.116698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:12:06.056674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-24T06:13:11.385157+0000","last_change":"2023-11-24T06:13:09.168558+0000","last_active":"2023-11-24T06:13:11.385157+0000","last_peered":"2023-11-24T06:13:11.385157+0000","last_clean":"2023-11-24T06:13:11.385157+0000","last_became_active":"2023-11-24T06:13:09.168217+0000","last_became_peered":"2023-11-24T06:13:09.168217+0000","last_unstale":"2023-11-24T06:13:11.385157+0000","last_undegraded":"2023-11-24T06:13:11.385157+0000","last_fullsized":"2023-11-24T06:13:11.385157+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_clean_scrub_stamp":"2023-11-24T06:13:08.130949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:31:57.274979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"62'2","reported_seq":14,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-24T06:13:09.191524+0000","last_change":"2023-11-24T06:13:08.183006+0000","last_active":"2023-11-24T06:13:09.191524+0000","last_peered":"2023-11-24T06:13:09.191524+0000","last_clean":"2023-11-24T06:13:09.191524+0000","last_became_active":"2023-11-24T06:13:08.182538+0000","last_became_peered":"2023-11-24T06:13:08.182538+0000","last_unstale":"2023-11-24T06:13:09.191524+0000","last_undegraded":"2023-11-24T06:13:09.191524+0000","last_fullsized":"2023-11-24T06:13:09.191524+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:13:07.116698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:13:07.116698+0000","last_clean_scrub_stamp":"2023-11-24T06:13:07.116698+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:45:51.001578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-24T06:13:10.168352+0000","last_change":"2023-11-24T06:13:09.173923+0000","last_active":"2023-11-24T06:13:10.168352+0000","last_peered":"2023-11-24T06:13:10.168352+0000","last_clean":"2023-11-24T06:13:10.168352+0000","last_became_active":"2023-11-24T06:13:09.173644+0000","last_became_peered":"2023-11-24T06:13:09.173644+0000","last_unstale":"2023-11-24T06:13:10.168352+0000","last_undegraded":"2023-11-24T06:13:10.168352+0000","last_fullsized":"2023-11-24T06:13:10.168352+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_clean_scrub_stamp":"2023-11-24T06:13:08.130949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:10:28.958299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-24T06:13:09.152496+0000","last_change":"2023-11-24T06:13:08.185150+0000","last_active":"2023-11-24T06:13:09.152496+0000","last_peered":"2023-11-24T06:13:09.152496+0000","last_clean":"2023-11-24T06:13:09.152496+0000","last_became_active":"2023-11-24T06:13:08.184873+0000","last_became_peered":"2023-11-24T06:13:08.184873+0000","last_unstale":"2023-11-24T06:13:09.152496+0000","last_undegraded":"2023-11-24T06:13:09.152496+0000","last_fullsized":"2023-11-24T06:13:09.152496+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:13:07.116698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:13:07.116698+0000","last_clean_scrub_stamp":"2023-11-24T06:13:07.116698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:49:37.745773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-24T06:13:10.171968+0000","last_change":"2023-11-24T06:13:09.176771+0000","last_active":"2023-11-24T06:13:10.171968+0000","last_peered":"2023-11-24T06:13:10.171968+0000","last_clean":"2023-11-24T06:13:10.171968+0000","last_became_active":"2023-11-24T06:13:09.175367+0000","last_became_peered":"2023-11-24T06:13:09.175367+0000","last_unstale":"2023-11-24T06:13:10.171968+0000","last_undegraded":"2023-11-24T06:13:10.171968+0000","last_fullsized":"2023-11-24T06:13:10.171968+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_clean_scrub_stamp":"2023-11-24T06:13:08.130949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:56:15.626987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-24T06:13:09.162203+0000","last_change":"2023-11-24T06:13:08.179883+0000","last_active":"2023-11-24T06:13:09.162203+0000","last_peered":"2023-11-24T06:13:09.162203+0000","last_clean":"2023-11-24T06:13:09.162203+0000","last_became_active":"2023-11-24T06:13:08.179388+0000","last_became_peered":"2023-11-24T06:13:08.179388+0000","last_unstale":"2023-11-24T06:13:09.162203+0000","last_undegraded":"2023-11-24T06:13:09.162203+0000","last_fullsized":"2023-11-24T06:13:09.162203+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:13:07.116698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:13:07.116698+0000","last_clean_scrub_stamp":"2023-11-24T06:13:07.116698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:23:32.526555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-24T06:13:10.172091+0000","last_change":"2023-11-24T06:13:09.170115+0000","last_active":"2023-11-24T06:13:10.172091+0000","last_peered":"2023-11-24T06:13:10.172091+0000","last_clean":"2023-11-24T06:13:10.172091+0000","last_became_active":"2023-11-24T06:13:09.169823+0000","last_became_peered":"2023-11-24T06:13:09.169823+0000","last_unstale":"2023-11-24T06:13:10.172091+0000","last_undegraded":"2023-11-24T06:13:10.172091+0000","last_fullsized":"2023-11-24T06:13:10.172091+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_clean_scrub_stamp":"2023-11-24T06:13:08.130949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:45:03.543887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-24T06:13:09.158840+0000","last_change":"2023-11-24T06:13:08.167670+0000","last_active":"2023-11-24T06:13:09.158840+0000","last_peered":"2023-11-24T06:13:09.158840+0000","last_clean":"2023-11-24T06:13:09.158840+0000","last_became_active":"2023-11-24T06:13:08.166123+0000","last_became_peered":"2023-11-24T06:13:08.166123+0000","last_unstale":"2023-11-24T06:13:09.158840+0000","last_undegraded":"2023-11-24T06:13:09.158840+0000","last_fullsized":"2023-11-24T06:13:09.158840+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:13:07.116698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:13:07.116698+0000","last_clean_scrub_stamp":"2023-11-24T06:13:07.116698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:32:53.662482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-24T06:13:11.383386+0000","last_change":"2023-11-24T06:13:09.180566+0000","last_active":"2023-11-24T06:13:11.383386+0000","last_peered":"2023-11-24T06:13:11.383386+0000","last_clean":"2023-11-24T06:13:11.383386+0000","last_became_active":"2023-11-24T06:13:09.176843+0000","last_became_peered":"2023-11-24T06:13:09.176843+0000","last_unstale":"2023-11-24T06:13:11.383386+0000","last_undegraded":"2023-11-24T06:13:11.383386+0000","last_fullsized":"2023-11-24T06:13:11.383386+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_clean_scrub_stamp":"2023-11-24T06:13:08.130949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:24:31.498179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-24T06:13:09.165050+0000","last_change":"2023-11-24T06:13:08.182920+0000","last_active":"2023-11-24T06:13:09.165050+0000","last_peered":"2023-11-24T06:13:09.165050+0000","last_clean":"2023-11-24T06:13:09.165050+0000","last_became_active":"2023-11-24T06:13:08.182638+0000","last_became_peered":"2023-11-24T06:13:08.182638+0000","last_unstale":"2023-11-24T06:13:09.165050+0000","last_undegraded":"2023-11-24T06:13:09.165050+0000","last_fullsized":"2023-11-24T06:13:09.165050+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:13:07.116698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:13:07.116698+0000","last_clean_scrub_stamp":"2023-11-24T06:13:07.116698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:30:09.204783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-24T06:13:10.172052+0000","last_change":"2023-11-24T06:13:09.176232+0000","last_active":"2023-11-24T06:13:10.172052+0000","last_peered":"2023-11-24T06:13:10.172052+0000","last_clean":"2023-11-24T06:13:10.172052+0000","last_became_active":"2023-11-24T06:13:09.175953+0000","last_became_peered":"2023-11-24T06:13:09.175953+0000","last_unstale":"2023-11-24T06:13:10.172052+0000","last_undegraded":"2023-11-24T06:13:10.172052+0000","last_fullsized":"2023-11-24T06:13:10.172052+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_clean_scrub_stamp":"2023-11-24T06:13:08.130949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:02:49.934847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-24T06:13:09.164097+0000","last_change":"2023-11-24T06:13:08.176458+0000","last_active":"2023-11-24T06:13:09.164097+0000","last_peered":"2023-11-24T06:13:09.164097+0000","last_clean":"2023-11-24T06:13:09.164097+0000","last_became_active":"2023-11-24T06:13:08.176179+0000","last_became_peered":"2023-11-24T06:13:08.176179+0000","last_unstale":"2023-11-24T06:13:09.164097+0000","last_undegraded":"2023-11-24T06:13:09.164097+0000","last_fullsized":"2023-11-24T06:13:09.164097+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:13:07.116698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:13:07.116698+0000","last_clean_scrub_stamp":"2023-11-24T06:13:07.116698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:28:11.494557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-24T06:13:10.808170+0000","last_change":"2023-11-24T06:13:09.173909+0000","last_active":"2023-11-24T06:13:10.808170+0000","last_peered":"2023-11-24T06:13:10.808170+0000","last_clean":"2023-11-24T06:13:10.808170+0000","last_became_active":"2023-11-24T06:13:09.171945+0000","last_became_peered":"2023-11-24T06:13:09.171945+0000","last_unstale":"2023-11-24T06:13:10.808170+0000","last_undegraded":"2023-11-24T06:13:10.808170+0000","last_fullsized":"2023-11-24T06:13:10.808170+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_clean_scrub_stamp":"2023-11-24T06:13:08.130949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:58:55.969306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-24T06:13:09.164737+0000","last_change":"2023-11-24T06:13:08.181184+0000","last_active":"2023-11-24T06:13:09.164737+0000","last_peered":"2023-11-24T06:13:09.164737+0000","last_clean":"2023-11-24T06:13:09.164737+0000","last_became_active":"2023-11-24T06:13:08.180870+0000","last_became_peered":"2023-11-24T06:13:08.180870+0000","last_unstale":"2023-11-24T06:13:09.164737+0000","last_undegraded":"2023-11-24T06:13:09.164737+0000","last_fullsized":"2023-11-24T06:13:09.164737+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:13:07.116698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:13:07.116698+0000","last_clean_scrub_stamp":"2023-11-24T06:13:07.116698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:02:48.920596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-24T06:13:10.170254+0000","last_change":"2023-11-24T06:13:09.177460+0000","last_active":"2023-11-24T06:13:10.170254+0000","last_peered":"2023-11-24T06:13:10.170254+0000","last_clean":"2023-11-24T06:13:10.170254+0000","last_became_active":"2023-11-24T06:13:09.177157+0000","last_became_peered":"2023-11-24T06:13:09.177157+0000","last_unstale":"2023-11-24T06:13:10.170254+0000","last_undegraded":"2023-11-24T06:13:10.170254+0000","last_fullsized":"2023-11-24T06:13:10.170254+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_clean_scrub_stamp":"2023-11-24T06:13:08.130949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:14:22.413849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-24T06:13:09.158374+0000","last_change":"2023-11-24T06:13:08.176974+0000","last_active":"2023-11-24T06:13:09.158374+0000","last_peered":"2023-11-24T06:13:09.158374+0000","last_clean":"2023-11-24T06:13:09.158374+0000","last_became_active":"2023-11-24T06:13:08.176700+0000","last_became_peered":"2023-11-24T06:13:08.176700+0000","last_unstale":"2023-11-24T06:13:09.158374+0000","last_undegraded":"2023-11-24T06:13:09.158374+0000","last_fullsized":"2023-11-24T06:13:09.158374+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:13:07.116698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:13:07.116698+0000","last_clean_scrub_stamp":"2023-11-24T06:13:07.116698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:11:47.599940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-24T06:13:10.622116+0000","last_change":"2023-11-24T06:13:09.171036+0000","last_active":"2023-11-24T06:13:10.622116+0000","last_peered":"2023-11-24T06:13:10.622116+0000","last_clean":"2023-11-24T06:13:10.622116+0000","last_became_active":"2023-11-24T06:13:09.170566+0000","last_became_peered":"2023-11-24T06:13:09.170566+0000","last_unstale":"2023-11-24T06:13:10.622116+0000","last_undegraded":"2023-11-24T06:13:10.622116+0000","last_fullsized":"2023-11-24T06:13:10.622116+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:13:08.130949+0000","last_clean_scrub_stamp":"2023-11-24T06:13:08.130949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:03:20.261960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"62'2","reported_seq":14,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-24T06:13:09.191935+0000","last_change":"2023-11-24T06:13:08.179536+0000","last_active":"2023-11-24T06:13:09.191935+0000","last_peered":"2023-11-24T06:13:09.191935+0000","last_clean":"2023-11-24T06:13:09.191935+0000","last_became_active":"2023-11-24T06:13:08.179206+0000","last_became_peered":"2023-11-24T06:13:08.179206+0000","last_unstale":"2023-11-24T06:13:09.191935+0000","last_undegraded":"2023-11-24T06:13:09.191935+0000","last_fullsized":"2023-11-24T06:13:09.191935+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:13:07.116698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:13:07.116698+0000","last_clean_scrub_stamp":"2023-11-24T06:13:07.116698+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:33:54.722506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":202,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-24T06:13:09.152988+0000","last_change":"2023-11-24T06:10:20.082677+0000","last_active":"2023-11-24T06:13:09.152988+0000","last_peered":"2023-11-24T06:13:09.152988+0000","last_clean":"2023-11-24T06:13:09.152988+0000","last_became_active":"2023-11-24T06:10:20.080551+0000","last_became_peered":"2023-11-24T06:10:20.080551+0000","last_unstale":"2023-11-24T06:13:09.152988+0000","last_undegraded":"2023-11-24T06:13:09.152988+0000","last_fullsized":"2023-11-24T06:13:09.152988+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:18.602209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:18.602209+0000","last_clean_scrub_stamp":"2023-11-24T06:10:18.602209+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:47:24.216235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574885,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32520,"kb_used_data":1336,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339320,"statfs":{"total":96636764160,"available":96603463680,"internally_reserved":0,"allocated":1368064,"data_stored":919231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:12:18 2023","interfaces":[{"interface":"back","average":{"1min":1.016,"5min":1.091,"15min":1.103},"min":{"1min":0.75600000000000001,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":1.5580000000000001,"5min":1.9630000000000001,"15min":1.9630000000000001},"last":0.93999999999999995},{"interface":"front","average":{"1min":1.0529999999999999,"5min":1.056,"15min":1.056},"min":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":1.605,"5min":1.621,"15min":1.621},"last":0.88700000000000001}]},{"osd":1,"last update":"Fri Nov 24 06:12:18 2023","interfaces":[{"interface":"back","average":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"min":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":1.657,"5min":2.0259999999999998,"15min":2.0259999999999998},"last":0.82399999999999995},{"interface":"front","average":{"1min":1.226,"5min":1.149,"15min":1.1359999999999999},"min":{"1min":0.70699999999999996,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.786,"5min":2.1480000000000001,"15min":2.1480000000000001},"last":1.5189999999999999}]},{"osd":2,"last update":"Fri Nov 24 06:12:18 2023","interfaces":[{"interface":"back","average":{"1min":1.1000000000000001,"5min":1.1339999999999999,"15min":1.1399999999999999},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.488,"5min":1.7809999999999999,"15min":1.7809999999999999},"last":1.101},{"interface":"front","average":{"1min":1.2370000000000001,"5min":1.1850000000000001,"15min":1.1759999999999999},"min":{"1min":0.90900000000000003,"5min":0.86599999999999999,"15min":0.86599999999999999},"max":{"1min":2.0619999999999998,"5min":2.0619999999999998,"15min":2.0619999999999998},"last":1.042}]},{"osd":3,"last update":"Fri Nov 24 06:12:18 2023","interfaces":[{"interface":"back","average":{"1min":1.1970000000000001,"5min":1.204,"15min":1.2050000000000001},"min":{"1min":0.89600000000000002,"5min":0.73999999999999999,"15min":0.73999999999999999},"max":{"1min":1.615,"5min":1.615,"15min":1.615},"last":0.99299999999999999},{"interface":"front","average":{"1min":1.2909999999999999,"5min":1.3069999999999999,"15min":1.3089999999999999},"min":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"max":{"1min":1.9570000000000001,"5min":1.9570000000000001,"15min":1.9570000000000001},"last":1.1399999999999999}]},{"osd":4,"last update":"Fri Nov 24 06:12:18 2023","interfaces":[{"interface":"back","average":{"1min":1.3660000000000001,"5min":1.304,"15min":1.294},"min":{"1min":0.748,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.9570000000000001,"5min":1.9570000000000001,"15min":1.9570000000000001},"last":1.651},{"interface":"front","average":{"1min":1.3320000000000001,"5min":1.329,"15min":1.329},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.8340000000000001,"5min":2.1680000000000001,"15min":2.1680000000000001},"last":1.3440000000000001}]},{"osd":5,"last update":"Fri Nov 24 06:12:18 2023","interfaces":[{"interface":"back","average":{"1min":1.546,"5min":1.466,"15min":1.452},"min":{"1min":0.75900000000000001,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":2.0379999999999998,"5min":2.0379999999999998,"15min":2.0379999999999998},"last":1.7150000000000001},{"interface":"front","average":{"1min":1.5720000000000001,"5min":1.4770000000000001,"15min":1.4610000000000001},"min":{"1min":1.077,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.923,"5min":2.0539999999999998,"15min":2.0539999999999998},"last":1.46}]},{"osd":6,"last update":"Fri Nov 24 06:12:18 2023","interfaces":[{"interface":"back","average":{"1min":1.6559999999999999,"5min":1.548,"15min":1.53},"min":{"1min":0.94499999999999995,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":2.0899999999999999,"5min":2.0899999999999999,"15min":2.0899999999999999},"last":1.794},{"interface":"front","average":{"1min":1.6719999999999999,"5min":1.5580000000000001,"15min":1.5389999999999999},"min":{"1min":1.127,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":2.0070000000000001,"5min":2.0680000000000001,"15min":2.0680000000000001},"last":1.7390000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574886,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31944,"kb_used_data":756,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339896,"statfs":{"total":96636764160,"available":96604053504,"internally_reserved":0,"allocated":774144,"data_stored":328339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:12:21 2023","interfaces":[{"interface":"back","average":{"1min":1.097,"5min":1.194,"15min":1.21},"min":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":2.032,"5min":2.032,"15min":2.032},"last":0.77100000000000002},{"interface":"front","average":{"1min":1.1639999999999999,"5min":1.212,"15min":1.22},"min":{"1min":0.81599999999999995,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.9199999999999999,"5min":2.0230000000000001,"15min":2.0230000000000001},"last":1.008}]},{"osd":1,"last update":"Fri Nov 24 06:12:21 2023","interfaces":[{"interface":"back","average":{"1min":1.149,"5min":1.163,"15min":1.165},"min":{"1min":0.72199999999999998,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":2.1499999999999999,"5min":2.1499999999999999,"15min":2.1499999999999999},"last":1.3500000000000001},{"interface":"front","average":{"1min":1.1970000000000001,"5min":1.2270000000000001,"15min":1.232},"min":{"1min":0.78700000000000003,"5min":0.77500000000000002,"15min":0.77500000000000002},"max":{"1min":1.645,"5min":1.8899999999999999,"15min":1.8899999999999999},"last":1.5720000000000001}]},{"osd":2,"last update":"Fri Nov 24 06:12:21 2023","interfaces":[{"interface":"back","average":{"1min":1.4179999999999999,"5min":1.2050000000000001,"15min":1.169},"min":{"1min":0.88,"5min":0.82299999999999995,"15min":0.82299999999999995},"max":{"1min":4.7359999999999998,"5min":4.7359999999999998,"15min":4.7359999999999998},"last":0.91900000000000004},{"interface":"front","average":{"1min":1.4970000000000001,"5min":1.45,"15min":1.4419999999999999},"min":{"1min":0.93799999999999994,"5min":0.85899999999999999,"15min":0.85899999999999999},"max":{"1min":3.9700000000000002,"5min":3.9700000000000002,"15min":3.9700000000000002},"last":1.071}]},{"osd":3,"last update":"Fri Nov 24 06:12:21 2023","interfaces":[{"interface":"back","average":{"1min":1.2809999999999999,"5min":1.3069999999999999,"15min":1.3109999999999999},"min":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"max":{"1min":1.798,"5min":1.919,"15min":1.919},"last":1.403},{"interface":"front","average":{"1min":1.28,"5min":1.292,"15min":1.294},"min":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":2.3929999999999998,"5min":2.3929999999999998,"15min":2.3929999999999998},"last":1.532}]},{"osd":4,"last update":"Fri Nov 24 06:12:21 2023","interfaces":[{"interface":"back","average":{"1min":1.4470000000000001,"5min":1.4550000000000001,"15min":1.4570000000000001},"min":{"1min":0.873,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":2.4039999999999999,"5min":2.4039999999999999,"15min":2.4039999999999999},"last":1.196},{"interface":"front","average":{"1min":1.5620000000000001,"5min":1.496,"15min":1.4850000000000001},"min":{"1min":0.88400000000000001,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":2.5609999999999999,"5min":2.5609999999999999,"15min":2.5609999999999999},"last":1.4910000000000001}]},{"osd":5,"last update":"Fri Nov 24 06:12:21 2023","interfaces":[{"interface":"back","average":{"1min":1.4610000000000001,"5min":1.4730000000000001,"15min":1.4750000000000001},"min":{"1min":0.90000000000000002,"5min":0.79100000000000004,"15min":0.79100000000000004},"max":{"1min":2.8290000000000002,"5min":2.8290000000000002,"15min":2.8290000000000002},"last":1.292},{"interface":"front","average":{"1min":1.639,"5min":1.653,"15min":1.655},"min":{"1min":1.2250000000000001,"5min":0.93400000000000005,"15min":0.93400000000000005},"max":{"1min":2.9420000000000002,"5min":2.9420000000000002,"15min":2.9420000000000002},"last":1.4339999999999999}]},{"osd":7,"last update":"Fri Nov 24 06:12:21 2023","interfaces":[{"interface":"back","average":{"1min":1.6160000000000001,"5min":1.5960000000000001,"15min":1.5920000000000001},"min":{"1min":1.1299999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":2.1859999999999999,"5min":2.5139999999999998,"15min":2.5139999999999998},"last":1.129},{"interface":"front","average":{"1min":1.7509999999999999,"5min":1.5880000000000001,"15min":1.5609999999999999},"min":{"1min":1.0249999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":4.7859999999999996,"5min":4.7859999999999996,"15min":4.7859999999999996},"last":1.6359999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574886,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31928,"kb_used_data":748,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339912,"statfs":{"total":96636764160,"available":96604069888,"internally_reserved":0,"allocated":765952,"data_stored":327723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:12:21 2023","interfaces":[{"interface":"back","average":{"1min":0.94099999999999995,"5min":1.1890000000000001,"15min":1.23},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.4750000000000001,"5min":3.0030000000000001,"15min":3.0030000000000001},"last":1.1439999999999999},{"interface":"front","average":{"1min":0.99199999999999999,"5min":1.1799999999999999,"15min":1.2110000000000001},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.5369999999999999,"5min":2.0979999999999999,"15min":2.0979999999999999},"last":0.90000000000000002}]},{"osd":1,"last update":"Fri Nov 24 06:12:21 2023","interfaces":[{"interface":"back","average":{"1min":1.1599999999999999,"5min":1.252,"15min":1.268},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.728,"5min":5.0499999999999998,"15min":5.0499999999999998},"last":1.24},{"interface":"front","average":{"1min":1.0529999999999999,"5min":1.3939999999999999,"15min":1.4510000000000001},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.5509999999999999,"5min":3.6930000000000001,"15min":3.6930000000000001},"last":0.999}]},{"osd":2,"last update":"Fri Nov 24 06:12:21 2023","interfaces":[{"interface":"back","average":{"1min":1.161,"5min":1.3080000000000001,"15min":1.3320000000000001},"min":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"max":{"1min":1.5389999999999999,"5min":3.085,"15min":3.085},"last":1.4139999999999999},{"interface":"front","average":{"1min":1.1879999999999999,"5min":1.53,"15min":1.587},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.8340000000000001,"5min":4.7359999999999998,"15min":4.7359999999999998},"last":1.4450000000000001}]},{"osd":3,"last update":"Fri Nov 24 06:12:21 2023","interfaces":[{"interface":"back","average":{"1min":1.2450000000000001,"5min":1.385,"15min":1.409},"min":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"max":{"1min":1.7629999999999999,"5min":3.2229999999999999,"15min":3.2229999999999999},"last":1.472},{"interface":"front","average":{"1min":1.2929999999999999,"5min":1.4139999999999999,"15min":1.4339999999999999},"min":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"max":{"1min":1.8069999999999999,"5min":2.3260000000000001,"15min":2.3260000000000001},"last":1.327}]},{"osd":4,"last update":"Fri Nov 24 06:12:21 2023","interfaces":[{"interface":"back","average":{"1min":1.3759999999999999,"5min":1.528,"15min":1.554},"min":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"max":{"1min":1.756,"5min":3.1230000000000002,"15min":3.1230000000000002},"last":1.5569999999999999},{"interface":"front","average":{"1min":1.375,"5min":1.522,"15min":1.546},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":2.044,"5min":2.4809999999999999,"15min":2.4809999999999999},"last":1.1830000000000001}]},{"osd":6,"last update":"Fri Nov 24 06:12:21 2023","interfaces":[{"interface":"back","average":{"1min":1.427,"5min":1.6910000000000001,"15min":1.7350000000000001},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.913,"5min":4.6529999999999996,"15min":4.6529999999999996},"last":1.5069999999999999},{"interface":"front","average":{"1min":1.4890000000000001,"5min":1.6970000000000001,"15min":1.732},"min":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"max":{"1min":2.266,"5min":4.3529999999999998,"15min":4.3529999999999998},"last":1.3600000000000001}]},{"osd":7,"last update":"Fri Nov 24 06:12:21 2023","interfaces":[{"interface":"back","average":{"1min":1.496,"5min":1.6679999999999999,"15min":1.696},"min":{"1min":0.874,"5min":0.874,"15min":0.874},"max":{"1min":1.9830000000000001,"5min":2.8839999999999999,"15min":2.8839999999999999},"last":1.3009999999999999},{"interface":"front","average":{"1min":1.575,"5min":1.8080000000000001,"15min":1.847},"min":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":2.0699999999999998,"5min":4.8739999999999997,"15min":4.8739999999999997},"last":1.597}]}]},{"osd":4,"up_from":13,"seq":55834574886,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31932,"kb_used_data":752,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339908,"statfs":{"total":96636764160,"available":96604065792,"internally_reserved":0,"allocated":770048,"data_stored":328317,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:12:21 2023","interfaces":[{"interface":"back","average":{"1min":1.127,"5min":1.0589999999999999,"15min":1.0469999999999999},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.585,"5min":1.585,"15min":1.585},"last":0.90200000000000002},{"interface":"front","average":{"1min":1.0429999999999999,"5min":1.1819999999999999,"15min":1.2050000000000001},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.744,"5min":1.8220000000000001,"15min":1.8220000000000001},"last":1.012}]},{"osd":1,"last update":"Fri Nov 24 06:12:21 2023","interfaces":[{"interface":"back","average":{"1min":1.1499999999999999,"5min":1.1259999999999999,"15min":1.1220000000000001},"min":{"1min":0.73499999999999999,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":1.97,"5min":1.97,"15min":1.97},"last":1.1859999999999999},{"interface":"front","average":{"1min":1.1819999999999999,"5min":1.1779999999999999,"15min":1.1779999999999999},"min":{"1min":0.79800000000000004,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":1.917,"5min":1.917,"15min":1.917},"last":1.5820000000000001}]},{"osd":2,"last update":"Fri Nov 24 06:12:21 2023","interfaces":[{"interface":"back","average":{"1min":1.0980000000000001,"5min":1.22,"15min":1.24},"min":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":1.6970000000000001,"5min":2.319,"15min":2.319},"last":1.2709999999999999},{"interface":"front","average":{"1min":1.3320000000000001,"5min":1.3129999999999999,"15min":1.3100000000000001},"min":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"max":{"1min":2.077,"5min":2.0990000000000002,"15min":2.0990000000000002},"last":1.522}]},{"osd":3,"last update":"Fri Nov 24 06:12:21 2023","interfaces":[{"interface":"back","average":{"1min":1.206,"5min":1.3700000000000001,"15min":1.3979999999999999},"min":{"1min":0.752,"5min":0.752,"15min":0.752},"max":{"1min":2.048,"5min":2.048,"15min":2.048},"last":1.4550000000000001},{"interface":"front","average":{"1min":1.3340000000000001,"5min":1.4199999999999999,"15min":1.4339999999999999},"min":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"max":{"1min":1.8149999999999999,"5min":2.1720000000000002,"15min":2.1720000000000002},"last":1.4219999999999999}]},{"osd":5,"last update":"Fri Nov 24 06:12:21 2023","interfaces":[{"interface":"back","average":{"1min":1.2829999999999999,"5min":1.327,"15min":1.3340000000000001},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.835,"5min":1.895,"15min":1.895},"last":1.377},{"interface":"front","average":{"1min":1.4359999999999999,"5min":1.528,"15min":1.544},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":2.044,"5min":2.5059999999999998,"15min":2.5059999999999998},"last":1.3340000000000001}]},{"osd":6,"last update":"Fri Nov 24 06:12:21 2023","interfaces":[{"interface":"back","average":{"1min":1.423,"5min":1.522,"15min":1.538},"min":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"max":{"1min":2.1509999999999998,"5min":2.2069999999999999,"15min":2.2069999999999999},"last":1.7729999999999999},{"interface":"front","average":{"1min":1.587,"5min":1.665,"15min":1.6779999999999999},"min":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":2.0139999999999998,"5min":2.4510000000000001,"15min":2.4510000000000001},"last":1.6990000000000001}]},{"osd":7,"last update":"Fri Nov 24 06:12:21 2023","interfaces":[{"interface":"back","average":{"1min":1.5640000000000001,"5min":1.6279999999999999,"15min":1.6379999999999999},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":2.214,"5min":2.4350000000000001,"15min":2.4350000000000001},"last":1.8480000000000001},{"interface":"front","average":{"1min":1.554,"5min":1.617,"15min":1.627},"min":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":2.1499999999999999,"5min":3.1120000000000001,"15min":3.1120000000000001},"last":1.627}]}]},{"osd":3,"up_from":13,"seq":55834574886,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27776,"kb_used_data":756,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344064,"statfs":{"total":96636764160,"available":96608321536,"internally_reserved":0,"allocated":774144,"data_stored":328359,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:12:26 2023","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.98599999999999999,"15min":1},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":1.8360000000000001,"5min":2.766,"15min":2.766},"last":0.95899999999999996},{"interface":"front","average":{"1min":1.0800000000000001,"5min":0.94799999999999995,"15min":0.92600000000000005},"min":{"1min":0.44600000000000001,"5min":0.441,"15min":0.441},"max":{"1min":1.873,"5min":1.873,"15min":1.873},"last":1.081}]},{"osd":1,"last update":"Fri Nov 24 06:12:26 2023","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.876,"15min":0.88200000000000001},"min":{"1min":0.46999999999999997,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.2569999999999999,"5min":2.5779999999999998,"15min":2.5779999999999998},"last":1.1759999999999999},{"interface":"front","average":{"1min":1.02,"5min":1.151,"15min":1.173},"min":{"1min":0.50700000000000001,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.7350000000000001,"5min":3.6899999999999999,"15min":3.6899999999999999},"last":1.141}]},{"osd":2,"last update":"Fri Nov 24 06:12:26 2023","interfaces":[{"interface":"back","average":{"1min":0.996,"5min":1.1399999999999999,"15min":1.1639999999999999},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.637,"5min":4.0620000000000003,"15min":4.0620000000000003},"last":1.3759999999999999},{"interface":"front","average":{"1min":1.113,"5min":1.204,"15min":1.2190000000000001},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.5309999999999999,"5min":2.9529999999999998,"15min":2.9529999999999998},"last":1.2230000000000001}]},{"osd":4,"last update":"Fri Nov 24 06:12:26 2023","interfaces":[{"interface":"back","average":{"1min":1.26,"5min":1.3939999999999999,"15min":1.4159999999999999},"min":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"max":{"1min":1.6819999999999999,"5min":4.125,"15min":4.125},"last":1.2410000000000001},{"interface":"front","average":{"1min":1.351,"5min":1.427,"15min":1.4399999999999999},"min":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"max":{"1min":1.7689999999999999,"5min":3.9409999999999998,"15min":3.9409999999999998},"last":1.633}]},{"osd":5,"last update":"Fri Nov 24 06:12:26 2023","interfaces":[{"interface":"back","average":{"1min":1.319,"5min":1.421,"15min":1.4379999999999999},"min":{"1min":0.749,"5min":0.749,"15min":0.749},"max":{"1min":1.6679999999999999,"5min":3.3359999999999999,"15min":3.3359999999999999},"last":1.417},{"interface":"front","average":{"1min":1.214,"5min":1.4099999999999999,"15min":1.4430000000000001},"min":{"1min":0.86499999999999999,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.542,"5min":4.1669999999999998,"15min":4.1669999999999998},"last":1.3320000000000001}]},{"osd":6,"last update":"Fri Nov 24 06:12:26 2023","interfaces":[{"interface":"back","average":{"1min":1.2230000000000001,"5min":1.3939999999999999,"15min":1.4219999999999999},"min":{"1min":0.871,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":1.571,"5min":4.0190000000000001,"15min":4.0190000000000001},"last":1.5700000000000001},{"interface":"front","average":{"1min":1.429,"5min":1.512,"15min":1.526},"min":{"1min":0.95999999999999996,"5min":0.92300000000000004,"15min":0.92300000000000004},"max":{"1min":1.966,"5min":3.2480000000000002,"15min":3.2480000000000002},"last":1.5329999999999999}]},{"osd":7,"last update":"Fri Nov 24 06:12:26 2023","interfaces":[{"interface":"back","average":{"1min":1.3400000000000001,"5min":1.5580000000000001,"15min":1.5940000000000001},"min":{"1min":0.79500000000000004,"5min":0.76600000000000001,"15min":0.76600000000000001},"max":{"1min":1.8049999999999999,"5min":4.202,"15min":4.202},"last":1.4610000000000001},{"interface":"front","average":{"1min":1.3480000000000001,"5min":1.5669999999999999,"15min":1.603},"min":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"max":{"1min":1.9199999999999999,"5min":3.9849999999999999,"15min":3.9849999999999999},"last":1.8799999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574886,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27776,"kb_used_data":756,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344064,"statfs":{"total":96636764160,"available":96608321536,"internally_reserved":0,"allocated":774144,"data_stored":329245,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:12:22 2023","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.88,"15min":0.88600000000000001},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":1.3149999999999999,"5min":1.6259999999999999,"15min":1.6259999999999999},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.96799999999999997,"5min":1.0740000000000001,"15min":1.0920000000000001},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.619,"5min":1.7669999999999999,"15min":1.7669999999999999},"last":0.98999999999999999}]},{"osd":1,"last update":"Fri Nov 24 06:12:22 2023","interfaces":[{"interface":"back","average":{"1min":0.99399999999999999,"5min":0.94999999999999996,"15min":0.94199999999999995},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.635,"5min":1.635,"15min":1.635},"last":0.871},{"interface":"front","average":{"1min":0.95499999999999996,"5min":0.97599999999999998,"15min":0.97999999999999998},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.498,"5min":1.8140000000000001,"15min":1.8140000000000001},"last":1.0800000000000001}]},{"osd":3,"last update":"Fri Nov 24 06:12:22 2023","interfaces":[{"interface":"back","average":{"1min":0.94499999999999995,"5min":1.0529999999999999,"15min":1.071},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.476,"5min":2.3559999999999999,"15min":2.3559999999999999},"last":0.92900000000000005},{"interface":"front","average":{"1min":1.1719999999999999,"5min":1.1319999999999999,"15min":1.1259999999999999},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.643,"5min":2.1840000000000002,"15min":2.1840000000000002},"last":1.127}]},{"osd":4,"last update":"Fri Nov 24 06:12:22 2023","interfaces":[{"interface":"back","average":{"1min":1.411,"5min":1.417,"15min":1.4179999999999999},"min":{"1min":0.72099999999999997,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.9350000000000001,"5min":3.411,"15min":3.411},"last":1.268},{"interface":"front","average":{"1min":1.333,"5min":1.4710000000000001,"15min":1.494},"min":{"1min":0.88400000000000001,"5min":0.82699999999999996,"15min":0.82699999999999996},"max":{"1min":1.964,"5min":3.7109999999999999,"15min":3.7109999999999999},"last":1.413}]},{"osd":5,"last update":"Fri Nov 24 06:12:22 2023","interfaces":[{"interface":"back","average":{"1min":1.246,"5min":1.3089999999999999,"15min":1.319},"min":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"max":{"1min":1.5620000000000001,"5min":2.5259999999999998,"15min":2.5259999999999998},"last":1.2110000000000001},{"interface":"front","average":{"1min":1.444,"5min":1.3300000000000001,"15min":1.3109999999999999},"min":{"1min":0.96799999999999997,"5min":0.90400000000000003,"15min":0.90400000000000003},"max":{"1min":1.8839999999999999,"5min":2.1640000000000001,"15min":2.1640000000000001},"last":1.472}]},{"osd":6,"last update":"Fri Nov 24 06:12:22 2023","interfaces":[{"interface":"back","average":{"1min":1.244,"5min":1.4139999999999999,"15min":1.4419999999999999},"min":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"max":{"1min":1.8400000000000001,"5min":2.3039999999999998,"15min":2.3039999999999998},"last":1.7170000000000001},{"interface":"front","average":{"1min":1.347,"5min":1.444,"15min":1.46},"min":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"max":{"1min":1.6850000000000001,"5min":2.605,"15min":2.605},"last":1.3080000000000001}]},{"osd":7,"last update":"Fri Nov 24 06:12:22 2023","interfaces":[{"interface":"back","average":{"1min":1.365,"5min":1.5489999999999999,"15min":1.579},"min":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"max":{"1min":1.9079999999999999,"5min":2.3740000000000001,"15min":2.3740000000000001},"last":1.7829999999999999},{"interface":"front","average":{"1min":1.3009999999999999,"5min":1.4330000000000001,"15min":1.4550000000000001},"min":{"1min":0.877,"5min":0.877,"15min":0.877},"max":{"1min":1.575,"5min":2.5699999999999998,"15min":2.5699999999999998},"last":1.3700000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574885,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27796,"kb_used_data":768,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344044,"statfs":{"total":96636764160,"available":96608301056,"internally_reserved":0,"allocated":786432,"data_stored":330475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:12:21 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.878,"15min":0.90800000000000003},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":1.3740000000000001,"5min":1.9299999999999999,"15min":1.9299999999999999},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.90700000000000003,"15min":0.92500000000000004},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":1.258,"5min":1.7669999999999999,"15min":1.7669999999999999},"last":0.93000000000000005}]},{"osd":2,"last update":"Fri Nov 24 06:12:21 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.96499999999999997,"15min":1.0009999999999999},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.0840000000000001,"5min":2.5329999999999999,"15min":2.5329999999999999},"last":1.1559999999999999},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.99099999999999999,"15min":1.0169999999999999},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.159,"5min":1.657,"15min":1.657},"last":1.3540000000000001}]},{"osd":3,"last update":"Fri Nov 24 06:12:21 2023","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.98899999999999999,"15min":1.0209999999999999},"min":{"1min":0.49399999999999999,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.2729999999999999,"5min":1.841,"15min":1.841},"last":1.002},{"interface":"front","average":{"1min":0.98399999999999999,"5min":1.121,"15min":1.1439999999999999},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.5249999999999999,"5min":2.8940000000000001,"15min":2.8940000000000001},"last":1.522}]},{"osd":4,"last update":"Fri Nov 24 06:12:21 2023","interfaces":[{"interface":"back","average":{"1min":1.145,"5min":1.254,"15min":1.272},"min":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":1.454,"5min":2.1930000000000001,"15min":2.1930000000000001},"last":1.131},{"interface":"front","average":{"1min":1.1200000000000001,"5min":1.1839999999999999,"15min":1.194},"min":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":1.569,"5min":1.784,"15min":1.784},"last":1.405}]},{"osd":5,"last update":"Fri Nov 24 06:12:21 2023","interfaces":[{"interface":"back","average":{"1min":1.1499999999999999,"5min":1.454,"15min":1.504},"min":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":1.5820000000000001,"5min":5.5449999999999999,"15min":5.5449999999999999},"last":1.732},{"interface":"front","average":{"1min":1.167,"5min":1.484,"15min":1.5369999999999999},"min":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":1.538,"5min":6.0030000000000001,"15min":6.0030000000000001},"last":1.679}]},{"osd":6,"last update":"Fri Nov 24 06:12:21 2023","interfaces":[{"interface":"back","average":{"1min":1.2030000000000001,"5min":1.4950000000000001,"15min":1.544},"min":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":1.5189999999999999,"5min":5.2809999999999997,"15min":5.2809999999999997},"last":1.8779999999999999},{"interface":"front","average":{"1min":1.2849999999999999,"5min":1.4370000000000001,"15min":1.462},"min":{"1min":0.79300000000000004,"5min":0.749,"15min":0.749},"max":{"1min":1.7969999999999999,"5min":4.327,"15min":4.327},"last":1.5800000000000001}]},{"osd":7,"last update":"Fri Nov 24 06:12:21 2023","interfaces":[{"interface":"back","average":{"1min":1.173,"5min":1.321,"15min":1.3460000000000001},"min":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":1.8540000000000001,"5min":2.3069999999999999,"15min":2.3069999999999999},"last":1.6299999999999999},{"interface":"front","average":{"1min":1.252,"5min":1.528,"15min":1.5740000000000001},"min":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":1.6819999999999999,"5min":5.4500000000000002,"15min":5.4500000000000002},"last":1.8049999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574885,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32512,"kb_used_data":1328,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339328,"statfs":{"total":96636764160,"available":96603471872,"internally_reserved":0,"allocated":1359872,"data_stored":918615,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Nov 24 06:12:20 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.90500000000000003,"15min":0.92700000000000005},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":1.675,"5min":2.3999999999999999,"15min":2.3999999999999999},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.79900000000000004,"5min":1.1100000000000001,"15min":1.1619999999999999},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.403,"5min":4.8289999999999997,"15min":4.8289999999999997},"last":0.83299999999999996}]},{"osd":2,"last update":"Fri Nov 24 06:12:20 2023","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":1.0249999999999999,"15min":1.05},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.5960000000000001,"5min":2.5139999999999998,"15min":2.5139999999999998},"last":0.95499999999999996},{"interface":"front","average":{"1min":0.98199999999999998,"5min":1.0860000000000001,"15min":1.103},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.4390000000000001,"5min":2.3599999999999999,"15min":2.3599999999999999},"last":0.88}]},{"osd":3,"last update":"Fri Nov 24 06:12:20 2023","interfaces":[{"interface":"back","average":{"1min":1.018,"5min":1.1040000000000001,"15min":1.1180000000000001},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.514,"5min":2.5840000000000001,"15min":2.5840000000000001},"last":1.204},{"interface":"front","average":{"1min":1.071,"5min":1.3460000000000001,"15min":1.3919999999999999},"min":{"1min":0.503,"5min":0.495,"15min":0.495},"max":{"1min":1.526,"5min":5.1230000000000002,"15min":5.1230000000000002},"last":0.72699999999999998}]},{"osd":4,"last update":"Fri Nov 24 06:12:20 2023","interfaces":[{"interface":"back","average":{"1min":1.0209999999999999,"5min":1.23,"15min":1.2649999999999999},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.3919999999999999,"5min":2.887,"15min":2.887},"last":0.81799999999999995},{"interface":"front","average":{"1min":1.0760000000000001,"5min":1.284,"15min":1.3180000000000001},"min":{"1min":0.71599999999999997,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":1.6819999999999999,"5min":2.8530000000000002,"15min":2.8530000000000002},"last":1.0549999999999999}]},{"osd":5,"last update":"Fri Nov 24 06:12:20 2023","interfaces":[{"interface":"back","average":{"1min":1.2370000000000001,"5min":1.5069999999999999,"15min":1.552},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.804,"5min":4.6299999999999999,"15min":4.6299999999999999},"last":1.1220000000000001},{"interface":"front","average":{"1min":1.056,"5min":1.2769999999999999,"15min":1.3140000000000001},"min":{"1min":0.72699999999999998,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.3560000000000001,"5min":2.794,"15min":2.794},"last":0.91000000000000003}]},{"osd":6,"last update":"Fri Nov 24 06:12:20 2023","interfaces":[{"interface":"back","average":{"1min":1.2829999999999999,"5min":1.5940000000000001,"15min":1.6459999999999999},"min":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.6779999999999999,"5min":4.798,"15min":4.798},"last":1.032},{"interface":"front","average":{"1min":1.375,"5min":1.5840000000000001,"15min":1.619},"min":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"max":{"1min":1.9350000000000001,"5min":4.7149999999999999,"15min":4.7149999999999999},"last":1.2509999999999999}]},{"osd":7,"last update":"Fri Nov 24 06:12:20 2023","interfaces":[{"interface":"back","average":{"1min":1.3109999999999999,"5min":1.635,"15min":1.6890000000000001},"min":{"1min":0.73799999999999999,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.9750000000000001,"5min":4.7649999999999997,"15min":4.7649999999999997},"last":1.302},{"interface":"front","average":{"1min":1.421,"5min":1.6399999999999999,"15min":1.6759999999999999},"min":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"max":{"1min":2.0430000000000001,"5min":4.6820000000000004,"15min":4.6820000000000004},"last":1.341}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-24T06:13:14.099 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-11-24T06:13:14.099 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-11-24T06:13:14.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:14.366+0000 7faaa359e640 1 Processor -- start 2023-11-24T06:13:14.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:14.366+0000 7faaa359e640 1 -- start start 2023-11-24T06:13:14.370 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:14.368+0000 7faaa359e640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7faaa40b43f0 0x7faaa415f520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:14.370 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:14.368+0000 7faaa359e640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7faaa40b47d0 0x7faaa40ad8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:14.370 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:14.368+0000 7faaa359e640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7faaa415b520 0x7faaa40addf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:14.370 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:14.368+0000 7faaa359e640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faaa415fa60 con 0x7faaa40b43f0 2023-11-24T06:13:14.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:14.368+0000 7faaa359e640 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faaa40ae330 con 0x7faaa40b47d0 2023-11-24T06:13:14.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:14.368+0000 7faaa359e640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faaa40ceb80 con 0x7faaa415b520 2023-11-24T06:13:14.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:14.368+0000 7faaa259c640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7faaa40b43f0 0x7faaa415f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:14.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:14.368+0000 7faaa259c640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7faaa40b43f0 0x7faaa415f520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52810/0 (socket says 172.21.15.92:52810) 2023-11-24T06:13:14.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:14.368+0000 7faaa259c640 1 -- 172.21.15.92:0/1587746019 learned_addr learned my addr 172.21.15.92:0/1587746019 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:13:14.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:14.368+0000 7faaa2d9d640 1 --2- 172.21.15.92:0/1587746019 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7faaa415b520 0x7faaa40addf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:14.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:14.368+0000 7faa9bfff640 1 --2- 172.21.15.92:0/1587746019 >> v2:172.21.15.118:3300/0 conn(0x7faaa40b47d0 0x7faaa40ad8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:14.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:14.369+0000 7faaa259c640 1 -- 172.21.15.92:0/1587746019 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7faaa415b520 msgr2=0x7faaa40addf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:13:14.374 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:14.369+0000 7faaa259c640 1 --2- 172.21.15.92:0/1587746019 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7faaa415b520 0x7faaa40addf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:14.374 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:14.369+0000 7faaa259c640 1 -- 172.21.15.92:0/1587746019 >> v2:172.21.15.118:3300/0 conn(0x7faaa40b47d0 msgr2=0x7faaa40ad8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:13:14.374 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:14.369+0000 7faaa259c640 1 --2- 172.21.15.92:0/1587746019 >> v2:172.21.15.118:3300/0 conn(0x7faaa40b47d0 0x7faaa40ad8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:14.374 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:14.369+0000 7faaa259c640 1 -- 172.21.15.92:0/1587746019 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faaa415bac0 con 0x7faaa40b43f0 2023-11-24T06:13:14.374 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:14.369+0000 7faaa2d9d640 1 --2- 172.21.15.92:0/1587746019 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7faaa415b520 0x7faaa40addf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:13:14.375 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:14.369+0000 7faa9bfff640 1 --2- 172.21.15.92:0/1587746019 >> v2:172.21.15.118:3300/0 conn(0x7faaa40b47d0 0x7faaa40ad8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:13:14.375 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:14.370+0000 7faaa259c640 1 --2- 172.21.15.92:0/1587746019 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faaa40b43f0 0x7faaa415f520 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7faaa40ce290 tx=0x7faa9402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac4116939960bf04 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:14.375 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:14.370+0000 7faaa1d9b640 1 -- 172.21.15.92:0/1587746019 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa94031640 con 0x7faaa40b43f0 2023-11-24T06:13:14.375 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:14.370+0000 7faaa1d9b640 1 -- 172.21.15.92:0/1587746019 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa940317a0 con 0x7faaa40b43f0 2023-11-24T06:13:14.375 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:14.370+0000 7faaa1d9b640 1 -- 172.21.15.92:0/1587746019 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa94031b30 con 0x7faaa40b43f0 2023-11-24T06:13:14.376 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:14.371+0000 7faaa359e640 1 -- 172.21.15.92:0/1587746019 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faaa40b43f0 msgr2=0x7faaa415f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:14.376 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:14.371+0000 7faaa359e640 1 --2- 172.21.15.92:0/1587746019 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faaa40b43f0 0x7faaa415f520 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7faaa40ce290 tx=0x7faa9402b8f0 comp rx=0 tx=0).stop 2023-11-24T06:13:14.376 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:14.371+0000 7faaa359e640 1 -- 172.21.15.92:0/1587746019 shutdown_connections 2023-11-24T06:13:14.376 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:14.371+0000 7faaa359e640 1 --2- 172.21.15.92:0/1587746019 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7faaa415b520 0x7faaa40addf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:14.376 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:14.371+0000 7faaa359e640 1 --2- 172.21.15.92:0/1587746019 >> v2:172.21.15.118:3300/0 conn(0x7faaa40b47d0 0x7faaa40ad8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:14.376 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:14.371+0000 7faaa359e640 1 --2- 172.21.15.92:0/1587746019 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faaa40b43f0 0x7faaa415f520 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:14.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:14.371+0000 7faaa359e640 1 -- 172.21.15.92:0/1587746019 >> 172.21.15.92:0/1587746019 conn(0x7faaa40b0b70 msgr2=0x7faaa40b0f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:13:14.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:14.371+0000 7faaa359e640 1 -- 172.21.15.92:0/1587746019 shutdown_connections 2023-11-24T06:13:14.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:14.371+0000 7faaa359e640 1 -- 172.21.15.92:0/1587746019 wait complete. 2023-11-24T06:13:14.379 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:14.373+0000 7faaa359e640 1 Processor -- start 2023-11-24T06:13:14.379 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:14.373+0000 7faaa359e640 1 -- start start 2023-11-24T06:13:14.380 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:14.374+0000 7faaa359e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faaa40b43f0 0x7faaa41563d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:14.380 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:14.374+0000 7faaa359e640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7faaa40b47d0 0x7faaa413f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:14.380 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:14.374+0000 7faaa359e640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7faaa415b520 0x7faaa415a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:14.380 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:14.374+0000 7faaa359e640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faaa4159230 con 0x7faaa40b43f0 2023-11-24T06:13:14.380 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:14.374+0000 7faaa359e640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faaa41594d0 con 0x7faaa40b47d0 2023-11-24T06:13:14.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:14.374+0000 7faaa359e640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faaa41597b0 con 0x7faaa415b520 2023-11-24T06:13:14.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:14.374+0000 7faaa259c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faaa40b43f0 0x7faaa41563d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:14.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:14.374+0000 7faaa259c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faaa40b43f0 0x7faaa41563d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52814/0 (socket says 172.21.15.92:52814) 2023-11-24T06:13:14.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:14.374+0000 7faaa259c640 1 -- 172.21.15.92:0/1014363337 learned_addr learned my addr 172.21.15.92:0/1014363337 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:13:14.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:14.374+0000 7faaa259c640 1 -- 172.21.15.92:0/1014363337 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7faaa415b520 msgr2=0x7faaa415a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:14.382 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:14.374+0000 7faaa259c640 1 --2- 172.21.15.92:0/1014363337 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7faaa415b520 0x7faaa415a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:14.382 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:14.374+0000 7faaa259c640 1 -- 172.21.15.92:0/1014363337 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7faaa40b47d0 msgr2=0x7faaa413f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:14.382 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:14.374+0000 7faaa259c640 1 --2- 172.21.15.92:0/1014363337 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7faaa40b47d0 0x7faaa413f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:14.382 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:14.374+0000 7faaa259c640 1 -- 172.21.15.92:0/1014363337 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa940094d0 con 0x7faaa40b43f0 2023-11-24T06:13:14.382 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:14.375+0000 7faaa259c640 1 --2- 172.21.15.92:0/1014363337 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faaa40b43f0 0x7faaa41563d0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7faa94000c00 tx=0x7faa940046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:14.383 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:14.375+0000 7faa9affd640 1 -- 172.21.15.92:0/1014363337 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa9403ac60 con 0x7faaa40b43f0 2023-11-24T06:13:14.383 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:14.375+0000 7faa9affd640 1 -- 172.21.15.92:0/1014363337 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa9403ade0 con 0x7faaa40b43f0 2023-11-24T06:13:14.383 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:14.375+0000 7faaa359e640 1 -- 172.21.15.92:0/1014363337 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faaa4159a90 con 0x7faaa40b43f0 2023-11-24T06:13:14.383 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:14.375+0000 7faa9affd640 1 -- 172.21.15.92:0/1014363337 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa94044630 con 0x7faaa40b43f0 2023-11-24T06:13:14.383 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:14.375+0000 7faaa359e640 1 -- 172.21.15.92:0/1014363337 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faaa41421a0 con 0x7faaa40b43f0 2023-11-24T06:13:14.383 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:14.376+0000 7faaa359e640 1 -- 172.21.15.92:0/1014363337 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faaa4159fb0 con 0x7faaa40b43f0 2023-11-24T06:13:14.384 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:14.377+0000 7faa9affd640 1 -- 172.21.15.92:0/1014363337 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7faa9402f050 con 0x7faaa40b43f0 2023-11-24T06:13:14.384 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:14.378+0000 7faa9affd640 1 --2- 172.21.15.92:0/1014363337 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7faa8406aed0 0x7faa8406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:14.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:14.378+0000 7faa9affd640 1 -- 172.21.15.92:0/1014363337 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5709+0+0 (secure 0 0 0) 0x7faa940ab6e0 con 0x7faaa40b43f0 2023-11-24T06:13:14.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:14.378+0000 7faa9bfff640 1 --2- 172.21.15.92:0/1014363337 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7faa8406aed0 0x7faa8406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:14.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:14.379+0000 7faa9bfff640 1 --2- 172.21.15.92:0/1014363337 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7faa8406aed0 0x7faa8406d390 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7faaa41f6750 tx=0x7faa8c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:14.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:14.383+0000 7faa9affd640 1 -- 172.21.15.92:0/1014363337 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7faaa4159fb0 con 0x7faaa40b43f0 2023-11-24T06:13:14.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:14.638+0000 7faaa359e640 1 -- 172.21.15.92:0/1014363337 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7faaa4159fb0 con 0x7faaa40b43f0 2023-11-24T06:13:14.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:14.644+0000 7faa9affd640 1 -- 172.21.15.92:0/1014363337 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v11) v1 ==== 190+0+147 (secure 0 0 0) 0x7faaa4159fb0 con 0x7faaa40b43f0 2023-11-24T06:13:14.647 INFO:teuthology.orchestra.run.smithi092.stdout:[client.0] 2023-11-24T06:13:14.648 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQARPmBlGMUsDhAAlgrJvA/TT5hpUYd5o9OzIQ== 2023-11-24T06:13:14.648 INFO:teuthology.orchestra.run.smithi092.stdout: caps mds = "allow" 2023-11-24T06:13:14.648 INFO:teuthology.orchestra.run.smithi092.stdout: caps mon = "allow r" 2023-11-24T06:13:14.648 INFO:teuthology.orchestra.run.smithi092.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-11-24T06:13:14.648 INFO:teuthology.orchestra.run.smithi092.stderr:updated caps for client.0 2023-11-24T06:13:14.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:14.648+0000 7faaa359e640 1 -- 172.21.15.92:0/1014363337 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7faa8406aed0 msgr2=0x7faa8406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:14.652 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:14.648+0000 7faaa359e640 1 --2- 172.21.15.92:0/1014363337 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7faa8406aed0 0x7faa8406d390 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7faaa41f6750 tx=0x7faa8c008040 comp rx=0 tx=0).stop 2023-11-24T06:13:14.652 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:14.648+0000 7faaa359e640 1 -- 172.21.15.92:0/1014363337 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faaa40b43f0 msgr2=0x7faaa41563d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:14.652 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:14.648+0000 7faaa359e640 1 --2- 172.21.15.92:0/1014363337 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faaa40b43f0 0x7faaa41563d0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7faa94000c00 tx=0x7faa940046c0 comp rx=0 tx=0).stop 2023-11-24T06:13:14.652 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:14.648+0000 7faaa359e640 1 -- 172.21.15.92:0/1014363337 shutdown_connections 2023-11-24T06:13:14.652 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:14.648+0000 7faaa359e640 1 --2- 172.21.15.92:0/1014363337 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7faa8406aed0 0x7faa8406d390 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:14.652 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:14.648+0000 7faaa359e640 1 --2- 172.21.15.92:0/1014363337 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7faaa415b520 0x7faaa415a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:14.652 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:14.648+0000 7faaa359e640 1 --2- 172.21.15.92:0/1014363337 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7faaa40b47d0 0x7faaa413f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:14.652 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:14.648+0000 7faaa359e640 1 --2- 172.21.15.92:0/1014363337 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faaa40b43f0 0x7faaa41563d0 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:14.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:14.648+0000 7faaa359e640 1 -- 172.21.15.92:0/1014363337 >> 172.21.15.92:0/1014363337 conn(0x7faaa40b0b70 msgr2=0x7faaa406c3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:13:14.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:14.649+0000 7faaa359e640 1 -- 172.21.15.92:0/1014363337 shutdown_connections 2023-11-24T06:13:14.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:14.649+0000 7faaa359e640 1 -- 172.21.15.92:0/1014363337 wait complete. 2023-11-24T06:13:14.669 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T06:13:14.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:14.874+0000 7f70f8bef640 1 Processor -- start 2023-11-24T06:13:14.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:14.874+0000 7f70f8bef640 1 -- start start 2023-11-24T06:13:14.879 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:14.875+0000 7f70f8bef640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f70f413cfa0 0x7f70f415afb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:14.879 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:14.875+0000 7f70f8bef640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f70f40cd530 0x7f70f413f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:14.879 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:14.875+0000 7f70f8bef640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f70f40cd9b0 0x7f70f413fae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:14.879 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:14.875+0000 7f70f8bef640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70f40cdec0 con 0x7f70f40cd9b0 2023-11-24T06:13:14.879 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:14.875+0000 7f70f8bef640 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70f40ce1a0 con 0x7f70f40cd530 2023-11-24T06:13:14.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:14.876+0000 7f70f8bef640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70f40ce480 con 0x7f70f413cfa0 2023-11-24T06:13:14.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:14.876+0000 7f70f3fff640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f70f40cd9b0 0x7f70f413fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:14.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:14.876+0000 7f70f3fff640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f70f40cd9b0 0x7f70f413fae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52820/0 (socket says 172.21.15.92:52820) 2023-11-24T06:13:14.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:14.876+0000 7f70f3fff640 1 -- 172.21.15.92:0/204777162 learned_addr learned my addr 172.21.15.92:0/204777162 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:13:14.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:14.876+0000 7f70ebfff640 1 --2- 172.21.15.92:0/204777162 >> v2:172.21.15.118:3300/0 conn(0x7f70f40cd530 0x7f70f413f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:14.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:14.876+0000 7f70f37fe640 1 --2- 172.21.15.92:0/204777162 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f70f413cfa0 0x7f70f415afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:14.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:14.876+0000 7f70f3fff640 1 -- 172.21.15.92:0/204777162 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f70f413cfa0 msgr2=0x7f70f415afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:13:14.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:14.876+0000 7f70f3fff640 1 --2- 172.21.15.92:0/204777162 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f70f413cfa0 0x7f70f415afb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:14.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:14.876+0000 7f70f3fff640 1 -- 172.21.15.92:0/204777162 >> v2:172.21.15.118:3300/0 conn(0x7f70f40cd530 msgr2=0x7f70f413f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:13:14.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:14.876+0000 7f70f3fff640 1 --2- 172.21.15.92:0/204777162 >> v2:172.21.15.118:3300/0 conn(0x7f70f40cd530 0x7f70f413f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:14.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:14.876+0000 7f70f3fff640 1 -- 172.21.15.92:0/204777162 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70f40ce760 con 0x7f70f40cd9b0 2023-11-24T06:13:14.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:14.877+0000 7f70f3fff640 1 --2- 172.21.15.92:0/204777162 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f70f40cd9b0 0x7f70f413fae0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f70f406e2a0 tx=0x7f70ec02be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2332fc3243cb23db server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:14.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:14.878+0000 7f70f2ffd640 1 -- 172.21.15.92:0/204777162 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70ec02c9e0 con 0x7f70f40cd9b0 2023-11-24T06:13:14.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:14.878+0000 7f70f2ffd640 1 -- 172.21.15.92:0/204777162 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70ec02cb40 con 0x7f70f40cd9b0 2023-11-24T06:13:14.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:14.878+0000 7f70f2ffd640 1 -- 172.21.15.92:0/204777162 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70ec0343f0 con 0x7f70f40cd9b0 2023-11-24T06:13:14.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:14.878+0000 7f70f8bef640 1 -- 172.21.15.92:0/204777162 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f70f40cd9b0 msgr2=0x7f70f413fae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:14.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:14.878+0000 7f70f8bef640 1 --2- 172.21.15.92:0/204777162 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f70f40cd9b0 0x7f70f413fae0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f70f406e2a0 tx=0x7f70ec02be40 comp rx=0 tx=0).stop 2023-11-24T06:13:14.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:14.878+0000 7f70f8bef640 1 -- 172.21.15.92:0/204777162 shutdown_connections 2023-11-24T06:13:14.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:14.878+0000 7f70f8bef640 1 --2- 172.21.15.92:0/204777162 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f70f40cd9b0 0x7f70f413fae0 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:14.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:14.878+0000 7f70f8bef640 1 --2- 172.21.15.92:0/204777162 >> v2:172.21.15.118:3300/0 conn(0x7f70f40cd530 0x7f70f413f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:14.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:14.878+0000 7f70f8bef640 1 --2- 172.21.15.92:0/204777162 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f70f413cfa0 0x7f70f415afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:14.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:14.878+0000 7f70f8bef640 1 -- 172.21.15.92:0/204777162 >> 172.21.15.92:0/204777162 conn(0x7f70f406a800 msgr2=0x7f70f406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:13:14.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:14.879+0000 7f70f8bef640 1 -- 172.21.15.92:0/204777162 shutdown_connections 2023-11-24T06:13:14.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:14.879+0000 7f70f8bef640 1 -- 172.21.15.92:0/204777162 wait complete. 2023-11-24T06:13:14.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:14.880+0000 7f70f8bef640 1 Processor -- start 2023-11-24T06:13:14.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:14.880+0000 7f70f8bef640 1 -- start start 2023-11-24T06:13:14.885 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:14.881+0000 7f70f8bef640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f70f40cd530 0x7f70f4159070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:14.885 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:14.881+0000 7f70f8bef640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f70f40cd9b0 0x7f70f41f6ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:14.885 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:14.881+0000 7f70f8bef640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f70f413cfa0 0x7f70f41f7530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:14.885 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:14.881+0000 7f70f8bef640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70f41f4160 con 0x7f70f40cd530 2023-11-24T06:13:14.885 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:14.881+0000 7f70f8bef640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70f41f34e0 con 0x7f70f413cfa0 2023-11-24T06:13:14.885 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:14.881+0000 7f70f8bef640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70f41f37c0 con 0x7f70f40cd9b0 2023-11-24T06:13:14.885 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:14.882+0000 7f70f37fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f70f40cd530 0x7f70f4159070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:14.885 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:14.882+0000 7f70f37fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f70f40cd530 0x7f70f4159070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52822/0 (socket says 172.21.15.92:52822) 2023-11-24T06:13:14.885 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:14.882+0000 7f70f37fe640 1 -- 172.21.15.92:0/438899603 learned_addr learned my addr 172.21.15.92:0/438899603 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:13:14.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:14.882+0000 7f70f3fff640 1 --2- 172.21.15.92:0/438899603 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f70f413cfa0 0x7f70f41f7530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:14.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:14.882+0000 7f70f37fe640 1 -- 172.21.15.92:0/438899603 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f70f40cd9b0 msgr2=0x7f70f41f6ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:14.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:14.882+0000 7f70f37fe640 1 --2- 172.21.15.92:0/438899603 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f70f40cd9b0 0x7f70f41f6ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:14.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:14.882+0000 7f70f37fe640 1 -- 172.21.15.92:0/438899603 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f70f413cfa0 msgr2=0x7f70f41f7530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:14.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:14.882+0000 7f70f37fe640 1 --2- 172.21.15.92:0/438899603 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f70f413cfa0 0x7f70f41f7530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:14.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:14.882+0000 7f70f37fe640 1 -- 172.21.15.92:0/438899603 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70ec009b30 con 0x7f70f40cd530 2023-11-24T06:13:14.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:14.882+0000 7f70f3fff640 1 --2- 172.21.15.92:0/438899603 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f70f413cfa0 0x7f70f41f7530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:13:14.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:14.882+0000 7f70f37fe640 1 --2- 172.21.15.92:0/438899603 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f70f40cd530 0x7f70f4159070 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f70e400d5f0 tx=0x7f70e400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:14.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:14.883+0000 7f70f0ff9640 1 -- 172.21.15.92:0/438899603 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70e40125f0 con 0x7f70f40cd530 2023-11-24T06:13:14.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:14.883+0000 7f70f0ff9640 1 -- 172.21.15.92:0/438899603 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70e4012750 con 0x7f70f40cd530 2023-11-24T06:13:14.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:14.883+0000 7f70f0ff9640 1 -- 172.21.15.92:0/438899603 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70e4012b00 con 0x7f70f40cd530 2023-11-24T06:13:14.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:14.883+0000 7f70f8bef640 1 -- 172.21.15.92:0/438899603 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70f41f3aa0 con 0x7f70f40cd530 2023-11-24T06:13:14.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:14.883+0000 7f70f8bef640 1 -- 172.21.15.92:0/438899603 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70f41595b0 con 0x7f70f40cd530 2023-11-24T06:13:14.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:14.884+0000 7f70f8bef640 1 -- 172.21.15.92:0/438899603 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70f4067450 con 0x7f70f40cd530 2023-11-24T06:13:14.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:14.885+0000 7f70f0ff9640 1 -- 172.21.15.92:0/438899603 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f70e4012c60 con 0x7f70f40cd530 2023-11-24T06:13:14.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:14.886+0000 7f70f0ff9640 1 --2- 172.21.15.92:0/438899603 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f70d406aed0 0x7f70d406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:14.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:14.886+0000 7f70f0ff9640 1 -- 172.21.15.92:0/438899603 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f70e4089af0 con 0x7f70f40cd530 2023-11-24T06:13:14.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:14.886+0000 7f70ebfff640 1 --2- 172.21.15.92:0/438899603 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f70d406aed0 0x7f70d406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:14.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:14.887+0000 7f70ebfff640 1 --2- 172.21.15.92:0/438899603 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f70d406aed0 0x7f70d406d390 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f70f406b030 tx=0x7f70dc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:14.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:14.890+0000 7f70f0ff9640 1 -- 172.21.15.92:0/438899603 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f70e4054b40 con 0x7f70f40cd530 2023-11-24T06:13:15.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.117+0000 7f70f8bef640 1 -- 172.21.15.92:0/438899603 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f70f40b2a70 con 0x7f70f40cd530 2023-11-24T06:13:15.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.118+0000 7f70f0ff9640 1 -- 172.21.15.92:0/438899603 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) v1 ==== 94+0+4921 (secure 0 0 0) 0x7f70e4058160 con 0x7f70f40cd530 2023-11-24T06:13:15.121 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-24T06:13:15.121 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":40,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5143,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6833/2094733752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":2094733752},{"type":"v1","addr":"172.21.15.118:6833","nonce":2094733752}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":36},{"gid":5146,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6835/2873546779","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":2873546779},{"type":"v1","addr":"172.21.15.118:6835","nonce":2873546779}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":36},{"gid":6141,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6836/3335501795","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":3335501795},{"type":"v1","addr":"172.21.15.92:6836","nonce":3335501795}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":35}],"filesystems":[{"mdsmap":{"epoch":40,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T06:13:08.625588+0000","modified":"2023-11-24T06:13:11.658333+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6144},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6144":{"gid":6144,"name":"a","rank":0,"incarnation":37,"state":"up:active","state_seq":3,"addr":"172.21.15.92:6837/3913457833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":3913457833},{"type":"v1","addr":"172.21.15.92:6837","nonce":3913457833}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-11-24T06:13:15.122 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 40 2023-11-24T06:13:15.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.121+0000 7f70f8bef640 1 -- 172.21.15.92:0/438899603 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f70d406aed0 msgr2=0x7f70d406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:15.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.121+0000 7f70f8bef640 1 --2- 172.21.15.92:0/438899603 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f70d406aed0 0x7f70d406d390 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f70f406b030 tx=0x7f70dc008040 comp rx=0 tx=0).stop 2023-11-24T06:13:15.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.121+0000 7f70f8bef640 1 -- 172.21.15.92:0/438899603 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f70f40cd530 msgr2=0x7f70f4159070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:15.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.121+0000 7f70f8bef640 1 --2- 172.21.15.92:0/438899603 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f70f40cd530 0x7f70f4159070 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f70e400d5f0 tx=0x7f70e400dab0 comp rx=0 tx=0).stop 2023-11-24T06:13:15.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.121+0000 7f70f8bef640 1 -- 172.21.15.92:0/438899603 shutdown_connections 2023-11-24T06:13:15.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.121+0000 7f70f8bef640 1 --2- 172.21.15.92:0/438899603 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f70d406aed0 0x7f70d406d390 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:15.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.121+0000 7f70f8bef640 1 --2- 172.21.15.92:0/438899603 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f70f413cfa0 0x7f70f41f7530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:15.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.121+0000 7f70f8bef640 1 --2- 172.21.15.92:0/438899603 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f70f40cd9b0 0x7f70f41f6ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:15.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.121+0000 7f70f8bef640 1 --2- 172.21.15.92:0/438899603 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f70f40cd530 0x7f70f4159070 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:15.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.121+0000 7f70f8bef640 1 -- 172.21.15.92:0/438899603 >> 172.21.15.92:0/438899603 conn(0x7f70f406a800 msgr2=0x7f70f415a290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:13:15.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.121+0000 7f70f8bef640 1 -- 172.21.15.92:0/438899603 shutdown_connections 2023-11-24T06:13:15.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.121+0000 7f70f8bef640 1 -- 172.21.15.92:0/438899603 wait complete. 2023-11-24T06:13:15.545 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 40, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-24T06:13:08.625588+0000', 'modified': '2023-11-24T06:13:11.658333+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6144}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6144': {'gid': 6144, 'name': 'a', 'rank': 0, 'incarnation': 37, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.92:6837/3913457833', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6835', 'nonce': 3913457833}, {'type': 'v1', 'addr': '172.21.15.92:6837', 'nonce': 3913457833}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [11], 'metadata_pool': 10, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-24T06:13:15.545 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-24T06:13:15.546 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-11-24T06:13:15.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.765+0000 7fc3898cf640 1 Processor -- start 2023-11-24T06:13:15.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.765+0000 7fc3898cf640 1 -- start start 2023-11-24T06:13:15.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.767+0000 7fc3898cf640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fc38413e360 0x7fc3840cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:15.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.767+0000 7fc3898cf640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc38415f9c0 0x7fc3840cf7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:15.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.767+0000 7fc3898cf640 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc38415c5f0 con 0x7fc38413df80 2023-11-24T06:13:15.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.767+0000 7fc3898cf640 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc38406c470 con 0x7fc38413e360 2023-11-24T06:13:15.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.767+0000 7fc3898cf640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc38406c750 con 0x7fc38415f9c0 2023-11-24T06:13:15.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.768+0000 7fc3837fe640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc38415f9c0 0x7fc3840cf7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:15.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.768+0000 7fc3837fe640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc38415f9c0 0x7fc3840cf7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3301/0 says I am v2:172.21.15.92:47158/0 (socket says 172.21.15.92:47158) 2023-11-24T06:13:15.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.768+0000 7fc37a7fc640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fc38413e360 0x7fc3840cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:15.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.768+0000 7fc3837fe640 1 -- 172.21.15.92:0/1426220470 learned_addr learned my addr 172.21.15.92:0/1426220470 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:13:15.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.768+0000 7fc3837fe640 1 -- 172.21.15.92:0/1426220470 >> v2:172.21.15.118:3300/0 conn(0x7fc38413e360 msgr2=0x7fc3840cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:13:15.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.768+0000 7fc3837fe640 1 --2- 172.21.15.92:0/1426220470 >> v2:172.21.15.118:3300/0 conn(0x7fc38413e360 0x7fc3840cf290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:15.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.768+0000 7fc3837fe640 1 -- 172.21.15.92:0/1426220470 >> v1:172.21.15.118:6789/0 conn(0x7fc38413df80 legacy=0x7fc38415bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:15.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.768+0000 7fc3837fe640 1 -- 172.21.15.92:0/1426220470 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc38406ca30 con 0x7fc38415f9c0 2023-11-24T06:13:15.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.768+0000 7fc37a7fc640 1 --2- 172.21.15.92:0/1426220470 >> v2:172.21.15.118:3300/0 conn(0x7fc38413e360 0x7fc3840cf290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:13:15.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.770+0000 7fc3837fe640 1 --2- 172.21.15.92:0/1426220470 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc38415f9c0 0x7fc3840cf7d0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fc37c009e50 tx=0x7fc37c02bf50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6151e00a07518cf6 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:15.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.770+0000 7fc3827fc640 1 -- 172.21.15.92:0/1426220470 <== mon.2 v2:172.21.15.118:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc37c02caf0 con 0x7fc38415f9c0 2023-11-24T06:13:15.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.770+0000 7fc3827fc640 1 -- 172.21.15.92:0/1426220470 <== mon.2 v2:172.21.15.118:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc37c02cc50 con 0x7fc38415f9c0 2023-11-24T06:13:15.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.770+0000 7fc3827fc640 1 -- 172.21.15.92:0/1426220470 <== mon.2 v2:172.21.15.118:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc37c0343f0 con 0x7fc38415f9c0 2023-11-24T06:13:15.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.771+0000 7fc3898cf640 1 -- 172.21.15.92:0/1426220470 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc38415f9c0 msgr2=0x7fc3840cf7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:15.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.771+0000 7fc3898cf640 1 --2- 172.21.15.92:0/1426220470 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc38415f9c0 0x7fc3840cf7d0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fc37c009e50 tx=0x7fc37c02bf50 comp rx=0 tx=0).stop 2023-11-24T06:13:15.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.771+0000 7fc3898cf640 1 -- 172.21.15.92:0/1426220470 shutdown_connections 2023-11-24T06:13:15.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.771+0000 7fc3898cf640 1 --2- 172.21.15.92:0/1426220470 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc38415f9c0 0x7fc3840cf7d0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:15.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.771+0000 7fc3898cf640 1 --2- 172.21.15.92:0/1426220470 >> v2:172.21.15.118:3300/0 conn(0x7fc38413e360 0x7fc3840cf290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:15.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.771+0000 7fc3898cf640 1 -- 172.21.15.92:0/1426220470 >> 172.21.15.92:0/1426220470 conn(0x7fc38406a800 msgr2=0x7fc38406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:13:15.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.771+0000 7fc3898cf640 1 -- 172.21.15.92:0/1426220470 shutdown_connections 2023-11-24T06:13:15.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.771+0000 7fc3898cf640 1 -- 172.21.15.92:0/1426220470 wait complete. 2023-11-24T06:13:15.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.773+0000 7fc3898cf640 1 Processor -- start 2023-11-24T06:13:15.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.773+0000 7fc3898cf640 1 -- start start 2023-11-24T06:13:15.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.774+0000 7fc3898cf640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc38413df80 0x7fc384132620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:15.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.774+0000 7fc3898cf640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc38413e360 0x7fc3841575d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:15.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.774+0000 7fc3898cf640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc38415f9c0 0x7fc38415cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:15.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.774+0000 7fc3898cf640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3840aa2c0 con 0x7fc38413df80 2023-11-24T06:13:15.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.774+0000 7fc3898cf640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc384158890 con 0x7fc38415f9c0 2023-11-24T06:13:15.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.774+0000 7fc3898cf640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc384158b70 con 0x7fc38413e360 2023-11-24T06:13:15.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.774+0000 7fc382ffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc38413df80 0x7fc384132620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:15.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.774+0000 7fc382ffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc38413df80 0x7fc384132620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52832/0 (socket says 172.21.15.92:52832) 2023-11-24T06:13:15.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.774+0000 7fc382ffd640 1 -- 172.21.15.92:0/380163041 learned_addr learned my addr 172.21.15.92:0/380163041 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:13:15.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.775+0000 7fc37a7fc640 1 --2- 172.21.15.92:0/380163041 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc38413e360 0x7fc3841575d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:15.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.775+0000 7fc382ffd640 1 -- 172.21.15.92:0/380163041 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc38413e360 msgr2=0x7fc3841575d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:15.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.775+0000 7fc382ffd640 1 --2- 172.21.15.92:0/380163041 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc38413e360 0x7fc3841575d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:15.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.775+0000 7fc3837fe640 1 --2- 172.21.15.92:0/380163041 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc38415f9c0 0x7fc38415cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:15.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.775+0000 7fc382ffd640 1 -- 172.21.15.92:0/380163041 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc38415f9c0 msgr2=0x7fc38415cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:15.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.775+0000 7fc382ffd640 1 --2- 172.21.15.92:0/380163041 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc38415f9c0 0x7fc38415cbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:15.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.775+0000 7fc382ffd640 1 -- 172.21.15.92:0/380163041 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc36c009400 con 0x7fc38413df80 2023-11-24T06:13:15.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.775+0000 7fc382ffd640 1 --2- 172.21.15.92:0/380163041 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc38413df80 0x7fc384132620 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7fc37400c390 tx=0x7fc37400c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:15.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.775+0000 7fc380ff9640 1 -- 172.21.15.92:0/380163041 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3740052d0 con 0x7fc38413df80 2023-11-24T06:13:15.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.775+0000 7fc380ff9640 1 -- 172.21.15.92:0/380163041 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc374005430 con 0x7fc38413df80 2023-11-24T06:13:15.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.775+0000 7fc380ff9640 1 -- 172.21.15.92:0/380163041 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc374003830 con 0x7fc38413df80 2023-11-24T06:13:15.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.775+0000 7fc3898cf640 1 -- 172.21.15.92:0/380163041 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc37c009b30 con 0x7fc38413df80 2023-11-24T06:13:15.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.775+0000 7fc3898cf640 1 -- 172.21.15.92:0/380163041 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc384159130 con 0x7fc38413df80 2023-11-24T06:13:15.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.779+0000 7fc380ff9640 1 -- 172.21.15.92:0/380163041 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7fc374003990 con 0x7fc38413df80 2023-11-24T06:13:15.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.780+0000 7fc380ff9640 1 --2- 172.21.15.92:0/380163041 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fc36406aed0 0x7fc36406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:15.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.780+0000 7fc380ff9640 1 -- 172.21.15.92:0/380163041 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc374088c40 con 0x7fc38413df80 2023-11-24T06:13:15.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.780+0000 7fc37a7fc640 1 --2- 172.21.15.92:0/380163041 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fc36406aed0 0x7fc36406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:15.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.780+0000 7fc38c463740 1 Processor -- start 2023-11-24T06:13:15.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.781+0000 7fc38c463740 1 -- start start 2023-11-24T06:13:15.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.781+0000 7fc37a7fc640 1 --2- 172.21.15.92:0/380163041 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fc36406aed0 0x7fc36406d390 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fc38415db10 tx=0x7fc36c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:15.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.781+0000 7fc38c463740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e0475972b0 0x55e047597690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:15.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.781+0000 7fc38c463740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e047591fb0 0x55e0475964b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:15.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.781+0000 7fc38c463740 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x55e0475969f0 0x55e0475da970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:15.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.781+0000 7fc38c463740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0474f1880 con 0x55e047591fb0 2023-11-24T06:13:15.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.781+0000 7fc38c463740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e047500ae0 con 0x55e0475972b0 2023-11-24T06:13:15.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.781+0000 7fc38c463740 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e0474fe1d0 con 0x55e0475969f0 2023-11-24T06:13:15.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.782+0000 7fc37a7fc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e047591fb0 0x55e0475964b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:15.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.782+0000 7fc37a7fc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e047591fb0 0x55e0475964b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52840/0 (socket says 172.21.15.92:52840) 2023-11-24T06:13:15.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.782+0000 7fc37a7fc640 1 -- 172.21.15.92:0/51217915 learned_addr learned my addr 172.21.15.92:0/51217915 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:13:15.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.782+0000 7fc37a7fc640 1 -- 172.21.15.92:0/51217915 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x55e0475969f0 msgr2=0x55e0475da970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:13:15.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.782+0000 7fc37a7fc640 1 --2- 172.21.15.92:0/51217915 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x55e0475969f0 0x55e0475da970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:15.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.782+0000 7fc37a7fc640 1 -- 172.21.15.92:0/51217915 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e0475972b0 msgr2=0x55e047597690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:13:15.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.782+0000 7fc382ffd640 1 --2- 172.21.15.92:0/51217915 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e0475972b0 0x55e047597690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:15.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.782+0000 7fc37a7fc640 1 --2- 172.21.15.92:0/51217915 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e0475972b0 0x55e047597690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:15.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.782+0000 7fc37a7fc640 1 -- 172.21.15.92:0/51217915 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e0474c1860 con 0x55e047591fb0 2023-11-24T06:13:15.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.782+0000 7fc382ffd640 1 --2- 172.21.15.92:0/51217915 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e0475972b0 0x55e047597690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:13:15.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.783+0000 7fc37a7fc640 1 --2- 172.21.15.92:0/51217915 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e047591fb0 0x55e0475964b0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7fc36c010bd0 tx=0x7fc36c013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fba2c286069aa0fc server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:15.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.783+0000 7fc3797fa640 1 -- 172.21.15.92:0/51217915 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc36c016620 con 0x55e047591fb0 2023-11-24T06:13:15.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.783+0000 7fc3797fa640 1 -- 172.21.15.92:0/51217915 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc36c016780 con 0x55e047591fb0 2023-11-24T06:13:15.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.783+0000 7fc3797fa640 1 -- 172.21.15.92:0/51217915 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc36c016b30 con 0x55e047591fb0 2023-11-24T06:13:15.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.783+0000 7fc38c463740 1 -- 172.21.15.92:0/51217915 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e047591fb0 msgr2=0x55e0475964b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:15.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.783+0000 7fc38c463740 1 --2- 172.21.15.92:0/51217915 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e047591fb0 0x55e0475964b0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7fc36c010bd0 tx=0x7fc36c013000 comp rx=0 tx=0).stop 2023-11-24T06:13:15.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.784+0000 7fc38c463740 1 -- 172.21.15.92:0/51217915 shutdown_connections 2023-11-24T06:13:15.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.784+0000 7fc38c463740 1 --2- 172.21.15.92:0/51217915 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x55e0475969f0 0x55e0475da970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:15.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.784+0000 7fc38c463740 1 --2- 172.21.15.92:0/51217915 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e047591fb0 0x55e0475964b0 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:15.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.784+0000 7fc38c463740 1 --2- 172.21.15.92:0/51217915 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e0475972b0 0x55e047597690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:15.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.784+0000 7fc38c463740 1 -- 172.21.15.92:0/51217915 >> 172.21.15.92:0/51217915 conn(0x55e047554c70 msgr2=0x55e0475dcda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:13:15.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.784+0000 7fc38c463740 1 -- 172.21.15.92:0/51217915 shutdown_connections 2023-11-24T06:13:15.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.784+0000 7fc38c463740 1 -- 172.21.15.92:0/51217915 wait complete. 2023-11-24T06:13:15.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.784+0000 7fc38c463740 1 -- start start 2023-11-24T06:13:15.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.785+0000 7fc38c463740 1 Processor -- start 2023-11-24T06:13:15.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.786+0000 7fc38c463740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e047591fb0 0x55e0475c0750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:15.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.786+0000 7fc38c463740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e0475969f0 0x55e0475e1a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:15.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.786+0000 7fc38c463740 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x55e0475972b0 0x55e0475c10d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:15.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.786+0000 7fc38c463740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0474f9ac0 con 0x55e047591fb0 2023-11-24T06:13:15.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.786+0000 7fc38c463740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0474962f0 con 0x55e0475969f0 2023-11-24T06:13:15.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.786+0000 7fc38c463740 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e0474b3c70 con 0x55e0475972b0 2023-11-24T06:13:15.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.786+0000 7fc38c463740 10 client.0 _ll_register_callbacks cb 0x55e0474b83b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-24T06:13:15.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.786+0000 7fc382ffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e047591fb0 0x55e0475c0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:15.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.786+0000 7fc382ffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e047591fb0 0x55e0475c0750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52848/0 (socket says 172.21.15.92:52848) 2023-11-24T06:13:15.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.786+0000 7fc382ffd640 1 -- 172.21.15.92:0/1261127629 learned_addr learned my addr 172.21.15.92:0/1261127629 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:13:15.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.786+0000 7fc37a7fc640 1 --2- 172.21.15.92:0/1261127629 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e0475969f0 0x55e0475e1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:15.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.786+0000 7fc382ffd640 1 -- 172.21.15.92:0/1261127629 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x55e0475972b0 msgr2=0x55e0475c10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:13:15.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.786+0000 7fc382ffd640 1 --2- 172.21.15.92:0/1261127629 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x55e0475972b0 0x55e0475c10d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:15.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.786+0000 7fc382ffd640 1 -- 172.21.15.92:0/1261127629 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e0475969f0 msgr2=0x55e0475e1a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:13:15.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.786+0000 7fc382ffd640 1 --2- 172.21.15.92:0/1261127629 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e0475969f0 0x55e0475e1a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:15.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.786+0000 7fc382ffd640 1 -- 172.21.15.92:0/1261127629 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc36c00e050 con 0x55e047591fb0 2023-11-24T06:13:15.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.787+0000 7fc382ffd640 1 --2- 172.21.15.92:0/1261127629 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e047591fb0 0x55e0475c0750 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fc37400cd50 tx=0x7fc374057a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ab1f6db48c51194 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:15.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.787+0000 7fc3627fc640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-11-24T06:13:15.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.787+0000 7fc3627fc640 1 -- 172.21.15.92:0/1261127629 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc374054060 con 0x55e047591fb0 2023-11-24T06:13:15.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.787+0000 7fc3627fc640 1 -- 172.21.15.92:0/1261127629 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(63..63 src has 1..63) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc3740541c0 con 0x55e047591fb0 2023-11-24T06:13:15.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.787+0000 7fc38c463740 1 -- 172.21.15.92:0/1261127629 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e0474f2aa0 con 0x55e047591fb0 2023-11-24T06:13:15.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.787+0000 7fc3627fc640 1 -- 172.21.15.92:0/1261127629 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc37405f7c0 con 0x55e047591fb0 2023-11-24T06:13:15.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.787+0000 7fc38c463740 10 client.6297 fetch_fsmap learned FSMap version 40 2023-11-24T06:13:15.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.787+0000 7fc38c463740 1 -- 172.21.15.92:0/1261127629 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=2+}) v3 -- 0x55e0474f8140 con 0x55e047591fb0 2023-11-24T06:13:15.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.787+0000 7fc3627fc640 1 -- 172.21.15.92:0/1261127629 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc374073020 con 0x55e047591fb0 2023-11-24T06:13:15.794 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.787+0000 7fc3627fc640 1 -- 172.21.15.92:0/1261127629 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 40) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fc374053740 con 0x55e047591fb0 2023-11-24T06:13:15.794 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.788+0000 7fc38c463740 10 client.6297 fetch_fsmap finished waiting for FSMap version 40 2023-11-24T06:13:15.794 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.788+0000 7fc38c463740 10 client.6297 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-24T06:13:15.794 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.788+0000 7fc38c463740 20 client.6297 populate_metadata read hostname 'smithi092' 2023-11-24T06:13:15.794 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.788+0000 7fc38c463740 1 --2- 172.21.15.92:0/1261127629 >> [v2:172.21.15.92:6835/3913457833,v1:172.21.15.92:6837/3913457833] conn(0x55e0475e20c0 0x55e0475f9eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:15.794 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.788+0000 7fc38c463740 4 client.6297 mds_command: new command op to 6144 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-24T06:13:15.794 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.788+0000 7fc38c463740 1 -- 172.21.15.92:0/1261127629 --> [v2:172.21.15.92:6835/3913457833,v1:172.21.15.92:6837/3913457833] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e0474f27c0 con 0x55e0475e20c0 2023-11-24T06:13:15.794 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.788+0000 7fc37a7fc640 1 --2- 172.21.15.92:0/1261127629 >> [v2:172.21.15.92:6835/3913457833,v1:172.21.15.92:6837/3913457833] conn(0x55e0475e20c0 0x55e0475f9eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:15.794 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.788+0000 7fc37a7fc640 1 --2- 172.21.15.92:0/1261127629 >> [v2:172.21.15.92:6835/3913457833,v1:172.21.15.92:6837/3913457833] conn(0x55e0475e20c0 0x55e0475f9eb0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=327c37c1a870d984 server_cookie=fc2941ac808c1627 in_seq=0 out_seq=0 2023-11-24T06:13:15.794 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.788+0000 7fc3627fc640 10 client.6297 ms_handle_connect on v2:172.21.15.92:6835/3913457833 2023-11-24T06:13:15.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.789+0000 7fc3627fc640 1 -- 172.21.15.92:0/1261127629 <== mds.0 v2:172.21.15.92:6835/3913457833 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e0474f27c0 con 0x55e0475e20c0 2023-11-24T06:13:15.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.789+0000 7fc3627fc640 10 client.6297 handle_command_reply: tid=0 2023-11-24T06:13:15.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.789+0000 7fc38c463740 1 client.6297 shutdown 2023-11-24T06:13:15.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.790+0000 7fc38c463740 1 -- 172.21.15.92:0/1261127629 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e047591fb0 msgr2=0x55e0475c0750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:15.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.790+0000 7fc38c463740 1 --2- 172.21.15.92:0/1261127629 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e047591fb0 0x55e0475c0750 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fc37400cd50 tx=0x7fc374057a10 comp rx=0 tx=0).stop 2023-11-24T06:13:15.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.790+0000 7fc38c463740 1 -- 172.21.15.92:0/1261127629 shutdown_connections 2023-11-24T06:13:15.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.790+0000 7fc38c463740 1 --2- 172.21.15.92:0/1261127629 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x55e0475972b0 0x55e0475c10d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:15.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.790+0000 7fc38c463740 1 --2- 172.21.15.92:0/1261127629 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e0475969f0 0x55e0475e1a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:15.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.790+0000 7fc38c463740 1 --2- 172.21.15.92:0/1261127629 >> [v2:172.21.15.92:6835/3913457833,v1:172.21.15.92:6837/3913457833] conn(0x55e0475e20c0 0x55e0475f9eb0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:15.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.790+0000 7fc38c463740 1 --2- 172.21.15.92:0/1261127629 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e047591fb0 0x55e0475c0750 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:15.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.790+0000 7fc38c463740 1 -- 172.21.15.92:0/1261127629 >> 172.21.15.92:0/1261127629 conn(0x55e047554c70 msgr2=0x55e04758cb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:13:15.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.790+0000 7fc3627fc640 0 client.6297 ms_handle_reset on v2:172.21.15.92:6835/3913457833 2023-11-24T06:13:15.797 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.790+0000 7fc38c463740 1 -- 172.21.15.92:0/1261127629 shutdown_connections 2023-11-24T06:13:15.797 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.790+0000 7fc38c463740 1 -- 172.21.15.92:0/1261127629 wait complete. 2023-11-24T06:13:15.797 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.790+0000 7fc38c463740 20 client.6297 trim_cache size 0 max 16384 2023-11-24T06:13:15.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.807+0000 7fc38c463740 1 Processor -- start 2023-11-24T06:13:15.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.807+0000 7fc38c463740 1 -- start start 2023-11-24T06:13:15.811 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.808+0000 7fc38c463740 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x55e047591fb0 0x55e0475e4a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:15.811 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.808+0000 7fc38c463740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e0475969f0 0x55e04758d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:15.811 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.808+0000 7fc38c463740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e0475972b0 0x55e047595ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:15.811 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.808+0000 7fc38c463740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e04725cf20 con 0x55e0475969f0 2023-11-24T06:13:15.811 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.808+0000 7fc38c463740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0475bbc80 con 0x55e0475972b0 2023-11-24T06:13:15.811 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.808+0000 7fc38c463740 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e0475bf180 con 0x55e047591fb0 2023-11-24T06:13:15.812 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.808+0000 7fc37a7fc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e0475969f0 0x55e04758d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:15.812 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.808+0000 7fc37a7fc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e0475969f0 0x55e04758d1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52856/0 (socket says 172.21.15.92:52856) 2023-11-24T06:13:15.812 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.808+0000 7fc37a7fc640 1 -- 172.21.15.92:0/3996369561 learned_addr learned my addr 172.21.15.92:0/3996369561 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:13:15.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.809+0000 7fc37a7fc640 1 -- 172.21.15.92:0/3996369561 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x55e047591fb0 msgr2=0x55e0475e4a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:13:15.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.809+0000 7fc37a7fc640 1 --2- 172.21.15.92:0/3996369561 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x55e047591fb0 0x55e0475e4a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:15.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.809+0000 7fc37a7fc640 1 -- 172.21.15.92:0/3996369561 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e0475972b0 msgr2=0x55e047595ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:13:15.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.809+0000 7fc37a7fc640 1 --2- 172.21.15.92:0/3996369561 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e0475972b0 0x55e047595ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:15.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.809+0000 7fc37a7fc640 1 -- 172.21.15.92:0/3996369561 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e047495b80 con 0x55e0475969f0 2023-11-24T06:13:15.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.809+0000 7fc37a7fc640 1 --2- 172.21.15.92:0/3996369561 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e0475969f0 0x55e04758d1c0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fc36c024040 tx=0x7fc36c015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf7775181976863c server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:15.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.809+0000 7fc379ffb640 1 -- 172.21.15.92:0/3996369561 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc36c0168f0 con 0x55e0475969f0 2023-11-24T06:13:15.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.809+0000 7fc379ffb640 1 -- 172.21.15.92:0/3996369561 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc36c016a50 con 0x55e0475969f0 2023-11-24T06:13:15.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.809+0000 7fc379ffb640 1 -- 172.21.15.92:0/3996369561 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc36c016dc0 con 0x55e0475969f0 2023-11-24T06:13:15.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.810+0000 7fc38c463740 1 -- 172.21.15.92:0/3996369561 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e0475969f0 msgr2=0x55e04758d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:15.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.810+0000 7fc38c463740 1 --2- 172.21.15.92:0/3996369561 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e0475969f0 0x55e04758d1c0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fc36c024040 tx=0x7fc36c015000 comp rx=0 tx=0).stop 2023-11-24T06:13:15.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.810+0000 7fc38c463740 1 -- 172.21.15.92:0/3996369561 shutdown_connections 2023-11-24T06:13:15.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.810+0000 7fc38c463740 1 --2- 172.21.15.92:0/3996369561 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e0475972b0 0x55e047595ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:15.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.810+0000 7fc38c463740 1 --2- 172.21.15.92:0/3996369561 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e0475969f0 0x55e04758d1c0 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:15.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.810+0000 7fc38c463740 1 --2- 172.21.15.92:0/3996369561 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x55e047591fb0 0x55e0475e4a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:15.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.810+0000 7fc38c463740 1 -- 172.21.15.92:0/3996369561 >> 172.21.15.92:0/3996369561 conn(0x55e047554c70 msgr2=0x55e047597690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:13:15.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.810+0000 7fc38c463740 1 -- 172.21.15.92:0/3996369561 shutdown_connections 2023-11-24T06:13:15.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.810+0000 7fc38c463740 1 -- 172.21.15.92:0/3996369561 wait complete. 2023-11-24T06:13:15.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.811+0000 7fc38c463740 1 -- start start 2023-11-24T06:13:15.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.811+0000 7fc38c463740 1 Processor -- start 2023-11-24T06:13:15.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.812+0000 7fc38c463740 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x55e047591fb0 0x55e0475da080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:15.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.812+0000 7fc38c463740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e0475969f0 0x55e047647790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:15.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.812+0000 7fc38c463740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e0475972b0 0x55e0475d8760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:15.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.812+0000 7fc38c463740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0475e18a0 con 0x55e0475972b0 2023-11-24T06:13:15.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.812+0000 7fc38c463740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e047647cd0 con 0x55e0475969f0 2023-11-24T06:13:15.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.812+0000 7fc38c463740 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e0475f9f50 con 0x55e047591fb0 2023-11-24T06:13:15.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.812+0000 7fc38c463740 10 client.0 _ll_register_callbacks cb 0x55e0475e5570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-24T06:13:15.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.812+0000 7fc3837fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e0475972b0 0x55e0475d8760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:15.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.812+0000 7fc3837fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e0475972b0 0x55e0475d8760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52864/0 (socket says 172.21.15.92:52864) 2023-11-24T06:13:15.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.812+0000 7fc3837fe640 1 -- 172.21.15.92:0/940962817 learned_addr learned my addr 172.21.15.92:0/940962817 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:13:15.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.812+0000 7fc37a7fc640 1 --2- 172.21.15.92:0/940962817 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e0475969f0 0x55e047647790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:15.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.812+0000 7fc382ffd640 1 --2- 172.21.15.92:0/940962817 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x55e047591fb0 0x55e0475da080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:15.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.813+0000 7fc3837fe640 1 -- 172.21.15.92:0/940962817 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x55e047591fb0 msgr2=0x55e0475da080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:13:15.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.813+0000 7fc3837fe640 1 --2- 172.21.15.92:0/940962817 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x55e047591fb0 0x55e0475da080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:15.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.813+0000 7fc3837fe640 1 -- 172.21.15.92:0/940962817 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e0475969f0 msgr2=0x55e047647790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:13:15.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.813+0000 7fc3837fe640 1 --2- 172.21.15.92:0/940962817 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e0475969f0 0x55e047647790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:15.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.813+0000 7fc3837fe640 1 -- 172.21.15.92:0/940962817 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e0474c1860 con 0x55e0475972b0 2023-11-24T06:13:15.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.813+0000 7fc37a7fc640 1 --2- 172.21.15.92:0/940962817 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e0475969f0 0x55e047647790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:13:15.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.813+0000 7fc3837fe640 1 --2- 172.21.15.92:0/940962817 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e0475972b0 0x55e0475d8760 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fc37c009e50 tx=0x7fc37c034620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b45bcdef9ceddbec server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:15.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.813+0000 7fc3627fc640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-11-24T06:13:15.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.813+0000 7fc3627fc640 1 -- 172.21.15.92:0/940962817 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc37c034ba0 con 0x55e0475972b0 2023-11-24T06:13:15.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.813+0000 7fc3627fc640 1 -- 172.21.15.92:0/940962817 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(63..63 src has 1..63) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc37c034d00 con 0x55e0475972b0 2023-11-24T06:13:15.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.813+0000 7fc38c463740 1 -- 172.21.15.92:0/940962817 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc36c016780 con 0x55e0475972b0 2023-11-24T06:13:15.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.814+0000 7fc3627fc640 1 -- 172.21.15.92:0/940962817 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc37c040470 con 0x55e0475972b0 2023-11-24T06:13:15.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.814+0000 7fc38c463740 10 client.6303 fetch_fsmap learned FSMap version 40 2023-11-24T06:13:15.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.814+0000 7fc38c463740 1 -- 172.21.15.92:0/940962817 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=2+}) v3 -- 0x55e0474bb170 con 0x55e0475972b0 2023-11-24T06:13:15.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.814+0000 7fc3627fc640 1 -- 172.21.15.92:0/940962817 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc36c016780 con 0x55e0475972b0 2023-11-24T06:13:15.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.814+0000 7fc3627fc640 1 -- 172.21.15.92:0/940962817 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 40) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fc37c03c040 con 0x55e0475972b0 2023-11-24T06:13:15.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.814+0000 7fc38c463740 10 client.6303 fetch_fsmap finished waiting for FSMap version 40 2023-11-24T06:13:15.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.814+0000 7fc38c463740 10 client.6303 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-24T06:13:15.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.814+0000 7fc38c463740 20 client.6303 populate_metadata read hostname 'smithi092' 2023-11-24T06:13:15.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.814+0000 7fc38c463740 1 --2- 172.21.15.92:0/940962817 >> [v2:172.21.15.92:6835/3913457833,v1:172.21.15.92:6837/3913457833] conn(0x55e0475fbb40 0x55e0475fbf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:15.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.814+0000 7fc38c463740 4 client.6303 mds_command: new command op to 6144 tid=0 multi_id=0 [{"prefix": "status"}] 2023-11-24T06:13:15.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.814+0000 7fc38c463740 1 -- 172.21.15.92:0/940962817 --> [v2:172.21.15.92:6835/3913457833,v1:172.21.15.92:6837/3913457833] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55e0471fa6c0 con 0x55e0475fbb40 2023-11-24T06:13:15.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.814+0000 7fc382ffd640 1 --2- 172.21.15.92:0/940962817 >> [v2:172.21.15.92:6835/3913457833,v1:172.21.15.92:6837/3913457833] conn(0x55e0475fbb40 0x55e0475fbf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:15.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.815+0000 7fc382ffd640 1 --2- 172.21.15.92:0/940962817 >> [v2:172.21.15.92:6835/3913457833,v1:172.21.15.92:6837/3913457833] conn(0x55e0475fbb40 0x55e0475fbf20 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c8ef843ca60a9bb server_cookie=ebe310a06128b3fe in_seq=0 out_seq=0 2023-11-24T06:13:15.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.815+0000 7fc3627fc640 10 client.6303 ms_handle_connect on v2:172.21.15.92:6835/3913457833 2023-11-24T06:13:15.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.815+0000 7fc3627fc640 1 -- 172.21.15.92:0/940962817 <== mds.0 v2:172.21.15.92:6835/3913457833 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x55e0471fa6c0 con 0x55e0475fbb40 2023-11-24T06:13:15.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.815+0000 7fc3627fc640 10 client.6303 handle_command_reply: tid=0 2023-11-24T06:13:15.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.815+0000 7fc38c463740 1 client.6303 shutdown 2023-11-24T06:13:15.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.815+0000 7fc38c463740 1 -- 172.21.15.92:0/940962817 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e0475972b0 msgr2=0x55e0475d8760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:15.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.815+0000 7fc38c463740 1 --2- 172.21.15.92:0/940962817 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e0475972b0 0x55e0475d8760 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fc37c009e50 tx=0x7fc37c034620 comp rx=0 tx=0).stop 2023-11-24T06:13:15.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.815+0000 7fc38c463740 1 -- 172.21.15.92:0/940962817 shutdown_connections 2023-11-24T06:13:15.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.815+0000 7fc38c463740 1 --2- 172.21.15.92:0/940962817 >> [v2:172.21.15.92:6835/3913457833,v1:172.21.15.92:6837/3913457833] conn(0x55e0475fbb40 0x55e0475fbf20 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:15.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.815+0000 7fc38c463740 1 --2- 172.21.15.92:0/940962817 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e0475972b0 0x55e0475d8760 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:15.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.815+0000 7fc38c463740 1 --2- 172.21.15.92:0/940962817 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e0475969f0 0x55e047647790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:15.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.815+0000 7fc3627fc640 0 client.6303 ms_handle_reset on v2:172.21.15.92:6835/3913457833 2023-11-24T06:13:15.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.815+0000 7fc38c463740 1 --2- 172.21.15.92:0/940962817 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x55e047591fb0 0x55e0475da080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:15.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.815+0000 7fc38c463740 1 -- 172.21.15.92:0/940962817 >> 172.21.15.92:0/940962817 conn(0x55e047554c70 msgr2=0x55e0475e4840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:13:15.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.816+0000 7fc38c463740 1 -- 172.21.15.92:0/940962817 shutdown_connections 2023-11-24T06:13:15.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.816+0000 7fc38c463740 1 -- 172.21.15.92:0/940962817 wait complete. 2023-11-24T06:13:15.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.816+0000 7fc38c463740 20 client.6303 trim_cache size 0 max 16384 2023-11-24T06:13:15.826 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-24T06:13:15.826 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_fsid": "2d6dee5f-8aa3-470f-931a-50ade963b1cb", 2023-11-24T06:13:15.826 INFO:teuthology.orchestra.run.smithi092.stdout: "whoami": 0, 2023-11-24T06:13:15.826 INFO:teuthology.orchestra.run.smithi092.stdout: "id": 6144, 2023-11-24T06:13:15.826 INFO:teuthology.orchestra.run.smithi092.stdout: "want_state": "up:active", 2023-11-24T06:13:15.826 INFO:teuthology.orchestra.run.smithi092.stdout: "state": "up:active", 2023-11-24T06:13:15.827 INFO:teuthology.orchestra.run.smithi092.stdout: "fs_name": "cephfs", 2023-11-24T06:13:15.827 INFO:teuthology.orchestra.run.smithi092.stdout: "rank_uptime": 6.6493746040000001, 2023-11-24T06:13:15.827 INFO:teuthology.orchestra.run.smithi092.stdout: "mdsmap_epoch": 40, 2023-11-24T06:13:15.827 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_epoch": 63, 2023-11-24T06:13:15.827 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_epoch_barrier": 63, 2023-11-24T06:13:15.827 INFO:teuthology.orchestra.run.smithi092.stdout: "uptime": 12.064354137 2023-11-24T06:13:15.827 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-24T06:13:15.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.816+0000 7fc3898cf640 1 -- 172.21.15.92:0/380163041 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fc36406aed0 msgr2=0x7fc36406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:15.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.816+0000 7fc3898cf640 1 --2- 172.21.15.92:0/380163041 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fc36406aed0 0x7fc36406d390 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fc38415db10 tx=0x7fc36c008040 comp rx=0 tx=0).stop 2023-11-24T06:13:15.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.816+0000 7fc3898cf640 1 -- 172.21.15.92:0/380163041 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc38413df80 msgr2=0x7fc384132620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:15.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.816+0000 7fc3898cf640 1 --2- 172.21.15.92:0/380163041 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc38413df80 0x7fc384132620 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7fc37400c390 tx=0x7fc37400c850 comp rx=0 tx=0).stop 2023-11-24T06:13:15.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.816+0000 7fc3898cf640 1 -- 172.21.15.92:0/380163041 shutdown_connections 2023-11-24T06:13:15.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.816+0000 7fc3898cf640 1 --2- 172.21.15.92:0/380163041 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fc36406aed0 0x7fc36406d390 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:15.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.816+0000 7fc3898cf640 1 --2- 172.21.15.92:0/380163041 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc38415f9c0 0x7fc38415cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:15.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.816+0000 7fc3898cf640 1 --2- 172.21.15.92:0/380163041 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc38413e360 0x7fc3841575d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:15.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.816+0000 7fc3898cf640 1 --2- 172.21.15.92:0/380163041 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc38413df80 0x7fc384132620 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:15.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.816+0000 7fc3898cf640 1 -- 172.21.15.92:0/380163041 >> 172.21.15.92:0/380163041 conn(0x7fc38406a800 msgr2=0x7fc38415e010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:13:15.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.817+0000 7fc3898cf640 1 -- 172.21.15.92:0/380163041 shutdown_connections 2023-11-24T06:13:15.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:15.817+0000 7fc3898cf640 1 -- 172.21.15.92:0/380163041 wait complete. 2023-11-24T06:13:15.836 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-24T06:13:16.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:16.042+0000 7f62f0859640 1 Processor -- start 2023-11-24T06:13:16.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:16.042+0000 7f62f0859640 1 -- start start 2023-11-24T06:13:16.047 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:16.044+0000 7f62f0859640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f62ec0cc990 0x7f62ec06d870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:16.047 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:16.044+0000 7f62f0859640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62ec070e50 con 0x7f62ec0cc500 2023-11-24T06:13:16.047 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:16.044+0000 7f62f0859640 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62ec144020 con 0x7f62ec143c40 2023-11-24T06:13:16.047 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:16.044+0000 7f62f0859640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62ec0723c0 con 0x7f62ec0cc990 2023-11-24T06:13:16.047 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:16.044+0000 7f62e3fff640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f62ec0cc500 0x7f62ec06d160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:59352/0 (socket says 172.21.15.92:59352) 2023-11-24T06:13:16.047 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:16.044+0000 7f62e3fff640 1 -- 172.21.15.92:0/208348872 learned_addr learned my addr 172.21.15.92:0/208348872 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-24T06:13:16.049 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:16.044+0000 7f62ebfff640 1 --2- 172.21.15.92:0/208348872 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f62ec0cc990 0x7f62ec06d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:16.049 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:16.045+0000 7f62eaffd640 1 -- 172.21.15.92:0/208348872 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 812987448 0 0) 0x7f62ec070e50 con 0x7f62ec0cc500 2023-11-24T06:13:16.049 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:16.045+0000 7f62eaffd640 1 -- 172.21.15.92:0/208348872 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f62d8003520 con 0x7f62ec0cc500 2023-11-24T06:13:16.050 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:16.045+0000 7f62eaffd640 1 -- 172.21.15.92:0/208348872 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1958635070 0 0) 0x7f62d8003520 con 0x7f62ec0cc500 2023-11-24T06:13:16.050 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:16.045+0000 7f62eaffd640 1 -- 172.21.15.92:0/208348872 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f62ec0cc990 msgr2=0x7f62ec06d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:13:16.050 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:16.045+0000 7f62eaffd640 1 --2- 172.21.15.92:0/208348872 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f62ec0cc990 0x7f62ec06d870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:16.050 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:16.045+0000 7f62eaffd640 1 -- 172.21.15.92:0/208348872 >> v1:172.21.15.118:6789/0 conn(0x7f62ec143c40 legacy=0x7f62ec071290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:16.050 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:16.045+0000 7f62eaffd640 1 -- 172.21.15.92:0/208348872 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62ec071e00 con 0x7f62ec0cc500 2023-11-24T06:13:16.050 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:16.045+0000 7f62eaffd640 1 -- 172.21.15.92:0/208348872 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2607661350 0 0) 0x7f62d4001690 con 0x7f62ec0cc500 2023-11-24T06:13:16.050 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:16.045+0000 7f62eaffd640 1 -- 172.21.15.92:0/208348872 >> v1:172.21.15.92:6789/0 conn(0x7f62ec0cc500 legacy=0x7f62ec06d160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:16.050 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:16.045+0000 7f62eaffd640 1 --2- 172.21.15.92:0/208348872 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f62d80039a0 0x7f62d8003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:16.050 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:16.045+0000 7f62eaffd640 1 --2- 172.21.15.92:0/208348872 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f62d80047c0 0x7f62d8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:16.050 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:16.045+0000 7f62eaffd640 1 --2- 172.21.15.92:0/208348872 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f62ec0cc990 0x7f62d80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:16.050 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:16.045+0000 7f62eaffd640 1 -- 172.21.15.92:0/208348872 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62ec0723c0 con 0x7f62d80047c0 2023-11-24T06:13:16.051 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:16.045+0000 7f62eaffd640 1 -- 172.21.15.92:0/208348872 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62d800bae0 con 0x7f62d80039a0 2023-11-24T06:13:16.051 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:16.045+0000 7f62eaffd640 1 -- 172.21.15.92:0/208348872 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62d800bca0 con 0x7f62ec0cc990 2023-11-24T06:13:16.051 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:16.045+0000 7f62e3fff640 1 --2- 172.21.15.92:0/208348872 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f62d80047c0 0x7f62d8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:16.051 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:16.046+0000 7f62eb7fe640 1 --2- 172.21.15.92:0/208348872 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f62d80039a0 0x7f62d8003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:16.051 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:16.046+0000 7f62e3fff640 1 -- 172.21.15.92:0/208348872 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f62ec0cc990 msgr2=0x7f62d80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:13:16.051 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:16.046+0000 7f62e3fff640 1 --2- 172.21.15.92:0/208348872 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f62ec0cc990 0x7f62d80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:16.051 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:16.046+0000 7f62e3fff640 1 -- 172.21.15.92:0/208348872 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f62d80039a0 msgr2=0x7f62d8003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:13:16.051 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:16.046+0000 7f62e3fff640 1 --2- 172.21.15.92:0/208348872 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f62d80039a0 0x7f62d8003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:16.051 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:16.046+0000 7f62e3fff640 1 -- 172.21.15.92:0/208348872 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62d8003520 con 0x7f62d80047c0 2023-11-24T06:13:16.051 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:16.046+0000 7f62eb7fe640 1 --2- 172.21.15.92:0/208348872 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f62d80039a0 0x7f62d8003dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:13:16.052 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:16.047+0000 7f62e3fff640 1 --2- 172.21.15.92:0/208348872 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f62d80047c0 0x7f62d8004c40 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f62ec0cbd00 tx=0x7f62d80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30ca07d07c66a91d server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:16.052 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:16.047+0000 7f62eaffd640 1 -- 172.21.15.92:0/208348872 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62d4005ad0 con 0x7f62d80047c0 2023-11-24T06:13:16.052 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:16.047+0000 7f62eaffd640 1 -- 172.21.15.92:0/208348872 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62d4005c30 con 0x7f62d80047c0 2023-11-24T06:13:16.053 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:16.047+0000 7f62eaffd640 1 -- 172.21.15.92:0/208348872 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62d402c890 con 0x7f62d80047c0 2023-11-24T06:13:16.053 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:16.048+0000 7f62f0859640 1 -- 172.21.15.92:0/208348872 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f62d80047c0 msgr2=0x7f62d8004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:16.053 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:16.048+0000 7f62f0859640 1 --2- 172.21.15.92:0/208348872 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f62d80047c0 0x7f62d8004c40 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f62ec0cbd00 tx=0x7f62d80057f0 comp rx=0 tx=0).stop 2023-11-24T06:13:16.053 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:16.048+0000 7f62ebfff640 1 -- 172.21.15.92:0/208348872 reap_dead start 2023-11-24T06:13:16.053 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:16.048+0000 7f62f0859640 1 -- 172.21.15.92:0/208348872 shutdown_connections 2023-11-24T06:13:16.053 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:16.048+0000 7f62f0859640 1 -- 172.21.15.92:0/208348872 >> 172.21.15.92:0/208348872 conn(0x7f62ec06a800 msgr2=0x7f62ec06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:13:16.053 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:16.048+0000 7f62f0859640 1 -- 172.21.15.92:0/208348872 shutdown_connections 2023-11-24T06:13:16.053 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:16.048+0000 7f62f0859640 1 -- 172.21.15.92:0/208348872 wait complete. 2023-11-24T06:13:16.054 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:16.050+0000 7f62f0859640 1 Processor -- start 2023-11-24T06:13:16.054 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:16.050+0000 7f62f0859640 1 -- start start 2023-11-24T06:13:16.054 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:16.051+0000 7f62f0859640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f62d80047c0 0x7f62ec1f6960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:16.054 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:16.051+0000 7f62f0859640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f62ec0c57a0 0x7f62ec0c7cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:16.055 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:16.051+0000 7f62f0859640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f62ec13cf80 0x7f62ec0afbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:16.055 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:16.051+0000 7f62f0859640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62ec0c3d70 con 0x7f62ec13cf80 2023-11-24T06:13:16.055 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:16.051+0000 7f62f0859640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62ec0b31d0 con 0x7f62d80047c0 2023-11-24T06:13:16.055 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:16.051+0000 7f62f0859640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62ec0c5d40 con 0x7f62ec0c57a0 2023-11-24T06:13:16.055 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:16.051+0000 7f62ebfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f62ec13cf80 0x7f62ec0afbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:16.055 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:16.051+0000 7f62ebfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f62ec13cf80 0x7f62ec0afbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52876/0 (socket says 172.21.15.92:52876) 2023-11-24T06:13:16.055 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:16.051+0000 7f62ebfff640 1 -- 172.21.15.92:0/1062232039 learned_addr learned my addr 172.21.15.92:0/1062232039 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:13:16.056 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:16.052+0000 7f62ebfff640 1 -- 172.21.15.92:0/1062232039 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f62ec0c57a0 msgr2=0x7f62ec0c7cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:16.056 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:16.052+0000 7f62ebfff640 1 --2- 172.21.15.92:0/1062232039 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f62ec0c57a0 0x7f62ec0c7cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:16.056 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:16.052+0000 7f62ebfff640 1 -- 172.21.15.92:0/1062232039 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f62d80047c0 msgr2=0x7f62ec1f6960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:16.056 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:16.052+0000 7f62ebfff640 1 --2- 172.21.15.92:0/1062232039 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f62d80047c0 0x7f62ec1f6960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:16.056 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:16.052+0000 7f62ebfff640 1 -- 172.21.15.92:0/1062232039 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62ec0c6020 con 0x7f62ec13cf80 2023-11-24T06:13:16.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:16.052+0000 7f62ebfff640 1 --2- 172.21.15.92:0/1062232039 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f62ec13cf80 0x7f62ec0afbe0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f62ec0cc3f0 tx=0x7f62e400b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:16.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:16.052+0000 7f62e8ff9640 1 -- 172.21.15.92:0/1062232039 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62e4010040 con 0x7f62ec13cf80 2023-11-24T06:13:16.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:16.052+0000 7f62e8ff9640 1 -- 172.21.15.92:0/1062232039 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62e400bae0 con 0x7f62ec13cf80 2023-11-24T06:13:16.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:16.052+0000 7f62e8ff9640 1 -- 172.21.15.92:0/1062232039 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62e400be70 con 0x7f62ec13cf80 2023-11-24T06:13:16.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:16.052+0000 7f62f0859640 1 -- 172.21.15.92:0/1062232039 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62ec0c6300 con 0x7f62ec13cf80 2023-11-24T06:13:16.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:16.053+0000 7f62f0859640 1 -- 172.21.15.92:0/1062232039 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62ec0c8210 con 0x7f62ec13cf80 2023-11-24T06:13:16.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:16.054+0000 7f62f0859640 1 -- 172.21.15.92:0/1062232039 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62ec0c6820 con 0x7f62ec13cf80 2023-11-24T06:13:16.058 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:16.054+0000 7f62e8ff9640 1 -- 172.21.15.92:0/1062232039 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f62e401d400 con 0x7f62ec13cf80 2023-11-24T06:13:16.058 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:16.055+0000 7f62e8ff9640 1 --2- 172.21.15.92:0/1062232039 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f62cc06aed0 0x7f62cc06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:16.058 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:16.055+0000 7f62e8ff9640 1 -- 172.21.15.92:0/1062232039 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f62e4089f30 con 0x7f62ec13cf80 2023-11-24T06:13:16.059 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:16.056+0000 7f62eb7fe640 1 --2- 172.21.15.92:0/1062232039 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f62cc06aed0 0x7f62cc06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:16.059 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:16.056+0000 7f62eb7fe640 1 --2- 172.21.15.92:0/1062232039 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f62cc06aed0 0x7f62cc06d390 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f62dc001670 tx=0x7f62dc009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:16.063 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:16.060+0000 7f62e8ff9640 1 -- 172.21.15.92:0/1062232039 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f62e4054e70 con 0x7f62ec13cf80 2023-11-24T06:13:16.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:16.290+0000 7f62f0859640 1 -- 172.21.15.92:0/1062232039 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f62ec067450 con 0x7f62ec13cf80 2023-11-24T06:13:16.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:16.291+0000 7f62e8ff9640 1 -- 172.21.15.92:0/1062232039 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v40) v1 ==== 53+0+73 (secure 0 0 0) 0x7f62e4018070 con 0x7f62ec13cf80 2023-11-24T06:13:16.294 INFO:teuthology.orchestra.run.smithi092.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-24T06:13:16.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:16.293+0000 7f62f0859640 1 -- 172.21.15.92:0/1062232039 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f62cc06aed0 msgr2=0x7f62cc06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:16.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:16.293+0000 7f62f0859640 1 --2- 172.21.15.92:0/1062232039 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f62cc06aed0 0x7f62cc06d390 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f62dc001670 tx=0x7f62dc009040 comp rx=0 tx=0).stop 2023-11-24T06:13:16.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:16.293+0000 7f62f0859640 1 -- 172.21.15.92:0/1062232039 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f62ec13cf80 msgr2=0x7f62ec0afbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:16.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:16.293+0000 7f62f0859640 1 --2- 172.21.15.92:0/1062232039 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f62ec13cf80 0x7f62ec0afbe0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f62ec0cc3f0 tx=0x7f62e400b480 comp rx=0 tx=0).stop 2023-11-24T06:13:16.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:16.293+0000 7f62f0859640 1 -- 172.21.15.92:0/1062232039 shutdown_connections 2023-11-24T06:13:16.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:16.293+0000 7f62f0859640 1 --2- 172.21.15.92:0/1062232039 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f62cc06aed0 0x7f62cc06d390 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:16.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:16.293+0000 7f62f0859640 1 --2- 172.21.15.92:0/1062232039 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f62ec13cf80 0x7f62ec0afbe0 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:16.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:16.293+0000 7f62f0859640 1 --2- 172.21.15.92:0/1062232039 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f62ec0c57a0 0x7f62ec0c7cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:16.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:16.293+0000 7f62f0859640 1 --2- 172.21.15.92:0/1062232039 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f62d80047c0 0x7f62ec1f6960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:16.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:16.293+0000 7f62f0859640 1 -- 172.21.15.92:0/1062232039 >> 172.21.15.92:0/1062232039 conn(0x7f62ec06a800 msgr2=0x7f62ec0aed80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:13:16.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:16.294+0000 7f62f0859640 1 -- 172.21.15.92:0/1062232039 shutdown_connections 2023-11-24T06:13:16.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:16.294+0000 7f62f0859640 1 -- 172.21.15.92:0/1062232039 wait complete. 2023-11-24T06:13:16.316 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-24T06:13:16.316 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-24T06:13:16.316 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi092.front.sepia.ceph.com 2023-11-24T06:13:16.316 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-24T06:13:16.316 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-24T06:13:16.316 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-24T06:13:16.317 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-24T06:13:16.317 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-24T06:13:16.317 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-24T06:13:16.317 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:13:16.317 DEBUG:teuthology.orchestra.run.smithi092:> ip addr 2023-11-24T06:13:16.370 INFO:teuthology.orchestra.run.smithi092.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-24T06:13:16.371 INFO:teuthology.orchestra.run.smithi092.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-24T06:13:16.371 INFO:teuthology.orchestra.run.smithi092.stdout: inet 127.0.0.1/8 scope host lo 2023-11-24T06:13:16.371 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:13:16.371 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 ::1/128 scope host 2023-11-24T06:13:16.371 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:13:16.371 INFO:teuthology.orchestra.run.smithi092.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T06:13:16.371 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:d8:a1:b2 brd ff:ff:ff:ff:ff:ff 2023-11-24T06:13:16.371 INFO:teuthology.orchestra.run.smithi092.stdout: altname enp7s0f0 2023-11-24T06:13:16.371 INFO:teuthology.orchestra.run.smithi092.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T06:13:16.371 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:d8:a1:b3 brd ff:ff:ff:ff:ff:ff 2023-11-24T06:13:16.371 INFO:teuthology.orchestra.run.smithi092.stdout: altname enp7s0f1 2023-11-24T06:13:16.372 INFO:teuthology.orchestra.run.smithi092.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T06:13:16.372 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:8f:cd:44 brd ff:ff:ff:ff:ff:ff 2023-11-24T06:13:16.372 INFO:teuthology.orchestra.run.smithi092.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-24T06:13:16.372 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:8f:cd:45 brd ff:ff:ff:ff:ff:ff 2023-11-24T06:13:16.372 INFO:teuthology.orchestra.run.smithi092.stdout: inet 172.21.15.92/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-24T06:13:16.372 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft 42045sec preferred_lft 42045sec 2023-11-24T06:13:16.372 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 fe80::ec4:7aff:fe8f:cd45/64 scope link 2023-11-24T06:13:16.372 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:13:16.372 INFO:teuthology.orchestra.run.smithi092.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-24T06:13:16.372 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 7a:4d:f0:06:a7:4e brd ff:ff:ff:ff:ff:ff 2023-11-24T06:13:16.372 INFO:teuthology.orchestra.run.smithi092.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-24T06:13:16.372 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:13:16.373 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 fe80::9055:2fff:fec5:ff24/64 scope link 2023-11-24T06:13:16.373 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:13:16.373 INFO:teuthology.orchestra.run.smithi092.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-24T06:13:16.373 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether fa:d3:03:06:91:4f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-24T06:13:16.373 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 fe80::f8d3:3ff:fe06:914f/64 scope link 2023-11-24T06:13:16.373 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:13:16.373 INFO:teuthology.orchestra.run.smithi092.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-24T06:13:16.373 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 7a:4d:f0:06:a7:4e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-24T06:13:16.373 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 fe80::784d:f0ff:fe06:a74e/64 scope link 2023-11-24T06:13:16.373 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:13:16.374 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:13:16.374 DEBUG:teuthology.orchestra.run.smithi092:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-24T06:13:16.462 INFO:teuthology.orchestra.run.smithi092.stdout:1 2023-11-24T06:13:16.465 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:13:16.465 DEBUG:teuthology.orchestra.run.smithi092:> route 2023-11-24T06:13:16.517 INFO:teuthology.orchestra.run.smithi092.stdout:Kernel IP routing table 2023-11-24T06:13:16.518 INFO:teuthology.orchestra.run.smithi092.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-24T06:13:16.518 INFO:teuthology.orchestra.run.smithi092.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-24T06:13:16.518 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-24T06:13:16.518 INFO:teuthology.orchestra.run.smithi092.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-24T06:13:16.519 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:13:16.519 DEBUG:teuthology.orchestra.run.smithi092:> (cd / && exec bash -c ' 2023-11-24T06:13:16.519 DEBUG:teuthology.orchestra.run.smithi092:> set -e 2023-11-24T06:13:16.519 DEBUG:teuthology.orchestra.run.smithi092:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-24T06:13:16.519 DEBUG:teuthology.orchestra.run.smithi092:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-24T06:13:16.519 DEBUG:teuthology.orchestra.run.smithi092:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-24T06:13:16.519 DEBUG:teuthology.orchestra.run.smithi092:> ') 2023-11-24T06:13:16.697 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:13:16.697 DEBUG:teuthology.orchestra.run.smithi092:> ip netns list 2023-11-24T06:13:16.748 INFO:teuthology.orchestra.run.smithi092.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-24T06:13:16.748 INFO:teuthology.orchestra.run.smithi092.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-24T06:13:16.749 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-24T06:13:16.749 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:13:16.750 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip link set brx.0 up 2023-11-24T06:13:16.846 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-11-24T06:13:16.846 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T06:13:16.846 DEBUG:teuthology.orchestra.run.smithi092:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-11-24T06:13:16.900 INFO:teuthology.orchestra.run.smithi092.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-11-24T06:13:16.900 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T06:13:16.900 DEBUG:teuthology.orchestra.run.smithi092:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-11-24T06:13:16.948 DEBUG:teuthology.orchestra.run.smithi092:> sudo modprobe fuse 2023-11-24T06:13:17.037 DEBUG:teuthology.orchestra.run.smithi092:> cat /proc/self/mounts | awk '{print $2}' 2023-11-24T06:13:17.088 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T06:13:17.094 DEBUG:teuthology.orchestra.run.smithi118:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T06:13:17.095 INFO:teuthology.orchestra.run.smithi092.stdout:/proc 2023-11-24T06:13:17.095 INFO:teuthology.orchestra.run.smithi092.stdout:/sys 2023-11-24T06:13:17.095 INFO:teuthology.orchestra.run.smithi092.stdout:/dev 2023-11-24T06:13:17.095 INFO:teuthology.orchestra.run.smithi092.stdout:/sys/kernel/security 2023-11-24T06:13:17.095 INFO:teuthology.orchestra.run.smithi092.stdout:/dev/shm 2023-11-24T06:13:17.095 INFO:teuthology.orchestra.run.smithi092.stdout:/dev/pts 2023-11-24T06:13:17.095 INFO:teuthology.orchestra.run.smithi092.stdout:/run 2023-11-24T06:13:17.095 INFO:teuthology.orchestra.run.smithi092.stdout:/sys/fs/cgroup 2023-11-24T06:13:17.095 INFO:teuthology.orchestra.run.smithi092.stdout:/sys/fs/pstore 2023-11-24T06:13:17.095 INFO:teuthology.orchestra.run.smithi092.stdout:/ 2023-11-24T06:13:17.096 INFO:teuthology.orchestra.run.smithi092.stdout:/sys/fs/selinux 2023-11-24T06:13:17.096 INFO:teuthology.orchestra.run.smithi092.stdout:/proc/sys/fs/binfmt_misc 2023-11-24T06:13:17.096 INFO:teuthology.orchestra.run.smithi092.stdout:/dev/hugepages 2023-11-24T06:13:17.096 INFO:teuthology.orchestra.run.smithi092.stdout:/dev/mqueue 2023-11-24T06:13:17.096 INFO:teuthology.orchestra.run.smithi092.stdout:/sys/kernel/debug 2023-11-24T06:13:17.096 INFO:teuthology.orchestra.run.smithi092.stdout:/sys/kernel/tracing 2023-11-24T06:13:17.096 INFO:teuthology.orchestra.run.smithi092.stdout:/sys/fs/fuse/connections 2023-11-24T06:13:17.096 INFO:teuthology.orchestra.run.smithi092.stdout:/sys/kernel/config 2023-11-24T06:13:17.096 INFO:teuthology.orchestra.run.smithi092.stdout:/run/credentials/systemd-sysctl.service 2023-11-24T06:13:17.096 INFO:teuthology.orchestra.run.smithi092.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-24T06:13:17.096 INFO:teuthology.orchestra.run.smithi092.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-24T06:13:17.096 INFO:teuthology.orchestra.run.smithi092.stdout:/var/lib/nfs/rpc_pipefs 2023-11-24T06:13:17.097 INFO:teuthology.orchestra.run.smithi092.stdout:/run/user/1000 2023-11-24T06:13:17.097 INFO:teuthology.orchestra.run.smithi092.stdout:/var/lib/ceph 2023-11-24T06:13:17.097 INFO:teuthology.orchestra.run.smithi092.stdout:/var/lib/ceph/osd/ceph-0 2023-11-24T06:13:17.097 INFO:teuthology.orchestra.run.smithi092.stdout:/var/lib/ceph/osd/ceph-1 2023-11-24T06:13:17.097 INFO:teuthology.orchestra.run.smithi092.stdout:/var/lib/ceph/osd/ceph-2 2023-11-24T06:13:17.097 INFO:teuthology.orchestra.run.smithi092.stdout:/var/lib/ceph/osd/ceph-3 2023-11-24T06:13:17.097 INFO:teuthology.orchestra.run.smithi092.stdout:/run/netns 2023-11-24T06:13:17.097 INFO:teuthology.orchestra.run.smithi092.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-24T06:13:17.097 INFO:teuthology.orchestra.run.smithi092.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-24T06:13:17.097 INFO:teuthology.orchestra.run.smithi092.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-24T06:13:17.097 INFO:teuthology.orchestra.run.smithi092.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-24T06:13:17.098 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:13:17.098 DEBUG:teuthology.orchestra.run.smithi092:> ls /sys/fs/fuse/connections 2023-11-24T06:13:17.187 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-11-24T06:13:17.187 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-11-24T06:13:17.191 DEBUG:teuthology.orchestra.run.smithi092:> sudo modprobe fuse 2023-11-24T06:13:17.265 DEBUG:teuthology.orchestra.run.smithi092:> cat /proc/self/mounts | awk '{print $2}' 2023-11-24T06:13:17.310 INFO:teuthology.orchestra.run.smithi092.stdout:/proc 2023-11-24T06:13:17.310 INFO:teuthology.orchestra.run.smithi092.stdout:/sys 2023-11-24T06:13:17.310 INFO:teuthology.orchestra.run.smithi092.stdout:/dev 2023-11-24T06:13:17.310 INFO:teuthology.orchestra.run.smithi092.stdout:/sys/kernel/security 2023-11-24T06:13:17.310 INFO:teuthology.orchestra.run.smithi092.stdout:/dev/shm 2023-11-24T06:13:17.310 INFO:teuthology.orchestra.run.smithi092.stdout:/dev/pts 2023-11-24T06:13:17.311 INFO:teuthology.orchestra.run.smithi092.stdout:/run 2023-11-24T06:13:17.311 INFO:teuthology.orchestra.run.smithi092.stdout:/sys/fs/cgroup 2023-11-24T06:13:17.311 INFO:teuthology.orchestra.run.smithi092.stdout:/sys/fs/pstore 2023-11-24T06:13:17.311 INFO:teuthology.orchestra.run.smithi092.stdout:/ 2023-11-24T06:13:17.311 INFO:teuthology.orchestra.run.smithi092.stdout:/sys/fs/selinux 2023-11-24T06:13:17.311 INFO:teuthology.orchestra.run.smithi092.stdout:/proc/sys/fs/binfmt_misc 2023-11-24T06:13:17.311 INFO:teuthology.orchestra.run.smithi092.stdout:/dev/hugepages 2023-11-24T06:13:17.311 INFO:teuthology.orchestra.run.smithi092.stdout:/dev/mqueue 2023-11-24T06:13:17.311 INFO:teuthology.orchestra.run.smithi092.stdout:/sys/kernel/debug 2023-11-24T06:13:17.311 INFO:teuthology.orchestra.run.smithi092.stdout:/sys/kernel/tracing 2023-11-24T06:13:17.311 INFO:teuthology.orchestra.run.smithi092.stdout:/sys/fs/fuse/connections 2023-11-24T06:13:17.311 INFO:teuthology.orchestra.run.smithi092.stdout:/sys/kernel/config 2023-11-24T06:13:17.312 INFO:teuthology.orchestra.run.smithi092.stdout:/run/credentials/systemd-sysctl.service 2023-11-24T06:13:17.312 INFO:teuthology.orchestra.run.smithi092.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-24T06:13:17.312 INFO:teuthology.orchestra.run.smithi092.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-24T06:13:17.312 INFO:teuthology.orchestra.run.smithi092.stdout:/var/lib/nfs/rpc_pipefs 2023-11-24T06:13:17.312 INFO:teuthology.orchestra.run.smithi092.stdout:/run/user/1000 2023-11-24T06:13:17.312 INFO:teuthology.orchestra.run.smithi092.stdout:/var/lib/ceph 2023-11-24T06:13:17.312 INFO:teuthology.orchestra.run.smithi092.stdout:/var/lib/ceph/osd/ceph-0 2023-11-24T06:13:17.312 INFO:teuthology.orchestra.run.smithi092.stdout:/var/lib/ceph/osd/ceph-1 2023-11-24T06:13:17.312 INFO:teuthology.orchestra.run.smithi092.stdout:/var/lib/ceph/osd/ceph-2 2023-11-24T06:13:17.312 INFO:teuthology.orchestra.run.smithi092.stdout:/var/lib/ceph/osd/ceph-3 2023-11-24T06:13:17.312 INFO:teuthology.orchestra.run.smithi092.stdout:/run/netns 2023-11-24T06:13:17.313 INFO:teuthology.orchestra.run.smithi092.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-24T06:13:17.313 INFO:teuthology.orchestra.run.smithi092.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-24T06:13:17.313 INFO:teuthology.orchestra.run.smithi092.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-24T06:13:17.313 INFO:teuthology.orchestra.run.smithi092.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-24T06:13:17.314 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:13:17.314 DEBUG:teuthology.orchestra.run.smithi092:> ls /sys/fs/fuse/connections 2023-11-24T06:13:17.377 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi092.stderr:2023-11-24T06:13:17.375+0000 7f63ade45480 -1 init, newargv = 0x563791681640 newargc=15 2023-11-24T06:13:17.378 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi092.stderr:ceph-fuse[109323]: starting ceph client 2023-11-24T06:13:17.385 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi092.stderr:ceph-fuse[109323]: starting fuse 2023-11-24T06:13:17.403 INFO:teuthology.orchestra.run.smithi092.stdout:40 2023-11-24T06:13:17.403 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [40] 2023-11-24T06:13:17.404 DEBUG:teuthology.orchestra.run.smithi092:> sudo stdin-killer -- python3 -c ' 2023-11-24T06:13:17.404 DEBUG:teuthology.orchestra.run.smithi092:> import glob 2023-11-24T06:13:17.404 DEBUG:teuthology.orchestra.run.smithi092:> import re 2023-11-24T06:13:17.404 DEBUG:teuthology.orchestra.run.smithi092:> import os 2023-11-24T06:13:17.404 DEBUG:teuthology.orchestra.run.smithi092:> import subprocess 2023-11-24T06:13:17.404 DEBUG:teuthology.orchestra.run.smithi092:> 2023-11-24T06:13:17.404 DEBUG:teuthology.orchestra.run.smithi092:> def _find_admin_socket(client_name): 2023-11-24T06:13:17.404 DEBUG:teuthology.orchestra.run.smithi092:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-11-24T06:13:17.404 DEBUG:teuthology.orchestra.run.smithi092:> files = glob.glob(asok_path) 2023-11-24T06:13:17.404 DEBUG:teuthology.orchestra.run.smithi092:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-11-24T06:13:17.404 DEBUG:teuthology.orchestra.run.smithi092:> 2023-11-24T06:13:17.404 DEBUG:teuthology.orchestra.run.smithi092:> # Given a non-glob path, it better be there 2023-11-24T06:13:17.405 DEBUG:teuthology.orchestra.run.smithi092:> if "*" not in asok_path: 2023-11-24T06:13:17.405 DEBUG:teuthology.orchestra.run.smithi092:> assert(len(files) == 1) 2023-11-24T06:13:17.405 DEBUG:teuthology.orchestra.run.smithi092:> return files[0] 2023-11-24T06:13:17.405 DEBUG:teuthology.orchestra.run.smithi092:> 2023-11-24T06:13:17.405 DEBUG:teuthology.orchestra.run.smithi092:> for f in files: 2023-11-24T06:13:17.405 DEBUG:teuthology.orchestra.run.smithi092:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-11-24T06:13:17.405 DEBUG:teuthology.orchestra.run.smithi092:> if os.path.exists("/proc/{0}".format(pid)): 2023-11-24T06:13:17.405 DEBUG:teuthology.orchestra.run.smithi092:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-11-24T06:13:17.405 DEBUG:teuthology.orchestra.run.smithi092:> contents = proc_f.read() 2023-11-24T06:13:17.405 DEBUG:teuthology.orchestra.run.smithi092:> if mountpoint in contents: 2023-11-24T06:13:17.405 DEBUG:teuthology.orchestra.run.smithi092:> return f 2023-11-24T06:13:17.405 DEBUG:teuthology.orchestra.run.smithi092:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-11-24T06:13:17.405 DEBUG:teuthology.orchestra.run.smithi092:> 2023-11-24T06:13:17.405 DEBUG:teuthology.orchestra.run.smithi092:> print(_find_admin_socket("client.0")) 2023-11-24T06:13:17.406 DEBUG:teuthology.orchestra.run.smithi092:> ' 2023-11-24T06:13:17.597 INFO:teuthology.orchestra.run.smithi092.stdout:/var/run/ceph/ceph-client.0.109323.asok 2023-11-24T06:13:17.601 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-24T06:13:17.614 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.109323.asok 2023-11-24T06:13:17.614 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:13:17.614 DEBUG:teuthology.orchestra.run.smithi092:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.109323.asok status 2023-11-24T06:13:17.849 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-24T06:13:17.849 INFO:teuthology.orchestra.run.smithi092.stdout: "metadata": { 2023-11-24T06:13:17.849 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph_sha1": "a13ea5ff00cb4df7a9cc941166ddeb358881144c", 2023-11-24T06:13:17.849 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph_version": "ceph version 18.0.0-7484-ga13ea5ff (a13ea5ff00cb4df7a9cc941166ddeb358881144c) reef (dev)", 2023-11-24T06:13:17.850 INFO:teuthology.orchestra.run.smithi092.stdout: "entity_id": "0", 2023-11-24T06:13:17.850 INFO:teuthology.orchestra.run.smithi092.stdout: "hostname": "smithi092", 2023-11-24T06:13:17.850 INFO:teuthology.orchestra.run.smithi092.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-11-24T06:13:17.850 INFO:teuthology.orchestra.run.smithi092.stdout: "pid": "109323", 2023-11-24T06:13:17.850 INFO:teuthology.orchestra.run.smithi092.stdout: "root": "/" 2023-11-24T06:13:17.850 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-24T06:13:17.850 INFO:teuthology.orchestra.run.smithi092.stdout: "dentry_count": 0, 2023-11-24T06:13:17.850 INFO:teuthology.orchestra.run.smithi092.stdout: "dentry_pinned_count": 0, 2023-11-24T06:13:17.850 INFO:teuthology.orchestra.run.smithi092.stdout: "id": 6318, 2023-11-24T06:13:17.850 INFO:teuthology.orchestra.run.smithi092.stdout: "inst": { 2023-11-24T06:13:17.851 INFO:teuthology.orchestra.run.smithi092.stdout: "name": { 2023-11-24T06:13:17.851 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "client", 2023-11-24T06:13:17.851 INFO:teuthology.orchestra.run.smithi092.stdout: "num": 6318 2023-11-24T06:13:17.851 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-24T06:13:17.851 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": { 2023-11-24T06:13:17.851 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2023-11-24T06:13:17.851 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "192.168.0.1:0", 2023-11-24T06:13:17.851 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 2223906739 2023-11-24T06:13:17.851 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-24T06:13:17.851 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-24T06:13:17.851 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": { 2023-11-24T06:13:17.851 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2023-11-24T06:13:17.852 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "192.168.0.1:0", 2023-11-24T06:13:17.852 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 2223906739 2023-11-24T06:13:17.852 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-24T06:13:17.852 INFO:teuthology.orchestra.run.smithi092.stdout: "inst_str": "client.6318 192.168.0.1:0/2223906739", 2023-11-24T06:13:17.852 INFO:teuthology.orchestra.run.smithi092.stdout: "addr_str": "192.168.0.1:0/2223906739", 2023-11-24T06:13:17.852 INFO:teuthology.orchestra.run.smithi092.stdout: "inode_count": 1, 2023-11-24T06:13:17.852 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_epoch": 40, 2023-11-24T06:13:17.852 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_epoch": 63, 2023-11-24T06:13:17.852 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_epoch_barrier": 0, 2023-11-24T06:13:17.852 INFO:teuthology.orchestra.run.smithi092.stdout: "blocklisted": false, 2023-11-24T06:13:17.852 INFO:teuthology.orchestra.run.smithi092.stdout: "fs_name": "cephfs" 2023-11-24T06:13:17.853 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-24T06:13:17.864 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:13:17.864 DEBUG:teuthology.orchestra.run.smithi092:> stat --file-system '--printf=%T 2023-11-24T06:13:17.864 DEBUG:teuthology.orchestra.run.smithi092:> ' -- /home/ubuntu/cephtest/mnt.0 2023-11-24T06:13:17.915 INFO:teuthology.orchestra.run.smithi092.stdout:fuseblk 2023-11-24T06:13:17.916 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-11-24T06:13:17.916 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:13:17.917 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-24T06:13:18.010 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-24T06:13:18.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:18.226+0000 7fcd32056640 1 Processor -- start 2023-11-24T06:13:18.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:18.226+0000 7fcd32056640 1 -- start start 2023-11-24T06:13:18.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:18.227+0000 7fcd32056640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fcd2c143ae0 0x7fcd2c0cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:18.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:18.227+0000 7fcd32056640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fcd2c0cf7d0 0x7fcd2c15dca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:18.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:18.228+0000 7fcd32056640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd2c0cfc30 con 0x7fcd2c15e1e0 2023-11-24T06:13:18.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:18.228+0000 7fcd32056640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd2c0b1ee0 con 0x7fcd2c143ae0 2023-11-24T06:13:18.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:18.228+0000 7fcd32056640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd2c0b2780 con 0x7fcd2c0cf7d0 2023-11-24T06:13:18.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:18.228+0000 7fcd2bfff640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fcd2c15e1e0 0x7fcd2c15a670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:59364/0 (socket says 172.21.15.92:59364) 2023-11-24T06:13:18.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:18.228+0000 7fcd2b7fe640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fcd2c143ae0 0x7fcd2c0cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:18.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:18.228+0000 7fcd2bfff640 1 -- 172.21.15.92:0/1116396235 learned_addr learned my addr 172.21.15.92:0/1116396235 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-24T06:13:18.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:18.228+0000 7fcd23fff640 1 --2- 172.21.15.92:0/1116396235 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fcd2c0cf7d0 0x7fcd2c15dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:18.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:18.228+0000 7fcd2b7fe640 1 -- 172.21.15.92:0/1116396235 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fcd2c0cf7d0 msgr2=0x7fcd2c15dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:13:18.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:18.228+0000 7fcd2b7fe640 1 --2- 172.21.15.92:0/1116396235 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fcd2c0cf7d0 0x7fcd2c15dca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:18.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:18.228+0000 7fcd2b7fe640 1 -- 172.21.15.92:0/1116396235 >> v1:172.21.15.92:6789/0 conn(0x7fcd2c15e1e0 legacy=0x7fcd2c15a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:18.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:18.228+0000 7fcd2b7fe640 1 -- 172.21.15.92:0/1116396235 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd2c15e990 con 0x7fcd2c143ae0 2023-11-24T06:13:18.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:18.228+0000 7fcd2affd640 1 -- 172.21.15.92:0/1116396235 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 985462299 0 0) 0x7fcd2c0cfc30 con 0x7fcd2c15e1e0 2023-11-24T06:13:18.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:18.228+0000 7fcd23fff640 1 --2- 172.21.15.92:0/1116396235 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fcd2c0cf7d0 0x7fcd2c15dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:13:18.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:18.229+0000 7fcd2b7fe640 1 --2- 172.21.15.92:0/1116396235 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcd2c143ae0 0x7fcd2c0cf290 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7fcd1c0097a0 tx=0x7fcd1c02b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2dfbbedd9d7136fe server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:18.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:18.230+0000 7fcd2affd640 1 -- 172.21.15.92:0/1116396235 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd1c02d040 con 0x7fcd2c143ae0 2023-11-24T06:13:18.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:18.230+0000 7fcd2affd640 1 -- 172.21.15.92:0/1116396235 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd1c031810 con 0x7fcd2c143ae0 2023-11-24T06:13:18.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:18.230+0000 7fcd2affd640 1 -- 172.21.15.92:0/1116396235 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd1c031ba0 con 0x7fcd2c143ae0 2023-11-24T06:13:18.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:18.230+0000 7fcd32056640 1 -- 172.21.15.92:0/1116396235 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcd2c143ae0 msgr2=0x7fcd2c0cf290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:18.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:18.230+0000 7fcd32056640 1 --2- 172.21.15.92:0/1116396235 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcd2c143ae0 0x7fcd2c0cf290 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7fcd1c0097a0 tx=0x7fcd1c02b8b0 comp rx=0 tx=0).stop 2023-11-24T06:13:18.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:18.230+0000 7fcd32056640 1 -- 172.21.15.92:0/1116396235 shutdown_connections 2023-11-24T06:13:18.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:18.230+0000 7fcd32056640 1 --2- 172.21.15.92:0/1116396235 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fcd2c0cf7d0 0x7fcd2c15dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:18.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:18.230+0000 7fcd32056640 1 --2- 172.21.15.92:0/1116396235 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcd2c143ae0 0x7fcd2c0cf290 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:18.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:18.230+0000 7fcd32056640 1 -- 172.21.15.92:0/1116396235 >> 172.21.15.92:0/1116396235 conn(0x7fcd2c06a800 msgr2=0x7fcd2c06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:13:18.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:18.231+0000 7fcd32056640 1 -- 172.21.15.92:0/1116396235 shutdown_connections 2023-11-24T06:13:18.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:18.231+0000 7fcd32056640 1 -- 172.21.15.92:0/1116396235 wait complete. 2023-11-24T06:13:18.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:18.232+0000 7fcd32056640 1 Processor -- start 2023-11-24T06:13:18.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:18.233+0000 7fcd32056640 1 -- start start 2023-11-24T06:13:18.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:18.233+0000 7fcd32056640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fcd2c0cf7d0 0x7fcd2c0c6850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:18.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:18.234+0000 7fcd32056640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fcd2c143ae0 0x7fcd2c0c6d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:18.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:18.234+0000 7fcd32056640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcd2c15e1e0 0x7fcd2c136980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:18.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:18.234+0000 7fcd32056640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd2c135120 con 0x7fcd2c15e1e0 2023-11-24T06:13:18.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:18.234+0000 7fcd32056640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd2c142250 con 0x7fcd2c143ae0 2023-11-24T06:13:18.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:18.234+0000 7fcd32056640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd2c0cdf50 con 0x7fcd2c0cf7d0 2023-11-24T06:13:18.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:18.234+0000 7fcd2bfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcd2c15e1e0 0x7fcd2c136980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:18.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:18.234+0000 7fcd2bfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcd2c15e1e0 0x7fcd2c136980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52902/0 (socket says 172.21.15.92:52902) 2023-11-24T06:13:18.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:18.234+0000 7fcd2bfff640 1 -- 172.21.15.92:0/94594790 learned_addr learned my addr 172.21.15.92:0/94594790 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:13:18.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:18.234+0000 7fcd23fff640 1 --2- 172.21.15.92:0/94594790 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fcd2c143ae0 0x7fcd2c0c6d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:18.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:18.234+0000 7fcd2b7fe640 1 --2- 172.21.15.92:0/94594790 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fcd2c0cf7d0 0x7fcd2c0c6850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:18.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:18.234+0000 7fcd2bfff640 1 -- 172.21.15.92:0/94594790 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fcd2c0cf7d0 msgr2=0x7fcd2c0c6850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:18.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:18.234+0000 7fcd2bfff640 1 --2- 172.21.15.92:0/94594790 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fcd2c0cf7d0 0x7fcd2c0c6850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:18.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:18.234+0000 7fcd2bfff640 1 -- 172.21.15.92:0/94594790 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fcd2c143ae0 msgr2=0x7fcd2c0c6d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:18.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:18.234+0000 7fcd2bfff640 1 --2- 172.21.15.92:0/94594790 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fcd2c143ae0 0x7fcd2c0c6d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:18.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:18.234+0000 7fcd2bfff640 1 -- 172.21.15.92:0/94594790 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd1c009400 con 0x7fcd2c15e1e0 2023-11-24T06:13:18.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:18.234+0000 7fcd23fff640 1 --2- 172.21.15.92:0/94594790 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fcd2c143ae0 0x7fcd2c0c6d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:13:18.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:18.235+0000 7fcd2bfff640 1 --2- 172.21.15.92:0/94594790 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcd2c15e1e0 0x7fcd2c136980 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fcd2400c7b0 tx=0x7fcd2400cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:18.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:18.235+0000 7fcd28ff9640 1 -- 172.21.15.92:0/94594790 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd2400a5e0 con 0x7fcd2c15e1e0 2023-11-24T06:13:18.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:18.235+0000 7fcd28ff9640 1 -- 172.21.15.92:0/94594790 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd2400a760 con 0x7fcd2c15e1e0 2023-11-24T06:13:18.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:18.235+0000 7fcd32056640 1 -- 172.21.15.92:0/94594790 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd2c0cdc70 con 0x7fcd2c15e1e0 2023-11-24T06:13:18.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:18.235+0000 7fcd28ff9640 1 -- 172.21.15.92:0/94594790 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd2400ab10 con 0x7fcd2c15e1e0 2023-11-24T06:13:18.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:18.235+0000 7fcd32056640 1 -- 172.21.15.92:0/94594790 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd2c134d40 con 0x7fcd2c15e1e0 2023-11-24T06:13:18.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:18.236+0000 7fcd32056640 1 -- 172.21.15.92:0/94594790 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd2c0b2bf0 con 0x7fcd2c15e1e0 2023-11-24T06:13:18.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:18.237+0000 7fcd28ff9640 1 -- 172.21.15.92:0/94594790 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7fcd2400ac70 con 0x7fcd2c15e1e0 2023-11-24T06:13:18.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:18.238+0000 7fcd28ff9640 1 --2- 172.21.15.92:0/94594790 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fcd0c06aed0 0x7fcd0c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:18.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:18.238+0000 7fcd28ff9640 1 -- 172.21.15.92:0/94594790 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcd24088e60 con 0x7fcd2c15e1e0 2023-11-24T06:13:18.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:18.238+0000 7fcd2b7fe640 1 --2- 172.21.15.92:0/94594790 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fcd0c06aed0 0x7fcd0c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:18.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:18.239+0000 7fcd2b7fe640 1 --2- 172.21.15.92:0/94594790 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fcd0c06aed0 0x7fcd0c06d390 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fcd1c036000 tx=0x7fcd1c03c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:18.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:18.243+0000 7fcd28ff9640 1 -- 172.21.15.92:0/94594790 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fcd2c0b2bf0 con 0x7fcd2c15e1e0 2023-11-24T06:13:18.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:18.470+0000 7fcd32056640 1 -- 172.21.15.92:0/94594790 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fcd2c135260 con 0x7fcd2c15e1e0 2023-11-24T06:13:18.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:18.471+0000 7fcd28ff9640 1 -- 172.21.15.92:0/94594790 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v40) v1 ==== 53+0+73 (secure 0 0 0) 0x7fcd2c135260 con 0x7fcd2c15e1e0 2023-11-24T06:13:18.474 INFO:teuthology.orchestra.run.smithi092.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-24T06:13:18.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:18.474+0000 7fcd32056640 1 -- 172.21.15.92:0/94594790 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fcd0c06aed0 msgr2=0x7fcd0c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:18.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:18.474+0000 7fcd32056640 1 --2- 172.21.15.92:0/94594790 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fcd0c06aed0 0x7fcd0c06d390 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fcd1c036000 tx=0x7fcd1c03c040 comp rx=0 tx=0).stop 2023-11-24T06:13:18.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:18.474+0000 7fcd32056640 1 -- 172.21.15.92:0/94594790 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcd2c15e1e0 msgr2=0x7fcd2c136980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:18.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:18.474+0000 7fcd32056640 1 --2- 172.21.15.92:0/94594790 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcd2c15e1e0 0x7fcd2c136980 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fcd2400c7b0 tx=0x7fcd2400cc70 comp rx=0 tx=0).stop 2023-11-24T06:13:18.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:18.474+0000 7fcd32056640 1 -- 172.21.15.92:0/94594790 shutdown_connections 2023-11-24T06:13:18.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:18.474+0000 7fcd32056640 1 --2- 172.21.15.92:0/94594790 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fcd0c06aed0 0x7fcd0c06d390 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:18.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:18.474+0000 7fcd32056640 1 --2- 172.21.15.92:0/94594790 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcd2c15e1e0 0x7fcd2c136980 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:18.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:18.474+0000 7fcd32056640 1 --2- 172.21.15.92:0/94594790 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fcd2c143ae0 0x7fcd2c0c6d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:18.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:18.474+0000 7fcd32056640 1 --2- 172.21.15.92:0/94594790 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fcd2c0cf7d0 0x7fcd2c0c6850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:18.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:18.474+0000 7fcd32056640 1 -- 172.21.15.92:0/94594790 >> 172.21.15.92:0/94594790 conn(0x7fcd2c06a800 msgr2=0x7fcd2c15b7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:13:18.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:18.474+0000 7fcd32056640 1 -- 172.21.15.92:0/94594790 shutdown_connections 2023-11-24T06:13:18.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:18.474+0000 7fcd32056640 1 -- 172.21.15.92:0/94594790 wait complete. 2023-11-24T06:13:18.497 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-24T06:13:18.497 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-24T06:13:18.497 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi118.front.sepia.ceph.com 2023-11-24T06:13:18.497 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-11-24T06:13:18.497 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-11-24T06:13:18.497 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-24T06:13:18.497 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-24T06:13:18.497 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-24T06:13:18.497 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-11-24T06:13:18.497 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:13:18.498 DEBUG:teuthology.orchestra.run.smithi118:> ip addr 2023-11-24T06:13:18.557 INFO:teuthology.orchestra.run.smithi118.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-24T06:13:18.557 INFO:teuthology.orchestra.run.smithi118.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-24T06:13:18.557 INFO:teuthology.orchestra.run.smithi118.stdout: inet 127.0.0.1/8 scope host lo 2023-11-24T06:13:18.558 INFO:teuthology.orchestra.run.smithi118.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:13:18.558 INFO:teuthology.orchestra.run.smithi118.stdout: inet6 ::1/128 scope host 2023-11-24T06:13:18.558 INFO:teuthology.orchestra.run.smithi118.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:13:18.558 INFO:teuthology.orchestra.run.smithi118.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T06:13:18.558 INFO:teuthology.orchestra.run.smithi118.stdout: link/ether 0c:c4:7a:d8:a4:cc brd ff:ff:ff:ff:ff:ff 2023-11-24T06:13:18.558 INFO:teuthology.orchestra.run.smithi118.stdout: altname enp7s0f0 2023-11-24T06:13:18.558 INFO:teuthology.orchestra.run.smithi118.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T06:13:18.558 INFO:teuthology.orchestra.run.smithi118.stdout: link/ether 0c:c4:7a:d8:a4:cd brd ff:ff:ff:ff:ff:ff 2023-11-24T06:13:18.558 INFO:teuthology.orchestra.run.smithi118.stdout: altname enp7s0f1 2023-11-24T06:13:18.558 INFO:teuthology.orchestra.run.smithi118.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T06:13:18.558 INFO:teuthology.orchestra.run.smithi118.stdout: link/ether 0c:c4:7a:8f:d7:24 brd ff:ff:ff:ff:ff:ff 2023-11-24T06:13:18.559 INFO:teuthology.orchestra.run.smithi118.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-24T06:13:18.559 INFO:teuthology.orchestra.run.smithi118.stdout: link/ether 0c:c4:7a:8f:d7:25 brd ff:ff:ff:ff:ff:ff 2023-11-24T06:13:18.559 INFO:teuthology.orchestra.run.smithi118.stdout: inet 172.21.15.118/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-24T06:13:18.559 INFO:teuthology.orchestra.run.smithi118.stdout: valid_lft 42049sec preferred_lft 42049sec 2023-11-24T06:13:18.559 INFO:teuthology.orchestra.run.smithi118.stdout: inet6 fe80::ec4:7aff:fe8f:d725/64 scope link 2023-11-24T06:13:18.559 INFO:teuthology.orchestra.run.smithi118.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:13:18.559 INFO:teuthology.orchestra.run.smithi118.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-24T06:13:18.559 INFO:teuthology.orchestra.run.smithi118.stdout: link/ether 16:ef:1e:16:88:fb brd ff:ff:ff:ff:ff:ff 2023-11-24T06:13:18.559 INFO:teuthology.orchestra.run.smithi118.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-24T06:13:18.559 INFO:teuthology.orchestra.run.smithi118.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:13:18.559 INFO:teuthology.orchestra.run.smithi118.stdout: inet6 fe80::50a3:24ff:fea4:c41e/64 scope link 2023-11-24T06:13:18.559 INFO:teuthology.orchestra.run.smithi118.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:13:18.559 INFO:teuthology.orchestra.run.smithi118.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-24T06:13:18.560 INFO:teuthology.orchestra.run.smithi118.stdout: link/ether 16:ef:1e:16:88:fb brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-24T06:13:18.560 INFO:teuthology.orchestra.run.smithi118.stdout: inet6 fe80::14ef:1eff:fe16:88fb/64 scope link 2023-11-24T06:13:18.560 INFO:teuthology.orchestra.run.smithi118.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:13:18.560 INFO:teuthology.orchestra.run.smithi118.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-24T06:13:18.560 INFO:teuthology.orchestra.run.smithi118.stdout: link/ether 6a:b2:50:ae:7a:72 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-24T06:13:18.560 INFO:teuthology.orchestra.run.smithi118.stdout: inet6 fe80::68b2:50ff:feae:7a72/64 scope link 2023-11-24T06:13:18.560 INFO:teuthology.orchestra.run.smithi118.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:13:18.561 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:13:18.561 DEBUG:teuthology.orchestra.run.smithi118:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-24T06:13:18.653 INFO:teuthology.orchestra.run.smithi118.stdout:1 2023-11-24T06:13:18.656 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:13:18.657 DEBUG:teuthology.orchestra.run.smithi118:> route 2023-11-24T06:13:18.710 INFO:teuthology.orchestra.run.smithi118.stdout:Kernel IP routing table 2023-11-24T06:13:18.710 INFO:teuthology.orchestra.run.smithi118.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-24T06:13:18.710 INFO:teuthology.orchestra.run.smithi118.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-24T06:13:18.711 INFO:teuthology.orchestra.run.smithi118.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-24T06:13:18.711 INFO:teuthology.orchestra.run.smithi118.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-24T06:13:18.711 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:13:18.711 DEBUG:teuthology.orchestra.run.smithi118:> (cd / && exec bash -c ' 2023-11-24T06:13:18.711 DEBUG:teuthology.orchestra.run.smithi118:> set -e 2023-11-24T06:13:18.711 DEBUG:teuthology.orchestra.run.smithi118:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-24T06:13:18.711 DEBUG:teuthology.orchestra.run.smithi118:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-24T06:13:18.711 DEBUG:teuthology.orchestra.run.smithi118:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-24T06:13:18.712 DEBUG:teuthology.orchestra.run.smithi118:> ') 2023-11-24T06:13:18.893 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:13:18.893 DEBUG:teuthology.orchestra.run.smithi118:> ip netns list 2023-11-24T06:13:18.944 INFO:teuthology.orchestra.run.smithi118.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-11-24T06:13:18.944 INFO:teuthology.orchestra.run.smithi118.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-11-24T06:13:18.945 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-11-24T06:13:18.945 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:13:18.945 DEBUG:teuthology.orchestra.run.smithi118:> sudo ip link set brx.0 up 2023-11-24T06:13:19.038 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2023-11-24T06:13:19.038 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T06:13:19.038 DEBUG:teuthology.orchestra.run.smithi118:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-11-24T06:13:19.090 INFO:teuthology.orchestra.run.smithi118.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-11-24T06:13:19.091 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T06:13:19.091 DEBUG:teuthology.orchestra.run.smithi118:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-11-24T06:13:19.141 DEBUG:teuthology.orchestra.run.smithi118:> sudo modprobe fuse 2023-11-24T06:13:19.233 DEBUG:teuthology.orchestra.run.smithi118:> cat /proc/self/mounts | awk '{print $2}' 2023-11-24T06:13:19.291 INFO:teuthology.orchestra.run.smithi118.stdout:/proc 2023-11-24T06:13:19.291 INFO:teuthology.orchestra.run.smithi118.stdout:/sys 2023-11-24T06:13:19.291 INFO:teuthology.orchestra.run.smithi118.stdout:/dev 2023-11-24T06:13:19.291 INFO:teuthology.orchestra.run.smithi118.stdout:/sys/kernel/security 2023-11-24T06:13:19.291 INFO:teuthology.orchestra.run.smithi118.stdout:/dev/shm 2023-11-24T06:13:19.292 INFO:teuthology.orchestra.run.smithi118.stdout:/dev/pts 2023-11-24T06:13:19.292 INFO:teuthology.orchestra.run.smithi118.stdout:/run 2023-11-24T06:13:19.292 INFO:teuthology.orchestra.run.smithi118.stdout:/sys/fs/cgroup 2023-11-24T06:13:19.292 INFO:teuthology.orchestra.run.smithi118.stdout:/sys/fs/pstore 2023-11-24T06:13:19.292 INFO:teuthology.orchestra.run.smithi118.stdout:/ 2023-11-24T06:13:19.292 INFO:teuthology.orchestra.run.smithi118.stdout:/sys/fs/selinux 2023-11-24T06:13:19.292 INFO:teuthology.orchestra.run.smithi118.stdout:/proc/sys/fs/binfmt_misc 2023-11-24T06:13:19.292 INFO:teuthology.orchestra.run.smithi118.stdout:/dev/mqueue 2023-11-24T06:13:19.292 INFO:teuthology.orchestra.run.smithi118.stdout:/dev/hugepages 2023-11-24T06:13:19.292 INFO:teuthology.orchestra.run.smithi118.stdout:/sys/kernel/debug 2023-11-24T06:13:19.292 INFO:teuthology.orchestra.run.smithi118.stdout:/sys/kernel/tracing 2023-11-24T06:13:19.292 INFO:teuthology.orchestra.run.smithi118.stdout:/sys/fs/fuse/connections 2023-11-24T06:13:19.292 INFO:teuthology.orchestra.run.smithi118.stdout:/sys/kernel/config 2023-11-24T06:13:19.293 INFO:teuthology.orchestra.run.smithi118.stdout:/run/credentials/systemd-sysctl.service 2023-11-24T06:13:19.293 INFO:teuthology.orchestra.run.smithi118.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-24T06:13:19.293 INFO:teuthology.orchestra.run.smithi118.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-24T06:13:19.293 INFO:teuthology.orchestra.run.smithi118.stdout:/var/lib/nfs/rpc_pipefs 2023-11-24T06:13:19.293 INFO:teuthology.orchestra.run.smithi118.stdout:/run/user/1000 2023-11-24T06:13:19.293 INFO:teuthology.orchestra.run.smithi118.stdout:/var/lib/ceph 2023-11-24T06:13:19.293 INFO:teuthology.orchestra.run.smithi118.stdout:/var/lib/ceph/osd/ceph-4 2023-11-24T06:13:19.293 INFO:teuthology.orchestra.run.smithi118.stdout:/var/lib/ceph/osd/ceph-5 2023-11-24T06:13:19.293 INFO:teuthology.orchestra.run.smithi118.stdout:/var/lib/ceph/osd/ceph-6 2023-11-24T06:13:19.293 INFO:teuthology.orchestra.run.smithi118.stdout:/var/lib/ceph/osd/ceph-7 2023-11-24T06:13:19.293 INFO:teuthology.orchestra.run.smithi118.stdout:/run/netns 2023-11-24T06:13:19.293 INFO:teuthology.orchestra.run.smithi118.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-24T06:13:19.294 INFO:teuthology.orchestra.run.smithi118.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-24T06:13:19.294 INFO:teuthology.orchestra.run.smithi118.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-24T06:13:19.294 INFO:teuthology.orchestra.run.smithi118.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-24T06:13:19.295 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:13:19.295 DEBUG:teuthology.orchestra.run.smithi118:> ls /sys/fs/fuse/connections 2023-11-24T06:13:19.347 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-11-24T06:13:19.347 DEBUG:teuthology.orchestra.run.smithi118:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2023-11-24T06:13:19.353 DEBUG:teuthology.orchestra.run.smithi118:> sudo modprobe fuse 2023-11-24T06:13:19.429 DEBUG:teuthology.orchestra.run.smithi118:> cat /proc/self/mounts | awk '{print $2}' 2023-11-24T06:13:19.471 INFO:teuthology.orchestra.run.smithi118.stdout:/proc 2023-11-24T06:13:19.471 INFO:teuthology.orchestra.run.smithi118.stdout:/sys 2023-11-24T06:13:19.471 INFO:teuthology.orchestra.run.smithi118.stdout:/dev 2023-11-24T06:13:19.471 INFO:teuthology.orchestra.run.smithi118.stdout:/sys/kernel/security 2023-11-24T06:13:19.471 INFO:teuthology.orchestra.run.smithi118.stdout:/dev/shm 2023-11-24T06:13:19.471 INFO:teuthology.orchestra.run.smithi118.stdout:/dev/pts 2023-11-24T06:13:19.472 INFO:teuthology.orchestra.run.smithi118.stdout:/run 2023-11-24T06:13:19.472 INFO:teuthology.orchestra.run.smithi118.stdout:/sys/fs/cgroup 2023-11-24T06:13:19.472 INFO:teuthology.orchestra.run.smithi118.stdout:/sys/fs/pstore 2023-11-24T06:13:19.472 INFO:teuthology.orchestra.run.smithi118.stdout:/ 2023-11-24T06:13:19.472 INFO:teuthology.orchestra.run.smithi118.stdout:/sys/fs/selinux 2023-11-24T06:13:19.472 INFO:teuthology.orchestra.run.smithi118.stdout:/proc/sys/fs/binfmt_misc 2023-11-24T06:13:19.472 INFO:teuthology.orchestra.run.smithi118.stdout:/dev/mqueue 2023-11-24T06:13:19.472 INFO:teuthology.orchestra.run.smithi118.stdout:/dev/hugepages 2023-11-24T06:13:19.472 INFO:teuthology.orchestra.run.smithi118.stdout:/sys/kernel/debug 2023-11-24T06:13:19.472 INFO:teuthology.orchestra.run.smithi118.stdout:/sys/kernel/tracing 2023-11-24T06:13:19.472 INFO:teuthology.orchestra.run.smithi118.stdout:/sys/fs/fuse/connections 2023-11-24T06:13:19.473 INFO:teuthology.orchestra.run.smithi118.stdout:/sys/kernel/config 2023-11-24T06:13:19.473 INFO:teuthology.orchestra.run.smithi118.stdout:/run/credentials/systemd-sysctl.service 2023-11-24T06:13:19.473 INFO:teuthology.orchestra.run.smithi118.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-24T06:13:19.473 INFO:teuthology.orchestra.run.smithi118.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-24T06:13:19.473 INFO:teuthology.orchestra.run.smithi118.stdout:/var/lib/nfs/rpc_pipefs 2023-11-24T06:13:19.473 INFO:teuthology.orchestra.run.smithi118.stdout:/run/user/1000 2023-11-24T06:13:19.473 INFO:teuthology.orchestra.run.smithi118.stdout:/var/lib/ceph 2023-11-24T06:13:19.473 INFO:teuthology.orchestra.run.smithi118.stdout:/var/lib/ceph/osd/ceph-4 2023-11-24T06:13:19.473 INFO:teuthology.orchestra.run.smithi118.stdout:/var/lib/ceph/osd/ceph-5 2023-11-24T06:13:19.473 INFO:teuthology.orchestra.run.smithi118.stdout:/var/lib/ceph/osd/ceph-6 2023-11-24T06:13:19.474 INFO:teuthology.orchestra.run.smithi118.stdout:/var/lib/ceph/osd/ceph-7 2023-11-24T06:13:19.474 INFO:teuthology.orchestra.run.smithi118.stdout:/run/netns 2023-11-24T06:13:19.474 INFO:teuthology.orchestra.run.smithi118.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-24T06:13:19.474 INFO:teuthology.orchestra.run.smithi118.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-24T06:13:19.474 INFO:teuthology.orchestra.run.smithi118.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-24T06:13:19.474 INFO:teuthology.orchestra.run.smithi118.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-24T06:13:19.475 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:13:19.475 DEBUG:teuthology.orchestra.run.smithi118:> ls /sys/fs/fuse/connections 2023-11-24T06:13:19.538 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi118.stderr:2023-11-24T06:13:19.535+0000 7fddf6967480 -1 init, newargv = 0x56423f176370 newargc=15 2023-11-24T06:13:19.539 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi118.stderr:ceph-fuse[95623]: starting ceph client 2023-11-24T06:13:19.550 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi118.stderr:ceph-fuse[95623]: starting fuse 2023-11-24T06:13:19.560 INFO:teuthology.orchestra.run.smithi118.stdout:41 2023-11-24T06:13:19.560 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2023-11-24T06:13:19.561 DEBUG:teuthology.orchestra.run.smithi118:> sudo stdin-killer -- python3 -c ' 2023-11-24T06:13:19.561 DEBUG:teuthology.orchestra.run.smithi118:> import glob 2023-11-24T06:13:19.561 DEBUG:teuthology.orchestra.run.smithi118:> import re 2023-11-24T06:13:19.561 DEBUG:teuthology.orchestra.run.smithi118:> import os 2023-11-24T06:13:19.561 DEBUG:teuthology.orchestra.run.smithi118:> import subprocess 2023-11-24T06:13:19.561 DEBUG:teuthology.orchestra.run.smithi118:> 2023-11-24T06:13:19.561 DEBUG:teuthology.orchestra.run.smithi118:> def _find_admin_socket(client_name): 2023-11-24T06:13:19.561 DEBUG:teuthology.orchestra.run.smithi118:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2023-11-24T06:13:19.561 DEBUG:teuthology.orchestra.run.smithi118:> files = glob.glob(asok_path) 2023-11-24T06:13:19.561 DEBUG:teuthology.orchestra.run.smithi118:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2023-11-24T06:13:19.561 DEBUG:teuthology.orchestra.run.smithi118:> 2023-11-24T06:13:19.561 DEBUG:teuthology.orchestra.run.smithi118:> # Given a non-glob path, it better be there 2023-11-24T06:13:19.562 DEBUG:teuthology.orchestra.run.smithi118:> if "*" not in asok_path: 2023-11-24T06:13:19.562 DEBUG:teuthology.orchestra.run.smithi118:> assert(len(files) == 1) 2023-11-24T06:13:19.562 DEBUG:teuthology.orchestra.run.smithi118:> return files[0] 2023-11-24T06:13:19.562 DEBUG:teuthology.orchestra.run.smithi118:> 2023-11-24T06:13:19.562 DEBUG:teuthology.orchestra.run.smithi118:> for f in files: 2023-11-24T06:13:19.562 DEBUG:teuthology.orchestra.run.smithi118:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-11-24T06:13:19.562 DEBUG:teuthology.orchestra.run.smithi118:> if os.path.exists("/proc/{0}".format(pid)): 2023-11-24T06:13:19.562 DEBUG:teuthology.orchestra.run.smithi118:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-11-24T06:13:19.562 DEBUG:teuthology.orchestra.run.smithi118:> contents = proc_f.read() 2023-11-24T06:13:19.562 DEBUG:teuthology.orchestra.run.smithi118:> if mountpoint in contents: 2023-11-24T06:13:19.562 DEBUG:teuthology.orchestra.run.smithi118:> return f 2023-11-24T06:13:19.562 DEBUG:teuthology.orchestra.run.smithi118:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-11-24T06:13:19.563 DEBUG:teuthology.orchestra.run.smithi118:> 2023-11-24T06:13:19.563 DEBUG:teuthology.orchestra.run.smithi118:> print(_find_admin_socket("client.1")) 2023-11-24T06:13:19.563 DEBUG:teuthology.orchestra.run.smithi118:> ' 2023-11-24T06:13:19.762 INFO:teuthology.orchestra.run.smithi118.stdout:/var/run/ceph/ceph-client.1.95623.asok 2023-11-24T06:13:19.767 INFO:teuthology.orchestra.run.smithi118.stderr:2023-11-24T06:13:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-24T06:13:19.779 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.95623.asok 2023-11-24T06:13:19.779 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:13:19.779 DEBUG:teuthology.orchestra.run.smithi118:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.95623.asok status 2023-11-24T06:13:20.018 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2023-11-24T06:13:20.018 INFO:teuthology.orchestra.run.smithi118.stdout: "metadata": { 2023-11-24T06:13:20.018 INFO:teuthology.orchestra.run.smithi118.stdout: "ceph_sha1": "a13ea5ff00cb4df7a9cc941166ddeb358881144c", 2023-11-24T06:13:20.018 INFO:teuthology.orchestra.run.smithi118.stdout: "ceph_version": "ceph version 18.0.0-7484-ga13ea5ff (a13ea5ff00cb4df7a9cc941166ddeb358881144c) reef (dev)", 2023-11-24T06:13:20.019 INFO:teuthology.orchestra.run.smithi118.stdout: "entity_id": "1", 2023-11-24T06:13:20.019 INFO:teuthology.orchestra.run.smithi118.stdout: "hostname": "smithi118", 2023-11-24T06:13:20.019 INFO:teuthology.orchestra.run.smithi118.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2023-11-24T06:13:20.019 INFO:teuthology.orchestra.run.smithi118.stdout: "pid": "95623", 2023-11-24T06:13:20.019 INFO:teuthology.orchestra.run.smithi118.stdout: "root": "/" 2023-11-24T06:13:20.019 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-11-24T06:13:20.019 INFO:teuthology.orchestra.run.smithi118.stdout: "dentry_count": 0, 2023-11-24T06:13:20.019 INFO:teuthology.orchestra.run.smithi118.stdout: "dentry_pinned_count": 0, 2023-11-24T06:13:20.019 INFO:teuthology.orchestra.run.smithi118.stdout: "id": 5206, 2023-11-24T06:13:20.019 INFO:teuthology.orchestra.run.smithi118.stdout: "inst": { 2023-11-24T06:13:20.019 INFO:teuthology.orchestra.run.smithi118.stdout: "name": { 2023-11-24T06:13:20.019 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "client", 2023-11-24T06:13:20.020 INFO:teuthology.orchestra.run.smithi118.stdout: "num": 5206 2023-11-24T06:13:20.020 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-11-24T06:13:20.020 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": { 2023-11-24T06:13:20.020 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v1", 2023-11-24T06:13:20.020 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "192.168.0.1:0", 2023-11-24T06:13:20.020 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 1567922837 2023-11-24T06:13:20.020 INFO:teuthology.orchestra.run.smithi118.stdout: } 2023-11-24T06:13:20.020 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-11-24T06:13:20.020 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": { 2023-11-24T06:13:20.020 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v1", 2023-11-24T06:13:20.020 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "192.168.0.1:0", 2023-11-24T06:13:20.021 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 1567922837 2023-11-24T06:13:20.021 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-11-24T06:13:20.021 INFO:teuthology.orchestra.run.smithi118.stdout: "inst_str": "client.5206 192.168.0.1:0/1567922837", 2023-11-24T06:13:20.021 INFO:teuthology.orchestra.run.smithi118.stdout: "addr_str": "192.168.0.1:0/1567922837", 2023-11-24T06:13:20.021 INFO:teuthology.orchestra.run.smithi118.stdout: "inode_count": 1, 2023-11-24T06:13:20.021 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_epoch": 40, 2023-11-24T06:13:20.021 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_epoch": 63, 2023-11-24T06:13:20.021 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_epoch_barrier": 0, 2023-11-24T06:13:20.021 INFO:teuthology.orchestra.run.smithi118.stdout: "blocklisted": false, 2023-11-24T06:13:20.021 INFO:teuthology.orchestra.run.smithi118.stdout: "fs_name": "cephfs" 2023-11-24T06:13:20.021 INFO:teuthology.orchestra.run.smithi118.stdout:} 2023-11-24T06:13:20.033 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:13:20.033 DEBUG:teuthology.orchestra.run.smithi118:> stat --file-system '--printf=%T 2023-11-24T06:13:20.033 DEBUG:teuthology.orchestra.run.smithi118:> ' -- /home/ubuntu/cephtest/mnt.1 2023-11-24T06:13:20.090 INFO:teuthology.orchestra.run.smithi118.stdout:fuseblk 2023-11-24T06:13:20.090 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2023-11-24T06:13:20.090 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:13:20.090 DEBUG:teuthology.orchestra.run.smithi118:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-11-24T06:13:20.183 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2023-11-24T06:13:20.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:20.406+0000 7f6ab2c00640 1 Processor -- start 2023-11-24T06:13:20.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:20.406+0000 7f6ab2c00640 1 -- start start 2023-11-24T06:13:20.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:20.408+0000 7f6ab2c00640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f6aac0d0b60 0x7f6aac0ac8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:20.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:20.408+0000 7f6ab2c00640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6aac0aede0 0x7f6aac06f080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:20.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:20.408+0000 7f6ab2c00640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6aac071de0 con 0x7f6aac06a3e0 2023-11-24T06:13:20.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:20.408+0000 7f6ab2c00640 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6aac0af270 con 0x7f6aac0d0b60 2023-11-24T06:13:20.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:20.408+0000 7f6ab2c00640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6aac072030 con 0x7f6aac0aede0 2023-11-24T06:13:20.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:20.408+0000 7f6ab099c640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f6aac06a3e0 0x7f6aac06a7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:53548/0 (socket says 172.21.15.92:53548) 2023-11-24T06:13:20.412 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:20.408+0000 7f6ab099c640 1 -- 172.21.15.92:0/3777762323 learned_addr learned my addr 172.21.15.92:0/3777762323 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-24T06:13:20.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:20.409+0000 7f6aab7fe640 1 -- 172.21.15.92:0/3777762323 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 482538840 0 0) 0x7f6aac071de0 con 0x7f6aac06a3e0 2023-11-24T06:13:20.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:20.409+0000 7f6aab7fe640 1 -- 172.21.15.92:0/3777762323 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6a9c003520 con 0x7f6aac06a3e0 2023-11-24T06:13:20.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:20.409+0000 7f6aabfff640 1 --2- 172.21.15.92:0/3777762323 >> v2:172.21.15.118:3300/0 conn(0x7f6aac0d0b60 0x7f6aac0ac8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:20.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:20.409+0000 7f6ab119d640 1 --2- 172.21.15.92:0/3777762323 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6aac0aede0 0x7f6aac06f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:20.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:20.409+0000 7f6aab7fe640 1 -- 172.21.15.92:0/3777762323 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1880005275 0 0) 0x7f6a9c003520 con 0x7f6aac06a3e0 2023-11-24T06:13:20.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:20.409+0000 7f6aab7fe640 1 -- 172.21.15.92:0/3777762323 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6aac0aede0 msgr2=0x7f6aac06f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:13:20.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:20.409+0000 7f6aab7fe640 1 --2- 172.21.15.92:0/3777762323 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6aac0aede0 0x7f6aac06f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:20.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:20.409+0000 7f6aab7fe640 1 -- 172.21.15.92:0/3777762323 >> v2:172.21.15.118:3300/0 conn(0x7f6aac0d0b60 msgr2=0x7f6aac0ac8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:13:20.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:20.409+0000 7f6aab7fe640 1 --2- 172.21.15.92:0/3777762323 >> v2:172.21.15.118:3300/0 conn(0x7f6aac0d0b60 0x7f6aac0ac8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:20.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:20.409+0000 7f6aab7fe640 1 -- 172.21.15.92:0/3777762323 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6aac072310 con 0x7f6aac06a3e0 2023-11-24T06:13:20.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:20.409+0000 7f6aab7fe640 1 -- 172.21.15.92:0/3777762323 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2607661350 0 0) 0x7f6aa0001670 con 0x7f6aac06a3e0 2023-11-24T06:13:20.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:20.409+0000 7f6aab7fe640 1 -- 172.21.15.92:0/3777762323 >> v1:172.21.15.92:6789/0 conn(0x7f6aac06a3e0 legacy=0x7f6aac06a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:20.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:20.409+0000 7f6aab7fe640 1 --2- 172.21.15.92:0/3777762323 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6aac0aede0 0x7f6a9c003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:20.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:20.409+0000 7f6aab7fe640 1 --2- 172.21.15.92:0/3777762323 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6a9c003fa0 0x7f6a9c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:20.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:20.409+0000 7f6aab7fe640 1 --2- 172.21.15.92:0/3777762323 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6a9c004d00 0x7f6a9c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:20.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:20.409+0000 7f6aab7fe640 1 -- 172.21.15.92:0/3777762323 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6aac0af270 con 0x7f6a9c003fa0 2023-11-24T06:13:20.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:20.409+0000 7f6aab7fe640 1 -- 172.21.15.92:0/3777762323 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6aac072030 con 0x7f6a9c004d00 2023-11-24T06:13:20.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:20.409+0000 7f6aab7fe640 1 -- 172.21.15.92:0/3777762323 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a9c00ba80 con 0x7f6aac0aede0 2023-11-24T06:13:20.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:20.410+0000 7f6aabfff640 1 --2- 172.21.15.92:0/3777762323 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6a9c003fa0 0x7f6a9c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:20.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:20.410+0000 7f6ab099c640 1 --2- 172.21.15.92:0/3777762323 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6aac0aede0 0x7f6a9c003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:20.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:20.410+0000 7f6aabfff640 1 -- 172.21.15.92:0/3777762323 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6aac0aede0 msgr2=0x7f6a9c003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:13:20.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:20.410+0000 7f6aabfff640 1 --2- 172.21.15.92:0/3777762323 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6aac0aede0 0x7f6a9c003970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:20.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:20.410+0000 7f6ab119d640 1 --2- 172.21.15.92:0/3777762323 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6a9c004d00 0x7f6a9c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:20.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:20.410+0000 7f6aabfff640 1 -- 172.21.15.92:0/3777762323 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6a9c004d00 msgr2=0x7f6a9c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:13:20.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:20.410+0000 7f6aabfff640 1 --2- 172.21.15.92:0/3777762323 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6a9c004d00 0x7f6a9c005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:20.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:20.410+0000 7f6aabfff640 1 -- 172.21.15.92:0/3777762323 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a9c003520 con 0x7f6a9c003fa0 2023-11-24T06:13:20.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:20.410+0000 7f6ab119d640 1 --2- 172.21.15.92:0/3777762323 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6a9c004d00 0x7f6a9c005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:13:20.417 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:20.411+0000 7f6aabfff640 1 --2- 172.21.15.92:0/3777762323 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6a9c003fa0 0x7f6a9c0047c0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f6a9800ba00 tx=0x7f6a9c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61ff4b2ff746b6fe server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:20.417 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:20.411+0000 7f6aab7fe640 1 -- 172.21.15.92:0/3777762323 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a98032640 con 0x7f6a9c003fa0 2023-11-24T06:13:20.417 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:20.411+0000 7f6aab7fe640 1 -- 172.21.15.92:0/3777762323 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a980327a0 con 0x7f6a9c003fa0 2023-11-24T06:13:20.417 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:20.411+0000 7f6aab7fe640 1 -- 172.21.15.92:0/3777762323 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a98032b30 con 0x7f6a9c003fa0 2023-11-24T06:13:20.417 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:20.412+0000 7f6ab2c00640 1 -- 172.21.15.92:0/3777762323 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6a9c003fa0 msgr2=0x7f6a9c0047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:20.417 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:20.412+0000 7f6ab2c00640 1 --2- 172.21.15.92:0/3777762323 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6a9c003fa0 0x7f6a9c0047c0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f6a9800ba00 tx=0x7f6a9c0057f0 comp rx=0 tx=0).stop 2023-11-24T06:13:20.417 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:20.412+0000 7f6ab119d640 1 -- 172.21.15.92:0/3777762323 reap_dead start 2023-11-24T06:13:20.417 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:20.412+0000 7f6ab2c00640 1 -- 172.21.15.92:0/3777762323 shutdown_connections 2023-11-24T06:13:20.417 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:20.412+0000 7f6ab2c00640 1 -- 172.21.15.92:0/3777762323 >> 172.21.15.92:0/3777762323 conn(0x7f6aac0cfef0 msgr2=0x7f6aac0d0300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:13:20.418 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:20.412+0000 7f6ab2c00640 1 -- 172.21.15.92:0/3777762323 shutdown_connections 2023-11-24T06:13:20.418 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:20.412+0000 7f6ab2c00640 1 -- 172.21.15.92:0/3777762323 wait complete. 2023-11-24T06:13:20.418 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:20.414+0000 7f6ab2c00640 1 Processor -- start 2023-11-24T06:13:20.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:20.414+0000 7f6ab2c00640 1 -- start start 2023-11-24T06:13:20.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:20.415+0000 7f6ab2c00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6a9c003fa0 0x7f6aac0cf570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:20.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:20.415+0000 7f6ab2c00640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6aac159e70 0x7f6aac150d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:20.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:20.415+0000 7f6ab2c00640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6aac14f9a0 0x7f6aac14fe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:20.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:20.415+0000 7f6ab2c00640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6aac15b2d0 con 0x7f6a9c003fa0 2023-11-24T06:13:20.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:20.415+0000 7f6ab2c00640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6aac150360 con 0x7f6aac14f9a0 2023-11-24T06:13:20.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:20.415+0000 7f6ab2c00640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6aac15a650 con 0x7f6aac159e70 2023-11-24T06:13:20.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:20.415+0000 7f6ab099c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6a9c003fa0 0x7f6aac0cf570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:20.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:20.415+0000 7f6ab099c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6a9c003fa0 0x7f6aac0cf570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43696/0 (socket says 172.21.15.92:43696) 2023-11-24T06:13:20.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:20.415+0000 7f6ab099c640 1 -- 172.21.15.92:0/2580909440 learned_addr learned my addr 172.21.15.92:0/2580909440 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:13:20.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:20.416+0000 7f6ab099c640 1 -- 172.21.15.92:0/2580909440 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6aac159e70 msgr2=0x7f6aac150d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:20.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:20.416+0000 7f6ab119d640 1 --2- 172.21.15.92:0/2580909440 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6aac14f9a0 0x7f6aac14fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:20.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:20.416+0000 7f6ab099c640 1 --2- 172.21.15.92:0/2580909440 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6aac159e70 0x7f6aac150d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:20.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:20.416+0000 7f6ab099c640 1 -- 172.21.15.92:0/2580909440 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6aac14f9a0 msgr2=0x7f6aac14fe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:20.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:20.416+0000 7f6ab099c640 1 --2- 172.21.15.92:0/2580909440 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6aac14f9a0 0x7f6aac14fe20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:20.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:20.416+0000 7f6ab099c640 1 -- 172.21.15.92:0/2580909440 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6aac15a930 con 0x7f6a9c003fa0 2023-11-24T06:13:20.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:20.416+0000 7f6ab099c640 1 --2- 172.21.15.92:0/2580909440 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6a9c003fa0 0x7f6aac0cf570 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f6aa000d8e0 tx=0x7f6aa000dda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:20.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:20.416+0000 7f6aa97fa640 1 -- 172.21.15.92:0/2580909440 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6aa00074e0 con 0x7f6a9c003fa0 2023-11-24T06:13:20.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:20.416+0000 7f6aa97fa640 1 -- 172.21.15.92:0/2580909440 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6aa0007640 con 0x7f6a9c003fa0 2023-11-24T06:13:20.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:20.416+0000 7f6ab2c00640 1 -- 172.21.15.92:0/2580909440 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6aac15ac10 con 0x7f6a9c003fa0 2023-11-24T06:13:20.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:20.416+0000 7f6aa97fa640 1 -- 172.21.15.92:0/2580909440 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6aa00049e0 con 0x7f6a9c003fa0 2023-11-24T06:13:20.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:20.417+0000 7f6ab2c00640 1 -- 172.21.15.92:0/2580909440 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6aac15a2f0 con 0x7f6a9c003fa0 2023-11-24T06:13:20.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:20.417+0000 7f6ab2c00640 1 -- 172.21.15.92:0/2580909440 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6aac073c00 con 0x7f6a9c003fa0 2023-11-24T06:13:20.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:20.418+0000 7f6aa97fa640 1 -- 172.21.15.92:0/2580909440 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f6aa0004b40 con 0x7f6a9c003fa0 2023-11-24T06:13:20.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:20.419+0000 7f6aa97fa640 1 --2- 172.21.15.92:0/2580909440 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f6a9006aed0 0x7f6a9006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:20.423 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:20.420+0000 7f6aa97fa640 1 -- 172.21.15.92:0/2580909440 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6aa0089260 con 0x7f6a9c003fa0 2023-11-24T06:13:20.423 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:20.420+0000 7f6aabfff640 1 --2- 172.21.15.92:0/2580909440 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f6a9006aed0 0x7f6a9006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:20.423 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:20.420+0000 7f6aabfff640 1 --2- 172.21.15.92:0/2580909440 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f6a9006aed0 0x7f6a9006d390 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f6aac0d0fa0 tx=0x7f6a98032000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:20.427 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:20.424+0000 7f6aa97fa640 1 -- 172.21.15.92:0/2580909440 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6aa00541d0 con 0x7f6a9c003fa0 2023-11-24T06:13:20.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:20.660+0000 7f6ab2c00640 1 -- 172.21.15.92:0/2580909440 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f6aac0675d0 con 0x7f6a9c003fa0 2023-11-24T06:13:21.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:21.542+0000 7f6aa97fa640 1 -- 172.21.15.92:0/2580909440 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v41) v1 ==== 136+0+0 (secure 0 0 0) 0x7f6aa00577f0 con 0x7f6a9c003fa0 2023-11-24T06:13:21.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:21.546+0000 7f6ab2c00640 1 -- 172.21.15.92:0/2580909440 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f6a9006aed0 msgr2=0x7f6a9006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:21.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:21.546+0000 7f6ab2c00640 1 --2- 172.21.15.92:0/2580909440 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f6a9006aed0 0x7f6a9006d390 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f6aac0d0fa0 tx=0x7f6a98032000 comp rx=0 tx=0).stop 2023-11-24T06:13:21.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:21.546+0000 7f6ab2c00640 1 -- 172.21.15.92:0/2580909440 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6a9c003fa0 msgr2=0x7f6aac0cf570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:21.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:21.546+0000 7f6ab2c00640 1 --2- 172.21.15.92:0/2580909440 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6a9c003fa0 0x7f6aac0cf570 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f6aa000d8e0 tx=0x7f6aa000dda0 comp rx=0 tx=0).stop 2023-11-24T06:13:21.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:21.546+0000 7f6ab2c00640 1 -- 172.21.15.92:0/2580909440 shutdown_connections 2023-11-24T06:13:21.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:21.546+0000 7f6ab2c00640 1 --2- 172.21.15.92:0/2580909440 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f6a9006aed0 0x7f6a9006d390 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:21.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:21.546+0000 7f6ab2c00640 1 --2- 172.21.15.92:0/2580909440 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6aac14f9a0 0x7f6aac14fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:21.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:21.546+0000 7f6ab2c00640 1 --2- 172.21.15.92:0/2580909440 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6aac159e70 0x7f6aac150d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:21.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:21.546+0000 7f6ab2c00640 1 --2- 172.21.15.92:0/2580909440 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6a9c003fa0 0x7f6aac0cf570 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:21.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:21.546+0000 7f6ab2c00640 1 -- 172.21.15.92:0/2580909440 >> 172.21.15.92:0/2580909440 conn(0x7f6aac0cfef0 msgr2=0x7f6aac0add40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:13:21.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:21.546+0000 7f6ab2c00640 1 -- 172.21.15.92:0/2580909440 shutdown_connections 2023-11-24T06:13:21.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:21.546+0000 7f6ab2c00640 1 -- 172.21.15.92:0/2580909440 wait complete. 2023-11-24T06:13:21.569 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-11-24T06:13:21.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:21.783+0000 7f171121c640 1 Processor -- start 2023-11-24T06:13:21.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:21.783+0000 7f171121c640 1 -- start start 2023-11-24T06:13:21.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:21.785+0000 7f171121c640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f170c15e880 0x7f170c06ff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:21.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:21.785+0000 7f171121c640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f170c070490 0x7f170c099a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:21.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:21.785+0000 7f171121c640 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f170c0cfa70 con 0x7f170c099fa0 2023-11-24T06:13:21.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:21.785+0000 7f171121c640 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f170c09a410 con 0x7f170c070490 2023-11-24T06:13:21.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:21.785+0000 7f171121c640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f170c0cff00 con 0x7f170c15e880 2023-11-24T06:13:21.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:21.786+0000 7f170ad9d640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f170c15e880 0x7f170c06ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:21.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:21.786+0000 7f170a59c640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f170c070490 0x7f170c099a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:21.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:21.786+0000 7f170a59c640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f170c070490 0x7f170c099a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.92:47022/0 (socket says 172.21.15.92:47022) 2023-11-24T06:13:21.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:21.786+0000 7f170a59c640 1 -- 172.21.15.92:0/381550314 learned_addr learned my addr 172.21.15.92:0/381550314 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:13:21.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:21.787+0000 7f170a59c640 1 -- 172.21.15.92:0/381550314 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f170c15e880 msgr2=0x7f170c06ff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:13:21.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:21.787+0000 7f170a59c640 1 --2- 172.21.15.92:0/381550314 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f170c15e880 0x7f170c06ff50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:21.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:21.787+0000 7f170a59c640 1 -- 172.21.15.92:0/381550314 >> v1:172.21.15.118:6789/0 conn(0x7f170c099fa0 legacy=0x7f170c0cbb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:21.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:21.787+0000 7f170a59c640 1 -- 172.21.15.92:0/381550314 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f170c06fc70 con 0x7f170c070490 2023-11-24T06:13:21.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:21.788+0000 7f170a59c640 1 --2- 172.21.15.92:0/381550314 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f170c070490 0x7f170c099a60 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f16f4009870 tx=0x7f16f402ba20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2c7da159a5e2214e server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:21.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:21.788+0000 7f1709d9b640 1 -- 172.21.15.92:0/381550314 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16f402f840 con 0x7f170c070490 2023-11-24T06:13:21.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:21.788+0000 7f1709d9b640 1 -- 172.21.15.92:0/381550314 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16f402f9a0 con 0x7f170c070490 2023-11-24T06:13:21.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:21.788+0000 7f1709d9b640 1 -- 172.21.15.92:0/381550314 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16f402fd30 con 0x7f170c070490 2023-11-24T06:13:21.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:21.789+0000 7f171121c640 1 -- 172.21.15.92:0/381550314 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f170c070490 msgr2=0x7f170c099a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:21.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:21.789+0000 7f171121c640 1 --2- 172.21.15.92:0/381550314 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f170c070490 0x7f170c099a60 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f16f4009870 tx=0x7f16f402ba20 comp rx=0 tx=0).stop 2023-11-24T06:13:21.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:21.789+0000 7f171121c640 1 -- 172.21.15.92:0/381550314 shutdown_connections 2023-11-24T06:13:21.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:21.789+0000 7f171121c640 1 --2- 172.21.15.92:0/381550314 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f170c070490 0x7f170c099a60 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:21.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:21.789+0000 7f171121c640 1 --2- 172.21.15.92:0/381550314 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f170c15e880 0x7f170c06ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:21.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:21.789+0000 7f171121c640 1 -- 172.21.15.92:0/381550314 >> 172.21.15.92:0/381550314 conn(0x7f170c0b2d00 msgr2=0x7f170c0b3110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:13:21.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:21.789+0000 7f171121c640 1 -- 172.21.15.92:0/381550314 shutdown_connections 2023-11-24T06:13:21.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:21.789+0000 7f171121c640 1 -- 172.21.15.92:0/381550314 wait complete. 2023-11-24T06:13:21.794 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:21.791+0000 7f171121c640 1 Processor -- start 2023-11-24T06:13:21.794 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:21.791+0000 7f171121c640 1 -- start start 2023-11-24T06:13:21.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:21.792+0000 7f171121c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f170c070490 0x7f170c159950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:21.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:21.792+0000 7f171121c640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f170c099fa0 0x7f170c0c6350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:21.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:21.792+0000 7f171121c640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f170c15e880 0x7f170c0ca750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:21.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:21.792+0000 7f171121c640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f170c0c7a20 con 0x7f170c070490 2023-11-24T06:13:21.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:21.792+0000 7f171121c640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f170c0c6e10 con 0x7f170c099fa0 2023-11-24T06:13:21.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:21.792+0000 7f171121c640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f170c0c70f0 con 0x7f170c15e880 2023-11-24T06:13:21.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:21.793+0000 7f170ad9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f170c070490 0x7f170c159950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:21.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:21.793+0000 7f170ad9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f170c070490 0x7f170c159950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43708/0 (socket says 172.21.15.92:43708) 2023-11-24T06:13:21.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:21.793+0000 7f170ad9d640 1 -- 172.21.15.92:0/928519223 learned_addr learned my addr 172.21.15.92:0/928519223 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:13:21.797 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:21.793+0000 7f170a59c640 1 --2- 172.21.15.92:0/928519223 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f170c099fa0 0x7f170c0c6350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:21.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:21.793+0000 7f170b59e640 1 --2- 172.21.15.92:0/928519223 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f170c15e880 0x7f170c0ca750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:21.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:21.793+0000 7f170ad9d640 1 -- 172.21.15.92:0/928519223 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f170c15e880 msgr2=0x7f170c0ca750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:21.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:21.793+0000 7f170ad9d640 1 --2- 172.21.15.92:0/928519223 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f170c15e880 0x7f170c0ca750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:21.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:21.793+0000 7f170ad9d640 1 -- 172.21.15.92:0/928519223 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f170c099fa0 msgr2=0x7f170c0c6350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:21.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:21.793+0000 7f170ad9d640 1 --2- 172.21.15.92:0/928519223 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f170c099fa0 0x7f170c0c6350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:21.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:21.793+0000 7f170ad9d640 1 -- 172.21.15.92:0/928519223 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16f40094d0 con 0x7f170c070490 2023-11-24T06:13:21.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:21.793+0000 7f170ad9d640 1 --2- 172.21.15.92:0/928519223 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f170c070490 0x7f170c159950 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f16fc00efc0 tx=0x7f16fc00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:21.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:21.794+0000 7f17037fe640 1 -- 172.21.15.92:0/928519223 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16fc009b80 con 0x7f170c070490 2023-11-24T06:13:21.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:21.794+0000 7f17037fe640 1 -- 172.21.15.92:0/928519223 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16fc00f040 con 0x7f170c070490 2023-11-24T06:13:21.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:21.794+0000 7f171121c640 1 -- 172.21.15.92:0/928519223 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f170c0c73d0 con 0x7f170c070490 2023-11-24T06:13:21.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:21.794+0000 7f17037fe640 1 -- 172.21.15.92:0/928519223 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16fc0146a0 con 0x7f170c070490 2023-11-24T06:13:21.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:21.794+0000 7f171121c640 1 -- 172.21.15.92:0/928519223 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f170c1f3010 con 0x7f170c070490 2023-11-24T06:13:21.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:21.795+0000 7f171121c640 1 -- 172.21.15.92:0/928519223 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f170c1f3200 con 0x7f170c070490 2023-11-24T06:13:21.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:21.796+0000 7f17037fe640 1 -- 172.21.15.92:0/928519223 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f16fc014800 con 0x7f170c070490 2023-11-24T06:13:21.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:21.797+0000 7f17037fe640 1 --2- 172.21.15.92:0/928519223 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f16ec06aed0 0x7f16ec06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:21.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:21.798+0000 7f17037fe640 1 -- 172.21.15.92:0/928519223 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f16fc08a290 con 0x7f170c070490 2023-11-24T06:13:21.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:21.798+0000 7f170a59c640 1 --2- 172.21.15.92:0/928519223 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f16ec06aed0 0x7f16ec06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:21.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:21.798+0000 7f170a59c640 1 --2- 172.21.15.92:0/928519223 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f16ec06aed0 0x7f16ec06d390 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f16f4036000 tx=0x7f16f4008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:21.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:21.802+0000 7f17037fe640 1 -- 172.21.15.92:0/928519223 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f170c1f3200 con 0x7f170c070490 2023-11-24T06:13:22.019 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:22.016+0000 7f171121c640 1 -- 172.21.15.92:0/928519223 --> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f170c0cf700 con 0x7f16ec06aed0 2023-11-24T06:13:22.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:22.018+0000 7f17037fe640 1 -- 172.21.15.92:0/928519223 <== mgr.4101 v2:172.21.15.92:6800/784027401 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f16f402fb10 con 0x7f16ec06aed0 2023-11-24T06:13:22.022 INFO:teuthology.orchestra.run.smithi092.stdout:[ 2023-11-24T06:13:22.022 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-24T06:13:22.022 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "cephfs" 2023-11-24T06:13:22.022 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-24T06:13:22.022 INFO:teuthology.orchestra.run.smithi092.stdout:] 2023-11-24T06:13:22.025 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:22.021+0000 7f171121c640 1 -- 172.21.15.92:0/928519223 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f16ec06aed0 msgr2=0x7f16ec06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:22.025 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:22.021+0000 7f171121c640 1 --2- 172.21.15.92:0/928519223 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f16ec06aed0 0x7f16ec06d390 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f16f4036000 tx=0x7f16f4008030 comp rx=0 tx=0).stop 2023-11-24T06:13:22.025 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:22.021+0000 7f171121c640 1 -- 172.21.15.92:0/928519223 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f170c070490 msgr2=0x7f170c159950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:22.025 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:22.021+0000 7f171121c640 1 --2- 172.21.15.92:0/928519223 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f170c070490 0x7f170c159950 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f16fc00efc0 tx=0x7f16fc00c490 comp rx=0 tx=0).stop 2023-11-24T06:13:22.025 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:22.021+0000 7f171121c640 1 -- 172.21.15.92:0/928519223 shutdown_connections 2023-11-24T06:13:22.025 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:22.021+0000 7f171121c640 1 --2- 172.21.15.92:0/928519223 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f16ec06aed0 0x7f16ec06d390 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:22.025 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:22.021+0000 7f171121c640 1 --2- 172.21.15.92:0/928519223 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f170c15e880 0x7f170c0ca750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:22.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:22.021+0000 7f171121c640 1 --2- 172.21.15.92:0/928519223 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f170c099fa0 0x7f170c0c6350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:22.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:22.021+0000 7f171121c640 1 --2- 172.21.15.92:0/928519223 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f170c070490 0x7f170c159950 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:22.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:22.022+0000 7f171121c640 1 -- 172.21.15.92:0/928519223 >> 172.21.15.92:0/928519223 conn(0x7f170c0b2d00 msgr2=0x7f170c0ceb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:13:22.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:22.022+0000 7f171121c640 1 -- 172.21.15.92:0/928519223 shutdown_connections 2023-11-24T06:13:22.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:22.022+0000 7f171121c640 1 -- 172.21.15.92:0/928519223 wait complete. 2023-11-24T06:13:22.043 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-24T06:13:22.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:22.253+0000 7f54178e0640 1 Processor -- start 2023-11-24T06:13:22.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:22.253+0000 7f54178e0640 1 -- start start 2023-11-24T06:13:22.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:22.255+0000 7f54178e0640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f541013df80 0x7f541015bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:22.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:22.255+0000 7f54178e0640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f541013e360 0x7f54100cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:22.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:22.255+0000 7f54178e0640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f541006c470 con 0x7f541015f9c0 2023-11-24T06:13:22.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:22.255+0000 7f54178e0640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f541006c750 con 0x7f541013e360 2023-11-24T06:13:22.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:22.255+0000 7f54178e0640 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f541006ca30 con 0x7f541013df80 2023-11-24T06:13:22.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:22.255+0000 7f5414e7b640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f541013e360 0x7f54100cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:22.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:22.255+0000 7f5414e7b640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f541013e360 0x7f54100cf290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43722/0 (socket says 172.21.15.92:43722) 2023-11-24T06:13:22.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:22.255+0000 7f5414e7b640 1 -- 172.21.15.92:0/2836543644 learned_addr learned my addr 172.21.15.92:0/2836543644 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:13:22.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:22.256+0000 7f541567c640 1 --2- 172.21.15.92:0/2836543644 >> v2:172.21.15.118:3300/0 conn(0x7f541013df80 0x7f541015bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:22.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:22.256+0000 7f5414e7b640 1 -- 172.21.15.92:0/2836543644 >> v2:172.21.15.118:3300/0 conn(0x7f541013df80 msgr2=0x7f541015bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:13:22.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:22.256+0000 7f5414e7b640 1 --2- 172.21.15.92:0/2836543644 >> v2:172.21.15.118:3300/0 conn(0x7f541013df80 0x7f541015bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:22.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:22.256+0000 7f5414e7b640 1 -- 172.21.15.92:0/2836543644 >> v1:172.21.15.92:6789/0 conn(0x7f541015f9c0 legacy=0x7f54101408d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:22.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:22.256+0000 7f5414e7b640 1 -- 172.21.15.92:0/2836543644 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f541006cd10 con 0x7f541013e360 2023-11-24T06:13:22.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:22.256+0000 7f5407fff640 1 -- 172.21.15.92:0/2836543644 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1189714130 0 0) 0x7f541006c470 con 0x7f541015f9c0 2023-11-24T06:13:22.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:22.257+0000 7f5414e7b640 1 --2- 172.21.15.92:0/2836543644 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f541013e360 0x7f54100cf290 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f5410072770 tx=0x7f540002b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98ed41469cfe9bef server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:22.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:22.258+0000 7f5407fff640 1 -- 172.21.15.92:0/2836543644 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5400030860 con 0x7f541013e360 2023-11-24T06:13:22.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:22.258+0000 7f5407fff640 1 -- 172.21.15.92:0/2836543644 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54000309c0 con 0x7f541013e360 2023-11-24T06:13:22.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:22.258+0000 7f5407fff640 1 -- 172.21.15.92:0/2836543644 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5400030d50 con 0x7f541013e360 2023-11-24T06:13:22.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:22.258+0000 7f54178e0640 1 -- 172.21.15.92:0/2836543644 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f541013e360 msgr2=0x7f54100cf290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:22.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:22.258+0000 7f54178e0640 1 --2- 172.21.15.92:0/2836543644 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f541013e360 0x7f54100cf290 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f5410072770 tx=0x7f540002b9a0 comp rx=0 tx=0).stop 2023-11-24T06:13:22.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:22.259+0000 7f54178e0640 1 -- 172.21.15.92:0/2836543644 shutdown_connections 2023-11-24T06:13:22.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:22.259+0000 7f54178e0640 1 --2- 172.21.15.92:0/2836543644 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f541013e360 0x7f54100cf290 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:22.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:22.259+0000 7f54178e0640 1 --2- 172.21.15.92:0/2836543644 >> v2:172.21.15.118:3300/0 conn(0x7f541013df80 0x7f541015bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:22.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:22.259+0000 7f54178e0640 1 -- 172.21.15.92:0/2836543644 >> 172.21.15.92:0/2836543644 conn(0x7f541006a800 msgr2=0x7f541006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:13:22.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:22.259+0000 7f54178e0640 1 -- 172.21.15.92:0/2836543644 shutdown_connections 2023-11-24T06:13:22.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:22.259+0000 7f54178e0640 1 -- 172.21.15.92:0/2836543644 wait complete. 2023-11-24T06:13:22.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:22.260+0000 7f54178e0640 1 Processor -- start 2023-11-24T06:13:22.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:22.261+0000 7f54178e0640 1 -- start start 2023-11-24T06:13:22.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:22.261+0000 7f54178e0640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f541013df80 0x7f54101f6090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:22.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:22.262+0000 7f54178e0640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f541013e360 0x7f5410070720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:22.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:22.262+0000 7f54178e0640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f541015f9c0 0x7f5410070c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:22.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:22.262+0000 7f54178e0640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5410135c40 con 0x7f541013e360 2023-11-24T06:13:22.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:22.262+0000 7f54178e0640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5410072330 con 0x7f541013df80 2023-11-24T06:13:22.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:22.262+0000 7f54178e0640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5410071290 con 0x7f541015f9c0 2023-11-24T06:13:22.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:22.262+0000 7f5414e7b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f541013e360 0x7f5410070720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:22.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:22.262+0000 7f5414e7b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f541013e360 0x7f5410070720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43728/0 (socket says 172.21.15.92:43728) 2023-11-24T06:13:22.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:22.262+0000 7f5414e7b640 1 -- 172.21.15.92:0/3293559136 learned_addr learned my addr 172.21.15.92:0/3293559136 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:13:22.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:22.262+0000 7f541567c640 1 --2- 172.21.15.92:0/3293559136 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f541013df80 0x7f54101f6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:22.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:22.262+0000 7f5415e7d640 1 --2- 172.21.15.92:0/3293559136 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f541015f9c0 0x7f5410070c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:22.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:22.262+0000 7f5414e7b640 1 -- 172.21.15.92:0/3293559136 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f541015f9c0 msgr2=0x7f5410070c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:22.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:22.262+0000 7f5414e7b640 1 --2- 172.21.15.92:0/3293559136 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f541015f9c0 0x7f5410070c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:22.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:22.262+0000 7f5414e7b640 1 -- 172.21.15.92:0/3293559136 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f541013df80 msgr2=0x7f54101f6090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:22.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:22.262+0000 7f5414e7b640 1 --2- 172.21.15.92:0/3293559136 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f541013df80 0x7f54101f6090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:22.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:22.262+0000 7f5414e7b640 1 -- 172.21.15.92:0/3293559136 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54000094d0 con 0x7f541013e360 2023-11-24T06:13:22.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:22.262+0000 7f541567c640 1 --2- 172.21.15.92:0/3293559136 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f541013df80 0x7f54101f6090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:13:22.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:22.263+0000 7f5414e7b640 1 --2- 172.21.15.92:0/3293559136 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f541013e360 0x7f5410070720 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f5400000c00 tx=0x7f5400004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:22.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:22.263+0000 7f5405ffb640 1 -- 172.21.15.92:0/3293559136 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5400039c90 con 0x7f541013e360 2023-11-24T06:13:22.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:22.263+0000 7f5405ffb640 1 -- 172.21.15.92:0/3293559136 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5400039df0 con 0x7f541013e360 2023-11-24T06:13:22.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:22.263+0000 7f5405ffb640 1 -- 172.21.15.92:0/3293559136 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5400043620 con 0x7f541013e360 2023-11-24T06:13:22.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:22.263+0000 7f54178e0640 1 -- 172.21.15.92:0/3293559136 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5410071570 con 0x7f541013e360 2023-11-24T06:13:22.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:22.263+0000 7f54178e0640 1 -- 172.21.15.92:0/3293559136 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5410071b30 con 0x7f541013e360 2023-11-24T06:13:22.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:22.264+0000 7f54178e0640 1 -- 172.21.15.92:0/3293559136 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54100675d0 con 0x7f541013e360 2023-11-24T06:13:22.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:22.265+0000 7f5405ffb640 1 -- 172.21.15.92:0/3293559136 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f5400043780 con 0x7f541013e360 2023-11-24T06:13:22.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:22.267+0000 7f5405ffb640 1 --2- 172.21.15.92:0/3293559136 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f53f406aed0 0x7f53f406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:22.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:22.267+0000 7f5405ffb640 1 -- 172.21.15.92:0/3293559136 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f540003e070 con 0x7f541013e360 2023-11-24T06:13:22.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:22.267+0000 7f541567c640 1 --2- 172.21.15.92:0/3293559136 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f53f406aed0 0x7f53f406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:22.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:22.268+0000 7f541567c640 1 --2- 172.21.15.92:0/3293559136 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f53f406aed0 0x7f53f406d390 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f540c006fd0 tx=0x7f540c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:22.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:22.271+0000 7f5405ffb640 1 -- 172.21.15.92:0/3293559136 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5400077a70 con 0x7f541013e360 2023-11-24T06:13:22.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:22.465+0000 7f54178e0640 1 -- 172.21.15.92:0/3293559136 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f541006df00 con 0x7f541013e360 2023-11-24T06:13:22.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:22.466+0000 7f5405ffb640 1 -- 172.21.15.92:0/3293559136 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v14)=0 v14) v1 ==== 122+0+0 (secure 0 0 0) 0x7f540007b090 con 0x7f541013e360 2023-11-24T06:13:22.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:22.469+0000 7f54178e0640 1 -- 172.21.15.92:0/3293559136 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f53f406aed0 msgr2=0x7f53f406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:22.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:22.469+0000 7f54178e0640 1 --2- 172.21.15.92:0/3293559136 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f53f406aed0 0x7f53f406d390 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f540c006fd0 tx=0x7f540c008040 comp rx=0 tx=0).stop 2023-11-24T06:13:22.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:22.469+0000 7f54178e0640 1 -- 172.21.15.92:0/3293559136 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f541013e360 msgr2=0x7f5410070720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:22.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:22.469+0000 7f54178e0640 1 --2- 172.21.15.92:0/3293559136 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f541013e360 0x7f5410070720 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f5400000c00 tx=0x7f5400004730 comp rx=0 tx=0).stop 2023-11-24T06:13:22.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:22.469+0000 7f54178e0640 1 -- 172.21.15.92:0/3293559136 shutdown_connections 2023-11-24T06:13:22.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:22.469+0000 7f54178e0640 1 --2- 172.21.15.92:0/3293559136 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f53f406aed0 0x7f53f406d390 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:22.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:22.469+0000 7f54178e0640 1 --2- 172.21.15.92:0/3293559136 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f541015f9c0 0x7f5410070c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:22.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:22.469+0000 7f54178e0640 1 --2- 172.21.15.92:0/3293559136 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f541013e360 0x7f5410070720 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:22.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:22.469+0000 7f54178e0640 1 --2- 172.21.15.92:0/3293559136 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f541013df80 0x7f54101f6090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:22.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:22.469+0000 7f54178e0640 1 -- 172.21.15.92:0/3293559136 >> 172.21.15.92:0/3293559136 conn(0x7f541006a800 msgr2=0x7f541015c850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:13:22.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:22.469+0000 7f54178e0640 1 -- 172.21.15.92:0/3293559136 shutdown_connections 2023-11-24T06:13:22.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:22.469+0000 7f54178e0640 1 -- 172.21.15.92:0/3293559136 wait complete. 2023-11-24T06:13:22.489 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs subvol9124 --mode=777 2023-11-24T06:13:22.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:22.702+0000 7f7329d0a640 1 Processor -- start 2023-11-24T06:13:22.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:22.702+0000 7f7329d0a640 1 -- start start 2023-11-24T06:13:22.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:22.703+0000 7f7329d0a640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f732413df80 0x7f732415bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:22.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:22.703+0000 7f7329d0a640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f732413e360 0x7f73240cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:22.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:22.703+0000 7f7329d0a640 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f732406c470 con 0x7f732415f9c0 2023-11-24T06:13:22.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:22.703+0000 7f7329d0a640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f732406c750 con 0x7f732413df80 2023-11-24T06:13:22.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:22.703+0000 7f7329d0a640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f732406ca30 con 0x7f732413e360 2023-11-24T06:13:22.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:22.704+0000 7f73237fe640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f732413df80 0x7f732415bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:22.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:22.704+0000 7f73237fe640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f732413df80 0x7f732415bee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43732/0 (socket says 172.21.15.92:43732) 2023-11-24T06:13:22.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:22.704+0000 7f73237fe640 1 -- 172.21.15.92:0/2631908627 learned_addr learned my addr 172.21.15.92:0/2631908627 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:13:22.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:22.704+0000 7f731bfff640 1 --2- 172.21.15.92:0/2631908627 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f732413e360 0x7f73240cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:22.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:22.704+0000 7f73237fe640 1 -- 172.21.15.92:0/2631908627 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f732413e360 msgr2=0x7f73240cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:13:22.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:22.704+0000 7f73237fe640 1 --2- 172.21.15.92:0/2631908627 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f732413e360 0x7f73240cf290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:22.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:22.704+0000 7f73237fe640 1 -- 172.21.15.92:0/2631908627 >> v1:172.21.15.118:6789/0 conn(0x7f732415f9c0 legacy=0x7f73241408d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:13:22.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:22.704+0000 7f73237fe640 1 -- 172.21.15.92:0/2631908627 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f732406cd10 con 0x7f732413df80 2023-11-24T06:13:22.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:22.706+0000 7f73237fe640 1 --2- 172.21.15.92:0/2631908627 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f732413df80 0x7f732415bee0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f7314009870 tx=0x7f731402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee7fe03360b4f02a server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:22.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:22.706+0000 7f7322ffd640 1 -- 172.21.15.92:0/2631908627 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7314031640 con 0x7f732413df80 2023-11-24T06:13:22.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:22.706+0000 7f7322ffd640 1 -- 172.21.15.92:0/2631908627 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73140317a0 con 0x7f732413df80 2023-11-24T06:13:22.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:22.706+0000 7f7322ffd640 1 -- 172.21.15.92:0/2631908627 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7314031b30 con 0x7f732413df80 2023-11-24T06:13:22.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:22.707+0000 7f7329d0a640 1 -- 172.21.15.92:0/2631908627 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f732413df80 msgr2=0x7f732415bee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:22.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:22.707+0000 7f7329d0a640 1 --2- 172.21.15.92:0/2631908627 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f732413df80 0x7f732415bee0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f7314009870 tx=0x7f731402b8f0 comp rx=0 tx=0).stop 2023-11-24T06:13:22.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:22.707+0000 7f7329d0a640 1 -- 172.21.15.92:0/2631908627 shutdown_connections 2023-11-24T06:13:22.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:22.707+0000 7f7329d0a640 1 --2- 172.21.15.92:0/2631908627 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f732413e360 0x7f73240cf290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:22.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:22.707+0000 7f7329d0a640 1 --2- 172.21.15.92:0/2631908627 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f732413df80 0x7f732415bee0 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:22.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:22.707+0000 7f7329d0a640 1 -- 172.21.15.92:0/2631908627 >> 172.21.15.92:0/2631908627 conn(0x7f732406a800 msgr2=0x7f732406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:13:22.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:22.707+0000 7f7329d0a640 1 -- 172.21.15.92:0/2631908627 shutdown_connections 2023-11-24T06:13:22.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:22.707+0000 7f7329d0a640 1 -- 172.21.15.92:0/2631908627 wait complete. 2023-11-24T06:13:22.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:22.708+0000 7f7329d0a640 1 Processor -- start 2023-11-24T06:13:22.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:22.709+0000 7f7329d0a640 1 -- start start 2023-11-24T06:13:22.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:22.709+0000 7f7329d0a640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f732413df80 0x7f7324150f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:22.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:22.710+0000 7f7329d0a640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f732413e360 0x7f732414df10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:22.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:22.710+0000 7f7329d0a640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f732415f9c0 0x7f732413d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:22.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:22.710+0000 7f7329d0a640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f732413de30 con 0x7f732413e360 2023-11-24T06:13:22.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:22.710+0000 7f7329d0a640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f732415a9b0 con 0x7f732415f9c0 2023-11-24T06:13:22.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:22.710+0000 7f7329d0a640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f732414e560 con 0x7f732413df80 2023-11-24T06:13:22.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:22.710+0000 7f731bfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f732413e360 0x7f732414df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:22.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:22.710+0000 7f731bfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f732413e360 0x7f732414df10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43740/0 (socket says 172.21.15.92:43740) 2023-11-24T06:13:22.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:22.710+0000 7f731bfff640 1 -- 172.21.15.92:0/4014299784 learned_addr learned my addr 172.21.15.92:0/4014299784 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:13:22.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:22.710+0000 7f73237fe640 1 --2- 172.21.15.92:0/4014299784 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f732413df80 0x7f7324150f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:22.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:22.710+0000 7f7323fff640 1 --2- 172.21.15.92:0/4014299784 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f732415f9c0 0x7f732413d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:22.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:22.710+0000 7f731bfff640 1 -- 172.21.15.92:0/4014299784 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f732413df80 msgr2=0x7f7324150f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:22.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:22.710+0000 7f731bfff640 1 --2- 172.21.15.92:0/4014299784 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f732413df80 0x7f7324150f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:22.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:22.710+0000 7f731bfff640 1 -- 172.21.15.92:0/4014299784 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f732415f9c0 msgr2=0x7f732413d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:22.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:22.710+0000 7f731bfff640 1 --2- 172.21.15.92:0/4014299784 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f732415f9c0 0x7f732413d8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:22.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:22.710+0000 7f731bfff640 1 -- 172.21.15.92:0/4014299784 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73140094d0 con 0x7f732413e360 2023-11-24T06:13:22.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:22.711+0000 7f731bfff640 1 --2- 172.21.15.92:0/4014299784 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f732413e360 0x7f732414df10 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f730c00d800 tx=0x7f730c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:22.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:22.711+0000 7f7320ff9640 1 -- 172.21.15.92:0/4014299784 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f730c00b6c0 con 0x7f732413e360 2023-11-24T06:13:22.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:22.711+0000 7f7320ff9640 1 -- 172.21.15.92:0/4014299784 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f730c00b820 con 0x7f732413e360 2023-11-24T06:13:22.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:22.711+0000 7f7329d0a640 1 -- 172.21.15.92:0/4014299784 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73241f0c40 con 0x7f732413e360 2023-11-24T06:13:22.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:22.711+0000 7f7320ff9640 1 -- 172.21.15.92:0/4014299784 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f730c00bbb0 con 0x7f732413e360 2023-11-24T06:13:22.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:22.711+0000 7f7329d0a640 1 -- 172.21.15.92:0/4014299784 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73241f1200 con 0x7f732413e360 2023-11-24T06:13:22.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:22.712+0000 7f7329d0a640 1 -- 172.21.15.92:0/4014299784 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73240675d0 con 0x7f732413e360 2023-11-24T06:13:22.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:22.713+0000 7f7320ff9640 1 -- 172.21.15.92:0/4014299784 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f730c00bd10 con 0x7f732413e360 2023-11-24T06:13:22.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:22.714+0000 7f7320ff9640 1 --2- 172.21.15.92:0/4014299784 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f730406aed0 0x7f730406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:22.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:22.714+0000 7f7320ff9640 1 -- 172.21.15.92:0/4014299784 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f730c089ca0 con 0x7f732413e360 2023-11-24T06:13:22.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:22.714+0000 7f73237fe640 1 --2- 172.21.15.92:0/4014299784 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f730406aed0 0x7f730406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:22.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:22.715+0000 7f73237fe640 1 --2- 172.21.15.92:0/4014299784 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f730406aed0 0x7f730406d390 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f7314036000 tx=0x7f731403c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:22.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:22.718+0000 7f7320ff9640 1 -- 172.21.15.92:0/4014299784 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f730c054c60 con 0x7f732413e360 2023-11-24T06:13:22.942 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:22.939+0000 7f7329d0a640 1 -- 172.21.15.92:0/4014299784 --> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvol9124", "mode": "777", "target": ["mon-mgr", ""]}) v1 -- 0x7f73240ce850 con 0x7f730406aed0 2023-11-24T06:13:22.950 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-11-24T06:13:22.947+0000 7ff62b16c640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T06:13:22.950 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-11-24T06:13:22.947+0000 7ff62b16c640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T06:13:22.950 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-11-24T06:13:22.947+0000 7ff62b16c640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T06:13:22.950 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-11-24T06:13:22.947+0000 7ff62b16c640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T06:13:22.951 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-11-24T06:13:22.947+0000 7ff62b16c640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T06:13:23.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:22.998+0000 7f7320ff9640 1 -- 172.21.15.92:0/4014299784 <== mgr.4101 v2:172.21.15.92:6800/784027401 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f7314031a80 con 0x7f730406aed0 2023-11-24T06:13:23.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:23.001+0000 7f7329d0a640 1 -- 172.21.15.92:0/4014299784 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f730406aed0 msgr2=0x7f730406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:23.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:23.001+0000 7f7329d0a640 1 --2- 172.21.15.92:0/4014299784 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f730406aed0 0x7f730406d390 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f7314036000 tx=0x7f731403c040 comp rx=0 tx=0).stop 2023-11-24T06:13:23.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:23.001+0000 7f7329d0a640 1 -- 172.21.15.92:0/4014299784 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f732413e360 msgr2=0x7f732414df10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:23.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:23.001+0000 7f7329d0a640 1 --2- 172.21.15.92:0/4014299784 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f732413e360 0x7f732414df10 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f730c00d800 tx=0x7f730c00dcc0 comp rx=0 tx=0).stop 2023-11-24T06:13:23.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:23.001+0000 7f7329d0a640 1 -- 172.21.15.92:0/4014299784 shutdown_connections 2023-11-24T06:13:23.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:23.001+0000 7f7329d0a640 1 --2- 172.21.15.92:0/4014299784 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f730406aed0 0x7f730406d390 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:23.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:23.001+0000 7f7329d0a640 1 --2- 172.21.15.92:0/4014299784 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f732415f9c0 0x7f732413d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:23.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:23.001+0000 7f7329d0a640 1 --2- 172.21.15.92:0/4014299784 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f732413e360 0x7f732414df10 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:23.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:23.001+0000 7f7329d0a640 1 --2- 172.21.15.92:0/4014299784 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f732413df80 0x7f7324150f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:23.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:23.001+0000 7f7329d0a640 1 -- 172.21.15.92:0/4014299784 >> 172.21.15.92:0/4014299784 conn(0x7f732406a800 msgr2=0x7f732415d260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:13:23.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:23.001+0000 7f7329d0a640 1 -- 172.21.15.92:0/4014299784 shutdown_connections 2023-11-24T06:13:23.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:23.002+0000 7f7329d0a640 1 -- 172.21.15.92:0/4014299784 wait complete. 2023-11-24T06:13:23.023 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol9124 2023-11-24T06:13:23.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:23.219+0000 7fb6d469b640 1 Processor -- start 2023-11-24T06:13:23.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:23.219+0000 7fb6d469b640 1 -- start start 2023-11-24T06:13:23.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:23.221+0000 7fb6d469b640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb6cc0b3ae0 0x7fb6cc15e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:23.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:23.221+0000 7fb6d469b640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb6cc0cf290 0x7fb6cc0cf720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:23.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:23.221+0000 7fb6d469b640 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6cc070c40 con 0x7fb6cc0ae5a0 2023-11-24T06:13:23.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:23.221+0000 7fb6d469b640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6cc071450 con 0x7fb6cc0b3ae0 2023-11-24T06:13:23.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:23.221+0000 7fb6d469b640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6cc0cfc60 con 0x7fb6cc0cf290 2023-11-24T06:13:23.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:23.221+0000 7fb6d1c36640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb6cc0b3ae0 0x7fb6cc15e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:23.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:23.221+0000 7fb6d1c36640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb6cc0b3ae0 0x7fb6cc15e430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43776/0 (socket says 172.21.15.92:43776) 2023-11-24T06:13:23.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:23.221+0000 7fb6d1c36640 1 -- 172.21.15.92:0/3520499842 learned_addr learned my addr 172.21.15.92:0/3520499842 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:13:23.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:23.222+0000 7fb6d1c36640 1 -- 172.21.15.92:0/3520499842 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb6cc0cf290 msgr2=0x7fb6cc0cf720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:13:23.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:23.222+0000 7fb6d1c36640 1 --2- 172.21.15.92:0/3520499842 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb6cc0cf290 0x7fb6cc0cf720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:23.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:23.222+0000 7fb6d1c36640 1 -- 172.21.15.92:0/3520499842 >> v1:172.21.15.118:6789/0 conn(0x7fb6cc0ae5a0 legacy=0x7fb6cc06cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:13:23.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:23.222+0000 7fb6d1c36640 1 -- 172.21.15.92:0/3520499842 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6cc06c470 con 0x7fb6cc0b3ae0 2023-11-24T06:13:23.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:23.223+0000 7fb6d1c36640 1 --2- 172.21.15.92:0/3520499842 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb6cc0b3ae0 0x7fb6cc15e430 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fb6c8009e50 tx=0x7fb6c802bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc1c7938924655b0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:23.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:23.223+0000 7fb6d1435640 1 -- 172.21.15.92:0/3520499842 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6c802cb00 con 0x7fb6cc0b3ae0 2023-11-24T06:13:23.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:23.223+0000 7fb6d1435640 1 -- 172.21.15.92:0/3520499842 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6c802cc60 con 0x7fb6cc0b3ae0 2023-11-24T06:13:23.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:23.223+0000 7fb6d1435640 1 -- 172.21.15.92:0/3520499842 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6c80343f0 con 0x7fb6cc0b3ae0 2023-11-24T06:13:23.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:23.224+0000 7fb6d469b640 1 -- 172.21.15.92:0/3520499842 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb6cc0b3ae0 msgr2=0x7fb6cc15e430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:23.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:23.224+0000 7fb6d469b640 1 --2- 172.21.15.92:0/3520499842 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb6cc0b3ae0 0x7fb6cc15e430 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fb6c8009e50 tx=0x7fb6c802bf60 comp rx=0 tx=0).stop 2023-11-24T06:13:23.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:23.224+0000 7fb6d469b640 1 -- 172.21.15.92:0/3520499842 shutdown_connections 2023-11-24T06:13:23.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:23.224+0000 7fb6d469b640 1 --2- 172.21.15.92:0/3520499842 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb6cc0cf290 0x7fb6cc0cf720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:23.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:23.224+0000 7fb6d469b640 1 --2- 172.21.15.92:0/3520499842 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb6cc0b3ae0 0x7fb6cc15e430 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:23.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:23.224+0000 7fb6d469b640 1 -- 172.21.15.92:0/3520499842 >> 172.21.15.92:0/3520499842 conn(0x7fb6cc06a800 msgr2=0x7fb6cc06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:13:23.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:23.224+0000 7fb6d469b640 1 -- 172.21.15.92:0/3520499842 shutdown_connections 2023-11-24T06:13:23.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:23.224+0000 7fb6d469b640 1 -- 172.21.15.92:0/3520499842 wait complete. 2023-11-24T06:13:23.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:23.227+0000 7fb6d469b640 1 Processor -- start 2023-11-24T06:13:23.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:23.227+0000 7fb6d469b640 1 -- start start 2023-11-24T06:13:23.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:23.228+0000 7fb6d469b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb6cc0ae5a0 0x7fb6cc15ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:23.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:23.228+0000 7fb6d469b640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb6cc0b3ae0 0x7fb6cc145850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:23.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:23.228+0000 7fb6d469b640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb6cc0cf290 0x7fb6cc1584f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:23.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:23.228+0000 7fb6d469b640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6cc160510 con 0x7fb6cc0ae5a0 2023-11-24T06:13:23.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:23.228+0000 7fb6d469b640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6cc15fb40 con 0x7fb6cc0cf290 2023-11-24T06:13:23.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:23.228+0000 7fb6d469b640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6cc15ef30 con 0x7fb6cc0b3ae0 2023-11-24T06:13:23.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:23.228+0000 7fb6d2437640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb6cc0ae5a0 0x7fb6cc15ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:23.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:23.228+0000 7fb6d2437640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb6cc0ae5a0 0x7fb6cc15ffd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43788/0 (socket says 172.21.15.92:43788) 2023-11-24T06:13:23.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:23.228+0000 7fb6d2437640 1 -- 172.21.15.92:0/3294275389 learned_addr learned my addr 172.21.15.92:0/3294275389 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:13:23.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:23.229+0000 7fb6d2437640 1 -- 172.21.15.92:0/3294275389 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb6cc0b3ae0 msgr2=0x7fb6cc145850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:23.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:23.229+0000 7fb6d2437640 1 --2- 172.21.15.92:0/3294275389 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb6cc0b3ae0 0x7fb6cc145850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:23.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:23.229+0000 7fb6d2437640 1 -- 172.21.15.92:0/3294275389 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb6cc0cf290 msgr2=0x7fb6cc1584f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:23.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:23.229+0000 7fb6d2437640 1 --2- 172.21.15.92:0/3294275389 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb6cc0cf290 0x7fb6cc1584f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:23.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:23.229+0000 7fb6d2437640 1 -- 172.21.15.92:0/3294275389 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6c8009b30 con 0x7fb6cc0ae5a0 2023-11-24T06:13:23.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:23.229+0000 7fb6d2437640 1 --2- 172.21.15.92:0/3294275389 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb6cc0ae5a0 0x7fb6cc15ffd0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fb6c000c410 tx=0x7fb6c000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:23.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:23.229+0000 7fb6beffd640 1 -- 172.21.15.92:0/3294275389 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6c00035f0 con 0x7fb6cc0ae5a0 2023-11-24T06:13:23.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:23.229+0000 7fb6beffd640 1 -- 172.21.15.92:0/3294275389 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6c0003750 con 0x7fb6cc0ae5a0 2023-11-24T06:13:23.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:23.229+0000 7fb6beffd640 1 -- 172.21.15.92:0/3294275389 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6c0003b00 con 0x7fb6cc0ae5a0 2023-11-24T06:13:23.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:23.229+0000 7fb6d469b640 1 -- 172.21.15.92:0/3294275389 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6cc15f210 con 0x7fb6cc0ae5a0 2023-11-24T06:13:23.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:23.229+0000 7fb6d469b640 1 -- 172.21.15.92:0/3294275389 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6cc15f7d0 con 0x7fb6cc0ae5a0 2023-11-24T06:13:23.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:23.230+0000 7fb6d469b640 1 -- 172.21.15.92:0/3294275389 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6cc0ac920 con 0x7fb6cc0ae5a0 2023-11-24T06:13:23.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:23.232+0000 7fb6beffd640 1 -- 172.21.15.92:0/3294275389 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7fb6c0003c60 con 0x7fb6cc0ae5a0 2023-11-24T06:13:23.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:23.233+0000 7fb6beffd640 1 --2- 172.21.15.92:0/3294275389 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fb6b006aed0 0x7fb6b006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:23.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:23.234+0000 7fb6beffd640 1 -- 172.21.15.92:0/3294275389 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb6c0088980 con 0x7fb6cc0ae5a0 2023-11-24T06:13:23.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:23.234+0000 7fb6d1c36640 1 --2- 172.21.15.92:0/3294275389 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fb6b006aed0 0x7fb6b006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:23.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:23.234+0000 7fb6d1c36640 1 --2- 172.21.15.92:0/3294275389 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fb6b006aed0 0x7fb6b006d390 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fb6c8035000 tx=0x7fb6c802e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:23.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:23.239+0000 7fb6beffd640 1 -- 172.21.15.92:0/3294275389 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb6c0053940 con 0x7fb6cc0ae5a0 2023-11-24T06:13:23.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:23.457+0000 7fb6d469b640 1 -- 172.21.15.92:0/3294275389 --> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol9124", "target": ["mon-mgr", ""]}) v1 -- 0x7fb6cc071170 con 0x7fb6b006aed0 2023-11-24T06:13:23.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:23.468+0000 7fb6beffd640 1 -- 172.21.15.92:0/3294275389 <== mgr.4101 v2:172.21.15.92:6800/784027401 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+66 (secure 0 0 0) 0x7fb6c802cad0 con 0x7fb6b006aed0 2023-11-24T06:13:23.471 INFO:teuthology.orchestra.run.smithi092.stdout:/volumes/_nogroup/subvol9124/dd273087-4728-4200-b033-360841e11f01 2023-11-24T06:13:23.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:23.471+0000 7fb6d469b640 1 -- 172.21.15.92:0/3294275389 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fb6b006aed0 msgr2=0x7fb6b006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:23.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:23.471+0000 7fb6d469b640 1 --2- 172.21.15.92:0/3294275389 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fb6b006aed0 0x7fb6b006d390 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fb6c8035000 tx=0x7fb6c802e000 comp rx=0 tx=0).stop 2023-11-24T06:13:23.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:23.471+0000 7fb6d469b640 1 -- 172.21.15.92:0/3294275389 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb6cc0ae5a0 msgr2=0x7fb6cc15ffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:23.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:23.471+0000 7fb6d469b640 1 --2- 172.21.15.92:0/3294275389 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb6cc0ae5a0 0x7fb6cc15ffd0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fb6c000c410 tx=0x7fb6c000c8d0 comp rx=0 tx=0).stop 2023-11-24T06:13:23.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:23.471+0000 7fb6d469b640 1 -- 172.21.15.92:0/3294275389 shutdown_connections 2023-11-24T06:13:23.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:23.471+0000 7fb6d469b640 1 --2- 172.21.15.92:0/3294275389 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fb6b006aed0 0x7fb6b006d390 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:23.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:23.471+0000 7fb6d469b640 1 --2- 172.21.15.92:0/3294275389 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb6cc0cf290 0x7fb6cc1584f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:23.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:23.471+0000 7fb6d469b640 1 --2- 172.21.15.92:0/3294275389 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb6cc0b3ae0 0x7fb6cc145850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:23.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:23.471+0000 7fb6d469b640 1 --2- 172.21.15.92:0/3294275389 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb6cc0ae5a0 0x7fb6cc15ffd0 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:23.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:23.471+0000 7fb6d469b640 1 -- 172.21.15.92:0/3294275389 >> 172.21.15.92:0/3294275389 conn(0x7fb6cc06a800 msgr2=0x7fb6cc160f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:13:23.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:23.471+0000 7fb6d469b640 1 -- 172.21.15.92:0/3294275389 shutdown_connections 2023-11-24T06:13:23.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:23.471+0000 7fb6d469b640 1 -- 172.21.15.92:0/3294275389 wait complete. 2023-11-24T06:13:23.492 DEBUG:tasks.cephfs.test_volumes:filling subvolume subvol9124 with 3000 files each 0MB size under directory volumes/_nogroup/subvol9124/dd273087-4728-4200-b033-360841e11f01 2023-11-24T06:13:23.492 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-24T06:13:23.492 DEBUG:teuthology.orchestra.run.smithi092:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-24T06:13:23.604 DEBUG:teuthology.orchestra.run.smithi118:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-24T06:13:23.726 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-24T06:13:23.726 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-11-24T06:13:23.727 DEBUG:teuthology.orchestra.run.smithi092:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:13:23.789 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:13:24.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.001+0000 7f58564bb640 1 Processor -- start 2023-11-24T06:13:24.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.002+0000 7f58564bb640 1 -- start start 2023-11-24T06:13:24.006 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.003+0000 7f58564bb640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f585013df80 0x7f585015bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:24.006 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.003+0000 7f58564bb640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f585013e360 0x7f58500cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:24.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.003+0000 7f58564bb640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f585015f9c0 0x7f58500cf7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:24.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.003+0000 7f58564bb640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f585006c470 con 0x7f585013df80 2023-11-24T06:13:24.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.003+0000 7f58564bb640 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f585006c750 con 0x7f585015f9c0 2023-11-24T06:13:24.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.003+0000 7f58564bb640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f585006ca30 con 0x7f585013e360 2023-11-24T06:13:24.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.003+0000 7f584ffff640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f585013df80 0x7f585015bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:24.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.003+0000 7f584ffff640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f585013df80 0x7f585015bee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43790/0 (socket says 172.21.15.92:43790) 2023-11-24T06:13:24.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.003+0000 7f584ffff640 1 -- 172.21.15.92:0/1503259307 learned_addr learned my addr 172.21.15.92:0/1503259307 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:13:24.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.004+0000 7f5847fff640 1 --2- 172.21.15.92:0/1503259307 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f585013e360 0x7f58500cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:24.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.004+0000 7f584ffff640 1 -- 172.21.15.92:0/1503259307 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f585013e360 msgr2=0x7f58500cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:13:24.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.004+0000 7f5854a58640 1 --2- 172.21.15.92:0/1503259307 >> v2:172.21.15.118:3300/0 conn(0x7f585015f9c0 0x7f58500cf7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:24.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.004+0000 7f584ffff640 1 --2- 172.21.15.92:0/1503259307 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f585013e360 0x7f58500cf290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:24.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.004+0000 7f584ffff640 1 -- 172.21.15.92:0/1503259307 >> v2:172.21.15.118:3300/0 conn(0x7f585015f9c0 msgr2=0x7f58500cf7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:13:24.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.004+0000 7f584ffff640 1 --2- 172.21.15.92:0/1503259307 >> v2:172.21.15.118:3300/0 conn(0x7f585015f9c0 0x7f58500cf7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:24.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.004+0000 7f584ffff640 1 -- 172.21.15.92:0/1503259307 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f585006cd10 con 0x7f585013df80 2023-11-24T06:13:24.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.004+0000 7f5854a58640 1 --2- 172.21.15.92:0/1503259307 >> v2:172.21.15.118:3300/0 conn(0x7f585015f9c0 0x7f58500cf7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:13:24.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.005+0000 7f584ffff640 1 --2- 172.21.15.92:0/1503259307 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f585013df80 0x7f585015bee0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f585006e6d0 tx=0x7f584002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=337cc88dbcb35b29 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:24.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.005+0000 7f584f7fe640 1 -- 172.21.15.92:0/1503259307 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5840031870 con 0x7f585013df80 2023-11-24T06:13:24.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.005+0000 7f584f7fe640 1 -- 172.21.15.92:0/1503259307 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58400319d0 con 0x7f585013df80 2023-11-24T06:13:24.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.005+0000 7f584f7fe640 1 -- 172.21.15.92:0/1503259307 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5840031d60 con 0x7f585013df80 2023-11-24T06:13:24.010 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.006+0000 7f58564bb640 1 -- 172.21.15.92:0/1503259307 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f585013df80 msgr2=0x7f585015bee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:24.010 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.006+0000 7f58564bb640 1 --2- 172.21.15.92:0/1503259307 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f585013df80 0x7f585015bee0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f585006e6d0 tx=0x7f584002b910 comp rx=0 tx=0).stop 2023-11-24T06:13:24.010 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.006+0000 7f58564bb640 1 -- 172.21.15.92:0/1503259307 shutdown_connections 2023-11-24T06:13:24.010 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.006+0000 7f58564bb640 1 --2- 172.21.15.92:0/1503259307 >> v2:172.21.15.118:3300/0 conn(0x7f585015f9c0 0x7f58500cf7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:24.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.006+0000 7f58564bb640 1 --2- 172.21.15.92:0/1503259307 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f585013e360 0x7f58500cf290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:24.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.006+0000 7f58564bb640 1 --2- 172.21.15.92:0/1503259307 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f585013df80 0x7f585015bee0 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:24.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.006+0000 7f58564bb640 1 -- 172.21.15.92:0/1503259307 >> 172.21.15.92:0/1503259307 conn(0x7f585006a800 msgr2=0x7f585006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:13:24.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.006+0000 7f58564bb640 1 -- 172.21.15.92:0/1503259307 shutdown_connections 2023-11-24T06:13:24.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.006+0000 7f58564bb640 1 -- 172.21.15.92:0/1503259307 wait complete. 2023-11-24T06:13:24.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.008+0000 7f58564bb640 1 Processor -- start 2023-11-24T06:13:24.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.008+0000 7f58564bb640 1 -- start start 2023-11-24T06:13:24.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.009+0000 7f58564bb640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f585013df80 0x7f585012f500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:24.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.009+0000 7f58564bb640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f585013e360 0x7f5850154ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:24.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.009+0000 7f58564bb640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f585015f9c0 0x7f58500aac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:24.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.009+0000 7f58564bb640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58500b1d40 con 0x7f585013e360 2023-11-24T06:13:24.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.009+0000 7f58564bb640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58500b2020 con 0x7f585013df80 2023-11-24T06:13:24.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.009+0000 7f58564bb640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58500b2300 con 0x7f585015f9c0 2023-11-24T06:13:24.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.009+0000 7f5847fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f585013e360 0x7f5850154ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:24.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.009+0000 7f5847fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f585013e360 0x7f5850154ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43804/0 (socket says 172.21.15.92:43804) 2023-11-24T06:13:24.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.009+0000 7f5847fff640 1 -- 172.21.15.92:0/3643761830 learned_addr learned my addr 172.21.15.92:0/3643761830 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:13:24.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.010+0000 7f584ffff640 1 --2- 172.21.15.92:0/3643761830 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f585013df80 0x7f585012f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:24.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.010+0000 7f5854a58640 1 --2- 172.21.15.92:0/3643761830 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f585015f9c0 0x7f58500aac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:24.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.010+0000 7f5847fff640 1 -- 172.21.15.92:0/3643761830 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f585015f9c0 msgr2=0x7f58500aac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:24.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.010+0000 7f5847fff640 1 --2- 172.21.15.92:0/3643761830 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f585015f9c0 0x7f58500aac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:24.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.010+0000 7f5847fff640 1 -- 172.21.15.92:0/3643761830 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f585013df80 msgr2=0x7f585012f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:24.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.010+0000 7f5847fff640 1 --2- 172.21.15.92:0/3643761830 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f585013df80 0x7f585012f500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:24.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.010+0000 7f5847fff640 1 -- 172.21.15.92:0/3643761830 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58400094d0 con 0x7f585013e360 2023-11-24T06:13:24.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.010+0000 7f5854a58640 1 --2- 172.21.15.92:0/3643761830 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f585015f9c0 0x7f58500aac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:13:24.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.010+0000 7f5847fff640 1 --2- 172.21.15.92:0/3643761830 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f585013e360 0x7f5850154ce0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f583800c7f0 tx=0x7f583800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:24.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.010+0000 7f584d7fa640 1 -- 172.21.15.92:0/3643761830 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5838004220 con 0x7f585013e360 2023-11-24T06:13:24.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.010+0000 7f584d7fa640 1 -- 172.21.15.92:0/3643761830 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58380043a0 con 0x7f585013e360 2023-11-24T06:13:24.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.010+0000 7f584d7fa640 1 -- 172.21.15.92:0/3643761830 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5838007860 con 0x7f585013e360 2023-11-24T06:13:24.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.010+0000 7f58564bb640 1 -- 172.21.15.92:0/3643761830 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58500b25e0 con 0x7f585013e360 2023-11-24T06:13:24.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.011+0000 7f58564bb640 1 -- 172.21.15.92:0/3643761830 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f585012fa40 con 0x7f585013e360 2023-11-24T06:13:24.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.012+0000 7f58564bb640 1 -- 172.21.15.92:0/3643761830 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58500675d0 con 0x7f585013e360 2023-11-24T06:13:24.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.012+0000 7f584d7fa640 1 -- 172.21.15.92:0/3643761830 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f5838007bd0 con 0x7f585013e360 2023-11-24T06:13:24.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.013+0000 7f584d7fa640 1 --2- 172.21.15.92:0/3643761830 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f583006aed0 0x7f583006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:24.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.013+0000 7f584d7fa640 1 -- 172.21.15.92:0/3643761830 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5838089880 con 0x7f585013e360 2023-11-24T06:13:24.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.014+0000 7f584ffff640 1 --2- 172.21.15.92:0/3643761830 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f583006aed0 0x7f583006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:24.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.014+0000 7f584ffff640 1 --2- 172.21.15.92:0/3643761830 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f583006aed0 0x7f583006d390 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f5840036000 tx=0x7f584003c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:24.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.018+0000 7f584d7fa640 1 -- 172.21.15.92:0/3643761830 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5838054730 con 0x7f585013e360 2023-11-24T06:13:24.215 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.212+0000 7f58564bb640 1 -- 172.21.15.92:0/3643761830 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f585015b960 con 0x7f585013e360 2023-11-24T06:13:24.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.215+0000 7f584d7fa640 1 -- 172.21.15.92:0/3643761830 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v63) v1 ==== 74+0+16520 (secure 0 0 0) 0x7f5838057d50 con 0x7f585013e360 2023-11-24T06:13:24.219 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-24T06:13:24.219 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":63,"fsid":"2d6dee5f-8aa3-470f-931a-50ade963b1cb","created":"2023-11-24T06:10:04.707204+0000","modified":"2023-11-24T06:13:10.158258+0000","last_up_change":"2023-11-24T06:10:17.598352+0000","last_in_change":"2023-11-24T06:10:09.040793+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:10:17.733098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:13:06.409727+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-11-24T06:13:07.623678+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ff37a9fb-135f-471f-b4b0-f7b5772006c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6803","nonce":20560714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6805","nonce":20560714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6809","nonce":20560714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6807","nonce":20560714}]},"public_addr":"172.21.15.92:6803/20560714","cluster_addr":"172.21.15.92:6805/20560714","heartbeat_back_addr":"172.21.15.92:6809/20560714","heartbeat_front_addr":"172.21.15.92:6807/20560714","state":["exists","up"]},{"osd":1,"uuid":"03cdd353-dcf1-4f38-94fe-a795137c06b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6819","nonce":2451239906}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6821","nonce":2451239906}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6825","nonce":2451239906}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6823","nonce":2451239906}]},"public_addr":"172.21.15.92:6819/2451239906","cluster_addr":"172.21.15.92:6821/2451239906","heartbeat_back_addr":"172.21.15.92:6825/2451239906","heartbeat_front_addr":"172.21.15.92:6823/2451239906","state":["exists","up"]},{"osd":2,"uuid":"a4a20d80-5d57-484b-becd-9aa8070dd3aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6811","nonce":1124979277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6813","nonce":1124979277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6817","nonce":1124979277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6815","nonce":1124979277}]},"public_addr":"172.21.15.92:6811/1124979277","cluster_addr":"172.21.15.92:6813/1124979277","heartbeat_back_addr":"172.21.15.92:6817/1124979277","heartbeat_front_addr":"172.21.15.92:6815/1124979277","state":["exists","up"]},{"osd":3,"uuid":"4648d7ce-9b29-4583-9425-4ba8fd2b1903","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6827","nonce":1734115782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6829","nonce":1734115782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6833","nonce":1734115782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6831","nonce":1734115782}]},"public_addr":"172.21.15.92:6827/1734115782","cluster_addr":"172.21.15.92:6829/1734115782","heartbeat_back_addr":"172.21.15.92:6833/1734115782","heartbeat_front_addr":"172.21.15.92:6831/1734115782","state":["exists","up"]},{"osd":4,"uuid":"3bc9d7eb-c6fa-4fcf-a3ec-ade604e05aac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6819","nonce":3425937229}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6822","nonce":3425937229}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6830","nonce":3425937229}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6826","nonce":3425937229}]},"public_addr":"172.21.15.118:6819/3425937229","cluster_addr":"172.21.15.118:6822/3425937229","heartbeat_back_addr":"172.21.15.118:6830/3425937229","heartbeat_front_addr":"172.21.15.118:6826/3425937229","state":["exists","up"]},{"osd":5,"uuid":"dce9ba6d-59ed-46d1-84fc-a84b13de10c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6801","nonce":827455984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6803","nonce":827455984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6807","nonce":827455984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6805","nonce":827455984}]},"public_addr":"172.21.15.118:6801/827455984","cluster_addr":"172.21.15.118:6803/827455984","heartbeat_back_addr":"172.21.15.118:6807/827455984","heartbeat_front_addr":"172.21.15.118:6805/827455984","state":["exists","up"]},{"osd":6,"uuid":"82aec504-ae58-4709-8d30-46f2b12081b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6809","nonce":2946986141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6811","nonce":2946986141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6815","nonce":2946986141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6813","nonce":2946986141}]},"public_addr":"172.21.15.118:6809/2946986141","cluster_addr":"172.21.15.118:6811/2946986141","heartbeat_back_addr":"172.21.15.118:6815/2946986141","heartbeat_front_addr":"172.21.15.118:6813/2946986141","state":["exists","up"]},{"osd":7,"uuid":"4ed4c624-c88f-4d35-bdb1-bf209b5dffb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6817","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6818","nonce":48122059}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6821","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6823","nonce":48122059}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6829","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6831","nonce":48122059}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6825","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6827","nonce":48122059}]},"public_addr":"172.21.15.118:6818/48122059","cluster_addr":"172.21.15.118:6823/48122059","heartbeat_back_addr":"172.21.15.118:6831/48122059","heartbeat_front_addr":"172.21.15.118:6827/48122059","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.536664+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.368324+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.271201+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.342971+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.621832+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.387110+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.632946+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.416032+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:13:24.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.218+0000 7f58564bb640 1 -- 172.21.15.92:0/3643761830 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f583006aed0 msgr2=0x7f583006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:24.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.218+0000 7f58564bb640 1 --2- 172.21.15.92:0/3643761830 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f583006aed0 0x7f583006d390 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f5840036000 tx=0x7f584003c040 comp rx=0 tx=0).stop 2023-11-24T06:13:24.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.218+0000 7f58564bb640 1 -- 172.21.15.92:0/3643761830 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f585013e360 msgr2=0x7f5850154ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:24.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.218+0000 7f58564bb640 1 --2- 172.21.15.92:0/3643761830 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f585013e360 0x7f5850154ce0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f583800c7f0 tx=0x7f583800ccb0 comp rx=0 tx=0).stop 2023-11-24T06:13:24.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.218+0000 7f58564bb640 1 -- 172.21.15.92:0/3643761830 shutdown_connections 2023-11-24T06:13:24.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.218+0000 7f58564bb640 1 --2- 172.21.15.92:0/3643761830 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f583006aed0 0x7f583006d390 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:24.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.218+0000 7f58564bb640 1 --2- 172.21.15.92:0/3643761830 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f585015f9c0 0x7f58500aac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:24.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.218+0000 7f58564bb640 1 --2- 172.21.15.92:0/3643761830 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f585013e360 0x7f5850154ce0 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:24.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.218+0000 7f58564bb640 1 --2- 172.21.15.92:0/3643761830 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f585013df80 0x7f585012f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:24.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.218+0000 7f58564bb640 1 -- 172.21.15.92:0/3643761830 >> 172.21.15.92:0/3643761830 conn(0x7f585006a800 msgr2=0x7f585015a140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:13:24.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.218+0000 7f58564bb640 1 -- 172.21.15.92:0/3643761830 shutdown_connections 2023-11-24T06:13:24.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.218+0000 7f58564bb640 1 -- 172.21.15.92:0/3643761830 wait complete. 2023-11-24T06:13:24.241 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T06:10:17.733098+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T06:13:06.409727+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '62', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T06:13:07.623678+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '63', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T06:13:24.241 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T06:13:24.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.457+0000 7fbbeab6c640 1 Processor -- start 2023-11-24T06:13:24.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.457+0000 7fbbeab6c640 1 -- start start 2023-11-24T06:13:24.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.459+0000 7fbbeab6c640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fbbe40ccb50 0x7fbbe4070a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:24.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.459+0000 7fbbeab6c640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbe415f4e0 con 0x7fbbe40adf20 2023-11-24T06:13:24.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.459+0000 7fbbeab6c640 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbe4071130 con 0x7fbbe415bf10 2023-11-24T06:13:24.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.459+0000 7fbbeab6c640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbbe415f970 con 0x7fbbe40ccb50 2023-11-24T06:13:24.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.460+0000 7fbbe9b6a640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fbbe40adf20 0x7fbbe40cc440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:53554/0 (socket says 172.21.15.92:53554) 2023-11-24T06:13:24.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.460+0000 7fbbe9369640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fbbe40ccb50 0x7fbbe4070a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:24.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.460+0000 7fbbe9b6a640 1 -- 172.21.15.92:0/1224741861 learned_addr learned my addr 172.21.15.92:0/1224741861 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-24T06:13:24.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.460+0000 7fbbe9369640 1 -- 172.21.15.92:0/1224741861 >> v1:172.21.15.118:6789/0 conn(0x7fbbe415bf10 legacy=0x7fbbe4143d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:13:24.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.460+0000 7fbbe9369640 1 -- 172.21.15.92:0/1224741861 >> v1:172.21.15.92:6789/0 conn(0x7fbbe40adf20 legacy=0x7fbbe40cc440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:24.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.460+0000 7fbbe9369640 1 -- 172.21.15.92:0/1224741861 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbe415ebb0 con 0x7fbbe40ccb50 2023-11-24T06:13:24.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.460+0000 7fbbe8b68640 1 -- 172.21.15.92:0/1224741861 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2439770276 0 0) 0x7fbbe415f4e0 con 0x7fbbe40adf20 2023-11-24T06:13:24.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.461+0000 7fbbe9369640 1 --2- 172.21.15.92:0/1224741861 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbbe40ccb50 0x7fbbe4070a30 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7fbbd00097a0 tx=0x7fbbd002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5ca8470ce533564 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:24.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.461+0000 7fbbe8b68640 1 -- 172.21.15.92:0/1224741861 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbd002e070 con 0x7fbbe40ccb50 2023-11-24T06:13:24.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.461+0000 7fbbe8b68640 1 -- 172.21.15.92:0/1224741861 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbd00315a0 con 0x7fbbe40ccb50 2023-11-24T06:13:24.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.461+0000 7fbbe8b68640 1 -- 172.21.15.92:0/1224741861 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbd0031930 con 0x7fbbe40ccb50 2023-11-24T06:13:24.466 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.462+0000 7fbbeab6c640 1 -- 172.21.15.92:0/1224741861 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbbe40ccb50 msgr2=0x7fbbe4070a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:24.466 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.462+0000 7fbbeab6c640 1 --2- 172.21.15.92:0/1224741861 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbbe40ccb50 0x7fbbe4070a30 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7fbbd00097a0 tx=0x7fbbd002b840 comp rx=0 tx=0).stop 2023-11-24T06:13:24.466 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.462+0000 7fbbeab6c640 1 -- 172.21.15.92:0/1224741861 shutdown_connections 2023-11-24T06:13:24.466 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.462+0000 7fbbeab6c640 1 --2- 172.21.15.92:0/1224741861 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbbe40ccb50 0x7fbbe4070a30 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:24.466 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.462+0000 7fbbeab6c640 1 -- 172.21.15.92:0/1224741861 >> 172.21.15.92:0/1224741861 conn(0x7fbbe406a7d0 msgr2=0x7fbbe406abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:13:24.466 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.462+0000 7fbbeab6c640 1 -- 172.21.15.92:0/1224741861 shutdown_connections 2023-11-24T06:13:24.466 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.462+0000 7fbbeab6c640 1 -- 172.21.15.92:0/1224741861 wait complete. 2023-11-24T06:13:24.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.464+0000 7fbbeab6c640 1 Processor -- start 2023-11-24T06:13:24.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.464+0000 7fbbeab6c640 1 -- start start 2023-11-24T06:13:24.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.465+0000 7fbbeab6c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbbe40adf20 0x7fbbe40aed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:24.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.465+0000 7fbbeab6c640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbbe40ccb50 0x7fbbe4139140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:24.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.465+0000 7fbbeab6c640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fbbe415bf10 0x7fbbe4139680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:24.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.465+0000 7fbbeab6c640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbe4135600 con 0x7fbbe40adf20 2023-11-24T06:13:24.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.465+0000 7fbbeab6c640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbe4134560 con 0x7fbbe40ccb50 2023-11-24T06:13:24.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.465+0000 7fbbeab6c640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbe4134840 con 0x7fbbe415bf10 2023-11-24T06:13:24.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.466+0000 7fbbe9b6a640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbbe40adf20 0x7fbbe40aed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:24.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.466+0000 7fbbe9b6a640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbbe40adf20 0x7fbbe40aed70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43822/0 (socket says 172.21.15.92:43822) 2023-11-24T06:13:24.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.466+0000 7fbbe9b6a640 1 -- 172.21.15.92:0/2046589860 learned_addr learned my addr 172.21.15.92:0/2046589860 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:13:24.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.466+0000 7fbbe9369640 1 --2- 172.21.15.92:0/2046589860 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbbe40ccb50 0x7fbbe4139140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:24.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.466+0000 7fbbea36b640 1 --2- 172.21.15.92:0/2046589860 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fbbe415bf10 0x7fbbe4139680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:24.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.466+0000 7fbbe9b6a640 1 -- 172.21.15.92:0/2046589860 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fbbe415bf10 msgr2=0x7fbbe4139680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:24.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.466+0000 7fbbe9b6a640 1 --2- 172.21.15.92:0/2046589860 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fbbe415bf10 0x7fbbe4139680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:24.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.466+0000 7fbbe9b6a640 1 -- 172.21.15.92:0/2046589860 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbbe40ccb50 msgr2=0x7fbbe4139140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:24.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.466+0000 7fbbe9b6a640 1 --2- 172.21.15.92:0/2046589860 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbbe40ccb50 0x7fbbe4139140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:24.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.466+0000 7fbbe9b6a640 1 -- 172.21.15.92:0/2046589860 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbd0009400 con 0x7fbbe40adf20 2023-11-24T06:13:24.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.466+0000 7fbbe9369640 1 --2- 172.21.15.92:0/2046589860 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbbe40ccb50 0x7fbbe4139140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:13:24.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.466+0000 7fbbe9b6a640 1 --2- 172.21.15.92:0/2046589860 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbbe40adf20 0x7fbbe40aed70 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7fbbd800c7c0 tx=0x7fbbd800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:24.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.467+0000 7fbbe27fc640 1 -- 172.21.15.92:0/2046589860 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbd800a640 con 0x7fbbe40adf20 2023-11-24T06:13:24.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.467+0000 7fbbe27fc640 1 -- 172.21.15.92:0/2046589860 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbd800a7c0 con 0x7fbbe40adf20 2023-11-24T06:13:24.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.467+0000 7fbbeab6c640 1 -- 172.21.15.92:0/2046589860 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbe4134b20 con 0x7fbbe40adf20 2023-11-24T06:13:24.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.467+0000 7fbbe27fc640 1 -- 172.21.15.92:0/2046589860 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbd800ab70 con 0x7fbbe40adf20 2023-11-24T06:13:24.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.467+0000 7fbbeab6c640 1 -- 172.21.15.92:0/2046589860 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbe41350e0 con 0x7fbbe40adf20 2023-11-24T06:13:24.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.468+0000 7fbbeab6c640 1 -- 172.21.15.92:0/2046589860 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbbe4067450 con 0x7fbbe40adf20 2023-11-24T06:13:24.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.469+0000 7fbbe27fc640 1 -- 172.21.15.92:0/2046589860 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7fbbd800acd0 con 0x7fbbe40adf20 2023-11-24T06:13:24.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.470+0000 7fbbe27fc640 1 --2- 172.21.15.92:0/2046589860 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fbbc806aed0 0x7fbbc806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:24.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.470+0000 7fbbe27fc640 1 -- 172.21.15.92:0/2046589860 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbbd8088fb0 con 0x7fbbe40adf20 2023-11-24T06:13:24.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.470+0000 7fbbe9369640 1 --2- 172.21.15.92:0/2046589860 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fbbc806aed0 0x7fbbc806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:24.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.471+0000 7fbbe9369640 1 --2- 172.21.15.92:0/2046589860 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fbbc806aed0 0x7fbbc806d390 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fbbd0036000 tx=0x7fbbd003d040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:24.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.474+0000 7fbbe27fc640 1 -- 172.21.15.92:0/2046589860 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fbbd8053f70 con 0x7fbbe40adf20 2023-11-24T06:13:24.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.583+0000 7fbbe27fc640 1 -- 172.21.15.92:0/2046589860 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mgrmap(e 7) v1 ==== 89333+0+0 (secure 0 0 0) 0x7fbbd8054340 con 0x7fbbe40adf20 2023-11-24T06:13:24.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.672+0000 7fbbeab6c640 1 -- 172.21.15.92:0/2046589860 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fbbe40cbea0 con 0x7fbbe40adf20 2023-11-24T06:13:24.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.673+0000 7fbbe27fc640 1 -- 172.21.15.92:0/2046589860 <== mon.0 v2:172.21.15.92:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v63) v1 ==== 93+0+10 (secure 0 0 0) 0x7fbbd8073600 con 0x7fbbe40adf20 2023-11-24T06:13:24.676 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 1 2023-11-24T06:13:24.679 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.676+0000 7fbbeab6c640 1 -- 172.21.15.92:0/2046589860 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fbbc806aed0 msgr2=0x7fbbc806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:24.679 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.676+0000 7fbbeab6c640 1 --2- 172.21.15.92:0/2046589860 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fbbc806aed0 0x7fbbc806d390 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fbbd0036000 tx=0x7fbbd003d040 comp rx=0 tx=0).stop 2023-11-24T06:13:24.679 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.676+0000 7fbbeab6c640 1 -- 172.21.15.92:0/2046589860 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbbe40adf20 msgr2=0x7fbbe40aed70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:24.679 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.676+0000 7fbbeab6c640 1 --2- 172.21.15.92:0/2046589860 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbbe40adf20 0x7fbbe40aed70 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7fbbd800c7c0 tx=0x7fbbd800cc80 comp rx=0 tx=0).stop 2023-11-24T06:13:24.680 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.676+0000 7fbbeab6c640 1 -- 172.21.15.92:0/2046589860 shutdown_connections 2023-11-24T06:13:24.680 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.676+0000 7fbbeab6c640 1 --2- 172.21.15.92:0/2046589860 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fbbc806aed0 0x7fbbc806d390 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:24.680 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.676+0000 7fbbeab6c640 1 --2- 172.21.15.92:0/2046589860 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fbbe415bf10 0x7fbbe4139680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:24.680 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.676+0000 7fbbeab6c640 1 --2- 172.21.15.92:0/2046589860 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbbe40ccb50 0x7fbbe4139140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:24.680 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.676+0000 7fbbeab6c640 1 --2- 172.21.15.92:0/2046589860 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbbe40adf20 0x7fbbe40aed70 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:24.680 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.676+0000 7fbbeab6c640 1 -- 172.21.15.92:0/2046589860 >> 172.21.15.92:0/2046589860 conn(0x7fbbe406a7d0 msgr2=0x7fbbe40aff20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:13:24.680 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.676+0000 7fbbeab6c640 1 -- 172.21.15.92:0/2046589860 shutdown_connections 2023-11-24T06:13:24.680 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.676+0000 7fbbeab6c640 1 -- 172.21.15.92:0/2046589860 wait complete. 2023-11-24T06:13:24.698 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T06:13:24.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.900+0000 7f2037b41640 1 Processor -- start 2023-11-24T06:13:24.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.900+0000 7f2037b41640 1 -- start start 2023-11-24T06:13:24.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.902+0000 7f2037b41640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f2030143ae0 0x7f20300cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:24.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.902+0000 7f2037b41640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f20300cf7d0 0x7f203015dca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:24.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.902+0000 7f2037b41640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20300cfc30 con 0x7f203015e1e0 2023-11-24T06:13:24.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.902+0000 7f2037b41640 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20300b1ee0 con 0x7f2030143ae0 2023-11-24T06:13:24.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.902+0000 7f2037b41640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20300b2780 con 0x7f20300cf7d0 2023-11-24T06:13:24.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.902+0000 7f20360de640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f203015e1e0 0x7f203015a670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:53562/0 (socket says 172.21.15.92:53562) 2023-11-24T06:13:24.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.902+0000 7f20360de640 1 -- 172.21.15.92:0/2087207786 learned_addr learned my addr 172.21.15.92:0/2087207786 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-24T06:13:24.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.902+0000 7f20348db640 1 -- 172.21.15.92:0/2087207786 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3627241792 0 0) 0x7f20300cfc30 con 0x7f203015e1e0 2023-11-24T06:13:24.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.902+0000 7f20348db640 1 -- 172.21.15.92:0/2087207786 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f201c005730 con 0x7f203015e1e0 2023-11-24T06:13:24.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.902+0000 7f20358dd640 1 --2- 172.21.15.92:0/2087207786 >> v2:172.21.15.118:3300/0 conn(0x7f2030143ae0 0x7f20300cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:24.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.902+0000 7f20350dc640 1 --2- 172.21.15.92:0/2087207786 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f20300cf7d0 0x7f203015dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:24.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.903+0000 7f20348db640 1 -- 172.21.15.92:0/2087207786 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2030178519 0 0) 0x7f201c005730 con 0x7f203015e1e0 2023-11-24T06:13:24.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.903+0000 7f20348db640 1 -- 172.21.15.92:0/2087207786 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f20300cf7d0 msgr2=0x7f203015dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:13:24.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.903+0000 7f20348db640 1 --2- 172.21.15.92:0/2087207786 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f20300cf7d0 0x7f203015dca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:24.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.903+0000 7f20348db640 1 -- 172.21.15.92:0/2087207786 >> v2:172.21.15.118:3300/0 conn(0x7f2030143ae0 msgr2=0x7f20300cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:13:24.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.903+0000 7f20348db640 1 --2- 172.21.15.92:0/2087207786 >> v2:172.21.15.118:3300/0 conn(0x7f2030143ae0 0x7f20300cf290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:24.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.903+0000 7f20348db640 1 -- 172.21.15.92:0/2087207786 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f203015e990 con 0x7f203015e1e0 2023-11-24T06:13:24.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.903+0000 7f20348db640 1 -- 172.21.15.92:0/2087207786 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2607661350 0 0) 0x7f20280019f0 con 0x7f203015e1e0 2023-11-24T06:13:24.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.903+0000 7f20348db640 1 -- 172.21.15.92:0/2087207786 >> v1:172.21.15.92:6789/0 conn(0x7f203015e1e0 legacy=0x7f203015a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:24.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.903+0000 7f20348db640 1 --2- 172.21.15.92:0/2087207786 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f201c005b80 0x7f201c005fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:24.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.903+0000 7f20348db640 1 --2- 172.21.15.92:0/2087207786 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f201c0069d0 0x7f201c006e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:24.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.903+0000 7f20348db640 1 --2- 172.21.15.92:0/2087207786 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f20300cf7d0 0x7f201c007390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:24.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.903+0000 7f20348db640 1 -- 172.21.15.92:0/2087207786 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20300b1ee0 con 0x7f201c005b80 2023-11-24T06:13:24.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.903+0000 7f20348db640 1 -- 172.21.15.92:0/2087207786 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20300b2780 con 0x7f201c0069d0 2023-11-24T06:13:24.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.903+0000 7f20348db640 1 -- 172.21.15.92:0/2087207786 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f201c00dc90 con 0x7f20300cf7d0 2023-11-24T06:13:24.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.903+0000 7f20358dd640 1 --2- 172.21.15.92:0/2087207786 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f201c005b80 0x7f201c005fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:24.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.903+0000 7f20358dd640 1 -- 172.21.15.92:0/2087207786 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f20300cf7d0 msgr2=0x7f201c007390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:13:24.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.903+0000 7f20358dd640 1 --2- 172.21.15.92:0/2087207786 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f20300cf7d0 0x7f201c007390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:24.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.903+0000 7f20358dd640 1 -- 172.21.15.92:0/2087207786 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f201c0069d0 msgr2=0x7f201c006e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:13:24.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.903+0000 7f20358dd640 1 --2- 172.21.15.92:0/2087207786 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f201c0069d0 0x7f201c006e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:24.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.903+0000 7f20358dd640 1 -- 172.21.15.92:0/2087207786 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f201c005730 con 0x7f201c005b80 2023-11-24T06:13:24.910 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.904+0000 7f20358dd640 1 --2- 172.21.15.92:0/2087207786 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f201c005b80 0x7f201c005fa0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f202c00ba00 tx=0x7f201c007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2440e3262efec378 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:24.910 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.905+0000 7f20348db640 1 -- 172.21.15.92:0/2087207786 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f202c032630 con 0x7f201c005b80 2023-11-24T06:13:24.910 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.905+0000 7f20348db640 1 -- 172.21.15.92:0/2087207786 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f202c032790 con 0x7f201c005b80 2023-11-24T06:13:24.910 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.905+0000 7f20348db640 1 -- 172.21.15.92:0/2087207786 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f202c032b20 con 0x7f201c005b80 2023-11-24T06:13:24.910 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.905+0000 7f2037b41640 1 -- 172.21.15.92:0/2087207786 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f201c005b80 msgr2=0x7f201c005fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:24.911 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.905+0000 7f2037b41640 1 --2- 172.21.15.92:0/2087207786 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f201c005b80 0x7f201c005fa0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f202c00ba00 tx=0x7f201c007a00 comp rx=0 tx=0).stop 2023-11-24T06:13:24.911 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.905+0000 7f20360de640 1 -- 172.21.15.92:0/2087207786 reap_dead start 2023-11-24T06:13:24.911 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.905+0000 7f2037b41640 1 -- 172.21.15.92:0/2087207786 shutdown_connections 2023-11-24T06:13:24.911 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.905+0000 7f2037b41640 1 -- 172.21.15.92:0/2087207786 >> 172.21.15.92:0/2087207786 conn(0x7f203006a800 msgr2=0x7f203006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:13:24.911 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.905+0000 7f2037b41640 1 -- 172.21.15.92:0/2087207786 shutdown_connections 2023-11-24T06:13:24.911 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.905+0000 7f2037b41640 1 -- 172.21.15.92:0/2087207786 wait complete. 2023-11-24T06:13:24.911 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.907+0000 7f2037b41640 1 Processor -- start 2023-11-24T06:13:24.911 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.907+0000 7f2037b41640 1 -- start start 2023-11-24T06:13:24.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.908+0000 7f2037b41640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f201c005b80 0x7f203014c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:24.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.908+0000 7f2037b41640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f203015ab20 0x7f2030140260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:24.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.908+0000 7f2037b41640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f20301407a0 0x7f203014df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:24.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.908+0000 7f2037b41640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f203013bff0 con 0x7f201c005b80 2023-11-24T06:13:24.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.908+0000 7f2037b41640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20300cb780 con 0x7f203015ab20 2023-11-24T06:13:24.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.908+0000 7f2037b41640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f203015af60 con 0x7f20301407a0 2023-11-24T06:13:24.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.908+0000 7f20358dd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f201c005b80 0x7f203014c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:24.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.908+0000 7f20358dd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f201c005b80 0x7f203014c4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43826/0 (socket says 172.21.15.92:43826) 2023-11-24T06:13:24.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.908+0000 7f20358dd640 1 -- 172.21.15.92:0/2303087622 learned_addr learned my addr 172.21.15.92:0/2303087622 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:13:24.914 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.909+0000 7f20358dd640 1 -- 172.21.15.92:0/2303087622 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f20301407a0 msgr2=0x7f203014df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:24.914 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.909+0000 7f20358dd640 1 --2- 172.21.15.92:0/2303087622 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f20301407a0 0x7f203014df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:24.914 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.909+0000 7f20358dd640 1 -- 172.21.15.92:0/2303087622 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f203015ab20 msgr2=0x7f2030140260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:24.914 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.909+0000 7f20358dd640 1 --2- 172.21.15.92:0/2303087622 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f203015ab20 0x7f2030140260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:24.914 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.909+0000 7f20358dd640 1 -- 172.21.15.92:0/2303087622 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f203013f160 con 0x7f201c005b80 2023-11-24T06:13:24.914 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.909+0000 7f20358dd640 1 --2- 172.21.15.92:0/2303087622 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f201c005b80 0x7f203014c4b0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f202c02beb0 tx=0x7f202c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:24.914 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.909+0000 7f20267fc640 1 -- 172.21.15.92:0/2303087622 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f202c004220 con 0x7f201c005b80 2023-11-24T06:13:24.914 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.909+0000 7f20267fc640 1 -- 172.21.15.92:0/2303087622 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f202c004380 con 0x7f201c005b80 2023-11-24T06:13:24.915 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.909+0000 7f2037b41640 1 -- 172.21.15.92:0/2303087622 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f203013c1d0 con 0x7f201c005b80 2023-11-24T06:13:24.915 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.909+0000 7f20267fc640 1 -- 172.21.15.92:0/2303087622 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f202c03ba70 con 0x7f201c005b80 2023-11-24T06:13:24.915 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.909+0000 7f2037b41640 1 -- 172.21.15.92:0/2303087622 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f203013c790 con 0x7f201c005b80 2023-11-24T06:13:24.915 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.911+0000 7f2037b41640 1 -- 172.21.15.92:0/2303087622 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20300675d0 con 0x7f201c005b80 2023-11-24T06:13:24.916 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.911+0000 7f20267fc640 1 -- 172.21.15.92:0/2303087622 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 7) v1 ==== 89333+0+0 (secure 0 0 0) 0x7f202c03bbd0 con 0x7f201c005b80 2023-11-24T06:13:24.916 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.912+0000 7f20267fc640 1 --2- 172.21.15.92:0/2303087622 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f201406af60 0x7f201406d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:24.916 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.912+0000 7f20267fc640 1 -- 172.21.15.92:0/2303087622 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f202c0ac1d0 con 0x7f201c005b80 2023-11-24T06:13:24.916 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.913+0000 7f20350dc640 1 --2- 172.21.15.92:0/2303087622 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f201406af60 0x7f201406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:24.916 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.913+0000 7f20350dc640 1 --2- 172.21.15.92:0/2303087622 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f201406af60 0x7f201406d420 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f2020006f00 tx=0x7f2020008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:24.920 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:24.917+0000 7f20267fc640 1 -- 172.21.15.92:0/2303087622 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f202c077050 con 0x7f201c005b80 2023-11-24T06:13:25.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:25.104+0000 7f2037b41640 1 -- 172.21.15.92:0/2303087622 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f20300675d0 con 0x7f201c005b80 2023-11-24T06:13:25.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:25.105+0000 7f20267fc640 1 -- 172.21.15.92:0/2303087622 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v63) v1 ==== 104+0+11 (secure 0 0 0) 0x7f202c07a670 con 0x7f201c005b80 2023-11-24T06:13:25.108 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 32 2023-11-24T06:13:25.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:25.107+0000 7f2037b41640 1 -- 172.21.15.92:0/2303087622 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f201406af60 msgr2=0x7f201406d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:25.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:25.107+0000 7f2037b41640 1 --2- 172.21.15.92:0/2303087622 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f201406af60 0x7f201406d420 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f2020006f00 tx=0x7f2020008040 comp rx=0 tx=0).stop 2023-11-24T06:13:25.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:25.108+0000 7f2037b41640 1 -- 172.21.15.92:0/2303087622 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f201c005b80 msgr2=0x7f203014c4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:25.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:25.108+0000 7f2037b41640 1 --2- 172.21.15.92:0/2303087622 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f201c005b80 0x7f203014c4b0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f202c02beb0 tx=0x7f202c004730 comp rx=0 tx=0).stop 2023-11-24T06:13:25.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:25.108+0000 7f2037b41640 1 -- 172.21.15.92:0/2303087622 shutdown_connections 2023-11-24T06:13:25.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:25.108+0000 7f2037b41640 1 --2- 172.21.15.92:0/2303087622 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f201406af60 0x7f201406d420 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:25.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:25.108+0000 7f2037b41640 1 --2- 172.21.15.92:0/2303087622 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f20301407a0 0x7f203014df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:25.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:25.108+0000 7f2037b41640 1 --2- 172.21.15.92:0/2303087622 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f203015ab20 0x7f2030140260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:25.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:25.108+0000 7f2037b41640 1 --2- 172.21.15.92:0/2303087622 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f201c005b80 0x7f203014c4b0 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:25.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:25.108+0000 7f2037b41640 1 -- 172.21.15.92:0/2303087622 >> 172.21.15.92:0/2303087622 conn(0x7f203006a800 msgr2=0x7f20300adbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:13:25.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:25.108+0000 7f2037b41640 1 -- 172.21.15.92:0/2303087622 shutdown_connections 2023-11-24T06:13:25.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:25.108+0000 7f2037b41640 1 -- 172.21.15.92:0/2303087622 wait complete. 2023-11-24T06:13:25.131 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T06:13:25.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:25.344+0000 7f02dfbf5640 1 Processor -- start 2023-11-24T06:13:25.348 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:25.344+0000 7f02dfbf5640 1 -- start start 2023-11-24T06:13:25.349 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:25.346+0000 7f02dfbf5640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f02d813e360 0x7f02d80cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:25.349 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:25.346+0000 7f02dfbf5640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02d815c5f0 con 0x7f02d813df80 2023-11-24T06:13:25.350 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:25.346+0000 7f02dfbf5640 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02d806c470 con 0x7f02d815f9c0 2023-11-24T06:13:25.350 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:25.346+0000 7f02dfbf5640 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02d806c750 con 0x7f02d813e360 2023-11-24T06:13:25.350 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:25.346+0000 7f02dd991640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f02d813df80 0x7f02d815bee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:53568/0 (socket says 172.21.15.92:53568) 2023-11-24T06:13:25.350 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:25.346+0000 7f02dd991640 1 -- 172.21.15.92:0/466406915 learned_addr learned my addr 172.21.15.92:0/466406915 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-24T06:13:25.352 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:25.347+0000 7f02dc98f640 1 -- 172.21.15.92:0/466406915 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2280184975 0 0) 0x7f02d815c5f0 con 0x7f02d813df80 2023-11-24T06:13:25.352 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:25.347+0000 7f02dd190640 1 --2- 172.21.15.92:0/466406915 >> v2:172.21.15.118:3300/0 conn(0x7f02d813e360 0x7f02d80cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:25.352 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:25.347+0000 7f02dc98f640 1 -- 172.21.15.92:0/466406915 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f02c4003520 con 0x7f02d813df80 2023-11-24T06:13:25.352 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:25.347+0000 7f02dc98f640 1 -- 172.21.15.92:0/466406915 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2457763863 0 0) 0x7f02c4003520 con 0x7f02d813df80 2023-11-24T06:13:25.352 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:25.347+0000 7f02dc98f640 1 -- 172.21.15.92:0/466406915 >> v2:172.21.15.118:3300/0 conn(0x7f02d813e360 msgr2=0x7f02d80cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:13:25.352 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:25.347+0000 7f02dc98f640 1 --2- 172.21.15.92:0/466406915 >> v2:172.21.15.118:3300/0 conn(0x7f02d813e360 0x7f02d80cf290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:25.352 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:25.347+0000 7f02dc98f640 1 -- 172.21.15.92:0/466406915 >> v1:172.21.15.118:6789/0 conn(0x7f02d815f9c0 legacy=0x7f02d80cd840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:13:25.352 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:25.347+0000 7f02dc98f640 1 -- 172.21.15.92:0/466406915 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02d806ca30 con 0x7f02d813df80 2023-11-24T06:13:25.352 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:25.347+0000 7f02dc98f640 1 -- 172.21.15.92:0/466406915 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2607661350 0 0) 0x7f02d4001690 con 0x7f02d813df80 2023-11-24T06:13:25.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:25.347+0000 7f02dc98f640 1 -- 172.21.15.92:0/466406915 >> v1:172.21.15.92:6789/0 conn(0x7f02d813df80 legacy=0x7f02d815bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:25.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:25.347+0000 7f02dc98f640 1 --2- 172.21.15.92:0/466406915 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f02c4003970 0x7f02c4003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:25.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:25.347+0000 7f02dc98f640 1 --2- 172.21.15.92:0/466406915 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f02c40047c0 0x7f02c4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:25.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:25.347+0000 7f02dc98f640 1 --2- 172.21.15.92:0/466406915 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f02c4005180 0x7f02c400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:25.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:25.347+0000 7f02dc98f640 1 -- 172.21.15.92:0/466406915 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02d806c750 con 0x7f02c40047c0 2023-11-24T06:13:25.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:25.347+0000 7f02dc98f640 1 -- 172.21.15.92:0/466406915 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02c400c020 con 0x7f02c4005180 2023-11-24T06:13:25.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:25.347+0000 7f02dc98f640 1 -- 172.21.15.92:0/466406915 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02c400c1e0 con 0x7f02c4003970 2023-11-24T06:13:25.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:25.348+0000 7f02dd190640 1 --2- 172.21.15.92:0/466406915 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f02c40047c0 0x7f02c4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:25.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:25.348+0000 7f02dd190640 1 -- 172.21.15.92:0/466406915 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f02c4003970 msgr2=0x7f02c4003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:13:25.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:25.348+0000 7f02dd190640 1 --2- 172.21.15.92:0/466406915 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f02c4003970 0x7f02c4003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:25.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:25.348+0000 7f02dd190640 1 -- 172.21.15.92:0/466406915 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f02c4005180 msgr2=0x7f02c400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:13:25.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:25.348+0000 7f02dd190640 1 --2- 172.21.15.92:0/466406915 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f02c4005180 0x7f02c400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:25.354 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:25.348+0000 7f02dd190640 1 -- 172.21.15.92:0/466406915 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02c4003520 con 0x7f02c40047c0 2023-11-24T06:13:25.354 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:25.348+0000 7f02de192640 1 -- 172.21.15.92:0/466406915 reap_dead start 2023-11-24T06:13:25.354 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:25.349+0000 7f02dd190640 1 --2- 172.21.15.92:0/466406915 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f02c40047c0 0x7f02c4004c40 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f02c8009a90 tx=0x7f02c40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8af4f9c986ce35e server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:25.355 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:25.349+0000 7f02dc98f640 1 -- 172.21.15.92:0/466406915 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02c802c6b0 con 0x7f02c40047c0 2023-11-24T06:13:25.355 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:25.349+0000 7f02dc98f640 1 -- 172.21.15.92:0/466406915 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02c802c810 con 0x7f02c40047c0 2023-11-24T06:13:25.355 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:25.349+0000 7f02dc98f640 1 -- 172.21.15.92:0/466406915 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02c802cbc0 con 0x7f02c40047c0 2023-11-24T06:13:25.355 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:25.350+0000 7f02dfbf5640 1 -- 172.21.15.92:0/466406915 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f02c40047c0 msgr2=0x7f02c4004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:25.355 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:25.350+0000 7f02dfbf5640 1 --2- 172.21.15.92:0/466406915 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f02c40047c0 0x7f02c4004c40 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f02c8009a90 tx=0x7f02c40057f0 comp rx=0 tx=0).stop 2023-11-24T06:13:25.355 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:25.350+0000 7f02dfbf5640 1 -- 172.21.15.92:0/466406915 shutdown_connections 2023-11-24T06:13:25.355 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:25.350+0000 7f02dfbf5640 1 --2- 172.21.15.92:0/466406915 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f02c40047c0 0x7f02c4004c40 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:25.355 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:25.350+0000 7f02dfbf5640 1 -- 172.21.15.92:0/466406915 >> 172.21.15.92:0/466406915 conn(0x7f02d806a800 msgr2=0x7f02d806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:13:25.355 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:25.350+0000 7f02dfbf5640 1 -- 172.21.15.92:0/466406915 shutdown_connections 2023-11-24T06:13:25.355 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:25.350+0000 7f02dfbf5640 1 -- 172.21.15.92:0/466406915 wait complete. 2023-11-24T06:13:25.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:25.352+0000 7f02dfbf5640 1 Processor -- start 2023-11-24T06:13:25.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:25.352+0000 7f02dfbf5640 1 -- start start 2023-11-24T06:13:25.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:25.353+0000 7f02dfbf5640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f02c40047c0 0x7f02d8137870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:25.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:25.353+0000 7f02dfbf5640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f02d8157b30 0x7f02d8155d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:25.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:25.353+0000 7f02dfbf5640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f02d8156270 0x7f02d80b3290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:25.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:25.353+0000 7f02dfbf5640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02d8137db0 con 0x7f02c40047c0 2023-11-24T06:13:25.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:25.353+0000 7f02dfbf5640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02d8145c00 con 0x7f02d8156270 2023-11-24T06:13:25.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:25.353+0000 7f02dfbf5640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02d8142d90 con 0x7f02d8157b30 2023-11-24T06:13:25.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:25.353+0000 7f02dd991640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f02c40047c0 0x7f02d8137870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:25.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:25.353+0000 7f02dd991640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f02c40047c0 0x7f02d8137870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43844/0 (socket says 172.21.15.92:43844) 2023-11-24T06:13:25.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:25.353+0000 7f02dd991640 1 -- 172.21.15.92:0/1560175345 learned_addr learned my addr 172.21.15.92:0/1560175345 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:13:25.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:25.354+0000 7f02dd190640 1 --2- 172.21.15.92:0/1560175345 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f02d8157b30 0x7f02d8155d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:25.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:25.354+0000 7f02de192640 1 --2- 172.21.15.92:0/1560175345 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f02d8156270 0x7f02d80b3290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:25.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:25.354+0000 7f02dd991640 1 -- 172.21.15.92:0/1560175345 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f02d8157b30 msgr2=0x7f02d8155d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:25.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:25.354+0000 7f02dd991640 1 --2- 172.21.15.92:0/1560175345 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f02d8157b30 0x7f02d8155d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:25.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:25.354+0000 7f02dd991640 1 -- 172.21.15.92:0/1560175345 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f02d8156270 msgr2=0x7f02d80b3290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:25.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:25.354+0000 7f02dd991640 1 --2- 172.21.15.92:0/1560175345 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f02d8156270 0x7f02d80b3290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:25.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:25.354+0000 7f02dd991640 1 -- 172.21.15.92:0/1560175345 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02d8143070 con 0x7f02c40047c0 2023-11-24T06:13:25.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:25.354+0000 7f02dd991640 1 --2- 172.21.15.92:0/1560175345 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f02c40047c0 0x7f02d8137870 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f02d400c770 tx=0x7f02d400cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:25.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:25.354+0000 7f02ce7fc640 1 -- 172.21.15.92:0/1560175345 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02d40074e0 con 0x7f02c40047c0 2023-11-24T06:13:25.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:25.354+0000 7f02ce7fc640 1 -- 172.21.15.92:0/1560175345 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02d4007640 con 0x7f02c40047c0 2023-11-24T06:13:25.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:25.354+0000 7f02dfbf5640 1 -- 172.21.15.92:0/1560175345 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02d8143350 con 0x7f02c40047c0 2023-11-24T06:13:25.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:25.354+0000 7f02ce7fc640 1 -- 172.21.15.92:0/1560175345 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02d4004a00 con 0x7f02c40047c0 2023-11-24T06:13:25.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:25.355+0000 7f02dfbf5640 1 -- 172.21.15.92:0/1560175345 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02d8143910 con 0x7f02c40047c0 2023-11-24T06:13:25.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:25.356+0000 7f02dfbf5640 1 -- 172.21.15.92:0/1560175345 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02d8156650 con 0x7f02c40047c0 2023-11-24T06:13:25.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:25.357+0000 7f02ce7fc640 1 -- 172.21.15.92:0/1560175345 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 7) v1 ==== 89333+0+0 (secure 0 0 0) 0x7f02d4004b60 con 0x7f02c40047c0 2023-11-24T06:13:25.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:25.357+0000 7f02ce7fc640 1 --2- 172.21.15.92:0/1560175345 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f02bc06af90 0x7f02bc06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:25.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:25.358+0000 7f02ce7fc640 1 -- 172.21.15.92:0/1560175345 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f02d4088ed0 con 0x7f02c40047c0 2023-11-24T06:13:25.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:25.358+0000 7f02dd190640 1 --2- 172.21.15.92:0/1560175345 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f02bc06af90 0x7f02bc06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:25.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:25.359+0000 7f02dd190640 1 --2- 172.21.15.92:0/1560175345 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f02bc06af90 0x7f02bc06d450 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f02c8036000 tx=0x7f02c8030000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:25.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:25.362+0000 7f02ce7fc640 1 -- 172.21.15.92:0/1560175345 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f02d4053f00 con 0x7f02c40047c0 2023-11-24T06:13:25.556 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:25.553+0000 7f02dfbf5640 1 -- 172.21.15.92:0/1560175345 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f02d80675d0 con 0x7f02c40047c0 2023-11-24T06:13:25.557 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:25.554+0000 7f02ce7fc640 1 -- 172.21.15.92:0/1560175345 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v63) v1 ==== 100+0+11 (secure 0 0 0) 0x7f02d4018020 con 0x7f02c40047c0 2023-11-24T06:13:25.557 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 64 2023-11-24T06:13:25.560 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:25.557+0000 7f02dfbf5640 1 -- 172.21.15.92:0/1560175345 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f02bc06af90 msgr2=0x7f02bc06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:25.560 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:25.557+0000 7f02dfbf5640 1 --2- 172.21.15.92:0/1560175345 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f02bc06af90 0x7f02bc06d450 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f02c8036000 tx=0x7f02c8030000 comp rx=0 tx=0).stop 2023-11-24T06:13:25.560 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:25.557+0000 7f02dfbf5640 1 -- 172.21.15.92:0/1560175345 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f02c40047c0 msgr2=0x7f02d8137870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:25.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:25.557+0000 7f02dfbf5640 1 --2- 172.21.15.92:0/1560175345 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f02c40047c0 0x7f02d8137870 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f02d400c770 tx=0x7f02d400cc30 comp rx=0 tx=0).stop 2023-11-24T06:13:25.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:25.557+0000 7f02dfbf5640 1 -- 172.21.15.92:0/1560175345 shutdown_connections 2023-11-24T06:13:25.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:25.557+0000 7f02dfbf5640 1 --2- 172.21.15.92:0/1560175345 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f02bc06af90 0x7f02bc06d450 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:25.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:25.557+0000 7f02dfbf5640 1 --2- 172.21.15.92:0/1560175345 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f02d8156270 0x7f02d80b3290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:25.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:25.557+0000 7f02dfbf5640 1 --2- 172.21.15.92:0/1560175345 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f02d8157b30 0x7f02d8155d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:25.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:25.557+0000 7f02dfbf5640 1 --2- 172.21.15.92:0/1560175345 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f02c40047c0 0x7f02d8137870 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:25.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:25.557+0000 7f02dfbf5640 1 -- 172.21.15.92:0/1560175345 >> 172.21.15.92:0/1560175345 conn(0x7f02d806a800 msgr2=0x7f02d815cf80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:13:25.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:25.557+0000 7f02dfbf5640 1 -- 172.21.15.92:0/1560175345 shutdown_connections 2023-11-24T06:13:25.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:25.557+0000 7f02dfbf5640 1 -- 172.21.15.92:0/1560175345 wait complete. 2023-11-24T06:13:25.578 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-24T06:13:25.794 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:25.790+0000 7fe57a259640 1 Processor -- start 2023-11-24T06:13:25.794 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:25.791+0000 7fe57a259640 1 -- start start 2023-11-24T06:13:25.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:25.792+0000 7fe57a259640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fe5741616d0 0x7fe5740aec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:25.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:25.792+0000 7fe57a259640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe5740af140 0x7fe57406d000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:25.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:25.792+0000 7fe57a259640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5740b2cc0 con 0x7fe57415d2c0 2023-11-24T06:13:25.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:25.792+0000 7fe57a259640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5740af5d0 con 0x7fe5741616d0 2023-11-24T06:13:25.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:25.792+0000 7fe57a259640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe574160c80 con 0x7fe5740af140 2023-11-24T06:13:25.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:25.793+0000 7fe578a56640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fe5741616d0 0x7fe5740aec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:25.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:25.793+0000 7fe579257640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fe57415d2c0 0x7fe5740b4890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:53582/0 (socket says 172.21.15.92:53582) 2023-11-24T06:13:25.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:25.793+0000 7fe579257640 1 -- 172.21.15.92:0/3404248568 learned_addr learned my addr 172.21.15.92:0/3404248568 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-24T06:13:25.797 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:25.793+0000 7fe579a58640 1 --2- 172.21.15.92:0/3404248568 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe5740af140 0x7fe57406d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:25.797 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:25.793+0000 7fe578a56640 1 -- 172.21.15.92:0/3404248568 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe5740af140 msgr2=0x7fe57406d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:13:25.797 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:25.793+0000 7fe578a56640 1 --2- 172.21.15.92:0/3404248568 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe5740af140 0x7fe57406d000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:25.797 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:25.793+0000 7fe578a56640 1 -- 172.21.15.92:0/3404248568 >> v1:172.21.15.92:6789/0 conn(0x7fe57415d2c0 legacy=0x7fe5740b4890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:25.797 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:25.793+0000 7fe573fff640 1 -- 172.21.15.92:0/3404248568 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1271009920 0 0) 0x7fe5740b2cc0 con 0x7fe57415d2c0 2023-11-24T06:13:25.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:25.793+0000 7fe578a56640 1 -- 172.21.15.92:0/3404248568 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe574161240 con 0x7fe5741616d0 2023-11-24T06:13:25.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:25.793+0000 7fe579a58640 1 --2- 172.21.15.92:0/3404248568 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe5740af140 0x7fe57406d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:13:25.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:25.795+0000 7fe578a56640 1 --2- 172.21.15.92:0/3404248568 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe5741616d0 0x7fe5740aec00 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7fe5600097a0 tx=0x7fe56002b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=961d866800527796 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:25.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:25.795+0000 7fe573fff640 1 -- 172.21.15.92:0/3404248568 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe56002d040 con 0x7fe5741616d0 2023-11-24T06:13:25.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:25.795+0000 7fe573fff640 1 -- 172.21.15.92:0/3404248568 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe560031810 con 0x7fe5741616d0 2023-11-24T06:13:25.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:25.795+0000 7fe573fff640 1 -- 172.21.15.92:0/3404248568 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe560031ba0 con 0x7fe5741616d0 2023-11-24T06:13:25.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:25.796+0000 7fe57a259640 1 -- 172.21.15.92:0/3404248568 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe5741616d0 msgr2=0x7fe5740aec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:25.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:25.796+0000 7fe57a259640 1 --2- 172.21.15.92:0/3404248568 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe5741616d0 0x7fe5740aec00 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7fe5600097a0 tx=0x7fe56002b8b0 comp rx=0 tx=0).stop 2023-11-24T06:13:25.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:25.796+0000 7fe57a259640 1 -- 172.21.15.92:0/3404248568 shutdown_connections 2023-11-24T06:13:25.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:25.796+0000 7fe57a259640 1 --2- 172.21.15.92:0/3404248568 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe5740af140 0x7fe57406d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:25.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:25.796+0000 7fe57a259640 1 --2- 172.21.15.92:0/3404248568 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe5741616d0 0x7fe5740aec00 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:25.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:25.796+0000 7fe57a259640 1 -- 172.21.15.92:0/3404248568 >> 172.21.15.92:0/3404248568 conn(0x7fe5740b19c0 msgr2=0x7fe5740b1dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:13:25.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:25.796+0000 7fe57a259640 1 -- 172.21.15.92:0/3404248568 shutdown_connections 2023-11-24T06:13:25.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:25.796+0000 7fe57a259640 1 -- 172.21.15.92:0/3404248568 wait complete. 2023-11-24T06:13:25.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:25.797+0000 7fe57a259640 1 Processor -- start 2023-11-24T06:13:25.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:25.798+0000 7fe57a259640 1 -- start start 2023-11-24T06:13:25.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:25.799+0000 7fe57a259640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe5740af140 0x7fe574160150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:25.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:25.799+0000 7fe57a259640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe57415d2c0 0x7fe574155230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:25.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:25.799+0000 7fe57a259640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe5741616d0 0x7fe57414f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:25.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:25.799+0000 7fe57a259640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe574155770 con 0x7fe5741616d0 2023-11-24T06:13:25.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:25.799+0000 7fe57a259640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe57415d7d0 con 0x7fe5740af140 2023-11-24T06:13:25.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:25.799+0000 7fe57a259640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe574155980 con 0x7fe57415d2c0 2023-11-24T06:13:25.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:25.799+0000 7fe579a58640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe5741616d0 0x7fe57414f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:25.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:25.799+0000 7fe579a58640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe5741616d0 0x7fe57414f090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43870/0 (socket says 172.21.15.92:43870) 2023-11-24T06:13:25.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:25.799+0000 7fe579a58640 1 -- 172.21.15.92:0/3220296171 learned_addr learned my addr 172.21.15.92:0/3220296171 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:13:25.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:25.799+0000 7fe579257640 1 --2- 172.21.15.92:0/3220296171 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe5740af140 0x7fe574160150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:25.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:25.799+0000 7fe578a56640 1 --2- 172.21.15.92:0/3220296171 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe57415d2c0 0x7fe574155230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:25.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:25.799+0000 7fe579a58640 1 -- 172.21.15.92:0/3220296171 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe57415d2c0 msgr2=0x7fe574155230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:25.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:25.799+0000 7fe579a58640 1 --2- 172.21.15.92:0/3220296171 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe57415d2c0 0x7fe574155230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:25.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:25.799+0000 7fe579a58640 1 -- 172.21.15.92:0/3220296171 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe5740af140 msgr2=0x7fe574160150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:25.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:25.799+0000 7fe579a58640 1 --2- 172.21.15.92:0/3220296171 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe5740af140 0x7fe574160150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:25.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:25.799+0000 7fe579a58640 1 -- 172.21.15.92:0/3220296171 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe560009400 con 0x7fe5741616d0 2023-11-24T06:13:25.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:25.799+0000 7fe579257640 1 --2- 172.21.15.92:0/3220296171 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe5740af140 0x7fe574160150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:13:25.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:25.800+0000 7fe579a58640 1 --2- 172.21.15.92:0/3220296171 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe5741616d0 0x7fe57414f090 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7fe56c00c7c0 tx=0x7fe56c00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:25.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:25.800+0000 7fe571ffb640 1 -- 172.21.15.92:0/3220296171 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe56c004230 con 0x7fe5741616d0 2023-11-24T06:13:25.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:25.800+0000 7fe571ffb640 1 -- 172.21.15.92:0/3220296171 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe56c004390 con 0x7fe5741616d0 2023-11-24T06:13:25.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:25.800+0000 7fe571ffb640 1 -- 172.21.15.92:0/3220296171 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe56c007860 con 0x7fe5741616d0 2023-11-24T06:13:25.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:25.800+0000 7fe57a259640 1 -- 172.21.15.92:0/3220296171 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe574155c60 con 0x7fe5741616d0 2023-11-24T06:13:25.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:25.800+0000 7fe57a259640 1 -- 172.21.15.92:0/3220296171 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe574156220 con 0x7fe5741616d0 2023-11-24T06:13:25.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:25.801+0000 7fe57a259640 1 -- 172.21.15.92:0/3220296171 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe57415d960 con 0x7fe5741616d0 2023-11-24T06:13:25.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:25.802+0000 7fe571ffb640 1 -- 172.21.15.92:0/3220296171 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 7) v1 ==== 89333+0+0 (secure 0 0 0) 0x7fe56c0079c0 con 0x7fe5741616d0 2023-11-24T06:13:25.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:25.803+0000 7fe571ffb640 1 --2- 172.21.15.92:0/3220296171 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fe55806af60 0x7fe55806d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:25.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:25.803+0000 7fe571ffb640 1 -- 172.21.15.92:0/3220296171 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe56c089b00 con 0x7fe5741616d0 2023-11-24T06:13:25.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:25.803+0000 7fe579257640 1 --2- 172.21.15.92:0/3220296171 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fe55806af60 0x7fe55806d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:25.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:25.804+0000 7fe579257640 1 --2- 172.21.15.92:0/3220296171 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fe55806af60 0x7fe55806d420 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fe568007fd0 tx=0x7fe568009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:25.811 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:25.808+0000 7fe571ffb640 1 -- 172.21.15.92:0/3220296171 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe56c08be60 con 0x7fe5741616d0 2023-11-24T06:13:26.020 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:26.016+0000 7fe57a259640 1 -- 172.21.15.92:0/3220296171 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fe574067b60 con 0x7fe5741616d0 2023-11-24T06:13:26.020 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:26.017+0000 7fe571ffb640 1 -- 172.21.15.92:0/3220296171 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v63) v1 ==== 80+0+0 (secure 0 0 0) 0x7fe56c015080 con 0x7fe5741616d0 2023-11-24T06:13:26.020 INFO:teuthology.orchestra.run.smithi092.stderr:listed 0 entries 2023-11-24T06:13:26.023 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:26.020+0000 7fe57a259640 1 -- 172.21.15.92:0/3220296171 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fe55806af60 msgr2=0x7fe55806d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:26.023 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:26.020+0000 7fe57a259640 1 --2- 172.21.15.92:0/3220296171 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fe55806af60 0x7fe55806d420 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fe568007fd0 tx=0x7fe568009040 comp rx=0 tx=0).stop 2023-11-24T06:13:26.024 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:26.020+0000 7fe57a259640 1 -- 172.21.15.92:0/3220296171 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe5741616d0 msgr2=0x7fe57414f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:26.024 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:26.020+0000 7fe57a259640 1 --2- 172.21.15.92:0/3220296171 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe5741616d0 0x7fe57414f090 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7fe56c00c7c0 tx=0x7fe56c00cc80 comp rx=0 tx=0).stop 2023-11-24T06:13:26.024 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:26.020+0000 7fe57a259640 1 -- 172.21.15.92:0/3220296171 shutdown_connections 2023-11-24T06:13:26.024 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:26.020+0000 7fe57a259640 1 --2- 172.21.15.92:0/3220296171 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fe55806af60 0x7fe55806d420 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:26.024 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:26.020+0000 7fe57a259640 1 --2- 172.21.15.92:0/3220296171 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe5741616d0 0x7fe57414f090 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:26.024 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:26.020+0000 7fe57a259640 1 --2- 172.21.15.92:0/3220296171 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe57415d2c0 0x7fe574155230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:26.024 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:26.020+0000 7fe57a259640 1 --2- 172.21.15.92:0/3220296171 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe5740af140 0x7fe574160150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:26.025 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:26.020+0000 7fe57a259640 1 -- 172.21.15.92:0/3220296171 >> 172.21.15.92:0/3220296171 conn(0x7fe5740b19c0 msgr2=0x7fe57415bfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:13:26.025 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:26.020+0000 7fe57a259640 1 -- 172.21.15.92:0/3220296171 shutdown_connections 2023-11-24T06:13:26.025 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:26.021+0000 7fe57a259640 1 -- 172.21.15.92:0/3220296171 wait complete. 2023-11-24T06:13:26.042 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi092.front.sepia.ceph.com... 2023-11-24T06:13:26.042 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:13:26.043 DEBUG:teuthology.orchestra.run.smithi092:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-11-24T06:13:26.132 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi092.stderr:ceph-fuse[109323]: fuse finished with error 0 and tester_r 0 2023-11-24T06:13:26.136 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi092.front.sepia.ceph.com 2023-11-24T06:13:26.136 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:13:26.137 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-24T06:13:26.183 DEBUG:teuthology.orchestra.run.smithi118:> set -ex 2023-11-24T06:13:26.183 DEBUG:teuthology.orchestra.run.smithi118:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:13:26.245 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:13:26.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:26.464+0000 7f704bebb640 1 Processor -- start 2023-11-24T06:13:26.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:26.464+0000 7f704bebb640 1 -- start start 2023-11-24T06:13:26.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:26.466+0000 7f704bebb640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f704414b2c0 0x7f70440cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:26.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:26.466+0000 7f704bebb640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7044071070 0x7f70440cc160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:26.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:26.466+0000 7f704bebb640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70440cfbf0 con 0x7f70440cf7d0 2023-11-24T06:13:26.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:26.466+0000 7f704bebb640 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f704414b6a0 con 0x7f704414b2c0 2023-11-24T06:13:26.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:26.466+0000 7f704bebb640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7044071690 con 0x7f7044071070 2023-11-24T06:13:26.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:26.466+0000 7f7049456640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f70440cf7d0 0x7f7044070960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:53588/0 (socket says 172.21.15.92:53588) 2023-11-24T06:13:26.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:26.466+0000 7f7049456640 1 -- 172.21.15.92:0/1382737117 learned_addr learned my addr 172.21.15.92:0/1382737117 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-24T06:13:26.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:26.467+0000 7f7049c57640 1 --2- 172.21.15.92:0/1382737117 >> v2:172.21.15.118:3300/0 conn(0x7f704414b2c0 0x7f70440cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:26.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:26.467+0000 7f704a458640 1 --2- 172.21.15.92:0/1382737117 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7044071070 0x7f70440cc160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:26.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:26.467+0000 7f7048c55640 1 -- 172.21.15.92:0/1382737117 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3162252395 0 0) 0x7f70440cfbf0 con 0x7f70440cf7d0 2023-11-24T06:13:26.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:26.467+0000 7f7048c55640 1 -- 172.21.15.92:0/1382737117 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7030003520 con 0x7f70440cf7d0 2023-11-24T06:13:26.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:26.467+0000 7f7048c55640 1 -- 172.21.15.92:0/1382737117 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1198661517 0 0) 0x7f7030003520 con 0x7f70440cf7d0 2023-11-24T06:13:26.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:26.467+0000 7f7048c55640 1 -- 172.21.15.92:0/1382737117 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7044071070 msgr2=0x7f70440cc160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:13:26.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:26.467+0000 7f7048c55640 1 --2- 172.21.15.92:0/1382737117 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7044071070 0x7f70440cc160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:26.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:26.467+0000 7f7048c55640 1 -- 172.21.15.92:0/1382737117 >> v2:172.21.15.118:3300/0 conn(0x7f704414b2c0 msgr2=0x7f70440cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:13:26.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:26.467+0000 7f7048c55640 1 --2- 172.21.15.92:0/1382737117 >> v2:172.21.15.118:3300/0 conn(0x7f704414b2c0 0x7f70440cf290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:26.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:26.467+0000 7f7048c55640 1 -- 172.21.15.92:0/1382737117 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7044071970 con 0x7f70440cf7d0 2023-11-24T06:13:26.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:26.467+0000 7f7048c55640 1 -- 172.21.15.92:0/1382737117 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2607661350 0 0) 0x7f7034001670 con 0x7f70440cf7d0 2023-11-24T06:13:26.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:26.467+0000 7f7048c55640 1 -- 172.21.15.92:0/1382737117 >> v1:172.21.15.92:6789/0 conn(0x7f70440cf7d0 legacy=0x7f7044070960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:26.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:26.468+0000 7f7048c55640 1 --2- 172.21.15.92:0/1382737117 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7030003970 0x7f7030003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:26.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:26.468+0000 7f7048c55640 1 --2- 172.21.15.92:0/1382737117 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7044071070 0x7f70300047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:26.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:26.468+0000 7f7048c55640 1 --2- 172.21.15.92:0/1382737117 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7030004d00 0x7f7030005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:26.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:26.468+0000 7f7048c55640 1 -- 172.21.15.92:0/1382737117 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f704414b6a0 con 0x7f7030004d00 2023-11-24T06:13:26.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:26.468+0000 7f7048c55640 1 -- 172.21.15.92:0/1382737117 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7044071690 con 0x7f7030003970 2023-11-24T06:13:26.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:26.468+0000 7f7048c55640 1 -- 172.21.15.92:0/1382737117 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f703000b980 con 0x7f7044071070 2023-11-24T06:13:26.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:26.468+0000 7f704a458640 1 --2- 172.21.15.92:0/1382737117 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7030004d00 0x7f7030005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:26.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:26.468+0000 7f7049456640 1 --2- 172.21.15.92:0/1382737117 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7044071070 0x7f70300047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:26.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:26.468+0000 7f7049c57640 1 --2- 172.21.15.92:0/1382737117 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7030003970 0x7f7030003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:26.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:26.468+0000 7f704a458640 1 -- 172.21.15.92:0/1382737117 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7044071070 msgr2=0x7f70300047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:13:26.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:26.468+0000 7f704a458640 1 --2- 172.21.15.92:0/1382737117 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f7044071070 0x7f70300047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:26.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:26.468+0000 7f704a458640 1 -- 172.21.15.92:0/1382737117 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7030003970 msgr2=0x7f7030003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:13:26.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:26.468+0000 7f704a458640 1 --2- 172.21.15.92:0/1382737117 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7030003970 0x7f7030003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:26.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:26.469+0000 7f704a458640 1 -- 172.21.15.92:0/1382737117 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7030003520 con 0x7f7030004d00 2023-11-24T06:13:26.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:26.470+0000 7f704a458640 1 --2- 172.21.15.92:0/1382737117 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7030004d00 0x7f7030005180 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f703c009fc0 tx=0x7f70300057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=948981bd4d2559f server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:26.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:26.470+0000 7f7048c55640 1 -- 172.21.15.92:0/1382737117 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f703c02cc20 con 0x7f7030004d00 2023-11-24T06:13:26.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:26.470+0000 7f7048c55640 1 -- 172.21.15.92:0/1382737117 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f703c00f040 con 0x7f7030004d00 2023-11-24T06:13:26.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:26.470+0000 7f7048c55640 1 -- 172.21.15.92:0/1382737117 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f703c034440 con 0x7f7030004d00 2023-11-24T06:13:26.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:26.471+0000 7f704bebb640 1 -- 172.21.15.92:0/1382737117 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7030004d00 msgr2=0x7f7030005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:26.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:26.471+0000 7f704bebb640 1 --2- 172.21.15.92:0/1382737117 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7030004d00 0x7f7030005180 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f703c009fc0 tx=0x7f70300057f0 comp rx=0 tx=0).stop 2023-11-24T06:13:26.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:26.471+0000 7f704a458640 1 -- 172.21.15.92:0/1382737117 reap_dead start 2023-11-24T06:13:26.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:26.471+0000 7f704bebb640 1 -- 172.21.15.92:0/1382737117 shutdown_connections 2023-11-24T06:13:26.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:26.471+0000 7f704bebb640 1 -- 172.21.15.92:0/1382737117 >> 172.21.15.92:0/1382737117 conn(0x7f704406a800 msgr2=0x7f704406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:13:26.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:26.471+0000 7f704bebb640 1 -- 172.21.15.92:0/1382737117 shutdown_connections 2023-11-24T06:13:26.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:26.471+0000 7f704bebb640 1 -- 172.21.15.92:0/1382737117 wait complete. 2023-11-24T06:13:26.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:26.474+0000 7f704bebb640 1 Processor -- start 2023-11-24T06:13:26.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:26.474+0000 7f704bebb640 1 -- start start 2023-11-24T06:13:26.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:26.475+0000 7f704bebb640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7030004d00 0x7f704413fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:26.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:26.475+0000 7f704bebb640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f70441613b0 0x7f70441f7090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:26.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:26.475+0000 7f704bebb640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f70440cd5b0 0x7f70440cda30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:26.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:26.475+0000 7f704bebb640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7044163810 con 0x7f7030004d00 2023-11-24T06:13:26.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:26.475+0000 7f704bebb640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70440cdf70 con 0x7f70440cd5b0 2023-11-24T06:13:26.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:26.475+0000 7f704bebb640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7044163480 con 0x7f70441613b0 2023-11-24T06:13:26.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:26.475+0000 7f7049c57640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7030004d00 0x7f704413fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:26.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:26.475+0000 7f7049c57640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7030004d00 0x7f704413fc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43888/0 (socket says 172.21.15.92:43888) 2023-11-24T06:13:26.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:26.475+0000 7f7049c57640 1 -- 172.21.15.92:0/2102273049 learned_addr learned my addr 172.21.15.92:0/2102273049 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:13:26.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:26.476+0000 7f7049c57640 1 -- 172.21.15.92:0/2102273049 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f70441613b0 msgr2=0x7f70441f7090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:26.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:26.476+0000 7f7049c57640 1 --2- 172.21.15.92:0/2102273049 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f70441613b0 0x7f70441f7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:26.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:26.476+0000 7f7049c57640 1 -- 172.21.15.92:0/2102273049 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f70440cd5b0 msgr2=0x7f70440cda30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:26.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:26.476+0000 7f7049c57640 1 --2- 172.21.15.92:0/2102273049 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f70440cd5b0 0x7f70440cda30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:26.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:26.476+0000 7f7049c57640 1 -- 172.21.15.92:0/2102273049 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7044161830 con 0x7f7030004d00 2023-11-24T06:13:26.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:26.476+0000 7f7049c57640 1 --2- 172.21.15.92:0/2102273049 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7030004d00 0x7f704413fc50 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f70400116d0 tx=0x7f7040011b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:26.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:26.476+0000 7f703a7fc640 1 -- 172.21.15.92:0/2102273049 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f704000f9c0 con 0x7f7030004d00 2023-11-24T06:13:26.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:26.476+0000 7f703a7fc640 1 -- 172.21.15.92:0/2102273049 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f704000fb20 con 0x7f7030004d00 2023-11-24T06:13:26.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:26.476+0000 7f703a7fc640 1 -- 172.21.15.92:0/2102273049 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f704000fe90 con 0x7f7030004d00 2023-11-24T06:13:26.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:26.476+0000 7f704bebb640 1 -- 172.21.15.92:0/2102273049 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70441f7910 con 0x7f7030004d00 2023-11-24T06:13:26.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:26.477+0000 7f704bebb640 1 -- 172.21.15.92:0/2102273049 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70441f7ed0 con 0x7f7030004d00 2023-11-24T06:13:26.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:26.478+0000 7f704bebb640 1 -- 172.21.15.92:0/2102273049 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70441619c0 con 0x7f7030004d00 2023-11-24T06:13:26.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:26.479+0000 7f703a7fc640 1 -- 172.21.15.92:0/2102273049 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 7) v1 ==== 89333+0+0 (secure 0 0 0) 0x7f7040015460 con 0x7f7030004d00 2023-11-24T06:13:26.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:26.480+0000 7f703a7fc640 1 --2- 172.21.15.92:0/2102273049 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f702806af60 0x7f702806d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:26.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:26.480+0000 7f703a7fc640 1 -- 172.21.15.92:0/2102273049 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7040089d10 con 0x7f7030004d00 2023-11-24T06:13:26.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:26.480+0000 7f7049456640 1 --2- 172.21.15.92:0/2102273049 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f702806af60 0x7f702806d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:26.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:26.480+0000 7f7049456640 1 --2- 172.21.15.92:0/2102273049 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f702806af60 0x7f702806d420 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f7044149730 tx=0x7f7034001d70 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:26.487 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:26.484+0000 7f703a7fc640 1 -- 172.21.15.92:0/2102273049 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f704008f020 con 0x7f7030004d00 2023-11-24T06:13:26.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:26.679+0000 7f704bebb640 1 -- 172.21.15.92:0/2102273049 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f70441635f0 con 0x7f7030004d00 2023-11-24T06:13:26.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:26.681+0000 7f703a7fc640 1 -- 172.21.15.92:0/2102273049 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v63) v1 ==== 74+0+16520 (secure 0 0 0) 0x7f7040022090 con 0x7f7030004d00 2023-11-24T06:13:26.685 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-24T06:13:26.686 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":63,"fsid":"2d6dee5f-8aa3-470f-931a-50ade963b1cb","created":"2023-11-24T06:10:04.707204+0000","modified":"2023-11-24T06:13:10.158258+0000","last_up_change":"2023-11-24T06:10:17.598352+0000","last_in_change":"2023-11-24T06:10:09.040793+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:10:17.733098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:13:06.409727+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-11-24T06:13:07.623678+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ff37a9fb-135f-471f-b4b0-f7b5772006c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6803","nonce":20560714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6805","nonce":20560714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6809","nonce":20560714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6807","nonce":20560714}]},"public_addr":"172.21.15.92:6803/20560714","cluster_addr":"172.21.15.92:6805/20560714","heartbeat_back_addr":"172.21.15.92:6809/20560714","heartbeat_front_addr":"172.21.15.92:6807/20560714","state":["exists","up"]},{"osd":1,"uuid":"03cdd353-dcf1-4f38-94fe-a795137c06b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6819","nonce":2451239906}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6821","nonce":2451239906}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6825","nonce":2451239906}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6823","nonce":2451239906}]},"public_addr":"172.21.15.92:6819/2451239906","cluster_addr":"172.21.15.92:6821/2451239906","heartbeat_back_addr":"172.21.15.92:6825/2451239906","heartbeat_front_addr":"172.21.15.92:6823/2451239906","state":["exists","up"]},{"osd":2,"uuid":"a4a20d80-5d57-484b-becd-9aa8070dd3aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6811","nonce":1124979277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6813","nonce":1124979277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6817","nonce":1124979277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6815","nonce":1124979277}]},"public_addr":"172.21.15.92:6811/1124979277","cluster_addr":"172.21.15.92:6813/1124979277","heartbeat_back_addr":"172.21.15.92:6817/1124979277","heartbeat_front_addr":"172.21.15.92:6815/1124979277","state":["exists","up"]},{"osd":3,"uuid":"4648d7ce-9b29-4583-9425-4ba8fd2b1903","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6827","nonce":1734115782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6829","nonce":1734115782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6833","nonce":1734115782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6831","nonce":1734115782}]},"public_addr":"172.21.15.92:6827/1734115782","cluster_addr":"172.21.15.92:6829/1734115782","heartbeat_back_addr":"172.21.15.92:6833/1734115782","heartbeat_front_addr":"172.21.15.92:6831/1734115782","state":["exists","up"]},{"osd":4,"uuid":"3bc9d7eb-c6fa-4fcf-a3ec-ade604e05aac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6819","nonce":3425937229}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6822","nonce":3425937229}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6830","nonce":3425937229}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6826","nonce":3425937229}]},"public_addr":"172.21.15.118:6819/3425937229","cluster_addr":"172.21.15.118:6822/3425937229","heartbeat_back_addr":"172.21.15.118:6830/3425937229","heartbeat_front_addr":"172.21.15.118:6826/3425937229","state":["exists","up"]},{"osd":5,"uuid":"dce9ba6d-59ed-46d1-84fc-a84b13de10c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6801","nonce":827455984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6803","nonce":827455984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6807","nonce":827455984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6805","nonce":827455984}]},"public_addr":"172.21.15.118:6801/827455984","cluster_addr":"172.21.15.118:6803/827455984","heartbeat_back_addr":"172.21.15.118:6807/827455984","heartbeat_front_addr":"172.21.15.118:6805/827455984","state":["exists","up"]},{"osd":6,"uuid":"82aec504-ae58-4709-8d30-46f2b12081b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6809","nonce":2946986141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6811","nonce":2946986141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6815","nonce":2946986141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6813","nonce":2946986141}]},"public_addr":"172.21.15.118:6809/2946986141","cluster_addr":"172.21.15.118:6811/2946986141","heartbeat_back_addr":"172.21.15.118:6815/2946986141","heartbeat_front_addr":"172.21.15.118:6813/2946986141","state":["exists","up"]},{"osd":7,"uuid":"4ed4c624-c88f-4d35-bdb1-bf209b5dffb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6817","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6818","nonce":48122059}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6821","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6823","nonce":48122059}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6829","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6831","nonce":48122059}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6825","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6827","nonce":48122059}]},"public_addr":"172.21.15.118:6818/48122059","cluster_addr":"172.21.15.118:6823/48122059","heartbeat_back_addr":"172.21.15.118:6831/48122059","heartbeat_front_addr":"172.21.15.118:6827/48122059","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.536664+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.368324+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.271201+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.342971+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.621832+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.387110+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.632946+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.416032+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:13:26.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:26.684+0000 7f704bebb640 1 -- 172.21.15.92:0/2102273049 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f702806af60 msgr2=0x7f702806d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:26.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:26.684+0000 7f704bebb640 1 --2- 172.21.15.92:0/2102273049 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f702806af60 0x7f702806d420 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f7044149730 tx=0x7f7034001d70 comp rx=0 tx=0).stop 2023-11-24T06:13:26.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:26.684+0000 7f704bebb640 1 -- 172.21.15.92:0/2102273049 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7030004d00 msgr2=0x7f704413fc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:26.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:26.684+0000 7f704bebb640 1 --2- 172.21.15.92:0/2102273049 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7030004d00 0x7f704413fc50 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f70400116d0 tx=0x7f7040011b90 comp rx=0 tx=0).stop 2023-11-24T06:13:26.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:26.685+0000 7f704bebb640 1 -- 172.21.15.92:0/2102273049 shutdown_connections 2023-11-24T06:13:26.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:26.685+0000 7f704bebb640 1 --2- 172.21.15.92:0/2102273049 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f702806af60 0x7f702806d420 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:26.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:26.685+0000 7f704bebb640 1 --2- 172.21.15.92:0/2102273049 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f70440cd5b0 0x7f70440cda30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:26.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:26.685+0000 7f704bebb640 1 --2- 172.21.15.92:0/2102273049 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f70441613b0 0x7f70441f7090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:26.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:26.685+0000 7f704bebb640 1 --2- 172.21.15.92:0/2102273049 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7030004d00 0x7f704413fc50 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:26.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:26.685+0000 7f704bebb640 1 -- 172.21.15.92:0/2102273049 >> 172.21.15.92:0/2102273049 conn(0x7f704406a800 msgr2=0x7f7044149bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:13:26.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:26.685+0000 7f704bebb640 1 -- 172.21.15.92:0/2102273049 shutdown_connections 2023-11-24T06:13:26.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:26.685+0000 7f704bebb640 1 -- 172.21.15.92:0/2102273049 wait complete. 2023-11-24T06:13:26.707 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T06:10:17.733098+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T06:13:06.409727+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '62', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T06:13:07.623678+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '63', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T06:13:26.708 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T06:13:26.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:26.922+0000 7fb01a11d640 1 Processor -- start 2023-11-24T06:13:26.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:26.923+0000 7fb01a11d640 1 -- start start 2023-11-24T06:13:26.928 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:26.924+0000 7fb01a11d640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb0140cf9a0 0x7fb01415dca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:26.928 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:26.924+0000 7fb01a11d640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb01415e1e0 0x7fb0140702c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:26.928 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:26.924+0000 7fb01a11d640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0140b1ee0 con 0x7fb014143ae0 2023-11-24T06:13:26.928 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:26.924+0000 7fb01a11d640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0140b2780 con 0x7fb01415e1e0 2023-11-24T06:13:26.928 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:26.924+0000 7fb01a11d640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb01415e990 con 0x7fb0140cf9a0 2023-11-24T06:13:26.928 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:26.924+0000 7fb013fff640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb01415e1e0 0x7fb0140702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:26.928 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:26.924+0000 7fb013fff640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb01415e1e0 0x7fb0140702c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43890/0 (socket says 172.21.15.92:43890) 2023-11-24T06:13:26.928 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:26.924+0000 7fb013fff640 1 -- 172.21.15.92:0/431741810 learned_addr learned my addr 172.21.15.92:0/431741810 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:13:26.928 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:26.925+0000 7fb012ffd640 1 --2- 172.21.15.92:0/431741810 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb0140cf9a0 0x7fb01415dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:26.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:26.925+0000 7fb013fff640 1 -- 172.21.15.92:0/431741810 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb0140cf9a0 msgr2=0x7fb01415dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:13:26.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:26.925+0000 7fb013fff640 1 --2- 172.21.15.92:0/431741810 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb0140cf9a0 0x7fb01415dca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:26.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:26.925+0000 7fb013fff640 1 -- 172.21.15.92:0/431741810 >> v1:172.21.15.92:6789/0 conn(0x7fb014143ae0 legacy=0x7fb0140cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:26.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:26.925+0000 7fb013fff640 1 -- 172.21.15.92:0/431741810 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0140b2130 con 0x7fb01415e1e0 2023-11-24T06:13:26.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:26.925+0000 7fb0127fc640 1 -- 172.21.15.92:0/431741810 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3606105572 0 0) 0x7fb0140b1ee0 con 0x7fb014143ae0 2023-11-24T06:13:26.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:26.925+0000 7fb012ffd640 1 --2- 172.21.15.92:0/431741810 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb0140cf9a0 0x7fb01415dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:13:26.930 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:26.926+0000 7fb013fff640 1 --2- 172.21.15.92:0/431741810 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb01415e1e0 0x7fb0140702c0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7fb00c00d150 tx=0x7fb00c02e160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=447fc1832cf545ba server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:26.930 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:26.926+0000 7fb0127fc640 1 -- 172.21.15.92:0/431741810 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb00c00b0d0 con 0x7fb01415e1e0 2023-11-24T06:13:26.930 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:26.926+0000 7fb0127fc640 1 -- 172.21.15.92:0/431741810 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb00c00b230 con 0x7fb01415e1e0 2023-11-24T06:13:26.930 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:26.926+0000 7fb0127fc640 1 -- 172.21.15.92:0/431741810 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb00c009600 con 0x7fb01415e1e0 2023-11-24T06:13:26.931 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:26.927+0000 7fb01a11d640 1 -- 172.21.15.92:0/431741810 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb01415e1e0 msgr2=0x7fb0140702c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:26.931 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:26.927+0000 7fb01a11d640 1 --2- 172.21.15.92:0/431741810 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb01415e1e0 0x7fb0140702c0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7fb00c00d150 tx=0x7fb00c02e160 comp rx=0 tx=0).stop 2023-11-24T06:13:26.931 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:26.927+0000 7fb01a11d640 1 -- 172.21.15.92:0/431741810 shutdown_connections 2023-11-24T06:13:26.931 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:26.927+0000 7fb01a11d640 1 --2- 172.21.15.92:0/431741810 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb01415e1e0 0x7fb0140702c0 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:26.931 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:26.927+0000 7fb01a11d640 1 --2- 172.21.15.92:0/431741810 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb0140cf9a0 0x7fb01415dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:26.931 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:26.927+0000 7fb01a11d640 1 -- 172.21.15.92:0/431741810 >> 172.21.15.92:0/431741810 conn(0x7fb01406a800 msgr2=0x7fb01406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:13:26.931 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:26.927+0000 7fb01a11d640 1 -- 172.21.15.92:0/431741810 shutdown_connections 2023-11-24T06:13:26.931 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:26.927+0000 7fb01a11d640 1 -- 172.21.15.92:0/431741810 wait complete. 2023-11-24T06:13:26.932 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:26.929+0000 7fb01a11d640 1 Processor -- start 2023-11-24T06:13:26.932 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:26.929+0000 7fb01a11d640 1 -- start start 2023-11-24T06:13:26.934 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:26.930+0000 7fb01a11d640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb0140cf9a0 0x7fb0140cd850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:26.934 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:26.930+0000 7fb01a11d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb014143ae0 0x7fb0141597b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:26.934 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:26.930+0000 7fb01a11d640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb01415e1e0 0x7fb01413d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:26.934 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:26.930+0000 7fb01a11d640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0140cd3c0 con 0x7fb014143ae0 2023-11-24T06:13:26.934 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:26.930+0000 7fb01a11d640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0140a9190 con 0x7fb0140cf9a0 2023-11-24T06:13:26.934 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:26.930+0000 7fb01a11d640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0140cc7b0 con 0x7fb01415e1e0 2023-11-24T06:13:26.934 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:26.930+0000 7fb012ffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb014143ae0 0x7fb0141597b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:26.934 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:26.930+0000 7fb012ffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb014143ae0 0x7fb0141597b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43896/0 (socket says 172.21.15.92:43896) 2023-11-24T06:13:26.934 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:26.930+0000 7fb012ffd640 1 -- 172.21.15.92:0/3704824773 learned_addr learned my addr 172.21.15.92:0/3704824773 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:13:26.934 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:26.930+0000 7fb013fff640 1 --2- 172.21.15.92:0/3704824773 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb01415e1e0 0x7fb01413d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:26.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:26.930+0000 7fb0137fe640 1 --2- 172.21.15.92:0/3704824773 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb0140cf9a0 0x7fb0140cd850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:26.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:26.930+0000 7fb012ffd640 1 -- 172.21.15.92:0/3704824773 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb01415e1e0 msgr2=0x7fb01413d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:26.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:26.930+0000 7fb012ffd640 1 --2- 172.21.15.92:0/3704824773 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb01415e1e0 0x7fb01413d350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:26.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:26.930+0000 7fb012ffd640 1 -- 172.21.15.92:0/3704824773 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb0140cf9a0 msgr2=0x7fb0140cd850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:26.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:26.930+0000 7fb012ffd640 1 --2- 172.21.15.92:0/3704824773 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb0140cf9a0 0x7fb0140cd850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:26.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:26.930+0000 7fb012ffd640 1 -- 172.21.15.92:0/3704824773 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb00c00bd40 con 0x7fb014143ae0 2023-11-24T06:13:26.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:26.930+0000 7fb0137fe640 1 --2- 172.21.15.92:0/3704824773 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb0140cf9a0 0x7fb0140cd850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:13:26.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:26.931+0000 7fb012ffd640 1 --2- 172.21.15.92:0/3704824773 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb014143ae0 0x7fb0141597b0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fb00800c820 tx=0x7fb00800cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:26.936 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:26.931+0000 7faffbfff640 1 -- 172.21.15.92:0/3704824773 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb008007a30 con 0x7fb014143ae0 2023-11-24T06:13:26.936 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:26.931+0000 7faffbfff640 1 -- 172.21.15.92:0/3704824773 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb008007b90 con 0x7fb014143ae0 2023-11-24T06:13:26.936 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:26.931+0000 7fb01a11d640 1 -- 172.21.15.92:0/3704824773 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0140cca90 con 0x7fb014143ae0 2023-11-24T06:13:26.936 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:26.931+0000 7faffbfff640 1 -- 172.21.15.92:0/3704824773 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb00800ce70 con 0x7fb014143ae0 2023-11-24T06:13:26.936 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:26.931+0000 7fb01a11d640 1 -- 172.21.15.92:0/3704824773 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0140cd050 con 0x7fb014143ae0 2023-11-24T06:13:26.936 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:26.932+0000 7fb01a11d640 1 -- 172.21.15.92:0/3704824773 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0140675d0 con 0x7fb014143ae0 2023-11-24T06:13:26.937 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:26.933+0000 7faffbfff640 1 -- 172.21.15.92:0/3704824773 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 7) v1 ==== 89333+0+0 (secure 0 0 0) 0x7fb0080113f0 con 0x7fb014143ae0 2023-11-24T06:13:26.937 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:26.934+0000 7faffbfff640 1 --2- 172.21.15.92:0/3704824773 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fb00006af60 0x7fb00006d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:26.937 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:26.934+0000 7faffbfff640 1 -- 172.21.15.92:0/3704824773 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb008089a30 con 0x7fb014143ae0 2023-11-24T06:13:26.937 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:26.934+0000 7fb0137fe640 1 --2- 172.21.15.92:0/3704824773 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fb00006af60 0x7fb00006d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:26.938 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:26.935+0000 7fb0137fe640 1 --2- 172.21.15.92:0/3704824773 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fb00006af60 0x7fb00006d420 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fb004000f60 tx=0x7fb004009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:26.942 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:26.939+0000 7faffbfff640 1 -- 172.21.15.92:0/3704824773 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb0080548c0 con 0x7fb014143ae0 2023-11-24T06:13:27.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:27.127+0000 7fb01a11d640 1 -- 172.21.15.92:0/3704824773 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb0140cf690 con 0x7fb014143ae0 2023-11-24T06:13:27.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:27.127+0000 7faffbfff640 1 -- 172.21.15.92:0/3704824773 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v63) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb00801f0a0 con 0x7fb014143ae0 2023-11-24T06:13:27.131 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 1 2023-11-24T06:13:27.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:27.130+0000 7fb01a11d640 1 -- 172.21.15.92:0/3704824773 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fb00006af60 msgr2=0x7fb00006d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:27.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:27.130+0000 7fb01a11d640 1 --2- 172.21.15.92:0/3704824773 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fb00006af60 0x7fb00006d420 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fb004000f60 tx=0x7fb004009040 comp rx=0 tx=0).stop 2023-11-24T06:13:27.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:27.130+0000 7fb01a11d640 1 -- 172.21.15.92:0/3704824773 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb014143ae0 msgr2=0x7fb0141597b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:27.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:27.130+0000 7fb01a11d640 1 --2- 172.21.15.92:0/3704824773 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb014143ae0 0x7fb0141597b0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fb00800c820 tx=0x7fb00800cc20 comp rx=0 tx=0).stop 2023-11-24T06:13:27.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:27.130+0000 7fb01a11d640 1 -- 172.21.15.92:0/3704824773 shutdown_connections 2023-11-24T06:13:27.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:27.130+0000 7fb01a11d640 1 --2- 172.21.15.92:0/3704824773 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fb00006af60 0x7fb00006d420 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:27.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:27.130+0000 7fb01a11d640 1 --2- 172.21.15.92:0/3704824773 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb01415e1e0 0x7fb01413d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:27.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:27.130+0000 7fb01a11d640 1 --2- 172.21.15.92:0/3704824773 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb014143ae0 0x7fb0141597b0 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:27.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:27.130+0000 7fb01a11d640 1 --2- 172.21.15.92:0/3704824773 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb0140cf9a0 0x7fb0140cd850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:27.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:27.130+0000 7fb01a11d640 1 -- 172.21.15.92:0/3704824773 >> 172.21.15.92:0/3704824773 conn(0x7fb01406a800 msgr2=0x7fb01406cbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:13:27.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:27.130+0000 7fb01a11d640 1 -- 172.21.15.92:0/3704824773 shutdown_connections 2023-11-24T06:13:27.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:27.131+0000 7fb01a11d640 1 -- 172.21.15.92:0/3704824773 wait complete. 2023-11-24T06:13:27.152 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T06:13:27.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:27.365+0000 7f63ba0ad640 1 Processor -- start 2023-11-24T06:13:27.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:27.365+0000 7f63ba0ad640 1 -- start start 2023-11-24T06:13:27.370 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:27.366+0000 7f63ba0ad640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f63b40adf20 0x7f63b4070a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:27.370 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:27.366+0000 7f63ba0ad640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f63b40ccb50 0x7f63b4143d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:27.370 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:27.366+0000 7f63ba0ad640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63b415f4e0 con 0x7f63b415bf10 2023-11-24T06:13:27.370 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:27.366+0000 7f63ba0ad640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63b415f970 con 0x7f63b40ccb50 2023-11-24T06:13:27.370 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:27.366+0000 7f63ba0ad640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63b4071130 con 0x7f63b40adf20 2023-11-24T06:13:27.370 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:27.367+0000 7f63b98ac640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f63b40ccb50 0x7f63b4143d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:27.370 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:27.367+0000 7f63b98ac640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f63b40ccb50 0x7f63b4143d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43902/0 (socket says 172.21.15.92:43902) 2023-11-24T06:13:27.370 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:27.367+0000 7f63b98ac640 1 -- 172.21.15.92:0/2353724536 learned_addr learned my addr 172.21.15.92:0/2353724536 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:13:27.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:27.367+0000 7f63b90ab640 1 --2- 172.21.15.92:0/2353724536 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f63b40adf20 0x7f63b4070a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:27.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:27.367+0000 7f63b98ac640 1 -- 172.21.15.92:0/2353724536 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f63b40adf20 msgr2=0x7f63b4070a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:13:27.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:27.367+0000 7f63b98ac640 1 --2- 172.21.15.92:0/2353724536 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f63b40adf20 0x7f63b4070a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:27.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:27.367+0000 7f63b98ac640 1 -- 172.21.15.92:0/2353724536 >> v1:172.21.15.92:6789/0 conn(0x7f63b415bf10 legacy=0x7f63b40cc440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:27.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:27.367+0000 7f63b98ac640 1 -- 172.21.15.92:0/2353724536 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63b415c390 con 0x7f63b40ccb50 2023-11-24T06:13:27.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:27.367+0000 7f63abfff640 1 -- 172.21.15.92:0/2353724536 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 495858472 0 0) 0x7f63b415f4e0 con 0x7f63b415bf10 2023-11-24T06:13:27.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:27.367+0000 7f63b90ab640 1 --2- 172.21.15.92:0/2353724536 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f63b40adf20 0x7f63b4070a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:13:27.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:27.368+0000 7f63b98ac640 1 --2- 172.21.15.92:0/2353724536 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f63b40ccb50 0x7f63b4143d30 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f63ac009e50 tx=0x7f63ac02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12c43672d494eb25 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:27.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:27.369+0000 7f63abfff640 1 -- 172.21.15.92:0/2353724536 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63ac02ca70 con 0x7f63b40ccb50 2023-11-24T06:13:27.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:27.369+0000 7f63abfff640 1 -- 172.21.15.92:0/2353724536 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63ac02cbd0 con 0x7f63b40ccb50 2023-11-24T06:13:27.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:27.369+0000 7f63abfff640 1 -- 172.21.15.92:0/2353724536 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63ac0343f0 con 0x7f63b40ccb50 2023-11-24T06:13:27.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:27.369+0000 7f63ba0ad640 1 -- 172.21.15.92:0/2353724536 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f63b40ccb50 msgr2=0x7f63b4143d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:27.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:27.369+0000 7f63ba0ad640 1 --2- 172.21.15.92:0/2353724536 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f63b40ccb50 0x7f63b4143d30 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f63ac009e50 tx=0x7f63ac02bed0 comp rx=0 tx=0).stop 2023-11-24T06:13:27.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:27.369+0000 7f63ba0ad640 1 -- 172.21.15.92:0/2353724536 shutdown_connections 2023-11-24T06:13:27.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:27.369+0000 7f63ba0ad640 1 --2- 172.21.15.92:0/2353724536 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f63b40ccb50 0x7f63b4143d30 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:27.374 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:27.369+0000 7f63ba0ad640 1 --2- 172.21.15.92:0/2353724536 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f63b40adf20 0x7f63b4070a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:27.374 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:27.369+0000 7f63ba0ad640 1 -- 172.21.15.92:0/2353724536 >> 172.21.15.92:0/2353724536 conn(0x7f63b406a7d0 msgr2=0x7f63b406abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:13:27.374 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:27.369+0000 7f63ba0ad640 1 -- 172.21.15.92:0/2353724536 shutdown_connections 2023-11-24T06:13:27.374 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:27.369+0000 7f63ba0ad640 1 -- 172.21.15.92:0/2353724536 wait complete. 2023-11-24T06:13:27.374 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:27.371+0000 7f63ba0ad640 1 Processor -- start 2023-11-24T06:13:27.374 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:27.371+0000 7f63ba0ad640 1 -- start start 2023-11-24T06:13:27.376 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:27.372+0000 7f63ba0ad640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f63b40adf20 0x7f63b40b37e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:27.376 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:27.372+0000 7f63ba0ad640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f63b40ccb50 0x7f63b4138a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:27.376 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:27.372+0000 7f63ba0ad640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f63b415bf10 0x7f63b40c4710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:27.376 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:27.372+0000 7f63ba0ad640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63b4138f60 con 0x7f63b40adf20 2023-11-24T06:13:27.376 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:27.372+0000 7f63ba0ad640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63b40c6000 con 0x7f63b415bf10 2023-11-24T06:13:27.376 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:27.372+0000 7f63ba0ad640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63b40c4f60 con 0x7f63b40ccb50 2023-11-24T06:13:27.376 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:27.372+0000 7f63b90ab640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f63b40adf20 0x7f63b40b37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:27.376 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:27.372+0000 7f63b90ab640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f63b40adf20 0x7f63b40b37e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43912/0 (socket says 172.21.15.92:43912) 2023-11-24T06:13:27.376 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:27.372+0000 7f63b90ab640 1 -- 172.21.15.92:0/2028489818 learned_addr learned my addr 172.21.15.92:0/2028489818 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:13:27.376 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:27.373+0000 7f63b90ab640 1 -- 172.21.15.92:0/2028489818 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f63b40ccb50 msgr2=0x7f63b4138a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:27.376 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:27.373+0000 7f63b90ab640 1 --2- 172.21.15.92:0/2028489818 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f63b40ccb50 0x7f63b4138a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:27.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:27.373+0000 7f63b90ab640 1 -- 172.21.15.92:0/2028489818 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f63b415bf10 msgr2=0x7f63b40c4710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:27.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:27.373+0000 7f63b90ab640 1 --2- 172.21.15.92:0/2028489818 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f63b415bf10 0x7f63b40c4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:27.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:27.373+0000 7f63b90ab640 1 -- 172.21.15.92:0/2028489818 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63ac009b30 con 0x7f63b40adf20 2023-11-24T06:13:27.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:27.373+0000 7f63b90ab640 1 --2- 172.21.15.92:0/2028489818 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f63b40adf20 0x7f63b40b37e0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f63b000c7f0 tx=0x7f63b000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:27.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:27.373+0000 7f63a9ffb640 1 -- 172.21.15.92:0/2028489818 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63b0007a30 con 0x7f63b40adf20 2023-11-24T06:13:27.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:27.373+0000 7f63a9ffb640 1 -- 172.21.15.92:0/2028489818 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63b0007b90 con 0x7f63b40adf20 2023-11-24T06:13:27.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:27.373+0000 7f63a9ffb640 1 -- 172.21.15.92:0/2028489818 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63b000ce40 con 0x7f63b40adf20 2023-11-24T06:13:27.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:27.373+0000 7f63ba0ad640 1 -- 172.21.15.92:0/2028489818 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63b40c5240 con 0x7f63b40adf20 2023-11-24T06:13:27.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:27.374+0000 7f63ba0ad640 1 -- 172.21.15.92:0/2028489818 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63b40c5800 con 0x7f63b40adf20 2023-11-24T06:13:27.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:27.375+0000 7f63ba0ad640 1 -- 172.21.15.92:0/2028489818 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63b4067450 con 0x7f63b40adf20 2023-11-24T06:13:27.379 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:27.376+0000 7f63a9ffb640 1 -- 172.21.15.92:0/2028489818 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 7) v1 ==== 89333+0+0 (secure 0 0 0) 0x7f63b00113f0 con 0x7f63b40adf20 2023-11-24T06:13:27.379 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:27.376+0000 7f63a9ffb640 1 --2- 172.21.15.92:0/2028489818 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f639806af60 0x7f639806d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:27.380 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:27.377+0000 7f63a9ffb640 1 -- 172.21.15.92:0/2028489818 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f63b0089aa0 con 0x7f63b40adf20 2023-11-24T06:13:27.380 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:27.377+0000 7f63b88aa640 1 --2- 172.21.15.92:0/2028489818 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f639806af60 0x7f639806d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:27.380 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:27.378+0000 7f63b88aa640 1 --2- 172.21.15.92:0/2028489818 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f639806af60 0x7f639806d420 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f63b41f6440 tx=0x7f63a4007930 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:27.384 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:27.381+0000 7f63a9ffb640 1 -- 172.21.15.92:0/2028489818 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f63b0054920 con 0x7f63b40adf20 2023-11-24T06:13:27.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:27.574+0000 7f63ba0ad640 1 -- 172.21.15.92:0/2028489818 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f63b406afe0 con 0x7f63b40adf20 2023-11-24T06:13:27.578 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:27.575+0000 7f63a9ffb640 1 -- 172.21.15.92:0/2028489818 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v63) v1 ==== 104+0+11 (secure 0 0 0) 0x7f63b001f020 con 0x7f63b40adf20 2023-11-24T06:13:27.579 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 32 2023-11-24T06:13:27.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:27.578+0000 7f63ba0ad640 1 -- 172.21.15.92:0/2028489818 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f639806af60 msgr2=0x7f639806d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:27.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:27.578+0000 7f63ba0ad640 1 --2- 172.21.15.92:0/2028489818 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f639806af60 0x7f639806d420 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f63b41f6440 tx=0x7f63a4007930 comp rx=0 tx=0).stop 2023-11-24T06:13:27.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:27.578+0000 7f63ba0ad640 1 -- 172.21.15.92:0/2028489818 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f63b40adf20 msgr2=0x7f63b40b37e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:27.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:27.578+0000 7f63ba0ad640 1 --2- 172.21.15.92:0/2028489818 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f63b40adf20 0x7f63b40b37e0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f63b000c7f0 tx=0x7f63b000cbf0 comp rx=0 tx=0).stop 2023-11-24T06:13:27.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:27.578+0000 7f63ba0ad640 1 -- 172.21.15.92:0/2028489818 shutdown_connections 2023-11-24T06:13:27.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:27.578+0000 7f63ba0ad640 1 --2- 172.21.15.92:0/2028489818 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f639806af60 0x7f639806d420 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:27.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:27.578+0000 7f63ba0ad640 1 --2- 172.21.15.92:0/2028489818 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f63b415bf10 0x7f63b40c4710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:27.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:27.578+0000 7f63ba0ad640 1 --2- 172.21.15.92:0/2028489818 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f63b40ccb50 0x7f63b4138a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:27.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:27.578+0000 7f63ba0ad640 1 --2- 172.21.15.92:0/2028489818 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f63b40adf20 0x7f63b40b37e0 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:27.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:27.578+0000 7f63ba0ad640 1 -- 172.21.15.92:0/2028489818 >> 172.21.15.92:0/2028489818 conn(0x7f63b406a7d0 msgr2=0x7f63b415b3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:13:27.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:27.578+0000 7f63ba0ad640 1 -- 172.21.15.92:0/2028489818 shutdown_connections 2023-11-24T06:13:27.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:27.578+0000 7f63ba0ad640 1 -- 172.21.15.92:0/2028489818 wait complete. 2023-11-24T06:13:27.602 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T06:13:27.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:27.814+0000 7f8877215640 1 Processor -- start 2023-11-24T06:13:27.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:27.815+0000 7f8877215640 1 -- start start 2023-11-24T06:13:27.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:27.816+0000 7f8877215640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f88700ac8d0 0x7f88700aede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:27.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:27.816+0000 7f8877215640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8870071de0 con 0x7f887006a3e0 2023-11-24T06:13:27.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:27.816+0000 7f8877215640 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8870072030 con 0x7f88700d0b60 2023-11-24T06:13:27.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:27.816+0000 7f8877215640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88700acd60 con 0x7f88700ac8d0 2023-11-24T06:13:27.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:27.816+0000 7f88757b2640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f88700ac8d0 0x7f88700aede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:27.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:27.816+0000 7f88757b2640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f88700ac8d0 0x7f88700aede0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43924/0 (socket says 172.21.15.92:43924) 2023-11-24T06:13:27.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:27.816+0000 7f8874fb1640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f887006a3e0 0x7f887006a7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:53616/0 (socket says 172.21.15.92:53616) 2023-11-24T06:13:27.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:27.816+0000 7f88757b2640 1 -- 172.21.15.92:0/1822104203 learned_addr learned my addr 172.21.15.92:0/1822104203 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:13:27.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:27.817+0000 7f88757b2640 1 -- 172.21.15.92:0/1822104203 >> v1:172.21.15.118:6789/0 conn(0x7f88700d0b60 legacy=0x7f887006f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:13:27.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:27.817+0000 7f88757b2640 1 -- 172.21.15.92:0/1822104203 >> v1:172.21.15.92:6789/0 conn(0x7f887006a3e0 legacy=0x7f887006a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:27.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:27.817+0000 7f88757b2640 1 -- 172.21.15.92:0/1822104203 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8870072310 con 0x7f88700ac8d0 2023-11-24T06:13:27.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:27.817+0000 7f886f7fe640 1 -- 172.21.15.92:0/1822104203 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1932208806 0 0) 0x7f8870071de0 con 0x7f887006a3e0 2023-11-24T06:13:27.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:27.818+0000 7f88757b2640 1 --2- 172.21.15.92:0/1822104203 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f88700ac8d0 0x7f88700aede0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f8868009870 tx=0x7f886802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5376c4b3e6fe543e server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:27.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:27.818+0000 7f886f7fe640 1 -- 172.21.15.92:0/1822104203 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8868030860 con 0x7f88700ac8d0 2023-11-24T06:13:27.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:27.818+0000 7f886f7fe640 1 -- 172.21.15.92:0/1822104203 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88680309c0 con 0x7f88700ac8d0 2023-11-24T06:13:27.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:27.818+0000 7f886f7fe640 1 -- 172.21.15.92:0/1822104203 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8868030d50 con 0x7f88700ac8d0 2023-11-24T06:13:27.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:27.819+0000 7f8877215640 1 -- 172.21.15.92:0/1822104203 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f88700ac8d0 msgr2=0x7f88700aede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:27.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:27.819+0000 7f8877215640 1 --2- 172.21.15.92:0/1822104203 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f88700ac8d0 0x7f88700aede0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f8868009870 tx=0x7f886802b9a0 comp rx=0 tx=0).stop 2023-11-24T06:13:27.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:27.819+0000 7f8877215640 1 -- 172.21.15.92:0/1822104203 shutdown_connections 2023-11-24T06:13:27.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:27.819+0000 7f8877215640 1 --2- 172.21.15.92:0/1822104203 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f88700ac8d0 0x7f88700aede0 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:27.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:27.819+0000 7f8877215640 1 -- 172.21.15.92:0/1822104203 >> 172.21.15.92:0/1822104203 conn(0x7f88700cfef0 msgr2=0x7f88700d0300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:13:27.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:27.819+0000 7f8877215640 1 -- 172.21.15.92:0/1822104203 shutdown_connections 2023-11-24T06:13:27.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:27.819+0000 7f8877215640 1 -- 172.21.15.92:0/1822104203 wait complete. 2023-11-24T06:13:27.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:27.820+0000 7f8877215640 1 Processor -- start 2023-11-24T06:13:27.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:27.821+0000 7f8877215640 1 -- start start 2023-11-24T06:13:27.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:27.822+0000 7f8877215640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f887006a3e0 0x7f8870157a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:27.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:27.822+0000 7f8877215640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f88700ac8d0 0x7f8870141a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:27.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:27.822+0000 7f8877215640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f88700d0b60 0x7f8870154480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:27.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:27.822+0000 7f8877215640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8870158e60 con 0x7f887006a3e0 2023-11-24T06:13:27.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:27.822+0000 7f8877215640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8870157f80 con 0x7f88700ac8d0 2023-11-24T06:13:27.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:27.822+0000 7f8877215640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8870158250 con 0x7f88700d0b60 2023-11-24T06:13:27.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:27.822+0000 7f8874fb1640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f887006a3e0 0x7f8870157a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:27.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:27.822+0000 7f8874fb1640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f887006a3e0 0x7f8870157a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43932/0 (socket says 172.21.15.92:43932) 2023-11-24T06:13:27.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:27.822+0000 7f8874fb1640 1 -- 172.21.15.92:0/2295833295 learned_addr learned my addr 172.21.15.92:0/2295833295 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:13:27.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:27.822+0000 7f886ffff640 1 --2- 172.21.15.92:0/2295833295 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f88700ac8d0 0x7f8870141a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:27.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:27.822+0000 7f8874fb1640 1 -- 172.21.15.92:0/2295833295 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f88700d0b60 msgr2=0x7f8870154480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:27.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:27.822+0000 7f8874fb1640 1 --2- 172.21.15.92:0/2295833295 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f88700d0b60 0x7f8870154480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:27.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:27.822+0000 7f8874fb1640 1 -- 172.21.15.92:0/2295833295 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f88700ac8d0 msgr2=0x7f8870141a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:27.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:27.822+0000 7f8874fb1640 1 --2- 172.21.15.92:0/2295833295 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f88700ac8d0 0x7f8870141a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:27.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:27.822+0000 7f8874fb1640 1 -- 172.21.15.92:0/2295833295 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88680094d0 con 0x7f887006a3e0 2023-11-24T06:13:27.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:27.823+0000 7f8874fb1640 1 --2- 172.21.15.92:0/2295833295 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f887006a3e0 0x7f8870157a40 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f886000c890 tx=0x7f886000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:27.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:27.823+0000 7f886d7fa640 1 -- 172.21.15.92:0/2295833295 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f886000a640 con 0x7f887006a3e0 2023-11-24T06:13:27.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:27.823+0000 7f886d7fa640 1 -- 172.21.15.92:0/2295833295 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f886000a7c0 con 0x7f887006a3e0 2023-11-24T06:13:27.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:27.823+0000 7f886d7fa640 1 -- 172.21.15.92:0/2295833295 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f886000ab70 con 0x7f887006a3e0 2023-11-24T06:13:27.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:27.823+0000 7f8877215640 1 -- 172.21.15.92:0/2295833295 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8870158530 con 0x7f887006a3e0 2023-11-24T06:13:27.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:27.823+0000 7f8877215640 1 -- 172.21.15.92:0/2295833295 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8870158af0 con 0x7f887006a3e0 2023-11-24T06:13:27.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:27.824+0000 7f8877215640 1 -- 172.21.15.92:0/2295833295 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88700675d0 con 0x7f887006a3e0 2023-11-24T06:13:27.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:27.825+0000 7f886d7fa640 1 -- 172.21.15.92:0/2295833295 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 7) v1 ==== 89333+0+0 (secure 0 0 0) 0x7f886000acd0 con 0x7f887006a3e0 2023-11-24T06:13:27.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:27.826+0000 7f886d7fa640 1 --2- 172.21.15.92:0/2295833295 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f885006af60 0x7f885006d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:27.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:27.826+0000 7f886d7fa640 1 -- 172.21.15.92:0/2295833295 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8860088d20 con 0x7f887006a3e0 2023-11-24T06:13:27.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:27.826+0000 7f886ffff640 1 --2- 172.21.15.92:0/2295833295 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f885006af60 0x7f885006d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:27.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:27.827+0000 7f886ffff640 1 --2- 172.21.15.92:0/2295833295 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f885006af60 0x7f885006d420 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f8870159cf0 tx=0x7f8858008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:27.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:27.830+0000 7f886d7fa640 1 -- 172.21.15.92:0/2295833295 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8860053cc0 con 0x7f887006a3e0 2023-11-24T06:13:28.023 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:28.020+0000 7f8877215640 1 -- 172.21.15.92:0/2295833295 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f887015c8a0 con 0x7f887006a3e0 2023-11-24T06:13:28.024 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:28.021+0000 7f886d7fa640 1 -- 172.21.15.92:0/2295833295 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v63) v1 ==== 100+0+11 (secure 0 0 0) 0x7f8860018020 con 0x7f887006a3e0 2023-11-24T06:13:28.024 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 64 2023-11-24T06:13:28.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:28.023+0000 7f8877215640 1 -- 172.21.15.92:0/2295833295 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f885006af60 msgr2=0x7f885006d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:28.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:28.023+0000 7f8877215640 1 --2- 172.21.15.92:0/2295833295 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f885006af60 0x7f885006d420 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f8870159cf0 tx=0x7f8858008040 comp rx=0 tx=0).stop 2023-11-24T06:13:28.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:28.023+0000 7f8877215640 1 -- 172.21.15.92:0/2295833295 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f887006a3e0 msgr2=0x7f8870157a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:28.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:28.023+0000 7f8877215640 1 --2- 172.21.15.92:0/2295833295 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f887006a3e0 0x7f8870157a40 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f886000c890 tx=0x7f886000cd50 comp rx=0 tx=0).stop 2023-11-24T06:13:28.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:28.023+0000 7f8877215640 1 -- 172.21.15.92:0/2295833295 shutdown_connections 2023-11-24T06:13:28.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:28.023+0000 7f8877215640 1 --2- 172.21.15.92:0/2295833295 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f885006af60 0x7f885006d420 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:28.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:28.023+0000 7f8877215640 1 --2- 172.21.15.92:0/2295833295 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f88700d0b60 0x7f8870154480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:28.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:28.023+0000 7f8877215640 1 --2- 172.21.15.92:0/2295833295 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f88700ac8d0 0x7f8870141a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:28.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:28.023+0000 7f8877215640 1 --2- 172.21.15.92:0/2295833295 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f887006a3e0 0x7f8870157a40 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:28.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:28.023+0000 7f8877215640 1 -- 172.21.15.92:0/2295833295 >> 172.21.15.92:0/2295833295 conn(0x7f88700cfef0 msgr2=0x7f88700713d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:13:28.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:28.024+0000 7f8877215640 1 -- 172.21.15.92:0/2295833295 shutdown_connections 2023-11-24T06:13:28.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:28.024+0000 7f8877215640 1 -- 172.21.15.92:0/2295833295 wait complete. 2023-11-24T06:13:28.046 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-24T06:13:28.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:28.254+0000 7f8225083640 1 Processor -- start 2023-11-24T06:13:28.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:28.254+0000 7f8225083640 1 -- start start 2023-11-24T06:13:28.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:28.256+0000 7f8225083640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f822013df80 0x7f822015bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:28.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:28.256+0000 7f8225083640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f82200cf9a0 0x7f82200cd840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:28.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:28.256+0000 7f8225083640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f822006c470 con 0x7f822013e360 2023-11-24T06:13:28.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:28.256+0000 7f8225083640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f822006c750 con 0x7f822013df80 2023-11-24T06:13:28.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:28.256+0000 7f8225083640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f822006ca30 con 0x7f82200cf9a0 2023-11-24T06:13:28.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:28.256+0000 7f821ed9d640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f822013df80 0x7f822015bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:28.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:28.256+0000 7f821ed9d640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f822013df80 0x7f822015bee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43948/0 (socket says 172.21.15.92:43948) 2023-11-24T06:13:28.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:28.256+0000 7f821ed9d640 1 -- 172.21.15.92:0/3580635902 learned_addr learned my addr 172.21.15.92:0/3580635902 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:13:28.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:28.256+0000 7f821f59e640 1 --2- 172.21.15.92:0/3580635902 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f82200cf9a0 0x7f82200cd840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:28.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:28.257+0000 7f821ed9d640 1 -- 172.21.15.92:0/3580635902 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f82200cf9a0 msgr2=0x7f82200cd840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:13:28.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:28.257+0000 7f821ed9d640 1 --2- 172.21.15.92:0/3580635902 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f82200cf9a0 0x7f82200cd840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:28.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:28.257+0000 7f821ed9d640 1 -- 172.21.15.92:0/3580635902 >> v1:172.21.15.92:6789/0 conn(0x7f822013e360 legacy=0x7f82200cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:28.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:28.257+0000 7f821ed9d640 1 -- 172.21.15.92:0/3580635902 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f822006cd10 con 0x7f822013df80 2023-11-24T06:13:28.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:28.257+0000 7f821f59e640 1 --2- 172.21.15.92:0/3580635902 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f82200cf9a0 0x7f82200cd840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:13:28.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:28.258+0000 7f821ed9d640 1 --2- 172.21.15.92:0/3580635902 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f822013df80 0x7f822015bee0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f8210009870 tx=0x7f821002b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69a481edb80bcbc2 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:28.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:28.258+0000 7f821dd9b640 1 -- 172.21.15.92:0/3580635902 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8210030840 con 0x7f822013df80 2023-11-24T06:13:28.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:28.258+0000 7f821dd9b640 1 -- 172.21.15.92:0/3580635902 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82100309a0 con 0x7f822013df80 2023-11-24T06:13:28.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:28.258+0000 7f821dd9b640 1 -- 172.21.15.92:0/3580635902 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8210030d30 con 0x7f822013df80 2023-11-24T06:13:28.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:28.259+0000 7f8225083640 1 -- 172.21.15.92:0/3580635902 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f822013df80 msgr2=0x7f822015bee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:28.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:28.259+0000 7f8225083640 1 --2- 172.21.15.92:0/3580635902 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f822013df80 0x7f822015bee0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f8210009870 tx=0x7f821002b980 comp rx=0 tx=0).stop 2023-11-24T06:13:28.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:28.259+0000 7f8225083640 1 -- 172.21.15.92:0/3580635902 shutdown_connections 2023-11-24T06:13:28.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:28.259+0000 7f8225083640 1 --2- 172.21.15.92:0/3580635902 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f82200cf9a0 0x7f82200cd840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:28.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:28.259+0000 7f8225083640 1 --2- 172.21.15.92:0/3580635902 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f822013df80 0x7f822015bee0 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:28.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:28.259+0000 7f8225083640 1 -- 172.21.15.92:0/3580635902 >> 172.21.15.92:0/3580635902 conn(0x7f822006a800 msgr2=0x7f822006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:13:28.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:28.259+0000 7f8225083640 1 -- 172.21.15.92:0/3580635902 shutdown_connections 2023-11-24T06:13:28.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:28.259+0000 7f8225083640 1 -- 172.21.15.92:0/3580635902 wait complete. 2023-11-24T06:13:28.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:28.261+0000 7f8225083640 1 Processor -- start 2023-11-24T06:13:28.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:28.261+0000 7f8225083640 1 -- start start 2023-11-24T06:13:28.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:28.262+0000 7f8225083640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f82200cf9a0 0x7f82201399b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:28.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:28.262+0000 7f8225083640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f822013df80 0x7f822013cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:28.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:28.262+0000 7f8225083640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f822013e360 0x7f8220153f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:28.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:28.262+0000 7f8225083640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8220139ef0 con 0x7f822013e360 2023-11-24T06:13:28.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:28.262+0000 7f8225083640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f822013a0a0 con 0x7f822013df80 2023-11-24T06:13:28.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:28.262+0000 7f8225083640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82201f4410 con 0x7f82200cf9a0 2023-11-24T06:13:28.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:28.262+0000 7f821f59e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f822013e360 0x7f8220153f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:28.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:28.262+0000 7f821f59e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f822013e360 0x7f8220153f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43952/0 (socket says 172.21.15.92:43952) 2023-11-24T06:13:28.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:28.262+0000 7f821f59e640 1 -- 172.21.15.92:0/128276910 learned_addr learned my addr 172.21.15.92:0/128276910 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:13:28.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:28.263+0000 7f821e59c640 1 --2- 172.21.15.92:0/128276910 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f822013df80 0x7f822013cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:28.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:28.263+0000 7f821ed9d640 1 --2- 172.21.15.92:0/128276910 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f82200cf9a0 0x7f82201399b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:28.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:28.263+0000 7f821f59e640 1 -- 172.21.15.92:0/128276910 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f82200cf9a0 msgr2=0x7f82201399b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:28.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:28.263+0000 7f821f59e640 1 --2- 172.21.15.92:0/128276910 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f82200cf9a0 0x7f82201399b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:28.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:28.263+0000 7f821f59e640 1 -- 172.21.15.92:0/128276910 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f822013df80 msgr2=0x7f822013cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:28.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:28.263+0000 7f821f59e640 1 --2- 172.21.15.92:0/128276910 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f822013df80 0x7f822013cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:28.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:28.263+0000 7f821f59e640 1 -- 172.21.15.92:0/128276910 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82100094d0 con 0x7f822013e360 2023-11-24T06:13:28.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:28.263+0000 7f821f59e640 1 --2- 172.21.15.92:0/128276910 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f822013e360 0x7f8220153f00 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f821800cd80 tx=0x7f821800a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:28.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:28.263+0000 7f82077fe640 1 -- 172.21.15.92:0/128276910 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f821800f680 con 0x7f822013e360 2023-11-24T06:13:28.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:28.263+0000 7f82077fe640 1 -- 172.21.15.92:0/128276910 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f821800f800 con 0x7f822013e360 2023-11-24T06:13:28.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:28.263+0000 7f82077fe640 1 -- 172.21.15.92:0/128276910 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f821800fbb0 con 0x7f822013e360 2023-11-24T06:13:28.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:28.263+0000 7f8225083640 1 -- 172.21.15.92:0/128276910 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f822013a340 con 0x7f822013e360 2023-11-24T06:13:28.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:28.264+0000 7f8225083640 1 -- 172.21.15.92:0/128276910 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f822013a900 con 0x7f822013e360 2023-11-24T06:13:28.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:28.265+0000 7f8225083640 1 -- 172.21.15.92:0/128276910 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82200675d0 con 0x7f822013e360 2023-11-24T06:13:28.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:28.265+0000 7f82077fe640 1 -- 172.21.15.92:0/128276910 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 7) v1 ==== 89333+0+0 (secure 0 0 0) 0x7f821800b040 con 0x7f822013e360 2023-11-24T06:13:28.269 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:28.266+0000 7f82077fe640 1 --2- 172.21.15.92:0/128276910 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f820c06af60 0x7f820c06d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:28.269 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:28.266+0000 7f82077fe640 1 -- 172.21.15.92:0/128276910 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f821808a2e0 con 0x7f822013e360 2023-11-24T06:13:28.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:28.267+0000 7f821ed9d640 1 --2- 172.21.15.92:0/128276910 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f820c06af60 0x7f820c06d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:28.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:28.267+0000 7f821ed9d640 1 --2- 172.21.15.92:0/128276910 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f820c06af60 0x7f820c06d420 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f8210036000 tx=0x7f821003b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:28.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:28.271+0000 7f82077fe640 1 -- 172.21.15.92:0/128276910 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f82180550d0 con 0x7f822013e360 2023-11-24T06:13:28.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:28.471+0000 7f8225083640 1 -- 172.21.15.92:0/128276910 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f822006df00 con 0x7f822013e360 2023-11-24T06:13:28.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:28.472+0000 7f82077fe640 1 -- 172.21.15.92:0/128276910 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v63) v1 ==== 80+0+0 (secure 0 0 0) 0x7f82180586f0 con 0x7f822013e360 2023-11-24T06:13:28.475 INFO:teuthology.orchestra.run.smithi092.stderr:listed 0 entries 2023-11-24T06:13:28.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:28.475+0000 7f8225083640 1 -- 172.21.15.92:0/128276910 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f820c06af60 msgr2=0x7f820c06d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:28.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:28.475+0000 7f8225083640 1 --2- 172.21.15.92:0/128276910 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f820c06af60 0x7f820c06d420 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f8210036000 tx=0x7f821003b040 comp rx=0 tx=0).stop 2023-11-24T06:13:28.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:28.475+0000 7f8225083640 1 -- 172.21.15.92:0/128276910 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f822013e360 msgr2=0x7f8220153f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:28.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:28.475+0000 7f8225083640 1 --2- 172.21.15.92:0/128276910 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f822013e360 0x7f8220153f00 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f821800cd80 tx=0x7f821800a650 comp rx=0 tx=0).stop 2023-11-24T06:13:28.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:28.475+0000 7f8225083640 1 -- 172.21.15.92:0/128276910 shutdown_connections 2023-11-24T06:13:28.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:28.475+0000 7f8225083640 1 --2- 172.21.15.92:0/128276910 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f820c06af60 0x7f820c06d420 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:28.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:28.475+0000 7f8225083640 1 --2- 172.21.15.92:0/128276910 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f822013e360 0x7f8220153f00 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:28.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:28.475+0000 7f8225083640 1 --2- 172.21.15.92:0/128276910 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f822013df80 0x7f822013cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:28.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:28.475+0000 7f8225083640 1 --2- 172.21.15.92:0/128276910 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f82200cf9a0 0x7f82201399b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:28.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:28.475+0000 7f8225083640 1 -- 172.21.15.92:0/128276910 >> 172.21.15.92:0/128276910 conn(0x7f822006a800 msgr2=0x7f82200cf260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:13:28.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:28.475+0000 7f8225083640 1 -- 172.21.15.92:0/128276910 shutdown_connections 2023-11-24T06:13:28.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:28.475+0000 7f8225083640 1 -- 172.21.15.92:0/128276910 wait complete. 2023-11-24T06:13:28.497 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi118.front.sepia.ceph.com... 2023-11-24T06:13:28.497 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:13:28.497 DEBUG:teuthology.orchestra.run.smithi118:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2023-11-24T06:13:28.599 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi118.stderr:ceph-fuse[95623]: fuse finished with error 0 and tester_r 0 2023-11-24T06:13:28.605 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi118.front.sepia.ceph.com 2023-11-24T06:13:28.605 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:13:28.605 DEBUG:teuthology.orchestra.run.smithi118:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-11-24T06:13:28.745 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-11-24T06:13:28.745 DEBUG:teuthology.orchestra.run.smithi092:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:13:28.809 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054298,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498824k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249408k,nr_inodes=812352,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498824k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-24T06:13:28.809 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi092.front.sepia.ceph.com 2023-11-24T06:13:28.809 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:13:28.809 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-11-24T06:13:28.860 INFO:teuthology.orchestra.run.smithi092.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-11-24T06:13:28.860 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-24T06:13:28.861 DEBUG:teuthology.orchestra.run.smithi118:> set -ex 2023-11-24T06:13:28.861 DEBUG:teuthology.orchestra.run.smithi118:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:13:28.920 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054284,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249396k,nr_inodes=812349,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-24T06:13:28.921 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi118.front.sepia.ceph.com 2023-11-24T06:13:28.921 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:13:28.921 DEBUG:teuthology.orchestra.run.smithi118:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-11-24T06:13:28.972 INFO:teuthology.orchestra.run.smithi118.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-11-24T06:13:28.973 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-24T06:13:28.974 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T06:13:29.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:29.190+0000 7fcc06290640 1 Processor -- start 2023-11-24T06:13:29.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:29.191+0000 7fcc06290640 1 -- start start 2023-11-24T06:13:29.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:29.192+0000 7fcc06290640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fcc0013df80 0x7fcc0015bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:29.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:29.192+0000 7fcc06290640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fcc0013e360 0x7fcc000cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:29.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:29.192+0000 7fcc06290640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fcc0015f9c0 0x7fcc000cf7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:29.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:29.192+0000 7fcc06290640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc0006c470 con 0x7fcc0013df80 2023-11-24T06:13:29.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:29.192+0000 7fcc06290640 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc0006c750 con 0x7fcc0015f9c0 2023-11-24T06:13:29.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:29.192+0000 7fcc06290640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc0006ca30 con 0x7fcc0013e360 2023-11-24T06:13:29.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:29.193+0000 7fcbfffff640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fcc0013df80 0x7fcc0015bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:29.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:29.193+0000 7fcbfffff640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fcc0013df80 0x7fcc0015bee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:33516/0 (socket says 172.21.15.92:33516) 2023-11-24T06:13:29.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:29.193+0000 7fcbfffff640 1 -- 172.21.15.92:0/390566283 learned_addr learned my addr 172.21.15.92:0/390566283 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:13:29.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:29.193+0000 7fcc0482d640 1 --2- 172.21.15.92:0/390566283 >> v2:172.21.15.118:3300/0 conn(0x7fcc0015f9c0 0x7fcc000cf7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:29.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:29.193+0000 7fcbff7fe640 1 --2- 172.21.15.92:0/390566283 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fcc0013e360 0x7fcc000cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:29.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:29.193+0000 7fcbfffff640 1 -- 172.21.15.92:0/390566283 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fcc0013e360 msgr2=0x7fcc000cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:13:29.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:29.193+0000 7fcbfffff640 1 --2- 172.21.15.92:0/390566283 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fcc0013e360 0x7fcc000cf290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:29.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:29.193+0000 7fcbfffff640 1 -- 172.21.15.92:0/390566283 >> v2:172.21.15.118:3300/0 conn(0x7fcc0015f9c0 msgr2=0x7fcc000cf7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:13:29.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:29.193+0000 7fcbfffff640 1 --2- 172.21.15.92:0/390566283 >> v2:172.21.15.118:3300/0 conn(0x7fcc0015f9c0 0x7fcc000cf7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:29.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:29.193+0000 7fcbfffff640 1 -- 172.21.15.92:0/390566283 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc0006cd10 con 0x7fcc0013df80 2023-11-24T06:13:29.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:29.195+0000 7fcbfffff640 1 --2- 172.21.15.92:0/390566283 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcc0013df80 0x7fcc0015bee0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fcc0006e6d0 tx=0x7fcbf002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6612c15f9a61750d server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:29.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:29.195+0000 7fcbfeffd640 1 -- 172.21.15.92:0/390566283 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbf002ca70 con 0x7fcc0013df80 2023-11-24T06:13:29.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:29.195+0000 7fcbfeffd640 1 -- 172.21.15.92:0/390566283 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbf002cbd0 con 0x7fcc0013df80 2023-11-24T06:13:29.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:29.195+0000 7fcbfeffd640 1 -- 172.21.15.92:0/390566283 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbf00343f0 con 0x7fcc0013df80 2023-11-24T06:13:29.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:29.196+0000 7fcc06290640 1 -- 172.21.15.92:0/390566283 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcc0013df80 msgr2=0x7fcc0015bee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:29.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:29.196+0000 7fcc06290640 1 --2- 172.21.15.92:0/390566283 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcc0013df80 0x7fcc0015bee0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fcc0006e6d0 tx=0x7fcbf002bed0 comp rx=0 tx=0).stop 2023-11-24T06:13:29.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:29.196+0000 7fcc06290640 1 -- 172.21.15.92:0/390566283 shutdown_connections 2023-11-24T06:13:29.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:29.196+0000 7fcc06290640 1 --2- 172.21.15.92:0/390566283 >> v2:172.21.15.118:3300/0 conn(0x7fcc0015f9c0 0x7fcc000cf7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:29.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:29.196+0000 7fcc06290640 1 --2- 172.21.15.92:0/390566283 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fcc0013e360 0x7fcc000cf290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:29.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:29.196+0000 7fcc06290640 1 --2- 172.21.15.92:0/390566283 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcc0013df80 0x7fcc0015bee0 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:29.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:29.196+0000 7fcc06290640 1 -- 172.21.15.92:0/390566283 >> 172.21.15.92:0/390566283 conn(0x7fcc0006a800 msgr2=0x7fcc0006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:13:29.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:29.196+0000 7fcc06290640 1 -- 172.21.15.92:0/390566283 shutdown_connections 2023-11-24T06:13:29.201 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:29.196+0000 7fcc06290640 1 -- 172.21.15.92:0/390566283 wait complete. 2023-11-24T06:13:29.201 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:29.198+0000 7fcc06290640 1 Processor -- start 2023-11-24T06:13:29.201 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:29.198+0000 7fcc06290640 1 -- start start 2023-11-24T06:13:29.203 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:29.199+0000 7fcc06290640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fcc0013df80 0x7fcc000b3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:29.204 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:29.199+0000 7fcc06290640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcc0013e360 0x7fcc001f3060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:29.204 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:29.199+0000 7fcc06290640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fcc0015f9c0 0x7fcc00139820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:29.204 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:29.199+0000 7fcc06290640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc001f35a0 con 0x7fcc0013e360 2023-11-24T06:13:29.204 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:29.199+0000 7fcc06290640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc001f4050 con 0x7fcc0013df80 2023-11-24T06:13:29.204 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:29.199+0000 7fcc06290640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc001f37b0 con 0x7fcc0015f9c0 2023-11-24T06:13:29.204 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:29.199+0000 7fcbff7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcc0013e360 0x7fcc001f3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:29.204 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:29.199+0000 7fcbff7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcc0013e360 0x7fcc001f3060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:33532/0 (socket says 172.21.15.92:33532) 2023-11-24T06:13:29.204 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:29.199+0000 7fcbff7fe640 1 -- 172.21.15.92:0/1274576399 learned_addr learned my addr 172.21.15.92:0/1274576399 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:13:29.204 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:29.200+0000 7fcbff7fe640 1 -- 172.21.15.92:0/1274576399 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fcc0015f9c0 msgr2=0x7fcc00139820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:29.204 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:29.200+0000 7fcbff7fe640 1 --2- 172.21.15.92:0/1274576399 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fcc0015f9c0 0x7fcc00139820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:29.205 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:29.200+0000 7fcbff7fe640 1 -- 172.21.15.92:0/1274576399 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fcc0013df80 msgr2=0x7fcc000b3a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:29.205 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:29.200+0000 7fcbfffff640 1 --2- 172.21.15.92:0/1274576399 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fcc0013df80 0x7fcc000b3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:29.205 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:29.200+0000 7fcbff7fe640 1 --2- 172.21.15.92:0/1274576399 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fcc0013df80 0x7fcc000b3a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:29.205 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:29.200+0000 7fcbff7fe640 1 -- 172.21.15.92:0/1274576399 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbf0009b30 con 0x7fcc0013e360 2023-11-24T06:13:29.205 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:29.200+0000 7fcbff7fe640 1 --2- 172.21.15.92:0/1274576399 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcc0013e360 0x7fcc001f3060 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7fcbf400d800 tx=0x7fcbf400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:29.205 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:29.200+0000 7fcbfcff9640 1 -- 172.21.15.92:0/1274576399 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbf400b6c0 con 0x7fcc0013e360 2023-11-24T06:13:29.205 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:29.200+0000 7fcbfcff9640 1 -- 172.21.15.92:0/1274576399 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbf400b820 con 0x7fcc0013e360 2023-11-24T06:13:29.205 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:29.200+0000 7fcc06290640 1 -- 172.21.15.92:0/1274576399 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc001f3a90 con 0x7fcc0013e360 2023-11-24T06:13:29.205 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:29.200+0000 7fcbfcff9640 1 -- 172.21.15.92:0/1274576399 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbf400bbb0 con 0x7fcc0013e360 2023-11-24T06:13:29.205 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:29.201+0000 7fcc06290640 1 -- 172.21.15.92:0/1274576399 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc0013a220 con 0x7fcc0013e360 2023-11-24T06:13:29.206 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:29.202+0000 7fcc06290640 1 -- 172.21.15.92:0/1274576399 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc000ce980 con 0x7fcc0013e360 2023-11-24T06:13:29.206 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:29.202+0000 7fcbfcff9640 1 -- 172.21.15.92:0/1274576399 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 7) v1 ==== 89333+0+0 (secure 0 0 0) 0x7fcbf400bd10 con 0x7fcc0013e360 2023-11-24T06:13:29.206 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:29.203+0000 7fcbfcff9640 1 --2- 172.21.15.92:0/1274576399 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fcbec06af60 0x7fcbec06d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:29.207 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:29.203+0000 7fcbfcff9640 1 -- 172.21.15.92:0/1274576399 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcbf4089d40 con 0x7fcc0013e360 2023-11-24T06:13:29.207 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:29.204+0000 7fcbfffff640 1 --2- 172.21.15.92:0/1274576399 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fcbec06af60 0x7fcbec06d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:29.207 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:29.204+0000 7fcbfffff640 1 --2- 172.21.15.92:0/1274576399 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fcbec06af60 0x7fcbec06d420 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fcbf0035000 tx=0x7fcbf0030000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:29.210 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:29.208+0000 7fcbfcff9640 1 -- 172.21.15.92:0/1274576399 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fcbf4054cd0 con 0x7fcc0013e360 2023-11-24T06:13:29.438 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:29.435+0000 7fcc06290640 1 -- 172.21.15.92:0/1274576399 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcc000675d0 con 0x7fcc0013e360 2023-11-24T06:13:29.439 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:29.436+0000 7fcbfcff9640 1 -- 172.21.15.92:0/1274576399 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) v1 ==== 94+0+4921 (secure 0 0 0) 0x7fcbf40582f0 con 0x7fcc0013e360 2023-11-24T06:13:29.440 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-24T06:13:29.440 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":41,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5143,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6833/2094733752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":2094733752},{"type":"v1","addr":"172.21.15.118:6833","nonce":2094733752}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":36},{"gid":5146,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6835/2873546779","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":2873546779},{"type":"v1","addr":"172.21.15.118:6835","nonce":2873546779}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":36},{"gid":6141,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6836/3335501795","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":3335501795},{"type":"v1","addr":"172.21.15.92:6836","nonce":3335501795}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":35}],"filesystems":[{"mdsmap":{"epoch":40,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T06:13:08.625588+0000","modified":"2023-11-24T06:13:11.658333+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6144},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6144":{"gid":6144,"name":"a","rank":0,"incarnation":37,"state":"up:active","state_seq":3,"addr":"172.21.15.92:6837/3913457833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":3913457833},{"type":"v1","addr":"172.21.15.92:6837","nonce":3913457833}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-11-24T06:13:29.440 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 41 2023-11-24T06:13:29.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:29.439+0000 7fcc06290640 1 -- 172.21.15.92:0/1274576399 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fcbec06af60 msgr2=0x7fcbec06d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:29.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:29.439+0000 7fcc06290640 1 --2- 172.21.15.92:0/1274576399 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fcbec06af60 0x7fcbec06d420 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fcbf0035000 tx=0x7fcbf0030000 comp rx=0 tx=0).stop 2023-11-24T06:13:29.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:29.439+0000 7fcc06290640 1 -- 172.21.15.92:0/1274576399 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcc0013e360 msgr2=0x7fcc001f3060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:29.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:29.439+0000 7fcc06290640 1 --2- 172.21.15.92:0/1274576399 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcc0013e360 0x7fcc001f3060 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7fcbf400d800 tx=0x7fcbf400dcc0 comp rx=0 tx=0).stop 2023-11-24T06:13:29.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:29.439+0000 7fcc06290640 1 -- 172.21.15.92:0/1274576399 shutdown_connections 2023-11-24T06:13:29.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:29.439+0000 7fcc06290640 1 --2- 172.21.15.92:0/1274576399 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fcbec06af60 0x7fcbec06d420 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:29.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:29.439+0000 7fcc06290640 1 --2- 172.21.15.92:0/1274576399 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fcc0015f9c0 0x7fcc00139820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:29.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:29.439+0000 7fcc06290640 1 --2- 172.21.15.92:0/1274576399 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcc0013e360 0x7fcc001f3060 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:31.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:29.439+0000 7fcc06290640 1 --2- 172.21.15.92:0/1274576399 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fcc0013df80 0x7fcc000b3a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:31.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:29.439+0000 7fcc06290640 1 -- 172.21.15.92:0/1274576399 >> 172.21.15.92:0/1274576399 conn(0x7fcc0006a800 msgr2=0x7fcc0006abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:13:31.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:29.439+0000 7fcc06290640 1 -- 172.21.15.92:0/1274576399 shutdown_connections 2023-11-24T06:13:31.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:29.439+0000 7fcc06290640 1 -- 172.21.15.92:0/1274576399 wait complete. 2023-11-24T06:13:31.628 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:13:31.852 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:31.848+0000 7f3639f0d640 1 Processor -- start 2023-11-24T06:13:31.852 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:31.849+0000 7f3639f0d640 1 -- start start 2023-11-24T06:13:31.854 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:31.850+0000 7f3639f0d640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f3634071410 0x7f3634146770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:31.854 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:31.850+0000 7f3639f0d640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f3634071f00 0x7f3634166cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:31.854 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:31.850+0000 7f3639f0d640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36340b2e10 con 0x7f363406c7c0 2023-11-24T06:13:31.854 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:31.850+0000 7f3639f0d640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f363406cc50 con 0x7f3634071410 2023-11-24T06:13:31.854 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:31.850+0000 7f3639f0d640 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f363406cec0 con 0x7f3634071f00 2023-11-24T06:13:31.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:31.850+0000 7f3638f0b640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f3634071410 0x7f3634146770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:31.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:31.851+0000 7f3638f0b640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f3634071410 0x7f3634146770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:33542/0 (socket says 172.21.15.92:33542) 2023-11-24T06:13:31.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:31.851+0000 7f3638f0b640 1 -- 172.21.15.92:0/3821954942 learned_addr learned my addr 172.21.15.92:0/3821954942 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:13:31.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:31.850+0000 7f3633fff640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f363406c7c0 0x7f36340717f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:33548/0 (socket says 172.21.15.92:33548) 2023-11-24T06:13:31.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:31.851+0000 7f363970c640 1 --2- 172.21.15.92:0/3821954942 >> v2:172.21.15.118:3300/0 conn(0x7f3634071f00 0x7f3634166cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:31.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:31.851+0000 7f3638f0b640 1 -- 172.21.15.92:0/3821954942 >> v2:172.21.15.118:3300/0 conn(0x7f3634071f00 msgr2=0x7f3634166cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:13:31.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:31.851+0000 7f3638f0b640 1 --2- 172.21.15.92:0/3821954942 >> v2:172.21.15.118:3300/0 conn(0x7f3634071f00 0x7f3634166cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:31.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:31.851+0000 7f3638f0b640 1 -- 172.21.15.92:0/3821954942 >> v1:172.21.15.92:6789/0 conn(0x7f363406c7c0 legacy=0x7f36340717f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:31.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:31.851+0000 7f3638f0b640 1 -- 172.21.15.92:0/3821954942 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f363406d1a0 con 0x7f3634071410 2023-11-24T06:13:31.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:31.851+0000 7f36337fe640 1 -- 172.21.15.92:0/3821954942 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 452375928 0 0) 0x7f36340b2e10 con 0x7f363406c7c0 2023-11-24T06:13:31.856 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:31.852+0000 7f3638f0b640 1 --2- 172.21.15.92:0/3821954942 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3634071410 0x7f3634146770 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f36340ad5c0 tx=0x7f362802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7147b62770f29a5c server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:31.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:31.852+0000 7f36337fe640 1 -- 172.21.15.92:0/3821954942 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3628030860 con 0x7f3634071410 2023-11-24T06:13:31.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:31.852+0000 7f36337fe640 1 -- 172.21.15.92:0/3821954942 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36280309c0 con 0x7f3634071410 2023-11-24T06:13:31.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:31.853+0000 7f36337fe640 1 -- 172.21.15.92:0/3821954942 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3628030d50 con 0x7f3634071410 2023-11-24T06:13:31.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:31.853+0000 7f3639f0d640 1 -- 172.21.15.92:0/3821954942 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3634071410 msgr2=0x7f3634146770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:31.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:31.853+0000 7f3639f0d640 1 --2- 172.21.15.92:0/3821954942 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3634071410 0x7f3634146770 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f36340ad5c0 tx=0x7f362802b9a0 comp rx=0 tx=0).stop 2023-11-24T06:13:31.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:31.853+0000 7f3639f0d640 1 -- 172.21.15.92:0/3821954942 shutdown_connections 2023-11-24T06:13:31.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:31.853+0000 7f3639f0d640 1 --2- 172.21.15.92:0/3821954942 >> v2:172.21.15.118:3300/0 conn(0x7f3634071f00 0x7f3634166cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:31.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:31.853+0000 7f3639f0d640 1 --2- 172.21.15.92:0/3821954942 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3634071410 0x7f3634146770 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:31.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:31.853+0000 7f3639f0d640 1 -- 172.21.15.92:0/3821954942 >> 172.21.15.92:0/3821954942 conn(0x7f363406a800 msgr2=0x7f363406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:13:31.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:31.853+0000 7f3639f0d640 1 -- 172.21.15.92:0/3821954942 shutdown_connections 2023-11-24T06:13:31.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:31.853+0000 7f3639f0d640 1 -- 172.21.15.92:0/3821954942 wait complete. 2023-11-24T06:13:31.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:31.855+0000 7f3639f0d640 1 Processor -- start 2023-11-24T06:13:31.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:31.855+0000 7f3639f0d640 1 -- start start 2023-11-24T06:13:31.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:31.856+0000 7f3639f0d640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f363406c7c0 0x7f3634136390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:31.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:31.856+0000 7f3639f0d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3634071410 0x7f36341f2b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:31.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:31.856+0000 7f3639f0d640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3634071f00 0x7f36340c5f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:31.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:31.856+0000 7f3639f0d640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3634143b50 con 0x7f3634071410 2023-11-24T06:13:31.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:31.856+0000 7f3639f0d640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3634142f60 con 0x7f3634071f00 2023-11-24T06:13:31.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:31.856+0000 7f3639f0d640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36341431b0 con 0x7f363406c7c0 2023-11-24T06:13:31.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:31.856+0000 7f3633fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3634071410 0x7f36341f2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:31.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:31.856+0000 7f3633fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3634071410 0x7f36341f2b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:33552/0 (socket says 172.21.15.92:33552) 2023-11-24T06:13:31.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:31.856+0000 7f3633fff640 1 -- 172.21.15.92:0/3371378811 learned_addr learned my addr 172.21.15.92:0/3371378811 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:13:31.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:31.857+0000 7f3633fff640 1 -- 172.21.15.92:0/3371378811 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f363406c7c0 msgr2=0x7f3634136390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:31.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:31.857+0000 7f3633fff640 1 --2- 172.21.15.92:0/3371378811 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f363406c7c0 0x7f3634136390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:31.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:31.857+0000 7f3633fff640 1 -- 172.21.15.92:0/3371378811 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3634071f00 msgr2=0x7f36340c5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:31.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:31.857+0000 7f3633fff640 1 --2- 172.21.15.92:0/3371378811 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3634071f00 0x7f36340c5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:31.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:31.857+0000 7f3633fff640 1 -- 172.21.15.92:0/3371378811 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36280094d0 con 0x7f3634071410 2023-11-24T06:13:31.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:31.857+0000 7f3633fff640 1 --2- 172.21.15.92:0/3371378811 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3634071410 0x7f36341f2b80 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f362000c4d0 tx=0x7f362000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:31.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:31.857+0000 7f36317fa640 1 -- 172.21.15.92:0/3371378811 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36200074e0 con 0x7f3634071410 2023-11-24T06:13:31.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:31.857+0000 7f36317fa640 1 -- 172.21.15.92:0/3371378811 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3620007660 con 0x7f3634071410 2023-11-24T06:13:31.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:31.857+0000 7f36317fa640 1 -- 172.21.15.92:0/3371378811 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3620004a20 con 0x7f3634071410 2023-11-24T06:13:31.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:31.857+0000 7f3639f0d640 1 -- 172.21.15.92:0/3371378811 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3634143490 con 0x7f3634071410 2023-11-24T06:13:31.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:31.858+0000 7f3639f0d640 1 -- 172.21.15.92:0/3371378811 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36341368d0 con 0x7f3634071410 2023-11-24T06:13:31.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:31.859+0000 7f3639f0d640 1 -- 172.21.15.92:0/3371378811 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3634143c90 con 0x7f3634071410 2023-11-24T06:13:31.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:31.860+0000 7f36317fa640 1 -- 172.21.15.92:0/3371378811 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 7) v1 ==== 89333+0+0 (secure 0 0 0) 0x7f3620004b80 con 0x7f3634071410 2023-11-24T06:13:31.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:31.860+0000 7f36317fa640 1 --2- 172.21.15.92:0/3371378811 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f361806af60 0x7f361806d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:31.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:31.861+0000 7f36317fa640 1 -- 172.21.15.92:0/3371378811 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3620088960 con 0x7f3634071410 2023-11-24T06:13:31.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:31.861+0000 7f3638f0b640 1 --2- 172.21.15.92:0/3371378811 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f361806af60 0x7f361806d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:31.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:31.862+0000 7f3638f0b640 1 --2- 172.21.15.92:0/3371378811 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f361806af60 0x7f361806d420 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f3628036000 tx=0x7f362803b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:31.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:31.865+0000 7f36317fa640 1 -- 172.21.15.92:0/3371378811 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3634143c90 con 0x7f3634071410 2023-11-24T06:13:32.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:32.058+0000 7f3639f0d640 1 -- 172.21.15.92:0/3371378811 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3634143c90 con 0x7f3634071410 2023-11-24T06:13:32.064 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:32.061+0000 7f36317fa640 1 -- 172.21.15.92:0/3371378811 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v63) v1 ==== 74+0+16520 (secure 0 0 0) 0x7f3634143c90 con 0x7f3634071410 2023-11-24T06:13:32.064 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-24T06:13:32.065 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":63,"fsid":"2d6dee5f-8aa3-470f-931a-50ade963b1cb","created":"2023-11-24T06:10:04.707204+0000","modified":"2023-11-24T06:13:10.158258+0000","last_up_change":"2023-11-24T06:10:17.598352+0000","last_in_change":"2023-11-24T06:10:09.040793+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:10:17.733098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:13:06.409727+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-11-24T06:13:07.623678+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ff37a9fb-135f-471f-b4b0-f7b5772006c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6803","nonce":20560714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6805","nonce":20560714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6809","nonce":20560714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6807","nonce":20560714}]},"public_addr":"172.21.15.92:6803/20560714","cluster_addr":"172.21.15.92:6805/20560714","heartbeat_back_addr":"172.21.15.92:6809/20560714","heartbeat_front_addr":"172.21.15.92:6807/20560714","state":["exists","up"]},{"osd":1,"uuid":"03cdd353-dcf1-4f38-94fe-a795137c06b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6819","nonce":2451239906}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6821","nonce":2451239906}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6825","nonce":2451239906}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6823","nonce":2451239906}]},"public_addr":"172.21.15.92:6819/2451239906","cluster_addr":"172.21.15.92:6821/2451239906","heartbeat_back_addr":"172.21.15.92:6825/2451239906","heartbeat_front_addr":"172.21.15.92:6823/2451239906","state":["exists","up"]},{"osd":2,"uuid":"a4a20d80-5d57-484b-becd-9aa8070dd3aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6811","nonce":1124979277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6813","nonce":1124979277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6817","nonce":1124979277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6815","nonce":1124979277}]},"public_addr":"172.21.15.92:6811/1124979277","cluster_addr":"172.21.15.92:6813/1124979277","heartbeat_back_addr":"172.21.15.92:6817/1124979277","heartbeat_front_addr":"172.21.15.92:6815/1124979277","state":["exists","up"]},{"osd":3,"uuid":"4648d7ce-9b29-4583-9425-4ba8fd2b1903","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6827","nonce":1734115782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6829","nonce":1734115782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6833","nonce":1734115782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6831","nonce":1734115782}]},"public_addr":"172.21.15.92:6827/1734115782","cluster_addr":"172.21.15.92:6829/1734115782","heartbeat_back_addr":"172.21.15.92:6833/1734115782","heartbeat_front_addr":"172.21.15.92:6831/1734115782","state":["exists","up"]},{"osd":4,"uuid":"3bc9d7eb-c6fa-4fcf-a3ec-ade604e05aac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6819","nonce":3425937229}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6822","nonce":3425937229}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6830","nonce":3425937229}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6826","nonce":3425937229}]},"public_addr":"172.21.15.118:6819/3425937229","cluster_addr":"172.21.15.118:6822/3425937229","heartbeat_back_addr":"172.21.15.118:6830/3425937229","heartbeat_front_addr":"172.21.15.118:6826/3425937229","state":["exists","up"]},{"osd":5,"uuid":"dce9ba6d-59ed-46d1-84fc-a84b13de10c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6801","nonce":827455984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6803","nonce":827455984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6807","nonce":827455984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6805","nonce":827455984}]},"public_addr":"172.21.15.118:6801/827455984","cluster_addr":"172.21.15.118:6803/827455984","heartbeat_back_addr":"172.21.15.118:6807/827455984","heartbeat_front_addr":"172.21.15.118:6805/827455984","state":["exists","up"]},{"osd":6,"uuid":"82aec504-ae58-4709-8d30-46f2b12081b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6809","nonce":2946986141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6811","nonce":2946986141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6815","nonce":2946986141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6813","nonce":2946986141}]},"public_addr":"172.21.15.118:6809/2946986141","cluster_addr":"172.21.15.118:6811/2946986141","heartbeat_back_addr":"172.21.15.118:6815/2946986141","heartbeat_front_addr":"172.21.15.118:6813/2946986141","state":["exists","up"]},{"osd":7,"uuid":"4ed4c624-c88f-4d35-bdb1-bf209b5dffb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6817","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6818","nonce":48122059}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6821","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6823","nonce":48122059}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6829","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6831","nonce":48122059}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6825","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6827","nonce":48122059}]},"public_addr":"172.21.15.118:6818/48122059","cluster_addr":"172.21.15.118:6823/48122059","heartbeat_back_addr":"172.21.15.118:6831/48122059","heartbeat_front_addr":"172.21.15.118:6827/48122059","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.536664+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.368324+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.271201+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.342971+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.621832+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.387110+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.632946+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.416032+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:13:32.067 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:32.063+0000 7f3639f0d640 1 -- 172.21.15.92:0/3371378811 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f361806af60 msgr2=0x7f361806d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:32.067 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:32.063+0000 7f3639f0d640 1 --2- 172.21.15.92:0/3371378811 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f361806af60 0x7f361806d420 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f3628036000 tx=0x7f362803b040 comp rx=0 tx=0).stop 2023-11-24T06:13:32.067 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:32.064+0000 7f3639f0d640 1 -- 172.21.15.92:0/3371378811 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3634071410 msgr2=0x7f36341f2b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:32.067 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:32.064+0000 7f3639f0d640 1 --2- 172.21.15.92:0/3371378811 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3634071410 0x7f36341f2b80 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f362000c4d0 tx=0x7f362000c990 comp rx=0 tx=0).stop 2023-11-24T06:13:32.067 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:32.064+0000 7f3639f0d640 1 -- 172.21.15.92:0/3371378811 shutdown_connections 2023-11-24T06:13:32.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:32.064+0000 7f3639f0d640 1 --2- 172.21.15.92:0/3371378811 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f361806af60 0x7f361806d420 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:32.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:32.064+0000 7f3639f0d640 1 --2- 172.21.15.92:0/3371378811 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3634071f00 0x7f36340c5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:32.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:32.064+0000 7f3639f0d640 1 --2- 172.21.15.92:0/3371378811 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3634071410 0x7f36341f2b80 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:32.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:32.064+0000 7f3639f0d640 1 --2- 172.21.15.92:0/3371378811 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f363406c7c0 0x7f3634136390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:32.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:32.064+0000 7f3639f0d640 1 -- 172.21.15.92:0/3371378811 >> 172.21.15.92:0/3371378811 conn(0x7f363406a800 msgr2=0x7f363415ffc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:13:32.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:32.064+0000 7f3639f0d640 1 -- 172.21.15.92:0/3371378811 shutdown_connections 2023-11-24T06:13:32.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:32.064+0000 7f3639f0d640 1 -- 172.21.15.92:0/3371378811 wait complete. 2023-11-24T06:13:32.087 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T06:10:17.733098+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T06:13:06.409727+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '62', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T06:13:07.623678+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '63', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T06:13:32.087 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T06:13:32.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:32.304+0000 7fcad0ce7640 1 Processor -- start 2023-11-24T06:13:32.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:32.304+0000 7fcad0ce7640 1 -- start start 2023-11-24T06:13:32.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:32.305+0000 7fcad0ce7640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fcacc13df80 0x7fcacc15bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:32.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:32.305+0000 7fcad0ce7640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fcacc13e360 0x7fcacc0cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:32.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:32.305+0000 7fcad0ce7640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fcacc15f9c0 0x7fcacc0cf7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:32.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:32.305+0000 7fcad0ce7640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcacc06c470 con 0x7fcacc13e360 2023-11-24T06:13:32.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:32.305+0000 7fcad0ce7640 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcacc06c750 con 0x7fcacc13df80 2023-11-24T06:13:32.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:32.305+0000 7fcad0ce7640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcacc06ca30 con 0x7fcacc15f9c0 2023-11-24T06:13:32.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:32.306+0000 7fcac3fff640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fcacc13e360 0x7fcacc0cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:32.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:32.306+0000 7fcac3fff640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fcacc13e360 0x7fcacc0cf290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:33564/0 (socket says 172.21.15.92:33564) 2023-11-24T06:13:32.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:32.306+0000 7fcac3fff640 1 -- 172.21.15.92:0/2133621070 learned_addr learned my addr 172.21.15.92:0/2133621070 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:13:32.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:32.306+0000 7fcaca59c640 1 --2- 172.21.15.92:0/2133621070 >> v2:172.21.15.118:3300/0 conn(0x7fcacc13df80 0x7fcacc15bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:32.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:32.306+0000 7fcac3fff640 1 -- 172.21.15.92:0/2133621070 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fcacc15f9c0 msgr2=0x7fcacc0cf7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:13:32.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:32.306+0000 7fcac3fff640 1 --2- 172.21.15.92:0/2133621070 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fcacc15f9c0 0x7fcacc0cf7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:32.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:32.306+0000 7fcac3fff640 1 -- 172.21.15.92:0/2133621070 >> v2:172.21.15.118:3300/0 conn(0x7fcacc13df80 msgr2=0x7fcacc15bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:13:32.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:32.306+0000 7fcac3fff640 1 --2- 172.21.15.92:0/2133621070 >> v2:172.21.15.118:3300/0 conn(0x7fcacc13df80 0x7fcacc15bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:32.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:32.306+0000 7fcac3fff640 1 -- 172.21.15.92:0/2133621070 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcacc06cd10 con 0x7fcacc13e360 2023-11-24T06:13:32.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:32.306+0000 7fcaca59c640 1 --2- 172.21.15.92:0/2133621070 >> v2:172.21.15.118:3300/0 conn(0x7fcacc13df80 0x7fcacc15bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:13:32.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:32.307+0000 7fcac3fff640 1 --2- 172.21.15.92:0/2133621070 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcacc13e360 0x7fcacc0cf290 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7fcacc072770 tx=0x7fcab402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e41b085fc535213 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:32.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:32.308+0000 7fcac9d9b640 1 -- 172.21.15.92:0/2133621070 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcab4031870 con 0x7fcacc13e360 2023-11-24T06:13:32.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:32.308+0000 7fcac9d9b640 1 -- 172.21.15.92:0/2133621070 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcab40319d0 con 0x7fcacc13e360 2023-11-24T06:13:32.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:32.308+0000 7fcac9d9b640 1 -- 172.21.15.92:0/2133621070 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcab4031d60 con 0x7fcacc13e360 2023-11-24T06:13:32.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:32.308+0000 7fcad0ce7640 1 -- 172.21.15.92:0/2133621070 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcacc13e360 msgr2=0x7fcacc0cf290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:32.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:32.308+0000 7fcad0ce7640 1 --2- 172.21.15.92:0/2133621070 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcacc13e360 0x7fcacc0cf290 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7fcacc072770 tx=0x7fcab402b910 comp rx=0 tx=0).stop 2023-11-24T06:13:32.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:32.308+0000 7fcad0ce7640 1 -- 172.21.15.92:0/2133621070 shutdown_connections 2023-11-24T06:13:32.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:32.308+0000 7fcad0ce7640 1 --2- 172.21.15.92:0/2133621070 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fcacc15f9c0 0x7fcacc0cf7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:32.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:32.308+0000 7fcad0ce7640 1 --2- 172.21.15.92:0/2133621070 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcacc13e360 0x7fcacc0cf290 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:32.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:32.308+0000 7fcad0ce7640 1 --2- 172.21.15.92:0/2133621070 >> v2:172.21.15.118:3300/0 conn(0x7fcacc13df80 0x7fcacc15bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:32.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:32.308+0000 7fcad0ce7640 1 -- 172.21.15.92:0/2133621070 >> 172.21.15.92:0/2133621070 conn(0x7fcacc06a800 msgr2=0x7fcacc06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:13:32.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:32.308+0000 7fcad0ce7640 1 -- 172.21.15.92:0/2133621070 shutdown_connections 2023-11-24T06:13:32.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:32.309+0000 7fcad0ce7640 1 -- 172.21.15.92:0/2133621070 wait complete. 2023-11-24T06:13:32.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:32.310+0000 7fcad0ce7640 1 Processor -- start 2023-11-24T06:13:32.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:32.310+0000 7fcad0ce7640 1 -- start start 2023-11-24T06:13:32.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:32.311+0000 7fcad0ce7640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcacc13df80 0x7fcacc13ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:32.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:32.311+0000 7fcad0ce7640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fcacc13e360 0x7fcacc154770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:32.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:32.311+0000 7fcad0ce7640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fcacc15f9c0 0x7fcacc0b33e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:32.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:32.311+0000 7fcad0ce7640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcacc0b2f50 con 0x7fcacc13df80 2023-11-24T06:13:32.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:32.311+0000 7fcad0ce7640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcacc0b2680 con 0x7fcacc13e360 2023-11-24T06:13:32.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:32.311+0000 7fcad0ce7640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcacc0ceb00 con 0x7fcacc15f9c0 2023-11-24T06:13:32.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:32.311+0000 7fcaca59c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcacc13df80 0x7fcacc13ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:32.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:32.312+0000 7fcaca59c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcacc13df80 0x7fcacc13ced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:33572/0 (socket says 172.21.15.92:33572) 2023-11-24T06:13:32.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:32.312+0000 7fcaca59c640 1 -- 172.21.15.92:0/1086260825 learned_addr learned my addr 172.21.15.92:0/1086260825 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:13:32.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:32.312+0000 7fcaca59c640 1 -- 172.21.15.92:0/1086260825 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fcacc15f9c0 msgr2=0x7fcacc0b33e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:32.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:32.312+0000 7fcaca59c640 1 --2- 172.21.15.92:0/1086260825 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fcacc15f9c0 0x7fcacc0b33e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:32.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:32.312+0000 7fcaca59c640 1 -- 172.21.15.92:0/1086260825 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fcacc13e360 msgr2=0x7fcacc154770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:32.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:32.312+0000 7fcaca59c640 1 --2- 172.21.15.92:0/1086260825 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fcacc13e360 0x7fcacc154770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:32.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:32.312+0000 7fcaca59c640 1 -- 172.21.15.92:0/1086260825 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcab40094d0 con 0x7fcacc13df80 2023-11-24T06:13:32.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:32.312+0000 7fcaca59c640 1 --2- 172.21.15.92:0/1086260825 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcacc13df80 0x7fcacc13ced0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fcabc00c7f0 tx=0x7fcabc00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:32.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:32.312+0000 7fcac2ffd640 1 -- 172.21.15.92:0/1086260825 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcabc007820 con 0x7fcacc13df80 2023-11-24T06:13:32.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:32.312+0000 7fcac2ffd640 1 -- 172.21.15.92:0/1086260825 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcabc007980 con 0x7fcacc13df80 2023-11-24T06:13:32.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:32.313+0000 7fcad0ce7640 1 -- 172.21.15.92:0/1086260825 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcacc0b2900 con 0x7fcacc13df80 2023-11-24T06:13:32.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:32.313+0000 7fcac2ffd640 1 -- 172.21.15.92:0/1086260825 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcabc007cf0 con 0x7fcacc13df80 2023-11-24T06:13:32.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:32.313+0000 7fcad0ce7640 1 -- 172.21.15.92:0/1086260825 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcacc0ce740 con 0x7fcacc13df80 2023-11-24T06:13:32.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:32.314+0000 7fcad0ce7640 1 -- 172.21.15.92:0/1086260825 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcacc0675d0 con 0x7fcacc13df80 2023-11-24T06:13:32.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:32.314+0000 7fcac2ffd640 1 -- 172.21.15.92:0/1086260825 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 7) v1 ==== 89333+0+0 (secure 0 0 0) 0x7fcabc0113f0 con 0x7fcacc13df80 2023-11-24T06:13:32.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:32.315+0000 7fcac2ffd640 1 --2- 172.21.15.92:0/1086260825 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fcaac06af60 0x7fcaac06d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:32.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:32.316+0000 7fcac2ffd640 1 -- 172.21.15.92:0/1086260825 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcabc089bc0 con 0x7fcacc13df80 2023-11-24T06:13:32.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:32.316+0000 7fcac3fff640 1 --2- 172.21.15.92:0/1086260825 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fcaac06af60 0x7fcaac06d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:32.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:32.316+0000 7fcac3fff640 1 --2- 172.21.15.92:0/1086260825 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fcaac06af60 0x7fcaac06d420 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fcab4036000 tx=0x7fcab4030000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:32.323 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:32.320+0000 7fcac2ffd640 1 -- 172.21.15.92:0/1086260825 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fcabc054a40 con 0x7fcacc13df80 2023-11-24T06:13:32.515 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:32.512+0000 7fcad0ce7640 1 -- 172.21.15.92:0/1086260825 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fcacc06df00 con 0x7fcacc13df80 2023-11-24T06:13:32.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:32.513+0000 7fcac2ffd640 1 -- 172.21.15.92:0/1086260825 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v63) v1 ==== 93+0+10 (secure 0 0 0) 0x7fcabc01f0a0 con 0x7fcacc13df80 2023-11-24T06:13:32.516 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 1 2023-11-24T06:13:32.519 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:32.516+0000 7fcad0ce7640 1 -- 172.21.15.92:0/1086260825 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fcaac06af60 msgr2=0x7fcaac06d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:32.519 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:32.516+0000 7fcad0ce7640 1 --2- 172.21.15.92:0/1086260825 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fcaac06af60 0x7fcaac06d420 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fcab4036000 tx=0x7fcab4030000 comp rx=0 tx=0).stop 2023-11-24T06:13:32.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:32.516+0000 7fcad0ce7640 1 -- 172.21.15.92:0/1086260825 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcacc13df80 msgr2=0x7fcacc13ced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:32.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:32.516+0000 7fcad0ce7640 1 --2- 172.21.15.92:0/1086260825 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcacc13df80 0x7fcacc13ced0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fcabc00c7f0 tx=0x7fcabc00ccb0 comp rx=0 tx=0).stop 2023-11-24T06:13:32.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:32.516+0000 7fcad0ce7640 1 -- 172.21.15.92:0/1086260825 shutdown_connections 2023-11-24T06:13:32.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:32.516+0000 7fcad0ce7640 1 --2- 172.21.15.92:0/1086260825 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fcaac06af60 0x7fcaac06d420 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:32.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:32.516+0000 7fcad0ce7640 1 --2- 172.21.15.92:0/1086260825 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fcacc15f9c0 0x7fcacc0b33e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:32.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:32.516+0000 7fcad0ce7640 1 --2- 172.21.15.92:0/1086260825 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fcacc13e360 0x7fcacc154770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:32.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:32.516+0000 7fcad0ce7640 1 --2- 172.21.15.92:0/1086260825 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcacc13df80 0x7fcacc13ced0 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:32.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:32.516+0000 7fcad0ce7640 1 -- 172.21.15.92:0/1086260825 >> 172.21.15.92:0/1086260825 conn(0x7fcacc06a800 msgr2=0x7fcacc0afd80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:13:32.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:32.516+0000 7fcad0ce7640 1 -- 172.21.15.92:0/1086260825 shutdown_connections 2023-11-24T06:13:32.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:32.517+0000 7fcad0ce7640 1 -- 172.21.15.92:0/1086260825 wait complete. 2023-11-24T06:13:32.537 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T06:13:32.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:32.752+0000 7fdd43fff640 1 Processor -- start 2023-11-24T06:13:32.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:32.752+0000 7fdd43fff640 1 -- start start 2023-11-24T06:13:32.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:32.754+0000 7fdd43fff640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fdd440ccb50 0x7fdd44070a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:32.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:32.754+0000 7fdd43fff640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd4415f4e0 con 0x7fdd4415bf10 2023-11-24T06:13:32.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:32.754+0000 7fdd43fff640 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd4415c3c0 con 0x7fdd440adf20 2023-11-24T06:13:32.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:32.754+0000 7fdd43fff640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd4415f970 con 0x7fdd440ccb50 2023-11-24T06:13:32.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:32.754+0000 7fdd437fe640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fdd4415bf10 0x7fdd44143d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:33558/0 (socket says 172.21.15.92:33558) 2023-11-24T06:13:32.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:32.754+0000 7fdd437fe640 1 -- 172.21.15.92:0/2113765226 learned_addr learned my addr 172.21.15.92:0/2113765226 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-24T06:13:32.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:32.754+0000 7fdd3bfff640 1 --2- 172.21.15.92:0/2113765226 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fdd440ccb50 0x7fdd44070a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:32.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:32.755+0000 7fdd427fc640 1 -- 172.21.15.92:0/2113765226 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1254146239 0 0) 0x7fdd4415f4e0 con 0x7fdd4415bf10 2023-11-24T06:13:32.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:32.755+0000 7fdd427fc640 1 -- 172.21.15.92:0/2113765226 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdd30003520 con 0x7fdd4415bf10 2023-11-24T06:13:32.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:32.755+0000 7fdd427fc640 1 -- 172.21.15.92:0/2113765226 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 517410913 0 0) 0x7fdd30003520 con 0x7fdd4415bf10 2023-11-24T06:13:32.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:32.755+0000 7fdd427fc640 1 -- 172.21.15.92:0/2113765226 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fdd440ccb50 msgr2=0x7fdd44070a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:13:32.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:32.755+0000 7fdd427fc640 1 --2- 172.21.15.92:0/2113765226 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fdd440ccb50 0x7fdd44070a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:32.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:32.755+0000 7fdd427fc640 1 -- 172.21.15.92:0/2113765226 >> v1:172.21.15.118:6789/0 conn(0x7fdd440adf20 legacy=0x7fdd440cc440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:32.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:32.755+0000 7fdd427fc640 1 -- 172.21.15.92:0/2113765226 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd44071130 con 0x7fdd4415bf10 2023-11-24T06:13:32.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:32.755+0000 7fdd427fc640 1 -- 172.21.15.92:0/2113765226 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2607661350 0 0) 0x7fdd3c0019f0 con 0x7fdd4415bf10 2023-11-24T06:13:32.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:32.755+0000 7fdd427fc640 1 -- 172.21.15.92:0/2113765226 >> v1:172.21.15.92:6789/0 conn(0x7fdd4415bf10 legacy=0x7fdd44143d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:32.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:32.755+0000 7fdd427fc640 1 --2- 172.21.15.92:0/2113765226 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fdd440ccb50 0x7fdd300039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:32.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:32.755+0000 7fdd427fc640 1 --2- 172.21.15.92:0/2113765226 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdd30003f70 0x7fdd300047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:32.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:32.755+0000 7fdd427fc640 1 --2- 172.21.15.92:0/2113765226 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fdd30004d00 0x7fdd30005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:32.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:32.755+0000 7fdd427fc640 1 -- 172.21.15.92:0/2113765226 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd4415f970 con 0x7fdd30003f70 2023-11-24T06:13:32.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:32.755+0000 7fdd427fc640 1 -- 172.21.15.92:0/2113765226 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd3000bb70 con 0x7fdd30004d00 2023-11-24T06:13:32.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:32.755+0000 7fdd427fc640 1 -- 172.21.15.92:0/2113765226 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd3000bd30 con 0x7fdd440ccb50 2023-11-24T06:13:32.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:32.756+0000 7fdd3bfff640 1 --2- 172.21.15.92:0/2113765226 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdd30003f70 0x7fdd300047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:32.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:32.756+0000 7fdd3bfff640 1 -- 172.21.15.92:0/2113765226 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fdd440ccb50 msgr2=0x7fdd300039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:13:32.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:32.756+0000 7fdd3bfff640 1 --2- 172.21.15.92:0/2113765226 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fdd440ccb50 0x7fdd300039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:32.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:32.756+0000 7fdd3bfff640 1 -- 172.21.15.92:0/2113765226 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fdd30004d00 msgr2=0x7fdd30005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:13:32.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:32.756+0000 7fdd3bfff640 1 --2- 172.21.15.92:0/2113765226 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fdd30004d00 0x7fdd30005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:32.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:32.756+0000 7fdd3bfff640 1 -- 172.21.15.92:0/2113765226 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd30003520 con 0x7fdd30003f70 2023-11-24T06:13:32.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:32.757+0000 7fdd3bfff640 1 --2- 172.21.15.92:0/2113765226 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdd30003f70 0x7fdd300047c0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7fdd2c009a90 tx=0x7fdd300057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ea4d7f8962bee99 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:32.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:32.757+0000 7fdd427fc640 1 -- 172.21.15.92:0/2113765226 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd2c02c6b0 con 0x7fdd30003f70 2023-11-24T06:13:32.766 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:32.757+0000 7fdd427fc640 1 -- 172.21.15.92:0/2113765226 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd2c02c810 con 0x7fdd30003f70 2023-11-24T06:13:32.766 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:32.757+0000 7fdd427fc640 1 -- 172.21.15.92:0/2113765226 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd2c02cba0 con 0x7fdd30003f70 2023-11-24T06:13:32.766 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:32.758+0000 7fdd43fff640 1 -- 172.21.15.92:0/2113765226 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdd30003f70 msgr2=0x7fdd300047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:32.766 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:32.758+0000 7fdd43fff640 1 --2- 172.21.15.92:0/2113765226 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdd30003f70 0x7fdd300047c0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7fdd2c009a90 tx=0x7fdd300057f0 comp rx=0 tx=0).stop 2023-11-24T06:13:32.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:32.758+0000 7fdd437fe640 1 -- 172.21.15.92:0/2113765226 reap_dead start 2023-11-24T06:13:32.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:32.758+0000 7fdd43fff640 1 -- 172.21.15.92:0/2113765226 shutdown_connections 2023-11-24T06:13:32.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:32.758+0000 7fdd43fff640 1 -- 172.21.15.92:0/2113765226 >> 172.21.15.92:0/2113765226 conn(0x7fdd4406a7d0 msgr2=0x7fdd4406abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:13:32.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:32.758+0000 7fdd43fff640 1 -- 172.21.15.92:0/2113765226 shutdown_connections 2023-11-24T06:13:32.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:32.758+0000 7fdd43fff640 1 -- 172.21.15.92:0/2113765226 wait complete. 2023-11-24T06:13:32.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:32.760+0000 7fdd43fff640 1 Processor -- start 2023-11-24T06:13:32.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:32.760+0000 7fdd43fff640 1 -- start start 2023-11-24T06:13:32.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:32.761+0000 7fdd43fff640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fdd30003f70 0x7fdd44141ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:32.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:32.761+0000 7fdd43fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdd44133f60 0x7fdd441369c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:32.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:32.761+0000 7fdd43fff640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fdd441354e0 0x7fdd44135990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:32.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:32.761+0000 7fdd43fff640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd44142420 con 0x7fdd44133f60 2023-11-24T06:13:32.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:32.761+0000 7fdd43fff640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd440c6e20 con 0x7fdd30003f70 2023-11-24T06:13:32.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:32.761+0000 7fdd43fff640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd44134440 con 0x7fdd441354e0 2023-11-24T06:13:32.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:32.761+0000 7fdd3bfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdd44133f60 0x7fdd441369c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:32.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:32.761+0000 7fdd3bfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdd44133f60 0x7fdd441369c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:33592/0 (socket says 172.21.15.92:33592) 2023-11-24T06:13:32.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:32.761+0000 7fdd3bfff640 1 -- 172.21.15.92:0/745498889 learned_addr learned my addr 172.21.15.92:0/745498889 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:13:32.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:32.761+0000 7fdd42ffd640 1 --2- 172.21.15.92:0/745498889 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fdd30003f70 0x7fdd44141ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:32.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:32.761+0000 7fdd437fe640 1 --2- 172.21.15.92:0/745498889 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fdd441354e0 0x7fdd44135990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:32.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:32.761+0000 7fdd3bfff640 1 -- 172.21.15.92:0/745498889 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fdd441354e0 msgr2=0x7fdd44135990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:32.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:32.761+0000 7fdd3bfff640 1 --2- 172.21.15.92:0/745498889 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fdd441354e0 0x7fdd44135990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:32.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:32.761+0000 7fdd3bfff640 1 -- 172.21.15.92:0/745498889 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fdd30003f70 msgr2=0x7fdd44141ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:32.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:32.761+0000 7fdd3bfff640 1 --2- 172.21.15.92:0/745498889 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fdd30003f70 0x7fdd44141ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:32.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:32.761+0000 7fdd3bfff640 1 -- 172.21.15.92:0/745498889 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd44134720 con 0x7fdd44133f60 2023-11-24T06:13:32.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:32.761+0000 7fdd437fe640 1 --2- 172.21.15.92:0/745498889 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fdd441354e0 0x7fdd44135990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:13:32.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:32.762+0000 7fdd3bfff640 1 --2- 172.21.15.92:0/745498889 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdd44133f60 0x7fdd441369c0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7fdd2c02c250 tx=0x7fdd2c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:32.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:32.762+0000 7fdd3b7fe640 1 -- 172.21.15.92:0/745498889 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd2c003810 con 0x7fdd44133f60 2023-11-24T06:13:32.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:32.762+0000 7fdd3b7fe640 1 -- 172.21.15.92:0/745498889 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd2c003970 con 0x7fdd44133f60 2023-11-24T06:13:32.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:32.762+0000 7fdd3b7fe640 1 -- 172.21.15.92:0/745498889 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd2c003d00 con 0x7fdd44133f60 2023-11-24T06:13:32.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:32.762+0000 7fdd43fff640 1 -- 172.21.15.92:0/745498889 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd44134a00 con 0x7fdd44133f60 2023-11-24T06:13:32.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:32.762+0000 7fdd43fff640 1 -- 172.21.15.92:0/745498889 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd44134fc0 con 0x7fdd44133f60 2023-11-24T06:13:32.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:32.763+0000 7fdd43fff640 1 -- 172.21.15.92:0/745498889 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd44135150 con 0x7fdd44133f60 2023-11-24T06:13:32.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:32.764+0000 7fdd3b7fe640 1 -- 172.21.15.92:0/745498889 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 7) v1 ==== 89333+0+0 (secure 0 0 0) 0x7fdd2c034440 con 0x7fdd44133f60 2023-11-24T06:13:32.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:32.765+0000 7fdd3b7fe640 1 --2- 172.21.15.92:0/745498889 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fdd2406af90 0x7fdd2406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:32.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:32.765+0000 7fdd3b7fe640 1 -- 172.21.15.92:0/745498889 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdd2c0ab6d0 con 0x7fdd44133f60 2023-11-24T06:13:32.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:32.765+0000 7fdd42ffd640 1 --2- 172.21.15.92:0/745498889 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fdd2406af90 0x7fdd2406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:32.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:32.766+0000 7fdd42ffd640 1 --2- 172.21.15.92:0/745498889 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fdd2406af90 0x7fdd2406d450 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fdd340016f0 tx=0x7fdd34009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:32.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:32.770+0000 7fdd3b7fe640 1 -- 172.21.15.92:0/745498889 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fdd2c076720 con 0x7fdd44133f60 2023-11-24T06:13:32.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:32.961+0000 7fdd43fff640 1 -- 172.21.15.92:0/745498889 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fdd44067450 con 0x7fdd44133f60 2023-11-24T06:13:32.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:32.962+0000 7fdd3b7fe640 1 -- 172.21.15.92:0/745498889 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v63) v1 ==== 104+0+11 (secure 0 0 0) 0x7fdd2c079d40 con 0x7fdd44133f60 2023-11-24T06:13:32.965 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 32 2023-11-24T06:13:32.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:32.965+0000 7fdd43fff640 1 -- 172.21.15.92:0/745498889 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fdd2406af90 msgr2=0x7fdd2406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:32.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:32.965+0000 7fdd43fff640 1 --2- 172.21.15.92:0/745498889 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fdd2406af90 0x7fdd2406d450 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fdd340016f0 tx=0x7fdd34009040 comp rx=0 tx=0).stop 2023-11-24T06:13:32.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:32.965+0000 7fdd43fff640 1 -- 172.21.15.92:0/745498889 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdd44133f60 msgr2=0x7fdd441369c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:32.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:32.965+0000 7fdd43fff640 1 --2- 172.21.15.92:0/745498889 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdd44133f60 0x7fdd441369c0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7fdd2c02c250 tx=0x7fdd2c004730 comp rx=0 tx=0).stop 2023-11-24T06:13:32.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:32.965+0000 7fdd43fff640 1 -- 172.21.15.92:0/745498889 shutdown_connections 2023-11-24T06:13:32.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:32.965+0000 7fdd43fff640 1 --2- 172.21.15.92:0/745498889 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fdd2406af90 0x7fdd2406d450 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:32.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:32.965+0000 7fdd43fff640 1 --2- 172.21.15.92:0/745498889 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fdd441354e0 0x7fdd44135990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:32.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:32.965+0000 7fdd43fff640 1 --2- 172.21.15.92:0/745498889 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdd44133f60 0x7fdd441369c0 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:32.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:32.965+0000 7fdd43fff640 1 --2- 172.21.15.92:0/745498889 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fdd30003f70 0x7fdd44141ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:32.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:32.965+0000 7fdd43fff640 1 -- 172.21.15.92:0/745498889 >> 172.21.15.92:0/745498889 conn(0x7fdd4406a7d0 msgr2=0x7fdd4415b710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:13:32.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:32.965+0000 7fdd43fff640 1 -- 172.21.15.92:0/745498889 shutdown_connections 2023-11-24T06:13:32.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:32.965+0000 7fdd43fff640 1 -- 172.21.15.92:0/745498889 wait complete. 2023-11-24T06:13:32.987 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T06:13:33.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:33.190+0000 7f1590dc9640 1 Processor -- start 2023-11-24T06:13:33.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:33.190+0000 7f1590dc9640 1 -- start start 2023-11-24T06:13:33.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:33.191+0000 7f1590dc9640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f158c143ae0 0x7f158c0cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:33.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:33.191+0000 7f1590dc9640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f158c0cf7d0 0x7f158c15dca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:33.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:33.191+0000 7f1590dc9640 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f158c0cfc30 con 0x7f158c15e1e0 2023-11-24T06:13:33.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:33.191+0000 7f1590dc9640 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f158c0b1ee0 con 0x7f158c0cf7d0 2023-11-24T06:13:33.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:33.191+0000 7f1590dc9640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f158c0b2780 con 0x7f158c143ae0 2023-11-24T06:13:33.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:33.192+0000 7f158a59c640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f158c143ae0 0x7f158c0cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:33.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:33.192+0000 7f1583fff640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f158c0cf7d0 0x7f158c15dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:33.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:33.192+0000 7f158a59c640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f158c143ae0 0x7f158c0cf290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3301/0 says I am v2:172.21.15.92:41116/0 (socket says 172.21.15.92:41116) 2023-11-24T06:13:33.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:33.192+0000 7f158a59c640 1 -- 172.21.15.92:0/1550507355 learned_addr learned my addr 172.21.15.92:0/1550507355 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:13:33.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:33.192+0000 7f158ad9d640 1 --1- 172.21.15.92:0/1550507355 >> v1:172.21.15.118:6789/0 conn(0x7f158c15e1e0 0x7f158c15a670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.118:6789/0 says I am v1:172.21.15.92:41100/0 (socket says 172.21.15.92:41100) 2023-11-24T06:13:33.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:33.193+0000 7f158a59c640 1 -- 172.21.15.92:0/1550507355 >> v2:172.21.15.118:3300/0 conn(0x7f158c0cf7d0 msgr2=0x7f158c15dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:13:33.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:33.193+0000 7f158a59c640 1 --2- 172.21.15.92:0/1550507355 >> v2:172.21.15.118:3300/0 conn(0x7f158c0cf7d0 0x7f158c15dca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:33.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:33.193+0000 7f158a59c640 1 -- 172.21.15.92:0/1550507355 >> v1:172.21.15.118:6789/0 conn(0x7f158c15e1e0 legacy=0x7f158c15a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:33.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:33.193+0000 7f158a59c640 1 -- 172.21.15.92:0/1550507355 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f158c15e990 con 0x7f158c143ae0 2023-11-24T06:13:33.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:33.193+0000 7f1583fff640 1 --2- 172.21.15.92:0/1550507355 >> v2:172.21.15.118:3300/0 conn(0x7f158c0cf7d0 0x7f158c15dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:13:33.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:33.195+0000 7f158a59c640 1 --2- 172.21.15.92:0/1550507355 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f158c143ae0 0x7f158c0cf290 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f157c009870 tx=0x7f157c02b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a580bc53b797e197 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:33.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:33.195+0000 7f1589d9b640 1 -- 172.21.15.92:0/1550507355 <== mon.2 v2:172.21.15.118:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f157c030840 con 0x7f158c143ae0 2023-11-24T06:13:33.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:33.195+0000 7f1589d9b640 1 -- 172.21.15.92:0/1550507355 <== mon.2 v2:172.21.15.118:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f157c0309a0 con 0x7f158c143ae0 2023-11-24T06:13:33.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:33.195+0000 7f1589d9b640 1 -- 172.21.15.92:0/1550507355 <== mon.2 v2:172.21.15.118:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f157c030d30 con 0x7f158c143ae0 2023-11-24T06:13:33.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:33.196+0000 7f1590dc9640 1 -- 172.21.15.92:0/1550507355 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f158c143ae0 msgr2=0x7f158c0cf290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:33.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:33.196+0000 7f1590dc9640 1 --2- 172.21.15.92:0/1550507355 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f158c143ae0 0x7f158c0cf290 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f157c009870 tx=0x7f157c02b980 comp rx=0 tx=0).stop 2023-11-24T06:13:33.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:33.196+0000 7f1590dc9640 1 -- 172.21.15.92:0/1550507355 shutdown_connections 2023-11-24T06:13:33.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:33.196+0000 7f1590dc9640 1 --2- 172.21.15.92:0/1550507355 >> v2:172.21.15.118:3300/0 conn(0x7f158c0cf7d0 0x7f158c15dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:33.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:33.196+0000 7f1590dc9640 1 --2- 172.21.15.92:0/1550507355 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f158c143ae0 0x7f158c0cf290 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:33.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:33.196+0000 7f1590dc9640 1 -- 172.21.15.92:0/1550507355 >> 172.21.15.92:0/1550507355 conn(0x7f158c06a800 msgr2=0x7f158c06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:13:33.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:33.196+0000 7f1590dc9640 1 -- 172.21.15.92:0/1550507355 shutdown_connections 2023-11-24T06:13:33.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:33.196+0000 7f1590dc9640 1 -- 172.21.15.92:0/1550507355 wait complete. 2023-11-24T06:13:33.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:33.197+0000 7f1590dc9640 1 Processor -- start 2023-11-24T06:13:33.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:33.198+0000 7f1590dc9640 1 -- start start 2023-11-24T06:13:33.203 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:33.198+0000 7f1590dc9640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f158c0cf7d0 0x7f158c159a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:33.203 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:33.198+0000 7f1590dc9640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f158c143ae0 0x7f158c0afa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:33.203 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:33.199+0000 7f1590dc9640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f158c15e1e0 0x7f158c0affb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:33.203 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:33.199+0000 7f1590dc9640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f158c13bcc0 con 0x7f158c15e1e0 2023-11-24T06:13:33.203 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:33.199+0000 7f1590dc9640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f158c13bfa0 con 0x7f158c143ae0 2023-11-24T06:13:33.203 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:33.199+0000 7f1590dc9640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f158c13c280 con 0x7f158c0cf7d0 2023-11-24T06:13:33.203 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:33.199+0000 7f158ad9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f158c15e1e0 0x7f158c0affb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:33.203 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:33.199+0000 7f158ad9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f158c15e1e0 0x7f158c0affb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:33608/0 (socket says 172.21.15.92:33608) 2023-11-24T06:13:33.204 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:33.199+0000 7f158ad9d640 1 -- 172.21.15.92:0/2639010926 learned_addr learned my addr 172.21.15.92:0/2639010926 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:13:33.204 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:33.199+0000 7f158a59c640 1 --2- 172.21.15.92:0/2639010926 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f158c0cf7d0 0x7f158c159a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:33.204 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:33.199+0000 7f1583fff640 1 --2- 172.21.15.92:0/2639010926 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f158c143ae0 0x7f158c0afa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:33.204 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:33.199+0000 7f158ad9d640 1 -- 172.21.15.92:0/2639010926 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f158c0cf7d0 msgr2=0x7f158c159a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:33.204 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:33.199+0000 7f158ad9d640 1 --2- 172.21.15.92:0/2639010926 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f158c0cf7d0 0x7f158c159a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:33.204 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:33.199+0000 7f158ad9d640 1 -- 172.21.15.92:0/2639010926 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f158c143ae0 msgr2=0x7f158c0afa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:33.204 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:33.199+0000 7f158ad9d640 1 --2- 172.21.15.92:0/2639010926 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f158c143ae0 0x7f158c0afa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:33.204 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:33.199+0000 7f158ad9d640 1 -- 172.21.15.92:0/2639010926 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1574009400 con 0x7f158c15e1e0 2023-11-24T06:13:33.204 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:33.199+0000 7f1583fff640 1 --2- 172.21.15.92:0/2639010926 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f158c143ae0 0x7f158c0afa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:13:33.204 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:33.200+0000 7f158ad9d640 1 --2- 172.21.15.92:0/2639010926 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f158c15e1e0 0x7f158c0affb0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f158400c730 tx=0x7f158400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:33.204 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:33.200+0000 7f1582ffd640 1 -- 172.21.15.92:0/2639010926 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f158400ce40 con 0x7f158c15e1e0 2023-11-24T06:13:33.204 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:33.200+0000 7f1582ffd640 1 -- 172.21.15.92:0/2639010926 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f158400a640 con 0x7f158c15e1e0 2023-11-24T06:13:33.204 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:33.200+0000 7f1582ffd640 1 -- 172.21.15.92:0/2639010926 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f158400a9f0 con 0x7f158c15e1e0 2023-11-24T06:13:33.205 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:33.200+0000 7f1590dc9640 1 -- 172.21.15.92:0/2639010926 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f157c0094d0 con 0x7f158c15e1e0 2023-11-24T06:13:33.205 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:33.200+0000 7f1590dc9640 1 -- 172.21.15.92:0/2639010926 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f158c13c840 con 0x7f158c15e1e0 2023-11-24T06:13:33.205 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:33.201+0000 7f1590dc9640 1 -- 172.21.15.92:0/2639010926 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f158c0b2bf0 con 0x7f158c15e1e0 2023-11-24T06:13:33.205 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:33.202+0000 7f1582ffd640 1 -- 172.21.15.92:0/2639010926 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 7) v1 ==== 89333+0+0 (secure 0 0 0) 0x7f158400ab50 con 0x7f158c15e1e0 2023-11-24T06:13:33.206 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:33.203+0000 7f1582ffd640 1 --2- 172.21.15.92:0/2639010926 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f156c06af60 0x7f156c06d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:33.206 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:33.203+0000 7f1582ffd640 1 -- 172.21.15.92:0/2639010926 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1584088e00 con 0x7f158c15e1e0 2023-11-24T06:13:33.206 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:33.203+0000 7f158a59c640 1 --2- 172.21.15.92:0/2639010926 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f156c06af60 0x7f156c06d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:33.206 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:33.204+0000 7f158a59c640 1 --2- 172.21.15.92:0/2639010926 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f156c06af60 0x7f156c06d420 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f157c036000 tx=0x7f157c03b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:33.210 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:33.207+0000 7f1582ffd640 1 -- 172.21.15.92:0/2639010926 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f158c0b2bf0 con 0x7f158c15e1e0 2023-11-24T06:13:33.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:33.398+0000 7f1590dc9640 1 -- 172.21.15.92:0/2639010926 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f158c0b2bf0 con 0x7f158c15e1e0 2023-11-24T06:13:33.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:33.399+0000 7f1582ffd640 1 -- 172.21.15.92:0/2639010926 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v63) v1 ==== 100+0+11 (secure 0 0 0) 0x7f158c0b2bf0 con 0x7f158c15e1e0 2023-11-24T06:13:33.402 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 64 2023-11-24T06:13:33.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:33.401+0000 7f1590dc9640 1 -- 172.21.15.92:0/2639010926 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f156c06af60 msgr2=0x7f156c06d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:33.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:33.401+0000 7f1590dc9640 1 --2- 172.21.15.92:0/2639010926 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f156c06af60 0x7f156c06d420 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f157c036000 tx=0x7f157c03b040 comp rx=0 tx=0).stop 2023-11-24T06:13:33.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:33.401+0000 7f1590dc9640 1 -- 172.21.15.92:0/2639010926 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f158c15e1e0 msgr2=0x7f158c0affb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:33.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:33.401+0000 7f1590dc9640 1 --2- 172.21.15.92:0/2639010926 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f158c15e1e0 0x7f158c0affb0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f158400c730 tx=0x7f158400cbf0 comp rx=0 tx=0).stop 2023-11-24T06:13:33.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:33.402+0000 7f1590dc9640 1 -- 172.21.15.92:0/2639010926 shutdown_connections 2023-11-24T06:13:33.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:33.402+0000 7f1590dc9640 1 --2- 172.21.15.92:0/2639010926 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f156c06af60 0x7f156c06d420 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:33.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:33.402+0000 7f1590dc9640 1 --2- 172.21.15.92:0/2639010926 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f158c15e1e0 0x7f158c0affb0 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:33.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:33.402+0000 7f1590dc9640 1 --2- 172.21.15.92:0/2639010926 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f158c143ae0 0x7f158c0afa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:33.406 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:33.402+0000 7f1590dc9640 1 --2- 172.21.15.92:0/2639010926 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f158c0cf7d0 0x7f158c159a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:33.406 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:33.402+0000 7f1590dc9640 1 -- 172.21.15.92:0/2639010926 >> 172.21.15.92:0/2639010926 conn(0x7f158c06a800 msgr2=0x7f158c15a4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:13:33.406 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:33.402+0000 7f1590dc9640 1 -- 172.21.15.92:0/2639010926 shutdown_connections 2023-11-24T06:13:33.406 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:33.402+0000 7f1590dc9640 1 -- 172.21.15.92:0/2639010926 wait complete. 2023-11-24T06:13:33.424 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T06:13:33.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:33.627+0000 7f7789272640 1 Processor -- start 2023-11-24T06:13:33.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:33.627+0000 7f7789272640 1 -- start start 2023-11-24T06:13:33.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:33.628+0000 7f7789272640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f778413cfa0 0x7f778415afb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:33.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:33.629+0000 7f7789272640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77840cdec0 con 0x7f77840cd530 2023-11-24T06:13:33.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:33.629+0000 7f7789272640 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77840ce1a0 con 0x7f778413fcb0 2023-11-24T06:13:33.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:33.629+0000 7f7789272640 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77840ce480 con 0x7f778413cfa0 2023-11-24T06:13:33.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:33.629+0000 7f777bfff640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f77840cd530 0x7f778413f5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:33572/0 (socket says 172.21.15.92:33572) 2023-11-24T06:13:33.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:33.629+0000 7f777bfff640 1 -- 172.21.15.92:0/1887693916 learned_addr learned my addr 172.21.15.92:0/1887693916 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-24T06:13:33.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:33.629+0000 7f77837fe640 1 -- 172.21.15.92:0/1887693916 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 152842032 0 0) 0x7f77840cdec0 con 0x7f77840cd530 2023-11-24T06:13:33.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:33.629+0000 7f77837fe640 1 -- 172.21.15.92:0/1887693916 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7770003520 con 0x7f77840cd530 2023-11-24T06:13:33.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:33.629+0000 7f7783fff640 1 --2- 172.21.15.92:0/1887693916 >> v2:172.21.15.118:3300/0 conn(0x7f778413cfa0 0x7f778415afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:33.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:33.630+0000 7f77837fe640 1 -- 172.21.15.92:0/1887693916 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 717044042 0 0) 0x7f7770003520 con 0x7f77840cd530 2023-11-24T06:13:33.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:33.630+0000 7f77837fe640 1 -- 172.21.15.92:0/1887693916 >> v2:172.21.15.118:3300/0 conn(0x7f778413cfa0 msgr2=0x7f778415afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:13:33.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:33.630+0000 7f77837fe640 1 --2- 172.21.15.92:0/1887693916 >> v2:172.21.15.118:3300/0 conn(0x7f778413cfa0 0x7f778415afb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:33.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:33.630+0000 7f77837fe640 1 -- 172.21.15.92:0/1887693916 >> v1:172.21.15.118:6789/0 conn(0x7f778413fcb0 legacy=0x7f77840b2040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:33.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:33.630+0000 7f77837fe640 1 -- 172.21.15.92:0/1887693916 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77840ce760 con 0x7f77840cd530 2023-11-24T06:13:33.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:33.630+0000 7f77837fe640 1 -- 172.21.15.92:0/1887693916 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2607661350 0 0) 0x7f776c001670 con 0x7f77840cd530 2023-11-24T06:13:33.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:33.630+0000 7f77837fe640 1 -- 172.21.15.92:0/1887693916 >> v1:172.21.15.92:6789/0 conn(0x7f77840cd530 legacy=0x7f778413f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:33.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:33.630+0000 7f77837fe640 1 --2- 172.21.15.92:0/1887693916 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7770003970 0x7f7770003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:33.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:33.630+0000 7f77837fe640 1 --2- 172.21.15.92:0/1887693916 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f77700047c0 0x7f7770004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:33.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:33.630+0000 7f77837fe640 1 --2- 172.21.15.92:0/1887693916 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f77700052a0 0x7f777000db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:33.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:33.630+0000 7f77837fe640 1 -- 172.21.15.92:0/1887693916 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77840ce480 con 0x7f77700052a0 2023-11-24T06:13:33.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:33.630+0000 7f77837fe640 1 -- 172.21.15.92:0/1887693916 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f777000e380 con 0x7f7770003970 2023-11-24T06:13:33.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:33.630+0000 7f77837fe640 1 -- 172.21.15.92:0/1887693916 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f777000e540 con 0x7f77700047c0 2023-11-24T06:13:33.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:33.630+0000 7f7788a71640 1 --2- 172.21.15.92:0/1887693916 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f77700052a0 0x7f777000db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:33.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:33.630+0000 7f7783fff640 1 --2- 172.21.15.92:0/1887693916 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7770003970 0x7f7770003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:33.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:33.630+0000 7f777bfff640 1 --2- 172.21.15.92:0/1887693916 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f77700047c0 0x7f7770004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:33.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:33.630+0000 7f7788a71640 1 -- 172.21.15.92:0/1887693916 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f77700047c0 msgr2=0x7f7770004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:13:33.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:33.630+0000 7f7788a71640 1 --2- 172.21.15.92:0/1887693916 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f77700047c0 0x7f7770004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:33.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:33.630+0000 7f7788a71640 1 -- 172.21.15.92:0/1887693916 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7770003970 msgr2=0x7f7770003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:13:33.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:33.630+0000 7f7788a71640 1 --2- 172.21.15.92:0/1887693916 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7770003970 0x7f7770003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:33.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:33.631+0000 7f7788a71640 1 -- 172.21.15.92:0/1887693916 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7770003520 con 0x7f77700052a0 2023-11-24T06:13:33.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:33.631+0000 7f777bfff640 1 --2- 172.21.15.92:0/1887693916 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f77700047c0 0x7f7770004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:13:33.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:33.631+0000 7f7788a71640 1 -- 172.21.15.92:0/1887693916 reap_dead start 2023-11-24T06:13:33.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:33.632+0000 7f7788a71640 1 --2- 172.21.15.92:0/1887693916 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f77700052a0 0x7f777000db90 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f777c001e20 tx=0x7f77700057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47ba4bb01096410c server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:33.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:33.632+0000 7f77837fe640 1 -- 172.21.15.92:0/1887693916 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f777c003b70 con 0x7f77700052a0 2023-11-24T06:13:33.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:33.632+0000 7f77837fe640 1 -- 172.21.15.92:0/1887693916 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f777c003cd0 con 0x7f77700052a0 2023-11-24T06:13:33.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:33.632+0000 7f77837fe640 1 -- 172.21.15.92:0/1887693916 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f777c0052f0 con 0x7f77700052a0 2023-11-24T06:13:33.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:33.632+0000 7f7789272640 1 -- 172.21.15.92:0/1887693916 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f77700052a0 msgr2=0x7f777000db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:33.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:33.632+0000 7f7789272640 1 --2- 172.21.15.92:0/1887693916 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f77700052a0 0x7f777000db90 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f777c001e20 tx=0x7f77700057f0 comp rx=0 tx=0).stop 2023-11-24T06:13:33.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:33.632+0000 7f7789272640 1 -- 172.21.15.92:0/1887693916 shutdown_connections 2023-11-24T06:13:33.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:33.632+0000 7f7789272640 1 --2- 172.21.15.92:0/1887693916 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f77700052a0 0x7f777000db90 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:33.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:33.632+0000 7f7789272640 1 -- 172.21.15.92:0/1887693916 >> 172.21.15.92:0/1887693916 conn(0x7f778406a800 msgr2=0x7f778406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:13:33.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:33.633+0000 7f7789272640 1 -- 172.21.15.92:0/1887693916 shutdown_connections 2023-11-24T06:13:33.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:33.633+0000 7f7789272640 1 -- 172.21.15.92:0/1887693916 wait complete. 2023-11-24T06:13:33.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:33.634+0000 7f7789272640 1 Processor -- start 2023-11-24T06:13:33.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:33.634+0000 7f7789272640 1 -- start start 2023-11-24T06:13:33.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:33.635+0000 7f7789272640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f77700052a0 0x7f77840adea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:33.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:33.635+0000 7f7789272640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f77841394d0 0x7f778415d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:33.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:33.635+0000 7f7789272640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f77841f66e0 0x7f77841f6b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:33.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:33.635+0000 7f7789272640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f778413b700 con 0x7f77700052a0 2023-11-24T06:13:33.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:33.635+0000 7f7789272640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77841f70a0 con 0x7f77841f66e0 2023-11-24T06:13:33.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:33.635+0000 7f7789272640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f778415d730 con 0x7f77841394d0 2023-11-24T06:13:33.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:33.636+0000 7f7783fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f77700052a0 0x7f77840adea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:33.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:33.636+0000 7f7783fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f77700052a0 0x7f77840adea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:33626/0 (socket says 172.21.15.92:33626) 2023-11-24T06:13:33.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:33.636+0000 7f7783fff640 1 -- 172.21.15.92:0/2873264458 learned_addr learned my addr 172.21.15.92:0/2873264458 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:13:33.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:33.636+0000 7f777bfff640 1 --2- 172.21.15.92:0/2873264458 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f77841394d0 0x7f778415d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:33.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:33.636+0000 7f7788a71640 1 --2- 172.21.15.92:0/2873264458 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f77841f66e0 0x7f77841f6b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:33.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:33.636+0000 7f7783fff640 1 -- 172.21.15.92:0/2873264458 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f77841394d0 msgr2=0x7f778415d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:33.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:33.636+0000 7f7783fff640 1 --2- 172.21.15.92:0/2873264458 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f77841394d0 0x7f778415d1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:33.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:33.636+0000 7f7783fff640 1 -- 172.21.15.92:0/2873264458 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f77841f66e0 msgr2=0x7f77841f6b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:33.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:33.636+0000 7f7783fff640 1 --2- 172.21.15.92:0/2873264458 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f77841f66e0 0x7f77841f6b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:33.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:33.636+0000 7f7783fff640 1 -- 172.21.15.92:0/2873264458 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7784139930 con 0x7f77700052a0 2023-11-24T06:13:33.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:33.636+0000 7f7788a71640 1 --2- 172.21.15.92:0/2873264458 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f77841f66e0 0x7f77841f6b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:13:33.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:33.636+0000 7f7783fff640 1 --2- 172.21.15.92:0/2873264458 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f77700052a0 0x7f77840adea0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f777400f580 tx=0x7f777400fa40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:33.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:33.637+0000 7f77817fa640 1 -- 172.21.15.92:0/2873264458 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7774013810 con 0x7f77700052a0 2023-11-24T06:13:33.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:33.637+0000 7f77817fa640 1 -- 172.21.15.92:0/2873264458 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7774013970 con 0x7f77700052a0 2023-11-24T06:13:33.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:33.637+0000 7f7789272640 1 -- 172.21.15.92:0/2873264458 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77841e5240 con 0x7f77700052a0 2023-11-24T06:13:33.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:33.637+0000 7f77817fa640 1 -- 172.21.15.92:0/2873264458 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7774013ce0 con 0x7f77700052a0 2023-11-24T06:13:33.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:33.637+0000 7f7789272640 1 -- 172.21.15.92:0/2873264458 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77841e4f60 con 0x7f77700052a0 2023-11-24T06:13:33.644 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:33.638+0000 7f7789272640 1 -- 172.21.15.92:0/2873264458 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7784067450 con 0x7f77700052a0 2023-11-24T06:13:33.644 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:33.639+0000 7f77817fa640 1 -- 172.21.15.92:0/2873264458 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 7) v1 ==== 89333+0+0 (secure 0 0 0) 0x7f777401f460 con 0x7f77700052a0 2023-11-24T06:13:33.644 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:33.640+0000 7f77817fa640 1 --2- 172.21.15.92:0/2873264458 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f776406af60 0x7f776406d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:33.644 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:33.640+0000 7f77817fa640 1 -- 172.21.15.92:0/2873264458 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7774089b30 con 0x7f77700052a0 2023-11-24T06:13:33.644 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:33.640+0000 7f777bfff640 1 --2- 172.21.15.92:0/2873264458 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f776406af60 0x7f776406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:33.644 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:33.641+0000 7f777bfff640 1 --2- 172.21.15.92:0/2873264458 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f776406af60 0x7f776406d420 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f778406c900 tx=0x7f776c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:33.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:33.645+0000 7f77817fa640 1 -- 172.21.15.92:0/2873264458 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7774054ae0 con 0x7f77700052a0 2023-11-24T06:13:33.878 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:33.875+0000 7f7789272640 1 -- 172.21.15.92:0/2873264458 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f77840cbff0 con 0x7f77700052a0 2023-11-24T06:13:33.879 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:33.876+0000 7f77817fa640 1 -- 172.21.15.92:0/2873264458 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) v1 ==== 94+0+4921 (secure 0 0 0) 0x7f7774058100 con 0x7f77700052a0 2023-11-24T06:13:33.879 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-24T06:13:33.879 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":41,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5143,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6833/2094733752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":2094733752},{"type":"v1","addr":"172.21.15.118:6833","nonce":2094733752}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":36},{"gid":5146,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6835/2873546779","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":2873546779},{"type":"v1","addr":"172.21.15.118:6835","nonce":2873546779}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":36},{"gid":6141,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6836/3335501795","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":3335501795},{"type":"v1","addr":"172.21.15.92:6836","nonce":3335501795}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":35}],"filesystems":[{"mdsmap":{"epoch":40,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T06:13:08.625588+0000","modified":"2023-11-24T06:13:11.658333+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6144},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6144":{"gid":6144,"name":"a","rank":0,"incarnation":37,"state":"up:active","state_seq":3,"addr":"172.21.15.92:6837/3913457833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":3913457833},{"type":"v1","addr":"172.21.15.92:6837","nonce":3913457833}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-11-24T06:13:33.879 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 41 2023-11-24T06:13:33.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:33.879+0000 7f7789272640 1 -- 172.21.15.92:0/2873264458 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f776406af60 msgr2=0x7f776406d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:33.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:33.879+0000 7f7789272640 1 --2- 172.21.15.92:0/2873264458 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f776406af60 0x7f776406d420 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f778406c900 tx=0x7f776c009040 comp rx=0 tx=0).stop 2023-11-24T06:13:33.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:33.879+0000 7f7789272640 1 -- 172.21.15.92:0/2873264458 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f77700052a0 msgr2=0x7f77840adea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:33.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:33.879+0000 7f7789272640 1 --2- 172.21.15.92:0/2873264458 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f77700052a0 0x7f77840adea0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f777400f580 tx=0x7f777400fa40 comp rx=0 tx=0).stop 2023-11-24T06:13:33.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:33.879+0000 7f7789272640 1 -- 172.21.15.92:0/2873264458 shutdown_connections 2023-11-24T06:13:33.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:33.879+0000 7f7789272640 1 --2- 172.21.15.92:0/2873264458 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f776406af60 0x7f776406d420 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:33.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:33.879+0000 7f7789272640 1 --2- 172.21.15.92:0/2873264458 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f77841f66e0 0x7f77841f6b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:33.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:33.879+0000 7f7789272640 1 --2- 172.21.15.92:0/2873264458 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f77841394d0 0x7f778415d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:33.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:33.879+0000 7f7789272640 1 --2- 172.21.15.92:0/2873264458 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f77700052a0 0x7f77840adea0 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:33.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:33.879+0000 7f7789272640 1 -- 172.21.15.92:0/2873264458 >> 172.21.15.92:0/2873264458 conn(0x7f778406a800 msgr2=0x7f77840b3280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:13:33.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:33.879+0000 7f7789272640 1 -- 172.21.15.92:0/2873264458 shutdown_connections 2023-11-24T06:13:33.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:33.879+0000 7f7789272640 1 -- 172.21.15.92:0/2873264458 wait complete. 2023-11-24T06:13:33.900 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:13:34.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:34.107+0000 7fe456378640 1 Processor -- start 2023-11-24T06:13:34.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:34.107+0000 7fe456378640 1 -- start start 2023-11-24T06:13:34.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:34.109+0000 7fe456378640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fe450143ae0 0x7fe4500cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:34.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:34.109+0000 7fe456378640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe45015e3b0 0x7fe4500702c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:34.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:34.109+0000 7fe456378640 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4500cfc30 con 0x7fe4500cf7d0 2023-11-24T06:13:34.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:34.109+0000 7fe456378640 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4500b1ee0 con 0x7fe450143ae0 2023-11-24T06:13:34.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:34.109+0000 7fe456378640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4500b2780 con 0x7fe45015e3b0 2023-11-24T06:13:34.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:34.110+0000 7fe454915640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe45015e3b0 0x7fe4500702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:34.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:34.110+0000 7fe44ffff640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fe450143ae0 0x7fe4500cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:34.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:34.110+0000 7fe454915640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe45015e3b0 0x7fe4500702c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3301/0 says I am v2:172.21.15.92:41136/0 (socket says 172.21.15.92:41136) 2023-11-24T06:13:34.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:34.110+0000 7fe454915640 1 -- 172.21.15.92:0/2648553323 learned_addr learned my addr 172.21.15.92:0/2648553323 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:13:34.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:34.110+0000 7fe44ffff640 1 -- 172.21.15.92:0/2648553323 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe45015e3b0 msgr2=0x7fe4500702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:13:34.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:34.110+0000 7fe44ffff640 1 --2- 172.21.15.92:0/2648553323 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe45015e3b0 0x7fe4500702c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:34.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:34.111+0000 7fe44ffff640 1 -- 172.21.15.92:0/2648553323 >> v1:172.21.15.118:6789/0 conn(0x7fe4500cf7d0 legacy=0x7fe45015dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:34.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:34.111+0000 7fe44ffff640 1 -- 172.21.15.92:0/2648553323 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe45015e990 con 0x7fe450143ae0 2023-11-24T06:13:34.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:34.112+0000 7fe44ffff640 1 --2- 172.21.15.92:0/2648553323 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe450143ae0 0x7fe4500cf290 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fe4400097a0 tx=0x7fe44002b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1ce6be7cee2d666e server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:34.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:34.112+0000 7fe44f7fe640 1 -- 172.21.15.92:0/2648553323 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe440031830 con 0x7fe450143ae0 2023-11-24T06:13:34.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:34.112+0000 7fe44f7fe640 1 -- 172.21.15.92:0/2648553323 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe440031990 con 0x7fe450143ae0 2023-11-24T06:13:34.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:34.112+0000 7fe44f7fe640 1 -- 172.21.15.92:0/2648553323 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe440031d20 con 0x7fe450143ae0 2023-11-24T06:13:34.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:34.113+0000 7fe456378640 1 -- 172.21.15.92:0/2648553323 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe450143ae0 msgr2=0x7fe4500cf290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:34.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:34.113+0000 7fe456378640 1 --2- 172.21.15.92:0/2648553323 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe450143ae0 0x7fe4500cf290 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fe4400097a0 tx=0x7fe44002b8d0 comp rx=0 tx=0).stop 2023-11-24T06:13:34.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:34.113+0000 7fe456378640 1 -- 172.21.15.92:0/2648553323 shutdown_connections 2023-11-24T06:13:34.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:34.113+0000 7fe456378640 1 --2- 172.21.15.92:0/2648553323 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe45015e3b0 0x7fe4500702c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:34.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:34.113+0000 7fe456378640 1 --2- 172.21.15.92:0/2648553323 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe450143ae0 0x7fe4500cf290 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:34.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:34.113+0000 7fe456378640 1 -- 172.21.15.92:0/2648553323 >> 172.21.15.92:0/2648553323 conn(0x7fe45006a800 msgr2=0x7fe45006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:13:34.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:34.113+0000 7fe456378640 1 -- 172.21.15.92:0/2648553323 shutdown_connections 2023-11-24T06:13:34.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:34.113+0000 7fe456378640 1 -- 172.21.15.92:0/2648553323 wait complete. 2023-11-24T06:13:34.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:34.115+0000 7fe456378640 1 Processor -- start 2023-11-24T06:13:34.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:34.115+0000 7fe456378640 1 -- start start 2023-11-24T06:13:34.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:34.116+0000 7fe456378640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe4500cf7d0 0x7fe4500b01c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:34.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:34.116+0000 7fe456378640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe450143ae0 0x7fe4500c6bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:34.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:34.116+0000 7fe456378640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe45015e3b0 0x7fe4501593f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:34.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:34.116+0000 7fe456378640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4501f7640 con 0x7fe45015e3b0 2023-11-24T06:13:34.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:34.116+0000 7fe456378640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4501f69c0 con 0x7fe4500cf7d0 2023-11-24T06:13:34.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:34.116+0000 7fe456378640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4501f6ca0 con 0x7fe450143ae0 2023-11-24T06:13:34.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:34.116+0000 7fe454915640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe45015e3b0 0x7fe4501593f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:34.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:34.116+0000 7fe454915640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe45015e3b0 0x7fe4501593f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:33638/0 (socket says 172.21.15.92:33638) 2023-11-24T06:13:34.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:34.116+0000 7fe454915640 1 -- 172.21.15.92:0/4195580944 learned_addr learned my addr 172.21.15.92:0/4195580944 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:13:34.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:34.117+0000 7fe447fff640 1 --2- 172.21.15.92:0/4195580944 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe450143ae0 0x7fe4500c6bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:34.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:34.117+0000 7fe454915640 1 -- 172.21.15.92:0/4195580944 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe450143ae0 msgr2=0x7fe4500c6bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:34.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:34.117+0000 7fe454915640 1 --2- 172.21.15.92:0/4195580944 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe450143ae0 0x7fe4500c6bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:34.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:34.117+0000 7fe44ffff640 1 --2- 172.21.15.92:0/4195580944 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe4500cf7d0 0x7fe4500b01c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:34.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:34.117+0000 7fe454915640 1 -- 172.21.15.92:0/4195580944 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe4500cf7d0 msgr2=0x7fe4500b01c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:34.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:34.117+0000 7fe454915640 1 --2- 172.21.15.92:0/4195580944 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe4500cf7d0 0x7fe4500b01c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:34.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:34.117+0000 7fe454915640 1 -- 172.21.15.92:0/4195580944 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe440009400 con 0x7fe45015e3b0 2023-11-24T06:13:34.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:34.117+0000 7fe454915640 1 --2- 172.21.15.92:0/4195580944 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe45015e3b0 0x7fe4501593f0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7fe44800efc0 tx=0x7fe44800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:34.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:34.117+0000 7fe44d7fa640 1 -- 172.21.15.92:0/4195580944 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe448009b60 con 0x7fe45015e3b0 2023-11-24T06:13:34.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:34.117+0000 7fe44d7fa640 1 -- 172.21.15.92:0/4195580944 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe44800f040 con 0x7fe45015e3b0 2023-11-24T06:13:34.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:34.117+0000 7fe44d7fa640 1 -- 172.21.15.92:0/4195580944 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe448003990 con 0x7fe45015e3b0 2023-11-24T06:13:34.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:34.117+0000 7fe456378640 1 -- 172.21.15.92:0/4195580944 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4501f6f80 con 0x7fe45015e3b0 2023-11-24T06:13:34.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:34.118+0000 7fe456378640 1 -- 172.21.15.92:0/4195580944 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe450142370 con 0x7fe45015e3b0 2023-11-24T06:13:34.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:34.119+0000 7fe456378640 1 -- 172.21.15.92:0/4195580944 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4501f7780 con 0x7fe45015e3b0 2023-11-24T06:13:34.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:34.120+0000 7fe44d7fa640 1 -- 172.21.15.92:0/4195580944 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 7) v1 ==== 89333+0+0 (secure 0 0 0) 0x7fe448003af0 con 0x7fe45015e3b0 2023-11-24T06:13:34.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:34.120+0000 7fe44d7fa640 1 --2- 172.21.15.92:0/4195580944 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fe43006af60 0x7fe43006d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:34.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:34.120+0000 7fe44d7fa640 1 -- 172.21.15.92:0/4195580944 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe448089f30 con 0x7fe45015e3b0 2023-11-24T06:13:34.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:34.121+0000 7fe44ffff640 1 --2- 172.21.15.92:0/4195580944 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fe43006af60 0x7fe43006d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:34.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:34.121+0000 7fe44ffff640 1 --2- 172.21.15.92:0/4195580944 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fe43006af60 0x7fe43006d420 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fe440036000 tx=0x7fe440008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:34.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:34.125+0000 7fe44d7fa640 1 -- 172.21.15.92:0/4195580944 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe4501f7780 con 0x7fe45015e3b0 2023-11-24T06:13:34.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:34.317+0000 7fe456378640 1 -- 172.21.15.92:0/4195580944 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe4501f7780 con 0x7fe45015e3b0 2023-11-24T06:13:34.323 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:34.320+0000 7fe44d7fa640 1 -- 172.21.15.92:0/4195580944 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v63) v1 ==== 74+0+16520 (secure 0 0 0) 0x7fe4501f7780 con 0x7fe45015e3b0 2023-11-24T06:13:34.323 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-24T06:13:34.324 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":63,"fsid":"2d6dee5f-8aa3-470f-931a-50ade963b1cb","created":"2023-11-24T06:10:04.707204+0000","modified":"2023-11-24T06:13:10.158258+0000","last_up_change":"2023-11-24T06:10:17.598352+0000","last_in_change":"2023-11-24T06:10:09.040793+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:10:17.733098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:13:06.409727+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-11-24T06:13:07.623678+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ff37a9fb-135f-471f-b4b0-f7b5772006c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6803","nonce":20560714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6805","nonce":20560714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6809","nonce":20560714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6807","nonce":20560714}]},"public_addr":"172.21.15.92:6803/20560714","cluster_addr":"172.21.15.92:6805/20560714","heartbeat_back_addr":"172.21.15.92:6809/20560714","heartbeat_front_addr":"172.21.15.92:6807/20560714","state":["exists","up"]},{"osd":1,"uuid":"03cdd353-dcf1-4f38-94fe-a795137c06b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6819","nonce":2451239906}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6821","nonce":2451239906}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6825","nonce":2451239906}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6823","nonce":2451239906}]},"public_addr":"172.21.15.92:6819/2451239906","cluster_addr":"172.21.15.92:6821/2451239906","heartbeat_back_addr":"172.21.15.92:6825/2451239906","heartbeat_front_addr":"172.21.15.92:6823/2451239906","state":["exists","up"]},{"osd":2,"uuid":"a4a20d80-5d57-484b-becd-9aa8070dd3aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6811","nonce":1124979277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6813","nonce":1124979277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6817","nonce":1124979277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6815","nonce":1124979277}]},"public_addr":"172.21.15.92:6811/1124979277","cluster_addr":"172.21.15.92:6813/1124979277","heartbeat_back_addr":"172.21.15.92:6817/1124979277","heartbeat_front_addr":"172.21.15.92:6815/1124979277","state":["exists","up"]},{"osd":3,"uuid":"4648d7ce-9b29-4583-9425-4ba8fd2b1903","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6827","nonce":1734115782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6829","nonce":1734115782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6833","nonce":1734115782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6831","nonce":1734115782}]},"public_addr":"172.21.15.92:6827/1734115782","cluster_addr":"172.21.15.92:6829/1734115782","heartbeat_back_addr":"172.21.15.92:6833/1734115782","heartbeat_front_addr":"172.21.15.92:6831/1734115782","state":["exists","up"]},{"osd":4,"uuid":"3bc9d7eb-c6fa-4fcf-a3ec-ade604e05aac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6819","nonce":3425937229}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6822","nonce":3425937229}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6830","nonce":3425937229}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6826","nonce":3425937229}]},"public_addr":"172.21.15.118:6819/3425937229","cluster_addr":"172.21.15.118:6822/3425937229","heartbeat_back_addr":"172.21.15.118:6830/3425937229","heartbeat_front_addr":"172.21.15.118:6826/3425937229","state":["exists","up"]},{"osd":5,"uuid":"dce9ba6d-59ed-46d1-84fc-a84b13de10c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6801","nonce":827455984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6803","nonce":827455984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6807","nonce":827455984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6805","nonce":827455984}]},"public_addr":"172.21.15.118:6801/827455984","cluster_addr":"172.21.15.118:6803/827455984","heartbeat_back_addr":"172.21.15.118:6807/827455984","heartbeat_front_addr":"172.21.15.118:6805/827455984","state":["exists","up"]},{"osd":6,"uuid":"82aec504-ae58-4709-8d30-46f2b12081b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6809","nonce":2946986141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6811","nonce":2946986141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6815","nonce":2946986141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6813","nonce":2946986141}]},"public_addr":"172.21.15.118:6809/2946986141","cluster_addr":"172.21.15.118:6811/2946986141","heartbeat_back_addr":"172.21.15.118:6815/2946986141","heartbeat_front_addr":"172.21.15.118:6813/2946986141","state":["exists","up"]},{"osd":7,"uuid":"4ed4c624-c88f-4d35-bdb1-bf209b5dffb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6817","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6818","nonce":48122059}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6821","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6823","nonce":48122059}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6829","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6831","nonce":48122059}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6825","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6827","nonce":48122059}]},"public_addr":"172.21.15.118:6818/48122059","cluster_addr":"172.21.15.118:6823/48122059","heartbeat_back_addr":"172.21.15.118:6831/48122059","heartbeat_front_addr":"172.21.15.118:6827/48122059","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.536664+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.368324+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.271201+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.342971+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.621832+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.387110+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.632946+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.416032+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:13:34.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:34.322+0000 7fe456378640 1 -- 172.21.15.92:0/4195580944 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fe43006af60 msgr2=0x7fe43006d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:34.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:34.322+0000 7fe456378640 1 --2- 172.21.15.92:0/4195580944 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fe43006af60 0x7fe43006d420 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fe440036000 tx=0x7fe440008030 comp rx=0 tx=0).stop 2023-11-24T06:13:34.326 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:34.322+0000 7fe456378640 1 -- 172.21.15.92:0/4195580944 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe45015e3b0 msgr2=0x7fe4501593f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:34.326 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:34.322+0000 7fe456378640 1 --2- 172.21.15.92:0/4195580944 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe45015e3b0 0x7fe4501593f0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7fe44800efc0 tx=0x7fe44800c490 comp rx=0 tx=0).stop 2023-11-24T06:13:34.326 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:34.322+0000 7fe456378640 1 -- 172.21.15.92:0/4195580944 shutdown_connections 2023-11-24T06:13:34.326 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:34.322+0000 7fe456378640 1 --2- 172.21.15.92:0/4195580944 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fe43006af60 0x7fe43006d420 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:34.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:34.322+0000 7fe456378640 1 --2- 172.21.15.92:0/4195580944 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe45015e3b0 0x7fe4501593f0 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:34.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:34.322+0000 7fe456378640 1 --2- 172.21.15.92:0/4195580944 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe450143ae0 0x7fe4500c6bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:34.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:34.322+0000 7fe456378640 1 --2- 172.21.15.92:0/4195580944 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe4500cf7d0 0x7fe4500b01c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:34.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:34.322+0000 7fe456378640 1 -- 172.21.15.92:0/4195580944 >> 172.21.15.92:0/4195580944 conn(0x7fe45006a800 msgr2=0x7fe45015b8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:13:34.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:34.322+0000 7fe456378640 1 -- 172.21.15.92:0/4195580944 shutdown_connections 2023-11-24T06:13:34.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:34.322+0000 7fe456378640 1 -- 172.21.15.92:0/4195580944 wait complete. 2023-11-24T06:13:34.345 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-24T06:13:34.345 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T06:13:34.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:34.550+0000 7f4c9c73c640 1 Processor -- start 2023-11-24T06:13:34.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:34.550+0000 7f4c9c73c640 1 -- start start 2023-11-24T06:13:34.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:34.552+0000 7f4c9c73c640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4c9415c300 0x7f4c9415f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:34.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:34.552+0000 7f4c9c73c640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f4c940b34e0 0x7f4c9415f920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:34.556 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:34.552+0000 7f4c9c73c640 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c9415fe60 con 0x7f4c94159040 2023-11-24T06:13:34.556 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:34.552+0000 7f4c9c73c640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c940c6060 con 0x7f4c940b34e0 2023-11-24T06:13:34.556 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:34.552+0000 7f4c9c73c640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c940cf710 con 0x7f4c9415c300 2023-11-24T06:13:34.556 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:34.552+0000 7f4c9acd9640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f4c940b34e0 0x7f4c9415f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:34.556 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:34.552+0000 7f4c9acd9640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f4c940b34e0 0x7f4c9415f920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:33652/0 (socket says 172.21.15.92:33652) 2023-11-24T06:13:34.557 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:34.552+0000 7f4c9acd9640 1 -- 172.21.15.92:0/1896897752 learned_addr learned my addr 172.21.15.92:0/1896897752 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:13:34.560 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:34.553+0000 7f4c99cd7640 1 --2- 172.21.15.92:0/1896897752 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4c9415c300 0x7f4c9415f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:34.560 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:34.553+0000 7f4c9acd9640 1 -- 172.21.15.92:0/1896897752 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4c9415c300 msgr2=0x7f4c9415f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:13:34.560 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:34.553+0000 7f4c9acd9640 1 --2- 172.21.15.92:0/1896897752 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4c9415c300 0x7f4c9415f3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:34.560 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:34.553+0000 7f4c9acd9640 1 -- 172.21.15.92:0/1896897752 >> v1:172.21.15.118:6789/0 conn(0x7f4c94159040 legacy=0x7f4c940cf000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:13:34.560 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:34.553+0000 7f4c9acd9640 1 -- 172.21.15.92:0/1896897752 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c940c62d0 con 0x7f4c940b34e0 2023-11-24T06:13:34.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:34.553+0000 7f4c99cd7640 1 --2- 172.21.15.92:0/1896897752 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4c9415c300 0x7f4c9415f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:13:34.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:34.554+0000 7f4c9acd9640 1 --2- 172.21.15.92:0/1896897752 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4c940b34e0 0x7f4c9415f920 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f4c8c009e50 tx=0x7f4c8c02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d743c4d9e75c8d3 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:34.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:34.554+0000 7f4c994d6640 1 -- 172.21.15.92:0/1896897752 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c8c02ca70 con 0x7f4c940b34e0 2023-11-24T06:13:34.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:34.554+0000 7f4c994d6640 1 -- 172.21.15.92:0/1896897752 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c8c02cbd0 con 0x7f4c940b34e0 2023-11-24T06:13:34.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:34.554+0000 7f4c994d6640 1 -- 172.21.15.92:0/1896897752 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c8c0343f0 con 0x7f4c940b34e0 2023-11-24T06:13:34.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:34.555+0000 7f4c9c73c640 1 -- 172.21.15.92:0/1896897752 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4c940b34e0 msgr2=0x7f4c9415f920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:34.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:34.555+0000 7f4c9c73c640 1 --2- 172.21.15.92:0/1896897752 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4c940b34e0 0x7f4c9415f920 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f4c8c009e50 tx=0x7f4c8c02bed0 comp rx=0 tx=0).stop 2023-11-24T06:13:34.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:34.555+0000 7f4c9c73c640 1 -- 172.21.15.92:0/1896897752 shutdown_connections 2023-11-24T06:13:34.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:34.555+0000 7f4c9c73c640 1 --2- 172.21.15.92:0/1896897752 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4c940b34e0 0x7f4c9415f920 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:34.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:34.555+0000 7f4c9c73c640 1 --2- 172.21.15.92:0/1896897752 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4c9415c300 0x7f4c9415f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:34.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:34.555+0000 7f4c9c73c640 1 -- 172.21.15.92:0/1896897752 >> 172.21.15.92:0/1896897752 conn(0x7f4c94066690 msgr2=0x7f4c9406a3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:13:34.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:34.555+0000 7f4c9c73c640 1 -- 172.21.15.92:0/1896897752 shutdown_connections 2023-11-24T06:13:34.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:34.555+0000 7f4c9c73c640 1 -- 172.21.15.92:0/1896897752 wait complete. 2023-11-24T06:13:34.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:34.557+0000 7f4c9c73c640 1 Processor -- start 2023-11-24T06:13:34.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:34.557+0000 7f4c9c73c640 1 -- start start 2023-11-24T06:13:34.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:34.558+0000 7f4c9c73c640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4c940b34e0 0x7f4c94151d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:34.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:34.558+0000 7f4c9c73c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4c94159040 0x7f4c940b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:34.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:34.558+0000 7f4c9c73c640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4c9415c300 0x7f4c941569b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:34.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:34.558+0000 7f4c9c73c640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c94152260 con 0x7f4c94159040 2023-11-24T06:13:34.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:34.558+0000 7f4c9c73c640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c940b6500 con 0x7f4c9415c300 2023-11-24T06:13:34.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:34.558+0000 7f4c9c73c640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c940b58f0 con 0x7f4c940b34e0 2023-11-24T06:13:34.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:34.559+0000 7f4c99cd7640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4c94159040 0x7f4c940b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:34.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:34.559+0000 7f4c99cd7640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4c94159040 0x7f4c940b4f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:33664/0 (socket says 172.21.15.92:33664) 2023-11-24T06:13:34.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:34.559+0000 7f4c99cd7640 1 -- 172.21.15.92:0/3205161706 learned_addr learned my addr 172.21.15.92:0/3205161706 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:13:34.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:34.559+0000 7f4c9a4d8640 1 --2- 172.21.15.92:0/3205161706 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4c940b34e0 0x7f4c94151d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:34.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:34.559+0000 7f4c99cd7640 1 -- 172.21.15.92:0/3205161706 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4c940b34e0 msgr2=0x7f4c94151d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:34.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:34.559+0000 7f4c99cd7640 1 --2- 172.21.15.92:0/3205161706 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4c940b34e0 0x7f4c94151d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:34.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:34.559+0000 7f4c99cd7640 1 -- 172.21.15.92:0/3205161706 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4c9415c300 msgr2=0x7f4c941569b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:34.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:34.559+0000 7f4c99cd7640 1 --2- 172.21.15.92:0/3205161706 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4c9415c300 0x7f4c941569b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:34.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:34.559+0000 7f4c99cd7640 1 -- 172.21.15.92:0/3205161706 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c8c009b30 con 0x7f4c94159040 2023-11-24T06:13:34.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:34.559+0000 7f4c9a4d8640 1 --2- 172.21.15.92:0/3205161706 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4c940b34e0 0x7f4c94151d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:13:34.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:34.559+0000 7f4c99cd7640 1 --2- 172.21.15.92:0/3205161706 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4c94159040 0x7f4c940b4f30 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f4c8800c820 tx=0x7f4c8800cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:34.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:34.560+0000 7f4c92ffd640 1 -- 172.21.15.92:0/3205161706 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c88007a30 con 0x7f4c94159040 2023-11-24T06:13:34.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:34.560+0000 7f4c92ffd640 1 -- 172.21.15.92:0/3205161706 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c88007b90 con 0x7f4c94159040 2023-11-24T06:13:34.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:34.560+0000 7f4c92ffd640 1 -- 172.21.15.92:0/3205161706 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c8800ce10 con 0x7f4c94159040 2023-11-24T06:13:34.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:34.560+0000 7f4c9c73c640 1 -- 172.21.15.92:0/3205161706 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c940b5bd0 con 0x7f4c94159040 2023-11-24T06:13:34.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:34.560+0000 7f4c9c73c640 1 -- 172.21.15.92:0/3205161706 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c940b6190 con 0x7f4c94159040 2023-11-24T06:13:34.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:34.561+0000 7f4c9c73c640 1 -- 172.21.15.92:0/3205161706 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c94074a50 con 0x7f4c94159040 2023-11-24T06:13:34.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:34.562+0000 7f4c92ffd640 1 -- 172.21.15.92:0/3205161706 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 7) v1 ==== 89333+0+0 (secure 0 0 0) 0x7f4c880113f0 con 0x7f4c94159040 2023-11-24T06:13:34.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:34.563+0000 7f4c92ffd640 1 --2- 172.21.15.92:0/3205161706 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f4c8006af60 0x7f4c8006d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:34.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:34.563+0000 7f4c92ffd640 1 -- 172.21.15.92:0/3205161706 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4c88089a30 con 0x7f4c94159040 2023-11-24T06:13:34.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:34.563+0000 7f4c9a4d8640 1 --2- 172.21.15.92:0/3205161706 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f4c8006af60 0x7f4c8006d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:34.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:34.564+0000 7f4c9a4d8640 1 --2- 172.21.15.92:0/3205161706 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f4c8006af60 0x7f4c8006d420 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f4c84003eb0 tx=0x7f4c84009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:34.572 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:34.568+0000 7f4c92ffd640 1 -- 172.21.15.92:0/3205161706 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4c88054820 con 0x7f4c94159040 2023-11-24T06:13:34.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:34.796+0000 7f4c9c73c640 1 -- 172.21.15.92:0/3205161706 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4c940675d0 con 0x7f4c94159040 2023-11-24T06:13:34.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:34.803+0000 7f4c92ffd640 1 -- 172.21.15.92:0/3205161706 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) v1 ==== 94+0+4921 (secure 0 0 0) 0x7f4c8801f090 con 0x7f4c94159040 2023-11-24T06:13:34.806 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-24T06:13:34.806 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":41,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5143,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6833/2094733752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":2094733752},{"type":"v1","addr":"172.21.15.118:6833","nonce":2094733752}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":36},{"gid":5146,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6835/2873546779","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":2873546779},{"type":"v1","addr":"172.21.15.118:6835","nonce":2873546779}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":36},{"gid":6141,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6836/3335501795","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":3335501795},{"type":"v1","addr":"172.21.15.92:6836","nonce":3335501795}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":35}],"filesystems":[{"mdsmap":{"epoch":40,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T06:13:08.625588+0000","modified":"2023-11-24T06:13:11.658333+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6144},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6144":{"gid":6144,"name":"a","rank":0,"incarnation":37,"state":"up:active","state_seq":3,"addr":"172.21.15.92:6837/3913457833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":3913457833},{"type":"v1","addr":"172.21.15.92:6837","nonce":3913457833}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-11-24T06:13:34.807 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 41 2023-11-24T06:13:34.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:34.806+0000 7f4c9c73c640 1 -- 172.21.15.92:0/3205161706 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f4c8006af60 msgr2=0x7f4c8006d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:34.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:34.806+0000 7f4c9c73c640 1 --2- 172.21.15.92:0/3205161706 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f4c8006af60 0x7f4c8006d420 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f4c84003eb0 tx=0x7f4c84009040 comp rx=0 tx=0).stop 2023-11-24T06:13:34.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:34.806+0000 7f4c9c73c640 1 -- 172.21.15.92:0/3205161706 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4c94159040 msgr2=0x7f4c940b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:34.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:34.806+0000 7f4c9c73c640 1 --2- 172.21.15.92:0/3205161706 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4c94159040 0x7f4c940b4f30 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f4c8800c820 tx=0x7f4c8800cce0 comp rx=0 tx=0).stop 2023-11-24T06:13:34.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:34.806+0000 7f4c9c73c640 1 -- 172.21.15.92:0/3205161706 shutdown_connections 2023-11-24T06:13:34.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:34.806+0000 7f4c9c73c640 1 --2- 172.21.15.92:0/3205161706 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f4c8006af60 0x7f4c8006d420 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:34.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:34.806+0000 7f4c9c73c640 1 --2- 172.21.15.92:0/3205161706 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4c9415c300 0x7f4c941569b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:34.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:34.806+0000 7f4c9c73c640 1 --2- 172.21.15.92:0/3205161706 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4c94159040 0x7f4c940b4f30 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:34.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:34.806+0000 7f4c9c73c640 1 --2- 172.21.15.92:0/3205161706 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4c940b34e0 0x7f4c94151d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:34.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:34.806+0000 7f4c9c73c640 1 -- 172.21.15.92:0/3205161706 >> 172.21.15.92:0/3205161706 conn(0x7f4c94066690 msgr2=0x7f4c940cf8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:13:34.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:34.806+0000 7f4c9c73c640 1 -- 172.21.15.92:0/3205161706 shutdown_connections 2023-11-24T06:13:34.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:34.806+0000 7f4c9c73c640 1 -- 172.21.15.92:0/3205161706 wait complete. 2023-11-24T06:13:34.828 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T06:13:35.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.033+0000 7fc6194bb640 1 Processor -- start 2023-11-24T06:13:35.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.033+0000 7fc6194bb640 1 -- start start 2023-11-24T06:13:35.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.035+0000 7fc6194bb640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc61413e360 0x7fc6140cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:35.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.035+0000 7fc6194bb640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fc61415f9c0 0x7fc6140cf7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:35.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.035+0000 7fc6194bb640 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc61415c5f0 con 0x7fc61413df80 2023-11-24T06:13:35.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.035+0000 7fc6194bb640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc61406c470 con 0x7fc61415f9c0 2023-11-24T06:13:35.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.035+0000 7fc6194bb640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc61406c750 con 0x7fc61413e360 2023-11-24T06:13:35.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.035+0000 7fc6137fe640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fc61415f9c0 0x7fc6140cf7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:35.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.035+0000 7fc6137fe640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fc61415f9c0 0x7fc6140cf7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:33668/0 (socket says 172.21.15.92:33668) 2023-11-24T06:13:35.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.035+0000 7fc6137fe640 1 -- 172.21.15.92:0/3854022768 learned_addr learned my addr 172.21.15.92:0/3854022768 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:13:35.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.035+0000 7fc6127fc640 1 --2- 172.21.15.92:0/3854022768 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc61413e360 0x7fc6140cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:35.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.036+0000 7fc6137fe640 1 -- 172.21.15.92:0/3854022768 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc61413e360 msgr2=0x7fc6140cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:13:35.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.036+0000 7fc6137fe640 1 --2- 172.21.15.92:0/3854022768 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc61413e360 0x7fc6140cf290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:35.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.036+0000 7fc6137fe640 1 -- 172.21.15.92:0/3854022768 >> v1:172.21.15.118:6789/0 conn(0x7fc61413df80 legacy=0x7fc61415bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:13:35.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.036+0000 7fc6137fe640 1 -- 172.21.15.92:0/3854022768 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc61406ca30 con 0x7fc61415f9c0 2023-11-24T06:13:35.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.036+0000 7fc6127fc640 1 --2- 172.21.15.92:0/3854022768 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc61413e360 0x7fc6140cf290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:13:35.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.037+0000 7fc6137fe640 1 --2- 172.21.15.92:0/3854022768 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc61415f9c0 0x7fc6140cf7d0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7fc60c009870 tx=0x7fc60c02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7d417f6ab951c27 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:35.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.037+0000 7fc611ffb640 1 -- 172.21.15.92:0/3854022768 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc60c031640 con 0x7fc61415f9c0 2023-11-24T06:13:35.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.037+0000 7fc611ffb640 1 -- 172.21.15.92:0/3854022768 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc60c0317a0 con 0x7fc61415f9c0 2023-11-24T06:13:35.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.037+0000 7fc611ffb640 1 -- 172.21.15.92:0/3854022768 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc60c031b30 con 0x7fc61415f9c0 2023-11-24T06:13:35.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.038+0000 7fc6194bb640 1 -- 172.21.15.92:0/3854022768 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc61415f9c0 msgr2=0x7fc6140cf7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:35.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.038+0000 7fc6194bb640 1 --2- 172.21.15.92:0/3854022768 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc61415f9c0 0x7fc6140cf7d0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7fc60c009870 tx=0x7fc60c02b8f0 comp rx=0 tx=0).stop 2023-11-24T06:13:35.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.038+0000 7fc6194bb640 1 -- 172.21.15.92:0/3854022768 shutdown_connections 2023-11-24T06:13:35.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.038+0000 7fc6194bb640 1 --2- 172.21.15.92:0/3854022768 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc61415f9c0 0x7fc6140cf7d0 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:35.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.038+0000 7fc6194bb640 1 --2- 172.21.15.92:0/3854022768 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc61413e360 0x7fc6140cf290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:35.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.038+0000 7fc6194bb640 1 -- 172.21.15.92:0/3854022768 >> 172.21.15.92:0/3854022768 conn(0x7fc61406a800 msgr2=0x7fc61406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:13:35.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.038+0000 7fc6194bb640 1 -- 172.21.15.92:0/3854022768 shutdown_connections 2023-11-24T06:13:35.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.038+0000 7fc6194bb640 1 -- 172.21.15.92:0/3854022768 wait complete. 2023-11-24T06:13:35.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.040+0000 7fc6194bb640 1 Processor -- start 2023-11-24T06:13:35.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.040+0000 7fc6194bb640 1 -- start start 2023-11-24T06:13:35.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.041+0000 7fc6194bb640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc61413df80 0x7fc6140ce790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:35.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.041+0000 7fc6194bb640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc61413e360 0x7fc61414afd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:35.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.041+0000 7fc6194bb640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc61415f9c0 0x7fc614159f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:35.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.041+0000 7fc6194bb640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc61414b510 con 0x7fc61413df80 2023-11-24T06:13:35.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.041+0000 7fc6194bb640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6141f2de0 con 0x7fc61415f9c0 2023-11-24T06:13:35.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.041+0000 7fc6194bb640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc61413a1b0 con 0x7fc61413e360 2023-11-24T06:13:35.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.041+0000 7fc612ffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc61413df80 0x7fc6140ce790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:35.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.041+0000 7fc612ffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc61413df80 0x7fc6140ce790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:33674/0 (socket says 172.21.15.92:33674) 2023-11-24T06:13:35.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.041+0000 7fc612ffd640 1 -- 172.21.15.92:0/2827787614 learned_addr learned my addr 172.21.15.92:0/2827787614 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:13:35.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.041+0000 7fc6137fe640 1 --2- 172.21.15.92:0/2827787614 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc61415f9c0 0x7fc614159f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:35.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.041+0000 7fc6127fc640 1 --2- 172.21.15.92:0/2827787614 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc61413e360 0x7fc61414afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:35.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.041+0000 7fc612ffd640 1 -- 172.21.15.92:0/2827787614 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc61413e360 msgr2=0x7fc61414afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:35.046 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.041+0000 7fc612ffd640 1 --2- 172.21.15.92:0/2827787614 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc61413e360 0x7fc61414afd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:35.046 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.041+0000 7fc612ffd640 1 -- 172.21.15.92:0/2827787614 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc61415f9c0 msgr2=0x7fc614159f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:35.046 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.041+0000 7fc612ffd640 1 --2- 172.21.15.92:0/2827787614 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc61415f9c0 0x7fc614159f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:35.046 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.041+0000 7fc612ffd640 1 -- 172.21.15.92:0/2827787614 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc60c0094d0 con 0x7fc61413df80 2023-11-24T06:13:35.046 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.041+0000 7fc6137fe640 1 --2- 172.21.15.92:0/2827787614 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc61415f9c0 0x7fc614159f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:13:35.046 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.042+0000 7fc612ffd640 1 --2- 172.21.15.92:0/2827787614 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc61413df80 0x7fc6140ce790 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7fc60400ba50 tx=0x7fc60400bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:35.047 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.042+0000 7fc60b7fe640 1 -- 172.21.15.92:0/2827787614 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc60400cbe0 con 0x7fc61413df80 2023-11-24T06:13:35.047 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.042+0000 7fc60b7fe640 1 -- 172.21.15.92:0/2827787614 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc60400cd80 con 0x7fc61413df80 2023-11-24T06:13:35.047 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.042+0000 7fc6194bb640 1 -- 172.21.15.92:0/2827787614 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc61413a490 con 0x7fc61413df80 2023-11-24T06:13:35.047 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.042+0000 7fc60b7fe640 1 -- 172.21.15.92:0/2827787614 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc604014680 con 0x7fc61413df80 2023-11-24T06:13:35.047 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.042+0000 7fc6194bb640 1 -- 172.21.15.92:0/2827787614 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc61413aa50 con 0x7fc61413df80 2023-11-24T06:13:35.047 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.043+0000 7fc6194bb640 1 -- 172.21.15.92:0/2827787614 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6140675d0 con 0x7fc61413df80 2023-11-24T06:13:35.047 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.044+0000 7fc60b7fe640 1 -- 172.21.15.92:0/2827787614 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 7) v1 ==== 89333+0+0 (secure 0 0 0) 0x7fc6040147e0 con 0x7fc61413df80 2023-11-24T06:13:35.048 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.045+0000 7fc60b7fe640 1 --2- 172.21.15.92:0/2827787614 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fc5f406af90 0x7fc5f406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:35.048 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.045+0000 7fc60b7fe640 1 -- 172.21.15.92:0/2827787614 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc604089640 con 0x7fc61413df80 2023-11-24T06:13:35.048 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.045+0000 7fc6127fc640 1 --2- 172.21.15.92:0/2827787614 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fc5f406af90 0x7fc5f406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:35.049 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.046+0000 7fc6127fc640 1 --2- 172.21.15.92:0/2827787614 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fc5f406af90 0x7fc5f406d450 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fc6141555a0 tx=0x7fc5fc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:35.052 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.049+0000 7fc60b7fe640 1 -- 172.21.15.92:0/2827787614 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc6040545f0 con 0x7fc61413df80 2023-11-24T06:13:35.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.277+0000 7fc6194bb640 1 -- 172.21.15.92:0/2827787614 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc61406df00 con 0x7fc61413df80 2023-11-24T06:13:35.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.278+0000 7fc60b7fe640 1 -- 172.21.15.92:0/2827787614 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) v1 ==== 94+0+4921 (secure 0 0 0) 0x7fc604057c10 con 0x7fc61413df80 2023-11-24T06:13:35.281 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-24T06:13:35.281 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":41,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5143,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6833/2094733752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":2094733752},{"type":"v1","addr":"172.21.15.118:6833","nonce":2094733752}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":36},{"gid":5146,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6835/2873546779","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":2873546779},{"type":"v1","addr":"172.21.15.118:6835","nonce":2873546779}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":36},{"gid":6141,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6836/3335501795","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":3335501795},{"type":"v1","addr":"172.21.15.92:6836","nonce":3335501795}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":35}],"filesystems":[{"mdsmap":{"epoch":40,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T06:13:08.625588+0000","modified":"2023-11-24T06:13:11.658333+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6144},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6144":{"gid":6144,"name":"a","rank":0,"incarnation":37,"state":"up:active","state_seq":3,"addr":"172.21.15.92:6837/3913457833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":3913457833},{"type":"v1","addr":"172.21.15.92:6837","nonce":3913457833}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-11-24T06:13:35.281 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 41 2023-11-24T06:13:35.283 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.280+0000 7fc6194bb640 1 -- 172.21.15.92:0/2827787614 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fc5f406af90 msgr2=0x7fc5f406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:35.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.280+0000 7fc6194bb640 1 --2- 172.21.15.92:0/2827787614 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fc5f406af90 0x7fc5f406d450 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fc6141555a0 tx=0x7fc5fc008040 comp rx=0 tx=0).stop 2023-11-24T06:13:35.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.280+0000 7fc6194bb640 1 -- 172.21.15.92:0/2827787614 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc61413df80 msgr2=0x7fc6140ce790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:35.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.280+0000 7fc6194bb640 1 --2- 172.21.15.92:0/2827787614 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc61413df80 0x7fc6140ce790 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7fc60400ba50 tx=0x7fc60400bf10 comp rx=0 tx=0).stop 2023-11-24T06:13:35.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.280+0000 7fc6194bb640 1 -- 172.21.15.92:0/2827787614 shutdown_connections 2023-11-24T06:13:35.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.280+0000 7fc6194bb640 1 --2- 172.21.15.92:0/2827787614 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fc5f406af90 0x7fc5f406d450 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:35.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.280+0000 7fc6194bb640 1 --2- 172.21.15.92:0/2827787614 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc61415f9c0 0x7fc614159f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:35.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.280+0000 7fc6194bb640 1 --2- 172.21.15.92:0/2827787614 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc61413e360 0x7fc61414afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:35.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.280+0000 7fc6194bb640 1 --2- 172.21.15.92:0/2827787614 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc61413df80 0x7fc6140ce790 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:35.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.280+0000 7fc6194bb640 1 -- 172.21.15.92:0/2827787614 >> 172.21.15.92:0/2827787614 conn(0x7fc61406a800 msgr2=0x7fc61415bd70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:13:35.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.281+0000 7fc6194bb640 1 -- 172.21.15.92:0/2827787614 shutdown_connections 2023-11-24T06:13:35.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.281+0000 7fc6194bb640 1 -- 172.21.15.92:0/2827787614 wait complete. 2023-11-24T06:13:35.303 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:13:35.515 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.512+0000 7fbdf46c9640 1 Processor -- start 2023-11-24T06:13:35.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.513+0000 7fbdf46c9640 1 -- start start 2023-11-24T06:13:35.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.514+0000 7fbdf46c9640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fbdec0ac8d0 0x7fbdec0aede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:35.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.514+0000 7fbdf46c9640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbdec071de0 con 0x7fbdec06a3e0 2023-11-24T06:13:35.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.514+0000 7fbdf46c9640 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbdec072030 con 0x7fbdec0d0b60 2023-11-24T06:13:35.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.514+0000 7fbdf46c9640 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbdec0acd60 con 0x7fbdec0ac8d0 2023-11-24T06:13:35.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.514+0000 7fbdf2465640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fbdec06a3e0 0x7fbdec06a7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:33582/0 (socket says 172.21.15.92:33582) 2023-11-24T06:13:35.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.514+0000 7fbdf2465640 1 -- 172.21.15.92:0/1452909983 learned_addr learned my addr 172.21.15.92:0/1452909983 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-24T06:13:35.519 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.515+0000 7fbdf1463640 1 -- 172.21.15.92:0/1452909983 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 404287656 0 0) 0x7fbdec071de0 con 0x7fbdec06a3e0 2023-11-24T06:13:35.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.515+0000 7fbdf1463640 1 -- 172.21.15.92:0/1452909983 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbddc003520 con 0x7fbdec06a3e0 2023-11-24T06:13:35.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.515+0000 7fbdf2c66640 1 --2- 172.21.15.92:0/1452909983 >> v2:172.21.15.118:3300/0 conn(0x7fbdec0ac8d0 0x7fbdec0aede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:35.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.515+0000 7fbdf1463640 1 -- 172.21.15.92:0/1452909983 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 397983162 0 0) 0x7fbddc003520 con 0x7fbdec06a3e0 2023-11-24T06:13:35.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.515+0000 7fbdf1463640 1 -- 172.21.15.92:0/1452909983 >> v2:172.21.15.118:3300/0 conn(0x7fbdec0ac8d0 msgr2=0x7fbdec0aede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:13:35.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.515+0000 7fbdf1463640 1 --2- 172.21.15.92:0/1452909983 >> v2:172.21.15.118:3300/0 conn(0x7fbdec0ac8d0 0x7fbdec0aede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:35.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.515+0000 7fbdf1463640 1 -- 172.21.15.92:0/1452909983 >> v1:172.21.15.118:6789/0 conn(0x7fbdec0d0b60 legacy=0x7fbdec06f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:35.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.515+0000 7fbdf1463640 1 -- 172.21.15.92:0/1452909983 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdec072310 con 0x7fbdec06a3e0 2023-11-24T06:13:35.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.515+0000 7fbdf1463640 1 -- 172.21.15.92:0/1452909983 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2607661350 0 0) 0x7fbde00019f0 con 0x7fbdec06a3e0 2023-11-24T06:13:35.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.515+0000 7fbdf1463640 1 -- 172.21.15.92:0/1452909983 >> v1:172.21.15.92:6789/0 conn(0x7fbdec06a3e0 legacy=0x7fbdec06a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:35.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.515+0000 7fbdf1463640 1 --2- 172.21.15.92:0/1452909983 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fbddc003970 0x7fbddc003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:35.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.515+0000 7fbdf1463640 1 --2- 172.21.15.92:0/1452909983 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbddc0047c0 0x7fbddc004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:35.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.515+0000 7fbdf1463640 1 --2- 172.21.15.92:0/1452909983 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbddc005180 0x7fbddc00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:35.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.516+0000 7fbdf1463640 1 -- 172.21.15.92:0/1452909983 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdec0acd60 con 0x7fbddc0047c0 2023-11-24T06:13:35.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.516+0000 7fbdf1463640 1 -- 172.21.15.92:0/1452909983 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbddc00c0d0 con 0x7fbddc005180 2023-11-24T06:13:35.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.516+0000 7fbdf1463640 1 -- 172.21.15.92:0/1452909983 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbddc00c290 con 0x7fbddc003970 2023-11-24T06:13:35.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.516+0000 7fbdf1c64640 1 --2- 172.21.15.92:0/1452909983 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbddc0047c0 0x7fbddc004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:35.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.516+0000 7fbdf2465640 1 --2- 172.21.15.92:0/1452909983 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fbddc003970 0x7fbddc003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:35.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.516+0000 7fbdf1c64640 1 -- 172.21.15.92:0/1452909983 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fbddc003970 msgr2=0x7fbddc003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:13:35.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.516+0000 7fbdf1c64640 1 --2- 172.21.15.92:0/1452909983 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fbddc003970 0x7fbddc003d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:35.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.516+0000 7fbdf1c64640 1 -- 172.21.15.92:0/1452909983 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbddc005180 msgr2=0x7fbddc00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:13:35.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.516+0000 7fbdf1c64640 1 --2- 172.21.15.92:0/1452909983 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbddc005180 0x7fbddc00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:35.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.516+0000 7fbdf1c64640 1 -- 172.21.15.92:0/1452909983 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbddc003520 con 0x7fbddc0047c0 2023-11-24T06:13:35.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.516+0000 7fbdf2c66640 1 -- 172.21.15.92:0/1452909983 reap_dead start 2023-11-24T06:13:35.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.517+0000 7fbdf1c64640 1 --2- 172.21.15.92:0/1452909983 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbddc0047c0 0x7fbddc004c40 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fbdd8001720 tx=0x7fbddc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc12b0712db4878a server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:35.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.518+0000 7fbdf1463640 1 -- 172.21.15.92:0/1452909983 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdd80052d0 con 0x7fbddc0047c0 2023-11-24T06:13:35.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.518+0000 7fbdf1463640 1 -- 172.21.15.92:0/1452909983 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdd802e050 con 0x7fbddc0047c0 2023-11-24T06:13:35.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.518+0000 7fbdf1463640 1 -- 172.21.15.92:0/1452909983 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdd8003830 con 0x7fbddc0047c0 2023-11-24T06:13:35.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.518+0000 7fbdf46c9640 1 -- 172.21.15.92:0/1452909983 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbddc0047c0 msgr2=0x7fbddc004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:35.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.518+0000 7fbdf46c9640 1 --2- 172.21.15.92:0/1452909983 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbddc0047c0 0x7fbddc004c40 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fbdd8001720 tx=0x7fbddc0057f0 comp rx=0 tx=0).stop 2023-11-24T06:13:35.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.518+0000 7fbdf46c9640 1 -- 172.21.15.92:0/1452909983 shutdown_connections 2023-11-24T06:13:35.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.518+0000 7fbdf46c9640 1 --2- 172.21.15.92:0/1452909983 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbddc0047c0 0x7fbddc004c40 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:35.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.518+0000 7fbdf46c9640 1 -- 172.21.15.92:0/1452909983 >> 172.21.15.92:0/1452909983 conn(0x7fbdec0cfef0 msgr2=0x7fbdec0d0300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:13:35.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.518+0000 7fbdf46c9640 1 -- 172.21.15.92:0/1452909983 shutdown_connections 2023-11-24T06:13:35.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.518+0000 7fbdf46c9640 1 -- 172.21.15.92:0/1452909983 wait complete. 2023-11-24T06:13:35.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.520+0000 7fbdf46c9640 1 Processor -- start 2023-11-24T06:13:35.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.520+0000 7fbdf46c9640 1 -- start start 2023-11-24T06:13:35.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.521+0000 7fbdf46c9640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbddc0047c0 0x7fbdec0cc940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:35.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.521+0000 7fbdf46c9640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fbdec148850 0x7fbdec0ce380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:35.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.521+0000 7fbdf46c9640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbdec0ce8c0 0x7fbdec15a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:35.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.521+0000 7fbdf46c9640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdec160710 con 0x7fbddc0047c0 2023-11-24T06:13:35.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.521+0000 7fbdf46c9640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdec0cf200 con 0x7fbdec0ce8c0 2023-11-24T06:13:35.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.521+0000 7fbdf46c9640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdec0cf4e0 con 0x7fbdec148850 2023-11-24T06:13:35.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.521+0000 7fbdf2465640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbddc0047c0 0x7fbdec0cc940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:35.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.521+0000 7fbdf2465640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbddc0047c0 0x7fbdec0cc940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:33696/0 (socket says 172.21.15.92:33696) 2023-11-24T06:13:35.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.521+0000 7fbdf2465640 1 -- 172.21.15.92:0/3250080585 learned_addr learned my addr 172.21.15.92:0/3250080585 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:13:35.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.522+0000 7fbdf1c64640 1 --2- 172.21.15.92:0/3250080585 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fbdec148850 0x7fbdec0ce380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:35.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.522+0000 7fbdf2c66640 1 --2- 172.21.15.92:0/3250080585 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbdec0ce8c0 0x7fbdec15a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:35.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.522+0000 7fbdf2465640 1 -- 172.21.15.92:0/3250080585 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fbdec148850 msgr2=0x7fbdec0ce380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:35.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.522+0000 7fbdf2465640 1 --2- 172.21.15.92:0/3250080585 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fbdec148850 0x7fbdec0ce380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:35.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.522+0000 7fbdf2465640 1 -- 172.21.15.92:0/3250080585 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbdec0ce8c0 msgr2=0x7fbdec15a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:35.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.522+0000 7fbdf2465640 1 --2- 172.21.15.92:0/3250080585 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbdec0ce8c0 0x7fbdec15a3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:35.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.522+0000 7fbdf2465640 1 -- 172.21.15.92:0/3250080585 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdec0cf7c0 con 0x7fbddc0047c0 2023-11-24T06:13:35.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.522+0000 7fbdf2465640 1 --2- 172.21.15.92:0/3250080585 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbddc0047c0 0x7fbdec0cc940 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fbde0002170 tx=0x7fbde000b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:35.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.522+0000 7fbdeaffd640 1 -- 172.21.15.92:0/3250080585 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbde0004d60 con 0x7fbddc0047c0 2023-11-24T06:13:35.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.522+0000 7fbdeaffd640 1 -- 172.21.15.92:0/3250080585 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbde00070d0 con 0x7fbddc0047c0 2023-11-24T06:13:35.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.522+0000 7fbdf46c9640 1 -- 172.21.15.92:0/3250080585 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbdec151d20 con 0x7fbddc0047c0 2023-11-24T06:13:35.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.523+0000 7fbdeaffd640 1 -- 172.21.15.92:0/3250080585 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbde0007460 con 0x7fbddc0047c0 2023-11-24T06:13:35.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.523+0000 7fbdf46c9640 1 -- 172.21.15.92:0/3250080585 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbdec160390 con 0x7fbddc0047c0 2023-11-24T06:13:35.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.524+0000 7fbdf46c9640 1 -- 172.21.15.92:0/3250080585 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbdec0cf950 con 0x7fbddc0047c0 2023-11-24T06:13:35.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.525+0000 7fbdeaffd640 1 -- 172.21.15.92:0/3250080585 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 7) v1 ==== 89333+0+0 (secure 0 0 0) 0x7fbde00075c0 con 0x7fbddc0047c0 2023-11-24T06:13:35.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.526+0000 7fbdeaffd640 1 --2- 172.21.15.92:0/3250080585 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fbdd006af60 0x7fbdd006d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:35.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.526+0000 7fbdeaffd640 1 -- 172.21.15.92:0/3250080585 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbde0089960 con 0x7fbddc0047c0 2023-11-24T06:13:35.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.526+0000 7fbdf1c64640 1 --2- 172.21.15.92:0/3250080585 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fbdd006af60 0x7fbdd006d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:35.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.527+0000 7fbdf1c64640 1 --2- 172.21.15.92:0/3250080585 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fbdd006af60 0x7fbdd006d420 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fbdd8033000 tx=0x7fbdd8009030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:35.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.531+0000 7fbdeaffd640 1 -- 172.21.15.92:0/3250080585 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fbde0054880 con 0x7fbddc0047c0 2023-11-24T06:13:35.724 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.721+0000 7fbdf46c9640 1 -- 172.21.15.92:0/3250080585 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbdec0675d0 con 0x7fbddc0047c0 2023-11-24T06:13:35.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.724+0000 7fbdeaffd640 1 -- 172.21.15.92:0/3250080585 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v63) v1 ==== 74+0+16520 (secure 0 0 0) 0x7fbde0019020 con 0x7fbddc0047c0 2023-11-24T06:13:35.728 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-24T06:13:35.728 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":63,"fsid":"2d6dee5f-8aa3-470f-931a-50ade963b1cb","created":"2023-11-24T06:10:04.707204+0000","modified":"2023-11-24T06:13:10.158258+0000","last_up_change":"2023-11-24T06:10:17.598352+0000","last_in_change":"2023-11-24T06:10:09.040793+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:10:17.733098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:13:06.409727+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-11-24T06:13:07.623678+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ff37a9fb-135f-471f-b4b0-f7b5772006c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6803","nonce":20560714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6805","nonce":20560714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6809","nonce":20560714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6807","nonce":20560714}]},"public_addr":"172.21.15.92:6803/20560714","cluster_addr":"172.21.15.92:6805/20560714","heartbeat_back_addr":"172.21.15.92:6809/20560714","heartbeat_front_addr":"172.21.15.92:6807/20560714","state":["exists","up"]},{"osd":1,"uuid":"03cdd353-dcf1-4f38-94fe-a795137c06b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6819","nonce":2451239906}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6821","nonce":2451239906}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6825","nonce":2451239906}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6823","nonce":2451239906}]},"public_addr":"172.21.15.92:6819/2451239906","cluster_addr":"172.21.15.92:6821/2451239906","heartbeat_back_addr":"172.21.15.92:6825/2451239906","heartbeat_front_addr":"172.21.15.92:6823/2451239906","state":["exists","up"]},{"osd":2,"uuid":"a4a20d80-5d57-484b-becd-9aa8070dd3aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6811","nonce":1124979277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6813","nonce":1124979277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6817","nonce":1124979277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6815","nonce":1124979277}]},"public_addr":"172.21.15.92:6811/1124979277","cluster_addr":"172.21.15.92:6813/1124979277","heartbeat_back_addr":"172.21.15.92:6817/1124979277","heartbeat_front_addr":"172.21.15.92:6815/1124979277","state":["exists","up"]},{"osd":3,"uuid":"4648d7ce-9b29-4583-9425-4ba8fd2b1903","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6827","nonce":1734115782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6829","nonce":1734115782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6833","nonce":1734115782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6831","nonce":1734115782}]},"public_addr":"172.21.15.92:6827/1734115782","cluster_addr":"172.21.15.92:6829/1734115782","heartbeat_back_addr":"172.21.15.92:6833/1734115782","heartbeat_front_addr":"172.21.15.92:6831/1734115782","state":["exists","up"]},{"osd":4,"uuid":"3bc9d7eb-c6fa-4fcf-a3ec-ade604e05aac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6819","nonce":3425937229}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6822","nonce":3425937229}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6830","nonce":3425937229}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6826","nonce":3425937229}]},"public_addr":"172.21.15.118:6819/3425937229","cluster_addr":"172.21.15.118:6822/3425937229","heartbeat_back_addr":"172.21.15.118:6830/3425937229","heartbeat_front_addr":"172.21.15.118:6826/3425937229","state":["exists","up"]},{"osd":5,"uuid":"dce9ba6d-59ed-46d1-84fc-a84b13de10c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6801","nonce":827455984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6803","nonce":827455984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6807","nonce":827455984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6805","nonce":827455984}]},"public_addr":"172.21.15.118:6801/827455984","cluster_addr":"172.21.15.118:6803/827455984","heartbeat_back_addr":"172.21.15.118:6807/827455984","heartbeat_front_addr":"172.21.15.118:6805/827455984","state":["exists","up"]},{"osd":6,"uuid":"82aec504-ae58-4709-8d30-46f2b12081b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6809","nonce":2946986141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6811","nonce":2946986141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6815","nonce":2946986141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6813","nonce":2946986141}]},"public_addr":"172.21.15.118:6809/2946986141","cluster_addr":"172.21.15.118:6811/2946986141","heartbeat_back_addr":"172.21.15.118:6815/2946986141","heartbeat_front_addr":"172.21.15.118:6813/2946986141","state":["exists","up"]},{"osd":7,"uuid":"4ed4c624-c88f-4d35-bdb1-bf209b5dffb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6817","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6818","nonce":48122059}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6821","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6823","nonce":48122059}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6829","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6831","nonce":48122059}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6825","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6827","nonce":48122059}]},"public_addr":"172.21.15.118:6818/48122059","cluster_addr":"172.21.15.118:6823/48122059","heartbeat_back_addr":"172.21.15.118:6831/48122059","heartbeat_front_addr":"172.21.15.118:6827/48122059","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.536664+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.368324+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.271201+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.342971+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.621832+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.387110+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.632946+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.416032+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:13:35.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.727+0000 7fbdf46c9640 1 -- 172.21.15.92:0/3250080585 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fbdd006af60 msgr2=0x7fbdd006d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:35.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.727+0000 7fbdf46c9640 1 --2- 172.21.15.92:0/3250080585 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fbdd006af60 0x7fbdd006d420 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fbdd8033000 tx=0x7fbdd8009030 comp rx=0 tx=0).stop 2023-11-24T06:13:35.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.727+0000 7fbdf46c9640 1 -- 172.21.15.92:0/3250080585 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbddc0047c0 msgr2=0x7fbdec0cc940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:35.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.727+0000 7fbdf46c9640 1 --2- 172.21.15.92:0/3250080585 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbddc0047c0 0x7fbdec0cc940 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fbde0002170 tx=0x7fbde000b4a0 comp rx=0 tx=0).stop 2023-11-24T06:13:35.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.727+0000 7fbdf46c9640 1 -- 172.21.15.92:0/3250080585 shutdown_connections 2023-11-24T06:13:35.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.727+0000 7fbdf46c9640 1 --2- 172.21.15.92:0/3250080585 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fbdd006af60 0x7fbdd006d420 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:35.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.727+0000 7fbdf46c9640 1 --2- 172.21.15.92:0/3250080585 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbdec0ce8c0 0x7fbdec15a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:35.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.727+0000 7fbdf46c9640 1 --2- 172.21.15.92:0/3250080585 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fbdec148850 0x7fbdec0ce380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:35.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.727+0000 7fbdf46c9640 1 --2- 172.21.15.92:0/3250080585 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbddc0047c0 0x7fbdec0cc940 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:35.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.727+0000 7fbdf46c9640 1 -- 172.21.15.92:0/3250080585 >> 172.21.15.92:0/3250080585 conn(0x7fbdec0cfef0 msgr2=0x7fbdec0aede0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:13:35.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.728+0000 7fbdf46c9640 1 -- 172.21.15.92:0/3250080585 shutdown_connections 2023-11-24T06:13:35.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.728+0000 7fbdf46c9640 1 -- 172.21.15.92:0/3250080585 wait complete. 2023-11-24T06:13:35.751 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-24T06:13:35.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.951+0000 7f10ada90640 1 Processor -- start 2023-11-24T06:13:35.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.952+0000 7f10ada90640 1 -- start start 2023-11-24T06:13:35.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.953+0000 7f10ada90640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f10a80ae5a0 0x7f10a815e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:35.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.953+0000 7f10ada90640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10a8070c40 con 0x7f10a80cf290 2023-11-24T06:13:35.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.953+0000 7f10ada90640 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10a806c470 con 0x7f10a80b3ae0 2023-11-24T06:13:35.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.953+0000 7f10ada90640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10a8071450 con 0x7f10a80ae5a0 2023-11-24T06:13:35.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.954+0000 7f10a77fe640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f10a80ae5a0 0x7f10a815e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:35.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.954+0000 7f10a7fff640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f10a80cf290 0x7f10a80b1fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:33598/0 (socket says 172.21.15.92:33598) 2023-11-24T06:13:35.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.954+0000 7f10a7fff640 1 -- 172.21.15.92:0/2296652068 learned_addr learned my addr 172.21.15.92:0/2296652068 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-24T06:13:35.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.954+0000 7f10a77fe640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f10a80ae5a0 0x7f10a815e430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:33700/0 (socket says 172.21.15.92:33700) 2023-11-24T06:13:35.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.954+0000 7f10a77fe640 1 -- 172.21.15.92:0/2296652068 >> v1:172.21.15.118:6789/0 conn(0x7f10a80b3ae0 legacy=0x7f10a806cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:13:35.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.954+0000 7f10a77fe640 1 -- 172.21.15.92:0/2296652068 >> v1:172.21.15.92:6789/0 conn(0x7f10a80cf290 legacy=0x7f10a80b1fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:35.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.954+0000 7f10a6ffd640 1 -- 172.21.15.92:0/2296652068 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 170723715 0 0) 0x7f10a8070c40 con 0x7f10a80cf290 2023-11-24T06:13:35.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.954+0000 7f10a77fe640 1 -- 172.21.15.92:0/2296652068 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10a806c750 con 0x7f10a80ae5a0 2023-11-24T06:13:35.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.955+0000 7f10a77fe640 1 --2- 172.21.15.92:0/2296652068 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f10a80ae5a0 0x7f10a815e430 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f10980097a0 tx=0x7f109802b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16588289406270d2 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:35.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.956+0000 7f10a6ffd640 1 -- 172.21.15.92:0/2296652068 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1098030610 con 0x7f10a80ae5a0 2023-11-24T06:13:35.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.956+0000 7f10a6ffd640 1 -- 172.21.15.92:0/2296652068 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1098030770 con 0x7f10a80ae5a0 2023-11-24T06:13:35.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.956+0000 7f10a6ffd640 1 -- 172.21.15.92:0/2296652068 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1098030b00 con 0x7f10a80ae5a0 2023-11-24T06:13:35.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.956+0000 7f10ada90640 1 -- 172.21.15.92:0/2296652068 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f10a80ae5a0 msgr2=0x7f10a815e430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:35.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.956+0000 7f10ada90640 1 --2- 172.21.15.92:0/2296652068 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f10a80ae5a0 0x7f10a815e430 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f10980097a0 tx=0x7f109802b960 comp rx=0 tx=0).stop 2023-11-24T06:13:35.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.956+0000 7f10ada90640 1 -- 172.21.15.92:0/2296652068 shutdown_connections 2023-11-24T06:13:35.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.956+0000 7f10ada90640 1 --2- 172.21.15.92:0/2296652068 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f10a80ae5a0 0x7f10a815e430 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:35.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.956+0000 7f10ada90640 1 -- 172.21.15.92:0/2296652068 >> 172.21.15.92:0/2296652068 conn(0x7f10a806a800 msgr2=0x7f10a806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:13:35.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.956+0000 7f10ada90640 1 -- 172.21.15.92:0/2296652068 shutdown_connections 2023-11-24T06:13:35.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.957+0000 7f10ada90640 1 -- 172.21.15.92:0/2296652068 wait complete. 2023-11-24T06:13:35.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.958+0000 7f10ada90640 1 Processor -- start 2023-11-24T06:13:35.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.958+0000 7f10ada90640 1 -- start start 2023-11-24T06:13:35.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.959+0000 7f10ada90640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f10a80ae5a0 0x7f10a80cc360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:35.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.959+0000 7f10ada90640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f10a80b3ae0 0x7f10a80ce710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:35.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.959+0000 7f10ada90640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f10a80cf290 0x7f10a8157d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:35.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.959+0000 7f10ada90640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10a80c5550 con 0x7f10a80cf290 2023-11-24T06:13:35.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.959+0000 7f10ada90640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10a80cec50 con 0x7f10a80b3ae0 2023-11-24T06:13:35.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.959+0000 7f10ada90640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10a80cc9f0 con 0x7f10a80ae5a0 2023-11-24T06:13:35.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.959+0000 7f10a7fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f10a80cf290 0x7f10a8157d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:35.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.959+0000 7f10a7fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f10a80cf290 0x7f10a8157d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:33708/0 (socket says 172.21.15.92:33708) 2023-11-24T06:13:35.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.959+0000 7f10a7fff640 1 -- 172.21.15.92:0/3171724973 learned_addr learned my addr 172.21.15.92:0/3171724973 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:13:35.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.960+0000 7f10a7fff640 1 -- 172.21.15.92:0/3171724973 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f10a80ae5a0 msgr2=0x7f10a80cc360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:35.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.960+0000 7f10a7fff640 1 --2- 172.21.15.92:0/3171724973 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f10a80ae5a0 0x7f10a80cc360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:35.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.960+0000 7f10a7fff640 1 -- 172.21.15.92:0/3171724973 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f10a80b3ae0 msgr2=0x7f10a80ce710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:35.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.960+0000 7f10a7fff640 1 --2- 172.21.15.92:0/3171724973 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f10a80b3ae0 0x7f10a80ce710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:35.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.960+0000 7f10a7fff640 1 -- 172.21.15.92:0/3171724973 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1098009400 con 0x7f10a80cf290 2023-11-24T06:13:35.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.960+0000 7f10a7fff640 1 --2- 172.21.15.92:0/3171724973 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f10a80cf290 0x7f10a8157d90 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f10a000c780 tx=0x7f10a000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:35.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.960+0000 7f10a4ff9640 1 -- 172.21.15.92:0/3171724973 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10a000ce90 con 0x7f10a80cf290 2023-11-24T06:13:35.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.960+0000 7f10a4ff9640 1 -- 172.21.15.92:0/3171724973 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10a000a640 con 0x7f10a80cf290 2023-11-24T06:13:35.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.960+0000 7f10a4ff9640 1 -- 172.21.15.92:0/3171724973 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10a000a9d0 con 0x7f10a80cf290 2023-11-24T06:13:35.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.960+0000 7f10ada90640 1 -- 172.21.15.92:0/3171724973 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10a80cccd0 con 0x7f10a80cf290 2023-11-24T06:13:35.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.961+0000 7f10ada90640 1 -- 172.21.15.92:0/3171724973 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10a80cd290 con 0x7f10a80cf290 2023-11-24T06:13:35.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.962+0000 7f10ada90640 1 -- 172.21.15.92:0/3171724973 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10a80675d0 con 0x7f10a80cf290 2023-11-24T06:13:35.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.962+0000 7f10a4ff9640 1 -- 172.21.15.92:0/3171724973 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 7) v1 ==== 89333+0+0 (secure 0 0 0) 0x7f10a000ab30 con 0x7f10a80cf290 2023-11-24T06:13:35.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.963+0000 7f10a4ff9640 1 --2- 172.21.15.92:0/3171724973 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f108806af90 0x7f108806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:35.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.963+0000 7f10a4ff9640 1 -- 172.21.15.92:0/3171724973 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f10a0088ba0 con 0x7f10a80cf290 2023-11-24T06:13:35.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.963+0000 7f10a77fe640 1 --2- 172.21.15.92:0/3171724973 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f108806af90 0x7f108806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:35.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.964+0000 7f10a77fe640 1 --2- 172.21.15.92:0/3171724973 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f108806af90 0x7f108806d450 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f1098036000 tx=0x7f109803b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:35.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:35.968+0000 7f10a4ff9640 1 -- 172.21.15.92:0/3171724973 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f10a0053b40 con 0x7f10a80cf290 2023-11-24T06:13:36.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:36.194+0000 7f10ada90640 1 -- 172.21.15.92:0/3171724973 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f10a80c5690 con 0x7f10a80cf290 2023-11-24T06:13:36.198 INFO:tasks.ceph.mon.a.smithi092.stderr:2023-11-24T06:13:36.195+0000 7f83f3ce4640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-24T06:13:36.202 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:36.199+0000 7f10a4ff9640 1 -- 172.21.15.92:0/3171724973 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v42) v1 ==== 161+0+0 (secure 0 0 0) 0x7f10a0018020 con 0x7f10a80cf290 2023-11-24T06:13:36.202 INFO:teuthology.orchestra.run.smithi092.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-24T06:13:36.205 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:36.202+0000 7f10ada90640 1 -- 172.21.15.92:0/3171724973 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f108806af90 msgr2=0x7f108806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:36.205 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:36.202+0000 7f10ada90640 1 --2- 172.21.15.92:0/3171724973 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f108806af90 0x7f108806d450 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f1098036000 tx=0x7f109803b040 comp rx=0 tx=0).stop 2023-11-24T06:13:36.205 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:36.202+0000 7f10ada90640 1 -- 172.21.15.92:0/3171724973 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f10a80cf290 msgr2=0x7f10a8157d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:36.205 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:36.202+0000 7f10ada90640 1 --2- 172.21.15.92:0/3171724973 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f10a80cf290 0x7f10a8157d90 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f10a000c780 tx=0x7f10a000cc40 comp rx=0 tx=0).stop 2023-11-24T06:13:36.205 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:36.202+0000 7f10ada90640 1 -- 172.21.15.92:0/3171724973 shutdown_connections 2023-11-24T06:13:36.205 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:36.202+0000 7f10ada90640 1 --2- 172.21.15.92:0/3171724973 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f108806af90 0x7f108806d450 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:36.206 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:36.202+0000 7f10ada90640 1 --2- 172.21.15.92:0/3171724973 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f10a80cf290 0x7f10a8157d90 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:36.206 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:36.202+0000 7f10ada90640 1 --2- 172.21.15.92:0/3171724973 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f10a80b3ae0 0x7f10a80ce710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:36.206 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:36.202+0000 7f10ada90640 1 --2- 172.21.15.92:0/3171724973 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f10a80ae5a0 0x7f10a80cc360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:36.206 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:36.202+0000 7f10ada90640 1 -- 172.21.15.92:0/3171724973 >> 172.21.15.92:0/3171724973 conn(0x7f10a806a800 msgr2=0x7f10a80b1ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:13:36.206 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:36.202+0000 7f10ada90640 1 -- 172.21.15.92:0/3171724973 shutdown_connections 2023-11-24T06:13:36.206 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:36.202+0000 7f10ada90640 1 -- 172.21.15.92:0/3171724973 wait complete. 2023-11-24T06:13:36.215 INFO:tasks.ceph.mds.a.smithi092.stderr: -5249> 2023-11-24T06:13:10.114+0000 7fd3afc08640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-24T06:13:36.225 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-11-24T06:13:36.279 INFO:tasks.ceph.mds.a.smithi092.stdout:starting mds.a at 2023-11-24T06:13:36.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:36.416+0000 7fa35e41c640 1 Processor -- start 2023-11-24T06:13:36.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:36.416+0000 7fa35e41c640 1 -- start start 2023-11-24T06:13:36.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:36.417+0000 7fa35e41c640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fa358143ae0 0x7fa3580cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:36.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:36.417+0000 7fa35e41c640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fa3580cf7d0 0x7fa35815dca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:36.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:36.417+0000 7fa35e41c640 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3580cfc30 con 0x7fa35815e1e0 2023-11-24T06:13:36.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:36.417+0000 7fa35e41c640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3580b1ee0 con 0x7fa358143ae0 2023-11-24T06:13:36.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:36.417+0000 7fa35e41c640 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3580b2780 con 0x7fa3580cf7d0 2023-11-24T06:13:36.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:36.418+0000 7fa357fff640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fa358143ae0 0x7fa3580cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:36.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:36.418+0000 7fa357fff640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fa358143ae0 0x7fa3580cf290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:33738/0 (socket says 172.21.15.92:33738) 2023-11-24T06:13:36.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:36.418+0000 7fa357fff640 1 -- 172.21.15.92:0/3018587406 learned_addr learned my addr 172.21.15.92:0/3018587406 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:13:36.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:36.418+0000 7fa357fff640 1 -- 172.21.15.92:0/3018587406 >> v2:172.21.15.118:3300/0 conn(0x7fa3580cf7d0 msgr2=0x7fa35815dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:13:36.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:36.418+0000 7fa357fff640 1 --2- 172.21.15.92:0/3018587406 >> v2:172.21.15.118:3300/0 conn(0x7fa3580cf7d0 0x7fa35815dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:36.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:36.418+0000 7fa357fff640 1 -- 172.21.15.92:0/3018587406 >> v1:172.21.15.118:6789/0 conn(0x7fa35815e1e0 legacy=0x7fa35815a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:13:36.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:36.418+0000 7fa357fff640 1 -- 172.21.15.92:0/3018587406 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa35815e990 con 0x7fa358143ae0 2023-11-24T06:13:36.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:36.419+0000 7fa357fff640 1 --2- 172.21.15.92:0/3018587406 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa358143ae0 0x7fa3580cf290 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7fa3580cb830 tx=0x7fa34802e0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1111b52eff12867 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:36.423 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:36.419+0000 7fa3577fe640 1 -- 172.21.15.92:0/3018587406 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa34802ee90 con 0x7fa358143ae0 2023-11-24T06:13:36.423 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:36.419+0000 7fa3577fe640 1 -- 172.21.15.92:0/3018587406 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa348036440 con 0x7fa358143ae0 2023-11-24T06:13:36.423 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:36.419+0000 7fa3577fe640 1 -- 172.21.15.92:0/3018587406 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3480367d0 con 0x7fa358143ae0 2023-11-24T06:13:36.424 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:36.420+0000 7fa35e41c640 1 -- 172.21.15.92:0/3018587406 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa358143ae0 msgr2=0x7fa3580cf290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:36.424 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:36.420+0000 7fa35e41c640 1 --2- 172.21.15.92:0/3018587406 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa358143ae0 0x7fa3580cf290 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7fa3580cb830 tx=0x7fa34802e0e0 comp rx=0 tx=0).stop 2023-11-24T06:13:36.424 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:36.420+0000 7fa35e41c640 1 -- 172.21.15.92:0/3018587406 shutdown_connections 2023-11-24T06:13:36.424 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:36.420+0000 7fa35e41c640 1 --2- 172.21.15.92:0/3018587406 >> v2:172.21.15.118:3300/0 conn(0x7fa3580cf7d0 0x7fa35815dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:36.424 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:36.420+0000 7fa35e41c640 1 --2- 172.21.15.92:0/3018587406 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa358143ae0 0x7fa3580cf290 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:36.424 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:36.420+0000 7fa35e41c640 1 -- 172.21.15.92:0/3018587406 >> 172.21.15.92:0/3018587406 conn(0x7fa35806a800 msgr2=0x7fa35806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:13:36.424 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:36.420+0000 7fa35e41c640 1 -- 172.21.15.92:0/3018587406 shutdown_connections 2023-11-24T06:13:36.424 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:36.420+0000 7fa35e41c640 1 -- 172.21.15.92:0/3018587406 wait complete. 2023-11-24T06:13:36.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:36.422+0000 7fa35e41c640 1 Processor -- start 2023-11-24T06:13:36.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:36.422+0000 7fa35e41c640 1 -- start start 2023-11-24T06:13:36.426 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:36.423+0000 7fa35e41c640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa3580cf7d0 0x7fa358159540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:36.427 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:36.423+0000 7fa35e41c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa358143ae0 0x7fa3581eec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:36.427 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:36.423+0000 7fa35e41c640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa35815e1e0 0x7fa35815ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:36.427 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:36.423+0000 7fa35e41c640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3581354a0 con 0x7fa358143ae0 2023-11-24T06:13:36.427 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:36.423+0000 7fa35e41c640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa358141670 con 0x7fa3580cf7d0 2023-11-24T06:13:36.427 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:36.423+0000 7fa35e41c640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa358134820 con 0x7fa35815e1e0 2023-11-24T06:13:36.427 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:36.423+0000 7fa34ffff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa358143ae0 0x7fa3581eec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:36.427 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:36.423+0000 7fa34ffff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa358143ae0 0x7fa3581eec70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:33748/0 (socket says 172.21.15.92:33748) 2023-11-24T06:13:36.427 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:36.423+0000 7fa34ffff640 1 -- 172.21.15.92:0/2833565963 learned_addr learned my addr 172.21.15.92:0/2833565963 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:13:36.427 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:36.424+0000 7fa357fff640 1 --2- 172.21.15.92:0/2833565963 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa3580cf7d0 0x7fa358159540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:36.427 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:36.424+0000 7fa34ffff640 1 -- 172.21.15.92:0/2833565963 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa35815e1e0 msgr2=0x7fa35815ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:36.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:36.424+0000 7fa34ffff640 1 --2- 172.21.15.92:0/2833565963 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa35815e1e0 0x7fa35815ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:36.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:36.424+0000 7fa34ffff640 1 -- 172.21.15.92:0/2833565963 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa3580cf7d0 msgr2=0x7fa358159540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:36.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:36.424+0000 7fa34ffff640 1 --2- 172.21.15.92:0/2833565963 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa3580cf7d0 0x7fa358159540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:36.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:36.424+0000 7fa34ffff640 1 -- 172.21.15.92:0/2833565963 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa34800b770 con 0x7fa358143ae0 2023-11-24T06:13:36.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:36.424+0000 7fa34ffff640 1 --2- 172.21.15.92:0/2833565963 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa358143ae0 0x7fa3581eec70 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7fa34000b410 tx=0x7fa34000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:36.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:36.424+0000 7fa3557fa640 1 -- 172.21.15.92:0/2833565963 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa340011830 con 0x7fa358143ae0 2023-11-24T06:13:36.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:36.424+0000 7fa3557fa640 1 -- 172.21.15.92:0/2833565963 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3400119b0 con 0x7fa358143ae0 2023-11-24T06:13:36.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:36.424+0000 7fa3557fa640 1 -- 172.21.15.92:0/2833565963 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa340011d60 con 0x7fa358143ae0 2023-11-24T06:13:36.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:36.424+0000 7fa35e41c640 1 -- 172.21.15.92:0/2833565963 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa358134b00 con 0x7fa358143ae0 2023-11-24T06:13:36.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:36.424+0000 7fa35e41c640 1 -- 172.21.15.92:0/2833565963 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3581350c0 con 0x7fa358143ae0 2023-11-24T06:13:36.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:36.425+0000 7fa35e41c640 1 -- 172.21.15.92:0/2833565963 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3580b2bf0 con 0x7fa358143ae0 2023-11-24T06:13:36.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:36.426+0000 7fa3557fa640 1 -- 172.21.15.92:0/2833565963 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 7) v1 ==== 89333+0+0 (secure 0 0 0) 0x7fa34001a400 con 0x7fa358143ae0 2023-11-24T06:13:36.430 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:36.427+0000 7fa3557fa640 1 --2- 172.21.15.92:0/2833565963 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fa33806af90 0x7fa33806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:36.430 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:36.427+0000 7fa3557fa640 1 -- 172.21.15.92:0/2833565963 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fa340089920 con 0x7fa358143ae0 2023-11-24T06:13:36.430 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:36.427+0000 7fa357fff640 1 --2- 172.21.15.92:0/2833565963 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fa33806af90 0x7fa33806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:36.431 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:36.428+0000 7fa357fff640 1 --2- 172.21.15.92:0/2833565963 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fa33806af90 0x7fa33806d450 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fa348038000 tx=0x7fa348032000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:36.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:36.431+0000 7fa3557fa640 1 -- 172.21.15.92:0/2833565963 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa3580b2bf0 con 0x7fa358143ae0 2023-11-24T06:13:36.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:36.653+0000 7fa35e41c640 1 -- 172.21.15.92:0/2833565963 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fa3581355e0 con 0x7fa358143ae0 2023-11-24T06:13:37.206 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:37.202+0000 7fa3557fa640 1 -- 172.21.15.92:0/2833565963 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v43) v1 ==== 104+0+0 (secure 0 0 0) 0x7fa3581355e0 con 0x7fa358143ae0 2023-11-24T06:13:37.209 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:37.206+0000 7fa35e41c640 1 -- 172.21.15.92:0/2833565963 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fa33806af90 msgr2=0x7fa33806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:37.209 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:37.206+0000 7fa35e41c640 1 --2- 172.21.15.92:0/2833565963 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fa33806af90 0x7fa33806d450 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fa348038000 tx=0x7fa348032000 comp rx=0 tx=0).stop 2023-11-24T06:13:37.209 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:37.206+0000 7fa35e41c640 1 -- 172.21.15.92:0/2833565963 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa358143ae0 msgr2=0x7fa3581eec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:37.209 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:37.206+0000 7fa35e41c640 1 --2- 172.21.15.92:0/2833565963 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa358143ae0 0x7fa3581eec70 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7fa34000b410 tx=0x7fa34000b8d0 comp rx=0 tx=0).stop 2023-11-24T06:13:37.209 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:37.206+0000 7fa35e41c640 1 -- 172.21.15.92:0/2833565963 shutdown_connections 2023-11-24T06:13:37.209 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:37.206+0000 7fa35e41c640 1 --2- 172.21.15.92:0/2833565963 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fa33806af90 0x7fa33806d450 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:37.210 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:37.206+0000 7fa35e41c640 1 --2- 172.21.15.92:0/2833565963 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fa35815e1e0 0x7fa35815ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:37.210 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:37.206+0000 7fa35e41c640 1 --2- 172.21.15.92:0/2833565963 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa358143ae0 0x7fa3581eec70 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:37.210 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:37.206+0000 7fa35e41c640 1 --2- 172.21.15.92:0/2833565963 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa3580cf7d0 0x7fa358159540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:37.210 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:37.206+0000 7fa35e41c640 1 -- 172.21.15.92:0/2833565963 >> 172.21.15.92:0/2833565963 conn(0x7fa35806a800 msgr2=0x7fa35815ae50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:13:37.210 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:37.206+0000 7fa35e41c640 1 -- 172.21.15.92:0/2833565963 shutdown_connections 2023-11-24T06:13:37.210 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:37.206+0000 7fa35e41c640 1 -- 172.21.15.92:0/2833565963 wait complete. 2023-11-24T06:13:37.230 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-11-24T06:13:37.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:37.431+0000 7fb61eb97640 1 Processor -- start 2023-11-24T06:13:37.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:37.431+0000 7fb61eb97640 1 -- start start 2023-11-24T06:13:37.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:37.432+0000 7fb61eb97640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb61813df80 0x7fb61815bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:37.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:37.432+0000 7fb61eb97640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fb61813e360 0x7fb6180cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:37.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:37.433+0000 7fb61eb97640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb61806c470 con 0x7fb61815f9c0 2023-11-24T06:13:37.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:37.433+0000 7fb61eb97640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb61806c750 con 0x7fb61813df80 2023-11-24T06:13:37.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:37.433+0000 7fb61eb97640 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb61806ca30 con 0x7fb61813e360 2023-11-24T06:13:37.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:37.433+0000 7fb61c933640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb61813df80 0x7fb61815bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:37.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:37.433+0000 7fb61d134640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fb61815f9c0 0x7fb6181408d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:33618/0 (socket says 172.21.15.92:33618) 2023-11-24T06:13:37.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:37.433+0000 7fb61d134640 1 -- 172.21.15.92:0/2997753527 learned_addr learned my addr 172.21.15.92:0/2997753527 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-24T06:13:37.437 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:37.433+0000 7fb61c933640 1 -- 172.21.15.92:0/2997753527 >> v2:172.21.15.118:3300/0 conn(0x7fb61813e360 msgr2=0x7fb6180cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:13:37.437 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:37.433+0000 7fb617fff640 1 --2- 172.21.15.92:0/2997753527 >> v2:172.21.15.118:3300/0 conn(0x7fb61813e360 0x7fb6180cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:37.437 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:37.433+0000 7fb61c933640 1 --2- 172.21.15.92:0/2997753527 >> v2:172.21.15.118:3300/0 conn(0x7fb61813e360 0x7fb6180cf290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:37.437 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:37.433+0000 7fb61c933640 1 -- 172.21.15.92:0/2997753527 >> v1:172.21.15.92:6789/0 conn(0x7fb61815f9c0 legacy=0x7fb6181408d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:37.437 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:37.433+0000 7fb61c933640 1 -- 172.21.15.92:0/2997753527 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb61806cd10 con 0x7fb61813df80 2023-11-24T06:13:37.438 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:37.433+0000 7fb617fff640 1 --2- 172.21.15.92:0/2997753527 >> v2:172.21.15.118:3300/0 conn(0x7fb61813e360 0x7fb6180cf290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:13:37.438 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:37.433+0000 7fb6177fe640 1 -- 172.21.15.92:0/2997753527 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2842462596 0 0) 0x7fb61806c470 con 0x7fb61815f9c0 2023-11-24T06:13:37.438 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:37.434+0000 7fb61c933640 1 --2- 172.21.15.92:0/2997753527 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb61813df80 0x7fb61815bee0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7fb61806ded0 tx=0x7fb60c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7582d967ed4f071 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:37.439 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:37.435+0000 7fb6177fe640 1 -- 172.21.15.92:0/2997753527 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb60c031830 con 0x7fb61813df80 2023-11-24T06:13:37.439 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:37.435+0000 7fb6177fe640 1 -- 172.21.15.92:0/2997753527 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb60c031990 con 0x7fb61813df80 2023-11-24T06:13:37.439 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:37.435+0000 7fb6177fe640 1 -- 172.21.15.92:0/2997753527 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb60c031d20 con 0x7fb61813df80 2023-11-24T06:13:37.439 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:37.435+0000 7fb61eb97640 1 -- 172.21.15.92:0/2997753527 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb61813df80 msgr2=0x7fb61815bee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:37.439 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:37.435+0000 7fb61eb97640 1 --2- 172.21.15.92:0/2997753527 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb61813df80 0x7fb61815bee0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7fb61806ded0 tx=0x7fb60c02b8d0 comp rx=0 tx=0).stop 2023-11-24T06:13:37.439 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:37.435+0000 7fb61eb97640 1 -- 172.21.15.92:0/2997753527 shutdown_connections 2023-11-24T06:13:37.439 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:37.435+0000 7fb61eb97640 1 --2- 172.21.15.92:0/2997753527 >> v2:172.21.15.118:3300/0 conn(0x7fb61813e360 0x7fb6180cf290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:37.439 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:37.435+0000 7fb61eb97640 1 --2- 172.21.15.92:0/2997753527 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb61813df80 0x7fb61815bee0 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:37.439 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:37.435+0000 7fb61eb97640 1 -- 172.21.15.92:0/2997753527 >> 172.21.15.92:0/2997753527 conn(0x7fb61806a800 msgr2=0x7fb61806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:13:37.439 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:37.435+0000 7fb61eb97640 1 -- 172.21.15.92:0/2997753527 shutdown_connections 2023-11-24T06:13:37.439 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:37.435+0000 7fb61eb97640 1 -- 172.21.15.92:0/2997753527 wait complete. 2023-11-24T06:13:37.440 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:37.437+0000 7fb61eb97640 1 Processor -- start 2023-11-24T06:13:37.440 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:37.437+0000 7fb61eb97640 1 -- start start 2023-11-24T06:13:37.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:37.438+0000 7fb61eb97640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb61813df80 0x7fb6181f40e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:37.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:37.438+0000 7fb61eb97640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb61813e360 0x7fb6181f05a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:37.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:37.438+0000 7fb61eb97640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb61815f9c0 0x7fb6180b22e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:37.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:37.438+0000 7fb61eb97640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6181f4f60 con 0x7fb61813e360 2023-11-24T06:13:37.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:37.438+0000 7fb61eb97640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6180b2820 con 0x7fb61813df80 2023-11-24T06:13:37.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:37.438+0000 7fb61eb97640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6181f46c0 con 0x7fb61815f9c0 2023-11-24T06:13:37.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:37.438+0000 7fb617fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb61813e360 0x7fb6181f05a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:37.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:37.438+0000 7fb617fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb61813e360 0x7fb6181f05a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:33764/0 (socket says 172.21.15.92:33764) 2023-11-24T06:13:37.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:37.438+0000 7fb617fff640 1 -- 172.21.15.92:0/2831400709 learned_addr learned my addr 172.21.15.92:0/2831400709 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:13:37.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:37.439+0000 7fb61c933640 1 --2- 172.21.15.92:0/2831400709 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb61813df80 0x7fb6181f40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:37.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:37.439+0000 7fb617fff640 1 -- 172.21.15.92:0/2831400709 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb61815f9c0 msgr2=0x7fb6180b22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:37.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:37.439+0000 7fb61d134640 1 --2- 172.21.15.92:0/2831400709 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb61815f9c0 0x7fb6180b22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:37.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:37.439+0000 7fb617fff640 1 --2- 172.21.15.92:0/2831400709 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb61815f9c0 0x7fb6180b22e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:37.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:37.439+0000 7fb617fff640 1 -- 172.21.15.92:0/2831400709 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb61813df80 msgr2=0x7fb6181f40e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:37.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:37.439+0000 7fb617fff640 1 --2- 172.21.15.92:0/2831400709 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb61813df80 0x7fb6181f40e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:37.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:37.439+0000 7fb617fff640 1 -- 172.21.15.92:0/2831400709 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb60c009400 con 0x7fb61813e360 2023-11-24T06:13:37.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:37.439+0000 7fb61c933640 1 --2- 172.21.15.92:0/2831400709 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb61813df80 0x7fb6181f40e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:13:37.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:37.439+0000 7fb617fff640 1 --2- 172.21.15.92:0/2831400709 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb61813e360 0x7fb6181f05a0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7fb60400c7f0 tx=0x7fb60400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:37.444 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:37.439+0000 7fb6157fa640 1 -- 172.21.15.92:0/2831400709 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb60400cea0 con 0x7fb61813e360 2023-11-24T06:13:37.444 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:37.439+0000 7fb6157fa640 1 -- 172.21.15.92:0/2831400709 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb604004230 con 0x7fb61813e360 2023-11-24T06:13:37.444 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:37.439+0000 7fb6157fa640 1 -- 172.21.15.92:0/2831400709 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb604007830 con 0x7fb61813e360 2023-11-24T06:13:37.444 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:37.439+0000 7fb61eb97640 1 -- 172.21.15.92:0/2831400709 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6181f49a0 con 0x7fb61813e360 2023-11-24T06:13:37.444 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:37.439+0000 7fb61eb97640 1 -- 172.21.15.92:0/2831400709 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6181f0c80 con 0x7fb61813e360 2023-11-24T06:13:37.444 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:37.440+0000 7fb61eb97640 1 -- 172.21.15.92:0/2831400709 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6181f0ae0 con 0x7fb61813e360 2023-11-24T06:13:37.444 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:37.442+0000 7fb6157fa640 1 -- 172.21.15.92:0/2831400709 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 7) v1 ==== 89333+0+0 (secure 0 0 0) 0x7fb604007990 con 0x7fb61813e360 2023-11-24T06:13:37.445 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:37.443+0000 7fb6157fa640 1 --2- 172.21.15.92:0/2831400709 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fb5fc06af60 0x7fb5fc06d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:37.445 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:37.443+0000 7fb6157fa640 1 -- 172.21.15.92:0/2831400709 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fb6040898a0 con 0x7fb61813e360 2023-11-24T06:13:37.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:37.443+0000 7fb61c933640 1 --2- 172.21.15.92:0/2831400709 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fb5fc06af60 0x7fb5fc06d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:37.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:37.444+0000 7fb61c933640 1 --2- 172.21.15.92:0/2831400709 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fb5fc06af60 0x7fb5fc06d420 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fb60c036000 tx=0x7fb60c03c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:37.450 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:37.447+0000 7fb6157fa640 1 -- 172.21.15.92:0/2831400709 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb6181f0ae0 con 0x7fb61813e360 2023-11-24T06:13:37.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:37.622+0000 7fb61eb97640 1 -- 172.21.15.92:0/2831400709 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fb6181f0ae0 con 0x7fb61813e360 2023-11-24T06:13:38.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:38.212+0000 7fb6157fa640 1 -- 172.21.15.92:0/2831400709 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v65) v1 ==== 181+0+0 (secure 0 0 0) 0x7fb6181f0ae0 con 0x7fb61813e360 2023-11-24T06:13:38.216 INFO:teuthology.orchestra.run.smithi092.stderr:pool 'cephfs_metadata' removed 2023-11-24T06:13:38.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:38.217+0000 7fb61eb97640 1 -- 172.21.15.92:0/2831400709 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fb5fc06af60 msgr2=0x7fb5fc06d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:38.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:38.217+0000 7fb61eb97640 1 --2- 172.21.15.92:0/2831400709 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fb5fc06af60 0x7fb5fc06d420 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fb60c036000 tx=0x7fb60c03c040 comp rx=0 tx=0).stop 2023-11-24T06:13:38.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:38.217+0000 7fb61eb97640 1 -- 172.21.15.92:0/2831400709 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb61813e360 msgr2=0x7fb6181f05a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:38.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:38.217+0000 7fb61eb97640 1 --2- 172.21.15.92:0/2831400709 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb61813e360 0x7fb6181f05a0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7fb60400c7f0 tx=0x7fb60400ccb0 comp rx=0 tx=0).stop 2023-11-24T06:13:38.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:38.218+0000 7fb61eb97640 1 -- 172.21.15.92:0/2831400709 shutdown_connections 2023-11-24T06:13:38.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:38.218+0000 7fb61eb97640 1 --2- 172.21.15.92:0/2831400709 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fb5fc06af60 0x7fb5fc06d420 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:38.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:38.218+0000 7fb61eb97640 1 --2- 172.21.15.92:0/2831400709 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb61815f9c0 0x7fb6180b22e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:38.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:38.218+0000 7fb61eb97640 1 --2- 172.21.15.92:0/2831400709 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb61813e360 0x7fb6181f05a0 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:38.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:38.218+0000 7fb61eb97640 1 --2- 172.21.15.92:0/2831400709 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb61813df80 0x7fb6181f40e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:38.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:38.218+0000 7fb61eb97640 1 -- 172.21.15.92:0/2831400709 >> 172.21.15.92:0/2831400709 conn(0x7fb61806a800 msgr2=0x7fb61815e480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:13:38.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:38.218+0000 7fb61eb97640 1 -- 172.21.15.92:0/2831400709 shutdown_connections 2023-11-24T06:13:38.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:38.218+0000 7fb61eb97640 1 -- 172.21.15.92:0/2831400709 wait complete. 2023-11-24T06:13:38.243 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-11-24T06:13:38.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:38.452+0000 7f6ce395a640 1 Processor -- start 2023-11-24T06:13:38.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:38.452+0000 7f6ce395a640 1 -- start start 2023-11-24T06:13:38.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:38.453+0000 7f6ce395a640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f6cdc13e360 0x7f6cdc0cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:38.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:38.453+0000 7f6ce395a640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6cdc15f9c0 0x7f6cdc0cf7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:38.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:38.453+0000 7f6ce395a640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6cdc15c5f0 con 0x7f6cdc13df80 2023-11-24T06:13:38.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:38.453+0000 7f6ce395a640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6cdc06c470 con 0x7f6cdc13e360 2023-11-24T06:13:38.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:38.453+0000 7f6ce395a640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cdc06c750 con 0x7f6cdc15f9c0 2023-11-24T06:13:38.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:38.454+0000 7f6ce0ef5640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f6cdc13e360 0x7f6cdc0cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:38.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:38.454+0000 7f6ce0ef5640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f6cdc13e360 0x7f6cdc0cf290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:33766/0 (socket says 172.21.15.92:33766) 2023-11-24T06:13:38.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:38.454+0000 7f6ce0ef5640 1 -- 172.21.15.92:0/1583821087 learned_addr learned my addr 172.21.15.92:0/1583821087 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:13:38.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:38.454+0000 7f6ce0ef5640 1 -- 172.21.15.92:0/1583821087 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6cdc15f9c0 msgr2=0x7f6cdc0cf7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:13:38.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:38.454+0000 7f6ce0ef5640 1 --2- 172.21.15.92:0/1583821087 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6cdc15f9c0 0x7f6cdc0cf7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:38.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:38.454+0000 7f6ce0ef5640 1 -- 172.21.15.92:0/1583821087 >> v1:172.21.15.92:6789/0 conn(0x7f6cdc13df80 legacy=0x7f6cdc15bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:38.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:38.454+0000 7f6ce0ef5640 1 -- 172.21.15.92:0/1583821087 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cdc06ca30 con 0x7f6cdc13e360 2023-11-24T06:13:38.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:38.455+0000 7f6ce0ef5640 1 --2- 172.21.15.92:0/1583821087 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6cdc13e360 0x7f6cdc0cf290 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f6ccc009870 tx=0x7f6ccc02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb17a3d72daf6fe5 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:38.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:38.456+0000 7f6cd3fff640 1 -- 172.21.15.92:0/1583821087 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ccc030840 con 0x7f6cdc13e360 2023-11-24T06:13:38.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:38.456+0000 7f6cd3fff640 1 -- 172.21.15.92:0/1583821087 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ccc0309a0 con 0x7f6cdc13e360 2023-11-24T06:13:38.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:38.456+0000 7f6cd3fff640 1 -- 172.21.15.92:0/1583821087 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ccc030d30 con 0x7f6cdc13e360 2023-11-24T06:13:38.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:38.456+0000 7f6ce395a640 1 -- 172.21.15.92:0/1583821087 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6cdc13e360 msgr2=0x7f6cdc0cf290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:38.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:38.456+0000 7f6ce395a640 1 --2- 172.21.15.92:0/1583821087 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6cdc13e360 0x7f6cdc0cf290 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f6ccc009870 tx=0x7f6ccc02b980 comp rx=0 tx=0).stop 2023-11-24T06:13:38.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:38.456+0000 7f6ce395a640 1 -- 172.21.15.92:0/1583821087 shutdown_connections 2023-11-24T06:13:38.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:38.456+0000 7f6ce395a640 1 --2- 172.21.15.92:0/1583821087 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6cdc15f9c0 0x7f6cdc0cf7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:38.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:38.456+0000 7f6ce395a640 1 --2- 172.21.15.92:0/1583821087 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6cdc13e360 0x7f6cdc0cf290 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:38.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:38.456+0000 7f6ce395a640 1 -- 172.21.15.92:0/1583821087 >> 172.21.15.92:0/1583821087 conn(0x7f6cdc06a800 msgr2=0x7f6cdc06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:13:38.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:38.457+0000 7f6ce395a640 1 -- 172.21.15.92:0/1583821087 shutdown_connections 2023-11-24T06:13:38.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:38.457+0000 7f6ce395a640 1 -- 172.21.15.92:0/1583821087 wait complete. 2023-11-24T06:13:38.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:38.459+0000 7f6ce395a640 1 Processor -- start 2023-11-24T06:13:38.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:38.459+0000 7f6ce395a640 1 -- start start 2023-11-24T06:13:38.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:38.460+0000 7f6ce395a640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6cdc13df80 0x7f6cdc152af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:38.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:38.460+0000 7f6ce395a640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6cdc13e360 0x7f6cdc0cdd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:38.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:38.460+0000 7f6ce395a640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6cdc15f9c0 0x7f6cdc0ce280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:38.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:38.460+0000 7f6ce395a640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cdc1f33c0 con 0x7f6cdc13df80 2023-11-24T06:13:38.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:38.460+0000 7f6ce395a640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cdc1f36a0 con 0x7f6cdc13e360 2023-11-24T06:13:38.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:38.460+0000 7f6ce395a640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cdc1f3980 con 0x7f6cdc15f9c0 2023-11-24T06:13:38.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:38.460+0000 7f6ce16f6640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6cdc13df80 0x7f6cdc152af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:38.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:38.460+0000 7f6ce16f6640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6cdc13df80 0x7f6cdc152af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:33772/0 (socket says 172.21.15.92:33772) 2023-11-24T06:13:38.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:38.460+0000 7f6ce16f6640 1 -- 172.21.15.92:0/4121136152 learned_addr learned my addr 172.21.15.92:0/4121136152 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:13:38.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:38.461+0000 7f6ce16f6640 1 -- 172.21.15.92:0/4121136152 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6cdc15f9c0 msgr2=0x7f6cdc0ce280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:38.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:38.461+0000 7f6ce16f6640 1 --2- 172.21.15.92:0/4121136152 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6cdc15f9c0 0x7f6cdc0ce280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:38.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:38.461+0000 7f6ce16f6640 1 -- 172.21.15.92:0/4121136152 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6cdc13e360 msgr2=0x7f6cdc0cdd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:38.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:38.461+0000 7f6ce16f6640 1 --2- 172.21.15.92:0/4121136152 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6cdc13e360 0x7f6cdc0cdd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:38.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:38.461+0000 7f6ce16f6640 1 -- 172.21.15.92:0/4121136152 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ccc0094d0 con 0x7f6cdc13df80 2023-11-24T06:13:38.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:38.461+0000 7f6ce16f6640 1 --2- 172.21.15.92:0/4121136152 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6cdc13df80 0x7f6cdc152af0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f6cd800c390 tx=0x7f6cd800c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:38.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:38.461+0000 7f6cd1ffb640 1 -- 172.21.15.92:0/4121136152 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cd80052d0 con 0x7f6cdc13df80 2023-11-24T06:13:38.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:38.461+0000 7f6cd1ffb640 1 -- 172.21.15.92:0/4121136152 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cd8005430 con 0x7f6cdc13df80 2023-11-24T06:13:38.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:38.461+0000 7f6ce395a640 1 -- 172.21.15.92:0/4121136152 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6cdc1f3c60 con 0x7f6cdc13df80 2023-11-24T06:13:38.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:38.461+0000 7f6cd1ffb640 1 -- 172.21.15.92:0/4121136152 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cd8003830 con 0x7f6cdc13df80 2023-11-24T06:13:38.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:38.461+0000 7f6ce395a640 1 -- 172.21.15.92:0/4121136152 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cdc153030 con 0x7f6cdc13df80 2023-11-24T06:13:38.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:38.462+0000 7f6ce395a640 1 -- 172.21.15.92:0/4121136152 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6cdc0675d0 con 0x7f6cdc13df80 2023-11-24T06:13:38.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:38.463+0000 7f6cd1ffb640 1 -- 172.21.15.92:0/4121136152 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 7) v1 ==== 89333+0+0 (secure 0 0 0) 0x7f6cd8003990 con 0x7f6cdc13df80 2023-11-24T06:13:38.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:38.464+0000 7f6cd1ffb640 1 --2- 172.21.15.92:0/4121136152 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f6cc006af60 0x7f6cc006d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:38.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:38.464+0000 7f6cd1ffb640 1 -- 172.21.15.92:0/4121136152 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5357+0+0 (secure 0 0 0) 0x7f6cd8088c60 con 0x7f6cdc13df80 2023-11-24T06:13:38.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:38.465+0000 7f6ce0ef5640 1 --2- 172.21.15.92:0/4121136152 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f6cc006af60 0x7f6cc006d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:38.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:38.465+0000 7f6ce0ef5640 1 --2- 172.21.15.92:0/4121136152 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f6cc006af60 0x7f6cc006d420 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f6ccc036000 tx=0x7f6ccc03b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:38.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:38.469+0000 7f6cd1ffb640 1 -- 172.21.15.92:0/4121136152 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6cd8053cc0 con 0x7f6cdc13df80 2023-11-24T06:13:38.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:38.670+0000 7f6ce395a640 1 -- 172.21.15.92:0/4121136152 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f6cdc0cc100 con 0x7f6cdc13df80 2023-11-24T06:13:39.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:39.224+0000 7f6cd1ffb640 1 -- 172.21.15.92:0/4121136152 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v66) v1 ==== 169+0+0 (secure 0 0 0) 0x7f6cd80170d0 con 0x7f6cdc13df80 2023-11-24T06:13:39.227 INFO:teuthology.orchestra.run.smithi092.stderr:pool 'cephfs_data' removed 2023-11-24T06:13:39.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:39.228+0000 7f6ce395a640 1 -- 172.21.15.92:0/4121136152 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f6cc006af60 msgr2=0x7f6cc006d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:39.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:39.228+0000 7f6ce395a640 1 --2- 172.21.15.92:0/4121136152 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f6cc006af60 0x7f6cc006d420 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f6ccc036000 tx=0x7f6ccc03b040 comp rx=0 tx=0).stop 2023-11-24T06:13:39.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:39.228+0000 7f6ce395a640 1 -- 172.21.15.92:0/4121136152 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6cdc13df80 msgr2=0x7f6cdc152af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:39.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:39.228+0000 7f6ce395a640 1 --2- 172.21.15.92:0/4121136152 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6cdc13df80 0x7f6cdc152af0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f6cd800c390 tx=0x7f6cd800c850 comp rx=0 tx=0).stop 2023-11-24T06:13:39.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:39.228+0000 7f6ce395a640 1 -- 172.21.15.92:0/4121136152 shutdown_connections 2023-11-24T06:13:39.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:39.228+0000 7f6ce395a640 1 --2- 172.21.15.92:0/4121136152 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f6cc006af60 0x7f6cc006d420 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:39.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:39.228+0000 7f6ce395a640 1 --2- 172.21.15.92:0/4121136152 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6cdc15f9c0 0x7f6cdc0ce280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:39.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:39.228+0000 7f6ce395a640 1 --2- 172.21.15.92:0/4121136152 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6cdc13e360 0x7f6cdc0cdd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:39.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:39.228+0000 7f6ce395a640 1 --2- 172.21.15.92:0/4121136152 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6cdc13df80 0x7f6cdc152af0 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:39.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:39.228+0000 7f6ce395a640 1 -- 172.21.15.92:0/4121136152 >> 172.21.15.92:0/4121136152 conn(0x7f6cdc06a800 msgr2=0x7f6cdc153c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:13:39.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:39.230+0000 7f6ce395a640 1 -- 172.21.15.92:0/4121136152 shutdown_connections 2023-11-24T06:13:39.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:39.230+0000 7f6ce395a640 1 -- 172.21.15.92:0/4121136152 wait complete. 2023-11-24T06:13:39.255 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-11-24T06:13:39.451 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:39.448+0000 7fb48fe51640 1 Processor -- start 2023-11-24T06:13:39.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:39.448+0000 7fb48fe51640 1 -- start start 2023-11-24T06:13:39.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:39.450+0000 7fb48fe51640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb488143ae0 0x7fb4880cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:39.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:39.450+0000 7fb48fe51640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fb4880cf7d0 0x7fb48815dca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:39.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:39.450+0000 7fb48fe51640 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4880cfc30 con 0x7fb48815e1e0 2023-11-24T06:13:39.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:39.450+0000 7fb48fe51640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4880b1ee0 con 0x7fb488143ae0 2023-11-24T06:13:39.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:39.450+0000 7fb48fe51640 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4880b2780 con 0x7fb4880cf7d0 2023-11-24T06:13:39.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:39.450+0000 7fb48dbed640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb488143ae0 0x7fb4880cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:39.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:39.450+0000 7fb48dbed640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb488143ae0 0x7fb4880cf290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47734/0 (socket says 172.21.15.92:47734) 2023-11-24T06:13:39.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:39.450+0000 7fb48dbed640 1 -- 172.21.15.92:0/258233805 learned_addr learned my addr 172.21.15.92:0/258233805 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:13:39.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:39.450+0000 7fb48dbed640 1 -- 172.21.15.92:0/258233805 >> v2:172.21.15.118:3300/0 conn(0x7fb4880cf7d0 msgr2=0x7fb48815dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:13:39.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:39.450+0000 7fb48dbed640 1 --2- 172.21.15.92:0/258233805 >> v2:172.21.15.118:3300/0 conn(0x7fb4880cf7d0 0x7fb48815dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:39.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:39.450+0000 7fb48dbed640 1 -- 172.21.15.92:0/258233805 >> v1:172.21.15.118:6789/0 conn(0x7fb48815e1e0 legacy=0x7fb48815a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:13:39.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:39.450+0000 7fb48dbed640 1 -- 172.21.15.92:0/258233805 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb48815e990 con 0x7fb488143ae0 2023-11-24T06:13:39.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:39.451+0000 7fb48dbed640 1 --2- 172.21.15.92:0/258233805 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb488143ae0 0x7fb4880cf290 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7fb4880cb830 tx=0x7fb48402e0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d5509a21e8632d server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:39.456 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:39.452+0000 7fb48cbeb640 1 -- 172.21.15.92:0/258233805 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb48402ee90 con 0x7fb488143ae0 2023-11-24T06:13:39.456 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:39.452+0000 7fb48cbeb640 1 -- 172.21.15.92:0/258233805 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb484036440 con 0x7fb488143ae0 2023-11-24T06:13:39.456 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:39.452+0000 7fb48cbeb640 1 -- 172.21.15.92:0/258233805 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4840367d0 con 0x7fb488143ae0 2023-11-24T06:13:39.456 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:39.452+0000 7fb48fe51640 1 -- 172.21.15.92:0/258233805 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb488143ae0 msgr2=0x7fb4880cf290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:39.456 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:39.452+0000 7fb48fe51640 1 --2- 172.21.15.92:0/258233805 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb488143ae0 0x7fb4880cf290 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7fb4880cb830 tx=0x7fb48402e0e0 comp rx=0 tx=0).stop 2023-11-24T06:13:39.456 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:39.452+0000 7fb48fe51640 1 -- 172.21.15.92:0/258233805 shutdown_connections 2023-11-24T06:13:39.456 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:39.452+0000 7fb48fe51640 1 --2- 172.21.15.92:0/258233805 >> v2:172.21.15.118:3300/0 conn(0x7fb4880cf7d0 0x7fb48815dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:39.456 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:39.452+0000 7fb48fe51640 1 --2- 172.21.15.92:0/258233805 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb488143ae0 0x7fb4880cf290 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:39.456 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:39.452+0000 7fb48fe51640 1 -- 172.21.15.92:0/258233805 >> 172.21.15.92:0/258233805 conn(0x7fb48806a800 msgr2=0x7fb48806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:13:39.456 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:39.453+0000 7fb48fe51640 1 -- 172.21.15.92:0/258233805 shutdown_connections 2023-11-24T06:13:39.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:39.453+0000 7fb48fe51640 1 -- 172.21.15.92:0/258233805 wait complete. 2023-11-24T06:13:39.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:39.454+0000 7fb48fe51640 1 Processor -- start 2023-11-24T06:13:39.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:39.454+0000 7fb48fe51640 1 -- start start 2023-11-24T06:13:39.459 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:39.455+0000 7fb48fe51640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb4880cf7d0 0x7fb488159a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:39.459 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:39.455+0000 7fb48fe51640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb488143ae0 0x7fb4880c4000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:39.459 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:39.455+0000 7fb48fe51640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb48815e1e0 0x7fb48815ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:39.459 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:39.455+0000 7fb48fe51640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4881f61b0 con 0x7fb4880cf7d0 2023-11-24T06:13:39.459 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:39.455+0000 7fb48fe51640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4881f6490 con 0x7fb488143ae0 2023-11-24T06:13:39.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:39.455+0000 7fb48fe51640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4881f6770 con 0x7fb48815e1e0 2023-11-24T06:13:39.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:39.456+0000 7fb48dbed640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb4880cf7d0 0x7fb488159a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:39.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:39.456+0000 7fb48dbed640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb4880cf7d0 0x7fb488159a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47744/0 (socket says 172.21.15.92:47744) 2023-11-24T06:13:39.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:39.456+0000 7fb48dbed640 1 -- 172.21.15.92:0/299438952 learned_addr learned my addr 172.21.15.92:0/299438952 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:13:39.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:39.456+0000 7fb48dbed640 1 -- 172.21.15.92:0/299438952 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb48815e1e0 msgr2=0x7fb48815ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:39.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:39.456+0000 7fb48dbed640 1 --2- 172.21.15.92:0/299438952 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb48815e1e0 0x7fb48815ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:39.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:39.456+0000 7fb48dbed640 1 -- 172.21.15.92:0/299438952 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb488143ae0 msgr2=0x7fb4880c4000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:39.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:39.456+0000 7fb48dbed640 1 --2- 172.21.15.92:0/299438952 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb488143ae0 0x7fb4880c4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:39.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:39.456+0000 7fb48dbed640 1 -- 172.21.15.92:0/299438952 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb48400b770 con 0x7fb4880cf7d0 2023-11-24T06:13:39.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:39.456+0000 7fb48dbed640 1 --2- 172.21.15.92:0/299438952 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb4880cf7d0 0x7fb488159a90 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7fb48402e120 tx=0x7fb484004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:39.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:39.457+0000 7fb47e7fc640 1 -- 172.21.15.92:0/299438952 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb484034950 con 0x7fb4880cf7d0 2023-11-24T06:13:39.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:39.457+0000 7fb47e7fc640 1 -- 172.21.15.92:0/299438952 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb484034ab0 con 0x7fb4880cf7d0 2023-11-24T06:13:39.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:39.457+0000 7fb48fe51640 1 -- 172.21.15.92:0/299438952 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4881f6a50 con 0x7fb4880cf7d0 2023-11-24T06:13:39.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:39.457+0000 7fb47e7fc640 1 -- 172.21.15.92:0/299438952 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb484034e60 con 0x7fb4880cf7d0 2023-11-24T06:13:39.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:39.457+0000 7fb48fe51640 1 -- 172.21.15.92:0/299438952 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb488159fd0 con 0x7fb4880cf7d0 2023-11-24T06:13:39.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:39.458+0000 7fb48fe51640 1 -- 172.21.15.92:0/299438952 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4881f6f70 con 0x7fb4880cf7d0 2023-11-24T06:13:39.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:39.459+0000 7fb47e7fc640 1 -- 172.21.15.92:0/299438952 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 7) v1 ==== 89333+0+0 (secure 0 0 0) 0x7fb484041480 con 0x7fb4880cf7d0 2023-11-24T06:13:39.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:39.460+0000 7fb47e7fc640 1 --2- 172.21.15.92:0/299438952 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fb46c06af90 0x7fb46c06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:39.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:39.460+0000 7fb47e7fc640 1 -- 172.21.15.92:0/299438952 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fb4840ac7b0 con 0x7fb4880cf7d0 2023-11-24T06:13:39.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:39.460+0000 7fb48d3ec640 1 --2- 172.21.15.92:0/299438952 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fb46c06af90 0x7fb46c06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:39.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:39.461+0000 7fb48d3ec640 1 --2- 172.21.15.92:0/299438952 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fb46c06af90 0x7fb46c06d450 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fb478009670 tx=0x7fb478006bf0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:39.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:39.465+0000 7fb47e7fc640 1 -- 172.21.15.92:0/299438952 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb4881f6f70 con 0x7fb4880cf7d0 2023-11-24T06:13:39.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:39.658+0000 7fb48fe51640 1 -- 172.21.15.92:0/299438952 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fb4881f6f70 con 0x7fb4880cf7d0 2023-11-24T06:13:39.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:39.665+0000 7fb47e7fc640 1 -- 172.21.15.92:0/299438952 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v15) v1 ==== 104+0+0 (secure 0 0 0) 0x7fb4881f6f70 con 0x7fb4880cf7d0 2023-11-24T06:13:39.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:39.667+0000 7fb48fe51640 1 -- 172.21.15.92:0/299438952 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fb46c06af90 msgr2=0x7fb46c06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:39.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:39.667+0000 7fb48fe51640 1 --2- 172.21.15.92:0/299438952 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fb46c06af90 0x7fb46c06d450 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fb478009670 tx=0x7fb478006bf0 comp rx=0 tx=0).stop 2023-11-24T06:13:39.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:39.667+0000 7fb48fe51640 1 -- 172.21.15.92:0/299438952 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb4880cf7d0 msgr2=0x7fb488159a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:39.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:39.667+0000 7fb48fe51640 1 --2- 172.21.15.92:0/299438952 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb4880cf7d0 0x7fb488159a90 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7fb48402e120 tx=0x7fb484004730 comp rx=0 tx=0).stop 2023-11-24T06:13:39.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:39.668+0000 7fb48fe51640 1 -- 172.21.15.92:0/299438952 shutdown_connections 2023-11-24T06:13:39.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:39.668+0000 7fb48fe51640 1 --2- 172.21.15.92:0/299438952 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fb46c06af90 0x7fb46c06d450 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:39.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:39.668+0000 7fb48fe51640 1 --2- 172.21.15.92:0/299438952 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb48815e1e0 0x7fb48815ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:39.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:39.668+0000 7fb48fe51640 1 --2- 172.21.15.92:0/299438952 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb488143ae0 0x7fb4880c4000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:39.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:39.668+0000 7fb48fe51640 1 --2- 172.21.15.92:0/299438952 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb4880cf7d0 0x7fb488159a90 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:39.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:39.668+0000 7fb48fe51640 1 -- 172.21.15.92:0/299438952 >> 172.21.15.92:0/299438952 conn(0x7fb48806a800 msgr2=0x7fb48806abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:13:39.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:39.668+0000 7fb48fe51640 1 -- 172.21.15.92:0/299438952 shutdown_connections 2023-11-24T06:13:39.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:39.668+0000 7fb48fe51640 1 -- 172.21.15.92:0/299438952 wait complete. 2023-11-24T06:13:39.688 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestVolumes.test_volume_info_pending_subvol_deletions' 2023-11-24T06:13:39.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:39.900+0000 7fba3f301640 1 Processor -- start 2023-11-24T06:13:39.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:39.900+0000 7fba3f301640 1 -- start start 2023-11-24T06:13:39.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:39.902+0000 7fba3f301640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fba3813e360 0x7fba380cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:39.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:39.902+0000 7fba3f301640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba3815c5f0 con 0x7fba3815f9c0 2023-11-24T06:13:39.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:39.902+0000 7fba3f301640 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba3806c470 con 0x7fba3813df80 2023-11-24T06:13:39.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:39.902+0000 7fba3f301640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba3806c750 con 0x7fba3813e360 2023-11-24T06:13:39.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:39.902+0000 7fba3d89e640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fba3815f9c0 0x7fba380cd840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:50316/0 (socket says 172.21.15.92:50316) 2023-11-24T06:13:39.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:39.902+0000 7fba3d89e640 1 -- 172.21.15.92:0/4269075696 learned_addr learned my addr 172.21.15.92:0/4269075696 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-24T06:13:39.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:39.902+0000 7fba3c89c640 1 --2- 172.21.15.92:0/4269075696 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fba3813e360 0x7fba380cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:39.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:39.902+0000 7fba2ffff640 1 -- 172.21.15.92:0/4269075696 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1818173682 0 0) 0x7fba3815c5f0 con 0x7fba3815f9c0 2023-11-24T06:13:39.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:39.902+0000 7fba2ffff640 1 -- 172.21.15.92:0/4269075696 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fba24003520 con 0x7fba3815f9c0 2023-11-24T06:13:39.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:39.902+0000 7fba2ffff640 1 -- 172.21.15.92:0/4269075696 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 52092585 0 0) 0x7fba24003520 con 0x7fba3815f9c0 2023-11-24T06:13:39.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:39.902+0000 7fba2ffff640 1 -- 172.21.15.92:0/4269075696 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fba3813e360 msgr2=0x7fba380cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:13:39.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:39.902+0000 7fba2ffff640 1 --2- 172.21.15.92:0/4269075696 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fba3813e360 0x7fba380cf290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:39.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:39.902+0000 7fba2ffff640 1 -- 172.21.15.92:0/4269075696 >> v1:172.21.15.118:6789/0 conn(0x7fba3813df80 legacy=0x7fba3815bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:39.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:39.902+0000 7fba2ffff640 1 -- 172.21.15.92:0/4269075696 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba3806ca30 con 0x7fba3815f9c0 2023-11-24T06:13:39.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:39.902+0000 7fba2ffff640 1 -- 172.21.15.92:0/4269075696 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2607661350 0 0) 0x7fba300019f0 con 0x7fba3815f9c0 2023-11-24T06:13:39.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:39.902+0000 7fba2ffff640 1 -- 172.21.15.92:0/4269075696 >> v1:172.21.15.92:6789/0 conn(0x7fba3815f9c0 legacy=0x7fba380cd840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:39.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:39.902+0000 7fba2ffff640 1 --2- 172.21.15.92:0/4269075696 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fba3813e360 0x7fba240039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:39.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:39.902+0000 7fba2ffff640 1 --2- 172.21.15.92:0/4269075696 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fba24003ee0 0x7fba240047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:39.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:39.902+0000 7fba2ffff640 1 --2- 172.21.15.92:0/4269075696 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fba24004d00 0x7fba24005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:39.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:39.902+0000 7fba2ffff640 1 -- 172.21.15.92:0/4269075696 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba3806c750 con 0x7fba24003ee0 2023-11-24T06:13:39.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:39.902+0000 7fba2ffff640 1 -- 172.21.15.92:0/4269075696 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba2400b9e0 con 0x7fba24004d00 2023-11-24T06:13:39.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:39.902+0000 7fba2ffff640 1 -- 172.21.15.92:0/4269075696 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba2400bba0 con 0x7fba3813e360 2023-11-24T06:13:39.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:39.903+0000 7fba3c89c640 1 --2- 172.21.15.92:0/4269075696 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fba24003ee0 0x7fba240047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:39.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:39.903+0000 7fba3d09d640 1 --2- 172.21.15.92:0/4269075696 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fba3813e360 0x7fba240039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:39.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:39.903+0000 7fba3d89e640 1 --2- 172.21.15.92:0/4269075696 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fba24004d00 0x7fba24005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:39.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:39.903+0000 7fba3c89c640 1 -- 172.21.15.92:0/4269075696 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fba3813e360 msgr2=0x7fba240039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:13:39.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:39.903+0000 7fba3c89c640 1 --2- 172.21.15.92:0/4269075696 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fba3813e360 0x7fba240039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:39.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:39.903+0000 7fba3c89c640 1 -- 172.21.15.92:0/4269075696 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fba24004d00 msgr2=0x7fba24005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:13:39.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:39.903+0000 7fba3c89c640 1 --2- 172.21.15.92:0/4269075696 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fba24004d00 0x7fba24005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:39.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:39.903+0000 7fba3c89c640 1 -- 172.21.15.92:0/4269075696 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba24003520 con 0x7fba24003ee0 2023-11-24T06:13:39.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:39.903+0000 7fba3d09d640 1 --2- 172.21.15.92:0/4269075696 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fba3813e360 0x7fba240039a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:13:39.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:39.903+0000 7fba3d89e640 1 --2- 172.21.15.92:0/4269075696 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fba24004d00 0x7fba24005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:13:39.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:39.904+0000 7fba3c89c640 1 --2- 172.21.15.92:0/4269075696 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fba24003ee0 0x7fba240047c0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7fba28009ac0 tx=0x7fba240057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3d3bf85363a2746 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:39.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:39.905+0000 7fba2ffff640 1 -- 172.21.15.92:0/4269075696 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba2802c600 con 0x7fba24003ee0 2023-11-24T06:13:39.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:39.905+0000 7fba2ffff640 1 -- 172.21.15.92:0/4269075696 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba2802c760 con 0x7fba24003ee0 2023-11-24T06:13:39.910 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:39.905+0000 7fba2ffff640 1 -- 172.21.15.92:0/4269075696 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba2802caf0 con 0x7fba24003ee0 2023-11-24T06:13:39.910 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:39.905+0000 7fba3f301640 1 -- 172.21.15.92:0/4269075696 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fba24003ee0 msgr2=0x7fba240047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:39.910 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:39.905+0000 7fba3f301640 1 --2- 172.21.15.92:0/4269075696 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fba24003ee0 0x7fba240047c0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7fba28009ac0 tx=0x7fba240057f0 comp rx=0 tx=0).stop 2023-11-24T06:13:39.910 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:39.905+0000 7fba3d89e640 1 -- 172.21.15.92:0/4269075696 reap_dead start 2023-11-24T06:13:39.910 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:39.905+0000 7fba3f301640 1 -- 172.21.15.92:0/4269075696 shutdown_connections 2023-11-24T06:13:39.910 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:39.905+0000 7fba3f301640 1 -- 172.21.15.92:0/4269075696 >> 172.21.15.92:0/4269075696 conn(0x7fba3806a800 msgr2=0x7fba3806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:13:39.910 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:39.905+0000 7fba3f301640 1 -- 172.21.15.92:0/4269075696 shutdown_connections 2023-11-24T06:13:39.910 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:39.905+0000 7fba3f301640 1 -- 172.21.15.92:0/4269075696 wait complete. 2023-11-24T06:13:39.911 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:39.907+0000 7fba3f301640 1 Processor -- start 2023-11-24T06:13:39.911 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:39.907+0000 7fba3f301640 1 -- start start 2023-11-24T06:13:39.912 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:39.908+0000 7fba3f301640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fba24003ee0 0x7fba380acf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:39.912 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:39.908+0000 7fba3f301640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fba3813d7d0 0x7fba380b2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:39.912 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:39.908+0000 7fba3f301640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fba380b3230 0x7fba380ab120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:39.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:39.908+0000 7fba3f301640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba380b36b0 con 0x7fba380b3230 2023-11-24T06:13:39.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:39.908+0000 7fba3f301640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba380ad4a0 con 0x7fba3813d7d0 2023-11-24T06:13:39.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:39.908+0000 7fba3f301640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba380abec0 con 0x7fba24003ee0 2023-11-24T06:13:39.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:39.908+0000 7fba3d89e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fba380b3230 0x7fba380ab120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:39.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:39.908+0000 7fba3d89e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fba380b3230 0x7fba380ab120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47758/0 (socket says 172.21.15.92:47758) 2023-11-24T06:13:39.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:39.909+0000 7fba3d89e640 1 -- 172.21.15.92:0/3042954031 learned_addr learned my addr 172.21.15.92:0/3042954031 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:13:39.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:39.909+0000 7fba3d09d640 1 --2- 172.21.15.92:0/3042954031 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fba24003ee0 0x7fba380acf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:39.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:39.909+0000 7fba3c89c640 1 --2- 172.21.15.92:0/3042954031 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fba3813d7d0 0x7fba380b2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:39.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:39.909+0000 7fba3d89e640 1 -- 172.21.15.92:0/3042954031 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fba24003ee0 msgr2=0x7fba380acf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:39.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:39.909+0000 7fba3d89e640 1 --2- 172.21.15.92:0/3042954031 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fba24003ee0 0x7fba380acf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:39.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:39.909+0000 7fba3d89e640 1 -- 172.21.15.92:0/3042954031 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fba3813d7d0 msgr2=0x7fba380b2cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:39.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:39.909+0000 7fba3d89e640 1 --2- 172.21.15.92:0/3042954031 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fba3813d7d0 0x7fba380b2cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:39.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:39.909+0000 7fba3d89e640 1 -- 172.21.15.92:0/3042954031 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba380ac1a0 con 0x7fba380b3230 2023-11-24T06:13:39.914 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:39.909+0000 7fba3d89e640 1 --2- 172.21.15.92:0/3042954031 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fba380b3230 0x7fba380ab120 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7fba300012e0 tx=0x7fba3000b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:39.914 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:39.909+0000 7fba2dffb640 1 -- 172.21.15.92:0/3042954031 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba3000bb80 con 0x7fba380b3230 2023-11-24T06:13:39.914 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:39.909+0000 7fba2dffb640 1 -- 172.21.15.92:0/3042954031 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba3000bce0 con 0x7fba380b3230 2023-11-24T06:13:39.914 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:39.909+0000 7fba2dffb640 1 -- 172.21.15.92:0/3042954031 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba30007310 con 0x7fba380b3230 2023-11-24T06:13:39.914 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:39.909+0000 7fba3f301640 1 -- 172.21.15.92:0/3042954031 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba380ac480 con 0x7fba380b3230 2023-11-24T06:13:39.914 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:39.910+0000 7fba3f301640 1 -- 172.21.15.92:0/3042954031 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba380aca40 con 0x7fba380b3230 2023-11-24T06:13:39.914 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:39.911+0000 7fba3f301640 1 -- 172.21.15.92:0/3042954031 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba380675d0 con 0x7fba380b3230 2023-11-24T06:13:39.914 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:39.912+0000 7fba2dffb640 1 -- 172.21.15.92:0/3042954031 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 7) v1 ==== 89333+0+0 (secure 0 0 0) 0x7fba30007470 con 0x7fba380b3230 2023-11-24T06:13:39.916 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:39.913+0000 7fba2dffb640 1 --2- 172.21.15.92:0/3042954031 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fba1c06af60 0x7fba1c06d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:39.916 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:39.913+0000 7fba2dffb640 1 -- 172.21.15.92:0/3042954031 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fba30089af0 con 0x7fba380b3230 2023-11-24T06:13:39.916 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:39.913+0000 7fba3d09d640 1 --2- 172.21.15.92:0/3042954031 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fba1c06af60 0x7fba1c06d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:39.916 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:39.913+0000 7fba3d09d640 1 --2- 172.21.15.92:0/3042954031 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fba1c06af60 0x7fba1c06d420 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fba34001860 tx=0x7fba34009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:39.920 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:39.917+0000 7fba2dffb640 1 -- 172.21.15.92:0/3042954031 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fba30054c70 con 0x7fba380b3230 2023-11-24T06:13:40.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:40.178+0000 7fba3f301640 1 -- 172.21.15.92:0/3042954031 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestVolumes.test_volume_info_pending_subvol_deletions"]} v 0) v1 -- 0x7fba3806df00 con 0x7fba380b3230 2023-11-24T06:13:40.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:40.228+0000 7fba2dffb640 1 -- 172.21.15.92:0/3042954031 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestVolumes.test_volume_info_pending_subvol_deletions"]}]=0 v202) v1 ==== 158+0+0 (secure 0 0 0) 0x7fba30058290 con 0x7fba380b3230 2023-11-24T06:13:40.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:40.231+0000 7fba3f301640 1 -- 172.21.15.92:0/3042954031 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fba1c06af60 msgr2=0x7fba1c06d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:40.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:40.231+0000 7fba3f301640 1 --2- 172.21.15.92:0/3042954031 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fba1c06af60 0x7fba1c06d420 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fba34001860 tx=0x7fba34009040 comp rx=0 tx=0).stop 2023-11-24T06:13:40.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:40.231+0000 7fba3f301640 1 -- 172.21.15.92:0/3042954031 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fba380b3230 msgr2=0x7fba380ab120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:40.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:40.231+0000 7fba3f301640 1 --2- 172.21.15.92:0/3042954031 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fba380b3230 0x7fba380ab120 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7fba300012e0 tx=0x7fba3000b4a0 comp rx=0 tx=0).stop 2023-11-24T06:13:40.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:40.231+0000 7fba3f301640 1 -- 172.21.15.92:0/3042954031 shutdown_connections 2023-11-24T06:13:40.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:40.231+0000 7fba3f301640 1 --2- 172.21.15.92:0/3042954031 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fba1c06af60 0x7fba1c06d420 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:40.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:40.231+0000 7fba3f301640 1 --2- 172.21.15.92:0/3042954031 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fba380b3230 0x7fba380ab120 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:40.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:40.231+0000 7fba3f301640 1 --2- 172.21.15.92:0/3042954031 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fba3813d7d0 0x7fba380b2cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:40.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:40.231+0000 7fba3f301640 1 --2- 172.21.15.92:0/3042954031 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fba24003ee0 0x7fba380acf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:40.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:40.231+0000 7fba3f301640 1 -- 172.21.15.92:0/3042954031 >> 172.21.15.92:0/3042954031 conn(0x7fba3806a800 msgr2=0x7fba3815a140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:13:40.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:40.231+0000 7fba3f301640 1 -- 172.21.15.92:0/3042954031 shutdown_connections 2023-11-24T06:13:40.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:40.231+0000 7fba3f301640 1 -- 172.21.15.92:0/3042954031 wait complete. 2023-11-24T06:13:40.255 INFO:tasks.cephfs_test_runner:Tests the pending_subvolume_deletions in 'fs volume info' command ... ERROR 2023-11-24T06:13:40.255 INFO:tasks.cephfs_test_runner: 2023-11-24T06:13:40.255 INFO:tasks.cephfs_test_runner:====================================================================== 2023-11-24T06:13:40.255 INFO:tasks.cephfs_test_runner:ERROR: test_volume_info_pending_subvol_deletions (tasks.cephfs.test_volumes.TestVolumes) 2023-11-24T06:13:40.255 INFO:tasks.cephfs_test_runner:Tests the pending_subvolume_deletions in 'fs volume info' command 2023-11-24T06:13:40.255 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-11-24T06:13:40.255 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2023-11-24T06:13:40.255 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_a13ea5ff00cb4df7a9cc941166ddeb358881144c/qa/tasks/cephfs/test_volumes.py", line 568, in test_volume_info_pending_subvol_deletions 2023-11-24T06:13:40.256 INFO:tasks.cephfs_test_runner: self._do_subvolume_io(subvolname, number_of_files=3000, file_size=0) 2023-11-24T06:13:40.256 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_a13ea5ff00cb4df7a9cc941166ddeb358881144c/qa/tasks/cephfs/test_volumes.py", line 278, in _do_subvolume_io 2023-11-24T06:13:40.256 INFO:tasks.cephfs_test_runner: filename = "{0}.{1}".format(TestVolumes.TEST_FILE_NAME_PREFIX, i) 2023-11-24T06:13:40.256 INFO:tasks.cephfs_test_runner:AttributeError: type object 'TestVolumes' has no attribute 'TEST_FILE_NAME_PREFIX' 2023-11-24T06:13:40.256 INFO:tasks.cephfs_test_runner: 2023-11-24T06:13:40.256 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-11-24T06:13:40.256 INFO:tasks.cephfs_test_runner:Ran 3 tests in 155.329s 2023-11-24T06:13:40.256 INFO:tasks.cephfs_test_runner: 2023-11-24T06:13:40.256 INFO:tasks.cephfs_test_runner:FAILED (errors=1) 2023-11-24T06:13:40.256 INFO:tasks.cephfs_test_runner: 2023-11-24T06:13:40.256 INFO:tasks.cephfs_test_runner:====================================================================== 2023-11-24T06:13:40.256 INFO:tasks.cephfs_test_runner:ERROR: test_volume_info_pending_subvol_deletions (tasks.cephfs.test_volumes.TestVolumes) 2023-11-24T06:13:40.257 INFO:tasks.cephfs_test_runner:Tests the pending_subvolume_deletions in 'fs volume info' command 2023-11-24T06:13:40.257 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-11-24T06:13:40.257 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2023-11-24T06:13:40.257 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_a13ea5ff00cb4df7a9cc941166ddeb358881144c/qa/tasks/cephfs/test_volumes.py", line 568, in test_volume_info_pending_subvol_deletions 2023-11-24T06:13:40.257 INFO:tasks.cephfs_test_runner: self._do_subvolume_io(subvolname, number_of_files=3000, file_size=0) 2023-11-24T06:13:40.257 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_a13ea5ff00cb4df7a9cc941166ddeb358881144c/qa/tasks/cephfs/test_volumes.py", line 278, in _do_subvolume_io 2023-11-24T06:13:40.257 INFO:tasks.cephfs_test_runner: filename = "{0}.{1}".format(TestVolumes.TEST_FILE_NAME_PREFIX, i) 2023-11-24T06:13:40.257 INFO:tasks.cephfs_test_runner:AttributeError: type object 'TestVolumes' has no attribute 'TEST_FILE_NAME_PREFIX' 2023-11-24T06:13:40.257 INFO:tasks.cephfs_test_runner: 2023-11-24T06:13:40.257 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_2442239d2653456406c25ae0c71b689c8f2657b6/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_a13ea5ff00cb4df7a9cc941166ddeb358881144c/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_volume_info_pending_subvol_deletions (tasks.cephfs.test_volumes.TestVolumes) 2023-11-24T06:13:40.325 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=79a1606a3ac048d792f56a773cfae9f0 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_2442239d2653456406c25ae0c71b689c8f2657b6/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_a13ea5ff00cb4df7a9cc941166ddeb358881144c/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_volume_info_pending_subvol_deletions (tasks.cephfs.test_volumes.TestVolumes) 2023-11-24T06:13:40.326 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2023-11-24T06:13:40.335 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2023-11-24T06:13:40.410 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2023-11-24T06:13:40.410 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-11-24T06:13:40.410 DEBUG:teuthology.orchestra.run.smithi092:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:13:40.472 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054298,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498824k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249408k,nr_inodes=812352,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498824k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-24T06:13:40.473 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-11-24T06:13:40.473 DEBUG:teuthology.orchestra.run.smithi092:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:13:40.525 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054298,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498824k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249408k,nr_inodes=812352,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498824k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-24T06:13:40.525 DEBUG:teuthology.orchestra.run.smithi118:> set -ex 2023-11-24T06:13:40.525 DEBUG:teuthology.orchestra.run.smithi118:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:13:40.587 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054284,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249396k,nr_inodes=812349,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-24T06:13:40.587 DEBUG:teuthology.orchestra.run.smithi118:> set -ex 2023-11-24T06:13:40.587 DEBUG:teuthology.orchestra.run.smithi118:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:13:40.648 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054284,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249396k,nr_inodes=812349,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-24T06:13:40.648 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:13:40.648 DEBUG:teuthology.orchestra.run.smithi092:> ip netns list 2023-11-24T06:13:40.705 INFO:teuthology.orchestra.run.smithi092.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-24T06:13:40.706 INFO:teuthology.orchestra.run.smithi092.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-24T06:13:40.707 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:13:40.707 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-24T06:13:40.802 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:13:40.803 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-24T06:13:40.898 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:13:40.899 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip link delete ceph-brx 2023-11-24T06:13:41.039 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:13:41.039 DEBUG:teuthology.orchestra.run.smithi118:> ip netns list 2023-11-24T06:13:41.099 INFO:teuthology.orchestra.run.smithi118.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-11-24T06:13:41.099 INFO:teuthology.orchestra.run.smithi118.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-11-24T06:13:41.100 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:13:41.100 DEBUG:teuthology.orchestra.run.smithi118:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-24T06:13:41.192 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:13:41.192 DEBUG:teuthology.orchestra.run.smithi118:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-24T06:13:41.281 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:13:41.281 DEBUG:teuthology.orchestra.run.smithi118:> sudo ip link delete ceph-brx 2023-11-24T06:13:41.413 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2023-11-24T06:13:41.422 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-11-24T06:13:41.423 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-24T06:13:41.652 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:41.650+0000 7fdd3d075640 1 Processor -- start 2023-11-24T06:13:41.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:41.650+0000 7fdd3d075640 1 -- start start 2023-11-24T06:13:41.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:41.652+0000 7fdd3d075640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fdd38143ae0 0x7fdd380cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:41.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:41.652+0000 7fdd3d075640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fdd380cf7d0 0x7fdd3815dca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:41.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:41.652+0000 7fdd3d075640 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd380cfc30 con 0x7fdd3815e1e0 2023-11-24T06:13:41.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:41.652+0000 7fdd3d075640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd380b1ee0 con 0x7fdd38143ae0 2023-11-24T06:13:41.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:41.652+0000 7fdd3d075640 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd380b2780 con 0x7fdd380cf7d0 2023-11-24T06:13:41.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:41.652+0000 7fdd36d9d640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fdd38143ae0 0x7fdd380cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:41.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:41.652+0000 7fdd36d9d640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fdd38143ae0 0x7fdd380cf290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47766/0 (socket says 172.21.15.92:47766) 2023-11-24T06:13:41.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:41.652+0000 7fdd36d9d640 1 -- 172.21.15.92:0/1110285444 learned_addr learned my addr 172.21.15.92:0/1110285444 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:13:41.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:41.652+0000 7fdd36d9d640 1 -- 172.21.15.92:0/1110285444 >> v2:172.21.15.118:3300/0 conn(0x7fdd380cf7d0 msgr2=0x7fdd3815dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:13:41.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:41.652+0000 7fdd2ffff640 1 --2- 172.21.15.92:0/1110285444 >> v2:172.21.15.118:3300/0 conn(0x7fdd380cf7d0 0x7fdd3815dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:41.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:41.653+0000 7fdd36d9d640 1 --2- 172.21.15.92:0/1110285444 >> v2:172.21.15.118:3300/0 conn(0x7fdd380cf7d0 0x7fdd3815dca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:41.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:41.653+0000 7fdd36d9d640 1 -- 172.21.15.92:0/1110285444 >> v1:172.21.15.118:6789/0 conn(0x7fdd3815e1e0 legacy=0x7fdd3815a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:13:41.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:41.653+0000 7fdd36d9d640 1 -- 172.21.15.92:0/1110285444 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd3815e990 con 0x7fdd38143ae0 2023-11-24T06:13:41.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:41.653+0000 7fdd2ffff640 1 --2- 172.21.15.92:0/1110285444 >> v2:172.21.15.118:3300/0 conn(0x7fdd380cf7d0 0x7fdd3815dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:13:41.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:41.654+0000 7fdd36d9d640 1 --2- 172.21.15.92:0/1110285444 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdd38143ae0 0x7fdd380cf290 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fdd380cb830 tx=0x7fdd2802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af00f1eb1213efa1 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:41.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:41.654+0000 7fdd3659c640 1 -- 172.21.15.92:0/1110285444 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd28031870 con 0x7fdd38143ae0 2023-11-24T06:13:41.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:41.654+0000 7fdd3659c640 1 -- 172.21.15.92:0/1110285444 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd280319d0 con 0x7fdd38143ae0 2023-11-24T06:13:41.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:41.654+0000 7fdd3659c640 1 -- 172.21.15.92:0/1110285444 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd28031d60 con 0x7fdd38143ae0 2023-11-24T06:13:41.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:41.655+0000 7fdd3d075640 1 -- 172.21.15.92:0/1110285444 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdd38143ae0 msgr2=0x7fdd380cf290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:41.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:41.655+0000 7fdd3d075640 1 --2- 172.21.15.92:0/1110285444 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdd38143ae0 0x7fdd380cf290 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fdd380cb830 tx=0x7fdd2802b910 comp rx=0 tx=0).stop 2023-11-24T06:13:41.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:41.655+0000 7fdd3d075640 1 -- 172.21.15.92:0/1110285444 shutdown_connections 2023-11-24T06:13:41.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:41.655+0000 7fdd3d075640 1 --2- 172.21.15.92:0/1110285444 >> v2:172.21.15.118:3300/0 conn(0x7fdd380cf7d0 0x7fdd3815dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:41.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:41.655+0000 7fdd3d075640 1 --2- 172.21.15.92:0/1110285444 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdd38143ae0 0x7fdd380cf290 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:41.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:41.655+0000 7fdd3d075640 1 -- 172.21.15.92:0/1110285444 >> 172.21.15.92:0/1110285444 conn(0x7fdd3806a800 msgr2=0x7fdd3806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:13:41.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:41.655+0000 7fdd3d075640 1 -- 172.21.15.92:0/1110285444 shutdown_connections 2023-11-24T06:13:41.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:41.655+0000 7fdd3d075640 1 -- 172.21.15.92:0/1110285444 wait complete. 2023-11-24T06:13:41.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:41.657+0000 7fdd3d075640 1 Processor -- start 2023-11-24T06:13:41.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:41.657+0000 7fdd3d075640 1 -- start start 2023-11-24T06:13:41.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:41.658+0000 7fdd3d075640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fdd380cf7d0 0x7fdd3815ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:41.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:41.658+0000 7fdd3d075640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdd38143ae0 0x7fdd3813e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:41.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:41.658+0000 7fdd3d075640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fdd3815e1e0 0x7fdd3813ec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:41.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:41.658+0000 7fdd3d075640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd38165080 con 0x7fdd38143ae0 2023-11-24T06:13:41.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:41.658+0000 7fdd3d075640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd38140210 con 0x7fdd3815e1e0 2023-11-24T06:13:41.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:41.658+0000 7fdd3d075640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd381404f0 con 0x7fdd380cf7d0 2023-11-24T06:13:41.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:41.659+0000 7fdd2ffff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdd38143ae0 0x7fdd3813e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:41.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:41.659+0000 7fdd2ffff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdd38143ae0 0x7fdd3813e700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47774/0 (socket says 172.21.15.92:47774) 2023-11-24T06:13:41.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:41.659+0000 7fdd2ffff640 1 -- 172.21.15.92:0/2284065780 learned_addr learned my addr 172.21.15.92:0/2284065780 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:13:41.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:41.659+0000 7fdd3759e640 1 --2- 172.21.15.92:0/2284065780 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fdd3815e1e0 0x7fdd3813ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:41.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:41.659+0000 7fdd36d9d640 1 --2- 172.21.15.92:0/2284065780 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fdd380cf7d0 0x7fdd3815ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:41.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:41.659+0000 7fdd2ffff640 1 -- 172.21.15.92:0/2284065780 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fdd380cf7d0 msgr2=0x7fdd3815ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:41.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:41.659+0000 7fdd2ffff640 1 --2- 172.21.15.92:0/2284065780 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fdd380cf7d0 0x7fdd3815ca60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:41.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:41.659+0000 7fdd2ffff640 1 -- 172.21.15.92:0/2284065780 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fdd3815e1e0 msgr2=0x7fdd3813ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:41.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:41.659+0000 7fdd2ffff640 1 --2- 172.21.15.92:0/2284065780 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fdd3815e1e0 0x7fdd3813ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:41.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:41.659+0000 7fdd2ffff640 1 -- 172.21.15.92:0/2284065780 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd280094d0 con 0x7fdd38143ae0 2023-11-24T06:13:41.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:41.659+0000 7fdd3759e640 1 --2- 172.21.15.92:0/2284065780 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fdd3815e1e0 0x7fdd3813ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:13:41.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:41.659+0000 7fdd2ffff640 1 --2- 172.21.15.92:0/2284065780 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdd38143ae0 0x7fdd3813e700 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7fdd2000c7f0 tx=0x7fdd2000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:41.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:41.660+0000 7fdd2f7fe640 1 -- 172.21.15.92:0/2284065780 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd20007a30 con 0x7fdd38143ae0 2023-11-24T06:13:41.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:41.660+0000 7fdd2f7fe640 1 -- 172.21.15.92:0/2284065780 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd20007b90 con 0x7fdd38143ae0 2023-11-24T06:13:41.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:41.660+0000 7fdd2f7fe640 1 -- 172.21.15.92:0/2284065780 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd2000ce40 con 0x7fdd38143ae0 2023-11-24T06:13:41.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:41.660+0000 7fdd3d075640 1 -- 172.21.15.92:0/2284065780 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd38164a30 con 0x7fdd38143ae0 2023-11-24T06:13:41.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:41.660+0000 7fdd3d075640 1 -- 172.21.15.92:0/2284065780 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd380af3b0 con 0x7fdd38143ae0 2023-11-24T06:13:41.666 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:41.661+0000 7fdd3d075640 1 -- 172.21.15.92:0/2284065780 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd380b2bf0 con 0x7fdd38143ae0 2023-11-24T06:13:41.666 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:41.662+0000 7fdd2f7fe640 1 -- 172.21.15.92:0/2284065780 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 7) v1 ==== 89333+0+0 (secure 0 0 0) 0x7fdd200113f0 con 0x7fdd38143ae0 2023-11-24T06:13:41.666 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:41.663+0000 7fdd2f7fe640 1 --2- 172.21.15.92:0/2284065780 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fdd1806af60 0x7fdd1806d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:41.666 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:41.663+0000 7fdd2f7fe640 1 -- 172.21.15.92:0/2284065780 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fdd20089700 con 0x7fdd38143ae0 2023-11-24T06:13:41.666 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:41.664+0000 7fdd36d9d640 1 --2- 172.21.15.92:0/2284065780 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fdd1806af60 0x7fdd1806d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:41.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:41.664+0000 7fdd36d9d640 1 --2- 172.21.15.92:0/2284065780 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fdd1806af60 0x7fdd1806d420 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fdd28036000 tx=0x7fdd2803c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:41.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:41.668+0000 7fdd2f7fe640 1 -- 172.21.15.92:0/2284065780 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fdd380b2bf0 con 0x7fdd38143ae0 2023-11-24T06:13:41.856 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:41.854+0000 7fdd3d075640 1 -- 172.21.15.92:0/2284065780 --> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fdd3815e670 con 0x7fdd1806af60 2023-11-24T06:13:41.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:41.857+0000 7fdd2f7fe640 1 -- 172.21.15.92:0/2284065780 <== mgr.4101 v2:172.21.15.92:6800/784027401 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+48683 (secure 0 0 0) 0x7fdd28031cc0 con 0x7fdd1806af60 2023-11-24T06:13:41.860 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-24T06:13:41.861 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2023-11-24T06:13:41.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:41.860+0000 7fdd3d075640 1 -- 172.21.15.92:0/2284065780 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fdd1806af60 msgr2=0x7fdd1806d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:41.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:41.860+0000 7fdd3d075640 1 --2- 172.21.15.92:0/2284065780 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fdd1806af60 0x7fdd1806d420 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fdd28036000 tx=0x7fdd2803c040 comp rx=0 tx=0).stop 2023-11-24T06:13:41.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:41.860+0000 7fdd3d075640 1 -- 172.21.15.92:0/2284065780 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdd38143ae0 msgr2=0x7fdd3813e700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:41.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:41.860+0000 7fdd3d075640 1 --2- 172.21.15.92:0/2284065780 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdd38143ae0 0x7fdd3813e700 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7fdd2000c7f0 tx=0x7fdd2000cbf0 comp rx=0 tx=0).stop 2023-11-24T06:13:41.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:41.860+0000 7fdd3d075640 1 -- 172.21.15.92:0/2284065780 shutdown_connections 2023-11-24T06:13:41.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:41.860+0000 7fdd3d075640 1 --2- 172.21.15.92:0/2284065780 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fdd1806af60 0x7fdd1806d420 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:41.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:41.860+0000 7fdd3d075640 1 --2- 172.21.15.92:0/2284065780 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fdd3815e1e0 0x7fdd3813ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:41.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:41.860+0000 7fdd3d075640 1 --2- 172.21.15.92:0/2284065780 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdd38143ae0 0x7fdd3813e700 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:41.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:41.860+0000 7fdd3d075640 1 --2- 172.21.15.92:0/2284065780 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fdd380cf7d0 0x7fdd3815ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:41.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:41.860+0000 7fdd3d075640 1 -- 172.21.15.92:0/2284065780 >> 172.21.15.92:0/2284065780 conn(0x7fdd3806a800 msgr2=0x7fdd3815b7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:13:41.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:41.860+0000 7fdd3d075640 1 -- 172.21.15.92:0/2284065780 shutdown_connections 2023-11-24T06:13:41.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:41.860+0000 7fdd3d075640 1 -- 172.21.15.92:0/2284065780 wait complete. 2023-11-24T06:13:41.881 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":161,"stamp":"2023-11-24T06:13:41.727316+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":248488,"kb_used_data":7368,"kb_used_omap":12,"kb_used_meta":241011,"kb_avail":754726232,"statfs":{"total":773094113280,"available":772839661568,"internally_reserved":0,"allocated":7544832,"data_stored":3930806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":246795825},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-34068,"num_objects":-23,"num_object_clones":0,"num_object_copies":-46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-23,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-68,"num_write_kb":-57,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001340"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":203,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-24T06:13:10.170077+0000","last_change":"2023-11-24T06:10:20.082677+0000","last_active":"2023-11-24T06:13:10.170077+0000","last_peered":"2023-11-24T06:13:10.170077+0000","last_clean":"2023-11-24T06:13:10.170077+0000","last_became_active":"2023-11-24T06:10:20.080551+0000","last_became_peered":"2023-11-24T06:10:20.080551+0000","last_unstale":"2023-11-24T06:13:10.170077+0000","last_undegraded":"2023-11-24T06:13:10.170077+0000","last_fullsized":"2023-11-24T06:13:10.170077+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:18.602209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:18.602209+0000","last_clean_scrub_stamp":"2023-11-24T06:10:18.602209+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:47:24.216235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574890,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32532,"kb_used_data":1352,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339308,"statfs":{"total":96636764160,"available":96603451392,"internally_reserved":0,"allocated":1384448,"data_stored":925441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:13:18 2023","interfaces":[{"interface":"back","average":{"1min":0.99199999999999999,"5min":1.0669999999999999,"15min":1.095},"min":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":1.581,"5min":1.9630000000000001,"15min":1.9630000000000001},"last":0.60099999999999998},{"interface":"front","average":{"1min":1.0640000000000001,"5min":1.0569999999999999,"15min":1.0569999999999999},"min":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.413,"5min":1.621,"15min":1.621},"last":0.872}]},{"osd":1,"last update":"Fri Nov 24 06:13:18 2023","interfaces":[{"interface":"back","average":{"1min":1.0620000000000001,"5min":1.113,"15min":1.121},"min":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":1.569,"5min":2.0259999999999998,"15min":2.0259999999999998},"last":1.123},{"interface":"front","average":{"1min":1.274,"5min":1.1779999999999999,"15min":1.1459999999999999},"min":{"1min":0.72799999999999998,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.6579999999999999,"5min":2.1480000000000001,"15min":2.1480000000000001},"last":0.75700000000000001}]},{"osd":2,"last update":"Fri Nov 24 06:13:18 2023","interfaces":[{"interface":"back","average":{"1min":1.1439999999999999,"5min":1.1339999999999999,"15min":1.1399999999999999},"min":{"1min":0.72399999999999998,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.4770000000000001,"5min":1.7809999999999999,"15min":1.7809999999999999},"last":0.81799999999999995},{"interface":"front","average":{"1min":1.222,"5min":1.1950000000000001,"15min":1.179},"min":{"1min":0.88100000000000001,"5min":0.86599999999999999,"15min":0.86599999999999999},"max":{"1min":1.599,"5min":2.0619999999999998,"15min":2.0619999999999998},"last":1.0029999999999999}]},{"osd":3,"last update":"Fri Nov 24 06:13:18 2023","interfaces":[{"interface":"back","average":{"1min":1.077,"5min":1.1779999999999999,"15min":1.196},"min":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":1.4339999999999999,"5min":1.615,"15min":1.615},"last":0.95399999999999996},{"interface":"front","average":{"1min":1.28,"5min":1.3,"15min":1.3069999999999999},"min":{"1min":0.82699999999999996,"5min":0.81999999999999995,"15min":0.81999999999999995},"max":{"1min":1.778,"5min":1.9570000000000001,"15min":1.9570000000000001},"last":0.72599999999999998}]},{"osd":4,"last update":"Fri Nov 24 06:13:18 2023","interfaces":[{"interface":"back","average":{"1min":1.379,"5min":1.3220000000000001,"15min":1.3},"min":{"1min":0.60399999999999998,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":2.1739999999999999,"5min":2.1739999999999999,"15min":2.1739999999999999},"last":1.091},{"interface":"front","average":{"1min":1.2829999999999999,"5min":1.3200000000000001,"15min":1.3260000000000001},"min":{"1min":0.621,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.7809999999999999,"5min":2.1680000000000001,"15min":2.1680000000000001},"last":1.0529999999999999}]},{"osd":5,"last update":"Fri Nov 24 06:13:18 2023","interfaces":[{"interface":"back","average":{"1min":1.502,"5min":1.4770000000000001,"15min":1.456},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.964,"5min":2.0379999999999998,"15min":2.0379999999999998},"last":1.294},{"interface":"front","average":{"1min":1.4730000000000001,"5min":1.4810000000000001,"15min":1.4630000000000001},"min":{"1min":0.69699999999999995,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.915,"5min":2.0539999999999998,"15min":2.0539999999999998},"last":1.4330000000000001}]},{"osd":6,"last update":"Fri Nov 24 06:13:18 2023","interfaces":[{"interface":"back","average":{"1min":1.647,"5min":1.573,"15min":1.538},"min":{"1min":0.66000000000000003,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":2.1320000000000001,"5min":2.1320000000000001,"15min":2.1320000000000001},"last":1.194},{"interface":"front","average":{"1min":1.5649999999999999,"5min":1.5649999999999999,"15min":1.5409999999999999},"min":{"1min":0.75800000000000001,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":2.0310000000000001,"5min":2.0680000000000001,"15min":2.0680000000000001},"last":1.327}]}]},{"osd":6,"up_from":13,"seq":55834574891,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31948,"kb_used_data":768,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339892,"statfs":{"total":96636764160,"available":96604049408,"internally_reserved":0,"allocated":786432,"data_stored":334420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:13:24 2023","interfaces":[{"interface":"back","average":{"1min":1.0580000000000001,"5min":1.1619999999999999,"15min":1.2},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.45,"5min":2.032,"15min":2.032},"last":1.022},{"interface":"front","average":{"1min":1.0860000000000001,"5min":1.1850000000000001,"15min":1.2110000000000001},"min":{"1min":0.65700000000000003,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.637,"5min":2.0230000000000001,"15min":2.0230000000000001},"last":0.83299999999999996}]},{"osd":1,"last update":"Fri Nov 24 06:13:24 2023","interfaces":[{"interface":"back","average":{"1min":0.97799999999999998,"5min":1.125,"15min":1.153},"min":{"1min":0.60099999999999998,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.571,"5min":2.1499999999999999,"15min":2.1499999999999999},"last":1.123},{"interface":"front","average":{"1min":1.1659999999999999,"5min":1.2130000000000001,"15min":1.2270000000000001},"min":{"1min":0.872,"5min":0.77500000000000002,"15min":0.77500000000000002},"max":{"1min":1.653,"5min":1.8899999999999999,"15min":1.8899999999999999},"last":1.0580000000000001}]},{"osd":2,"last update":"Fri Nov 24 06:13:24 2023","interfaces":[{"interface":"back","average":{"1min":1.05,"5min":1.1839999999999999,"15min":1.1619999999999999},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.5009999999999999,"5min":4.7359999999999998,"15min":4.7359999999999998},"last":0.872},{"interface":"front","average":{"1min":1.252,"5min":1.413,"15min":1.4299999999999999},"min":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":1.8380000000000001,"5min":3.9700000000000002,"15min":3.9700000000000002},"last":1.2010000000000001}]},{"osd":3,"last update":"Fri Nov 24 06:13:24 2023","interfaces":[{"interface":"back","average":{"1min":1.274,"5min":1.2989999999999999,"15min":1.3089999999999999},"min":{"1min":0.94999999999999996,"5min":0.78500000000000003,"15min":0.78500000000000003},"max":{"1min":1.6120000000000001,"5min":1.919,"15min":1.919},"last":0.94999999999999996},{"interface":"front","average":{"1min":1.2609999999999999,"5min":1.2849999999999999,"15min":1.292},"min":{"1min":0.77200000000000002,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":1.605,"5min":2.3929999999999998,"15min":2.3929999999999998},"last":0.97999999999999998}]},{"osd":4,"last update":"Fri Nov 24 06:13:24 2023","interfaces":[{"interface":"back","average":{"1min":1.264,"5min":1.417,"15min":1.444},"min":{"1min":0.70699999999999996,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.9610000000000001,"5min":2.4039999999999999,"15min":2.4039999999999999},"last":1.446},{"interface":"front","average":{"1min":1.335,"5min":1.4670000000000001,"15min":1.4750000000000001},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.8029999999999999,"5min":2.5609999999999999,"15min":2.5609999999999999},"last":1.151}]},{"osd":5,"last update":"Fri Nov 24 06:13:24 2023","interfaces":[{"interface":"back","average":{"1min":1.306,"5min":1.4390000000000001,"15min":1.464},"min":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"max":{"1min":1.9159999999999999,"5min":2.8290000000000002,"15min":2.8290000000000002},"last":1.0920000000000001},{"interface":"front","average":{"1min":1.4590000000000001,"5min":1.613,"15min":1.6419999999999999},"min":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"max":{"1min":2.0259999999999998,"5min":2.9420000000000002,"15min":2.9420000000000002},"last":1.5}]},{"osd":7,"last update":"Fri Nov 24 06:13:24 2023","interfaces":[{"interface":"back","average":{"1min":1.403,"5min":1.5580000000000001,"15min":1.5800000000000001},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.772,"5min":2.5139999999999998,"15min":2.5139999999999998},"last":1.367},{"interface":"front","average":{"1min":1.421,"5min":1.5629999999999999,"15min":1.5529999999999999},"min":{"1min":0.79300000000000004,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":1.992,"5min":4.7859999999999996,"15min":4.7859999999999996},"last":1.29}]}]},{"osd":5,"up_from":13,"seq":55834574891,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31944,"kb_used_data":764,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339896,"statfs":{"total":96636764160,"available":96604053504,"internally_reserved":0,"allocated":782336,"data_stored":333933,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:13:25 2023","interfaces":[{"interface":"back","average":{"1min":0.97999999999999998,"5min":1.1339999999999999,"15min":1.212},"min":{"1min":0.62,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.526,"5min":3.0030000000000001,"15min":3.0030000000000001},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.998,"5min":1.1339999999999999,"15min":1.196},"min":{"1min":0.623,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.504,"5min":2.0979999999999999,"15min":2.0979999999999999},"last":0.81100000000000005}]},{"osd":1,"last update":"Fri Nov 24 06:13:25 2023","interfaces":[{"interface":"back","average":{"1min":1.0720000000000001,"5min":1.212,"15min":1.254},"min":{"1min":0.63300000000000001,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.665,"5min":5.0499999999999998,"15min":5.0499999999999998},"last":0.98099999999999998},{"interface":"front","average":{"1min":1.101,"5min":1.3180000000000001,"15min":1.4259999999999999},"min":{"1min":0.72599999999999998,"5min":0.435,"15min":0.435},"max":{"1min":1.5449999999999999,"5min":3.6930000000000001,"15min":3.6930000000000001},"last":1.046}]},{"osd":2,"last update":"Fri Nov 24 06:13:25 2023","interfaces":[{"interface":"back","average":{"1min":1.103,"5min":1.2589999999999999,"15min":1.3160000000000001},"min":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.5960000000000001,"5min":3.085,"15min":3.085},"last":1.4059999999999999},{"interface":"front","average":{"1min":1.2230000000000001,"5min":1.4510000000000001,"15min":1.5609999999999999},"min":{"1min":0.746,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.5980000000000001,"5min":4.7359999999999998,"15min":4.7359999999999998},"last":1.085}]},{"osd":3,"last update":"Fri Nov 24 06:13:25 2023","interfaces":[{"interface":"back","average":{"1min":1.246,"5min":1.3500000000000001,"15min":1.397},"min":{"1min":0.747,"5min":0.73999999999999999,"15min":0.73999999999999999},"max":{"1min":1.6579999999999999,"5min":3.2229999999999999,"15min":3.2229999999999999},"last":1.347},{"interface":"front","average":{"1min":1.333,"5min":1.3919999999999999,"15min":1.427},"min":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"max":{"1min":1.7789999999999999,"5min":2.3260000000000001,"15min":2.3260000000000001},"last":0.88300000000000001}]},{"osd":4,"last update":"Fri Nov 24 06:13:25 2023","interfaces":[{"interface":"back","average":{"1min":1.333,"5min":1.482,"15min":1.538},"min":{"1min":0.95399999999999996,"5min":0.84299999999999997,"15min":0.84299999999999997},"max":{"1min":1.762,"5min":3.1230000000000002,"15min":3.1230000000000002},"last":1.147},{"interface":"front","average":{"1min":1.3400000000000001,"5min":1.478,"15min":1.532},"min":{"1min":0.84399999999999997,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.7509999999999999,"5min":2.4809999999999999,"15min":2.4809999999999999},"last":1.2170000000000001}]},{"osd":6,"last update":"Fri Nov 24 06:13:25 2023","interfaces":[{"interface":"back","average":{"1min":1.4630000000000001,"5min":1.6319999999999999,"15min":1.7150000000000001},"min":{"1min":1.2010000000000001,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.887,"5min":4.6529999999999996,"15min":4.6529999999999996},"last":1.1739999999999999},{"interface":"front","average":{"1min":1.5049999999999999,"5min":1.6479999999999999,"15min":1.716},"min":{"1min":1.226,"5min":0.90400000000000003,"15min":0.90400000000000003},"max":{"1min":1.863,"5min":4.3529999999999998,"15min":4.3529999999999998},"last":1.4710000000000001}]},{"osd":7,"last update":"Fri Nov 24 06:13:25 2023","interfaces":[{"interface":"back","average":{"1min":1.4490000000000001,"5min":1.615,"15min":1.679},"min":{"1min":0.98499999999999999,"5min":0.874,"15min":0.874},"max":{"1min":1.847,"5min":2.8839999999999999,"15min":2.8839999999999999},"last":1.514},{"interface":"front","average":{"1min":1.579,"5min":1.7509999999999999,"15min":1.8280000000000001},"min":{"1min":1.2709999999999999,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.9770000000000001,"5min":4.8739999999999997,"15min":4.8739999999999997},"last":1.3049999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574891,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31944,"kb_used_data":764,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339896,"statfs":{"total":96636764160,"available":96604053504,"internally_reserved":0,"allocated":782336,"data_stored":334398,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:13:24 2023","interfaces":[{"interface":"back","average":{"1min":1.01,"5min":1.052,"15min":1.0449999999999999},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.907,"5min":1.907,"15min":1.907},"last":0.66700000000000004},{"interface":"front","average":{"1min":1.046,"5min":1.1479999999999999,"15min":1.194},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.732,"5min":1.8220000000000001,"15min":1.8220000000000001},"last":0.753}]},{"osd":1,"last update":"Fri Nov 24 06:13:24 2023","interfaces":[{"interface":"back","average":{"1min":1.1990000000000001,"5min":1.141,"15min":1.127},"min":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.655,"5min":1.97,"15min":1.97},"last":1.6639999999999999},{"interface":"front","average":{"1min":1.0920000000000001,"5min":1.161,"15min":1.1719999999999999},"min":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.831,"5min":1.917,"15min":1.917},"last":0.82199999999999995}]},{"osd":2,"last update":"Fri Nov 24 06:13:24 2023","interfaces":[{"interface":"back","average":{"1min":1.119,"5min":1.194,"15min":1.232},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.744,"5min":2.319,"15min":2.319},"last":1.0820000000000001},{"interface":"front","average":{"1min":1.2809999999999999,"5min":1.3080000000000001,"15min":1.3080000000000001},"min":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":1.956,"5min":2.0990000000000002,"15min":2.0990000000000002},"last":1.389}]},{"osd":3,"last update":"Fri Nov 24 06:13:24 2023","interfaces":[{"interface":"back","average":{"1min":1.1839999999999999,"5min":1.325,"15min":1.383},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":2.2269999999999999,"5min":2.2269999999999999,"15min":2.2269999999999999},"last":1.464},{"interface":"front","average":{"1min":1.3180000000000001,"5min":1.395,"15min":1.4259999999999999},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":2.0990000000000002,"5min":2.1720000000000002,"15min":2.1720000000000002},"last":1.5349999999999999}]},{"osd":5,"last update":"Fri Nov 24 06:13:24 2023","interfaces":[{"interface":"back","average":{"1min":1.274,"5min":1.3140000000000001,"15min":1.3300000000000001},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":1.9870000000000001,"5min":1.9870000000000001,"15min":1.9870000000000001},"last":1.2410000000000001},{"interface":"front","average":{"1min":1.3979999999999999,"5min":1.498,"15min":1.534},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":2.2890000000000001,"5min":2.5059999999999998,"15min":2.5059999999999998},"last":1.7569999999999999}]},{"osd":6,"last update":"Fri Nov 24 06:13:24 2023","interfaces":[{"interface":"back","average":{"1min":1.2569999999999999,"5min":1.464,"15min":1.5189999999999999},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.7729999999999999,"5min":2.2069999999999999,"15min":2.2069999999999999},"last":1.5669999999999999},{"interface":"front","average":{"1min":1.5109999999999999,"5min":1.6299999999999999,"15min":1.6659999999999999},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":2.3679999999999999,"5min":2.4510000000000001,"15min":2.4510000000000001},"last":1.8580000000000001}]},{"osd":7,"last update":"Fri Nov 24 06:13:24 2023","interfaces":[{"interface":"back","average":{"1min":1.369,"5min":1.573,"15min":1.6200000000000001},"min":{"1min":0.61299999999999999,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.8480000000000001,"5min":2.4350000000000001,"15min":2.4350000000000001},"last":1.6220000000000001},{"interface":"front","average":{"1min":1.454,"5min":1.581,"15min":1.615},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":2.173,"5min":3.1120000000000001,"15min":3.1120000000000001},"last":1.355}]}]},{"osd":3,"up_from":13,"seq":55834574891,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27788,"kb_used_data":768,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344052,"statfs":{"total":96636764160,"available":96608309248,"internally_reserved":0,"allocated":786432,"data_stored":334440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:13:30 2023","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.95599999999999996,"15min":0.98999999999999999},"min":{"1min":0.41799999999999998,"5min":0.314,"15min":0.314},"max":{"1min":1.3300000000000001,"5min":2.766,"15min":2.766},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.872,"5min":0.93999999999999995,"15min":0.92400000000000004},"min":{"1min":0.503,"5min":0.441,"15min":0.441},"max":{"1min":1.4850000000000001,"5min":1.873,"15min":1.873},"last":0.40000000000000002}]},{"osd":1,"last update":"Fri Nov 24 06:13:30 2023","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.86499999999999999,"15min":0.878},"min":{"1min":0.434,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.3129999999999999,"5min":2.5779999999999998,"15min":2.5779999999999998},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.96599999999999997,"5min":1.107,"15min":1.1579999999999999},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.49,"5min":3.6899999999999999,"15min":3.6899999999999999},"last":0.57299999999999995}]},{"osd":2,"last update":"Fri Nov 24 06:13:30 2023","interfaces":[{"interface":"back","average":{"1min":1.012,"5min":1.107,"15min":1.153},"min":{"1min":0.53100000000000003,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.431,"5min":4.0620000000000003,"15min":4.0620000000000003},"last":0.55000000000000004},{"interface":"front","average":{"1min":1.131,"5min":1.1850000000000001,"15min":1.2130000000000001},"min":{"1min":0.58499999999999996,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.659,"5min":2.9529999999999998,"15min":2.9529999999999998},"last":0.53700000000000003}]},{"osd":4,"last update":"Fri Nov 24 06:13:30 2023","interfaces":[{"interface":"back","average":{"1min":1.117,"5min":1.3320000000000001,"15min":1.3959999999999999},"min":{"1min":0.71999999999999997,"5min":0.70999999999999996,"15min":0.70999999999999996},"max":{"1min":1.51,"5min":4.125,"15min":4.125},"last":0.81200000000000006},{"interface":"front","average":{"1min":1.206,"5min":1.379,"15min":1.4239999999999999},"min":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"max":{"1min":1.633,"5min":3.9409999999999998,"15min":3.9409999999999998},"last":0.86699999999999999}]},{"osd":5,"last update":"Fri Nov 24 06:13:30 2023","interfaces":[{"interface":"back","average":{"1min":1.335,"5min":1.399,"15min":1.431},"min":{"1min":0.79500000000000004,"5min":0.749,"15min":0.749},"max":{"1min":2.0609999999999999,"5min":3.3359999999999999,"15min":3.3359999999999999},"last":1.0609999999999999},{"interface":"front","average":{"1min":1.1910000000000001,"5min":1.357,"15min":1.425},"min":{"1min":0.78200000000000003,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.7929999999999999,"5min":4.1669999999999998,"15min":4.1669999999999998},"last":0.69699999999999995}]},{"osd":6,"last update":"Fri Nov 24 06:13:30 2023","interfaces":[{"interface":"back","average":{"1min":1.3260000000000001,"5min":1.3720000000000001,"15min":1.415},"min":{"1min":0.74199999999999999,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":2.169,"5min":4.0190000000000001,"15min":4.0190000000000001},"last":0.93500000000000005},{"interface":"front","average":{"1min":1.3759999999999999,"5min":1.48,"15min":1.5149999999999999},"min":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"max":{"1min":2.004,"5min":3.2480000000000002,"15min":3.2480000000000002},"last":1.0209999999999999}]},{"osd":7,"last update":"Fri Nov 24 06:13:30 2023","interfaces":[{"interface":"back","average":{"1min":1.258,"5min":1.4870000000000001,"15min":1.571},"min":{"1min":0.79700000000000004,"5min":0.76600000000000001,"15min":0.76600000000000001},"max":{"1min":1.831,"5min":4.202,"15min":4.202},"last":1.0409999999999999},{"interface":"front","average":{"1min":1.4099999999999999,"5min":1.524,"15min":1.589},"min":{"1min":1.0149999999999999,"5min":0.89700000000000002,"15min":0.89700000000000002},"max":{"1min":2.2989999999999999,"5min":3.9849999999999999,"15min":3.9849999999999999},"last":0.89600000000000002}]}]},{"osd":2,"up_from":13,"seq":55834574891,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27820,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344020,"statfs":{"total":96636764160,"available":96608276480,"internally_reserved":0,"allocated":819200,"data_stored":371124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:13:25 2023","interfaces":[{"interface":"back","average":{"1min":0.88800000000000001,"5min":0.88,"15min":0.88600000000000001},"min":{"1min":0.34000000000000002,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":1.1850000000000001,"5min":1.6259999999999999,"15min":1.6259999999999999},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.90700000000000003,"5min":1.0349999999999999,"15min":1.079},"min":{"1min":0.51300000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.1799999999999999,"5min":1.7669999999999999,"15min":1.7669999999999999},"last":0.95699999999999996}]},{"osd":1,"last update":"Fri Nov 24 06:13:25 2023","interfaces":[{"interface":"back","average":{"1min":0.90700000000000003,"5min":0.94299999999999995,"15min":0.93999999999999995},"min":{"1min":0.38200000000000001,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.294,"5min":1.635,"15min":1.635},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.91400000000000003,"5min":0.96299999999999997,"15min":0.97499999999999998},"min":{"1min":0.51600000000000001,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.579,"5min":1.8140000000000001,"15min":1.8140000000000001},"last":0.66400000000000003}]},{"osd":3,"last update":"Fri Nov 24 06:13:25 2023","interfaces":[{"interface":"back","average":{"1min":0.88700000000000001,"5min":1.014,"15min":1.0580000000000001},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.3400000000000001,"5min":2.3559999999999999,"15min":2.3559999999999999},"last":0.97599999999999998},{"interface":"front","average":{"1min":1.0489999999999999,"5min":1.1180000000000001,"15min":1.121},"min":{"1min":0.47899999999999998,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.339,"5min":2.1840000000000002,"15min":2.1840000000000002},"last":0.503}]},{"osd":4,"last update":"Fri Nov 24 06:13:25 2023","interfaces":[{"interface":"back","average":{"1min":1.228,"5min":1.379,"15min":1.405},"min":{"1min":0.73999999999999999,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.488,"5min":3.411,"15min":3.411},"last":1.046},{"interface":"front","average":{"1min":1.179,"5min":1.4059999999999999,"15min":1.472},"min":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"max":{"1min":1.5409999999999999,"5min":3.7109999999999999,"15min":3.7109999999999999},"last":0.72499999999999998}]},{"osd":5,"last update":"Fri Nov 24 06:13:25 2023","interfaces":[{"interface":"back","average":{"1min":1.1970000000000001,"5min":1.2829999999999999,"15min":1.3109999999999999},"min":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"max":{"1min":1.498,"5min":2.5259999999999998,"15min":2.5259999999999998},"last":1.2729999999999999},{"interface":"front","average":{"1min":1.3100000000000001,"5min":1.3320000000000001,"15min":1.3120000000000001},"min":{"1min":0.999,"5min":0.90400000000000003,"15min":0.90400000000000003},"max":{"1min":1.5409999999999999,"5min":2.1640000000000001,"15min":2.1640000000000001},"last":1.165}]},{"osd":6,"last update":"Fri Nov 24 06:13:25 2023","interfaces":[{"interface":"back","average":{"1min":1.2529999999999999,"5min":1.373,"15min":1.429},"min":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"max":{"1min":1.7170000000000001,"5min":2.3039999999999998,"15min":2.3039999999999998},"last":0.76700000000000002},{"interface":"front","average":{"1min":1.218,"5min":1.3939999999999999,"15min":1.444},"min":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":1.474,"5min":2.605,"15min":2.605},"last":1.1100000000000001}]},{"osd":7,"last update":"Fri Nov 24 06:13:25 2023","interfaces":[{"interface":"back","average":{"1min":1.2889999999999999,"5min":1.4870000000000001,"15min":1.5589999999999999},"min":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"max":{"1min":1.7829999999999999,"5min":2.3740000000000001,"15min":2.3740000000000001},"last":0.83399999999999996},{"interface":"front","average":{"1min":1.2689999999999999,"5min":1.3939999999999999,"15min":1.4419999999999999},"min":{"1min":0.89200000000000002,"5min":0.877,"15min":0.877},"max":{"1min":1.6299999999999999,"5min":2.5699999999999998,"15min":2.5699999999999998},"last":1.218}]}]},{"osd":1,"up_from":13,"seq":55834574890,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31992,"kb_used_data":812,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339848,"statfs":{"total":96636764160,"available":96604004352,"internally_reserved":0,"allocated":831488,"data_stored":372354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:13:21 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.85099999999999998,"15min":0.89900000000000002},"min":{"1min":0.36799999999999999,"5min":0.311,"15min":0.311},"max":{"1min":1.212,"5min":1.9299999999999999,"15min":1.9299999999999999},"last":0.317},{"interface":"front","average":{"1min":0.873,"5min":0.89500000000000002,"15min":0.92100000000000004},"min":{"1min":0.35099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":1.339,"5min":1.7669999999999999,"15min":1.7669999999999999},"last":0.40999999999999998}]},{"osd":2,"last update":"Fri Nov 24 06:13:21 2023","interfaces":[{"interface":"back","average":{"1min":0.97499999999999998,"5min":0.95599999999999996,"15min":0.998},"min":{"1min":0.496,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.605,"5min":2.5329999999999999,"15min":2.5329999999999999},"last":0.497},{"interface":"front","average":{"1min":1.079,"5min":1.0009999999999999,"15min":1.02},"min":{"1min":0.502,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.7190000000000001,"5min":1.7190000000000001,"15min":1.7190000000000001},"last":0.46300000000000002}]},{"osd":3,"last update":"Fri Nov 24 06:13:21 2023","interfaces":[{"interface":"back","average":{"1min":0.91700000000000004,"5min":0.96399999999999997,"15min":1.0129999999999999},"min":{"1min":0.46800000000000003,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.4119999999999999,"5min":1.841,"15min":1.841},"last":0.435},{"interface":"front","average":{"1min":1.1479999999999999,"5min":1.1200000000000001,"15min":1.1439999999999999},"min":{"1min":0.61899999999999999,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.643,"5min":2.8940000000000001,"15min":2.8940000000000001},"last":0.54900000000000004}]},{"osd":4,"last update":"Fri Nov 24 06:13:21 2023","interfaces":[{"interface":"back","average":{"1min":1.3149999999999999,"5min":1.2609999999999999,"15min":1.2749999999999999},"min":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.728,"5min":2.1930000000000001,"15min":2.1930000000000001},"last":1.4790000000000001},{"interface":"front","average":{"1min":1.1899999999999999,"5min":1.1819999999999999,"15min":1.194},"min":{"1min":0.86799999999999999,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":1.5640000000000001,"5min":1.784,"15min":1.784},"last":1.3560000000000001}]},{"osd":5,"last update":"Fri Nov 24 06:13:21 2023","interfaces":[{"interface":"back","average":{"1min":1.2430000000000001,"5min":1.3959999999999999,"15min":1.4850000000000001},"min":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.732,"5min":5.5449999999999999,"15min":5.5449999999999999},"last":1.5009999999999999},{"interface":"front","average":{"1min":1.361,"5min":1.444,"15min":1.524},"min":{"1min":0.90900000000000003,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":1.7270000000000001,"5min":6.0030000000000001,"15min":6.0030000000000001},"last":1.536}]},{"osd":6,"last update":"Fri Nov 24 06:13:21 2023","interfaces":[{"interface":"back","average":{"1min":1.3220000000000001,"5min":1.446,"15min":1.528},"min":{"1min":0.81100000000000005,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":1.8779999999999999,"5min":5.2809999999999997,"15min":5.2809999999999997},"last":1.395},{"interface":"front","average":{"1min":1.4079999999999999,"5min":1.423,"15min":1.4570000000000001},"min":{"1min":0.82999999999999996,"5min":0.749,"15min":0.749},"max":{"1min":1.819,"5min":4.327,"15min":4.327},"last":1.5760000000000001}]},{"osd":7,"last update":"Fri Nov 24 06:13:21 2023","interfaces":[{"interface":"back","average":{"1min":1.264,"5min":1.302,"15min":1.3400000000000001},"min":{"1min":0.85099999999999998,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":1.7849999999999999,"5min":2.3069999999999999,"15min":2.3069999999999999},"last":1.413},{"interface":"front","average":{"1min":1.415,"5min":1.492,"15min":1.5620000000000001},"min":{"1min":0.91700000000000004,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":2.0179999999999998,"5min":5.4500000000000002,"15min":5.4500000000000002},"last":1.4530000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574890,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32520,"kb_used_data":1340,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339320,"statfs":{"total":96636764160,"available":96603463680,"internally_reserved":0,"allocated":1372160,"data_stored":924696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Nov 24 06:13:25 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.84799999999999998,"15min":0.90800000000000003},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.145,"5min":2.3999999999999999,"15min":2.3999999999999999},"last":0.91300000000000003},{"interface":"front","average":{"1min":0.75800000000000001,"5min":1.024,"15min":1.133},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":1.54,"5min":4.8289999999999997,"15min":4.8289999999999997},"last":0.58799999999999997}]},{"osd":2,"last update":"Fri Nov 24 06:13:25 2023","interfaces":[{"interface":"back","average":{"1min":0.83799999999999997,"5min":0.97999999999999998,"15min":1.0349999999999999},"min":{"1min":0.42999999999999999,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.4019999999999999,"5min":2.5139999999999998,"15min":2.5139999999999998},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.79800000000000004,"5min":1.0229999999999999,"15min":1.0820000000000001},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.26,"5min":2.3599999999999999,"15min":2.3599999999999999},"last":0.51800000000000002}]},{"osd":3,"last update":"Fri Nov 24 06:13:25 2023","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":1.056,"15min":1.1020000000000001},"min":{"1min":0.48699999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.5369999999999999,"5min":2.5840000000000001,"15min":2.5840000000000001},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.91600000000000004,"5min":1.246,"15min":1.3580000000000001},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.4630000000000001,"5min":5.1230000000000002,"15min":5.1230000000000002},"last":0.97899999999999998}]},{"osd":4,"last update":"Fri Nov 24 06:13:25 2023","interfaces":[{"interface":"back","average":{"1min":1.0329999999999999,"5min":1.1799999999999999,"15min":1.248},"min":{"1min":0.70499999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.488,"5min":2.887,"15min":2.887},"last":1.173},{"interface":"front","average":{"1min":1.0720000000000001,"5min":1.2310000000000001,"15min":1.3009999999999999},"min":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.581,"5min":2.8530000000000002,"15min":2.8530000000000002},"last":1.238}]},{"osd":5,"last update":"Fri Nov 24 06:13:25 2023","interfaces":[{"interface":"back","average":{"1min":1.149,"5min":1.4219999999999999,"15min":1.524},"min":{"1min":0.72199999999999998,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.978,"5min":4.6299999999999999,"15min":4.6299999999999999},"last":1.288},{"interface":"front","average":{"1min":1.1060000000000001,"5min":1.232,"15min":1.2989999999999999},"min":{"1min":0.85399999999999998,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.7010000000000001,"5min":2.794,"15min":2.794},"last":1.1120000000000001}]},{"osd":6,"last update":"Fri Nov 24 06:13:25 2023","interfaces":[{"interface":"back","average":{"1min":1.244,"5min":1.508,"15min":1.617},"min":{"1min":0.73899999999999999,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.8620000000000001,"5min":4.798,"15min":4.798},"last":1.0940000000000001},{"interface":"front","average":{"1min":1.3100000000000001,"5min":1.5189999999999999,"15min":1.597},"min":{"1min":0.97099999999999997,"5min":0.77800000000000002,"15min":0.77800000000000002},"max":{"1min":1.841,"5min":4.7149999999999999,"15min":4.7149999999999999},"last":1.399}]},{"osd":7,"last update":"Fri Nov 24 06:13:25 2023","interfaces":[{"interface":"back","average":{"1min":1.2629999999999999,"5min":1.544,"15min":1.6579999999999999},"min":{"1min":0.85099999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.9770000000000001,"5min":4.7649999999999997,"15min":4.7649999999999997},"last":1.204},{"interface":"front","average":{"1min":1.3,"5min":1.5609999999999999,"15min":1.6499999999999999},"min":{"1min":0.89900000000000002,"5min":0.84599999999999997,"15min":0.84599999999999997},"max":{"1min":1.9159999999999999,"5min":4.6820000000000004,"15min":4.6820000000000004},"last":1.5149999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-24T06:13:41.883 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-24T06:13:42.095 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.093+0000 7f74d16e0640 1 Processor -- start 2023-11-24T06:13:42.095 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.093+0000 7f74d16e0640 1 -- start start 2023-11-24T06:13:42.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.095+0000 7f74d16e0640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f74cc13df80 0x7f74cc15bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:42.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.095+0000 7f74d16e0640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f74cc13e360 0x7f74cc0cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:42.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.095+0000 7f74d16e0640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f74cc15f9c0 0x7f74cc0cf7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:42.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.095+0000 7f74d16e0640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74cc06c470 con 0x7f74cc15f9c0 2023-11-24T06:13:42.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.095+0000 7f74d16e0640 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74cc06c750 con 0x7f74cc13df80 2023-11-24T06:13:42.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.095+0000 7f74d16e0640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74cc06ca30 con 0x7f74cc13e360 2023-11-24T06:13:42.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.095+0000 7f74cb7fe640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f74cc15f9c0 0x7f74cc0cf7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:42.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.095+0000 7f74cb7fe640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f74cc15f9c0 0x7f74cc0cf7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47782/0 (socket says 172.21.15.92:47782) 2023-11-24T06:13:42.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.095+0000 7f74cb7fe640 1 -- 172.21.15.92:0/690113132 learned_addr learned my addr 172.21.15.92:0/690113132 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:13:42.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.096+0000 7f74caffd640 1 --2- 172.21.15.92:0/690113132 >> v2:172.21.15.118:3300/0 conn(0x7f74cc13df80 0x7f74cc15bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:42.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.096+0000 7f74c3fff640 1 --2- 172.21.15.92:0/690113132 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f74cc13e360 0x7f74cc0cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:42.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.096+0000 7f74cb7fe640 1 -- 172.21.15.92:0/690113132 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f74cc13e360 msgr2=0x7f74cc0cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:13:42.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.096+0000 7f74cb7fe640 1 --2- 172.21.15.92:0/690113132 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f74cc13e360 0x7f74cc0cf290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:42.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.096+0000 7f74cb7fe640 1 -- 172.21.15.92:0/690113132 >> v2:172.21.15.118:3300/0 conn(0x7f74cc13df80 msgr2=0x7f74cc15bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:13:42.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.096+0000 7f74cb7fe640 1 --2- 172.21.15.92:0/690113132 >> v2:172.21.15.118:3300/0 conn(0x7f74cc13df80 0x7f74cc15bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:42.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.096+0000 7f74cb7fe640 1 -- 172.21.15.92:0/690113132 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74cc06cd10 con 0x7f74cc15f9c0 2023-11-24T06:13:42.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.097+0000 7f74cb7fe640 1 --2- 172.21.15.92:0/690113132 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f74cc15f9c0 0x7f74cc0cf7d0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f74cc072770 tx=0x7f74c402bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d28758781d7a569 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:42.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.098+0000 7f74ca7fc640 1 -- 172.21.15.92:0/690113132 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74c402ca80 con 0x7f74cc15f9c0 2023-11-24T06:13:42.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.098+0000 7f74ca7fc640 1 -- 172.21.15.92:0/690113132 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74c402cbe0 con 0x7f74cc15f9c0 2023-11-24T06:13:42.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.098+0000 7f74ca7fc640 1 -- 172.21.15.92:0/690113132 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74c40343f0 con 0x7f74cc15f9c0 2023-11-24T06:13:42.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.098+0000 7f74d16e0640 1 -- 172.21.15.92:0/690113132 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f74cc15f9c0 msgr2=0x7f74cc0cf7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:42.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.098+0000 7f74d16e0640 1 --2- 172.21.15.92:0/690113132 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f74cc15f9c0 0x7f74cc0cf7d0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f74cc072770 tx=0x7f74c402bee0 comp rx=0 tx=0).stop 2023-11-24T06:13:42.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.099+0000 7f74d16e0640 1 -- 172.21.15.92:0/690113132 shutdown_connections 2023-11-24T06:13:42.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.099+0000 7f74d16e0640 1 --2- 172.21.15.92:0/690113132 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f74cc15f9c0 0x7f74cc0cf7d0 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:42.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.099+0000 7f74d16e0640 1 --2- 172.21.15.92:0/690113132 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f74cc13e360 0x7f74cc0cf290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:42.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.099+0000 7f74d16e0640 1 --2- 172.21.15.92:0/690113132 >> v2:172.21.15.118:3300/0 conn(0x7f74cc13df80 0x7f74cc15bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:42.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.099+0000 7f74d16e0640 1 -- 172.21.15.92:0/690113132 >> 172.21.15.92:0/690113132 conn(0x7f74cc06a800 msgr2=0x7f74cc06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:13:42.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.099+0000 7f74d16e0640 1 -- 172.21.15.92:0/690113132 shutdown_connections 2023-11-24T06:13:42.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.099+0000 7f74d16e0640 1 -- 172.21.15.92:0/690113132 wait complete. 2023-11-24T06:13:42.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.101+0000 7f74d16e0640 1 Processor -- start 2023-11-24T06:13:42.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.101+0000 7f74d16e0640 1 -- start start 2023-11-24T06:13:42.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.102+0000 7f74d16e0640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f74cc13df80 0x7f74cc1505f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:42.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.102+0000 7f74d16e0640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f74cc13e360 0x7f74cc0b2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:42.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.102+0000 7f74d16e0640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f74cc15f9c0 0x7f74cc0b3230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:42.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.102+0000 7f74d16e0640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74cc14fc70 con 0x7f74cc13e360 2023-11-24T06:13:42.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.102+0000 7f74d16e0640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74cc151800 con 0x7f74cc15f9c0 2023-11-24T06:13:42.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.102+0000 7f74d16e0640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74cc14f990 con 0x7f74cc13df80 2023-11-24T06:13:42.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.102+0000 7f74c3fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f74cc13e360 0x7f74cc0b2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:42.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.102+0000 7f74c3fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f74cc13e360 0x7f74cc0b2cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47792/0 (socket says 172.21.15.92:47792) 2023-11-24T06:13:42.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.102+0000 7f74c3fff640 1 -- 172.21.15.92:0/2457204386 learned_addr learned my addr 172.21.15.92:0/2457204386 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:13:42.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.102+0000 7f74cb7fe640 1 --2- 172.21.15.92:0/2457204386 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f74cc15f9c0 0x7f74cc0b3230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:42.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.102+0000 7f74c3fff640 1 -- 172.21.15.92:0/2457204386 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f74cc13df80 msgr2=0x7f74cc1505f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:42.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.102+0000 7f74caffd640 1 --2- 172.21.15.92:0/2457204386 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f74cc13df80 0x7f74cc1505f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:42.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.102+0000 7f74c3fff640 1 --2- 172.21.15.92:0/2457204386 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f74cc13df80 0x7f74cc1505f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:42.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.102+0000 7f74c3fff640 1 -- 172.21.15.92:0/2457204386 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f74cc15f9c0 msgr2=0x7f74cc0b3230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:42.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.102+0000 7f74c3fff640 1 --2- 172.21.15.92:0/2457204386 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f74cc15f9c0 0x7f74cc0b3230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:42.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.102+0000 7f74c3fff640 1 -- 172.21.15.92:0/2457204386 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74c4009b30 con 0x7f74cc13e360 2023-11-24T06:13:42.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.102+0000 7f74cb7fe640 1 --2- 172.21.15.92:0/2457204386 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f74cc15f9c0 0x7f74cc0b3230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:13:42.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.103+0000 7f74c3fff640 1 --2- 172.21.15.92:0/2457204386 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f74cc13e360 0x7f74cc0b2cf0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f74b400d800 tx=0x7f74b400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:42.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.103+0000 7f74c37fe640 1 -- 172.21.15.92:0/2457204386 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74b400b6c0 con 0x7f74cc13e360 2023-11-24T06:13:42.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.103+0000 7f74c37fe640 1 -- 172.21.15.92:0/2457204386 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74b400b820 con 0x7f74cc13e360 2023-11-24T06:13:42.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.103+0000 7f74d16e0640 1 -- 172.21.15.92:0/2457204386 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74cc1511b0 con 0x7f74cc13e360 2023-11-24T06:13:42.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.103+0000 7f74c37fe640 1 -- 172.21.15.92:0/2457204386 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74b400bbb0 con 0x7f74cc13e360 2023-11-24T06:13:42.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.103+0000 7f74d16e0640 1 -- 172.21.15.92:0/2457204386 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74cc13ccb0 con 0x7f74cc13e360 2023-11-24T06:13:42.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.104+0000 7f74d16e0640 1 -- 172.21.15.92:0/2457204386 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74cc151010 con 0x7f74cc13e360 2023-11-24T06:13:42.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.105+0000 7f74c37fe640 1 -- 172.21.15.92:0/2457204386 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 7) v1 ==== 89333+0+0 (secure 0 0 0) 0x7f74b400bd10 con 0x7f74cc13e360 2023-11-24T06:13:42.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.106+0000 7f74c37fe640 1 --2- 172.21.15.92:0/2457204386 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f74ac06af60 0x7f74ac06d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:42.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.106+0000 7f74c37fe640 1 -- 172.21.15.92:0/2457204386 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f74b40899a0 con 0x7f74cc13e360 2023-11-24T06:13:42.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.106+0000 7f74caffd640 1 --2- 172.21.15.92:0/2457204386 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f74ac06af60 0x7f74ac06d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:42.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.107+0000 7f74caffd640 1 --2- 172.21.15.92:0/2457204386 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f74ac06af60 0x7f74ac06d420 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f74bc006fd0 tx=0x7f74bc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:42.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.110+0000 7f74c37fe640 1 -- 172.21.15.92:0/2457204386 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f74cc151010 con 0x7f74cc13e360 2023-11-24T06:13:42.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.298+0000 7f74d16e0640 1 -- 172.21.15.92:0/2457204386 --> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f74cc15ba80 con 0x7f74ac06af60 2023-11-24T06:13:42.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.301+0000 7f74c37fe640 1 -- 172.21.15.92:0/2457204386 <== mgr.4101 v2:172.21.15.92:6800/784027401 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+48683 (secure 0 0 0) 0x7f74cc15ba80 con 0x7f74ac06af60 2023-11-24T06:13:42.304 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-24T06:13:42.304 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2023-11-24T06:13:42.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.304+0000 7f74d16e0640 1 -- 172.21.15.92:0/2457204386 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f74ac06af60 msgr2=0x7f74ac06d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:42.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.304+0000 7f74d16e0640 1 --2- 172.21.15.92:0/2457204386 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f74ac06af60 0x7f74ac06d420 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f74bc006fd0 tx=0x7f74bc008040 comp rx=0 tx=0).stop 2023-11-24T06:13:42.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.304+0000 7f74d16e0640 1 -- 172.21.15.92:0/2457204386 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f74cc13e360 msgr2=0x7f74cc0b2cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:42.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.304+0000 7f74d16e0640 1 --2- 172.21.15.92:0/2457204386 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f74cc13e360 0x7f74cc0b2cf0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f74b400d800 tx=0x7f74b400dcc0 comp rx=0 tx=0).stop 2023-11-24T06:13:42.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.304+0000 7f74d16e0640 1 -- 172.21.15.92:0/2457204386 shutdown_connections 2023-11-24T06:13:42.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.304+0000 7f74d16e0640 1 --2- 172.21.15.92:0/2457204386 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f74ac06af60 0x7f74ac06d420 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:42.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.304+0000 7f74d16e0640 1 --2- 172.21.15.92:0/2457204386 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f74cc15f9c0 0x7f74cc0b3230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:42.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.304+0000 7f74d16e0640 1 --2- 172.21.15.92:0/2457204386 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f74cc13e360 0x7f74cc0b2cf0 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:42.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.304+0000 7f74d16e0640 1 --2- 172.21.15.92:0/2457204386 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f74cc13df80 0x7f74cc1505f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:42.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.304+0000 7f74d16e0640 1 -- 172.21.15.92:0/2457204386 >> 172.21.15.92:0/2457204386 conn(0x7f74cc06a800 msgr2=0x7f74cc13eb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:13:42.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.304+0000 7f74d16e0640 1 -- 172.21.15.92:0/2457204386 shutdown_connections 2023-11-24T06:13:42.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.304+0000 7f74d16e0640 1 -- 172.21.15.92:0/2457204386 wait complete. 2023-11-24T06:13:42.324 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":161,"stamp":"2023-11-24T06:13:41.727316+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":248488,"kb_used_data":7368,"kb_used_omap":12,"kb_used_meta":241011,"kb_avail":754726232,"statfs":{"total":773094113280,"available":772839661568,"internally_reserved":0,"allocated":7544832,"data_stored":3930806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":246795825},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-34068,"num_objects":-23,"num_object_clones":0,"num_object_copies":-46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-23,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-68,"num_write_kb":-57,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001340"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":203,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-24T06:13:10.170077+0000","last_change":"2023-11-24T06:10:20.082677+0000","last_active":"2023-11-24T06:13:10.170077+0000","last_peered":"2023-11-24T06:13:10.170077+0000","last_clean":"2023-11-24T06:13:10.170077+0000","last_became_active":"2023-11-24T06:10:20.080551+0000","last_became_peered":"2023-11-24T06:10:20.080551+0000","last_unstale":"2023-11-24T06:13:10.170077+0000","last_undegraded":"2023-11-24T06:13:10.170077+0000","last_fullsized":"2023-11-24T06:13:10.170077+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:18.602209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:18.602209+0000","last_clean_scrub_stamp":"2023-11-24T06:10:18.602209+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:47:24.216235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574890,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32532,"kb_used_data":1352,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339308,"statfs":{"total":96636764160,"available":96603451392,"internally_reserved":0,"allocated":1384448,"data_stored":925441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:13:18 2023","interfaces":[{"interface":"back","average":{"1min":0.99199999999999999,"5min":1.0669999999999999,"15min":1.095},"min":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":1.581,"5min":1.9630000000000001,"15min":1.9630000000000001},"last":0.60099999999999998},{"interface":"front","average":{"1min":1.0640000000000001,"5min":1.0569999999999999,"15min":1.0569999999999999},"min":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.413,"5min":1.621,"15min":1.621},"last":0.872}]},{"osd":1,"last update":"Fri Nov 24 06:13:18 2023","interfaces":[{"interface":"back","average":{"1min":1.0620000000000001,"5min":1.113,"15min":1.121},"min":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":1.569,"5min":2.0259999999999998,"15min":2.0259999999999998},"last":1.123},{"interface":"front","average":{"1min":1.274,"5min":1.1779999999999999,"15min":1.1459999999999999},"min":{"1min":0.72799999999999998,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.6579999999999999,"5min":2.1480000000000001,"15min":2.1480000000000001},"last":0.75700000000000001}]},{"osd":2,"last update":"Fri Nov 24 06:13:18 2023","interfaces":[{"interface":"back","average":{"1min":1.1439999999999999,"5min":1.1339999999999999,"15min":1.1399999999999999},"min":{"1min":0.72399999999999998,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.4770000000000001,"5min":1.7809999999999999,"15min":1.7809999999999999},"last":0.81799999999999995},{"interface":"front","average":{"1min":1.222,"5min":1.1950000000000001,"15min":1.179},"min":{"1min":0.88100000000000001,"5min":0.86599999999999999,"15min":0.86599999999999999},"max":{"1min":1.599,"5min":2.0619999999999998,"15min":2.0619999999999998},"last":1.0029999999999999}]},{"osd":3,"last update":"Fri Nov 24 06:13:18 2023","interfaces":[{"interface":"back","average":{"1min":1.077,"5min":1.1779999999999999,"15min":1.196},"min":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":1.4339999999999999,"5min":1.615,"15min":1.615},"last":0.95399999999999996},{"interface":"front","average":{"1min":1.28,"5min":1.3,"15min":1.3069999999999999},"min":{"1min":0.82699999999999996,"5min":0.81999999999999995,"15min":0.81999999999999995},"max":{"1min":1.778,"5min":1.9570000000000001,"15min":1.9570000000000001},"last":0.72599999999999998}]},{"osd":4,"last update":"Fri Nov 24 06:13:18 2023","interfaces":[{"interface":"back","average":{"1min":1.379,"5min":1.3220000000000001,"15min":1.3},"min":{"1min":0.60399999999999998,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":2.1739999999999999,"5min":2.1739999999999999,"15min":2.1739999999999999},"last":1.091},{"interface":"front","average":{"1min":1.2829999999999999,"5min":1.3200000000000001,"15min":1.3260000000000001},"min":{"1min":0.621,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.7809999999999999,"5min":2.1680000000000001,"15min":2.1680000000000001},"last":1.0529999999999999}]},{"osd":5,"last update":"Fri Nov 24 06:13:18 2023","interfaces":[{"interface":"back","average":{"1min":1.502,"5min":1.4770000000000001,"15min":1.456},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.964,"5min":2.0379999999999998,"15min":2.0379999999999998},"last":1.294},{"interface":"front","average":{"1min":1.4730000000000001,"5min":1.4810000000000001,"15min":1.4630000000000001},"min":{"1min":0.69699999999999995,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.915,"5min":2.0539999999999998,"15min":2.0539999999999998},"last":1.4330000000000001}]},{"osd":6,"last update":"Fri Nov 24 06:13:18 2023","interfaces":[{"interface":"back","average":{"1min":1.647,"5min":1.573,"15min":1.538},"min":{"1min":0.66000000000000003,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":2.1320000000000001,"5min":2.1320000000000001,"15min":2.1320000000000001},"last":1.194},{"interface":"front","average":{"1min":1.5649999999999999,"5min":1.5649999999999999,"15min":1.5409999999999999},"min":{"1min":0.75800000000000001,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":2.0310000000000001,"5min":2.0680000000000001,"15min":2.0680000000000001},"last":1.327}]}]},{"osd":6,"up_from":13,"seq":55834574891,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31948,"kb_used_data":768,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339892,"statfs":{"total":96636764160,"available":96604049408,"internally_reserved":0,"allocated":786432,"data_stored":334420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:13:24 2023","interfaces":[{"interface":"back","average":{"1min":1.0580000000000001,"5min":1.1619999999999999,"15min":1.2},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.45,"5min":2.032,"15min":2.032},"last":1.022},{"interface":"front","average":{"1min":1.0860000000000001,"5min":1.1850000000000001,"15min":1.2110000000000001},"min":{"1min":0.65700000000000003,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.637,"5min":2.0230000000000001,"15min":2.0230000000000001},"last":0.83299999999999996}]},{"osd":1,"last update":"Fri Nov 24 06:13:24 2023","interfaces":[{"interface":"back","average":{"1min":0.97799999999999998,"5min":1.125,"15min":1.153},"min":{"1min":0.60099999999999998,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.571,"5min":2.1499999999999999,"15min":2.1499999999999999},"last":1.123},{"interface":"front","average":{"1min":1.1659999999999999,"5min":1.2130000000000001,"15min":1.2270000000000001},"min":{"1min":0.872,"5min":0.77500000000000002,"15min":0.77500000000000002},"max":{"1min":1.653,"5min":1.8899999999999999,"15min":1.8899999999999999},"last":1.0580000000000001}]},{"osd":2,"last update":"Fri Nov 24 06:13:24 2023","interfaces":[{"interface":"back","average":{"1min":1.05,"5min":1.1839999999999999,"15min":1.1619999999999999},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.5009999999999999,"5min":4.7359999999999998,"15min":4.7359999999999998},"last":0.872},{"interface":"front","average":{"1min":1.252,"5min":1.413,"15min":1.4299999999999999},"min":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":1.8380000000000001,"5min":3.9700000000000002,"15min":3.9700000000000002},"last":1.2010000000000001}]},{"osd":3,"last update":"Fri Nov 24 06:13:24 2023","interfaces":[{"interface":"back","average":{"1min":1.274,"5min":1.2989999999999999,"15min":1.3089999999999999},"min":{"1min":0.94999999999999996,"5min":0.78500000000000003,"15min":0.78500000000000003},"max":{"1min":1.6120000000000001,"5min":1.919,"15min":1.919},"last":0.94999999999999996},{"interface":"front","average":{"1min":1.2609999999999999,"5min":1.2849999999999999,"15min":1.292},"min":{"1min":0.77200000000000002,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":1.605,"5min":2.3929999999999998,"15min":2.3929999999999998},"last":0.97999999999999998}]},{"osd":4,"last update":"Fri Nov 24 06:13:24 2023","interfaces":[{"interface":"back","average":{"1min":1.264,"5min":1.417,"15min":1.444},"min":{"1min":0.70699999999999996,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.9610000000000001,"5min":2.4039999999999999,"15min":2.4039999999999999},"last":1.446},{"interface":"front","average":{"1min":1.335,"5min":1.4670000000000001,"15min":1.4750000000000001},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.8029999999999999,"5min":2.5609999999999999,"15min":2.5609999999999999},"last":1.151}]},{"osd":5,"last update":"Fri Nov 24 06:13:24 2023","interfaces":[{"interface":"back","average":{"1min":1.306,"5min":1.4390000000000001,"15min":1.464},"min":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"max":{"1min":1.9159999999999999,"5min":2.8290000000000002,"15min":2.8290000000000002},"last":1.0920000000000001},{"interface":"front","average":{"1min":1.4590000000000001,"5min":1.613,"15min":1.6419999999999999},"min":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"max":{"1min":2.0259999999999998,"5min":2.9420000000000002,"15min":2.9420000000000002},"last":1.5}]},{"osd":7,"last update":"Fri Nov 24 06:13:24 2023","interfaces":[{"interface":"back","average":{"1min":1.403,"5min":1.5580000000000001,"15min":1.5800000000000001},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.772,"5min":2.5139999999999998,"15min":2.5139999999999998},"last":1.367},{"interface":"front","average":{"1min":1.421,"5min":1.5629999999999999,"15min":1.5529999999999999},"min":{"1min":0.79300000000000004,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":1.992,"5min":4.7859999999999996,"15min":4.7859999999999996},"last":1.29}]}]},{"osd":5,"up_from":13,"seq":55834574891,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31944,"kb_used_data":764,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339896,"statfs":{"total":96636764160,"available":96604053504,"internally_reserved":0,"allocated":782336,"data_stored":333933,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:13:25 2023","interfaces":[{"interface":"back","average":{"1min":0.97999999999999998,"5min":1.1339999999999999,"15min":1.212},"min":{"1min":0.62,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.526,"5min":3.0030000000000001,"15min":3.0030000000000001},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.998,"5min":1.1339999999999999,"15min":1.196},"min":{"1min":0.623,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.504,"5min":2.0979999999999999,"15min":2.0979999999999999},"last":0.81100000000000005}]},{"osd":1,"last update":"Fri Nov 24 06:13:25 2023","interfaces":[{"interface":"back","average":{"1min":1.0720000000000001,"5min":1.212,"15min":1.254},"min":{"1min":0.63300000000000001,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.665,"5min":5.0499999999999998,"15min":5.0499999999999998},"last":0.98099999999999998},{"interface":"front","average":{"1min":1.101,"5min":1.3180000000000001,"15min":1.4259999999999999},"min":{"1min":0.72599999999999998,"5min":0.435,"15min":0.435},"max":{"1min":1.5449999999999999,"5min":3.6930000000000001,"15min":3.6930000000000001},"last":1.046}]},{"osd":2,"last update":"Fri Nov 24 06:13:25 2023","interfaces":[{"interface":"back","average":{"1min":1.103,"5min":1.2589999999999999,"15min":1.3160000000000001},"min":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.5960000000000001,"5min":3.085,"15min":3.085},"last":1.4059999999999999},{"interface":"front","average":{"1min":1.2230000000000001,"5min":1.4510000000000001,"15min":1.5609999999999999},"min":{"1min":0.746,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.5980000000000001,"5min":4.7359999999999998,"15min":4.7359999999999998},"last":1.085}]},{"osd":3,"last update":"Fri Nov 24 06:13:25 2023","interfaces":[{"interface":"back","average":{"1min":1.246,"5min":1.3500000000000001,"15min":1.397},"min":{"1min":0.747,"5min":0.73999999999999999,"15min":0.73999999999999999},"max":{"1min":1.6579999999999999,"5min":3.2229999999999999,"15min":3.2229999999999999},"last":1.347},{"interface":"front","average":{"1min":1.333,"5min":1.3919999999999999,"15min":1.427},"min":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"max":{"1min":1.7789999999999999,"5min":2.3260000000000001,"15min":2.3260000000000001},"last":0.88300000000000001}]},{"osd":4,"last update":"Fri Nov 24 06:13:25 2023","interfaces":[{"interface":"back","average":{"1min":1.333,"5min":1.482,"15min":1.538},"min":{"1min":0.95399999999999996,"5min":0.84299999999999997,"15min":0.84299999999999997},"max":{"1min":1.762,"5min":3.1230000000000002,"15min":3.1230000000000002},"last":1.147},{"interface":"front","average":{"1min":1.3400000000000001,"5min":1.478,"15min":1.532},"min":{"1min":0.84399999999999997,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.7509999999999999,"5min":2.4809999999999999,"15min":2.4809999999999999},"last":1.2170000000000001}]},{"osd":6,"last update":"Fri Nov 24 06:13:25 2023","interfaces":[{"interface":"back","average":{"1min":1.4630000000000001,"5min":1.6319999999999999,"15min":1.7150000000000001},"min":{"1min":1.2010000000000001,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.887,"5min":4.6529999999999996,"15min":4.6529999999999996},"last":1.1739999999999999},{"interface":"front","average":{"1min":1.5049999999999999,"5min":1.6479999999999999,"15min":1.716},"min":{"1min":1.226,"5min":0.90400000000000003,"15min":0.90400000000000003},"max":{"1min":1.863,"5min":4.3529999999999998,"15min":4.3529999999999998},"last":1.4710000000000001}]},{"osd":7,"last update":"Fri Nov 24 06:13:25 2023","interfaces":[{"interface":"back","average":{"1min":1.4490000000000001,"5min":1.615,"15min":1.679},"min":{"1min":0.98499999999999999,"5min":0.874,"15min":0.874},"max":{"1min":1.847,"5min":2.8839999999999999,"15min":2.8839999999999999},"last":1.514},{"interface":"front","average":{"1min":1.579,"5min":1.7509999999999999,"15min":1.8280000000000001},"min":{"1min":1.2709999999999999,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.9770000000000001,"5min":4.8739999999999997,"15min":4.8739999999999997},"last":1.3049999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574891,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31944,"kb_used_data":764,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339896,"statfs":{"total":96636764160,"available":96604053504,"internally_reserved":0,"allocated":782336,"data_stored":334398,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:13:24 2023","interfaces":[{"interface":"back","average":{"1min":1.01,"5min":1.052,"15min":1.0449999999999999},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.907,"5min":1.907,"15min":1.907},"last":0.66700000000000004},{"interface":"front","average":{"1min":1.046,"5min":1.1479999999999999,"15min":1.194},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.732,"5min":1.8220000000000001,"15min":1.8220000000000001},"last":0.753}]},{"osd":1,"last update":"Fri Nov 24 06:13:24 2023","interfaces":[{"interface":"back","average":{"1min":1.1990000000000001,"5min":1.141,"15min":1.127},"min":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.655,"5min":1.97,"15min":1.97},"last":1.6639999999999999},{"interface":"front","average":{"1min":1.0920000000000001,"5min":1.161,"15min":1.1719999999999999},"min":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.831,"5min":1.917,"15min":1.917},"last":0.82199999999999995}]},{"osd":2,"last update":"Fri Nov 24 06:13:24 2023","interfaces":[{"interface":"back","average":{"1min":1.119,"5min":1.194,"15min":1.232},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.744,"5min":2.319,"15min":2.319},"last":1.0820000000000001},{"interface":"front","average":{"1min":1.2809999999999999,"5min":1.3080000000000001,"15min":1.3080000000000001},"min":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":1.956,"5min":2.0990000000000002,"15min":2.0990000000000002},"last":1.389}]},{"osd":3,"last update":"Fri Nov 24 06:13:24 2023","interfaces":[{"interface":"back","average":{"1min":1.1839999999999999,"5min":1.325,"15min":1.383},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":2.2269999999999999,"5min":2.2269999999999999,"15min":2.2269999999999999},"last":1.464},{"interface":"front","average":{"1min":1.3180000000000001,"5min":1.395,"15min":1.4259999999999999},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":2.0990000000000002,"5min":2.1720000000000002,"15min":2.1720000000000002},"last":1.5349999999999999}]},{"osd":5,"last update":"Fri Nov 24 06:13:24 2023","interfaces":[{"interface":"back","average":{"1min":1.274,"5min":1.3140000000000001,"15min":1.3300000000000001},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":1.9870000000000001,"5min":1.9870000000000001,"15min":1.9870000000000001},"last":1.2410000000000001},{"interface":"front","average":{"1min":1.3979999999999999,"5min":1.498,"15min":1.534},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":2.2890000000000001,"5min":2.5059999999999998,"15min":2.5059999999999998},"last":1.7569999999999999}]},{"osd":6,"last update":"Fri Nov 24 06:13:24 2023","interfaces":[{"interface":"back","average":{"1min":1.2569999999999999,"5min":1.464,"15min":1.5189999999999999},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.7729999999999999,"5min":2.2069999999999999,"15min":2.2069999999999999},"last":1.5669999999999999},{"interface":"front","average":{"1min":1.5109999999999999,"5min":1.6299999999999999,"15min":1.6659999999999999},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":2.3679999999999999,"5min":2.4510000000000001,"15min":2.4510000000000001},"last":1.8580000000000001}]},{"osd":7,"last update":"Fri Nov 24 06:13:24 2023","interfaces":[{"interface":"back","average":{"1min":1.369,"5min":1.573,"15min":1.6200000000000001},"min":{"1min":0.61299999999999999,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.8480000000000001,"5min":2.4350000000000001,"15min":2.4350000000000001},"last":1.6220000000000001},{"interface":"front","average":{"1min":1.454,"5min":1.581,"15min":1.615},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":2.173,"5min":3.1120000000000001,"15min":3.1120000000000001},"last":1.355}]}]},{"osd":3,"up_from":13,"seq":55834574891,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27788,"kb_used_data":768,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344052,"statfs":{"total":96636764160,"available":96608309248,"internally_reserved":0,"allocated":786432,"data_stored":334440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:13:30 2023","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.95599999999999996,"15min":0.98999999999999999},"min":{"1min":0.41799999999999998,"5min":0.314,"15min":0.314},"max":{"1min":1.3300000000000001,"5min":2.766,"15min":2.766},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.872,"5min":0.93999999999999995,"15min":0.92400000000000004},"min":{"1min":0.503,"5min":0.441,"15min":0.441},"max":{"1min":1.4850000000000001,"5min":1.873,"15min":1.873},"last":0.40000000000000002}]},{"osd":1,"last update":"Fri Nov 24 06:13:30 2023","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.86499999999999999,"15min":0.878},"min":{"1min":0.434,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.3129999999999999,"5min":2.5779999999999998,"15min":2.5779999999999998},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.96599999999999997,"5min":1.107,"15min":1.1579999999999999},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.49,"5min":3.6899999999999999,"15min":3.6899999999999999},"last":0.57299999999999995}]},{"osd":2,"last update":"Fri Nov 24 06:13:30 2023","interfaces":[{"interface":"back","average":{"1min":1.012,"5min":1.107,"15min":1.153},"min":{"1min":0.53100000000000003,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.431,"5min":4.0620000000000003,"15min":4.0620000000000003},"last":0.55000000000000004},{"interface":"front","average":{"1min":1.131,"5min":1.1850000000000001,"15min":1.2130000000000001},"min":{"1min":0.58499999999999996,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.659,"5min":2.9529999999999998,"15min":2.9529999999999998},"last":0.53700000000000003}]},{"osd":4,"last update":"Fri Nov 24 06:13:30 2023","interfaces":[{"interface":"back","average":{"1min":1.117,"5min":1.3320000000000001,"15min":1.3959999999999999},"min":{"1min":0.71999999999999997,"5min":0.70999999999999996,"15min":0.70999999999999996},"max":{"1min":1.51,"5min":4.125,"15min":4.125},"last":0.81200000000000006},{"interface":"front","average":{"1min":1.206,"5min":1.379,"15min":1.4239999999999999},"min":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"max":{"1min":1.633,"5min":3.9409999999999998,"15min":3.9409999999999998},"last":0.86699999999999999}]},{"osd":5,"last update":"Fri Nov 24 06:13:30 2023","interfaces":[{"interface":"back","average":{"1min":1.335,"5min":1.399,"15min":1.431},"min":{"1min":0.79500000000000004,"5min":0.749,"15min":0.749},"max":{"1min":2.0609999999999999,"5min":3.3359999999999999,"15min":3.3359999999999999},"last":1.0609999999999999},{"interface":"front","average":{"1min":1.1910000000000001,"5min":1.357,"15min":1.425},"min":{"1min":0.78200000000000003,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.7929999999999999,"5min":4.1669999999999998,"15min":4.1669999999999998},"last":0.69699999999999995}]},{"osd":6,"last update":"Fri Nov 24 06:13:30 2023","interfaces":[{"interface":"back","average":{"1min":1.3260000000000001,"5min":1.3720000000000001,"15min":1.415},"min":{"1min":0.74199999999999999,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":2.169,"5min":4.0190000000000001,"15min":4.0190000000000001},"last":0.93500000000000005},{"interface":"front","average":{"1min":1.3759999999999999,"5min":1.48,"15min":1.5149999999999999},"min":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"max":{"1min":2.004,"5min":3.2480000000000002,"15min":3.2480000000000002},"last":1.0209999999999999}]},{"osd":7,"last update":"Fri Nov 24 06:13:30 2023","interfaces":[{"interface":"back","average":{"1min":1.258,"5min":1.4870000000000001,"15min":1.571},"min":{"1min":0.79700000000000004,"5min":0.76600000000000001,"15min":0.76600000000000001},"max":{"1min":1.831,"5min":4.202,"15min":4.202},"last":1.0409999999999999},{"interface":"front","average":{"1min":1.4099999999999999,"5min":1.524,"15min":1.589},"min":{"1min":1.0149999999999999,"5min":0.89700000000000002,"15min":0.89700000000000002},"max":{"1min":2.2989999999999999,"5min":3.9849999999999999,"15min":3.9849999999999999},"last":0.89600000000000002}]}]},{"osd":2,"up_from":13,"seq":55834574891,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27820,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344020,"statfs":{"total":96636764160,"available":96608276480,"internally_reserved":0,"allocated":819200,"data_stored":371124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:13:25 2023","interfaces":[{"interface":"back","average":{"1min":0.88800000000000001,"5min":0.88,"15min":0.88600000000000001},"min":{"1min":0.34000000000000002,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":1.1850000000000001,"5min":1.6259999999999999,"15min":1.6259999999999999},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.90700000000000003,"5min":1.0349999999999999,"15min":1.079},"min":{"1min":0.51300000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.1799999999999999,"5min":1.7669999999999999,"15min":1.7669999999999999},"last":0.95699999999999996}]},{"osd":1,"last update":"Fri Nov 24 06:13:25 2023","interfaces":[{"interface":"back","average":{"1min":0.90700000000000003,"5min":0.94299999999999995,"15min":0.93999999999999995},"min":{"1min":0.38200000000000001,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.294,"5min":1.635,"15min":1.635},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.91400000000000003,"5min":0.96299999999999997,"15min":0.97499999999999998},"min":{"1min":0.51600000000000001,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.579,"5min":1.8140000000000001,"15min":1.8140000000000001},"last":0.66400000000000003}]},{"osd":3,"last update":"Fri Nov 24 06:13:25 2023","interfaces":[{"interface":"back","average":{"1min":0.88700000000000001,"5min":1.014,"15min":1.0580000000000001},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.3400000000000001,"5min":2.3559999999999999,"15min":2.3559999999999999},"last":0.97599999999999998},{"interface":"front","average":{"1min":1.0489999999999999,"5min":1.1180000000000001,"15min":1.121},"min":{"1min":0.47899999999999998,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.339,"5min":2.1840000000000002,"15min":2.1840000000000002},"last":0.503}]},{"osd":4,"last update":"Fri Nov 24 06:13:25 2023","interfaces":[{"interface":"back","average":{"1min":1.228,"5min":1.379,"15min":1.405},"min":{"1min":0.73999999999999999,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.488,"5min":3.411,"15min":3.411},"last":1.046},{"interface":"front","average":{"1min":1.179,"5min":1.4059999999999999,"15min":1.472},"min":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"max":{"1min":1.5409999999999999,"5min":3.7109999999999999,"15min":3.7109999999999999},"last":0.72499999999999998}]},{"osd":5,"last update":"Fri Nov 24 06:13:25 2023","interfaces":[{"interface":"back","average":{"1min":1.1970000000000001,"5min":1.2829999999999999,"15min":1.3109999999999999},"min":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"max":{"1min":1.498,"5min":2.5259999999999998,"15min":2.5259999999999998},"last":1.2729999999999999},{"interface":"front","average":{"1min":1.3100000000000001,"5min":1.3320000000000001,"15min":1.3120000000000001},"min":{"1min":0.999,"5min":0.90400000000000003,"15min":0.90400000000000003},"max":{"1min":1.5409999999999999,"5min":2.1640000000000001,"15min":2.1640000000000001},"last":1.165}]},{"osd":6,"last update":"Fri Nov 24 06:13:25 2023","interfaces":[{"interface":"back","average":{"1min":1.2529999999999999,"5min":1.373,"15min":1.429},"min":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"max":{"1min":1.7170000000000001,"5min":2.3039999999999998,"15min":2.3039999999999998},"last":0.76700000000000002},{"interface":"front","average":{"1min":1.218,"5min":1.3939999999999999,"15min":1.444},"min":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":1.474,"5min":2.605,"15min":2.605},"last":1.1100000000000001}]},{"osd":7,"last update":"Fri Nov 24 06:13:25 2023","interfaces":[{"interface":"back","average":{"1min":1.2889999999999999,"5min":1.4870000000000001,"15min":1.5589999999999999},"min":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"max":{"1min":1.7829999999999999,"5min":2.3740000000000001,"15min":2.3740000000000001},"last":0.83399999999999996},{"interface":"front","average":{"1min":1.2689999999999999,"5min":1.3939999999999999,"15min":1.4419999999999999},"min":{"1min":0.89200000000000002,"5min":0.877,"15min":0.877},"max":{"1min":1.6299999999999999,"5min":2.5699999999999998,"15min":2.5699999999999998},"last":1.218}]}]},{"osd":1,"up_from":13,"seq":55834574890,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31992,"kb_used_data":812,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339848,"statfs":{"total":96636764160,"available":96604004352,"internally_reserved":0,"allocated":831488,"data_stored":372354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:13:21 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.85099999999999998,"15min":0.89900000000000002},"min":{"1min":0.36799999999999999,"5min":0.311,"15min":0.311},"max":{"1min":1.212,"5min":1.9299999999999999,"15min":1.9299999999999999},"last":0.317},{"interface":"front","average":{"1min":0.873,"5min":0.89500000000000002,"15min":0.92100000000000004},"min":{"1min":0.35099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":1.339,"5min":1.7669999999999999,"15min":1.7669999999999999},"last":0.40999999999999998}]},{"osd":2,"last update":"Fri Nov 24 06:13:21 2023","interfaces":[{"interface":"back","average":{"1min":0.97499999999999998,"5min":0.95599999999999996,"15min":0.998},"min":{"1min":0.496,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.605,"5min":2.5329999999999999,"15min":2.5329999999999999},"last":0.497},{"interface":"front","average":{"1min":1.079,"5min":1.0009999999999999,"15min":1.02},"min":{"1min":0.502,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.7190000000000001,"5min":1.7190000000000001,"15min":1.7190000000000001},"last":0.46300000000000002}]},{"osd":3,"last update":"Fri Nov 24 06:13:21 2023","interfaces":[{"interface":"back","average":{"1min":0.91700000000000004,"5min":0.96399999999999997,"15min":1.0129999999999999},"min":{"1min":0.46800000000000003,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.4119999999999999,"5min":1.841,"15min":1.841},"last":0.435},{"interface":"front","average":{"1min":1.1479999999999999,"5min":1.1200000000000001,"15min":1.1439999999999999},"min":{"1min":0.61899999999999999,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.643,"5min":2.8940000000000001,"15min":2.8940000000000001},"last":0.54900000000000004}]},{"osd":4,"last update":"Fri Nov 24 06:13:21 2023","interfaces":[{"interface":"back","average":{"1min":1.3149999999999999,"5min":1.2609999999999999,"15min":1.2749999999999999},"min":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.728,"5min":2.1930000000000001,"15min":2.1930000000000001},"last":1.4790000000000001},{"interface":"front","average":{"1min":1.1899999999999999,"5min":1.1819999999999999,"15min":1.194},"min":{"1min":0.86799999999999999,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":1.5640000000000001,"5min":1.784,"15min":1.784},"last":1.3560000000000001}]},{"osd":5,"last update":"Fri Nov 24 06:13:21 2023","interfaces":[{"interface":"back","average":{"1min":1.2430000000000001,"5min":1.3959999999999999,"15min":1.4850000000000001},"min":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.732,"5min":5.5449999999999999,"15min":5.5449999999999999},"last":1.5009999999999999},{"interface":"front","average":{"1min":1.361,"5min":1.444,"15min":1.524},"min":{"1min":0.90900000000000003,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":1.7270000000000001,"5min":6.0030000000000001,"15min":6.0030000000000001},"last":1.536}]},{"osd":6,"last update":"Fri Nov 24 06:13:21 2023","interfaces":[{"interface":"back","average":{"1min":1.3220000000000001,"5min":1.446,"15min":1.528},"min":{"1min":0.81100000000000005,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":1.8779999999999999,"5min":5.2809999999999997,"15min":5.2809999999999997},"last":1.395},{"interface":"front","average":{"1min":1.4079999999999999,"5min":1.423,"15min":1.4570000000000001},"min":{"1min":0.82999999999999996,"5min":0.749,"15min":0.749},"max":{"1min":1.819,"5min":4.327,"15min":4.327},"last":1.5760000000000001}]},{"osd":7,"last update":"Fri Nov 24 06:13:21 2023","interfaces":[{"interface":"back","average":{"1min":1.264,"5min":1.302,"15min":1.3400000000000001},"min":{"1min":0.85099999999999998,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":1.7849999999999999,"5min":2.3069999999999999,"15min":2.3069999999999999},"last":1.413},{"interface":"front","average":{"1min":1.415,"5min":1.492,"15min":1.5620000000000001},"min":{"1min":0.91700000000000004,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":2.0179999999999998,"5min":5.4500000000000002,"15min":5.4500000000000002},"last":1.4530000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574890,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32520,"kb_used_data":1340,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339320,"statfs":{"total":96636764160,"available":96603463680,"internally_reserved":0,"allocated":1372160,"data_stored":924696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Nov 24 06:13:25 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.84799999999999998,"15min":0.90800000000000003},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.145,"5min":2.3999999999999999,"15min":2.3999999999999999},"last":0.91300000000000003},{"interface":"front","average":{"1min":0.75800000000000001,"5min":1.024,"15min":1.133},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":1.54,"5min":4.8289999999999997,"15min":4.8289999999999997},"last":0.58799999999999997}]},{"osd":2,"last update":"Fri Nov 24 06:13:25 2023","interfaces":[{"interface":"back","average":{"1min":0.83799999999999997,"5min":0.97999999999999998,"15min":1.0349999999999999},"min":{"1min":0.42999999999999999,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.4019999999999999,"5min":2.5139999999999998,"15min":2.5139999999999998},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.79800000000000004,"5min":1.0229999999999999,"15min":1.0820000000000001},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.26,"5min":2.3599999999999999,"15min":2.3599999999999999},"last":0.51800000000000002}]},{"osd":3,"last update":"Fri Nov 24 06:13:25 2023","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":1.056,"15min":1.1020000000000001},"min":{"1min":0.48699999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.5369999999999999,"5min":2.5840000000000001,"15min":2.5840000000000001},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.91600000000000004,"5min":1.246,"15min":1.3580000000000001},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.4630000000000001,"5min":5.1230000000000002,"15min":5.1230000000000002},"last":0.97899999999999998}]},{"osd":4,"last update":"Fri Nov 24 06:13:25 2023","interfaces":[{"interface":"back","average":{"1min":1.0329999999999999,"5min":1.1799999999999999,"15min":1.248},"min":{"1min":0.70499999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.488,"5min":2.887,"15min":2.887},"last":1.173},{"interface":"front","average":{"1min":1.0720000000000001,"5min":1.2310000000000001,"15min":1.3009999999999999},"min":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.581,"5min":2.8530000000000002,"15min":2.8530000000000002},"last":1.238}]},{"osd":5,"last update":"Fri Nov 24 06:13:25 2023","interfaces":[{"interface":"back","average":{"1min":1.149,"5min":1.4219999999999999,"15min":1.524},"min":{"1min":0.72199999999999998,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.978,"5min":4.6299999999999999,"15min":4.6299999999999999},"last":1.288},{"interface":"front","average":{"1min":1.1060000000000001,"5min":1.232,"15min":1.2989999999999999},"min":{"1min":0.85399999999999998,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.7010000000000001,"5min":2.794,"15min":2.794},"last":1.1120000000000001}]},{"osd":6,"last update":"Fri Nov 24 06:13:25 2023","interfaces":[{"interface":"back","average":{"1min":1.244,"5min":1.508,"15min":1.617},"min":{"1min":0.73899999999999999,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.8620000000000001,"5min":4.798,"15min":4.798},"last":1.0940000000000001},{"interface":"front","average":{"1min":1.3100000000000001,"5min":1.5189999999999999,"15min":1.597},"min":{"1min":0.97099999999999997,"5min":0.77800000000000002,"15min":0.77800000000000002},"max":{"1min":1.841,"5min":4.7149999999999999,"15min":4.7149999999999999},"last":1.399}]},{"osd":7,"last update":"Fri Nov 24 06:13:25 2023","interfaces":[{"interface":"back","average":{"1min":1.2629999999999999,"5min":1.544,"15min":1.6579999999999999},"min":{"1min":0.85099999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.9770000000000001,"5min":4.7649999999999997,"15min":4.7649999999999997},"last":1.204},{"interface":"front","average":{"1min":1.3,"5min":1.5609999999999999,"15min":1.6499999999999999},"min":{"1min":0.89900000000000002,"5min":0.84599999999999997,"15min":0.84599999999999997},"max":{"1min":1.9159999999999999,"5min":4.6820000000000004,"15min":4.6820000000000004},"last":1.5149999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-24T06:13:42.325 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-11-24T06:13:42.326 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-24T06:13:42.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.525+0000 7fc94422d640 1 Processor -- start 2023-11-24T06:13:42.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.525+0000 7fc94422d640 1 -- start start 2023-11-24T06:13:42.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.527+0000 7fc94422d640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc93c0cf9a0 0x7fc93c166030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:42.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.527+0000 7fc94422d640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc93c15c5f0 con 0x7fc93c13e360 2023-11-24T06:13:42.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.527+0000 7fc94422d640 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc93c06c470 con 0x7fc93c13df80 2023-11-24T06:13:42.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.527+0000 7fc94422d640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc93c06c750 con 0x7fc93c0cf9a0 2023-11-24T06:13:42.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.527+0000 7fc9417c8640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fc93c13e360 0x7fc93c0cf290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:50326/0 (socket says 172.21.15.92:50326) 2023-11-24T06:13:42.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.527+0000 7fc9417c8640 1 -- 172.21.15.92:0/701469633 learned_addr learned my addr 172.21.15.92:0/701469633 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-24T06:13:42.532 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.528+0000 7fc940fc7640 1 -- 172.21.15.92:0/701469633 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 842423677 0 0) 0x7fc93c15c5f0 con 0x7fc93c13e360 2023-11-24T06:13:42.532 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.528+0000 7fc940fc7640 1 -- 172.21.15.92:0/701469633 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc928003520 con 0x7fc93c13e360 2023-11-24T06:13:42.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.528+0000 7fc9427ca640 1 --2- 172.21.15.92:0/701469633 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc93c0cf9a0 0x7fc93c166030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:42.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.528+0000 7fc940fc7640 1 -- 172.21.15.92:0/701469633 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3756971431 0 0) 0x7fc928003520 con 0x7fc93c13e360 2023-11-24T06:13:42.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.528+0000 7fc940fc7640 1 -- 172.21.15.92:0/701469633 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc93c0cf9a0 msgr2=0x7fc93c166030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:13:42.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.528+0000 7fc940fc7640 1 --2- 172.21.15.92:0/701469633 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc93c0cf9a0 0x7fc93c166030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:42.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.528+0000 7fc940fc7640 1 -- 172.21.15.92:0/701469633 >> v1:172.21.15.118:6789/0 conn(0x7fc93c13df80 legacy=0x7fc93c15bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:42.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.528+0000 7fc940fc7640 1 -- 172.21.15.92:0/701469633 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc93c06ca30 con 0x7fc93c13e360 2023-11-24T06:13:42.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.528+0000 7fc940fc7640 1 -- 172.21.15.92:0/701469633 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2607661350 0 0) 0x7fc92c001670 con 0x7fc93c13e360 2023-11-24T06:13:42.534 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.528+0000 7fc940fc7640 1 -- 172.21.15.92:0/701469633 >> v1:172.21.15.92:6789/0 conn(0x7fc93c13e360 legacy=0x7fc93c0cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:42.534 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.528+0000 7fc940fc7640 1 --2- 172.21.15.92:0/701469633 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc9280039a0 0x7fc928003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:42.534 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.528+0000 7fc940fc7640 1 --2- 172.21.15.92:0/701469633 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc93c0cf9a0 0x7fc9280047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:42.534 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.528+0000 7fc940fc7640 1 --2- 172.21.15.92:0/701469633 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc928004d00 0x7fc928005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:42.534 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.528+0000 7fc940fc7640 1 -- 172.21.15.92:0/701469633 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc93c06c750 con 0x7fc928004d00 2023-11-24T06:13:42.534 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.528+0000 7fc940fc7640 1 -- 172.21.15.92:0/701469633 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc92800bb10 con 0x7fc9280039a0 2023-11-24T06:13:42.534 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.528+0000 7fc940fc7640 1 -- 172.21.15.92:0/701469633 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc92800bcd0 con 0x7fc93c0cf9a0 2023-11-24T06:13:42.534 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.528+0000 7fc9427ca640 1 --2- 172.21.15.92:0/701469633 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc928004d00 0x7fc928005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:42.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.529+0000 7fc9417c8640 1 --2- 172.21.15.92:0/701469633 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc93c0cf9a0 0x7fc9280047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:42.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.529+0000 7fc941fc9640 1 --2- 172.21.15.92:0/701469633 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc9280039a0 0x7fc928003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:42.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.529+0000 7fc9427ca640 1 -- 172.21.15.92:0/701469633 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc93c0cf9a0 msgr2=0x7fc9280047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:13:42.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.529+0000 7fc9427ca640 1 --2- 172.21.15.92:0/701469633 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc93c0cf9a0 0x7fc9280047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:42.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.529+0000 7fc9427ca640 1 -- 172.21.15.92:0/701469633 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc9280039a0 msgr2=0x7fc928003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:13:42.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.529+0000 7fc9427ca640 1 --2- 172.21.15.92:0/701469633 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc9280039a0 0x7fc928003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:42.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.529+0000 7fc9427ca640 1 -- 172.21.15.92:0/701469633 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc928003520 con 0x7fc928004d00 2023-11-24T06:13:42.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.529+0000 7fc9417c8640 1 --2- 172.21.15.92:0/701469633 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc93c0cf9a0 0x7fc9280047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:13:42.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.529+0000 7fc941fc9640 1 --2- 172.21.15.92:0/701469633 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc9280039a0 0x7fc928003dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:13:42.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.530+0000 7fc9427ca640 1 --2- 172.21.15.92:0/701469633 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc928004d00 0x7fc928005180 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7fc934007330 tx=0x7fc9280057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8df1b113b5ae472 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:42.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.530+0000 7fc940fc7640 1 -- 172.21.15.92:0/701469633 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc93402cca0 con 0x7fc928004d00 2023-11-24T06:13:42.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.530+0000 7fc940fc7640 1 -- 172.21.15.92:0/701469633 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc93402ce00 con 0x7fc928004d00 2023-11-24T06:13:42.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.530+0000 7fc940fc7640 1 -- 172.21.15.92:0/701469633 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc934034660 con 0x7fc928004d00 2023-11-24T06:13:42.539 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.531+0000 7fc94422d640 1 -- 172.21.15.92:0/701469633 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc928004d00 msgr2=0x7fc928005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:42.539 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.531+0000 7fc94422d640 1 --2- 172.21.15.92:0/701469633 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc928004d00 0x7fc928005180 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7fc934007330 tx=0x7fc9280057f0 comp rx=0 tx=0).stop 2023-11-24T06:13:42.539 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.531+0000 7fc9427ca640 1 -- 172.21.15.92:0/701469633 reap_dead start 2023-11-24T06:13:42.539 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.531+0000 7fc94422d640 1 -- 172.21.15.92:0/701469633 shutdown_connections 2023-11-24T06:13:42.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.531+0000 7fc94422d640 1 -- 172.21.15.92:0/701469633 >> 172.21.15.92:0/701469633 conn(0x7fc93c06a800 msgr2=0x7fc93c06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:13:42.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.531+0000 7fc94422d640 1 -- 172.21.15.92:0/701469633 shutdown_connections 2023-11-24T06:13:42.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.531+0000 7fc94422d640 1 -- 172.21.15.92:0/701469633 wait complete. 2023-11-24T06:13:42.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.533+0000 7fc94422d640 1 Processor -- start 2023-11-24T06:13:42.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.533+0000 7fc94422d640 1 -- start start 2023-11-24T06:13:42.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.534+0000 7fc94422d640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc93c13cb50 0x7fc93c13d200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:42.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.534+0000 7fc94422d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc93c150720 0x7fc93c15d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:42.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.534+0000 7fc94422d640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc93c15dde0 0x7fc93c151490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:42.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.534+0000 7fc94422d640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc93c152ff0 con 0x7fc93c150720 2023-11-24T06:13:42.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.534+0000 7fc94422d640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc93c15e260 con 0x7fc93c13cb50 2023-11-24T06:13:42.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.534+0000 7fc94422d640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc93c1512c0 con 0x7fc93c15dde0 2023-11-24T06:13:42.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.535+0000 7fc9417c8640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc93c150720 0x7fc93c15d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:42.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.535+0000 7fc9417c8640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc93c150720 0x7fc93c15d8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47814/0 (socket says 172.21.15.92:47814) 2023-11-24T06:13:42.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.535+0000 7fc9417c8640 1 -- 172.21.15.92:0/3691345901 learned_addr learned my addr 172.21.15.92:0/3691345901 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:13:42.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.535+0000 7fc941fc9640 1 --2- 172.21.15.92:0/3691345901 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc93c13cb50 0x7fc93c13d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:42.542 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.535+0000 7fc9417c8640 1 -- 172.21.15.92:0/3691345901 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc93c15dde0 msgr2=0x7fc93c151490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:42.542 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.535+0000 7fc9427ca640 1 --2- 172.21.15.92:0/3691345901 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc93c15dde0 0x7fc93c151490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:42.542 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.535+0000 7fc9417c8640 1 --2- 172.21.15.92:0/3691345901 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc93c15dde0 0x7fc93c151490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:42.542 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.535+0000 7fc9417c8640 1 -- 172.21.15.92:0/3691345901 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc93c13cb50 msgr2=0x7fc93c13d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:42.542 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.535+0000 7fc9417c8640 1 --2- 172.21.15.92:0/3691345901 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc93c13cb50 0x7fc93c13d200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:42.542 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.535+0000 7fc9417c8640 1 -- 172.21.15.92:0/3691345901 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc93c150fe0 con 0x7fc93c150720 2023-11-24T06:13:42.542 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.535+0000 7fc9417c8640 1 --2- 172.21.15.92:0/3691345901 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc93c150720 0x7fc93c15d8a0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7fc92c00dae0 tx=0x7fc92c007e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:42.543 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.536+0000 7fc9327fc640 1 -- 172.21.15.92:0/3691345901 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc92c004cb0 con 0x7fc93c150720 2023-11-24T06:13:42.543 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.536+0000 7fc9327fc640 1 -- 172.21.15.92:0/3691345901 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc92c004e10 con 0x7fc93c150720 2023-11-24T06:13:42.543 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.536+0000 7fc94422d640 1 -- 172.21.15.92:0/3691345901 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc93c152930 con 0x7fc93c150720 2023-11-24T06:13:42.543 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.536+0000 7fc9327fc640 1 -- 172.21.15.92:0/3691345901 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc92c0072d0 con 0x7fc93c150720 2023-11-24T06:13:42.543 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.536+0000 7fc94422d640 1 -- 172.21.15.92:0/3691345901 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc93c13cf30 con 0x7fc93c150720 2023-11-24T06:13:42.543 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.537+0000 7fc94422d640 1 -- 172.21.15.92:0/3691345901 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc93c153130 con 0x7fc93c150720 2023-11-24T06:13:42.544 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.538+0000 7fc9327fc640 1 -- 172.21.15.92:0/3691345901 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 7) v1 ==== 89333+0+0 (secure 0 0 0) 0x7fc92c007430 con 0x7fc93c150720 2023-11-24T06:13:42.544 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.539+0000 7fc9327fc640 1 --2- 172.21.15.92:0/3691345901 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fc92006af60 0x7fc92006d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:42.544 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.539+0000 7fc9327fc640 1 -- 172.21.15.92:0/3691345901 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fc92c089390 con 0x7fc93c150720 2023-11-24T06:13:42.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.539+0000 7fc941fc9640 1 --2- 172.21.15.92:0/3691345901 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fc92006af60 0x7fc92006d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:42.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.540+0000 7fc941fc9640 1 --2- 172.21.15.92:0/3691345901 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fc92006af60 0x7fc92006d420 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fc93c14e7e0 tx=0x7fc938009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:42.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.543+0000 7fc9327fc640 1 -- 172.21.15.92:0/3691345901 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc93c153130 con 0x7fc93c150720 2023-11-24T06:13:42.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.730+0000 7fc94422d640 1 -- 172.21.15.92:0/3691345901 --> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc93c06cff0 con 0x7fc92006af60 2023-11-24T06:13:42.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.736+0000 7fc9327fc640 1 -- 172.21.15.92:0/3691345901 <== mgr.4101 v2:172.21.15.92:6800/784027401 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+48683 (secure 0 0 0) 0x7fc93800d050 con 0x7fc92006af60 2023-11-24T06:13:42.738 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-24T06:13:42.739 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2023-11-24T06:13:42.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.738+0000 7fc94422d640 1 -- 172.21.15.92:0/3691345901 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fc92006af60 msgr2=0x7fc92006d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:42.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.738+0000 7fc94422d640 1 --2- 172.21.15.92:0/3691345901 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fc92006af60 0x7fc92006d420 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fc93c14e7e0 tx=0x7fc938009040 comp rx=0 tx=0).stop 2023-11-24T06:13:42.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.738+0000 7fc94422d640 1 -- 172.21.15.92:0/3691345901 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc93c150720 msgr2=0x7fc93c15d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:42.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.738+0000 7fc94422d640 1 --2- 172.21.15.92:0/3691345901 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc93c150720 0x7fc93c15d8a0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7fc92c00dae0 tx=0x7fc92c007e50 comp rx=0 tx=0).stop 2023-11-24T06:13:42.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.738+0000 7fc94422d640 1 -- 172.21.15.92:0/3691345901 shutdown_connections 2023-11-24T06:13:42.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.739+0000 7fc94422d640 1 --2- 172.21.15.92:0/3691345901 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fc92006af60 0x7fc92006d420 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:42.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.739+0000 7fc94422d640 1 --2- 172.21.15.92:0/3691345901 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc93c15dde0 0x7fc93c151490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:42.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.739+0000 7fc94422d640 1 --2- 172.21.15.92:0/3691345901 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc93c150720 0x7fc93c15d8a0 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:42.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.739+0000 7fc94422d640 1 --2- 172.21.15.92:0/3691345901 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc93c13cb50 0x7fc93c13d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:42.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.739+0000 7fc94422d640 1 -- 172.21.15.92:0/3691345901 >> 172.21.15.92:0/3691345901 conn(0x7fc93c06a800 msgr2=0x7fc93c0b01c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:13:42.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.739+0000 7fc94422d640 1 -- 172.21.15.92:0/3691345901 shutdown_connections 2023-11-24T06:13:42.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.739+0000 7fc94422d640 1 -- 172.21.15.92:0/3691345901 wait complete. 2023-11-24T06:13:42.759 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":161,"stamp":"2023-11-24T06:13:41.727316+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":248488,"kb_used_data":7368,"kb_used_omap":12,"kb_used_meta":241011,"kb_avail":754726232,"statfs":{"total":773094113280,"available":772839661568,"internally_reserved":0,"allocated":7544832,"data_stored":3930806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":246795825},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-34068,"num_objects":-23,"num_object_clones":0,"num_object_copies":-46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-23,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-68,"num_write_kb":-57,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001340"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":203,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-24T06:13:10.170077+0000","last_change":"2023-11-24T06:10:20.082677+0000","last_active":"2023-11-24T06:13:10.170077+0000","last_peered":"2023-11-24T06:13:10.170077+0000","last_clean":"2023-11-24T06:13:10.170077+0000","last_became_active":"2023-11-24T06:10:20.080551+0000","last_became_peered":"2023-11-24T06:10:20.080551+0000","last_unstale":"2023-11-24T06:13:10.170077+0000","last_undegraded":"2023-11-24T06:13:10.170077+0000","last_fullsized":"2023-11-24T06:13:10.170077+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:18.602209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:18.602209+0000","last_clean_scrub_stamp":"2023-11-24T06:10:18.602209+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:47:24.216235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574890,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32532,"kb_used_data":1352,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339308,"statfs":{"total":96636764160,"available":96603451392,"internally_reserved":0,"allocated":1384448,"data_stored":925441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:13:18 2023","interfaces":[{"interface":"back","average":{"1min":0.99199999999999999,"5min":1.0669999999999999,"15min":1.095},"min":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":1.581,"5min":1.9630000000000001,"15min":1.9630000000000001},"last":0.60099999999999998},{"interface":"front","average":{"1min":1.0640000000000001,"5min":1.0569999999999999,"15min":1.0569999999999999},"min":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.413,"5min":1.621,"15min":1.621},"last":0.872}]},{"osd":1,"last update":"Fri Nov 24 06:13:18 2023","interfaces":[{"interface":"back","average":{"1min":1.0620000000000001,"5min":1.113,"15min":1.121},"min":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":1.569,"5min":2.0259999999999998,"15min":2.0259999999999998},"last":1.123},{"interface":"front","average":{"1min":1.274,"5min":1.1779999999999999,"15min":1.1459999999999999},"min":{"1min":0.72799999999999998,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.6579999999999999,"5min":2.1480000000000001,"15min":2.1480000000000001},"last":0.75700000000000001}]},{"osd":2,"last update":"Fri Nov 24 06:13:18 2023","interfaces":[{"interface":"back","average":{"1min":1.1439999999999999,"5min":1.1339999999999999,"15min":1.1399999999999999},"min":{"1min":0.72399999999999998,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.4770000000000001,"5min":1.7809999999999999,"15min":1.7809999999999999},"last":0.81799999999999995},{"interface":"front","average":{"1min":1.222,"5min":1.1950000000000001,"15min":1.179},"min":{"1min":0.88100000000000001,"5min":0.86599999999999999,"15min":0.86599999999999999},"max":{"1min":1.599,"5min":2.0619999999999998,"15min":2.0619999999999998},"last":1.0029999999999999}]},{"osd":3,"last update":"Fri Nov 24 06:13:18 2023","interfaces":[{"interface":"back","average":{"1min":1.077,"5min":1.1779999999999999,"15min":1.196},"min":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":1.4339999999999999,"5min":1.615,"15min":1.615},"last":0.95399999999999996},{"interface":"front","average":{"1min":1.28,"5min":1.3,"15min":1.3069999999999999},"min":{"1min":0.82699999999999996,"5min":0.81999999999999995,"15min":0.81999999999999995},"max":{"1min":1.778,"5min":1.9570000000000001,"15min":1.9570000000000001},"last":0.72599999999999998}]},{"osd":4,"last update":"Fri Nov 24 06:13:18 2023","interfaces":[{"interface":"back","average":{"1min":1.379,"5min":1.3220000000000001,"15min":1.3},"min":{"1min":0.60399999999999998,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":2.1739999999999999,"5min":2.1739999999999999,"15min":2.1739999999999999},"last":1.091},{"interface":"front","average":{"1min":1.2829999999999999,"5min":1.3200000000000001,"15min":1.3260000000000001},"min":{"1min":0.621,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.7809999999999999,"5min":2.1680000000000001,"15min":2.1680000000000001},"last":1.0529999999999999}]},{"osd":5,"last update":"Fri Nov 24 06:13:18 2023","interfaces":[{"interface":"back","average":{"1min":1.502,"5min":1.4770000000000001,"15min":1.456},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.964,"5min":2.0379999999999998,"15min":2.0379999999999998},"last":1.294},{"interface":"front","average":{"1min":1.4730000000000001,"5min":1.4810000000000001,"15min":1.4630000000000001},"min":{"1min":0.69699999999999995,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.915,"5min":2.0539999999999998,"15min":2.0539999999999998},"last":1.4330000000000001}]},{"osd":6,"last update":"Fri Nov 24 06:13:18 2023","interfaces":[{"interface":"back","average":{"1min":1.647,"5min":1.573,"15min":1.538},"min":{"1min":0.66000000000000003,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":2.1320000000000001,"5min":2.1320000000000001,"15min":2.1320000000000001},"last":1.194},{"interface":"front","average":{"1min":1.5649999999999999,"5min":1.5649999999999999,"15min":1.5409999999999999},"min":{"1min":0.75800000000000001,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":2.0310000000000001,"5min":2.0680000000000001,"15min":2.0680000000000001},"last":1.327}]}]},{"osd":6,"up_from":13,"seq":55834574891,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31948,"kb_used_data":768,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339892,"statfs":{"total":96636764160,"available":96604049408,"internally_reserved":0,"allocated":786432,"data_stored":334420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:13:24 2023","interfaces":[{"interface":"back","average":{"1min":1.0580000000000001,"5min":1.1619999999999999,"15min":1.2},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.45,"5min":2.032,"15min":2.032},"last":1.022},{"interface":"front","average":{"1min":1.0860000000000001,"5min":1.1850000000000001,"15min":1.2110000000000001},"min":{"1min":0.65700000000000003,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.637,"5min":2.0230000000000001,"15min":2.0230000000000001},"last":0.83299999999999996}]},{"osd":1,"last update":"Fri Nov 24 06:13:24 2023","interfaces":[{"interface":"back","average":{"1min":0.97799999999999998,"5min":1.125,"15min":1.153},"min":{"1min":0.60099999999999998,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.571,"5min":2.1499999999999999,"15min":2.1499999999999999},"last":1.123},{"interface":"front","average":{"1min":1.1659999999999999,"5min":1.2130000000000001,"15min":1.2270000000000001},"min":{"1min":0.872,"5min":0.77500000000000002,"15min":0.77500000000000002},"max":{"1min":1.653,"5min":1.8899999999999999,"15min":1.8899999999999999},"last":1.0580000000000001}]},{"osd":2,"last update":"Fri Nov 24 06:13:24 2023","interfaces":[{"interface":"back","average":{"1min":1.05,"5min":1.1839999999999999,"15min":1.1619999999999999},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.5009999999999999,"5min":4.7359999999999998,"15min":4.7359999999999998},"last":0.872},{"interface":"front","average":{"1min":1.252,"5min":1.413,"15min":1.4299999999999999},"min":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":1.8380000000000001,"5min":3.9700000000000002,"15min":3.9700000000000002},"last":1.2010000000000001}]},{"osd":3,"last update":"Fri Nov 24 06:13:24 2023","interfaces":[{"interface":"back","average":{"1min":1.274,"5min":1.2989999999999999,"15min":1.3089999999999999},"min":{"1min":0.94999999999999996,"5min":0.78500000000000003,"15min":0.78500000000000003},"max":{"1min":1.6120000000000001,"5min":1.919,"15min":1.919},"last":0.94999999999999996},{"interface":"front","average":{"1min":1.2609999999999999,"5min":1.2849999999999999,"15min":1.292},"min":{"1min":0.77200000000000002,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":1.605,"5min":2.3929999999999998,"15min":2.3929999999999998},"last":0.97999999999999998}]},{"osd":4,"last update":"Fri Nov 24 06:13:24 2023","interfaces":[{"interface":"back","average":{"1min":1.264,"5min":1.417,"15min":1.444},"min":{"1min":0.70699999999999996,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.9610000000000001,"5min":2.4039999999999999,"15min":2.4039999999999999},"last":1.446},{"interface":"front","average":{"1min":1.335,"5min":1.4670000000000001,"15min":1.4750000000000001},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.8029999999999999,"5min":2.5609999999999999,"15min":2.5609999999999999},"last":1.151}]},{"osd":5,"last update":"Fri Nov 24 06:13:24 2023","interfaces":[{"interface":"back","average":{"1min":1.306,"5min":1.4390000000000001,"15min":1.464},"min":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"max":{"1min":1.9159999999999999,"5min":2.8290000000000002,"15min":2.8290000000000002},"last":1.0920000000000001},{"interface":"front","average":{"1min":1.4590000000000001,"5min":1.613,"15min":1.6419999999999999},"min":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"max":{"1min":2.0259999999999998,"5min":2.9420000000000002,"15min":2.9420000000000002},"last":1.5}]},{"osd":7,"last update":"Fri Nov 24 06:13:24 2023","interfaces":[{"interface":"back","average":{"1min":1.403,"5min":1.5580000000000001,"15min":1.5800000000000001},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.772,"5min":2.5139999999999998,"15min":2.5139999999999998},"last":1.367},{"interface":"front","average":{"1min":1.421,"5min":1.5629999999999999,"15min":1.5529999999999999},"min":{"1min":0.79300000000000004,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":1.992,"5min":4.7859999999999996,"15min":4.7859999999999996},"last":1.29}]}]},{"osd":5,"up_from":13,"seq":55834574891,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31944,"kb_used_data":764,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339896,"statfs":{"total":96636764160,"available":96604053504,"internally_reserved":0,"allocated":782336,"data_stored":333933,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:13:25 2023","interfaces":[{"interface":"back","average":{"1min":0.97999999999999998,"5min":1.1339999999999999,"15min":1.212},"min":{"1min":0.62,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.526,"5min":3.0030000000000001,"15min":3.0030000000000001},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.998,"5min":1.1339999999999999,"15min":1.196},"min":{"1min":0.623,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.504,"5min":2.0979999999999999,"15min":2.0979999999999999},"last":0.81100000000000005}]},{"osd":1,"last update":"Fri Nov 24 06:13:25 2023","interfaces":[{"interface":"back","average":{"1min":1.0720000000000001,"5min":1.212,"15min":1.254},"min":{"1min":0.63300000000000001,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.665,"5min":5.0499999999999998,"15min":5.0499999999999998},"last":0.98099999999999998},{"interface":"front","average":{"1min":1.101,"5min":1.3180000000000001,"15min":1.4259999999999999},"min":{"1min":0.72599999999999998,"5min":0.435,"15min":0.435},"max":{"1min":1.5449999999999999,"5min":3.6930000000000001,"15min":3.6930000000000001},"last":1.046}]},{"osd":2,"last update":"Fri Nov 24 06:13:25 2023","interfaces":[{"interface":"back","average":{"1min":1.103,"5min":1.2589999999999999,"15min":1.3160000000000001},"min":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.5960000000000001,"5min":3.085,"15min":3.085},"last":1.4059999999999999},{"interface":"front","average":{"1min":1.2230000000000001,"5min":1.4510000000000001,"15min":1.5609999999999999},"min":{"1min":0.746,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.5980000000000001,"5min":4.7359999999999998,"15min":4.7359999999999998},"last":1.085}]},{"osd":3,"last update":"Fri Nov 24 06:13:25 2023","interfaces":[{"interface":"back","average":{"1min":1.246,"5min":1.3500000000000001,"15min":1.397},"min":{"1min":0.747,"5min":0.73999999999999999,"15min":0.73999999999999999},"max":{"1min":1.6579999999999999,"5min":3.2229999999999999,"15min":3.2229999999999999},"last":1.347},{"interface":"front","average":{"1min":1.333,"5min":1.3919999999999999,"15min":1.427},"min":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"max":{"1min":1.7789999999999999,"5min":2.3260000000000001,"15min":2.3260000000000001},"last":0.88300000000000001}]},{"osd":4,"last update":"Fri Nov 24 06:13:25 2023","interfaces":[{"interface":"back","average":{"1min":1.333,"5min":1.482,"15min":1.538},"min":{"1min":0.95399999999999996,"5min":0.84299999999999997,"15min":0.84299999999999997},"max":{"1min":1.762,"5min":3.1230000000000002,"15min":3.1230000000000002},"last":1.147},{"interface":"front","average":{"1min":1.3400000000000001,"5min":1.478,"15min":1.532},"min":{"1min":0.84399999999999997,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.7509999999999999,"5min":2.4809999999999999,"15min":2.4809999999999999},"last":1.2170000000000001}]},{"osd":6,"last update":"Fri Nov 24 06:13:25 2023","interfaces":[{"interface":"back","average":{"1min":1.4630000000000001,"5min":1.6319999999999999,"15min":1.7150000000000001},"min":{"1min":1.2010000000000001,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.887,"5min":4.6529999999999996,"15min":4.6529999999999996},"last":1.1739999999999999},{"interface":"front","average":{"1min":1.5049999999999999,"5min":1.6479999999999999,"15min":1.716},"min":{"1min":1.226,"5min":0.90400000000000003,"15min":0.90400000000000003},"max":{"1min":1.863,"5min":4.3529999999999998,"15min":4.3529999999999998},"last":1.4710000000000001}]},{"osd":7,"last update":"Fri Nov 24 06:13:25 2023","interfaces":[{"interface":"back","average":{"1min":1.4490000000000001,"5min":1.615,"15min":1.679},"min":{"1min":0.98499999999999999,"5min":0.874,"15min":0.874},"max":{"1min":1.847,"5min":2.8839999999999999,"15min":2.8839999999999999},"last":1.514},{"interface":"front","average":{"1min":1.579,"5min":1.7509999999999999,"15min":1.8280000000000001},"min":{"1min":1.2709999999999999,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.9770000000000001,"5min":4.8739999999999997,"15min":4.8739999999999997},"last":1.3049999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574891,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31944,"kb_used_data":764,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339896,"statfs":{"total":96636764160,"available":96604053504,"internally_reserved":0,"allocated":782336,"data_stored":334398,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:13:24 2023","interfaces":[{"interface":"back","average":{"1min":1.01,"5min":1.052,"15min":1.0449999999999999},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.907,"5min":1.907,"15min":1.907},"last":0.66700000000000004},{"interface":"front","average":{"1min":1.046,"5min":1.1479999999999999,"15min":1.194},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.732,"5min":1.8220000000000001,"15min":1.8220000000000001},"last":0.753}]},{"osd":1,"last update":"Fri Nov 24 06:13:24 2023","interfaces":[{"interface":"back","average":{"1min":1.1990000000000001,"5min":1.141,"15min":1.127},"min":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.655,"5min":1.97,"15min":1.97},"last":1.6639999999999999},{"interface":"front","average":{"1min":1.0920000000000001,"5min":1.161,"15min":1.1719999999999999},"min":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.831,"5min":1.917,"15min":1.917},"last":0.82199999999999995}]},{"osd":2,"last update":"Fri Nov 24 06:13:24 2023","interfaces":[{"interface":"back","average":{"1min":1.119,"5min":1.194,"15min":1.232},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.744,"5min":2.319,"15min":2.319},"last":1.0820000000000001},{"interface":"front","average":{"1min":1.2809999999999999,"5min":1.3080000000000001,"15min":1.3080000000000001},"min":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":1.956,"5min":2.0990000000000002,"15min":2.0990000000000002},"last":1.389}]},{"osd":3,"last update":"Fri Nov 24 06:13:24 2023","interfaces":[{"interface":"back","average":{"1min":1.1839999999999999,"5min":1.325,"15min":1.383},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":2.2269999999999999,"5min":2.2269999999999999,"15min":2.2269999999999999},"last":1.464},{"interface":"front","average":{"1min":1.3180000000000001,"5min":1.395,"15min":1.4259999999999999},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":2.0990000000000002,"5min":2.1720000000000002,"15min":2.1720000000000002},"last":1.5349999999999999}]},{"osd":5,"last update":"Fri Nov 24 06:13:24 2023","interfaces":[{"interface":"back","average":{"1min":1.274,"5min":1.3140000000000001,"15min":1.3300000000000001},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":1.9870000000000001,"5min":1.9870000000000001,"15min":1.9870000000000001},"last":1.2410000000000001},{"interface":"front","average":{"1min":1.3979999999999999,"5min":1.498,"15min":1.534},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":2.2890000000000001,"5min":2.5059999999999998,"15min":2.5059999999999998},"last":1.7569999999999999}]},{"osd":6,"last update":"Fri Nov 24 06:13:24 2023","interfaces":[{"interface":"back","average":{"1min":1.2569999999999999,"5min":1.464,"15min":1.5189999999999999},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.7729999999999999,"5min":2.2069999999999999,"15min":2.2069999999999999},"last":1.5669999999999999},{"interface":"front","average":{"1min":1.5109999999999999,"5min":1.6299999999999999,"15min":1.6659999999999999},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":2.3679999999999999,"5min":2.4510000000000001,"15min":2.4510000000000001},"last":1.8580000000000001}]},{"osd":7,"last update":"Fri Nov 24 06:13:24 2023","interfaces":[{"interface":"back","average":{"1min":1.369,"5min":1.573,"15min":1.6200000000000001},"min":{"1min":0.61299999999999999,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.8480000000000001,"5min":2.4350000000000001,"15min":2.4350000000000001},"last":1.6220000000000001},{"interface":"front","average":{"1min":1.454,"5min":1.581,"15min":1.615},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":2.173,"5min":3.1120000000000001,"15min":3.1120000000000001},"last":1.355}]}]},{"osd":3,"up_from":13,"seq":55834574891,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27788,"kb_used_data":768,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344052,"statfs":{"total":96636764160,"available":96608309248,"internally_reserved":0,"allocated":786432,"data_stored":334440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:13:30 2023","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.95599999999999996,"15min":0.98999999999999999},"min":{"1min":0.41799999999999998,"5min":0.314,"15min":0.314},"max":{"1min":1.3300000000000001,"5min":2.766,"15min":2.766},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.872,"5min":0.93999999999999995,"15min":0.92400000000000004},"min":{"1min":0.503,"5min":0.441,"15min":0.441},"max":{"1min":1.4850000000000001,"5min":1.873,"15min":1.873},"last":0.40000000000000002}]},{"osd":1,"last update":"Fri Nov 24 06:13:30 2023","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.86499999999999999,"15min":0.878},"min":{"1min":0.434,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.3129999999999999,"5min":2.5779999999999998,"15min":2.5779999999999998},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.96599999999999997,"5min":1.107,"15min":1.1579999999999999},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.49,"5min":3.6899999999999999,"15min":3.6899999999999999},"last":0.57299999999999995}]},{"osd":2,"last update":"Fri Nov 24 06:13:30 2023","interfaces":[{"interface":"back","average":{"1min":1.012,"5min":1.107,"15min":1.153},"min":{"1min":0.53100000000000003,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.431,"5min":4.0620000000000003,"15min":4.0620000000000003},"last":0.55000000000000004},{"interface":"front","average":{"1min":1.131,"5min":1.1850000000000001,"15min":1.2130000000000001},"min":{"1min":0.58499999999999996,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.659,"5min":2.9529999999999998,"15min":2.9529999999999998},"last":0.53700000000000003}]},{"osd":4,"last update":"Fri Nov 24 06:13:30 2023","interfaces":[{"interface":"back","average":{"1min":1.117,"5min":1.3320000000000001,"15min":1.3959999999999999},"min":{"1min":0.71999999999999997,"5min":0.70999999999999996,"15min":0.70999999999999996},"max":{"1min":1.51,"5min":4.125,"15min":4.125},"last":0.81200000000000006},{"interface":"front","average":{"1min":1.206,"5min":1.379,"15min":1.4239999999999999},"min":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"max":{"1min":1.633,"5min":3.9409999999999998,"15min":3.9409999999999998},"last":0.86699999999999999}]},{"osd":5,"last update":"Fri Nov 24 06:13:30 2023","interfaces":[{"interface":"back","average":{"1min":1.335,"5min":1.399,"15min":1.431},"min":{"1min":0.79500000000000004,"5min":0.749,"15min":0.749},"max":{"1min":2.0609999999999999,"5min":3.3359999999999999,"15min":3.3359999999999999},"last":1.0609999999999999},{"interface":"front","average":{"1min":1.1910000000000001,"5min":1.357,"15min":1.425},"min":{"1min":0.78200000000000003,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.7929999999999999,"5min":4.1669999999999998,"15min":4.1669999999999998},"last":0.69699999999999995}]},{"osd":6,"last update":"Fri Nov 24 06:13:30 2023","interfaces":[{"interface":"back","average":{"1min":1.3260000000000001,"5min":1.3720000000000001,"15min":1.415},"min":{"1min":0.74199999999999999,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":2.169,"5min":4.0190000000000001,"15min":4.0190000000000001},"last":0.93500000000000005},{"interface":"front","average":{"1min":1.3759999999999999,"5min":1.48,"15min":1.5149999999999999},"min":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"max":{"1min":2.004,"5min":3.2480000000000002,"15min":3.2480000000000002},"last":1.0209999999999999}]},{"osd":7,"last update":"Fri Nov 24 06:13:30 2023","interfaces":[{"interface":"back","average":{"1min":1.258,"5min":1.4870000000000001,"15min":1.571},"min":{"1min":0.79700000000000004,"5min":0.76600000000000001,"15min":0.76600000000000001},"max":{"1min":1.831,"5min":4.202,"15min":4.202},"last":1.0409999999999999},{"interface":"front","average":{"1min":1.4099999999999999,"5min":1.524,"15min":1.589},"min":{"1min":1.0149999999999999,"5min":0.89700000000000002,"15min":0.89700000000000002},"max":{"1min":2.2989999999999999,"5min":3.9849999999999999,"15min":3.9849999999999999},"last":0.89600000000000002}]}]},{"osd":2,"up_from":13,"seq":55834574891,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27820,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344020,"statfs":{"total":96636764160,"available":96608276480,"internally_reserved":0,"allocated":819200,"data_stored":371124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:13:25 2023","interfaces":[{"interface":"back","average":{"1min":0.88800000000000001,"5min":0.88,"15min":0.88600000000000001},"min":{"1min":0.34000000000000002,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":1.1850000000000001,"5min":1.6259999999999999,"15min":1.6259999999999999},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.90700000000000003,"5min":1.0349999999999999,"15min":1.079},"min":{"1min":0.51300000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.1799999999999999,"5min":1.7669999999999999,"15min":1.7669999999999999},"last":0.95699999999999996}]},{"osd":1,"last update":"Fri Nov 24 06:13:25 2023","interfaces":[{"interface":"back","average":{"1min":0.90700000000000003,"5min":0.94299999999999995,"15min":0.93999999999999995},"min":{"1min":0.38200000000000001,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.294,"5min":1.635,"15min":1.635},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.91400000000000003,"5min":0.96299999999999997,"15min":0.97499999999999998},"min":{"1min":0.51600000000000001,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.579,"5min":1.8140000000000001,"15min":1.8140000000000001},"last":0.66400000000000003}]},{"osd":3,"last update":"Fri Nov 24 06:13:25 2023","interfaces":[{"interface":"back","average":{"1min":0.88700000000000001,"5min":1.014,"15min":1.0580000000000001},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.3400000000000001,"5min":2.3559999999999999,"15min":2.3559999999999999},"last":0.97599999999999998},{"interface":"front","average":{"1min":1.0489999999999999,"5min":1.1180000000000001,"15min":1.121},"min":{"1min":0.47899999999999998,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.339,"5min":2.1840000000000002,"15min":2.1840000000000002},"last":0.503}]},{"osd":4,"last update":"Fri Nov 24 06:13:25 2023","interfaces":[{"interface":"back","average":{"1min":1.228,"5min":1.379,"15min":1.405},"min":{"1min":0.73999999999999999,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.488,"5min":3.411,"15min":3.411},"last":1.046},{"interface":"front","average":{"1min":1.179,"5min":1.4059999999999999,"15min":1.472},"min":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"max":{"1min":1.5409999999999999,"5min":3.7109999999999999,"15min":3.7109999999999999},"last":0.72499999999999998}]},{"osd":5,"last update":"Fri Nov 24 06:13:25 2023","interfaces":[{"interface":"back","average":{"1min":1.1970000000000001,"5min":1.2829999999999999,"15min":1.3109999999999999},"min":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"max":{"1min":1.498,"5min":2.5259999999999998,"15min":2.5259999999999998},"last":1.2729999999999999},{"interface":"front","average":{"1min":1.3100000000000001,"5min":1.3320000000000001,"15min":1.3120000000000001},"min":{"1min":0.999,"5min":0.90400000000000003,"15min":0.90400000000000003},"max":{"1min":1.5409999999999999,"5min":2.1640000000000001,"15min":2.1640000000000001},"last":1.165}]},{"osd":6,"last update":"Fri Nov 24 06:13:25 2023","interfaces":[{"interface":"back","average":{"1min":1.2529999999999999,"5min":1.373,"15min":1.429},"min":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"max":{"1min":1.7170000000000001,"5min":2.3039999999999998,"15min":2.3039999999999998},"last":0.76700000000000002},{"interface":"front","average":{"1min":1.218,"5min":1.3939999999999999,"15min":1.444},"min":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":1.474,"5min":2.605,"15min":2.605},"last":1.1100000000000001}]},{"osd":7,"last update":"Fri Nov 24 06:13:25 2023","interfaces":[{"interface":"back","average":{"1min":1.2889999999999999,"5min":1.4870000000000001,"15min":1.5589999999999999},"min":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"max":{"1min":1.7829999999999999,"5min":2.3740000000000001,"15min":2.3740000000000001},"last":0.83399999999999996},{"interface":"front","average":{"1min":1.2689999999999999,"5min":1.3939999999999999,"15min":1.4419999999999999},"min":{"1min":0.89200000000000002,"5min":0.877,"15min":0.877},"max":{"1min":1.6299999999999999,"5min":2.5699999999999998,"15min":2.5699999999999998},"last":1.218}]}]},{"osd":1,"up_from":13,"seq":55834574890,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31992,"kb_used_data":812,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339848,"statfs":{"total":96636764160,"available":96604004352,"internally_reserved":0,"allocated":831488,"data_stored":372354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:13:21 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.85099999999999998,"15min":0.89900000000000002},"min":{"1min":0.36799999999999999,"5min":0.311,"15min":0.311},"max":{"1min":1.212,"5min":1.9299999999999999,"15min":1.9299999999999999},"last":0.317},{"interface":"front","average":{"1min":0.873,"5min":0.89500000000000002,"15min":0.92100000000000004},"min":{"1min":0.35099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":1.339,"5min":1.7669999999999999,"15min":1.7669999999999999},"last":0.40999999999999998}]},{"osd":2,"last update":"Fri Nov 24 06:13:21 2023","interfaces":[{"interface":"back","average":{"1min":0.97499999999999998,"5min":0.95599999999999996,"15min":0.998},"min":{"1min":0.496,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.605,"5min":2.5329999999999999,"15min":2.5329999999999999},"last":0.497},{"interface":"front","average":{"1min":1.079,"5min":1.0009999999999999,"15min":1.02},"min":{"1min":0.502,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.7190000000000001,"5min":1.7190000000000001,"15min":1.7190000000000001},"last":0.46300000000000002}]},{"osd":3,"last update":"Fri Nov 24 06:13:21 2023","interfaces":[{"interface":"back","average":{"1min":0.91700000000000004,"5min":0.96399999999999997,"15min":1.0129999999999999},"min":{"1min":0.46800000000000003,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.4119999999999999,"5min":1.841,"15min":1.841},"last":0.435},{"interface":"front","average":{"1min":1.1479999999999999,"5min":1.1200000000000001,"15min":1.1439999999999999},"min":{"1min":0.61899999999999999,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.643,"5min":2.8940000000000001,"15min":2.8940000000000001},"last":0.54900000000000004}]},{"osd":4,"last update":"Fri Nov 24 06:13:21 2023","interfaces":[{"interface":"back","average":{"1min":1.3149999999999999,"5min":1.2609999999999999,"15min":1.2749999999999999},"min":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.728,"5min":2.1930000000000001,"15min":2.1930000000000001},"last":1.4790000000000001},{"interface":"front","average":{"1min":1.1899999999999999,"5min":1.1819999999999999,"15min":1.194},"min":{"1min":0.86799999999999999,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":1.5640000000000001,"5min":1.784,"15min":1.784},"last":1.3560000000000001}]},{"osd":5,"last update":"Fri Nov 24 06:13:21 2023","interfaces":[{"interface":"back","average":{"1min":1.2430000000000001,"5min":1.3959999999999999,"15min":1.4850000000000001},"min":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.732,"5min":5.5449999999999999,"15min":5.5449999999999999},"last":1.5009999999999999},{"interface":"front","average":{"1min":1.361,"5min":1.444,"15min":1.524},"min":{"1min":0.90900000000000003,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":1.7270000000000001,"5min":6.0030000000000001,"15min":6.0030000000000001},"last":1.536}]},{"osd":6,"last update":"Fri Nov 24 06:13:21 2023","interfaces":[{"interface":"back","average":{"1min":1.3220000000000001,"5min":1.446,"15min":1.528},"min":{"1min":0.81100000000000005,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":1.8779999999999999,"5min":5.2809999999999997,"15min":5.2809999999999997},"last":1.395},{"interface":"front","average":{"1min":1.4079999999999999,"5min":1.423,"15min":1.4570000000000001},"min":{"1min":0.82999999999999996,"5min":0.749,"15min":0.749},"max":{"1min":1.819,"5min":4.327,"15min":4.327},"last":1.5760000000000001}]},{"osd":7,"last update":"Fri Nov 24 06:13:21 2023","interfaces":[{"interface":"back","average":{"1min":1.264,"5min":1.302,"15min":1.3400000000000001},"min":{"1min":0.85099999999999998,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":1.7849999999999999,"5min":2.3069999999999999,"15min":2.3069999999999999},"last":1.413},{"interface":"front","average":{"1min":1.415,"5min":1.492,"15min":1.5620000000000001},"min":{"1min":0.91700000000000004,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":2.0179999999999998,"5min":5.4500000000000002,"15min":5.4500000000000002},"last":1.4530000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574890,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32520,"kb_used_data":1340,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339320,"statfs":{"total":96636764160,"available":96603463680,"internally_reserved":0,"allocated":1372160,"data_stored":924696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Nov 24 06:13:25 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.84799999999999998,"15min":0.90800000000000003},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.145,"5min":2.3999999999999999,"15min":2.3999999999999999},"last":0.91300000000000003},{"interface":"front","average":{"1min":0.75800000000000001,"5min":1.024,"15min":1.133},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":1.54,"5min":4.8289999999999997,"15min":4.8289999999999997},"last":0.58799999999999997}]},{"osd":2,"last update":"Fri Nov 24 06:13:25 2023","interfaces":[{"interface":"back","average":{"1min":0.83799999999999997,"5min":0.97999999999999998,"15min":1.0349999999999999},"min":{"1min":0.42999999999999999,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.4019999999999999,"5min":2.5139999999999998,"15min":2.5139999999999998},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.79800000000000004,"5min":1.0229999999999999,"15min":1.0820000000000001},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.26,"5min":2.3599999999999999,"15min":2.3599999999999999},"last":0.51800000000000002}]},{"osd":3,"last update":"Fri Nov 24 06:13:25 2023","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":1.056,"15min":1.1020000000000001},"min":{"1min":0.48699999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.5369999999999999,"5min":2.5840000000000001,"15min":2.5840000000000001},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.91600000000000004,"5min":1.246,"15min":1.3580000000000001},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.4630000000000001,"5min":5.1230000000000002,"15min":5.1230000000000002},"last":0.97899999999999998}]},{"osd":4,"last update":"Fri Nov 24 06:13:25 2023","interfaces":[{"interface":"back","average":{"1min":1.0329999999999999,"5min":1.1799999999999999,"15min":1.248},"min":{"1min":0.70499999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.488,"5min":2.887,"15min":2.887},"last":1.173},{"interface":"front","average":{"1min":1.0720000000000001,"5min":1.2310000000000001,"15min":1.3009999999999999},"min":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.581,"5min":2.8530000000000002,"15min":2.8530000000000002},"last":1.238}]},{"osd":5,"last update":"Fri Nov 24 06:13:25 2023","interfaces":[{"interface":"back","average":{"1min":1.149,"5min":1.4219999999999999,"15min":1.524},"min":{"1min":0.72199999999999998,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.978,"5min":4.6299999999999999,"15min":4.6299999999999999},"last":1.288},{"interface":"front","average":{"1min":1.1060000000000001,"5min":1.232,"15min":1.2989999999999999},"min":{"1min":0.85399999999999998,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.7010000000000001,"5min":2.794,"15min":2.794},"last":1.1120000000000001}]},{"osd":6,"last update":"Fri Nov 24 06:13:25 2023","interfaces":[{"interface":"back","average":{"1min":1.244,"5min":1.508,"15min":1.617},"min":{"1min":0.73899999999999999,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.8620000000000001,"5min":4.798,"15min":4.798},"last":1.0940000000000001},{"interface":"front","average":{"1min":1.3100000000000001,"5min":1.5189999999999999,"15min":1.597},"min":{"1min":0.97099999999999997,"5min":0.77800000000000002,"15min":0.77800000000000002},"max":{"1min":1.841,"5min":4.7149999999999999,"15min":4.7149999999999999},"last":1.399}]},{"osd":7,"last update":"Fri Nov 24 06:13:25 2023","interfaces":[{"interface":"back","average":{"1min":1.2629999999999999,"5min":1.544,"15min":1.6579999999999999},"min":{"1min":0.85099999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.9770000000000001,"5min":4.7649999999999997,"15min":4.7649999999999997},"last":1.204},{"interface":"front","average":{"1min":1.3,"5min":1.5609999999999999,"15min":1.6499999999999999},"min":{"1min":0.89900000000000002,"5min":0.84599999999999997,"15min":0.84599999999999997},"max":{"1min":1.9159999999999999,"5min":4.6820000000000004,"15min":4.6820000000000004},"last":1.5149999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-24T06:13:42.761 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:13:42.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.978+0000 7f65eb6ad640 1 Processor -- start 2023-11-24T06:13:42.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.978+0000 7f65eb6ad640 1 -- start start 2023-11-24T06:13:42.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.980+0000 7f65eb6ad640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f65e4159040 0x7f65e40cf000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:42.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.980+0000 7f65eb6ad640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f65e415c300 0x7f65e415f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:42.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.980+0000 7f65eb6ad640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65e40c6060 con 0x7f65e40b34e0 2023-11-24T06:13:42.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.980+0000 7f65eb6ad640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65e40c62d0 con 0x7f65e415c300 2023-11-24T06:13:42.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.980+0000 7f65eb6ad640 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65e40c65b0 con 0x7f65e4159040 2023-11-24T06:13:42.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.980+0000 7f65e8c48640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f65e415c300 0x7f65e415f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:42.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.980+0000 7f65e8c48640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f65e415c300 0x7f65e415f3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47830/0 (socket says 172.21.15.92:47830) 2023-11-24T06:13:42.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.980+0000 7f65e8c48640 1 -- 172.21.15.92:0/2168190148 learned_addr learned my addr 172.21.15.92:0/2168190148 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:13:42.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.980+0000 7f65e9c4a640 1 --1- 172.21.15.92:0/2168190148 >> v1:172.21.15.92:6789/0 conn(0x7f65e40b34e0 0x7f65e40cdb10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:50340/0 (socket says 172.21.15.92:50340) 2023-11-24T06:13:42.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.981+0000 7f65e9449640 1 --2- 172.21.15.92:0/2168190148 >> v2:172.21.15.118:3300/0 conn(0x7f65e4159040 0x7f65e40cf000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:42.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.981+0000 7f65e8c48640 1 -- 172.21.15.92:0/2168190148 >> v2:172.21.15.118:3300/0 conn(0x7f65e4159040 msgr2=0x7f65e40cf000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:13:42.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.981+0000 7f65e8c48640 1 --2- 172.21.15.92:0/2168190148 >> v2:172.21.15.118:3300/0 conn(0x7f65e4159040 0x7f65e40cf000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:42.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.981+0000 7f65e8c48640 1 -- 172.21.15.92:0/2168190148 >> v1:172.21.15.92:6789/0 conn(0x7f65e40b34e0 legacy=0x7f65e40cdb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:42.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.981+0000 7f65e3fff640 1 -- 172.21.15.92:0/2168190148 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2100901057 0 0) 0x7f65e40c6060 con 0x7f65e40b34e0 2023-11-24T06:13:42.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.981+0000 7f65e8c48640 1 -- 172.21.15.92:0/2168190148 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65e415eec0 con 0x7f65e415c300 2023-11-24T06:13:42.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.982+0000 7f65e8c48640 1 --2- 172.21.15.92:0/2168190148 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f65e415c300 0x7f65e415f3e0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f65e40d1e90 tx=0x7f65d802bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60b71d85290b42e5 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:42.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.983+0000 7f65e3fff640 1 -- 172.21.15.92:0/2168190148 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65d802cb20 con 0x7f65e415c300 2023-11-24T06:13:42.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.983+0000 7f65e3fff640 1 -- 172.21.15.92:0/2168190148 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65d802cc80 con 0x7f65e415c300 2023-11-24T06:13:42.988 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.983+0000 7f65e3fff640 1 -- 172.21.15.92:0/2168190148 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65d80343f0 con 0x7f65e415c300 2023-11-24T06:13:42.988 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.983+0000 7f65eb6ad640 1 -- 172.21.15.92:0/2168190148 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f65e415c300 msgr2=0x7f65e415f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:42.988 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.983+0000 7f65eb6ad640 1 --2- 172.21.15.92:0/2168190148 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f65e415c300 0x7f65e415f3e0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f65e40d1e90 tx=0x7f65d802bf80 comp rx=0 tx=0).stop 2023-11-24T06:13:42.988 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.984+0000 7f65eb6ad640 1 -- 172.21.15.92:0/2168190148 shutdown_connections 2023-11-24T06:13:42.988 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.984+0000 7f65eb6ad640 1 --2- 172.21.15.92:0/2168190148 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f65e415c300 0x7f65e415f3e0 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:42.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.984+0000 7f65eb6ad640 1 --2- 172.21.15.92:0/2168190148 >> v2:172.21.15.118:3300/0 conn(0x7f65e4159040 0x7f65e40cf000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:42.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.984+0000 7f65eb6ad640 1 -- 172.21.15.92:0/2168190148 >> 172.21.15.92:0/2168190148 conn(0x7f65e4066690 msgr2=0x7f65e406a3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:13:42.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.984+0000 7f65eb6ad640 1 -- 172.21.15.92:0/2168190148 shutdown_connections 2023-11-24T06:13:42.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.984+0000 7f65eb6ad640 1 -- 172.21.15.92:0/2168190148 wait complete. 2023-11-24T06:13:42.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.985+0000 7f65eb6ad640 1 Processor -- start 2023-11-24T06:13:42.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.986+0000 7f65eb6ad640 1 -- start start 2023-11-24T06:13:42.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.987+0000 7f65eb6ad640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f65e40b34e0 0x7f65e4157360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:42.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.987+0000 7f65eb6ad640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f65e4159040 0x7f65e40b5030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:42.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.987+0000 7f65eb6ad640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f65e415c300 0x7f65e40c8140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:42.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.987+0000 7f65eb6ad640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65e40b5570 con 0x7f65e40b34e0 2023-11-24T06:13:42.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.987+0000 7f65eb6ad640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65e40c9510 con 0x7f65e4159040 2023-11-24T06:13:42.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.987+0000 7f65eb6ad640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65e4158280 con 0x7f65e415c300 2023-11-24T06:13:42.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.987+0000 7f65e9449640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f65e40b34e0 0x7f65e4157360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:42.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.987+0000 7f65e9449640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f65e40b34e0 0x7f65e4157360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47834/0 (socket says 172.21.15.92:47834) 2023-11-24T06:13:42.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.987+0000 7f65e9449640 1 -- 172.21.15.92:0/151181838 learned_addr learned my addr 172.21.15.92:0/151181838 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:13:42.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.987+0000 7f65e8c48640 1 --2- 172.21.15.92:0/151181838 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f65e4159040 0x7f65e40b5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:42.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.987+0000 7f65e9449640 1 -- 172.21.15.92:0/151181838 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f65e415c300 msgr2=0x7f65e40c8140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:42.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.987+0000 7f65e9449640 1 --2- 172.21.15.92:0/151181838 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f65e415c300 0x7f65e40c8140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:42.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.987+0000 7f65e9449640 1 -- 172.21.15.92:0/151181838 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f65e4159040 msgr2=0x7f65e40b5030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:42.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.987+0000 7f65e9449640 1 --2- 172.21.15.92:0/151181838 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f65e4159040 0x7f65e40b5030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:42.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.987+0000 7f65e9449640 1 -- 172.21.15.92:0/151181838 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65d8009b30 con 0x7f65e40b34e0 2023-11-24T06:13:42.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.987+0000 7f65e8c48640 1 --2- 172.21.15.92:0/151181838 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f65e4159040 0x7f65e40b5030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:13:42.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.988+0000 7f65e9449640 1 --2- 172.21.15.92:0/151181838 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f65e40b34e0 0x7f65e4157360 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f65d400d830 tx=0x7f65d400dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:42.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.988+0000 7f65e1ffb640 1 -- 172.21.15.92:0/151181838 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65d400b6c0 con 0x7f65e40b34e0 2023-11-24T06:13:42.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.988+0000 7f65e1ffb640 1 -- 172.21.15.92:0/151181838 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65d400b820 con 0x7f65e40b34e0 2023-11-24T06:13:42.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.988+0000 7f65eb6ad640 1 -- 172.21.15.92:0/151181838 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65e415bac0 con 0x7f65e40b34e0 2023-11-24T06:13:42.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.988+0000 7f65e1ffb640 1 -- 172.21.15.92:0/151181838 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65d400bbb0 con 0x7f65e40b34e0 2023-11-24T06:13:42.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.988+0000 7f65eb6ad640 1 -- 172.21.15.92:0/151181838 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65e40c91a0 con 0x7f65e40b34e0 2023-11-24T06:13:42.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.989+0000 7f65eb6ad640 1 -- 172.21.15.92:0/151181838 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65e4074a50 con 0x7f65e40b34e0 2023-11-24T06:13:42.999 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.990+0000 7f65e1ffb640 1 -- 172.21.15.92:0/151181838 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 7) v1 ==== 89333+0+0 (secure 0 0 0) 0x7f65d401e460 con 0x7f65e40b34e0 2023-11-24T06:13:42.999 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.991+0000 7f65e1ffb640 1 --2- 172.21.15.92:0/151181838 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f65d006af90 0x7f65d006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:42.999 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.991+0000 7f65e1ffb640 1 -- 172.21.15.92:0/151181838 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f65d4089b70 con 0x7f65e40b34e0 2023-11-24T06:13:42.999 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.991+0000 7f65e8c48640 1 --2- 172.21.15.92:0/151181838 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f65d006af90 0x7f65d006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:43.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.992+0000 7f65e8c48640 1 --2- 172.21.15.92:0/151181838 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f65d006af90 0x7f65d006d450 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f65d8035000 tx=0x7f65d8030000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:43.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:42.996+0000 7f65e1ffb640 1 -- 172.21.15.92:0/151181838 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f65d4054ea0 con 0x7f65e40b34e0 2023-11-24T06:13:43.180 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:43.178+0000 7f65eb6ad640 1 -- 172.21.15.92:0/151181838 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f65e40675d0 con 0x7f65e40b34e0 2023-11-24T06:13:43.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:43.180+0000 7f65e1ffb640 1 -- 172.21.15.92:0/151181838 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v66) v1 ==== 74+0+12939 (secure 0 0 0) 0x7f65d40584c0 con 0x7f65e40b34e0 2023-11-24T06:13:43.182 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-24T06:13:43.182 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":66,"fsid":"2d6dee5f-8aa3-470f-931a-50ade963b1cb","created":"2023-11-24T06:10:04.707204+0000","modified":"2023-11-24T06:13:39.214223+0000","last_up_change":"2023-11-24T06:10:17.598352+0000","last_in_change":"2023-11-24T06:10:09.040793+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:10:17.733098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ff37a9fb-135f-471f-b4b0-f7b5772006c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6803","nonce":20560714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6805","nonce":20560714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6809","nonce":20560714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":20560714},{"type":"v1","addr":"172.21.15.92:6807","nonce":20560714}]},"public_addr":"172.21.15.92:6803/20560714","cluster_addr":"172.21.15.92:6805/20560714","heartbeat_back_addr":"172.21.15.92:6809/20560714","heartbeat_front_addr":"172.21.15.92:6807/20560714","state":["exists","up"]},{"osd":1,"uuid":"03cdd353-dcf1-4f38-94fe-a795137c06b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6819","nonce":2451239906}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6821","nonce":2451239906}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6825","nonce":2451239906}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":2451239906},{"type":"v1","addr":"172.21.15.92:6823","nonce":2451239906}]},"public_addr":"172.21.15.92:6819/2451239906","cluster_addr":"172.21.15.92:6821/2451239906","heartbeat_back_addr":"172.21.15.92:6825/2451239906","heartbeat_front_addr":"172.21.15.92:6823/2451239906","state":["exists","up"]},{"osd":2,"uuid":"a4a20d80-5d57-484b-becd-9aa8070dd3aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6811","nonce":1124979277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6813","nonce":1124979277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6817","nonce":1124979277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":1124979277},{"type":"v1","addr":"172.21.15.92:6815","nonce":1124979277}]},"public_addr":"172.21.15.92:6811/1124979277","cluster_addr":"172.21.15.92:6813/1124979277","heartbeat_back_addr":"172.21.15.92:6817/1124979277","heartbeat_front_addr":"172.21.15.92:6815/1124979277","state":["exists","up"]},{"osd":3,"uuid":"4648d7ce-9b29-4583-9425-4ba8fd2b1903","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6827","nonce":1734115782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6829","nonce":1734115782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6833","nonce":1734115782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":1734115782},{"type":"v1","addr":"172.21.15.92:6831","nonce":1734115782}]},"public_addr":"172.21.15.92:6827/1734115782","cluster_addr":"172.21.15.92:6829/1734115782","heartbeat_back_addr":"172.21.15.92:6833/1734115782","heartbeat_front_addr":"172.21.15.92:6831/1734115782","state":["exists","up"]},{"osd":4,"uuid":"3bc9d7eb-c6fa-4fcf-a3ec-ade604e05aac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6819","nonce":3425937229}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6822","nonce":3425937229}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6830","nonce":3425937229}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":3425937229},{"type":"v1","addr":"172.21.15.118:6826","nonce":3425937229}]},"public_addr":"172.21.15.118:6819/3425937229","cluster_addr":"172.21.15.118:6822/3425937229","heartbeat_back_addr":"172.21.15.118:6830/3425937229","heartbeat_front_addr":"172.21.15.118:6826/3425937229","state":["exists","up"]},{"osd":5,"uuid":"dce9ba6d-59ed-46d1-84fc-a84b13de10c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6801","nonce":827455984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6803","nonce":827455984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6807","nonce":827455984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":827455984},{"type":"v1","addr":"172.21.15.118:6805","nonce":827455984}]},"public_addr":"172.21.15.118:6801/827455984","cluster_addr":"172.21.15.118:6803/827455984","heartbeat_back_addr":"172.21.15.118:6807/827455984","heartbeat_front_addr":"172.21.15.118:6805/827455984","state":["exists","up"]},{"osd":6,"uuid":"82aec504-ae58-4709-8d30-46f2b12081b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6809","nonce":2946986141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6811","nonce":2946986141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6815","nonce":2946986141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":2946986141},{"type":"v1","addr":"172.21.15.118:6813","nonce":2946986141}]},"public_addr":"172.21.15.118:6809/2946986141","cluster_addr":"172.21.15.118:6811/2946986141","heartbeat_back_addr":"172.21.15.118:6815/2946986141","heartbeat_front_addr":"172.21.15.118:6813/2946986141","state":["exists","up"]},{"osd":7,"uuid":"4ed4c624-c88f-4d35-bdb1-bf209b5dffb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6817","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6818","nonce":48122059}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6821","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6823","nonce":48122059}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6829","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6831","nonce":48122059}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6825","nonce":48122059},{"type":"v1","addr":"172.21.15.118:6827","nonce":48122059}]},"public_addr":"172.21.15.118:6818/48122059","cluster_addr":"172.21.15.118:6823/48122059","heartbeat_back_addr":"172.21.15.118:6831/48122059","heartbeat_front_addr":"172.21.15.118:6827/48122059","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.536664+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.368324+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.271201+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.342971+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.621832+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.387110+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.632946+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:10:14.416032+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.92:6837/3913457833":"2023-11-25T06:13:36.196499+0000","172.21.15.92:6835/3913457833":"2023-11-25T06:13:36.196499+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:13:43.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:43.182+0000 7f65eb6ad640 1 -- 172.21.15.92:0/151181838 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f65d006af90 msgr2=0x7f65d006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:43.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:43.182+0000 7f65eb6ad640 1 --2- 172.21.15.92:0/151181838 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f65d006af90 0x7f65d006d450 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f65d8035000 tx=0x7f65d8030000 comp rx=0 tx=0).stop 2023-11-24T06:13:43.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:43.182+0000 7f65eb6ad640 1 -- 172.21.15.92:0/151181838 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f65e40b34e0 msgr2=0x7f65e4157360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:43.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:43.182+0000 7f65eb6ad640 1 --2- 172.21.15.92:0/151181838 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f65e40b34e0 0x7f65e4157360 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f65d400d830 tx=0x7f65d400dcf0 comp rx=0 tx=0).stop 2023-11-24T06:13:43.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:43.182+0000 7f65eb6ad640 1 -- 172.21.15.92:0/151181838 shutdown_connections 2023-11-24T06:13:43.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:43.182+0000 7f65eb6ad640 1 --2- 172.21.15.92:0/151181838 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f65d006af90 0x7f65d006d450 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:43.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:43.182+0000 7f65eb6ad640 1 --2- 172.21.15.92:0/151181838 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f65e415c300 0x7f65e40c8140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:43.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:43.182+0000 7f65eb6ad640 1 --2- 172.21.15.92:0/151181838 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f65e4159040 0x7f65e40b5030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:43.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:43.182+0000 7f65eb6ad640 1 --2- 172.21.15.92:0/151181838 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f65e40b34e0 0x7f65e4157360 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:43.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:43.182+0000 7f65eb6ad640 1 -- 172.21.15.92:0/151181838 >> 172.21.15.92:0/151181838 conn(0x7f65e4066690 msgr2=0x7f65e406e750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:13:43.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:43.182+0000 7f65eb6ad640 1 -- 172.21.15.92:0/151181838 shutdown_connections 2023-11-24T06:13:43.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:43.182+0000 7f65eb6ad640 1 -- 172.21.15.92:0/151181838 wait complete. 2023-11-24T06:13:44.202 INFO:tasks.ceph:Scrubbing osd.0 2023-11-24T06:13:44.203 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2023-11-24T06:13:44.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:44.430+0000 7f946dd43640 1 Processor -- start 2023-11-24T06:13:44.433 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:44.430+0000 7f946dd43640 1 -- start start 2023-11-24T06:13:44.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:44.432+0000 7f946dd43640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f94680cf9a0 0x7f946815e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:44.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:44.432+0000 7f946dd43640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9468070c40 con 0x7f94680b3ae0 2023-11-24T06:13:44.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:44.432+0000 7f946dd43640 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f946806c470 con 0x7f94680ae5a0 2023-11-24T06:13:44.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:44.432+0000 7f946dd43640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9468071450 con 0x7f94680cf9a0 2023-11-24T06:13:44.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:44.432+0000 7f9467fff640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f94680cf9a0 0x7f946815e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:44.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:44.432+0000 7f9467fff640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f94680cf9a0 0x7f946815e430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47848/0 (socket says 172.21.15.92:47848) 2023-11-24T06:13:44.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:44.432+0000 7f9467fff640 1 -- 172.21.15.92:0/3228261588 learned_addr learned my addr 172.21.15.92:0/3228261588 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:13:44.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:44.433+0000 7f9467fff640 1 -- 172.21.15.92:0/3228261588 >> v1:172.21.15.118:6789/0 conn(0x7f94680ae5a0 legacy=0x7f946806cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:13:44.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:44.433+0000 7f9467fff640 1 -- 172.21.15.92:0/3228261588 >> v1:172.21.15.92:6789/0 conn(0x7f94680b3ae0 legacy=0x7f94680cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:44.437 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:44.433+0000 7f9467fff640 1 -- 172.21.15.92:0/3228261588 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f946806c750 con 0x7f94680cf9a0 2023-11-24T06:13:44.437 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:44.434+0000 7f9467fff640 1 --2- 172.21.15.92:0/3228261588 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f94680cf9a0 0x7f946815e430 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f9460009e50 tx=0x7f946002c010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f54b48a46e16e1a server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:44.437 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:44.434+0000 7f94667fc640 1 -- 172.21.15.92:0/3228261588 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f946002cbb0 con 0x7f94680cf9a0 2023-11-24T06:13:44.437 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:44.434+0000 7f94667fc640 1 -- 172.21.15.92:0/3228261588 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f946002cd10 con 0x7f94680cf9a0 2023-11-24T06:13:44.437 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:44.434+0000 7f94667fc640 1 -- 172.21.15.92:0/3228261588 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9460034610 con 0x7f94680cf9a0 2023-11-24T06:13:44.438 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:44.435+0000 7f946dd43640 1 -- 172.21.15.92:0/3228261588 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f94680cf9a0 msgr2=0x7f946815e430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:44.438 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:44.435+0000 7f946dd43640 1 --2- 172.21.15.92:0/3228261588 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f94680cf9a0 0x7f946815e430 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f9460009e50 tx=0x7f946002c010 comp rx=0 tx=0).stop 2023-11-24T06:13:44.438 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:44.435+0000 7f946dd43640 1 -- 172.21.15.92:0/3228261588 shutdown_connections 2023-11-24T06:13:44.438 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:44.435+0000 7f946dd43640 1 --2- 172.21.15.92:0/3228261588 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f94680cf9a0 0x7f946815e430 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:44.438 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:44.435+0000 7f946dd43640 1 -- 172.21.15.92:0/3228261588 >> 172.21.15.92:0/3228261588 conn(0x7f946806a800 msgr2=0x7f946806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:13:44.439 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:44.435+0000 7f946dd43640 1 -- 172.21.15.92:0/3228261588 shutdown_connections 2023-11-24T06:13:44.439 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:44.435+0000 7f946dd43640 1 -- 172.21.15.92:0/3228261588 wait complete. 2023-11-24T06:13:44.439 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:44.437+0000 7f946dd43640 1 Processor -- start 2023-11-24T06:13:44.439 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:44.437+0000 7f946dd43640 1 -- start start 2023-11-24T06:13:44.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:44.438+0000 7f946dd43640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f94680ae5a0 0x7f94681f3e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:44.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:44.438+0000 7f946dd43640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f94680b3ae0 0x7f9468153300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:44.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:44.438+0000 7f946dd43640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f94680cf9a0 0x7f946815fe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:44.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:44.438+0000 7f946dd43640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94680cb980 con 0x7f94680ae5a0 2023-11-24T06:13:44.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:44.438+0000 7f946dd43640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94681f4380 con 0x7f94680b3ae0 2023-11-24T06:13:44.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:44.438+0000 7f946dd43640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9468153840 con 0x7f94680cf9a0 2023-11-24T06:13:44.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:44.438+0000 7f94677fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f94680ae5a0 0x7f94681f3e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:44.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:44.438+0000 7f94677fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f94680ae5a0 0x7f94681f3e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47854/0 (socket says 172.21.15.92:47854) 2023-11-24T06:13:44.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:44.438+0000 7f94677fe640 1 -- 172.21.15.92:0/40654163 learned_addr learned my addr 172.21.15.92:0/40654163 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:13:44.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:44.438+0000 7f9466ffd640 1 --2- 172.21.15.92:0/40654163 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f94680b3ae0 0x7f9468153300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:44.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:44.439+0000 7f94677fe640 1 -- 172.21.15.92:0/40654163 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f94680cf9a0 msgr2=0x7f946815fe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:44.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:44.439+0000 7f94677fe640 1 --2- 172.21.15.92:0/40654163 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f94680cf9a0 0x7f946815fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:44.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:44.439+0000 7f94677fe640 1 -- 172.21.15.92:0/40654163 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f94680b3ae0 msgr2=0x7f9468153300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:44.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:44.439+0000 7f94677fe640 1 --2- 172.21.15.92:0/40654163 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f94680b3ae0 0x7f9468153300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:44.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:44.439+0000 7f94677fe640 1 -- 172.21.15.92:0/40654163 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9460009b30 con 0x7f94680ae5a0 2023-11-24T06:13:44.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:44.439+0000 7f9466ffd640 1 --2- 172.21.15.92:0/40654163 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f94680b3ae0 0x7f9468153300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:13:44.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:44.439+0000 7f94677fe640 1 --2- 172.21.15.92:0/40654163 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f94680ae5a0 0x7f94681f3e40 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f945800c430 tx=0x7f945800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:44.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:44.439+0000 7f944ffff640 1 -- 172.21.15.92:0/40654163 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9458011790 con 0x7f94680ae5a0 2023-11-24T06:13:44.444 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:44.439+0000 7f944ffff640 1 -- 172.21.15.92:0/40654163 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94580118f0 con 0x7f94680ae5a0 2023-11-24T06:13:44.444 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:44.439+0000 7f944ffff640 1 -- 172.21.15.92:0/40654163 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9458011ce0 con 0x7f94680ae5a0 2023-11-24T06:13:44.444 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:44.439+0000 7f946dd43640 1 -- 172.21.15.92:0/40654163 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94681f6e90 con 0x7f94680ae5a0 2023-11-24T06:13:44.444 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:44.439+0000 7f946dd43640 1 -- 172.21.15.92:0/40654163 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94681f7450 con 0x7f94680ae5a0 2023-11-24T06:13:44.445 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:44.440+0000 7f946dd43640 1 -- 172.21.15.92:0/40654163 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f9468071ec0 con 0x7f94680ae5a0 2023-11-24T06:13:44.445 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:44.441+0000 7f944ffff640 1 -- 172.21.15.92:0/40654163 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 7) v1 ==== 89333+0+0 (secure 0 0 0) 0x7f945801c460 con 0x7f94680ae5a0 2023-11-24T06:13:44.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:44.442+0000 7f944ffff640 1 --2- 172.21.15.92:0/40654163 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f945406af60 0x7f945406d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:44.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:44.442+0000 7f944ffff640 1 -- 172.21.15.92:0/40654163 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f94580893f0 con 0x7f94680ae5a0 2023-11-24T06:13:44.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:44.442+0000 7f9466ffd640 1 --2- 172.21.15.92:0/40654163 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f945406af60 0x7f945406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:44.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:44.442+0000 7f944ffff640 1 --2- 172.21.15.92:0/40654163 >> [v2:172.21.15.92:6802/20560714,v1:172.21.15.92:6803/20560714] conn(0x7f9454070dc0 0x7f94540731e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:44.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:44.442+0000 7f944ffff640 1 -- 172.21.15.92:0/40654163 --> [v2:172.21.15.92:6802/20560714,v1:172.21.15.92:6803/20560714] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9454073890 con 0x7f9454070dc0 2023-11-24T06:13:44.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:44.442+0000 7f944ffff640 1 -- 172.21.15.92:0/40654163 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_get_version_reply(handle=1 version=66) v2 ==== 24+0+0 (secure 0 0 0) 0x7f945808b9d0 con 0x7f94680ae5a0 2023-11-24T06:13:44.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:44.443+0000 7f9467fff640 1 --2- 172.21.15.92:0/40654163 >> [v2:172.21.15.92:6802/20560714,v1:172.21.15.92:6803/20560714] conn(0x7f9454070dc0 0x7f94540731e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:44.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:44.443+0000 7f9466ffd640 1 --2- 172.21.15.92:0/40654163 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f945406af60 0x7f945406d420 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f9468159c30 tx=0x7f945c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:44.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:44.443+0000 7f9467fff640 1 --2- 172.21.15.92:0/40654163 >> [v2:172.21.15.92:6802/20560714,v1:172.21.15.92:6803/20560714] conn(0x7f9454070dc0 0x7f94540731e0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:44.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:44.444+0000 7f944ffff640 1 -- 172.21.15.92:0/40654163 <== osd.0 v2:172.21.15.92:6802/20560714 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f9454073890 con 0x7f9454070dc0 2023-11-24T06:13:44.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:44.459+0000 7f946dd43640 1 -- 172.21.15.92:0/40654163 --> [v2:172.21.15.92:6802/20560714,v1:172.21.15.92:6803/20560714] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f9468071170 con 0x7f9454070dc0 2023-11-24T06:13:44.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:44.459+0000 7f944ffff640 1 -- 172.21.15.92:0/40654163 <== osd.0 v2:172.21.15.92:6802/20560714 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f946002ca30 con 0x7f9454070dc0 2023-11-24T06:13:44.461 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-24T06:13:44.462 INFO:teuthology.orchestra.run.smithi092.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2023-11-24T06:13:44.462 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-24T06:13:44.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:44.460+0000 7f946dd43640 1 -- 172.21.15.92:0/40654163 >> [v2:172.21.15.92:6802/20560714,v1:172.21.15.92:6803/20560714] conn(0x7f9454070dc0 msgr2=0x7f94540731e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:44.466 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:44.460+0000 7f946dd43640 1 --2- 172.21.15.92:0/40654163 >> [v2:172.21.15.92:6802/20560714,v1:172.21.15.92:6803/20560714] conn(0x7f9454070dc0 0x7f94540731e0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:44.466 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:44.460+0000 7f946dd43640 1 -- 172.21.15.92:0/40654163 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f945406af60 msgr2=0x7f945406d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:44.466 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:44.460+0000 7f946dd43640 1 --2- 172.21.15.92:0/40654163 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f945406af60 0x7f945406d420 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f9468159c30 tx=0x7f945c009040 comp rx=0 tx=0).stop 2023-11-24T06:13:44.466 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:44.460+0000 7f946dd43640 1 -- 172.21.15.92:0/40654163 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f94680ae5a0 msgr2=0x7f94681f3e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:44.466 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:44.460+0000 7f946dd43640 1 --2- 172.21.15.92:0/40654163 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f94680ae5a0 0x7f94681f3e40 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f945800c430 tx=0x7f945800c8f0 comp rx=0 tx=0).stop 2023-11-24T06:13:44.466 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:44.460+0000 7f9467fff640 1 -- 172.21.15.92:0/40654163 reap_dead start 2023-11-24T06:13:44.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:44.460+0000 7f946dd43640 1 -- 172.21.15.92:0/40654163 shutdown_connections 2023-11-24T06:13:44.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:44.460+0000 7f946dd43640 1 -- 172.21.15.92:0/40654163 >> 172.21.15.92:0/40654163 conn(0x7f946806a800 msgr2=0x7f94680cf260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:13:44.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:44.460+0000 7f946dd43640 1 -- 172.21.15.92:0/40654163 shutdown_connections 2023-11-24T06:13:44.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:44.460+0000 7f946dd43640 1 -- 172.21.15.92:0/40654163 wait complete. 2023-11-24T06:13:44.478 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2023-11-24T06:13:44.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:44.685+0000 7f3ee8543640 1 Processor -- start 2023-11-24T06:13:44.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:44.685+0000 7f3ee8543640 1 -- start start 2023-11-24T06:13:44.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:44.687+0000 7f3ee8543640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3ee013df80 0x7f3ee015bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:44.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:44.687+0000 7f3ee8543640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f3ee00cf9a0 0x7f3ee00cd840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:44.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:44.687+0000 7f3ee8543640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ee006c470 con 0x7f3ee013e360 2023-11-24T06:13:44.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:44.687+0000 7f3ee8543640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ee006c750 con 0x7f3ee00cf9a0 2023-11-24T06:13:44.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:44.687+0000 7f3ee8543640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ee006ca30 con 0x7f3ee013df80 2023-11-24T06:13:44.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:44.687+0000 7f3ee5ade640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f3ee013e360 0x7f3ee00cf290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:50348/0 (socket says 172.21.15.92:50348) 2023-11-24T06:13:44.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:44.687+0000 7f3ee5ade640 1 -- 172.21.15.92:0/4003690615 learned_addr learned my addr 172.21.15.92:0/4003690615 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-24T06:13:44.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:44.688+0000 7f3ee6ae0640 1 --2- 172.21.15.92:0/4003690615 >> v2:172.21.15.92:3300/0 conn(0x7f3ee00cf9a0 0x7f3ee00cd840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:44.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:44.688+0000 7f3ee62df640 1 --2- 172.21.15.92:0/4003690615 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3ee013df80 0x7f3ee015bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:44.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:44.688+0000 7f3ee52dd640 1 -- 172.21.15.92:0/4003690615 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2572342975 0 0) 0x7f3ee006c470 con 0x7f3ee013e360 2023-11-24T06:13:44.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:44.688+0000 7f3ee52dd640 1 -- 172.21.15.92:0/4003690615 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3ecc003520 con 0x7f3ee013e360 2023-11-24T06:13:44.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:44.688+0000 7f3ee6ae0640 1 -- 172.21.15.92:0/4003690615 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3ee013df80 msgr2=0x7f3ee015bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:13:44.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:44.688+0000 7f3ee6ae0640 1 --2- 172.21.15.92:0/4003690615 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3ee013df80 0x7f3ee015bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:44.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:44.688+0000 7f3ee6ae0640 1 -- 172.21.15.92:0/4003690615 >> v1:172.21.15.92:6789/0 conn(0x7f3ee013e360 legacy=0x7f3ee00cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:44.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:44.688+0000 7f3ee6ae0640 1 -- 172.21.15.92:0/4003690615 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ee006cd10 con 0x7f3ee00cf9a0 2023-11-24T06:13:44.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:44.688+0000 7f3ee62df640 1 --2- 172.21.15.92:0/4003690615 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3ee013df80 0x7f3ee015bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:13:44.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:44.689+0000 7f3ee6ae0640 1 --2- 172.21.15.92:0/4003690615 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3ee00cf9a0 0x7f3ee00cd840 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f3ed8009d90 tx=0x7f3ed802beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c27f78c373be9a7 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:44.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:44.690+0000 7f3ee52dd640 1 -- 172.21.15.92:0/4003690615 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ed802cc60 con 0x7f3ee00cf9a0 2023-11-24T06:13:44.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:44.690+0000 7f3ee52dd640 1 -- 172.21.15.92:0/4003690615 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ed802cdc0 con 0x7f3ee00cf9a0 2023-11-24T06:13:44.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:44.690+0000 7f3ee52dd640 1 -- 172.21.15.92:0/4003690615 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ed8034660 con 0x7f3ee00cf9a0 2023-11-24T06:13:44.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:44.690+0000 7f3ee8543640 1 -- 172.21.15.92:0/4003690615 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3ee00cf9a0 msgr2=0x7f3ee00cd840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:44.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:44.690+0000 7f3ee8543640 1 --2- 172.21.15.92:0/4003690615 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3ee00cf9a0 0x7f3ee00cd840 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f3ed8009d90 tx=0x7f3ed802beb0 comp rx=0 tx=0).stop 2023-11-24T06:13:44.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:44.690+0000 7f3ee8543640 1 -- 172.21.15.92:0/4003690615 shutdown_connections 2023-11-24T06:13:44.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:44.690+0000 7f3ee8543640 1 --2- 172.21.15.92:0/4003690615 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3ee00cf9a0 0x7f3ee00cd840 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:44.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:44.690+0000 7f3ee8543640 1 --2- 172.21.15.92:0/4003690615 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3ee013df80 0x7f3ee015bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:44.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:44.690+0000 7f3ee8543640 1 -- 172.21.15.92:0/4003690615 >> 172.21.15.92:0/4003690615 conn(0x7f3ee006a800 msgr2=0x7f3ee006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:13:44.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:44.690+0000 7f3ee8543640 1 -- 172.21.15.92:0/4003690615 shutdown_connections 2023-11-24T06:13:44.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:44.690+0000 7f3ee8543640 1 -- 172.21.15.92:0/4003690615 wait complete. 2023-11-24T06:13:44.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:44.692+0000 7f3ee8543640 1 Processor -- start 2023-11-24T06:13:44.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:44.692+0000 7f3ee8543640 1 -- start start 2023-11-24T06:13:44.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:44.693+0000 7f3ee8543640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3ee00cf9a0 0x7f3ee01595a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:44.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:44.693+0000 7f3ee8543640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3ee013df80 0x7f3ee0157660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:44.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:44.693+0000 7f3ee8543640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3ee013e360 0x7f3ee00b3980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:44.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:44.693+0000 7f3ee8543640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ee0159ae0 con 0x7f3ee00cf9a0 2023-11-24T06:13:44.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:44.693+0000 7f3ee8543640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ee00b1d40 con 0x7f3ee013df80 2023-11-24T06:13:44.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:44.693+0000 7f3ee8543640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ee0157ba0 con 0x7f3ee013e360 2023-11-24T06:13:44.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:44.693+0000 7f3ee62df640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3ee00cf9a0 0x7f3ee01595a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:44.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:44.694+0000 7f3ee62df640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3ee00cf9a0 0x7f3ee01595a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47866/0 (socket says 172.21.15.92:47866) 2023-11-24T06:13:44.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:44.694+0000 7f3ee62df640 1 -- 172.21.15.92:0/3320308625 learned_addr learned my addr 172.21.15.92:0/3320308625 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:13:44.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:44.694+0000 7f3ee5ade640 1 --2- 172.21.15.92:0/3320308625 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3ee013df80 0x7f3ee0157660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:44.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:44.694+0000 7f3ee62df640 1 -- 172.21.15.92:0/3320308625 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3ee013e360 msgr2=0x7f3ee00b3980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:44.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:44.694+0000 7f3ee62df640 1 --2- 172.21.15.92:0/3320308625 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3ee013e360 0x7f3ee00b3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:44.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:44.694+0000 7f3ee62df640 1 -- 172.21.15.92:0/3320308625 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3ee013df80 msgr2=0x7f3ee0157660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:44.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:44.694+0000 7f3ee62df640 1 --2- 172.21.15.92:0/3320308625 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3ee013df80 0x7f3ee0157660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:44.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:44.694+0000 7f3ee62df640 1 -- 172.21.15.92:0/3320308625 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ed8009a70 con 0x7f3ee00cf9a0 2023-11-24T06:13:44.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:44.694+0000 7f3ee62df640 1 --2- 172.21.15.92:0/3320308625 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3ee00cf9a0 0x7f3ee01595a0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f3edc00e730 tx=0x7f3edc00ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:44.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:44.694+0000 7f3ed6ffd640 1 -- 172.21.15.92:0/3320308625 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3edc004230 con 0x7f3ee00cf9a0 2023-11-24T06:13:44.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:44.695+0000 7f3ed6ffd640 1 -- 172.21.15.92:0/3320308625 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3edc004390 con 0x7f3ee00cf9a0 2023-11-24T06:13:44.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:44.695+0000 7f3ed6ffd640 1 -- 172.21.15.92:0/3320308625 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3edc00c8f0 con 0x7f3ee00cf9a0 2023-11-24T06:13:44.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:44.695+0000 7f3ee8543640 1 -- 172.21.15.92:0/3320308625 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ee015a3a0 con 0x7f3ee00cf9a0 2023-11-24T06:13:44.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:44.695+0000 7f3ee8543640 1 -- 172.21.15.92:0/3320308625 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ee00b1fe0 con 0x7f3ee00cf9a0 2023-11-24T06:13:44.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:44.696+0000 7f3ee8543640 1 -- 172.21.15.92:0/3320308625 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ee00675d0 con 0x7f3ee00cf9a0 2023-11-24T06:13:44.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:44.697+0000 7f3ed6ffd640 1 -- 172.21.15.92:0/3320308625 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 7) v1 ==== 89333+0+0 (secure 0 0 0) 0x7f3edc00ca50 con 0x7f3ee00cf9a0 2023-11-24T06:13:44.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:44.698+0000 7f3ed6ffd640 1 --2- 172.21.15.92:0/3320308625 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f3ec406af60 0x7f3ec406d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:44.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:44.698+0000 7f3ed6ffd640 1 -- 172.21.15.92:0/3320308625 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f3edc089260 con 0x7f3ee00cf9a0 2023-11-24T06:13:44.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:44.698+0000 7f3ee5ade640 1 --2- 172.21.15.92:0/3320308625 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f3ec406af60 0x7f3ec406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:44.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:44.699+0000 7f3ee5ade640 1 --2- 172.21.15.92:0/3320308625 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f3ec406af60 0x7f3ec406d420 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f3ee00cf380 tx=0x7f3ed0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:44.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:44.702+0000 7f3ed6ffd640 1 -- 172.21.15.92:0/3320308625 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3edc054350 con 0x7f3ee00cf9a0 2023-11-24T06:13:44.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:44.891+0000 7f3ee8543640 1 -- 172.21.15.92:0/3320308625 --> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) v1 -- 0x7f3ee015f9c0 con 0x7f3ec406af60 2023-11-24T06:13:44.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:44.892+0000 7f3ed6ffd640 1 -- 172.21.15.92:0/3320308625 <== mgr.4101 v2:172.21.15.92:6800/784027401 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f3ed0001c40 con 0x7f3ec406af60 2023-11-24T06:13:44.894 INFO:teuthology.orchestra.run.smithi092.stderr:instructed osd(s) 0 to deep-scrub 2023-11-24T06:13:44.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:44.894+0000 7f3ee8543640 1 -- 172.21.15.92:0/3320308625 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f3ec406af60 msgr2=0x7f3ec406d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:44.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:44.894+0000 7f3ee8543640 1 --2- 172.21.15.92:0/3320308625 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f3ec406af60 0x7f3ec406d420 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f3ee00cf380 tx=0x7f3ed0009040 comp rx=0 tx=0).stop 2023-11-24T06:13:44.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:44.894+0000 7f3ee8543640 1 -- 172.21.15.92:0/3320308625 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3ee00cf9a0 msgr2=0x7f3ee01595a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:44.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:44.894+0000 7f3ee8543640 1 --2- 172.21.15.92:0/3320308625 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3ee00cf9a0 0x7f3ee01595a0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f3edc00e730 tx=0x7f3edc00ebf0 comp rx=0 tx=0).stop 2023-11-24T06:13:44.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:44.895+0000 7f3ee8543640 1 -- 172.21.15.92:0/3320308625 shutdown_connections 2023-11-24T06:13:44.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:44.895+0000 7f3ee8543640 1 --2- 172.21.15.92:0/3320308625 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f3ec406af60 0x7f3ec406d420 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:44.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:44.895+0000 7f3ee8543640 1 --2- 172.21.15.92:0/3320308625 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f3ee013e360 0x7f3ee00b3980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:44.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:44.895+0000 7f3ee8543640 1 --2- 172.21.15.92:0/3320308625 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3ee013df80 0x7f3ee0157660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:44.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:44.895+0000 7f3ee8543640 1 --2- 172.21.15.92:0/3320308625 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3ee00cf9a0 0x7f3ee01595a0 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:44.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:44.895+0000 7f3ee8543640 1 -- 172.21.15.92:0/3320308625 >> 172.21.15.92:0/3320308625 conn(0x7f3ee006a800 msgr2=0x7f3ee006abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:13:44.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:44.895+0000 7f3ee8543640 1 -- 172.21.15.92:0/3320308625 shutdown_connections 2023-11-24T06:13:44.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:44.895+0000 7f3ee8543640 1 -- 172.21.15.92:0/3320308625 wait complete. 2023-11-24T06:13:44.914 INFO:tasks.ceph:Scrubbing osd.1 2023-11-24T06:13:44.914 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2023-11-24T06:13:45.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.127+0000 7fe7e1fdc640 1 Processor -- start 2023-11-24T06:13:45.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.127+0000 7fe7e1fdc640 1 -- start start 2023-11-24T06:13:45.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.128+0000 7fe7e1fdc640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fe7dc150760 0x7fe7dc072230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:45.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.129+0000 7fe7e1fdc640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fe7dc0b32e0 0x7fe7dc15d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:45.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.129+0000 7fe7e1fdc640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7dc15dc80 con 0x7fe7dc15d300 2023-11-24T06:13:45.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.129+0000 7fe7e1fdc640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7dc0b3710 con 0x7fe7dc0b32e0 2023-11-24T06:13:45.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.129+0000 7fe7e1fdc640 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7dc071190 con 0x7fe7dc150760 2023-11-24T06:13:45.132 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.129+0000 7fe7daffd640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fe7dc15d300 0x7fe7dc0b2bd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:50354/0 (socket says 172.21.15.92:50354) 2023-11-24T06:13:45.132 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.129+0000 7fe7daffd640 1 -- 172.21.15.92:0/3090343421 learned_addr learned my addr 172.21.15.92:0/3090343421 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-24T06:13:45.132 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.129+0000 7fe7dbfff640 1 --2- 172.21.15.92:0/3090343421 >> v2:172.21.15.92:3300/0 conn(0x7fe7dc0b32e0 0x7fe7dc15d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:45.132 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.129+0000 7fe7db7fe640 1 --2- 172.21.15.92:0/3090343421 >> v2:172.21.15.118:3300/0 conn(0x7fe7dc150760 0x7fe7dc072230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:45.133 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.129+0000 7fe7da7fc640 1 -- 172.21.15.92:0/3090343421 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2070061523 0 0) 0x7fe7dc15dc80 con 0x7fe7dc15d300 2023-11-24T06:13:45.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.129+0000 7fe7da7fc640 1 -- 172.21.15.92:0/3090343421 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe7c8003520 con 0x7fe7dc15d300 2023-11-24T06:13:45.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.129+0000 7fe7dbfff640 1 -- 172.21.15.92:0/3090343421 >> v2:172.21.15.118:3300/0 conn(0x7fe7dc150760 msgr2=0x7fe7dc072230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:13:45.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.129+0000 7fe7dbfff640 1 --2- 172.21.15.92:0/3090343421 >> v2:172.21.15.118:3300/0 conn(0x7fe7dc150760 0x7fe7dc072230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:45.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.130+0000 7fe7dbfff640 1 -- 172.21.15.92:0/3090343421 >> v1:172.21.15.92:6789/0 conn(0x7fe7dc15d300 legacy=0x7fe7dc0b2bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:45.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.130+0000 7fe7dbfff640 1 -- 172.21.15.92:0/3090343421 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7dc071470 con 0x7fe7dc0b32e0 2023-11-24T06:13:45.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.131+0000 7fe7dbfff640 1 --2- 172.21.15.92:0/3090343421 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe7dc0b32e0 0x7fe7dc15d740 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7fe7dc06e420 tx=0x7fe7d402cfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48ab9773d01b3306 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:45.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.131+0000 7fe7da7fc640 1 -- 172.21.15.92:0/3090343421 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7d402db70 con 0x7fe7dc0b32e0 2023-11-24T06:13:45.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.131+0000 7fe7da7fc640 1 -- 172.21.15.92:0/3090343421 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7d402dcd0 con 0x7fe7dc0b32e0 2023-11-24T06:13:45.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.131+0000 7fe7da7fc640 1 -- 172.21.15.92:0/3090343421 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7d4008660 con 0x7fe7dc0b32e0 2023-11-24T06:13:45.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.131+0000 7fe7e1fdc640 1 -- 172.21.15.92:0/3090343421 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe7dc0b32e0 msgr2=0x7fe7dc15d740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:45.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.131+0000 7fe7e1fdc640 1 --2- 172.21.15.92:0/3090343421 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe7dc0b32e0 0x7fe7dc15d740 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7fe7dc06e420 tx=0x7fe7d402cfd0 comp rx=0 tx=0).stop 2023-11-24T06:13:45.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.132+0000 7fe7e1fdc640 1 -- 172.21.15.92:0/3090343421 shutdown_connections 2023-11-24T06:13:45.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.132+0000 7fe7e1fdc640 1 --2- 172.21.15.92:0/3090343421 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe7dc0b32e0 0x7fe7dc15d740 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:45.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.132+0000 7fe7e1fdc640 1 --2- 172.21.15.92:0/3090343421 >> v2:172.21.15.118:3300/0 conn(0x7fe7dc150760 0x7fe7dc072230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:45.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.132+0000 7fe7e1fdc640 1 -- 172.21.15.92:0/3090343421 >> 172.21.15.92:0/3090343421 conn(0x7fe7dc06a800 msgr2=0x7fe7dc06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:13:45.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.132+0000 7fe7e1fdc640 1 -- 172.21.15.92:0/3090343421 shutdown_connections 2023-11-24T06:13:45.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.132+0000 7fe7e1fdc640 1 -- 172.21.15.92:0/3090343421 wait complete. 2023-11-24T06:13:45.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.133+0000 7fe7e1fdc640 1 Processor -- start 2023-11-24T06:13:45.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.133+0000 7fe7e1fdc640 1 -- start start 2023-11-24T06:13:45.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.134+0000 7fe7e1fdc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe7dc0b32e0 0x7fe7dc14eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:45.138 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.134+0000 7fe7e1fdc640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe7dc150760 0x7fe7dc146f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:45.138 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.134+0000 7fe7e1fdc640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe7dc15d300 0x7fe7dc1411e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:45.138 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.134+0000 7fe7e1fdc640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7dc147880 con 0x7fe7dc0b32e0 2023-11-24T06:13:45.138 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.134+0000 7fe7e1fdc640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7dc147b60 con 0x7fe7dc15d300 2023-11-24T06:13:45.138 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.134+0000 7fe7e1fdc640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7dc147e40 con 0x7fe7dc150760 2023-11-24T06:13:45.138 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.135+0000 7fe7db7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe7dc0b32e0 0x7fe7dc14eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:45.138 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.135+0000 7fe7db7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe7dc0b32e0 0x7fe7dc14eb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47878/0 (socket says 172.21.15.92:47878) 2023-11-24T06:13:45.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.135+0000 7fe7db7fe640 1 -- 172.21.15.92:0/3880063939 learned_addr learned my addr 172.21.15.92:0/3880063939 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:13:45.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.135+0000 7fe7db7fe640 1 -- 172.21.15.92:0/3880063939 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe7dc150760 msgr2=0x7fe7dc146f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:45.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.135+0000 7fe7db7fe640 1 --2- 172.21.15.92:0/3880063939 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fe7dc150760 0x7fe7dc146f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:45.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.135+0000 7fe7db7fe640 1 -- 172.21.15.92:0/3880063939 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe7dc15d300 msgr2=0x7fe7dc1411e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:45.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.135+0000 7fe7db7fe640 1 --2- 172.21.15.92:0/3880063939 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe7dc15d300 0x7fe7dc1411e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:45.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.135+0000 7fe7db7fe640 1 -- 172.21.15.92:0/3880063939 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7d400ab80 con 0x7fe7dc0b32e0 2023-11-24T06:13:45.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.135+0000 7fe7db7fe640 1 --2- 172.21.15.92:0/3880063939 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe7dc0b32e0 0x7fe7dc14eb40 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7fe7cc00d800 tx=0x7fe7cc00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:45.140 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.136+0000 7fe7d3fff640 1 -- 172.21.15.92:0/3880063939 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7cc00b6c0 con 0x7fe7dc0b32e0 2023-11-24T06:13:45.140 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.136+0000 7fe7d3fff640 1 -- 172.21.15.92:0/3880063939 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7cc00b820 con 0x7fe7dc0b32e0 2023-11-24T06:13:45.140 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.136+0000 7fe7d3fff640 1 -- 172.21.15.92:0/3880063939 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7cc00bbb0 con 0x7fe7dc0b32e0 2023-11-24T06:13:45.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.136+0000 7fe7e1fdc640 1 -- 172.21.15.92:0/3880063939 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7dc148120 con 0x7fe7dc0b32e0 2023-11-24T06:13:45.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.136+0000 7fe7e1fdc640 1 -- 172.21.15.92:0/3880063939 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7dc14f080 con 0x7fe7dc0b32e0 2023-11-24T06:13:45.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.137+0000 7fe7e1fdc640 1 -- 172.21.15.92:0/3880063939 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fe7dc071d10 con 0x7fe7dc0b32e0 2023-11-24T06:13:45.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.138+0000 7fe7d3fff640 1 -- 172.21.15.92:0/3880063939 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 7) v1 ==== 89333+0+0 (secure 0 0 0) 0x7fe7cc00bd10 con 0x7fe7dc0b32e0 2023-11-24T06:13:45.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.139+0000 7fe7d3fff640 1 --2- 172.21.15.92:0/3880063939 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fe7bc06af60 0x7fe7bc06d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:45.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.139+0000 7fe7d3fff640 1 -- 172.21.15.92:0/3880063939 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fe7cc089a30 con 0x7fe7dc0b32e0 2023-11-24T06:13:45.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.139+0000 7fe7daffd640 1 --2- 172.21.15.92:0/3880063939 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fe7bc06af60 0x7fe7bc06d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:45.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.139+0000 7fe7d3fff640 1 --2- 172.21.15.92:0/3880063939 >> [v2:172.21.15.92:6818/2451239906,v1:172.21.15.92:6819/2451239906] conn(0x7fe7bc070dc0 0x7fe7bc0731e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:45.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.139+0000 7fe7d3fff640 1 -- 172.21.15.92:0/3880063939 --> [v2:172.21.15.92:6818/2451239906,v1:172.21.15.92:6819/2451239906] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe7bc073890 con 0x7fe7bc070dc0 2023-11-24T06:13:45.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.139+0000 7fe7d3fff640 1 -- 172.21.15.92:0/3880063939 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_get_version_reply(handle=1 version=66) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe7cc08e2c0 con 0x7fe7dc0b32e0 2023-11-24T06:13:45.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.139+0000 7fe7dbfff640 1 --2- 172.21.15.92:0/3880063939 >> [v2:172.21.15.92:6818/2451239906,v1:172.21.15.92:6819/2451239906] conn(0x7fe7bc070dc0 0x7fe7bc0731e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:45.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.140+0000 7fe7daffd640 1 --2- 172.21.15.92:0/3880063939 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fe7bc06af60 0x7fe7bc06d420 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fe7dc1f61f0 tx=0x7fe7c4007930 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:45.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.140+0000 7fe7dbfff640 1 --2- 172.21.15.92:0/3880063939 >> [v2:172.21.15.92:6818/2451239906,v1:172.21.15.92:6819/2451239906] conn(0x7fe7bc070dc0 0x7fe7bc0731e0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:45.144 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.141+0000 7fe7d3fff640 1 -- 172.21.15.92:0/3880063939 <== osd.1 v2:172.21.15.92:6818/2451239906 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7fe7bc073890 con 0x7fe7bc070dc0 2023-11-24T06:13:45.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.155+0000 7fe7e1fdc640 1 -- 172.21.15.92:0/3880063939 --> [v2:172.21.15.92:6818/2451239906,v1:172.21.15.92:6819/2451239906] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fe7dc071a30 con 0x7fe7bc070dc0 2023-11-24T06:13:45.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.155+0000 7fe7d3fff640 1 -- 172.21.15.92:0/3880063939 <== osd.1 v2:172.21.15.92:6818/2451239906 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fe7d400a560 con 0x7fe7bc070dc0 2023-11-24T06:13:45.158 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-24T06:13:45.158 INFO:teuthology.orchestra.run.smithi092.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2023-11-24T06:13:45.158 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-24T06:13:45.159 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.156+0000 7fe7e1fdc640 1 -- 172.21.15.92:0/3880063939 >> [v2:172.21.15.92:6818/2451239906,v1:172.21.15.92:6819/2451239906] conn(0x7fe7bc070dc0 msgr2=0x7fe7bc0731e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:45.159 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.156+0000 7fe7e1fdc640 1 --2- 172.21.15.92:0/3880063939 >> [v2:172.21.15.92:6818/2451239906,v1:172.21.15.92:6819/2451239906] conn(0x7fe7bc070dc0 0x7fe7bc0731e0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:45.159 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.156+0000 7fe7e1fdc640 1 -- 172.21.15.92:0/3880063939 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fe7bc06af60 msgr2=0x7fe7bc06d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:45.159 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.156+0000 7fe7e1fdc640 1 --2- 172.21.15.92:0/3880063939 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fe7bc06af60 0x7fe7bc06d420 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fe7dc1f61f0 tx=0x7fe7c4007930 comp rx=0 tx=0).stop 2023-11-24T06:13:45.159 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.156+0000 7fe7e1fdc640 1 -- 172.21.15.92:0/3880063939 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe7dc0b32e0 msgr2=0x7fe7dc14eb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:45.159 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.156+0000 7fe7e1fdc640 1 --2- 172.21.15.92:0/3880063939 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe7dc0b32e0 0x7fe7dc14eb40 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7fe7cc00d800 tx=0x7fe7cc00dcc0 comp rx=0 tx=0).stop 2023-11-24T06:13:45.160 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.156+0000 7fe7dbfff640 1 -- 172.21.15.92:0/3880063939 reap_dead start 2023-11-24T06:13:45.160 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.156+0000 7fe7e1fdc640 1 -- 172.21.15.92:0/3880063939 shutdown_connections 2023-11-24T06:13:45.160 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.156+0000 7fe7e1fdc640 1 -- 172.21.15.92:0/3880063939 >> 172.21.15.92:0/3880063939 conn(0x7fe7dc06a800 msgr2=0x7fe7dc06c470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:13:45.160 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.156+0000 7fe7e1fdc640 1 -- 172.21.15.92:0/3880063939 shutdown_connections 2023-11-24T06:13:45.160 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.156+0000 7fe7e1fdc640 1 -- 172.21.15.92:0/3880063939 wait complete. 2023-11-24T06:13:45.174 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2023-11-24T06:13:45.382 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.379+0000 7f5641197640 1 Processor -- start 2023-11-24T06:13:45.382 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.380+0000 7f5641197640 1 -- start start 2023-11-24T06:13:45.383 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.381+0000 7f5641197640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f563c15da10 0x7f563c072230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:45.383 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.381+0000 7f5641197640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f563c0b3730 con 0x7f563c0b2bd0 2023-11-24T06:13:45.384 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.381+0000 7f5641197640 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f563c071190 con 0x7f563c150760 2023-11-24T06:13:45.384 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.381+0000 7f5641197640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f563c071470 con 0x7f563c15da10 2023-11-24T06:13:45.384 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.381+0000 7f563a59c640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f563c15da10 0x7f563c072230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:45.384 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.381+0000 7f563b59e640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f563c0b2bd0 0x7f563c0b3020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:50368/0 (socket says 172.21.15.92:50368) 2023-11-24T06:13:45.384 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.381+0000 7f563b59e640 1 -- 172.21.15.92:0/1334128096 learned_addr learned my addr 172.21.15.92:0/1334128096 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-24T06:13:45.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.382+0000 7f563a59c640 1 -- 172.21.15.92:0/1334128096 >> v1:172.21.15.118:6789/0 conn(0x7f563c150760 legacy=0x7f563c15d300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:45.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.382+0000 7f563a59c640 1 -- 172.21.15.92:0/1334128096 >> v1:172.21.15.92:6789/0 conn(0x7f563c0b2bd0 legacy=0x7f563c0b3020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:45.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.382+0000 7f563a59c640 1 -- 172.21.15.92:0/1334128096 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f563c071750 con 0x7f563c15da10 2023-11-24T06:13:45.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.382+0000 7f5639d9b640 1 -- 172.21.15.92:0/1334128096 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1033866493 0 0) 0x7f563c0b3730 con 0x7f563c0b2bd0 2023-11-24T06:13:45.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.383+0000 7f563a59c640 1 --2- 172.21.15.92:0/1334128096 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f563c15da10 0x7f563c072230 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f56240097a0 tx=0x7f562402b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a168308178f2bb0a server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:45.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.383+0000 7f5639d9b640 1 -- 172.21.15.92:0/1334128096 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5624030610 con 0x7f563c15da10 2023-11-24T06:13:45.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.383+0000 7f5639d9b640 1 -- 172.21.15.92:0/1334128096 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5624030770 con 0x7f563c15da10 2023-11-24T06:13:45.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.383+0000 7f5639d9b640 1 -- 172.21.15.92:0/1334128096 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5624030b00 con 0x7f563c15da10 2023-11-24T06:13:45.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.384+0000 7f5641197640 1 -- 172.21.15.92:0/1334128096 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f563c15da10 msgr2=0x7f563c072230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:45.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.384+0000 7f5641197640 1 --2- 172.21.15.92:0/1334128096 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f563c15da10 0x7f563c072230 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f56240097a0 tx=0x7f562402b960 comp rx=0 tx=0).stop 2023-11-24T06:13:45.388 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.384+0000 7f5641197640 1 -- 172.21.15.92:0/1334128096 shutdown_connections 2023-11-24T06:13:45.388 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.384+0000 7f5641197640 1 --2- 172.21.15.92:0/1334128096 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f563c15da10 0x7f563c072230 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:45.388 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.384+0000 7f5641197640 1 -- 172.21.15.92:0/1334128096 >> 172.21.15.92:0/1334128096 conn(0x7f563c06a800 msgr2=0x7f563c06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:13:45.388 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.384+0000 7f5641197640 1 -- 172.21.15.92:0/1334128096 shutdown_connections 2023-11-24T06:13:45.388 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.384+0000 7f5641197640 1 -- 172.21.15.92:0/1334128096 wait complete. 2023-11-24T06:13:45.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.386+0000 7f5641197640 1 Processor -- start 2023-11-24T06:13:45.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.386+0000 7f5641197640 1 -- start start 2023-11-24T06:13:45.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.387+0000 7f5641197640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f563c0b2bd0 0x7f563c13fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:45.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.387+0000 7f5641197640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f563c150760 0x7f563c0cd320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:45.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.387+0000 7f5641197640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f563c15da10 0x7f563c14e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:45.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.387+0000 7f5641197640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f563c0cd860 con 0x7f563c0b2bd0 2023-11-24T06:13:45.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.387+0000 7f5641197640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f563c0ccbf0 con 0x7f563c15da10 2023-11-24T06:13:45.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.387+0000 7f5641197640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f563c0cbfe0 con 0x7f563c150760 2023-11-24T06:13:45.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.387+0000 7f563ad9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f563c0b2bd0 0x7f563c13fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:45.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.387+0000 7f563ad9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f563c0b2bd0 0x7f563c13fc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47892/0 (socket says 172.21.15.92:47892) 2023-11-24T06:13:45.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.387+0000 7f563ad9d640 1 -- 172.21.15.92:0/3303450406 learned_addr learned my addr 172.21.15.92:0/3303450406 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:13:45.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.388+0000 7f563b59e640 1 --2- 172.21.15.92:0/3303450406 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f563c15da10 0x7f563c14e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:45.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.388+0000 7f563a59c640 1 --2- 172.21.15.92:0/3303450406 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f563c150760 0x7f563c0cd320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:45.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.388+0000 7f563ad9d640 1 -- 172.21.15.92:0/3303450406 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f563c150760 msgr2=0x7f563c0cd320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:45.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.388+0000 7f563ad9d640 1 --2- 172.21.15.92:0/3303450406 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f563c150760 0x7f563c0cd320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:45.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.388+0000 7f563ad9d640 1 -- 172.21.15.92:0/3303450406 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f563c15da10 msgr2=0x7f563c14e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:45.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.388+0000 7f563ad9d640 1 --2- 172.21.15.92:0/3303450406 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f563c15da10 0x7f563c14e910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:45.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.388+0000 7f563ad9d640 1 -- 172.21.15.92:0/3303450406 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5624009400 con 0x7f563c0b2bd0 2023-11-24T06:13:45.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.388+0000 7f563b59e640 1 --2- 172.21.15.92:0/3303450406 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f563c15da10 0x7f563c14e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:13:45.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.388+0000 7f563ad9d640 1 --2- 172.21.15.92:0/3303450406 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f563c0b2bd0 0x7f563c13fc80 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f562c00c450 tx=0x7f562c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:45.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.388+0000 7f56337fe640 1 -- 172.21.15.92:0/3303450406 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f562c0052d0 con 0x7f563c0b2bd0 2023-11-24T06:13:45.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.388+0000 7f56337fe640 1 -- 172.21.15.92:0/3303450406 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f562c005430 con 0x7f563c0b2bd0 2023-11-24T06:13:45.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.388+0000 7f56337fe640 1 -- 172.21.15.92:0/3303450406 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f562c012870 con 0x7f563c0b2bd0 2023-11-24T06:13:45.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.388+0000 7f5641197640 1 -- 172.21.15.92:0/3303450406 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f563c0cc2c0 con 0x7f563c0b2bd0 2023-11-24T06:13:45.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.389+0000 7f5641197640 1 -- 172.21.15.92:0/3303450406 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f563c0cc880 con 0x7f563c0b2bd0 2023-11-24T06:13:45.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.390+0000 7f5641197640 1 -- 172.21.15.92:0/3303450406 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f563c0675d0 con 0x7f563c0b2bd0 2023-11-24T06:13:45.395 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.391+0000 7f56337fe640 1 -- 172.21.15.92:0/3303450406 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 7) v1 ==== 89333+0+0 (secure 0 0 0) 0x7f562c0129d0 con 0x7f563c0b2bd0 2023-11-24T06:13:45.395 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.392+0000 7f56337fe640 1 --2- 172.21.15.92:0/3303450406 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f561c06af90 0x7f561c06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:45.395 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.392+0000 7f56337fe640 1 -- 172.21.15.92:0/3303450406 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f562c088f90 con 0x7f563c0b2bd0 2023-11-24T06:13:45.395 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.392+0000 7f563a59c640 1 --2- 172.21.15.92:0/3303450406 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f561c06af90 0x7f561c06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:45.395 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.393+0000 7f563a59c640 1 --2- 172.21.15.92:0/3303450406 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f561c06af90 0x7f561c06d450 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f5624036000 tx=0x7f562403b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:45.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.397+0000 7f56337fe640 1 -- 172.21.15.92:0/3303450406 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f562c0bbe90 con 0x7f563c0b2bd0 2023-11-24T06:13:45.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.581+0000 7f5641197640 1 -- 172.21.15.92:0/3303450406 --> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7f563c071d10 con 0x7f561c06af90 2023-11-24T06:13:45.584 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.582+0000 7f56337fe640 1 -- 172.21.15.92:0/3303450406 <== mgr.4101 v2:172.21.15.92:6800/784027401 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f56240309e0 con 0x7f561c06af90 2023-11-24T06:13:45.584 INFO:teuthology.orchestra.run.smithi092.stderr:instructed osd(s) 1 to deep-scrub 2023-11-24T06:13:45.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.584+0000 7f5641197640 1 -- 172.21.15.92:0/3303450406 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f561c06af90 msgr2=0x7f561c06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:45.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.584+0000 7f5641197640 1 --2- 172.21.15.92:0/3303450406 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f561c06af90 0x7f561c06d450 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f5624036000 tx=0x7f562403b040 comp rx=0 tx=0).stop 2023-11-24T06:13:45.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.584+0000 7f5641197640 1 -- 172.21.15.92:0/3303450406 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f563c0b2bd0 msgr2=0x7f563c13fc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:45.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.584+0000 7f5641197640 1 --2- 172.21.15.92:0/3303450406 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f563c0b2bd0 0x7f563c13fc80 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f562c00c450 tx=0x7f562c00c910 comp rx=0 tx=0).stop 2023-11-24T06:13:45.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.585+0000 7f5641197640 1 -- 172.21.15.92:0/3303450406 shutdown_connections 2023-11-24T06:13:45.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.585+0000 7f5641197640 1 --2- 172.21.15.92:0/3303450406 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f561c06af90 0x7f561c06d450 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:45.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.585+0000 7f5641197640 1 --2- 172.21.15.92:0/3303450406 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f563c15da10 0x7f563c14e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:45.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.585+0000 7f5641197640 1 --2- 172.21.15.92:0/3303450406 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f563c150760 0x7f563c0cd320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:45.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.585+0000 7f5641197640 1 --2- 172.21.15.92:0/3303450406 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f563c0b2bd0 0x7f563c13fc80 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:45.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.585+0000 7f5641197640 1 -- 172.21.15.92:0/3303450406 >> 172.21.15.92:0/3303450406 conn(0x7f563c06a800 msgr2=0x7f563c06cd00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:13:45.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.585+0000 7f5641197640 1 -- 172.21.15.92:0/3303450406 shutdown_connections 2023-11-24T06:13:45.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.585+0000 7f5641197640 1 -- 172.21.15.92:0/3303450406 wait complete. 2023-11-24T06:13:45.604 INFO:tasks.ceph:Scrubbing osd.2 2023-11-24T06:13:45.605 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2023-11-24T06:13:45.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.808+0000 7fc0eb5be640 1 Processor -- start 2023-11-24T06:13:45.811 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.808+0000 7fc0eb5be640 1 -- start start 2023-11-24T06:13:45.812 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.810+0000 7fc0eb5be640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fc0e4143ae0 0x7fc0e40cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:45.812 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.810+0000 7fc0eb5be640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fc0e415e3b0 0x7fc0e40702c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:45.812 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.810+0000 7fc0eb5be640 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0e40cfc30 con 0x7fc0e40cf7d0 2023-11-24T06:13:45.812 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.810+0000 7fc0eb5be640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0e40b1ee0 con 0x7fc0e4143ae0 2023-11-24T06:13:45.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.810+0000 7fc0eb5be640 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0e40b2780 con 0x7fc0e415e3b0 2023-11-24T06:13:45.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.810+0000 7fc0e935a640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fc0e4143ae0 0x7fc0e40cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:45.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.810+0000 7fc0e935a640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fc0e4143ae0 0x7fc0e40cf290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47902/0 (socket says 172.21.15.92:47902) 2023-11-24T06:13:45.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.810+0000 7fc0e935a640 1 -- 172.21.15.92:0/3822718161 learned_addr learned my addr 172.21.15.92:0/3822718161 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:13:45.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.811+0000 7fc0e935a640 1 -- 172.21.15.92:0/3822718161 >> v2:172.21.15.118:3300/0 conn(0x7fc0e415e3b0 msgr2=0x7fc0e40702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:13:45.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.811+0000 7fc0e935a640 1 --2- 172.21.15.92:0/3822718161 >> v2:172.21.15.118:3300/0 conn(0x7fc0e415e3b0 0x7fc0e40702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:45.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.811+0000 7fc0e935a640 1 -- 172.21.15.92:0/3822718161 >> v1:172.21.15.118:6789/0 conn(0x7fc0e40cf7d0 legacy=0x7fc0e415dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:13:45.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.811+0000 7fc0e935a640 1 -- 172.21.15.92:0/3822718161 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0e415e990 con 0x7fc0e4143ae0 2023-11-24T06:13:45.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.812+0000 7fc0e935a640 1 --2- 172.21.15.92:0/3822718161 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc0e4143ae0 0x7fc0e40cf290 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7fc0e406e4f0 tx=0x7fc0e002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d209b2102cef108f server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:45.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.812+0000 7fc0dbfff640 1 -- 172.21.15.92:0/3822718161 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0e0031870 con 0x7fc0e4143ae0 2023-11-24T06:13:45.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.812+0000 7fc0dbfff640 1 -- 172.21.15.92:0/3822718161 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0e00319d0 con 0x7fc0e4143ae0 2023-11-24T06:13:45.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.812+0000 7fc0dbfff640 1 -- 172.21.15.92:0/3822718161 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0e0031d60 con 0x7fc0e4143ae0 2023-11-24T06:13:45.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.812+0000 7fc0eb5be640 1 -- 172.21.15.92:0/3822718161 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc0e4143ae0 msgr2=0x7fc0e40cf290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:45.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.812+0000 7fc0eb5be640 1 --2- 172.21.15.92:0/3822718161 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc0e4143ae0 0x7fc0e40cf290 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7fc0e406e4f0 tx=0x7fc0e002b910 comp rx=0 tx=0).stop 2023-11-24T06:13:45.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.813+0000 7fc0eb5be640 1 -- 172.21.15.92:0/3822718161 shutdown_connections 2023-11-24T06:13:45.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.813+0000 7fc0eb5be640 1 --2- 172.21.15.92:0/3822718161 >> v2:172.21.15.118:3300/0 conn(0x7fc0e415e3b0 0x7fc0e40702c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:45.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.813+0000 7fc0eb5be640 1 --2- 172.21.15.92:0/3822718161 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc0e4143ae0 0x7fc0e40cf290 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:45.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.813+0000 7fc0eb5be640 1 -- 172.21.15.92:0/3822718161 >> 172.21.15.92:0/3822718161 conn(0x7fc0e406a800 msgr2=0x7fc0e406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:13:45.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.813+0000 7fc0eb5be640 1 -- 172.21.15.92:0/3822718161 shutdown_connections 2023-11-24T06:13:45.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.813+0000 7fc0eb5be640 1 -- 172.21.15.92:0/3822718161 wait complete. 2023-11-24T06:13:45.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.814+0000 7fc0eb5be640 1 Processor -- start 2023-11-24T06:13:45.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.814+0000 7fc0eb5be640 1 -- start start 2023-11-24T06:13:45.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.815+0000 7fc0eb5be640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc0e40cf7d0 0x7fc0e413c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:45.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.815+0000 7fc0eb5be640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc0e4143ae0 0x7fc0e41eefd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:45.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.815+0000 7fc0eb5be640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc0e415e3b0 0x7fc0e415c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:45.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.815+0000 7fc0eb5be640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0e40cc9d0 con 0x7fc0e40cf7d0 2023-11-24T06:13:45.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.815+0000 7fc0eb5be640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0e40cccb0 con 0x7fc0e415e3b0 2023-11-24T06:13:45.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.815+0000 7fc0eb5be640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0e40ccf90 con 0x7fc0e4143ae0 2023-11-24T06:13:45.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.816+0000 7fc0e935a640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc0e40cf7d0 0x7fc0e413c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:45.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.816+0000 7fc0e935a640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc0e40cf7d0 0x7fc0e413c8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47908/0 (socket says 172.21.15.92:47908) 2023-11-24T06:13:45.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.816+0000 7fc0e935a640 1 -- 172.21.15.92:0/1317534365 learned_addr learned my addr 172.21.15.92:0/1317534365 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:13:45.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.816+0000 7fc0e8b59640 1 --2- 172.21.15.92:0/1317534365 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc0e4143ae0 0x7fc0e41eefd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:45.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.816+0000 7fc0e935a640 1 -- 172.21.15.92:0/1317534365 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc0e4143ae0 msgr2=0x7fc0e41eefd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:45.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.816+0000 7fc0e9b5b640 1 --2- 172.21.15.92:0/1317534365 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc0e415e3b0 0x7fc0e415c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:45.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.816+0000 7fc0e935a640 1 --2- 172.21.15.92:0/1317534365 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fc0e4143ae0 0x7fc0e41eefd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:45.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.816+0000 7fc0e935a640 1 -- 172.21.15.92:0/1317534365 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc0e415e3b0 msgr2=0x7fc0e415c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:45.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.816+0000 7fc0e935a640 1 --2- 172.21.15.92:0/1317534365 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc0e415e3b0 0x7fc0e415c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:45.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.816+0000 7fc0e935a640 1 -- 172.21.15.92:0/1317534365 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0e00094d0 con 0x7fc0e40cf7d0 2023-11-24T06:13:45.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.816+0000 7fc0e9b5b640 1 --2- 172.21.15.92:0/1317534365 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc0e415e3b0 0x7fc0e415c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:13:45.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.816+0000 7fc0e935a640 1 --2- 172.21.15.92:0/1317534365 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc0e40cf7d0 0x7fc0e413c8c0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7fc0e0000c00 tx=0x7fc0e00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:45.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.817+0000 7fc0d9ffb640 1 -- 172.21.15.92:0/1317534365 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0e003ad10 con 0x7fc0e40cf7d0 2023-11-24T06:13:45.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.817+0000 7fc0d9ffb640 1 -- 172.21.15.92:0/1317534365 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0e003ae70 con 0x7fc0e40cf7d0 2023-11-24T06:13:45.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.817+0000 7fc0d9ffb640 1 -- 172.21.15.92:0/1317534365 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0e0044610 con 0x7fc0e40cf7d0 2023-11-24T06:13:45.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.817+0000 7fc0eb5be640 1 -- 172.21.15.92:0/1317534365 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0e40cd270 con 0x7fc0e40cf7d0 2023-11-24T06:13:45.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.817+0000 7fc0eb5be640 1 -- 172.21.15.92:0/1317534365 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0e413ce00 con 0x7fc0e40cf7d0 2023-11-24T06:13:45.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.818+0000 7fc0eb5be640 1 -- 172.21.15.92:0/1317534365 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fc0e40cce20 con 0x7fc0e40cf7d0 2023-11-24T06:13:45.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.819+0000 7fc0d9ffb640 1 -- 172.21.15.92:0/1317534365 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 7) v1 ==== 89333+0+0 (secure 0 0 0) 0x7fc0e004d450 con 0x7fc0e40cf7d0 2023-11-24T06:13:45.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.820+0000 7fc0d9ffb640 1 --2- 172.21.15.92:0/1317534365 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fc0c806af60 0x7fc0c806d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:45.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.820+0000 7fc0d9ffb640 1 -- 172.21.15.92:0/1317534365 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fc0e002d040 con 0x7fc0e40cf7d0 2023-11-24T06:13:45.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.820+0000 7fc0d9ffb640 1 --2- 172.21.15.92:0/1317534365 >> [v2:172.21.15.92:6810/1124979277,v1:172.21.15.92:6811/1124979277] conn(0x7fc0c8070dc0 0x7fc0c80731e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:45.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.820+0000 7fc0e8b59640 1 --2- 172.21.15.92:0/1317534365 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fc0c806af60 0x7fc0c806d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:45.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.820+0000 7fc0d9ffb640 1 -- 172.21.15.92:0/1317534365 --> [v2:172.21.15.92:6810/1124979277,v1:172.21.15.92:6811/1124979277] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc0c8073890 con 0x7fc0c8070dc0 2023-11-24T06:13:45.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.820+0000 7fc0d9ffb640 1 -- 172.21.15.92:0/1317534365 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_get_version_reply(handle=1 version=66) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc0e00b02a0 con 0x7fc0e40cf7d0 2023-11-24T06:13:45.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.820+0000 7fc0e9b5b640 1 --2- 172.21.15.92:0/1317534365 >> [v2:172.21.15.92:6810/1124979277,v1:172.21.15.92:6811/1124979277] conn(0x7fc0c8070dc0 0x7fc0c80731e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:45.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.820+0000 7fc0e8b59640 1 --2- 172.21.15.92:0/1317534365 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fc0c806af60 0x7fc0c806d420 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fc0e413db40 tx=0x7fc0d4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:45.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.821+0000 7fc0e9b5b640 1 --2- 172.21.15.92:0/1317534365 >> [v2:172.21.15.92:6810/1124979277,v1:172.21.15.92:6811/1124979277] conn(0x7fc0c8070dc0 0x7fc0c80731e0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:45.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.822+0000 7fc0d9ffb640 1 -- 172.21.15.92:0/1317534365 <== osd.2 v2:172.21.15.92:6810/1124979277 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7fc0c8073890 con 0x7fc0c8070dc0 2023-11-24T06:13:45.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.836+0000 7fc0eb5be640 1 -- 172.21.15.92:0/1317534365 --> [v2:172.21.15.92:6810/1124979277,v1:172.21.15.92:6811/1124979277] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fc0e40b2410 con 0x7fc0c8070dc0 2023-11-24T06:13:45.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.836+0000 7fc0d9ffb640 1 -- 172.21.15.92:0/1317534365 <== osd.2 v2:172.21.15.92:6810/1124979277 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fc0dc007ac0 con 0x7fc0c8070dc0 2023-11-24T06:13:45.839 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-24T06:13:45.839 INFO:teuthology.orchestra.run.smithi092.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2023-11-24T06:13:45.839 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-24T06:13:45.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.837+0000 7fc0eb5be640 1 -- 172.21.15.92:0/1317534365 >> [v2:172.21.15.92:6810/1124979277,v1:172.21.15.92:6811/1124979277] conn(0x7fc0c8070dc0 msgr2=0x7fc0c80731e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:45.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.837+0000 7fc0eb5be640 1 --2- 172.21.15.92:0/1317534365 >> [v2:172.21.15.92:6810/1124979277,v1:172.21.15.92:6811/1124979277] conn(0x7fc0c8070dc0 0x7fc0c80731e0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:45.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.837+0000 7fc0eb5be640 1 -- 172.21.15.92:0/1317534365 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fc0c806af60 msgr2=0x7fc0c806d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:45.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.837+0000 7fc0eb5be640 1 --2- 172.21.15.92:0/1317534365 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fc0c806af60 0x7fc0c806d420 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fc0e413db40 tx=0x7fc0d4009040 comp rx=0 tx=0).stop 2023-11-24T06:13:45.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.837+0000 7fc0eb5be640 1 -- 172.21.15.92:0/1317534365 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc0e40cf7d0 msgr2=0x7fc0e413c8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:45.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.837+0000 7fc0eb5be640 1 --2- 172.21.15.92:0/1317534365 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc0e40cf7d0 0x7fc0e413c8c0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7fc0e0000c00 tx=0x7fc0e00046c0 comp rx=0 tx=0).stop 2023-11-24T06:13:45.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.837+0000 7fc0e9b5b640 1 -- 172.21.15.92:0/1317534365 reap_dead start 2023-11-24T06:13:45.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.837+0000 7fc0eb5be640 1 -- 172.21.15.92:0/1317534365 shutdown_connections 2023-11-24T06:13:45.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.837+0000 7fc0eb5be640 1 -- 172.21.15.92:0/1317534365 >> 172.21.15.92:0/1317534365 conn(0x7fc0e406a800 msgr2=0x7fc0e406ce20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:13:45.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.837+0000 7fc0eb5be640 1 -- 172.21.15.92:0/1317534365 shutdown_connections 2023-11-24T06:13:45.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:45.837+0000 7fc0eb5be640 1 -- 172.21.15.92:0/1317534365 wait complete. 2023-11-24T06:13:45.855 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2023-11-24T06:13:46.065 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.063+0000 7fb51bfff640 1 Processor -- start 2023-11-24T06:13:46.066 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.063+0000 7fb51bfff640 1 -- start start 2023-11-24T06:13:46.067 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.065+0000 7fb51bfff640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fb51c06c470 0x7fb51c0cc5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:46.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.065+0000 7fb51bfff640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb51c06dab0 0x7fb51c0ccb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:46.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.065+0000 7fb51bfff640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb51c06ca10 con 0x7fb51c1447f0 2023-11-24T06:13:46.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.065+0000 7fb51bfff640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb51c06d690 con 0x7fb51c06dab0 2023-11-24T06:13:46.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.065+0000 7fb51bfff640 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb51c0cebb0 con 0x7fb51c06c470 2023-11-24T06:13:46.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.065+0000 7fb51affd640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fb51c1447f0 0x7fb51c15c080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:50376/0 (socket says 172.21.15.92:50376) 2023-11-24T06:13:46.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.066+0000 7fb51affd640 1 -- 172.21.15.92:0/516946471 learned_addr learned my addr 172.21.15.92:0/516946471 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-24T06:13:46.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.066+0000 7fb51b7fe640 1 --2- 172.21.15.92:0/516946471 >> v2:172.21.15.92:3300/0 conn(0x7fb51c06dab0 0x7fb51c0ccb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:46.070 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.066+0000 7fb51a7fc640 1 --2- 172.21.15.92:0/516946471 >> v2:172.21.15.118:3300/0 conn(0x7fb51c06c470 0x7fb51c0cc5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:46.070 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.066+0000 7fb51b7fe640 1 -- 172.21.15.92:0/516946471 >> v2:172.21.15.118:3300/0 conn(0x7fb51c06c470 msgr2=0x7fb51c0cc5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:13:46.070 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.066+0000 7fb51b7fe640 1 --2- 172.21.15.92:0/516946471 >> v2:172.21.15.118:3300/0 conn(0x7fb51c06c470 0x7fb51c0cc5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:46.070 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.066+0000 7fb51b7fe640 1 -- 172.21.15.92:0/516946471 >> v1:172.21.15.92:6789/0 conn(0x7fb51c1447f0 legacy=0x7fb51c15c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:46.071 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.066+0000 7fb51b7fe640 1 -- 172.21.15.92:0/516946471 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb51c06ccf0 con 0x7fb51c06dab0 2023-11-24T06:13:46.071 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.066+0000 7fb51a7fc640 1 --2- 172.21.15.92:0/516946471 >> v2:172.21.15.118:3300/0 conn(0x7fb51c06c470 0x7fb51c0cc5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:13:46.071 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.066+0000 7fb519ffb640 1 -- 172.21.15.92:0/516946471 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4229456359 0 0) 0x7fb51c06ca10 con 0x7fb51c1447f0 2023-11-24T06:13:46.071 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.067+0000 7fb51b7fe640 1 --2- 172.21.15.92:0/516946471 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb51c06dab0 0x7fb51c0ccb20 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7fb51c06c900 tx=0x7fb51402be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3039de190d61b2fe server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:46.071 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.067+0000 7fb519ffb640 1 -- 172.21.15.92:0/516946471 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb51402cbe0 con 0x7fb51c06dab0 2023-11-24T06:13:46.071 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.067+0000 7fb519ffb640 1 -- 172.21.15.92:0/516946471 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb51402cd40 con 0x7fb51c06dab0 2023-11-24T06:13:46.071 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.067+0000 7fb519ffb640 1 -- 172.21.15.92:0/516946471 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb514034660 con 0x7fb51c06dab0 2023-11-24T06:13:46.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.068+0000 7fb51bfff640 1 -- 172.21.15.92:0/516946471 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb51c06dab0 msgr2=0x7fb51c0ccb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:46.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.068+0000 7fb51bfff640 1 --2- 172.21.15.92:0/516946471 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb51c06dab0 0x7fb51c0ccb20 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7fb51c06c900 tx=0x7fb51402be30 comp rx=0 tx=0).stop 2023-11-24T06:13:46.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.068+0000 7fb51bfff640 1 -- 172.21.15.92:0/516946471 shutdown_connections 2023-11-24T06:13:46.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.068+0000 7fb51bfff640 1 --2- 172.21.15.92:0/516946471 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb51c06dab0 0x7fb51c0ccb20 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:46.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.068+0000 7fb51bfff640 1 --2- 172.21.15.92:0/516946471 >> v2:172.21.15.118:3300/0 conn(0x7fb51c06c470 0x7fb51c0cc5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:46.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.068+0000 7fb51bfff640 1 -- 172.21.15.92:0/516946471 >> 172.21.15.92:0/516946471 conn(0x7fb51c06a3e0 msgr2=0x7fb51c06a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:13:46.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.068+0000 7fb51bfff640 1 -- 172.21.15.92:0/516946471 shutdown_connections 2023-11-24T06:13:46.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.068+0000 7fb51bfff640 1 -- 172.21.15.92:0/516946471 wait complete. 2023-11-24T06:13:46.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.070+0000 7fb51bfff640 1 Processor -- start 2023-11-24T06:13:46.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.070+0000 7fb51bfff640 1 -- start start 2023-11-24T06:13:46.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.071+0000 7fb51bfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb51c06c470 0x7fb51c0700f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:46.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.071+0000 7fb51bfff640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb51c06dab0 0x7fb51c1f65d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:46.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.071+0000 7fb51bfff640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb51c1447f0 0x7fb51c142df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:46.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.071+0000 7fb51bfff640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb51c13b090 con 0x7fb51c06c470 2023-11-24T06:13:46.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.071+0000 7fb51bfff640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb51c1f6e20 con 0x7fb51c06dab0 2023-11-24T06:13:46.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.071+0000 7fb51bfff640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb51c1f7100 con 0x7fb51c1447f0 2023-11-24T06:13:46.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.071+0000 7fb51affd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb51c06c470 0x7fb51c0700f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:46.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.071+0000 7fb51affd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb51c06c470 0x7fb51c0700f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47936/0 (socket says 172.21.15.92:47936) 2023-11-24T06:13:46.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.071+0000 7fb51affd640 1 -- 172.21.15.92:0/384626305 learned_addr learned my addr 172.21.15.92:0/384626305 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:13:46.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.072+0000 7fb51affd640 1 -- 172.21.15.92:0/384626305 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb51c1447f0 msgr2=0x7fb51c142df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:46.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.072+0000 7fb51affd640 1 --2- 172.21.15.92:0/384626305 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb51c1447f0 0x7fb51c142df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:46.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.072+0000 7fb51affd640 1 -- 172.21.15.92:0/384626305 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb51c06dab0 msgr2=0x7fb51c1f65d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:46.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.072+0000 7fb51a7fc640 1 --2- 172.21.15.92:0/384626305 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb51c06dab0 0x7fb51c1f65d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:46.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.072+0000 7fb51affd640 1 --2- 172.21.15.92:0/384626305 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb51c06dab0 0x7fb51c1f65d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:46.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.072+0000 7fb51affd640 1 -- 172.21.15.92:0/384626305 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb514009a70 con 0x7fb51c06c470 2023-11-24T06:13:46.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.072+0000 7fb51a7fc640 1 --2- 172.21.15.92:0/384626305 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb51c06dab0 0x7fb51c1f65d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:13:46.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.072+0000 7fb51affd640 1 --2- 172.21.15.92:0/384626305 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb51c06c470 0x7fb51c0700f0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7fb50c00c430 tx=0x7fb50c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:46.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.072+0000 7fb5137fe640 1 -- 172.21.15.92:0/384626305 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb50c0074e0 con 0x7fb51c06c470 2023-11-24T06:13:46.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.072+0000 7fb5137fe640 1 -- 172.21.15.92:0/384626305 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb50c007660 con 0x7fb51c06c470 2023-11-24T06:13:46.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.072+0000 7fb51bfff640 1 -- 172.21.15.92:0/384626305 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb51c1f73e0 con 0x7fb51c06c470 2023-11-24T06:13:46.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.072+0000 7fb5137fe640 1 -- 172.21.15.92:0/384626305 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb50c004a20 con 0x7fb51c06c470 2023-11-24T06:13:46.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.072+0000 7fb51bfff640 1 -- 172.21.15.92:0/384626305 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb51c1f6b10 con 0x7fb51c06c470 2023-11-24T06:13:46.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.073+0000 7fb51bfff640 1 -- 172.21.15.92:0/384626305 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb51c067450 con 0x7fb51c06c470 2023-11-24T06:13:46.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.074+0000 7fb5137fe640 1 -- 172.21.15.92:0/384626305 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 7) v1 ==== 89333+0+0 (secure 0 0 0) 0x7fb50c004b80 con 0x7fb51c06c470 2023-11-24T06:13:46.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.075+0000 7fb5137fe640 1 --2- 172.21.15.92:0/384626305 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fb4fc06af60 0x7fb4fc06d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:46.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.075+0000 7fb5137fe640 1 -- 172.21.15.92:0/384626305 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fb50c0893c0 con 0x7fb51c06c470 2023-11-24T06:13:46.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.075+0000 7fb51a7fc640 1 --2- 172.21.15.92:0/384626305 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fb4fc06af60 0x7fb4fc06d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:46.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.076+0000 7fb51a7fc640 1 --2- 172.21.15.92:0/384626305 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fb4fc06af60 0x7fb4fc06d420 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fb504006fd0 tx=0x7fb504008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:46.081 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.080+0000 7fb5137fe640 1 -- 172.21.15.92:0/384626305 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb50c0545c0 con 0x7fb51c06c470 2023-11-24T06:13:46.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.272+0000 7fb51bfff640 1 -- 172.21.15.92:0/384626305 --> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) v1 -- 0x7fb51c0cea40 con 0x7fb4fc06af60 2023-11-24T06:13:46.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.273+0000 7fb5137fe640 1 -- 172.21.15.92:0/384626305 <== mgr.4101 v2:172.21.15.92:6800/784027401 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fb51c0cea40 con 0x7fb4fc06af60 2023-11-24T06:13:46.276 INFO:teuthology.orchestra.run.smithi092.stderr:instructed osd(s) 2 to deep-scrub 2023-11-24T06:13:46.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.275+0000 7fb51bfff640 1 -- 172.21.15.92:0/384626305 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fb4fc06af60 msgr2=0x7fb4fc06d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:46.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.275+0000 7fb51bfff640 1 --2- 172.21.15.92:0/384626305 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fb4fc06af60 0x7fb4fc06d420 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fb504006fd0 tx=0x7fb504008040 comp rx=0 tx=0).stop 2023-11-24T06:13:46.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.276+0000 7fb51bfff640 1 -- 172.21.15.92:0/384626305 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb51c06c470 msgr2=0x7fb51c0700f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:46.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.276+0000 7fb51bfff640 1 --2- 172.21.15.92:0/384626305 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb51c06c470 0x7fb51c0700f0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7fb50c00c430 tx=0x7fb50c00c8f0 comp rx=0 tx=0).stop 2023-11-24T06:13:46.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.276+0000 7fb51bfff640 1 -- 172.21.15.92:0/384626305 shutdown_connections 2023-11-24T06:13:46.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.276+0000 7fb51bfff640 1 --2- 172.21.15.92:0/384626305 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fb4fc06af60 0x7fb4fc06d420 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:46.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.276+0000 7fb51bfff640 1 --2- 172.21.15.92:0/384626305 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb51c1447f0 0x7fb51c142df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:46.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.276+0000 7fb51bfff640 1 --2- 172.21.15.92:0/384626305 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb51c06dab0 0x7fb51c1f65d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:46.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.276+0000 7fb51bfff640 1 --2- 172.21.15.92:0/384626305 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb51c06c470 0x7fb51c0700f0 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:46.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.276+0000 7fb51bfff640 1 -- 172.21.15.92:0/384626305 >> 172.21.15.92:0/384626305 conn(0x7fb51c06a3e0 msgr2=0x7fb51c142640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:13:46.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.276+0000 7fb51bfff640 1 -- 172.21.15.92:0/384626305 shutdown_connections 2023-11-24T06:13:46.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.276+0000 7fb51bfff640 1 -- 172.21.15.92:0/384626305 wait complete. 2023-11-24T06:13:46.295 INFO:tasks.ceph:Scrubbing osd.3 2023-11-24T06:13:46.295 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2023-11-24T06:13:46.494 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.491+0000 7fdff6a52640 1 Processor -- start 2023-11-24T06:13:46.494 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.492+0000 7fdff6a52640 1 -- start start 2023-11-24T06:13:46.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.493+0000 7fdff6a52640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fdff015d3c0 0x7fdff00d1f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:46.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.493+0000 7fdff6a52640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fdff00b4110 0x7fdff00cfeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:46.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.493+0000 7fdff6a52640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdff00b2c70 con 0x7fdff00cfa40 2023-11-24T06:13:46.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.493+0000 7fdff6a52640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdff00d03f0 con 0x7fdff015d3c0 2023-11-24T06:13:46.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.493+0000 7fdff6a52640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdff00cf6d0 con 0x7fdff00b4110 2023-11-24T06:13:46.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.493+0000 7fdff5a50640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fdff015d3c0 0x7fdff00d1f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:46.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.493+0000 7fdff524f640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fdff00cfa40 0x7fdff00cde90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:50388/0 (socket says 172.21.15.92:50388) 2023-11-24T06:13:46.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.493+0000 7fdff524f640 1 -- 172.21.15.92:0/163817989 learned_addr learned my addr 172.21.15.92:0/163817989 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-24T06:13:46.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.493+0000 7fdff5a50640 1 --2- 172.21.15.92:0/163817989 >> v2:172.21.15.92:3300/0 conn(0x7fdff015d3c0 0x7fdff00d1f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47942/0 (socket says 172.21.15.92:47942) 2023-11-24T06:13:46.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.494+0000 7fdff5a50640 1 -- 172.21.15.92:0/163817989 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fdff00b4110 msgr2=0x7fdff00cfeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:13:46.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.494+0000 7fdff5a50640 1 --2- 172.21.15.92:0/163817989 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fdff00b4110 0x7fdff00cfeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:46.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.494+0000 7fdff5a50640 1 -- 172.21.15.92:0/163817989 >> v1:172.21.15.92:6789/0 conn(0x7fdff00cfa40 legacy=0x7fdff00cde90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:46.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.494+0000 7fdff5a50640 1 -- 172.21.15.92:0/163817989 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdff00cf3f0 con 0x7fdff015d3c0 2023-11-24T06:13:46.499 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.495+0000 7fdff5a50640 1 --2- 172.21.15.92:0/163817989 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdff015d3c0 0x7fdff00d1f20 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7fdfe0009d90 tx=0x7fdfe002be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60405914c030e866 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:46.499 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.495+0000 7fdff4a4e640 1 -- 172.21.15.92:0/163817989 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfe002cbd0 con 0x7fdff015d3c0 2023-11-24T06:13:46.499 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.495+0000 7fdff4a4e640 1 -- 172.21.15.92:0/163817989 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfe002cd30 con 0x7fdff015d3c0 2023-11-24T06:13:46.499 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.495+0000 7fdff4a4e640 1 -- 172.21.15.92:0/163817989 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfe002cea0 con 0x7fdff015d3c0 2023-11-24T06:13:46.499 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.496+0000 7fdff6a52640 1 -- 172.21.15.92:0/163817989 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdff015d3c0 msgr2=0x7fdff00d1f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:46.499 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.496+0000 7fdff6a52640 1 --2- 172.21.15.92:0/163817989 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdff015d3c0 0x7fdff00d1f20 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7fdfe0009d90 tx=0x7fdfe002be20 comp rx=0 tx=0).stop 2023-11-24T06:13:46.499 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.496+0000 7fdff6a52640 1 -- 172.21.15.92:0/163817989 shutdown_connections 2023-11-24T06:13:46.499 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.496+0000 7fdff6a52640 1 --2- 172.21.15.92:0/163817989 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fdff00b4110 0x7fdff00cfeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:46.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.496+0000 7fdff6a52640 1 --2- 172.21.15.92:0/163817989 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdff015d3c0 0x7fdff00d1f20 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:46.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.496+0000 7fdff6a52640 1 -- 172.21.15.92:0/163817989 >> 172.21.15.92:0/163817989 conn(0x7fdff00d3050 msgr2=0x7fdff00d3460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:13:46.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.496+0000 7fdff6a52640 1 -- 172.21.15.92:0/163817989 shutdown_connections 2023-11-24T06:13:46.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.496+0000 7fdff6a52640 1 -- 172.21.15.92:0/163817989 wait complete. 2023-11-24T06:13:46.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.498+0000 7fdff6a52640 1 Processor -- start 2023-11-24T06:13:46.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.498+0000 7fdff6a52640 1 -- start start 2023-11-24T06:13:46.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.499+0000 7fdff6a52640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdff00b4110 0x7fdff00c7c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:46.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.499+0000 7fdff6a52640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fdff00cfa40 0x7fdff00c7590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:46.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.499+0000 7fdff6a52640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fdff015d3c0 0x7fdff00c93a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:46.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.499+0000 7fdff6a52640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdff00c8300 con 0x7fdff00b4110 2023-11-24T06:13:46.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.499+0000 7fdff6a52640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdff00c85e0 con 0x7fdff00cfa40 2023-11-24T06:13:46.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.499+0000 7fdff6a52640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdff00c88c0 con 0x7fdff015d3c0 2023-11-24T06:13:46.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.499+0000 7fdff5a50640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdff00b4110 0x7fdff00c7c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:46.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.499+0000 7fdff5a50640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdff00b4110 0x7fdff00c7c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47944/0 (socket says 172.21.15.92:47944) 2023-11-24T06:13:46.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.499+0000 7fdff5a50640 1 -- 172.21.15.92:0/1380922162 learned_addr learned my addr 172.21.15.92:0/1380922162 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:13:46.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.499+0000 7fdff524f640 1 --2- 172.21.15.92:0/1380922162 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fdff00cfa40 0x7fdff00c7590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:46.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.500+0000 7fdff6251640 1 --2- 172.21.15.92:0/1380922162 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fdff015d3c0 0x7fdff00c93a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:46.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.500+0000 7fdff5a50640 1 -- 172.21.15.92:0/1380922162 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fdff015d3c0 msgr2=0x7fdff00c93a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:46.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.500+0000 7fdff5a50640 1 --2- 172.21.15.92:0/1380922162 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fdff015d3c0 0x7fdff00c93a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:46.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.500+0000 7fdff5a50640 1 -- 172.21.15.92:0/1380922162 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fdff00cfa40 msgr2=0x7fdff00c7590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:46.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.500+0000 7fdff5a50640 1 --2- 172.21.15.92:0/1380922162 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fdff00cfa40 0x7fdff00c7590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:46.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.500+0000 7fdff5a50640 1 -- 172.21.15.92:0/1380922162 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfe0009a70 con 0x7fdff00b4110 2023-11-24T06:13:46.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.500+0000 7fdff524f640 1 --2- 172.21.15.92:0/1380922162 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fdff00cfa40 0x7fdff00c7590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:13:46.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.500+0000 7fdff5a50640 1 --2- 172.21.15.92:0/1380922162 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdff00b4110 0x7fdff00c7c40 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7fdfe0037040 tx=0x7fdfe0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:46.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.500+0000 7fdfee7fc640 1 -- 172.21.15.92:0/1380922162 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfe002cea0 con 0x7fdff00b4110 2023-11-24T06:13:46.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.500+0000 7fdfee7fc640 1 -- 172.21.15.92:0/1380922162 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfe0040650 con 0x7fdff00b4110 2023-11-24T06:13:46.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.500+0000 7fdff6a52640 1 -- 172.21.15.92:0/1380922162 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdff00c8ba0 con 0x7fdff00b4110 2023-11-24T06:13:46.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.500+0000 7fdfee7fc640 1 -- 172.21.15.92:0/1380922162 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfe0040a00 con 0x7fdff00b4110 2023-11-24T06:13:46.506 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.501+0000 7fdff6a52640 1 -- 172.21.15.92:0/1380922162 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdff00c98e0 con 0x7fdff00b4110 2023-11-24T06:13:46.507 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.501+0000 7fdff6a52640 1 -- 172.21.15.92:0/1380922162 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fdff00c8750 con 0x7fdff00b4110 2023-11-24T06:13:46.507 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.503+0000 7fdfee7fc640 1 -- 172.21.15.92:0/1380922162 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 7) v1 ==== 89333+0+0 (secure 0 0 0) 0x7fdfe0040b60 con 0x7fdff00b4110 2023-11-24T06:13:46.507 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.503+0000 7fdfee7fc640 1 --2- 172.21.15.92:0/1380922162 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fdfdc06af60 0x7fdfdc06d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:46.507 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.504+0000 7fdfee7fc640 1 -- 172.21.15.92:0/1380922162 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fdfe00ac1e0 con 0x7fdff00b4110 2023-11-24T06:13:46.507 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.504+0000 7fdff524f640 1 --2- 172.21.15.92:0/1380922162 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fdfdc06af60 0x7fdfdc06d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:46.508 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.504+0000 7fdfee7fc640 1 --2- 172.21.15.92:0/1380922162 >> [v2:172.21.15.92:6826/1734115782,v1:172.21.15.92:6827/1734115782] conn(0x7fdfdc070dc0 0x7fdfdc0731e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:46.508 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.504+0000 7fdfee7fc640 1 -- 172.21.15.92:0/1380922162 --> [v2:172.21.15.92:6826/1734115782,v1:172.21.15.92:6827/1734115782] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fdfdc073890 con 0x7fdfdc070dc0 2023-11-24T06:13:46.508 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.504+0000 7fdfee7fc640 1 -- 172.21.15.92:0/1380922162 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_get_version_reply(handle=1 version=66) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdfe00ae8a0 con 0x7fdff00b4110 2023-11-24T06:13:46.508 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.504+0000 7fdff6251640 1 --2- 172.21.15.92:0/1380922162 >> [v2:172.21.15.92:6826/1734115782,v1:172.21.15.92:6827/1734115782] conn(0x7fdfdc070dc0 0x7fdfdc0731e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:46.508 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.505+0000 7fdff524f640 1 --2- 172.21.15.92:0/1380922162 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fdfdc06af60 0x7fdfdc06d420 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fdff00b5690 tx=0x7fdfe4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:46.508 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.505+0000 7fdff6251640 1 --2- 172.21.15.92:0/1380922162 >> [v2:172.21.15.92:6826/1734115782,v1:172.21.15.92:6827/1734115782] conn(0x7fdfdc070dc0 0x7fdfdc0731e0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:46.509 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.505+0000 7fdfee7fc640 1 -- 172.21.15.92:0/1380922162 <== osd.3 v2:172.21.15.92:6826/1734115782 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7fdfdc073890 con 0x7fdfdc070dc0 2023-11-24T06:13:46.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.520+0000 7fdff6a52640 1 -- 172.21.15.92:0/1380922162 --> [v2:172.21.15.92:6826/1734115782,v1:172.21.15.92:6827/1734115782] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fdff00b2900 con 0x7fdfdc070dc0 2023-11-24T06:13:46.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.520+0000 7fdfee7fc640 1 -- 172.21.15.92:0/1380922162 <== osd.3 v2:172.21.15.92:6826/1734115782 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fdfe800a8f0 con 0x7fdfdc070dc0 2023-11-24T06:13:46.523 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-24T06:13:46.523 INFO:teuthology.orchestra.run.smithi092.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2023-11-24T06:13:46.523 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-24T06:13:46.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.521+0000 7fdff6a52640 1 -- 172.21.15.92:0/1380922162 >> [v2:172.21.15.92:6826/1734115782,v1:172.21.15.92:6827/1734115782] conn(0x7fdfdc070dc0 msgr2=0x7fdfdc0731e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:46.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.521+0000 7fdff6a52640 1 --2- 172.21.15.92:0/1380922162 >> [v2:172.21.15.92:6826/1734115782,v1:172.21.15.92:6827/1734115782] conn(0x7fdfdc070dc0 0x7fdfdc0731e0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:46.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.521+0000 7fdff6a52640 1 -- 172.21.15.92:0/1380922162 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fdfdc06af60 msgr2=0x7fdfdc06d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:46.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.521+0000 7fdff6a52640 1 --2- 172.21.15.92:0/1380922162 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fdfdc06af60 0x7fdfdc06d420 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fdff00b5690 tx=0x7fdfe4009040 comp rx=0 tx=0).stop 2023-11-24T06:13:46.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.521+0000 7fdff6a52640 1 -- 172.21.15.92:0/1380922162 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdff00b4110 msgr2=0x7fdff00c7c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:46.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.521+0000 7fdff6a52640 1 --2- 172.21.15.92:0/1380922162 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdff00b4110 0x7fdff00c7c40 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7fdfe0037040 tx=0x7fdfe0004a40 comp rx=0 tx=0).stop 2023-11-24T06:13:46.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.521+0000 7fdff6251640 1 -- 172.21.15.92:0/1380922162 reap_dead start 2023-11-24T06:13:46.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.521+0000 7fdff6a52640 1 -- 172.21.15.92:0/1380922162 shutdown_connections 2023-11-24T06:13:46.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.521+0000 7fdff6a52640 1 -- 172.21.15.92:0/1380922162 >> 172.21.15.92:0/1380922162 conn(0x7fdff00d3050 msgr2=0x7fdff00b45d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:13:46.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.521+0000 7fdff6a52640 1 -- 172.21.15.92:0/1380922162 shutdown_connections 2023-11-24T06:13:46.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.521+0000 7fdff6a52640 1 -- 172.21.15.92:0/1380922162 wait complete. 2023-11-24T06:13:46.540 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2023-11-24T06:13:46.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.738+0000 7ff595c0e640 1 Processor -- start 2023-11-24T06:13:46.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.739+0000 7ff595c0e640 1 -- start start 2023-11-24T06:13:46.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.740+0000 7ff595c0e640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7ff590070b20 0x7ff5900b28a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:46.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.740+0000 7ff595c0e640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff59006e0f0 0x7ff5900b2de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:46.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.740+0000 7ff595c0e640 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5900b2410 con 0x7ff590161810 2023-11-24T06:13:46.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.740+0000 7ff595c0e640 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff59006d7f0 con 0x7ff590070b20 2023-11-24T06:13:46.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.740+0000 7ff595c0e640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff590072730 con 0x7ff59006e0f0 2023-11-24T06:13:46.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.741+0000 7ff58ffff640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff59006e0f0 0x7ff5900b2de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:46.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.741+0000 7ff58ffff640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff59006e0f0 0x7ff5900b2de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3301/0 says I am v2:172.21.15.92:34728/0 (socket says 172.21.15.92:34728) 2023-11-24T06:13:46.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.741+0000 7ff58ffff640 1 -- 172.21.15.92:0/3430356150 learned_addr learned my addr 172.21.15.92:0/3430356150 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:13:46.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.741+0000 7ff58effd640 1 --2- 172.21.15.92:0/3430356150 >> v2:172.21.15.118:3300/0 conn(0x7ff590070b20 0x7ff5900b28a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:46.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.742+0000 7ff58effd640 1 -- 172.21.15.92:0/3430356150 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff59006e0f0 msgr2=0x7ff5900b2de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:13:46.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.742+0000 7ff58effd640 1 --2- 172.21.15.92:0/3430356150 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff59006e0f0 0x7ff5900b2de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:46.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.742+0000 7ff58effd640 1 -- 172.21.15.92:0/3430356150 >> v1:172.21.15.118:6789/0 conn(0x7ff590161810 legacy=0x7ff590070410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:46.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.742+0000 7ff58effd640 1 -- 172.21.15.92:0/3430356150 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff590161bf0 con 0x7ff590070b20 2023-11-24T06:13:46.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.743+0000 7ff58effd640 1 --2- 172.21.15.92:0/3430356150 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff590070b20 0x7ff5900b28a0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7ff5780097a0 tx=0x7ff57802b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8ea378c2e20f0f39 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:46.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.743+0000 7ff58e7fc640 1 -- 172.21.15.92:0/3430356150 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff57802d040 con 0x7ff590070b20 2023-11-24T06:13:46.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.743+0000 7ff58e7fc640 1 -- 172.21.15.92:0/3430356150 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff578031810 con 0x7ff590070b20 2023-11-24T06:13:46.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.743+0000 7ff58e7fc640 1 -- 172.21.15.92:0/3430356150 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff578031ba0 con 0x7ff590070b20 2023-11-24T06:13:46.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.744+0000 7ff595c0e640 1 -- 172.21.15.92:0/3430356150 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff590070b20 msgr2=0x7ff5900b28a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:46.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.744+0000 7ff595c0e640 1 --2- 172.21.15.92:0/3430356150 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff590070b20 0x7ff5900b28a0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7ff5780097a0 tx=0x7ff57802b8b0 comp rx=0 tx=0).stop 2023-11-24T06:13:46.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.744+0000 7ff595c0e640 1 -- 172.21.15.92:0/3430356150 shutdown_connections 2023-11-24T06:13:46.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.744+0000 7ff595c0e640 1 --2- 172.21.15.92:0/3430356150 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff59006e0f0 0x7ff5900b2de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:46.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.744+0000 7ff595c0e640 1 --2- 172.21.15.92:0/3430356150 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff590070b20 0x7ff5900b28a0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:46.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.744+0000 7ff595c0e640 1 -- 172.21.15.92:0/3430356150 >> 172.21.15.92:0/3430356150 conn(0x7ff5900b0c00 msgr2=0x7ff5900b1010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:13:46.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.744+0000 7ff595c0e640 1 -- 172.21.15.92:0/3430356150 shutdown_connections 2023-11-24T06:13:46.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.744+0000 7ff595c0e640 1 -- 172.21.15.92:0/3430356150 wait complete. 2023-11-24T06:13:46.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.746+0000 7ff595c0e640 1 Processor -- start 2023-11-24T06:13:46.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.746+0000 7ff595c0e640 1 -- start start 2023-11-24T06:13:46.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.747+0000 7ff595c0e640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff59006e0f0 0x7ff5901fb3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:46.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.747+0000 7ff595c0e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff590070b20 0x7ff590149840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:46.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.747+0000 7ff595c0e640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff590161810 0x7ff59015bf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:46.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.747+0000 7ff595c0e640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff59015bb00 con 0x7ff590070b20 2023-11-24T06:13:46.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.747+0000 7ff595c0e640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5901fb8f0 con 0x7ff590161810 2023-11-24T06:13:46.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.747+0000 7ff595c0e640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff590149d80 con 0x7ff59006e0f0 2023-11-24T06:13:46.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.747+0000 7ff58effd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff590070b20 0x7ff590149840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:46.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.747+0000 7ff58effd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff590070b20 0x7ff590149840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47956/0 (socket says 172.21.15.92:47956) 2023-11-24T06:13:46.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.747+0000 7ff58effd640 1 -- 172.21.15.92:0/1974871397 learned_addr learned my addr 172.21.15.92:0/1974871397 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:13:46.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.748+0000 7ff58ffff640 1 --2- 172.21.15.92:0/1974871397 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff590161810 0x7ff59015bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:46.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.748+0000 7ff58f7fe640 1 --2- 172.21.15.92:0/1974871397 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff59006e0f0 0x7ff5901fb3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:46.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.748+0000 7ff58effd640 1 -- 172.21.15.92:0/1974871397 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff59006e0f0 msgr2=0x7ff5901fb3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:46.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.748+0000 7ff58effd640 1 --2- 172.21.15.92:0/1974871397 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff59006e0f0 0x7ff5901fb3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:46.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.748+0000 7ff58effd640 1 -- 172.21.15.92:0/1974871397 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff590161810 msgr2=0x7ff59015bf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:46.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.748+0000 7ff58effd640 1 --2- 172.21.15.92:0/1974871397 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff590161810 0x7ff59015bf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:46.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.748+0000 7ff58effd640 1 -- 172.21.15.92:0/1974871397 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff578009400 con 0x7ff590070b20 2023-11-24T06:13:46.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.748+0000 7ff58effd640 1 --2- 172.21.15.92:0/1974871397 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff590070b20 0x7ff590149840 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7ff57802bec0 tx=0x7ff5780046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:46.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.748+0000 7ff587fff640 1 -- 172.21.15.92:0/1974871397 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff57803ad60 con 0x7ff590070b20 2023-11-24T06:13:46.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.748+0000 7ff587fff640 1 -- 172.21.15.92:0/1974871397 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5780423f0 con 0x7ff590070b20 2023-11-24T06:13:46.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.748+0000 7ff587fff640 1 -- 172.21.15.92:0/1974871397 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5780427a0 con 0x7ff590070b20 2023-11-24T06:13:46.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.748+0000 7ff595c0e640 1 -- 172.21.15.92:0/1974871397 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff59015ae80 con 0x7ff590070b20 2023-11-24T06:13:46.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.749+0000 7ff595c0e640 1 -- 172.21.15.92:0/1974871397 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff59015b440 con 0x7ff590070b20 2023-11-24T06:13:46.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.750+0000 7ff595c0e640 1 -- 172.21.15.92:0/1974871397 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff59015bc40 con 0x7ff590070b20 2023-11-24T06:13:46.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.751+0000 7ff587fff640 1 -- 172.21.15.92:0/1974871397 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 7) v1 ==== 89333+0+0 (secure 0 0 0) 0x7ff578042900 con 0x7ff590070b20 2023-11-24T06:13:46.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.751+0000 7ff587fff640 1 --2- 172.21.15.92:0/1974871397 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7ff57006af90 0x7ff57006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:46.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.752+0000 7ff587fff640 1 -- 172.21.15.92:0/1974871397 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 4943+0+0 (secure 0 0 0) 0x7ff5780ab530 con 0x7ff590070b20 2023-11-24T06:13:46.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.752+0000 7ff58f7fe640 1 --2- 172.21.15.92:0/1974871397 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7ff57006af90 0x7ff57006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:46.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.752+0000 7ff58f7fe640 1 --2- 172.21.15.92:0/1974871397 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7ff57006af90 0x7ff57006d450 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7ff5800012e0 tx=0x7ff580009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:46.758 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.756+0000 7ff587fff640 1 -- 172.21.15.92:0/1974871397 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff59015bc40 con 0x7ff590070b20 2023-11-24T06:13:46.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.950+0000 7ff595c0e640 1 -- 172.21.15.92:0/1974871397 --> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) v1 -- 0x7ff590160810 con 0x7ff57006af90 2023-11-24T06:13:46.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.951+0000 7ff587fff640 1 -- 172.21.15.92:0/1974871397 <== mgr.4101 v2:172.21.15.92:6800/784027401 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7ff590160810 con 0x7ff57006af90 2023-11-24T06:13:46.953 INFO:teuthology.orchestra.run.smithi092.stderr:instructed osd(s) 3 to deep-scrub 2023-11-24T06:13:46.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.953+0000 7ff595c0e640 1 -- 172.21.15.92:0/1974871397 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7ff57006af90 msgr2=0x7ff57006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:46.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.953+0000 7ff595c0e640 1 --2- 172.21.15.92:0/1974871397 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7ff57006af90 0x7ff57006d450 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7ff5800012e0 tx=0x7ff580009040 comp rx=0 tx=0).stop 2023-11-24T06:13:46.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.953+0000 7ff595c0e640 1 -- 172.21.15.92:0/1974871397 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff590070b20 msgr2=0x7ff590149840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:46.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.953+0000 7ff595c0e640 1 --2- 172.21.15.92:0/1974871397 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff590070b20 0x7ff590149840 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7ff57802bec0 tx=0x7ff5780046c0 comp rx=0 tx=0).stop 2023-11-24T06:13:46.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.953+0000 7ff595c0e640 1 -- 172.21.15.92:0/1974871397 shutdown_connections 2023-11-24T06:13:46.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.953+0000 7ff595c0e640 1 --2- 172.21.15.92:0/1974871397 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7ff57006af90 0x7ff57006d450 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:46.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.953+0000 7ff595c0e640 1 --2- 172.21.15.92:0/1974871397 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff590161810 0x7ff59015bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:46.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.953+0000 7ff595c0e640 1 --2- 172.21.15.92:0/1974871397 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff590070b20 0x7ff590149840 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:46.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.953+0000 7ff595c0e640 1 --2- 172.21.15.92:0/1974871397 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7ff59006e0f0 0x7ff5901fb3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:46.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.953+0000 7ff595c0e640 1 -- 172.21.15.92:0/1974871397 >> 172.21.15.92:0/1974871397 conn(0x7ff5900b0c00 msgr2=0x7ff59015fbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:13:46.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.954+0000 7ff595c0e640 1 -- 172.21.15.92:0/1974871397 shutdown_connections 2023-11-24T06:13:46.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:46.954+0000 7ff595c0e640 1 -- 172.21.15.92:0/1974871397 wait complete. 2023-11-24T06:13:46.974 INFO:tasks.ceph:Scrubbing osd.4 2023-11-24T06:13:46.974 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2023-11-24T06:13:47.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.187+0000 7fd3a4967640 1 Processor -- start 2023-11-24T06:13:47.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.187+0000 7fd3a4967640 1 -- start start 2023-11-24T06:13:47.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.189+0000 7fd3a4967640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fd3a00cc500 0x7fd3a00cc990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:47.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.189+0000 7fd3a4967640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fd3a006d160 0x7fd3a006d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:47.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.189+0000 7fd3a4967640 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3a0070e50 con 0x7fd3a0143c40 2023-11-24T06:13:47.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.189+0000 7fd3a4967640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3a00723c0 con 0x7fd3a00cc500 2023-11-24T06:13:47.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.189+0000 7fd3a4967640 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3a00cced0 con 0x7fd3a006d160 2023-11-24T06:13:47.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.189+0000 7fd397fff640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fd3a00cc500 0x7fd3a00cc990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:47.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.190+0000 7fd397fff640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fd3a00cc500 0x7fd3a00cc990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47960/0 (socket says 172.21.15.92:47960) 2023-11-24T06:13:47.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.190+0000 7fd397fff640 1 -- 172.21.15.92:0/1710478035 learned_addr learned my addr 172.21.15.92:0/1710478035 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:13:47.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.190+0000 7fd39f59e640 1 --2- 172.21.15.92:0/1710478035 >> v2:172.21.15.118:3300/0 conn(0x7fd3a006d160 0x7fd3a006d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:47.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.190+0000 7fd397fff640 1 -- 172.21.15.92:0/1710478035 >> v2:172.21.15.118:3300/0 conn(0x7fd3a006d160 msgr2=0x7fd3a006d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:13:47.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.190+0000 7fd397fff640 1 --2- 172.21.15.92:0/1710478035 >> v2:172.21.15.118:3300/0 conn(0x7fd3a006d160 0x7fd3a006d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:47.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.190+0000 7fd397fff640 1 -- 172.21.15.92:0/1710478035 >> v1:172.21.15.118:6789/0 conn(0x7fd3a0143c40 legacy=0x7fd3a0071290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:13:47.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.190+0000 7fd397fff640 1 -- 172.21.15.92:0/1710478035 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3a0144020 con 0x7fd3a00cc500 2023-11-24T06:13:47.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.191+0000 7fd397fff640 1 --2- 172.21.15.92:0/1710478035 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd3a00cc500 0x7fd3a00cc990 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7fd3a0070690 tx=0x7fd38802b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c184cc5de4fa4284 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:47.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.192+0000 7fd39e59c640 1 -- 172.21.15.92:0/1710478035 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3880317e0 con 0x7fd3a00cc500 2023-11-24T06:13:47.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.192+0000 7fd39e59c640 1 -- 172.21.15.92:0/1710478035 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd388031940 con 0x7fd3a00cc500 2023-11-24T06:13:47.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.192+0000 7fd39e59c640 1 -- 172.21.15.92:0/1710478035 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd388031cd0 con 0x7fd3a00cc500 2023-11-24T06:13:47.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.192+0000 7fd3a4967640 1 -- 172.21.15.92:0/1710478035 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd3a00cc500 msgr2=0x7fd3a00cc990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:47.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.192+0000 7fd3a4967640 1 --2- 172.21.15.92:0/1710478035 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd3a00cc500 0x7fd3a00cc990 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7fd3a0070690 tx=0x7fd38802b880 comp rx=0 tx=0).stop 2023-11-24T06:13:47.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.193+0000 7fd3a4967640 1 -- 172.21.15.92:0/1710478035 shutdown_connections 2023-11-24T06:13:47.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.193+0000 7fd3a4967640 1 --2- 172.21.15.92:0/1710478035 >> v2:172.21.15.118:3300/0 conn(0x7fd3a006d160 0x7fd3a006d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:47.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.193+0000 7fd3a4967640 1 --2- 172.21.15.92:0/1710478035 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd3a00cc500 0x7fd3a00cc990 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:47.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.193+0000 7fd3a4967640 1 -- 172.21.15.92:0/1710478035 >> 172.21.15.92:0/1710478035 conn(0x7fd3a006a800 msgr2=0x7fd3a006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:13:47.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.193+0000 7fd3a4967640 1 -- 172.21.15.92:0/1710478035 shutdown_connections 2023-11-24T06:13:47.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.193+0000 7fd3a4967640 1 -- 172.21.15.92:0/1710478035 wait complete. 2023-11-24T06:13:47.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.194+0000 7fd3a4967640 1 Processor -- start 2023-11-24T06:13:47.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.195+0000 7fd3a4967640 1 -- start start 2023-11-24T06:13:47.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.196+0000 7fd3a4967640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd3a006d160 0x7fd3a00c8da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:47.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.196+0000 7fd3a4967640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd3a00cc500 0x7fd3a00ca420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:47.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.196+0000 7fd3a4967640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd3a0143c40 0x7fd3a01423b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:47.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.196+0000 7fd3a4967640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3a00c9f90 con 0x7fd3a00cc500 2023-11-24T06:13:47.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.196+0000 7fd3a4967640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3a00c9380 con 0x7fd3a006d160 2023-11-24T06:13:47.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.196+0000 7fd3a4967640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3a00c9660 con 0x7fd3a0143c40 2023-11-24T06:13:47.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.196+0000 7fd397fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd3a00cc500 0x7fd3a00ca420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:47.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.196+0000 7fd397fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd3a00cc500 0x7fd3a00ca420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47966/0 (socket says 172.21.15.92:47966) 2023-11-24T06:13:47.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.196+0000 7fd397fff640 1 -- 172.21.15.92:0/3027141248 learned_addr learned my addr 172.21.15.92:0/3027141248 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:13:47.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.196+0000 7fd39ed9d640 1 --2- 172.21.15.92:0/3027141248 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd3a006d160 0x7fd3a00c8da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:47.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.196+0000 7fd397fff640 1 -- 172.21.15.92:0/3027141248 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd3a0143c40 msgr2=0x7fd3a01423b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:47.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.196+0000 7fd397fff640 1 --2- 172.21.15.92:0/3027141248 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd3a0143c40 0x7fd3a01423b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:47.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.196+0000 7fd397fff640 1 -- 172.21.15.92:0/3027141248 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd3a006d160 msgr2=0x7fd3a00c8da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:47.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.196+0000 7fd397fff640 1 --2- 172.21.15.92:0/3027141248 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd3a006d160 0x7fd3a00c8da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:47.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.196+0000 7fd397fff640 1 -- 172.21.15.92:0/3027141248 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3880094d0 con 0x7fd3a00cc500 2023-11-24T06:13:47.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.196+0000 7fd39ed9d640 1 --2- 172.21.15.92:0/3027141248 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd3a006d160 0x7fd3a00c8da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:13:47.201 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.197+0000 7fd397fff640 1 --2- 172.21.15.92:0/3027141248 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd3a00cc500 0x7fd3a00ca420 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7fd388000c00 tx=0x7fd3880046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:47.201 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.197+0000 7fd3977fe640 1 -- 172.21.15.92:0/3027141248 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd388032880 con 0x7fd3a00cc500 2023-11-24T06:13:47.202 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.197+0000 7fd3977fe640 1 -- 172.21.15.92:0/3027141248 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3880329e0 con 0x7fd3a00cc500 2023-11-24T06:13:47.202 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.197+0000 7fd3977fe640 1 -- 172.21.15.92:0/3027141248 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd388032d90 con 0x7fd3a00cc500 2023-11-24T06:13:47.202 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.197+0000 7fd3a4967640 1 -- 172.21.15.92:0/3027141248 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3a00c9940 con 0x7fd3a00cc500 2023-11-24T06:13:47.202 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.197+0000 7fd3a4967640 1 -- 172.21.15.92:0/3027141248 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3a00ca960 con 0x7fd3a00cc500 2023-11-24T06:13:47.202 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.198+0000 7fd3a4967640 1 -- 172.21.15.92:0/3027141248 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fd3a00cb780 con 0x7fd3a00cc500 2023-11-24T06:13:47.202 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.199+0000 7fd3977fe640 1 -- 172.21.15.92:0/3027141248 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 7) v1 ==== 89333+0+0 (secure 0 0 0) 0x7fd38804d3f0 con 0x7fd3a00cc500 2023-11-24T06:13:47.203 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.200+0000 7fd3977fe640 1 --2- 172.21.15.92:0/3027141248 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fd38006af60 0x7fd38006d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:47.204 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.200+0000 7fd3977fe640 1 -- 172.21.15.92:0/3027141248 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fd3880ab450 con 0x7fd3a00cc500 2023-11-24T06:13:47.204 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.200+0000 7fd39ed9d640 1 --2- 172.21.15.92:0/3027141248 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fd38006af60 0x7fd38006d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:47.204 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.200+0000 7fd3977fe640 1 --2- 172.21.15.92:0/3027141248 >> [v2:172.21.15.118:6816/3425937229,v1:172.21.15.118:6819/3425937229] conn(0x7fd380070dc0 0x7fd3800731e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:47.204 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.200+0000 7fd3977fe640 1 -- 172.21.15.92:0/3027141248 --> [v2:172.21.15.118:6816/3425937229,v1:172.21.15.118:6819/3425937229] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd380073890 con 0x7fd380070dc0 2023-11-24T06:13:47.204 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.200+0000 7fd3977fe640 1 -- 172.21.15.92:0/3027141248 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_get_version_reply(handle=1 version=66) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd3880adae0 con 0x7fd3a00cc500 2023-11-24T06:13:47.204 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.201+0000 7fd39ed9d640 1 --2- 172.21.15.92:0/3027141248 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fd38006af60 0x7fd38006d420 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fd39000a830 tx=0x7fd390008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:47.204 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.201+0000 7fd39f59e640 1 --2- 172.21.15.92:0/3027141248 >> [v2:172.21.15.118:6816/3425937229,v1:172.21.15.118:6819/3425937229] conn(0x7fd380070dc0 0x7fd3800731e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:47.205 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.202+0000 7fd39f59e640 1 --2- 172.21.15.92:0/3027141248 >> [v2:172.21.15.118:6816/3425937229,v1:172.21.15.118:6819/3425937229] conn(0x7fd380070dc0 0x7fd3800731e0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:47.206 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.204+0000 7fd3977fe640 1 -- 172.21.15.92:0/3027141248 <== osd.4 v2:172.21.15.118:6816/3425937229 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7fd380073890 con 0x7fd380070dc0 2023-11-24T06:13:47.220 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.218+0000 7fd3a4967640 1 -- 172.21.15.92:0/3027141248 --> [v2:172.21.15.118:6816/3425937229,v1:172.21.15.118:6819/3425937229] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fd3a006c960 con 0x7fd380070dc0 2023-11-24T06:13:47.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.219+0000 7fd3977fe640 1 -- 172.21.15.92:0/3027141248 <== osd.4 v2:172.21.15.118:6816/3425937229 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fd398009ad0 con 0x7fd380070dc0 2023-11-24T06:13:47.222 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-24T06:13:47.222 INFO:teuthology.orchestra.run.smithi092.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2023-11-24T06:13:47.222 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-24T06:13:47.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.220+0000 7fd3a4967640 1 -- 172.21.15.92:0/3027141248 >> [v2:172.21.15.118:6816/3425937229,v1:172.21.15.118:6819/3425937229] conn(0x7fd380070dc0 msgr2=0x7fd3800731e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:47.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.220+0000 7fd3a4967640 1 --2- 172.21.15.92:0/3027141248 >> [v2:172.21.15.118:6816/3425937229,v1:172.21.15.118:6819/3425937229] conn(0x7fd380070dc0 0x7fd3800731e0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:47.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.220+0000 7fd3a4967640 1 -- 172.21.15.92:0/3027141248 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fd38006af60 msgr2=0x7fd38006d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:47.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.220+0000 7fd3a4967640 1 --2- 172.21.15.92:0/3027141248 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fd38006af60 0x7fd38006d420 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fd39000a830 tx=0x7fd390008040 comp rx=0 tx=0).stop 2023-11-24T06:13:47.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.220+0000 7fd3a4967640 1 -- 172.21.15.92:0/3027141248 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd3a00cc500 msgr2=0x7fd3a00ca420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:47.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.220+0000 7fd3a4967640 1 --2- 172.21.15.92:0/3027141248 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd3a00cc500 0x7fd3a00ca420 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7fd388000c00 tx=0x7fd3880046c0 comp rx=0 tx=0).stop 2023-11-24T06:13:47.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.220+0000 7fd39f59e640 1 -- 172.21.15.92:0/3027141248 reap_dead start 2023-11-24T06:13:47.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.220+0000 7fd3a4967640 1 -- 172.21.15.92:0/3027141248 shutdown_connections 2023-11-24T06:13:47.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.220+0000 7fd3a4967640 1 -- 172.21.15.92:0/3027141248 >> 172.21.15.92:0/3027141248 conn(0x7fd3a006a800 msgr2=0x7fd3a006abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:13:47.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.220+0000 7fd3a4967640 1 -- 172.21.15.92:0/3027141248 shutdown_connections 2023-11-24T06:13:47.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.220+0000 7fd3a4967640 1 -- 172.21.15.92:0/3027141248 wait complete. 2023-11-24T06:13:47.239 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2023-11-24T06:13:47.255 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T06:13:47.291 DEBUG:teuthology.orchestra.run.smithi118:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T06:13:47.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.444+0000 7f26bf59e640 1 Processor -- start 2023-11-24T06:13:47.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.444+0000 7f26bf59e640 1 -- start start 2023-11-24T06:13:47.448 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.446+0000 7f26bf59e640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f26c01447f0 0x7f26c015c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:47.448 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.446+0000 7f26bf59e640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f26c006c470 0x7f26c00cc5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:47.448 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.446+0000 7f26bf59e640 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26c015c5c0 con 0x7f26c006dab0 2023-11-24T06:13:47.448 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.446+0000 7f26bf59e640 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26c006d690 con 0x7f26c01447f0 2023-11-24T06:13:47.448 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.446+0000 7f26bf59e640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26c00cebb0 con 0x7f26c006c470 2023-11-24T06:13:47.449 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.446+0000 7f26b5d9b640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f26c006c470 0x7f26c00cc5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:47.449 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.446+0000 7f26bed9d640 1 --1- >> v1:172.21.15.118:6789/0 conn(0x7f26c006dab0 0x7f26c015c7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.118:6789/0 says I am v1:172.21.15.92:35748/0 (socket says 172.21.15.92:35748) 2023-11-24T06:13:47.450 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.446+0000 7f26bed9d640 1 -- 172.21.15.92:0/4275637190 learned_addr learned my addr 172.21.15.92:0/4275637190 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-24T06:13:47.450 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.446+0000 7f26be59c640 1 --2- 172.21.15.92:0/4275637190 >> v2:172.21.15.118:3300/0 conn(0x7f26c01447f0 0x7f26c015c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:47.450 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.447+0000 7f26bdd9b640 1 -- 172.21.15.92:0/4275637190 <== mon.1 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 690869001 0 0) 0x7f26c015c5c0 con 0x7f26c006dab0 2023-11-24T06:13:47.450 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.447+0000 7f26bdd9b640 1 -- 172.21.15.92:0/4275637190 --> v1:172.21.15.118:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f26ac003520 con 0x7f26c006dab0 2023-11-24T06:13:47.450 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.447+0000 7f26be59c640 1 -- 172.21.15.92:0/4275637190 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f26c006c470 msgr2=0x7f26c00cc5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:13:47.450 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.447+0000 7f26be59c640 1 --2- 172.21.15.92:0/4275637190 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f26c006c470 0x7f26c00cc5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:47.450 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.447+0000 7f26be59c640 1 -- 172.21.15.92:0/4275637190 >> v1:172.21.15.118:6789/0 conn(0x7f26c006dab0 legacy=0x7f26c015c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:47.451 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.447+0000 7f26be59c640 1 -- 172.21.15.92:0/4275637190 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26c006ca10 con 0x7f26c01447f0 2023-11-24T06:13:47.451 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.447+0000 7f26b5d9b640 1 --2- 172.21.15.92:0/4275637190 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f26c006c470 0x7f26c00cc5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:13:47.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.449+0000 7f26be59c640 1 --2- 172.21.15.92:0/4275637190 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f26c01447f0 0x7f26c015c080 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f26b00097a0 tx=0x7f26b002b820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d2c7acdc1b2ff44a server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:47.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.449+0000 7f26bdd9b640 1 -- 172.21.15.92:0/4275637190 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26b0031780 con 0x7f26c01447f0 2023-11-24T06:13:47.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.449+0000 7f26bdd9b640 1 -- 172.21.15.92:0/4275637190 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26b00318e0 con 0x7f26c01447f0 2023-11-24T06:13:47.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.449+0000 7f26bdd9b640 1 -- 172.21.15.92:0/4275637190 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26b0031c50 con 0x7f26c01447f0 2023-11-24T06:13:47.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.450+0000 7f26bf59e640 1 -- 172.21.15.92:0/4275637190 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f26c01447f0 msgr2=0x7f26c015c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:47.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.450+0000 7f26bf59e640 1 --2- 172.21.15.92:0/4275637190 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f26c01447f0 0x7f26c015c080 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f26b00097a0 tx=0x7f26b002b820 comp rx=0 tx=0).stop 2023-11-24T06:13:47.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.450+0000 7f26bf59e640 1 -- 172.21.15.92:0/4275637190 shutdown_connections 2023-11-24T06:13:47.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.450+0000 7f26bf59e640 1 --2- 172.21.15.92:0/4275637190 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f26c006c470 0x7f26c00cc5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:47.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.450+0000 7f26bf59e640 1 --2- 172.21.15.92:0/4275637190 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f26c01447f0 0x7f26c015c080 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:47.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.450+0000 7f26bf59e640 1 -- 172.21.15.92:0/4275637190 >> 172.21.15.92:0/4275637190 conn(0x7f26c006a3e0 msgr2=0x7f26c006a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:13:47.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.450+0000 7f26bf59e640 1 -- 172.21.15.92:0/4275637190 shutdown_connections 2023-11-24T06:13:47.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.450+0000 7f26bf59e640 1 -- 172.21.15.92:0/4275637190 wait complete. 2023-11-24T06:13:47.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.452+0000 7f26bf59e640 1 Processor -- start 2023-11-24T06:13:47.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.452+0000 7f26bf59e640 1 -- start start 2023-11-24T06:13:47.456 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.453+0000 7f26bf59e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f26c006c470 0x7f26c01343e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:47.456 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.453+0000 7f26bf59e640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f26c006dab0 0x7f26c00c5310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:47.456 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.453+0000 7f26bf59e640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f26c01447f0 0x7f26c00cbb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:47.456 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.453+0000 7f26bf59e640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26c00c4ef0 con 0x7f26c006c470 2023-11-24T06:13:47.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.453+0000 7f26bf59e640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26c015b1a0 con 0x7f26c006dab0 2023-11-24T06:13:47.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.453+0000 7f26bf59e640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26c0134920 con 0x7f26c01447f0 2023-11-24T06:13:47.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.454+0000 7f26be59c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f26c006c470 0x7f26c01343e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:47.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.454+0000 7f26be59c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f26c006c470 0x7f26c01343e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47976/0 (socket says 172.21.15.92:47976) 2023-11-24T06:13:47.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.454+0000 7f26be59c640 1 -- 172.21.15.92:0/2635731146 learned_addr learned my addr 172.21.15.92:0/2635731146 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:13:47.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.454+0000 7f26b5d9b640 1 --2- 172.21.15.92:0/2635731146 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f26c006dab0 0x7f26c00c5310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:47.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.454+0000 7f26be59c640 1 -- 172.21.15.92:0/2635731146 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f26c01447f0 msgr2=0x7f26c00cbb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:47.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.454+0000 7f26be59c640 1 --2- 172.21.15.92:0/2635731146 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f26c01447f0 0x7f26c00cbb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:47.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.454+0000 7f26be59c640 1 -- 172.21.15.92:0/2635731146 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f26c006dab0 msgr2=0x7f26c00c5310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:47.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.454+0000 7f26be59c640 1 --2- 172.21.15.92:0/2635731146 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f26c006dab0 0x7f26c00c5310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:47.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.454+0000 7f26be59c640 1 -- 172.21.15.92:0/2635731146 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26a8009400 con 0x7f26c006c470 2023-11-24T06:13:47.459 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.455+0000 7f26be59c640 1 --2- 172.21.15.92:0/2635731146 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f26c006c470 0x7f26c01343e0 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f26b00098c0 tx=0x7f26b00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:47.459 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.455+0000 7f26b77fe640 1 -- 172.21.15.92:0/2635731146 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26b0032820 con 0x7f26c006c470 2023-11-24T06:13:47.459 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.455+0000 7f26b77fe640 1 -- 172.21.15.92:0/2635731146 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26b0032980 con 0x7f26c006c470 2023-11-24T06:13:47.459 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.455+0000 7f26b77fe640 1 -- 172.21.15.92:0/2635731146 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26b0032cf0 con 0x7f26c006c470 2023-11-24T06:13:47.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.455+0000 7f26bf59e640 1 -- 172.21.15.92:0/2635731146 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26b0009400 con 0x7f26c006c470 2023-11-24T06:13:47.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.455+0000 7f26bf59e640 1 -- 172.21.15.92:0/2635731146 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26c00c4550 con 0x7f26c006c470 2023-11-24T06:13:47.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.456+0000 7f26bf59e640 1 -- 172.21.15.92:0/2635731146 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26c0067450 con 0x7f26c006c470 2023-11-24T06:13:47.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.457+0000 7f26b77fe640 1 -- 172.21.15.92:0/2635731146 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 7) v1 ==== 89333+0+0 (secure 0 0 0) 0x7f26b004e440 con 0x7f26c006c470 2023-11-24T06:13:47.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.458+0000 7f26b77fe640 1 --2- 172.21.15.92:0/2635731146 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f26a006af60 0x7f26a006d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:47.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.458+0000 7f26b77fe640 1 -- 172.21.15.92:0/2635731146 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f26b00ab610 con 0x7f26c006c470 2023-11-24T06:13:47.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.458+0000 7f26b5d9b640 1 --2- 172.21.15.92:0/2635731146 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f26a006af60 0x7f26a006d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:47.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.459+0000 7f26b5d9b640 1 --2- 172.21.15.92:0/2635731146 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f26a006af60 0x7f26a006d420 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f26c0067e60 tx=0x7f26a8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:47.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.463+0000 7f26b77fe640 1 -- 172.21.15.92:0/2635731146 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f26b0076700 con 0x7f26c006c470 2023-11-24T06:13:47.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.651+0000 7f26bf59e640 1 -- 172.21.15.92:0/2635731146 --> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7f26c006d2b0 con 0x7f26a006af60 2023-11-24T06:13:47.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.652+0000 7f26b77fe640 1 -- 172.21.15.92:0/2635731146 <== mgr.4101 v2:172.21.15.92:6800/784027401 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f26a8010930 con 0x7f26a006af60 2023-11-24T06:13:47.654 INFO:teuthology.orchestra.run.smithi092.stderr:instructed osd(s) 4 to deep-scrub 2023-11-24T06:13:47.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.654+0000 7f26bf59e640 1 -- 172.21.15.92:0/2635731146 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f26a006af60 msgr2=0x7f26a006d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:47.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.654+0000 7f26bf59e640 1 --2- 172.21.15.92:0/2635731146 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f26a006af60 0x7f26a006d420 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f26c0067e60 tx=0x7f26a8008040 comp rx=0 tx=0).stop 2023-11-24T06:13:47.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.654+0000 7f26bf59e640 1 -- 172.21.15.92:0/2635731146 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f26c006c470 msgr2=0x7f26c01343e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:47.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.654+0000 7f26bf59e640 1 --2- 172.21.15.92:0/2635731146 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f26c006c470 0x7f26c01343e0 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f26b00098c0 tx=0x7f26b00046c0 comp rx=0 tx=0).stop 2023-11-24T06:13:47.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.655+0000 7f26bf59e640 1 -- 172.21.15.92:0/2635731146 shutdown_connections 2023-11-24T06:13:47.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.655+0000 7f26bf59e640 1 --2- 172.21.15.92:0/2635731146 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f26a006af60 0x7f26a006d420 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:47.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.655+0000 7f26bf59e640 1 --2- 172.21.15.92:0/2635731146 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f26c01447f0 0x7f26c00cbb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:47.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.655+0000 7f26bf59e640 1 --2- 172.21.15.92:0/2635731146 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f26c006dab0 0x7f26c00c5310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:47.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.655+0000 7f26bf59e640 1 --2- 172.21.15.92:0/2635731146 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f26c006c470 0x7f26c01343e0 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:47.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.655+0000 7f26bf59e640 1 -- 172.21.15.92:0/2635731146 >> 172.21.15.92:0/2635731146 conn(0x7f26c006a3e0 msgr2=0x7f26c015dfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:13:47.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.655+0000 7f26bf59e640 1 -- 172.21.15.92:0/2635731146 shutdown_connections 2023-11-24T06:13:47.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.655+0000 7f26bf59e640 1 -- 172.21.15.92:0/2635731146 wait complete. 2023-11-24T06:13:47.676 INFO:tasks.ceph:Scrubbing osd.5 2023-11-24T06:13:47.676 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2023-11-24T06:13:47.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.893+0000 7eff73167640 1 Processor -- start 2023-11-24T06:13:47.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.893+0000 7eff73167640 1 -- start start 2023-11-24T06:13:47.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.895+0000 7eff73167640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7eff6c15afb0 0x7eff6c13f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:47.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.895+0000 7eff73167640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7eff6c13fae0 0x7eff6c0b2040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:47.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.895+0000 7eff73167640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff6c13ff30 con 0x7eff6c13cfa0 2023-11-24T06:13:47.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.895+0000 7eff73167640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff6c0cdc40 con 0x7eff6c13fae0 2023-11-24T06:13:47.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.895+0000 7eff73167640 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff6c0cdec0 con 0x7eff6c15afb0 2023-11-24T06:13:47.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.895+0000 7eff72165640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7eff6c13cfa0 0x7eff6c0cd530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:50404/0 (socket says 172.21.15.92:50404) 2023-11-24T06:13:47.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.895+0000 7eff72165640 1 -- 172.21.15.92:0/3847238946 learned_addr learned my addr 172.21.15.92:0/3847238946 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-24T06:13:47.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.895+0000 7eff72966640 1 --2- 172.21.15.92:0/3847238946 >> v2:172.21.15.92:3300/0 conn(0x7eff6c13fae0 0x7eff6c0b2040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:47.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.895+0000 7eff71964640 1 --2- 172.21.15.92:0/3847238946 >> v2:172.21.15.118:3300/0 conn(0x7eff6c15afb0 0x7eff6c13f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:47.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.896+0000 7eff72966640 1 -- 172.21.15.92:0/3847238946 >> v2:172.21.15.118:3300/0 conn(0x7eff6c15afb0 msgr2=0x7eff6c13f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:13:47.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.896+0000 7eff72966640 1 --2- 172.21.15.92:0/3847238946 >> v2:172.21.15.118:3300/0 conn(0x7eff6c15afb0 0x7eff6c13f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:47.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.896+0000 7eff72966640 1 -- 172.21.15.92:0/3847238946 >> v1:172.21.15.92:6789/0 conn(0x7eff6c13cfa0 legacy=0x7eff6c0cd530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:47.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.896+0000 7eff72966640 1 -- 172.21.15.92:0/3847238946 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff6c0ce1a0 con 0x7eff6c13fae0 2023-11-24T06:13:47.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.896+0000 7eff71964640 1 --2- 172.21.15.92:0/3847238946 >> v2:172.21.15.118:3300/0 conn(0x7eff6c15afb0 0x7eff6c13f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:13:47.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.897+0000 7eff72966640 1 --2- 172.21.15.92:0/3847238946 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7eff6c13fae0 0x7eff6c0b2040 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7eff6c0b3fd0 tx=0x7eff6402e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a57378fb6a4c621a server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:47.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.897+0000 7eff71163640 1 -- 172.21.15.92:0/3847238946 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff6402edf0 con 0x7eff6c13fae0 2023-11-24T06:13:47.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.897+0000 7eff71163640 1 -- 172.21.15.92:0/3847238946 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff64036440 con 0x7eff6c13fae0 2023-11-24T06:13:47.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.897+0000 7eff71163640 1 -- 172.21.15.92:0/3847238946 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff640367d0 con 0x7eff6c13fae0 2023-11-24T06:13:47.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.897+0000 7eff73167640 1 -- 172.21.15.92:0/3847238946 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7eff6c13fae0 msgr2=0x7eff6c0b2040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:47.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.897+0000 7eff73167640 1 --2- 172.21.15.92:0/3847238946 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7eff6c13fae0 0x7eff6c0b2040 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7eff6c0b3fd0 tx=0x7eff6402e040 comp rx=0 tx=0).stop 2023-11-24T06:13:47.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.898+0000 7eff73167640 1 -- 172.21.15.92:0/3847238946 shutdown_connections 2023-11-24T06:13:47.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.898+0000 7eff73167640 1 --2- 172.21.15.92:0/3847238946 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7eff6c13fae0 0x7eff6c0b2040 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:47.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.898+0000 7eff73167640 1 --2- 172.21.15.92:0/3847238946 >> v2:172.21.15.118:3300/0 conn(0x7eff6c15afb0 0x7eff6c13f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:47.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.898+0000 7eff73167640 1 -- 172.21.15.92:0/3847238946 >> 172.21.15.92:0/3847238946 conn(0x7eff6c06a800 msgr2=0x7eff6c06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:13:47.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.898+0000 7eff73167640 1 -- 172.21.15.92:0/3847238946 shutdown_connections 2023-11-24T06:13:47.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.898+0000 7eff73167640 1 -- 172.21.15.92:0/3847238946 wait complete. 2023-11-24T06:13:47.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.899+0000 7eff73167640 1 Processor -- start 2023-11-24T06:13:47.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.900+0000 7eff73167640 1 -- start start 2023-11-24T06:13:47.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.900+0000 7eff73167640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7eff6c13cfa0 0x7eff6c1537e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:47.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.900+0000 7eff73167640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7eff6c13fae0 0x7eff6c153d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:47.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.900+0000 7eff73167640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7eff6c15afb0 0x7eff6c147990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:47.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.900+0000 7eff73167640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff6c1457a0 con 0x7eff6c15afb0 2023-11-24T06:13:47.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.900+0000 7eff73167640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff6c13ad00 con 0x7eff6c13fae0 2023-11-24T06:13:47.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.900+0000 7eff73167640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff6c145a40 con 0x7eff6c13cfa0 2023-11-24T06:13:47.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.901+0000 7eff72966640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7eff6c15afb0 0x7eff6c147990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:47.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.901+0000 7eff72966640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7eff6c15afb0 0x7eff6c147990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47994/0 (socket says 172.21.15.92:47994) 2023-11-24T06:13:47.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.901+0000 7eff72966640 1 -- 172.21.15.92:0/2285966438 learned_addr learned my addr 172.21.15.92:0/2285966438 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:13:47.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.901+0000 7eff72966640 1 -- 172.21.15.92:0/2285966438 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7eff6c13cfa0 msgr2=0x7eff6c1537e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:47.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.901+0000 7eff72966640 1 --2- 172.21.15.92:0/2285966438 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7eff6c13cfa0 0x7eff6c1537e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:47.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.901+0000 7eff72966640 1 -- 172.21.15.92:0/2285966438 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7eff6c13fae0 msgr2=0x7eff6c153d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:47.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.901+0000 7eff71964640 1 --2- 172.21.15.92:0/2285966438 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7eff6c13fae0 0x7eff6c153d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:47.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.901+0000 7eff72966640 1 --2- 172.21.15.92:0/2285966438 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7eff6c13fae0 0x7eff6c153d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:47.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.901+0000 7eff72966640 1 -- 172.21.15.92:0/2285966438 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff64009a70 con 0x7eff6c15afb0 2023-11-24T06:13:47.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.902+0000 7eff72966640 1 --2- 172.21.15.92:0/2285966438 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7eff6c15afb0 0x7eff6c147990 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7eff64009e80 tx=0x7eff64004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:47.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.902+0000 7eff62ffd640 1 -- 172.21.15.92:0/2285966438 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff6403f850 con 0x7eff6c15afb0 2023-11-24T06:13:47.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.902+0000 7eff62ffd640 1 -- 172.21.15.92:0/2285966438 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff6403f9d0 con 0x7eff6c15afb0 2023-11-24T06:13:47.910 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.902+0000 7eff62ffd640 1 -- 172.21.15.92:0/2285966438 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff6403fd80 con 0x7eff6c15afb0 2023-11-24T06:13:47.910 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.902+0000 7eff73167640 1 -- 172.21.15.92:0/2285966438 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff6c145d20 con 0x7eff6c15afb0 2023-11-24T06:13:47.910 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.902+0000 7eff73167640 1 -- 172.21.15.92:0/2285966438 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff6c13a9a0 con 0x7eff6c15afb0 2023-11-24T06:13:47.911 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.903+0000 7eff73167640 1 -- 172.21.15.92:0/2285966438 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7eff6c0cea40 con 0x7eff6c15afb0 2023-11-24T06:13:47.912 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.904+0000 7eff62ffd640 1 -- 172.21.15.92:0/2285966438 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 7) v1 ==== 89333+0+0 (secure 0 0 0) 0x7eff6403d070 con 0x7eff6c15afb0 2023-11-24T06:13:47.912 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.905+0000 7eff62ffd640 1 --2- 172.21.15.92:0/2285966438 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7eff5006af60 0x7eff5006d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:47.912 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.905+0000 7eff62ffd640 1 -- 172.21.15.92:0/2285966438 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 4943+0+0 (secure 0 0 0) 0x7eff640abe10 con 0x7eff6c15afb0 2023-11-24T06:13:47.912 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.905+0000 7eff62ffd640 1 --2- 172.21.15.92:0/2285966438 >> [v2:172.21.15.118:6800/827455984,v1:172.21.15.118:6801/827455984] conn(0x7eff50070dc0 0x7eff500731e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:47.912 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.905+0000 7eff62ffd640 1 -- 172.21.15.92:0/2285966438 --> [v2:172.21.15.118:6800/827455984,v1:172.21.15.118:6801/827455984] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7eff50073890 con 0x7eff50070dc0 2023-11-24T06:13:47.912 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.905+0000 7eff62ffd640 1 -- 172.21.15.92:0/2285966438 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_get_version_reply(handle=1 version=66) v2 ==== 24+0+0 (secure 0 0 0) 0x7eff640ae470 con 0x7eff6c15afb0 2023-11-24T06:13:47.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.905+0000 7eff72165640 1 --2- 172.21.15.92:0/2285966438 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7eff5006af60 0x7eff5006d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:47.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.906+0000 7eff72165640 1 --2- 172.21.15.92:0/2285966438 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7eff5006af60 0x7eff5006d420 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7eff680015f0 tx=0x7eff68007a40 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:47.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.906+0000 7eff71964640 1 --2- 172.21.15.92:0/2285966438 >> [v2:172.21.15.118:6800/827455984,v1:172.21.15.118:6801/827455984] conn(0x7eff50070dc0 0x7eff500731e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:47.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.907+0000 7eff71964640 1 --2- 172.21.15.92:0/2285966438 >> [v2:172.21.15.118:6800/827455984,v1:172.21.15.118:6801/827455984] conn(0x7eff50070dc0 0x7eff500731e0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:47.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.909+0000 7eff62ffd640 1 -- 172.21.15.92:0/2285966438 <== osd.5 v2:172.21.15.118:6800/827455984 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7eff50073890 con 0x7eff50070dc0 2023-11-24T06:13:47.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.923+0000 7eff73167640 1 -- 172.21.15.92:0/2285966438 --> [v2:172.21.15.118:6800/827455984,v1:172.21.15.118:6801/827455984] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7eff6c0ce760 con 0x7eff50070dc0 2023-11-24T06:13:47.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.924+0000 7eff62ffd640 1 -- 172.21.15.92:0/2285966438 <== osd.5 v2:172.21.15.118:6800/827455984 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7eff5c00c040 con 0x7eff50070dc0 2023-11-24T06:13:47.926 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-24T06:13:47.927 INFO:teuthology.orchestra.run.smithi092.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2023-11-24T06:13:47.927 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-24T06:13:47.928 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.925+0000 7eff73167640 1 -- 172.21.15.92:0/2285966438 >> [v2:172.21.15.118:6800/827455984,v1:172.21.15.118:6801/827455984] conn(0x7eff50070dc0 msgr2=0x7eff500731e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:47.928 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.925+0000 7eff73167640 1 --2- 172.21.15.92:0/2285966438 >> [v2:172.21.15.118:6800/827455984,v1:172.21.15.118:6801/827455984] conn(0x7eff50070dc0 0x7eff500731e0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:47.928 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.925+0000 7eff73167640 1 -- 172.21.15.92:0/2285966438 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7eff5006af60 msgr2=0x7eff5006d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:47.928 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.925+0000 7eff73167640 1 --2- 172.21.15.92:0/2285966438 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7eff5006af60 0x7eff5006d420 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7eff680015f0 tx=0x7eff68007a40 comp rx=0 tx=0).stop 2023-11-24T06:13:47.928 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.925+0000 7eff73167640 1 -- 172.21.15.92:0/2285966438 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7eff6c15afb0 msgr2=0x7eff6c147990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:47.928 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.925+0000 7eff73167640 1 --2- 172.21.15.92:0/2285966438 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7eff6c15afb0 0x7eff6c147990 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7eff64009e80 tx=0x7eff64004a40 comp rx=0 tx=0).stop 2023-11-24T06:13:47.928 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.925+0000 7eff72966640 1 -- 172.21.15.92:0/2285966438 reap_dead start 2023-11-24T06:13:47.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.925+0000 7eff73167640 1 -- 172.21.15.92:0/2285966438 shutdown_connections 2023-11-24T06:13:47.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.925+0000 7eff73167640 1 -- 172.21.15.92:0/2285966438 >> 172.21.15.92:0/2285966438 conn(0x7eff6c06a800 msgr2=0x7eff6c0b1230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:13:47.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.925+0000 7eff73167640 1 -- 172.21.15.92:0/2285966438 shutdown_connections 2023-11-24T06:13:47.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:47.925+0000 7eff73167640 1 -- 172.21.15.92:0/2285966438 wait complete. 2023-11-24T06:13:47.944 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2023-11-24T06:13:48.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:48.146+0000 7f1ccb2d1640 1 Processor -- start 2023-11-24T06:13:48.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:48.146+0000 7f1ccb2d1640 1 -- start start 2023-11-24T06:13:48.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:48.147+0000 7f1ccb2d1640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f1cc41447f0 0x7f1cc415c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:48.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:48.148+0000 7f1ccb2d1640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f1cc40cccf0 0x7f1cc415c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:48.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:48.148+0000 7f1ccb2d1640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1cc415c5c0 con 0x7f1cc406c470 2023-11-24T06:13:48.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:48.148+0000 7f1ccb2d1640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1cc406d690 con 0x7f1cc40cccf0 2023-11-24T06:13:48.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:48.148+0000 7f1ccb2d1640 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1cc40cebb0 con 0x7f1cc41447f0 2023-11-24T06:13:48.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:48.148+0000 7f1cc9ace640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f1cc406c470 0x7f1cc40cc5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:50418/0 (socket says 172.21.15.92:50418) 2023-11-24T06:13:48.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:48.148+0000 7f1cc9ace640 1 -- 172.21.15.92:0/993166530 learned_addr learned my addr 172.21.15.92:0/993166530 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-24T06:13:48.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:48.148+0000 7f1ccaad0640 1 --2- 172.21.15.92:0/993166530 >> v2:172.21.15.92:3300/0 conn(0x7f1cc40cccf0 0x7f1cc415c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:48.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:48.148+0000 7f1cca2cf640 1 --2- 172.21.15.92:0/993166530 >> v2:172.21.15.118:3300/0 conn(0x7f1cc41447f0 0x7f1cc415c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:48.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:48.148+0000 7f1ccaad0640 1 -- 172.21.15.92:0/993166530 >> v2:172.21.15.118:3300/0 conn(0x7f1cc41447f0 msgr2=0x7f1cc415c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:13:48.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:48.148+0000 7f1ccaad0640 1 --2- 172.21.15.92:0/993166530 >> v2:172.21.15.118:3300/0 conn(0x7f1cc41447f0 0x7f1cc415c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:48.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:48.148+0000 7f1ccaad0640 1 -- 172.21.15.92:0/993166530 >> v1:172.21.15.92:6789/0 conn(0x7f1cc406c470 legacy=0x7f1cc40cc5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:48.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:48.148+0000 7f1ccaad0640 1 -- 172.21.15.92:0/993166530 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cc406ca10 con 0x7f1cc40cccf0 2023-11-24T06:13:48.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:48.148+0000 7f1cc92cd640 1 -- 172.21.15.92:0/993166530 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3857009750 0 0) 0x7f1cc415c5c0 con 0x7f1cc406c470 2023-11-24T06:13:48.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:48.150+0000 7f1ccaad0640 1 --2- 172.21.15.92:0/993166530 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1cc40cccf0 0x7f1cc415c7c0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f1cc4072520 tx=0x7f1cbc02be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c78e6e0f91a3bac server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:48.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:48.150+0000 7f1cc92cd640 1 -- 172.21.15.92:0/993166530 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cbc02cbf0 con 0x7f1cc40cccf0 2023-11-24T06:13:48.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:48.150+0000 7f1cc92cd640 1 -- 172.21.15.92:0/993166530 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cbc02cd50 con 0x7f1cc40cccf0 2023-11-24T06:13:48.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:48.150+0000 7f1cc92cd640 1 -- 172.21.15.92:0/993166530 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cbc034660 con 0x7f1cc40cccf0 2023-11-24T06:13:48.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:48.150+0000 7f1ccb2d1640 1 -- 172.21.15.92:0/993166530 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1cc40cccf0 msgr2=0x7f1cc415c7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:48.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:48.150+0000 7f1ccb2d1640 1 --2- 172.21.15.92:0/993166530 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1cc40cccf0 0x7f1cc415c7c0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f1cc4072520 tx=0x7f1cbc02be40 comp rx=0 tx=0).stop 2023-11-24T06:13:48.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:48.151+0000 7f1ccb2d1640 1 -- 172.21.15.92:0/993166530 shutdown_connections 2023-11-24T06:13:48.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:48.151+0000 7f1ccb2d1640 1 --2- 172.21.15.92:0/993166530 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1cc40cccf0 0x7f1cc415c7c0 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:48.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:48.151+0000 7f1ccb2d1640 1 --2- 172.21.15.92:0/993166530 >> v2:172.21.15.118:3300/0 conn(0x7f1cc41447f0 0x7f1cc415c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:48.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:48.151+0000 7f1ccb2d1640 1 -- 172.21.15.92:0/993166530 >> 172.21.15.92:0/993166530 conn(0x7f1cc406a3e0 msgr2=0x7f1cc406a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:13:48.155 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:48.151+0000 7f1ccb2d1640 1 -- 172.21.15.92:0/993166530 shutdown_connections 2023-11-24T06:13:48.155 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:48.151+0000 7f1ccb2d1640 1 -- 172.21.15.92:0/993166530 wait complete. 2023-11-24T06:13:48.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:48.152+0000 7f1ccb2d1640 1 Processor -- start 2023-11-24T06:13:48.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:48.153+0000 7f1ccb2d1640 1 -- start start 2023-11-24T06:13:48.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:48.154+0000 7f1ccb2d1640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1cc406c470 0x7f1cc4138740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:48.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:48.154+0000 7f1ccb2d1640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1cc40cccf0 0x7f1cc4139a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:48.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:48.154+0000 7f1ccb2d1640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1cc41447f0 0x7f1cc415c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:48.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:48.154+0000 7f1ccb2d1640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cc4139650 con 0x7f1cc40cccf0 2023-11-24T06:13:48.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:48.154+0000 7f1ccb2d1640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cc413f640 con 0x7f1cc406c470 2023-11-24T06:13:48.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:48.154+0000 7f1ccb2d1640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cc40b1520 con 0x7f1cc41447f0 2023-11-24T06:13:48.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:48.154+0000 7f1cc9ace640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1cc40cccf0 0x7f1cc4139a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:48.158 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:48.154+0000 7f1cc9ace640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1cc40cccf0 0x7f1cc4139a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48016/0 (socket says 172.21.15.92:48016) 2023-11-24T06:13:48.158 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:48.154+0000 7f1cc9ace640 1 -- 172.21.15.92:0/677528135 learned_addr learned my addr 172.21.15.92:0/677528135 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:13:48.158 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:48.154+0000 7f1cca2cf640 1 --2- 172.21.15.92:0/677528135 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1cc406c470 0x7f1cc4138740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:48.158 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:48.154+0000 7f1ccaad0640 1 --2- 172.21.15.92:0/677528135 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1cc41447f0 0x7f1cc415c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:48.158 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:48.154+0000 7f1cc9ace640 1 -- 172.21.15.92:0/677528135 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1cc41447f0 msgr2=0x7f1cc415c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:48.158 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:48.154+0000 7f1cc9ace640 1 --2- 172.21.15.92:0/677528135 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1cc41447f0 0x7f1cc415c1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:48.161 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:48.154+0000 7f1cc9ace640 1 -- 172.21.15.92:0/677528135 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1cc406c470 msgr2=0x7f1cc4138740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:48.161 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:48.154+0000 7f1cc9ace640 1 --2- 172.21.15.92:0/677528135 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1cc406c470 0x7f1cc4138740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:48.161 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:48.154+0000 7f1cc9ace640 1 -- 172.21.15.92:0/677528135 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cbc009a70 con 0x7f1cc40cccf0 2023-11-24T06:13:48.161 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:48.155+0000 7f1cc9ace640 1 --2- 172.21.15.92:0/677528135 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1cc40cccf0 0x7f1cc4139a70 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f1cb400c500 tx=0x7f1cb400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:48.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:48.155+0000 7f1cbaffd640 1 -- 172.21.15.92:0/677528135 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cb4007560 con 0x7f1cc40cccf0 2023-11-24T06:13:48.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:48.155+0000 7f1cbaffd640 1 -- 172.21.15.92:0/677528135 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cb40047e0 con 0x7f1cc40cccf0 2023-11-24T06:13:48.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:48.155+0000 7f1cbaffd640 1 -- 172.21.15.92:0/677528135 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cb4004b90 con 0x7f1cc40cccf0 2023-11-24T06:13:48.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:48.155+0000 7f1ccb2d1640 1 -- 172.21.15.92:0/677528135 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1cc40b1800 con 0x7f1cc40cccf0 2023-11-24T06:13:48.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:48.155+0000 7f1ccb2d1640 1 -- 172.21.15.92:0/677528135 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cc4139270 con 0x7f1cc40cccf0 2023-11-24T06:13:48.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:48.156+0000 7f1ccb2d1640 1 -- 172.21.15.92:0/677528135 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1cc4139790 con 0x7f1cc40cccf0 2023-11-24T06:13:48.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:48.157+0000 7f1cbaffd640 1 -- 172.21.15.92:0/677528135 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 7) v1 ==== 89333+0+0 (secure 0 0 0) 0x7f1cb4004cf0 con 0x7f1cc40cccf0 2023-11-24T06:13:48.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:48.158+0000 7f1cbaffd640 1 --2- 172.21.15.92:0/677528135 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f1ca806af60 0x7f1ca806d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:48.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:48.158+0000 7f1cbaffd640 1 -- 172.21.15.92:0/677528135 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f1cb40885f0 con 0x7f1cc40cccf0 2023-11-24T06:13:48.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:48.158+0000 7f1cca2cf640 1 --2- 172.21.15.92:0/677528135 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f1ca806af60 0x7f1ca806d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:48.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:48.159+0000 7f1cca2cf640 1 --2- 172.21.15.92:0/677528135 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f1ca806af60 0x7f1ca806d420 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f1cc0006fd0 tx=0x7f1cc0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:48.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:48.162+0000 7f1cbaffd640 1 -- 172.21.15.92:0/677528135 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1cc4139790 con 0x7f1cc40cccf0 2023-11-24T06:13:48.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:48.354+0000 7f1ccb2d1640 1 -- 172.21.15.92:0/677528135 --> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) v1 -- 0x7f1cc406d2b0 con 0x7f1ca806af60 2023-11-24T06:13:48.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:48.355+0000 7f1cbaffd640 1 -- 172.21.15.92:0/677528135 <== mgr.4101 v2:172.21.15.92:6800/784027401 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f1cc000e4b0 con 0x7f1ca806af60 2023-11-24T06:13:48.358 INFO:teuthology.orchestra.run.smithi092.stderr:instructed osd(s) 5 to deep-scrub 2023-11-24T06:13:48.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:48.357+0000 7f1ccb2d1640 1 -- 172.21.15.92:0/677528135 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f1ca806af60 msgr2=0x7f1ca806d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:48.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:48.357+0000 7f1ccb2d1640 1 --2- 172.21.15.92:0/677528135 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f1ca806af60 0x7f1ca806d420 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f1cc0006fd0 tx=0x7f1cc0008040 comp rx=0 tx=0).stop 2023-11-24T06:13:48.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:48.357+0000 7f1ccb2d1640 1 -- 172.21.15.92:0/677528135 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1cc40cccf0 msgr2=0x7f1cc4139a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:48.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:48.357+0000 7f1ccb2d1640 1 --2- 172.21.15.92:0/677528135 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1cc40cccf0 0x7f1cc4139a70 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f1cb400c500 tx=0x7f1cb400c9c0 comp rx=0 tx=0).stop 2023-11-24T06:13:48.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:48.358+0000 7f1ccb2d1640 1 -- 172.21.15.92:0/677528135 shutdown_connections 2023-11-24T06:13:48.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:48.358+0000 7f1ccb2d1640 1 --2- 172.21.15.92:0/677528135 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f1ca806af60 0x7f1ca806d420 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:48.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:48.358+0000 7f1ccb2d1640 1 --2- 172.21.15.92:0/677528135 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f1cc41447f0 0x7f1cc415c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:48.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:48.358+0000 7f1ccb2d1640 1 --2- 172.21.15.92:0/677528135 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1cc40cccf0 0x7f1cc4139a70 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:48.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:48.358+0000 7f1ccb2d1640 1 --2- 172.21.15.92:0/677528135 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1cc406c470 0x7f1cc4138740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:48.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:48.358+0000 7f1ccb2d1640 1 -- 172.21.15.92:0/677528135 >> 172.21.15.92:0/677528135 conn(0x7f1cc406a3e0 msgr2=0x7f1cc415b670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:13:48.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:48.358+0000 7f1ccb2d1640 1 -- 172.21.15.92:0/677528135 shutdown_connections 2023-11-24T06:13:48.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:48.358+0000 7f1ccb2d1640 1 -- 172.21.15.92:0/677528135 wait complete. 2023-11-24T06:13:48.378 INFO:tasks.ceph:Scrubbing osd.6 2023-11-24T06:13:48.378 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2023-11-24T06:13:48.593 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:48.591+0000 7fd5fc748640 1 Processor -- start 2023-11-24T06:13:48.594 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:48.591+0000 7fd5fc748640 1 -- start start 2023-11-24T06:13:48.595 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:48.593+0000 7fd5fc748640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd5f413df80 0x7fd5f415bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:48.595 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:48.593+0000 7fd5fc748640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fd5f413e360 0x7fd5f40cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:48.595 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:48.593+0000 7fd5fc748640 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5f406c470 con 0x7fd5f415f9c0 2023-11-24T06:13:48.595 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:48.593+0000 7fd5fc748640 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5f406c750 con 0x7fd5f413e360 2023-11-24T06:13:48.596 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:48.593+0000 7fd5fc748640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5f406ca30 con 0x7fd5f413df80 2023-11-24T06:13:48.596 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:48.594+0000 7fd5fa4e4640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd5f413df80 0x7fd5f415bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:48.596 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:48.594+0000 7fd5fa4e4640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd5f413df80 0x7fd5f415bee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3301/0 says I am v2:172.21.15.92:34796/0 (socket says 172.21.15.92:34796) 2023-11-24T06:13:48.596 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:48.594+0000 7fd5fa4e4640 1 -- 172.21.15.92:0/1454480614 learned_addr learned my addr 172.21.15.92:0/1454480614 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:13:48.597 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:48.594+0000 7fd5f9ce3640 1 --2- 172.21.15.92:0/1454480614 >> v2:172.21.15.118:3300/0 conn(0x7fd5f413e360 0x7fd5f40cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:48.597 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:48.595+0000 7fd5fa4e4640 1 -- 172.21.15.92:0/1454480614 >> v2:172.21.15.118:3300/0 conn(0x7fd5f413e360 msgr2=0x7fd5f40cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:13:48.597 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:48.595+0000 7fd5fa4e4640 1 --2- 172.21.15.92:0/1454480614 >> v2:172.21.15.118:3300/0 conn(0x7fd5f413e360 0x7fd5f40cf290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:48.598 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:48.595+0000 7fd5fa4e4640 1 -- 172.21.15.92:0/1454480614 >> v1:172.21.15.118:6789/0 conn(0x7fd5f415f9c0 legacy=0x7fd5f41408d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:48.598 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:48.595+0000 7fd5fa4e4640 1 -- 172.21.15.92:0/1454480614 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5f406cd10 con 0x7fd5f413df80 2023-11-24T06:13:48.598 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:48.596+0000 7fd5fa4e4640 1 --2- 172.21.15.92:0/1454480614 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd5f413df80 0x7fd5f415bee0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fd5f0009870 tx=0x7fd5f002b7e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=46d0391d856bea43 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:48.599 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:48.597+0000 7fd5f94e2640 1 -- 172.21.15.92:0/1454480614 <== mon.2 v2:172.21.15.118:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5f0004230 con 0x7fd5f413df80 2023-11-24T06:13:48.599 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:48.597+0000 7fd5f94e2640 1 -- 172.21.15.92:0/1454480614 <== mon.2 v2:172.21.15.118:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5f002e050 con 0x7fd5f413df80 2023-11-24T06:13:48.599 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:48.597+0000 7fd5f94e2640 1 -- 172.21.15.92:0/1454480614 <== mon.2 v2:172.21.15.118:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5f0003610 con 0x7fd5f413df80 2023-11-24T06:13:48.600 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:48.598+0000 7fd5fc748640 1 -- 172.21.15.92:0/1454480614 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd5f413df80 msgr2=0x7fd5f415bee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:48.600 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:48.598+0000 7fd5fc748640 1 --2- 172.21.15.92:0/1454480614 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd5f413df80 0x7fd5f415bee0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fd5f0009870 tx=0x7fd5f002b7e0 comp rx=0 tx=0).stop 2023-11-24T06:13:48.600 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:48.598+0000 7fd5fc748640 1 -- 172.21.15.92:0/1454480614 shutdown_connections 2023-11-24T06:13:48.601 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:48.598+0000 7fd5fc748640 1 --2- 172.21.15.92:0/1454480614 >> v2:172.21.15.118:3300/0 conn(0x7fd5f413e360 0x7fd5f40cf290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:48.601 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:48.598+0000 7fd5fc748640 1 --2- 172.21.15.92:0/1454480614 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd5f413df80 0x7fd5f415bee0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:48.601 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:48.598+0000 7fd5fc748640 1 -- 172.21.15.92:0/1454480614 >> 172.21.15.92:0/1454480614 conn(0x7fd5f406a800 msgr2=0x7fd5f406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:13:48.601 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:48.598+0000 7fd5fc748640 1 -- 172.21.15.92:0/1454480614 shutdown_connections 2023-11-24T06:13:48.601 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:48.598+0000 7fd5fc748640 1 -- 172.21.15.92:0/1454480614 wait complete. 2023-11-24T06:13:48.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:48.600+0000 7fd5fc748640 1 Processor -- start 2023-11-24T06:13:48.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:48.600+0000 7fd5fc748640 1 -- start start 2023-11-24T06:13:48.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:48.601+0000 7fd5fc748640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd5f413df80 0x7fd5f41f2a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:48.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:48.601+0000 7fd5fc748640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd5f413e360 0x7fd5f414f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:48.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:48.601+0000 7fd5fc748640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd5f415f9c0 0x7fd5f414fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:48.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:48.601+0000 7fd5fc748640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5f414cbe0 con 0x7fd5f413e360 2023-11-24T06:13:48.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:48.601+0000 7fd5fc748640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5f4152af0 con 0x7fd5f415f9c0 2023-11-24T06:13:48.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:48.601+0000 7fd5fc748640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5f4152d90 con 0x7fd5f413df80 2023-11-24T06:13:48.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:48.601+0000 7fd5f9ce3640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd5f413e360 0x7fd5f414f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:48.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:48.601+0000 7fd5f9ce3640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd5f413e360 0x7fd5f414f630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48032/0 (socket says 172.21.15.92:48032) 2023-11-24T06:13:48.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:48.601+0000 7fd5f9ce3640 1 -- 172.21.15.92:0/3653049636 learned_addr learned my addr 172.21.15.92:0/3653049636 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:13:48.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:48.601+0000 7fd5fa4e4640 1 --2- 172.21.15.92:0/3653049636 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd5f413df80 0x7fd5f41f2a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:48.605 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:48.601+0000 7fd5face5640 1 --2- 172.21.15.92:0/3653049636 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd5f415f9c0 0x7fd5f414fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:48.605 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:48.601+0000 7fd5f9ce3640 1 -- 172.21.15.92:0/3653049636 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd5f413df80 msgr2=0x7fd5f41f2a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:48.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:48.601+0000 7fd5f9ce3640 1 --2- 172.21.15.92:0/3653049636 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fd5f413df80 0x7fd5f41f2a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:48.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:48.602+0000 7fd5f9ce3640 1 -- 172.21.15.92:0/3653049636 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd5f415f9c0 msgr2=0x7fd5f414fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:48.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:48.602+0000 7fd5f9ce3640 1 --2- 172.21.15.92:0/3653049636 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd5f415f9c0 0x7fd5f414fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:48.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:48.602+0000 7fd5f9ce3640 1 -- 172.21.15.92:0/3653049636 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5f00094d0 con 0x7fd5f413e360 2023-11-24T06:13:48.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:48.602+0000 7fd5face5640 1 --2- 172.21.15.92:0/3653049636 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd5f415f9c0 0x7fd5f414fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:13:48.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:48.602+0000 7fd5f9ce3640 1 --2- 172.21.15.92:0/3653049636 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd5f413e360 0x7fd5f414f630 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7fd5e400e9b0 tx=0x7fd5e400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:48.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:48.602+0000 7fd5eaffd640 1 -- 172.21.15.92:0/3653049636 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5e4009670 con 0x7fd5f413e360 2023-11-24T06:13:48.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:48.602+0000 7fd5eaffd640 1 -- 172.21.15.92:0/3653049636 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5e400c9a0 con 0x7fd5f413e360 2023-11-24T06:13:48.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:48.602+0000 7fd5fc748640 1 -- 172.21.15.92:0/3653049636 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5f4153070 con 0x7fd5f413e360 2023-11-24T06:13:48.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:48.602+0000 7fd5eaffd640 1 -- 172.21.15.92:0/3653049636 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5e400cd50 con 0x7fd5f413e360 2023-11-24T06:13:48.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:48.602+0000 7fd5fc748640 1 -- 172.21.15.92:0/3653049636 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5f413dab0 con 0x7fd5f413e360 2023-11-24T06:13:48.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:48.603+0000 7fd5fc748640 1 -- 172.21.15.92:0/3653049636 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fd5f4071f80 con 0x7fd5f413e360 2023-11-24T06:13:48.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:48.605+0000 7fd5eaffd640 1 -- 172.21.15.92:0/3653049636 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 7) v1 ==== 89333+0+0 (secure 0 0 0) 0x7fd5e4013470 con 0x7fd5f413e360 2023-11-24T06:13:48.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:48.606+0000 7fd5eaffd640 1 --2- 172.21.15.92:0/3653049636 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fd5d806af90 0x7fd5d806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:48.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:48.606+0000 7fd5eaffd640 1 -- 172.21.15.92:0/3653049636 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fd5e4089620 con 0x7fd5f413e360 2023-11-24T06:13:48.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:48.606+0000 7fd5fa4e4640 1 --2- 172.21.15.92:0/3653049636 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fd5d806af90 0x7fd5d806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:48.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:48.606+0000 7fd5eaffd640 1 --2- 172.21.15.92:0/3653049636 >> [v2:172.21.15.118:6808/2946986141,v1:172.21.15.118:6809/2946986141] conn(0x7fd5d8070df0 0x7fd5d8073210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:48.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:48.606+0000 7fd5eaffd640 1 -- 172.21.15.92:0/3653049636 --> [v2:172.21.15.118:6808/2946986141,v1:172.21.15.118:6809/2946986141] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd5d80738c0 con 0x7fd5d8070df0 2023-11-24T06:13:48.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:48.606+0000 7fd5eaffd640 1 -- 172.21.15.92:0/3653049636 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_get_version_reply(handle=1 version=66) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd5e408bc40 con 0x7fd5f413e360 2023-11-24T06:13:48.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:48.607+0000 7fd5fa4e4640 1 --2- 172.21.15.92:0/3653049636 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fd5d806af90 0x7fd5d806d450 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fd5f0034000 tx=0x7fd5f0008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:48.610 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:48.607+0000 7fd5face5640 1 --2- 172.21.15.92:0/3653049636 >> [v2:172.21.15.118:6808/2946986141,v1:172.21.15.118:6809/2946986141] conn(0x7fd5d8070df0 0x7fd5d8073210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:48.610 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:48.608+0000 7fd5face5640 1 --2- 172.21.15.92:0/3653049636 >> [v2:172.21.15.118:6808/2946986141,v1:172.21.15.118:6809/2946986141] conn(0x7fd5d8070df0 0x7fd5d8073210 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:48.611 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:48.610+0000 7fd5eaffd640 1 -- 172.21.15.92:0/3653049636 <== osd.6 v2:172.21.15.118:6808/2946986141 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7fd5d80738c0 con 0x7fd5d8070df0 2023-11-24T06:13:48.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:48.624+0000 7fd5fc748640 1 -- 172.21.15.92:0/3653049636 --> [v2:172.21.15.118:6808/2946986141,v1:172.21.15.118:6809/2946986141] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fd5f40ce850 con 0x7fd5d8070df0 2023-11-24T06:13:48.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:48.625+0000 7fd5eaffd640 1 -- 172.21.15.92:0/3653049636 <== osd.6 v2:172.21.15.118:6808/2946986141 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fd5ec001b80 con 0x7fd5d8070df0 2023-11-24T06:13:48.627 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-24T06:13:48.627 INFO:teuthology.orchestra.run.smithi092.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2023-11-24T06:13:48.628 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-24T06:13:48.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:48.625+0000 7fd5fc748640 1 -- 172.21.15.92:0/3653049636 >> [v2:172.21.15.118:6808/2946986141,v1:172.21.15.118:6809/2946986141] conn(0x7fd5d8070df0 msgr2=0x7fd5d8073210 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:48.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:48.625+0000 7fd5fc748640 1 --2- 172.21.15.92:0/3653049636 >> [v2:172.21.15.118:6808/2946986141,v1:172.21.15.118:6809/2946986141] conn(0x7fd5d8070df0 0x7fd5d8073210 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:48.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:48.626+0000 7fd5fc748640 1 -- 172.21.15.92:0/3653049636 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fd5d806af90 msgr2=0x7fd5d806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:48.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:48.626+0000 7fd5fc748640 1 --2- 172.21.15.92:0/3653049636 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fd5d806af90 0x7fd5d806d450 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fd5f0034000 tx=0x7fd5f0008030 comp rx=0 tx=0).stop 2023-11-24T06:13:48.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:48.626+0000 7fd5fc748640 1 -- 172.21.15.92:0/3653049636 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd5f413e360 msgr2=0x7fd5f414f630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:48.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:48.626+0000 7fd5fc748640 1 --2- 172.21.15.92:0/3653049636 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd5f413e360 0x7fd5f414f630 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7fd5e400e9b0 tx=0x7fd5e400ee70 comp rx=0 tx=0).stop 2023-11-24T06:13:48.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:48.626+0000 7fd5face5640 1 -- 172.21.15.92:0/3653049636 reap_dead start 2023-11-24T06:13:48.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:48.626+0000 7fd5fc748640 1 -- 172.21.15.92:0/3653049636 shutdown_connections 2023-11-24T06:13:48.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:48.626+0000 7fd5fc748640 1 -- 172.21.15.92:0/3653049636 >> 172.21.15.92:0/3653049636 conn(0x7fd5f406a800 msgr2=0x7fd5f40b0bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:13:48.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:48.626+0000 7fd5fc748640 1 -- 172.21.15.92:0/3653049636 shutdown_connections 2023-11-24T06:13:48.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:48.626+0000 7fd5fc748640 1 -- 172.21.15.92:0/3653049636 wait complete. 2023-11-24T06:13:48.644 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2023-11-24T06:13:48.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:48.853+0000 7f2bba65d640 1 Processor -- start 2023-11-24T06:13:48.856 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:48.853+0000 7f2bba65d640 1 -- start start 2023-11-24T06:13:48.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:48.855+0000 7f2bba65d640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f2bb4143ae0 0x7f2bb40cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:48.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:48.855+0000 7f2bba65d640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2bb40cf7d0 0x7f2bb415dca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:48.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:48.855+0000 7f2bba65d640 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2bb40cfc30 con 0x7f2bb415e1e0 2023-11-24T06:13:48.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:48.855+0000 7f2bba65d640 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2bb40b1ee0 con 0x7f2bb4143ae0 2023-11-24T06:13:48.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:48.855+0000 7f2bba65d640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bb40b2780 con 0x7f2bb40cf7d0 2023-11-24T06:13:48.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:48.855+0000 7f2bb37fe640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2bb40cf7d0 0x7f2bb415dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:48.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:48.855+0000 7f2bb37fe640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2bb40cf7d0 0x7f2bb415dca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3301/0 says I am v2:172.21.15.92:34816/0 (socket says 172.21.15.92:34816) 2023-11-24T06:13:48.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:48.855+0000 7f2bb3fff640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f2bb4143ae0 0x7f2bb40cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:48.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:48.855+0000 7f2bb37fe640 1 -- 172.21.15.92:0/1165965990 learned_addr learned my addr 172.21.15.92:0/1165965990 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:13:48.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:48.856+0000 7f2bb3fff640 1 -- 172.21.15.92:0/1165965990 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2bb40cf7d0 msgr2=0x7f2bb415dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:13:48.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:48.856+0000 7f2bb3fff640 1 --2- 172.21.15.92:0/1165965990 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2bb40cf7d0 0x7f2bb415dca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:48.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:48.856+0000 7f2bb3fff640 1 -- 172.21.15.92:0/1165965990 >> v1:172.21.15.118:6789/0 conn(0x7f2bb415e1e0 legacy=0x7f2bb415a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:48.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:48.856+0000 7f2bb3fff640 1 -- 172.21.15.92:0/1165965990 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bb415e990 con 0x7f2bb4143ae0 2023-11-24T06:13:48.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:48.858+0000 7f2bb3fff640 1 --2- 172.21.15.92:0/1165965990 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2bb4143ae0 0x7f2bb40cf290 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f2ba40097a0 tx=0x7f2ba402b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=12f01e12d8ccbf92 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:48.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:48.858+0000 7f2bb2ffd640 1 -- 172.21.15.92:0/1165965990 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ba4031830 con 0x7f2bb4143ae0 2023-11-24T06:13:48.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:48.858+0000 7f2bb2ffd640 1 -- 172.21.15.92:0/1165965990 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ba4031990 con 0x7f2bb4143ae0 2023-11-24T06:13:48.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:48.858+0000 7f2bb2ffd640 1 -- 172.21.15.92:0/1165965990 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ba4031d20 con 0x7f2bb4143ae0 2023-11-24T06:13:48.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:48.859+0000 7f2bba65d640 1 -- 172.21.15.92:0/1165965990 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2bb4143ae0 msgr2=0x7f2bb40cf290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:48.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:48.859+0000 7f2bba65d640 1 --2- 172.21.15.92:0/1165965990 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2bb4143ae0 0x7f2bb40cf290 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f2ba40097a0 tx=0x7f2ba402b8d0 comp rx=0 tx=0).stop 2023-11-24T06:13:48.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:48.859+0000 7f2bba65d640 1 -- 172.21.15.92:0/1165965990 shutdown_connections 2023-11-24T06:13:48.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:48.859+0000 7f2bba65d640 1 --2- 172.21.15.92:0/1165965990 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2bb40cf7d0 0x7f2bb415dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:48.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:48.859+0000 7f2bba65d640 1 --2- 172.21.15.92:0/1165965990 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2bb4143ae0 0x7f2bb40cf290 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:48.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:48.859+0000 7f2bba65d640 1 -- 172.21.15.92:0/1165965990 >> 172.21.15.92:0/1165965990 conn(0x7f2bb406a800 msgr2=0x7f2bb406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:13:48.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:48.859+0000 7f2bba65d640 1 -- 172.21.15.92:0/1165965990 shutdown_connections 2023-11-24T06:13:48.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:48.859+0000 7f2bba65d640 1 -- 172.21.15.92:0/1165965990 wait complete. 2023-11-24T06:13:48.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:48.862+0000 7f2bba65d640 1 Processor -- start 2023-11-24T06:13:48.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:48.862+0000 7f2bba65d640 1 -- start start 2023-11-24T06:13:48.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:48.863+0000 7f2bba65d640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2bb40cf7d0 0x7f2bb40ce3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:48.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:48.863+0000 7f2bba65d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2bb4143ae0 0x7f2bb414de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:48.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:48.863+0000 7f2bba65d640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2bb415e1e0 0x7f2bb406c470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:48.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:48.863+0000 7f2bba65d640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bb414f250 con 0x7f2bb4143ae0 2023-11-24T06:13:48.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:48.863+0000 7f2bba65d640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bb414e920 con 0x7f2bb415e1e0 2023-11-24T06:13:48.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:48.863+0000 7f2bba65d640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bb414ec00 con 0x7f2bb40cf7d0 2023-11-24T06:13:48.869 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:48.863+0000 7f2bb37fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2bb4143ae0 0x7f2bb414de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:48.869 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:48.864+0000 7f2bb37fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2bb4143ae0 0x7f2bb414de80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48036/0 (socket says 172.21.15.92:48036) 2023-11-24T06:13:48.869 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:48.864+0000 7f2bb37fe640 1 -- 172.21.15.92:0/3740815563 learned_addr learned my addr 172.21.15.92:0/3740815563 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:13:48.869 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:48.864+0000 7f2bb3fff640 1 --2- 172.21.15.92:0/3740815563 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2bb40cf7d0 0x7f2bb40ce3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:48.869 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:48.864+0000 7f2bb8bfa640 1 --2- 172.21.15.92:0/3740815563 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2bb415e1e0 0x7f2bb406c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:48.870 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:48.864+0000 7f2bb37fe640 1 -- 172.21.15.92:0/3740815563 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2bb40cf7d0 msgr2=0x7f2bb40ce3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:48.870 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:48.864+0000 7f2bb37fe640 1 --2- 172.21.15.92:0/3740815563 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2bb40cf7d0 0x7f2bb40ce3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:48.870 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:48.864+0000 7f2bb37fe640 1 -- 172.21.15.92:0/3740815563 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2bb415e1e0 msgr2=0x7f2bb406c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:48.870 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:48.864+0000 7f2bb37fe640 1 --2- 172.21.15.92:0/3740815563 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2bb415e1e0 0x7f2bb406c470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:48.871 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:48.864+0000 7f2bb37fe640 1 -- 172.21.15.92:0/3740815563 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ba4009400 con 0x7f2bb4143ae0 2023-11-24T06:13:48.871 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:48.864+0000 7f2bb3fff640 1 --2- 172.21.15.92:0/3740815563 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2bb40cf7d0 0x7f2bb40ce3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:13:48.871 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:48.864+0000 7f2bb8bfa640 1 --2- 172.21.15.92:0/3740815563 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2bb415e1e0 0x7f2bb406c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:13:48.871 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:48.864+0000 7f2bb37fe640 1 --2- 172.21.15.92:0/3740815563 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2bb4143ae0 0x7f2bb414de80 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f2ba800eb30 tx=0x7f2ba800d350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:48.872 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:48.865+0000 7f2bb0ff9640 1 -- 172.21.15.92:0/3740815563 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ba8019480 con 0x7f2bb4143ae0 2023-11-24T06:13:48.872 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:48.865+0000 7f2bb0ff9640 1 -- 172.21.15.92:0/3740815563 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ba80195e0 con 0x7f2bb4143ae0 2023-11-24T06:13:48.872 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:48.865+0000 7f2bb0ff9640 1 -- 172.21.15.92:0/3740815563 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ba8019970 con 0x7f2bb4143ae0 2023-11-24T06:13:48.872 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:48.865+0000 7f2bba65d640 1 -- 172.21.15.92:0/3740815563 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bb414eee0 con 0x7f2bb4143ae0 2023-11-24T06:13:48.872 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:48.865+0000 7f2bba65d640 1 -- 172.21.15.92:0/3740815563 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bb40ceaf0 con 0x7f2bb4143ae0 2023-11-24T06:13:48.873 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:48.866+0000 7f2bba65d640 1 -- 172.21.15.92:0/3740815563 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2bb40b2bf0 con 0x7f2bb4143ae0 2023-11-24T06:13:48.874 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:48.867+0000 7f2bb0ff9640 1 -- 172.21.15.92:0/3740815563 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 7) v1 ==== 89333+0+0 (secure 0 0 0) 0x7f2ba8019ad0 con 0x7f2bb4143ae0 2023-11-24T06:13:48.874 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:48.868+0000 7f2bb0ff9640 1 --2- 172.21.15.92:0/3740815563 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f2ba006af60 0x7f2ba006d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:48.874 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:48.868+0000 7f2bb0ff9640 1 -- 172.21.15.92:0/3740815563 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f2ba8014070 con 0x7f2bb4143ae0 2023-11-24T06:13:48.874 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:48.868+0000 7f2bb3fff640 1 --2- 172.21.15.92:0/3740815563 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f2ba006af60 0x7f2ba006d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:48.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:48.869+0000 7f2bb3fff640 1 --2- 172.21.15.92:0/3740815563 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f2ba006af60 0x7f2ba006d420 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f2ba4036000 tx=0x7f2ba4008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:48.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:48.872+0000 7f2bb0ff9640 1 -- 172.21.15.92:0/3740815563 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2bb40b2bf0 con 0x7f2bb4143ae0 2023-11-24T06:13:49.059 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:49.057+0000 7f2bba65d640 1 -- 172.21.15.92:0/3740815563 --> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) v1 -- 0x7f2bb414ea90 con 0x7f2ba006af60 2023-11-24T06:13:49.059 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:49.057+0000 7f2bb0ff9640 1 -- 172.21.15.92:0/3740815563 <== mgr.4101 v2:172.21.15.92:6800/784027401 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f2bb414ea90 con 0x7f2ba006af60 2023-11-24T06:13:49.060 INFO:teuthology.orchestra.run.smithi092.stderr:instructed osd(s) 6 to deep-scrub 2023-11-24T06:13:49.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:49.060+0000 7f2bba65d640 1 -- 172.21.15.92:0/3740815563 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f2ba006af60 msgr2=0x7f2ba006d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:49.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:49.060+0000 7f2bba65d640 1 --2- 172.21.15.92:0/3740815563 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f2ba006af60 0x7f2ba006d420 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f2ba4036000 tx=0x7f2ba4008030 comp rx=0 tx=0).stop 2023-11-24T06:13:49.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:49.060+0000 7f2bba65d640 1 -- 172.21.15.92:0/3740815563 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2bb4143ae0 msgr2=0x7f2bb414de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:49.063 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:49.060+0000 7f2bba65d640 1 --2- 172.21.15.92:0/3740815563 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2bb4143ae0 0x7f2bb414de80 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f2ba800eb30 tx=0x7f2ba800d350 comp rx=0 tx=0).stop 2023-11-24T06:13:49.063 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:49.060+0000 7f2bba65d640 1 -- 172.21.15.92:0/3740815563 shutdown_connections 2023-11-24T06:13:49.063 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:49.060+0000 7f2bba65d640 1 --2- 172.21.15.92:0/3740815563 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f2ba006af60 0x7f2ba006d420 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:49.063 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:49.060+0000 7f2bba65d640 1 --2- 172.21.15.92:0/3740815563 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2bb415e1e0 0x7f2bb406c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:49.063 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:49.060+0000 7f2bba65d640 1 --2- 172.21.15.92:0/3740815563 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2bb4143ae0 0x7f2bb414de80 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:49.063 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:49.060+0000 7f2bba65d640 1 --2- 172.21.15.92:0/3740815563 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f2bb40cf7d0 0x7f2bb40ce3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:49.063 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:49.060+0000 7f2bba65d640 1 -- 172.21.15.92:0/3740815563 >> 172.21.15.92:0/3740815563 conn(0x7f2bb406a800 msgr2=0x7f2bb40702c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:13:49.064 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:49.060+0000 7f2bba65d640 1 -- 172.21.15.92:0/3740815563 shutdown_connections 2023-11-24T06:13:49.064 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:49.060+0000 7f2bba65d640 1 -- 172.21.15.92:0/3740815563 wait complete. 2023-11-24T06:13:49.079 INFO:tasks.ceph:Scrubbing osd.7 2023-11-24T06:13:49.079 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2023-11-24T06:13:49.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:49.283+0000 7fce08fe5640 1 Processor -- start 2023-11-24T06:13:49.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:49.283+0000 7fce08fe5640 1 -- start start 2023-11-24T06:13:49.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:49.285+0000 7fce08fe5640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fce0413df80 0x7fce0415bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:49.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:49.285+0000 7fce08fe5640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fce0413e360 0x7fce040cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:49.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:49.285+0000 7fce08fe5640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fce0415f9c0 0x7fce040cf7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:49.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:49.285+0000 7fce08fe5640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce0406c470 con 0x7fce0415f9c0 2023-11-24T06:13:49.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:49.285+0000 7fce08fe5640 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce0406c750 con 0x7fce0413df80 2023-11-24T06:13:49.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:49.285+0000 7fce08fe5640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce0406ca30 con 0x7fce0413e360 2023-11-24T06:13:49.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:49.285+0000 7fce02d9d640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fce0415f9c0 0x7fce040cf7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:49.289 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:49.285+0000 7fce02d9d640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fce0415f9c0 0x7fce040cf7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:54298/0 (socket says 172.21.15.92:54298) 2023-11-24T06:13:49.289 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:49.285+0000 7fce02d9d640 1 -- 172.21.15.92:0/3266405658 learned_addr learned my addr 172.21.15.92:0/3266405658 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:13:49.290 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:49.286+0000 7fce02d9d640 1 -- 172.21.15.92:0/3266405658 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fce0413e360 msgr2=0x7fce040cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:13:49.290 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:49.286+0000 7fcdfbfff640 1 --2- 172.21.15.92:0/3266405658 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fce0413e360 0x7fce040cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:49.290 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:49.286+0000 7fce0259c640 1 --2- 172.21.15.92:0/3266405658 >> v2:172.21.15.118:3300/0 conn(0x7fce0413df80 0x7fce0415bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:49.291 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:49.286+0000 7fce02d9d640 1 --2- 172.21.15.92:0/3266405658 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fce0413e360 0x7fce040cf290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:49.291 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:49.286+0000 7fce02d9d640 1 -- 172.21.15.92:0/3266405658 >> v2:172.21.15.118:3300/0 conn(0x7fce0413df80 msgr2=0x7fce0415bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:13:49.291 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:49.286+0000 7fce02d9d640 1 --2- 172.21.15.92:0/3266405658 >> v2:172.21.15.118:3300/0 conn(0x7fce0413df80 0x7fce0415bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:49.291 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:49.286+0000 7fce02d9d640 1 -- 172.21.15.92:0/3266405658 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce0406cd10 con 0x7fce0415f9c0 2023-11-24T06:13:49.291 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:49.286+0000 7fce0259c640 1 --2- 172.21.15.92:0/3266405658 >> v2:172.21.15.118:3300/0 conn(0x7fce0413df80 0x7fce0415bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:13:49.291 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:49.287+0000 7fce02d9d640 1 --2- 172.21.15.92:0/3266405658 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fce0415f9c0 0x7fce040cf7d0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7fce04072770 tx=0x7fcdfc02bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f88b0e7f00c39d6 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:49.291 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:49.287+0000 7fce01d9b640 1 -- 172.21.15.92:0/3266405658 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdfc02ca80 con 0x7fce0415f9c0 2023-11-24T06:13:49.292 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:49.287+0000 7fce01d9b640 1 -- 172.21.15.92:0/3266405658 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdfc02cbe0 con 0x7fce0415f9c0 2023-11-24T06:13:49.292 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:49.287+0000 7fce01d9b640 1 -- 172.21.15.92:0/3266405658 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdfc0343f0 con 0x7fce0415f9c0 2023-11-24T06:13:49.292 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:49.288+0000 7fce08fe5640 1 -- 172.21.15.92:0/3266405658 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fce0415f9c0 msgr2=0x7fce040cf7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:49.292 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:49.288+0000 7fce08fe5640 1 --2- 172.21.15.92:0/3266405658 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fce0415f9c0 0x7fce040cf7d0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7fce04072770 tx=0x7fcdfc02bee0 comp rx=0 tx=0).stop 2023-11-24T06:13:49.292 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:49.288+0000 7fce08fe5640 1 -- 172.21.15.92:0/3266405658 shutdown_connections 2023-11-24T06:13:49.292 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:49.288+0000 7fce08fe5640 1 --2- 172.21.15.92:0/3266405658 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fce0415f9c0 0x7fce040cf7d0 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:49.292 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:49.288+0000 7fce08fe5640 1 --2- 172.21.15.92:0/3266405658 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fce0413e360 0x7fce040cf290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:49.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:49.288+0000 7fce08fe5640 1 --2- 172.21.15.92:0/3266405658 >> v2:172.21.15.118:3300/0 conn(0x7fce0413df80 0x7fce0415bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:49.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:49.288+0000 7fce08fe5640 1 -- 172.21.15.92:0/3266405658 >> 172.21.15.92:0/3266405658 conn(0x7fce0406a800 msgr2=0x7fce0406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:13:49.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:49.288+0000 7fce08fe5640 1 -- 172.21.15.92:0/3266405658 shutdown_connections 2023-11-24T06:13:49.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:49.288+0000 7fce08fe5640 1 -- 172.21.15.92:0/3266405658 wait complete. 2023-11-24T06:13:49.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:49.290+0000 7fce08fe5640 1 Processor -- start 2023-11-24T06:13:49.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:49.290+0000 7fce08fe5640 1 -- start start 2023-11-24T06:13:49.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:49.291+0000 7fce08fe5640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fce0413df80 0x7fce0413cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:49.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:49.291+0000 7fce08fe5640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fce0413e360 0x7fce041f3530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:49.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:49.291+0000 7fce08fe5640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fce0415f9c0 0x7fce04071310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:49.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:49.291+0000 7fce08fe5640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce041f4920 con 0x7fce0413df80 2023-11-24T06:13:49.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:49.291+0000 7fce08fe5640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce041f3a70 con 0x7fce0413e360 2023-11-24T06:13:49.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:49.291+0000 7fce08fe5640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce041f3d10 con 0x7fce0415f9c0 2023-11-24T06:13:49.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:49.291+0000 7fce0259c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fce0413df80 0x7fce0413cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:49.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:49.291+0000 7fce0259c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fce0413df80 0x7fce0413cc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:54308/0 (socket says 172.21.15.92:54308) 2023-11-24T06:13:49.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:49.291+0000 7fce0259c640 1 -- 172.21.15.92:0/689580555 learned_addr learned my addr 172.21.15.92:0/689580555 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:13:49.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:49.291+0000 7fcdfbfff640 1 --2- 172.21.15.92:0/689580555 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fce0413e360 0x7fce041f3530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:49.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:49.291+0000 7fce0259c640 1 -- 172.21.15.92:0/689580555 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fce0415f9c0 msgr2=0x7fce04071310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:49.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:49.291+0000 7fce0259c640 1 --2- 172.21.15.92:0/689580555 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fce0415f9c0 0x7fce04071310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:49.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:49.291+0000 7fce0259c640 1 -- 172.21.15.92:0/689580555 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fce0413e360 msgr2=0x7fce041f3530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:49.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:49.291+0000 7fce0259c640 1 --2- 172.21.15.92:0/689580555 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fce0413e360 0x7fce041f3530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:49.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:49.291+0000 7fce0259c640 1 -- 172.21.15.92:0/689580555 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdfc009b30 con 0x7fce0413df80 2023-11-24T06:13:49.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:49.291+0000 7fce0259c640 1 --2- 172.21.15.92:0/689580555 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fce0413df80 0x7fce0413cc00 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7fcdf400c820 tx=0x7fcdf400cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:49.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:49.292+0000 7fcdfaffd640 1 -- 172.21.15.92:0/689580555 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdf4007a30 con 0x7fce0413df80 2023-11-24T06:13:49.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:49.292+0000 7fcdfaffd640 1 -- 172.21.15.92:0/689580555 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdf4007b90 con 0x7fce0413df80 2023-11-24T06:13:49.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:49.292+0000 7fcdfaffd640 1 -- 172.21.15.92:0/689580555 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdf400ce10 con 0x7fce0413df80 2023-11-24T06:13:49.299 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:49.292+0000 7fce08fe5640 1 -- 172.21.15.92:0/689580555 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce041f3ff0 con 0x7fce0413df80 2023-11-24T06:13:49.299 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:49.292+0000 7fce08fe5640 1 -- 172.21.15.92:0/689580555 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce041f45b0 con 0x7fce0413df80 2023-11-24T06:13:49.299 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:49.293+0000 7fce08fe5640 1 -- 172.21.15.92:0/689580555 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fce041408d0 con 0x7fce0413df80 2023-11-24T06:13:49.300 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:49.294+0000 7fcdfaffd640 1 -- 172.21.15.92:0/689580555 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 7) v1 ==== 89333+0+0 (secure 0 0 0) 0x7fcdf40113f0 con 0x7fce0413df80 2023-11-24T06:13:49.300 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:49.295+0000 7fcdfaffd640 1 --2- 172.21.15.92:0/689580555 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fcde406af60 0x7fcde406d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:49.300 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:49.295+0000 7fcdfaffd640 1 -- 172.21.15.92:0/689580555 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fcdf4089700 con 0x7fce0413df80 2023-11-24T06:13:49.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:49.295+0000 7fcdfbfff640 1 --2- 172.21.15.92:0/689580555 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fcde406af60 0x7fcde406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:49.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:49.295+0000 7fcdfaffd640 1 --2- 172.21.15.92:0/689580555 >> [v2:172.21.15.118:6817/48122059,v1:172.21.15.118:6818/48122059] conn(0x7fcde4070dc0 0x7fcde40731e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:49.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:49.295+0000 7fcdfaffd640 1 -- 172.21.15.92:0/689580555 --> [v2:172.21.15.118:6817/48122059,v1:172.21.15.118:6818/48122059] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fcde4073890 con 0x7fcde4070dc0 2023-11-24T06:13:49.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:49.295+0000 7fcdfaffd640 1 -- 172.21.15.92:0/689580555 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_get_version_reply(handle=1 version=66) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcdf408bd60 con 0x7fce0413df80 2023-11-24T06:13:49.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:49.296+0000 7fcdfbfff640 1 --2- 172.21.15.92:0/689580555 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fcde406af60 0x7fcde406d420 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fce040b3580 tx=0x7fcdec008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:49.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:49.296+0000 7fce02d9d640 1 --2- 172.21.15.92:0/689580555 >> [v2:172.21.15.118:6817/48122059,v1:172.21.15.118:6818/48122059] conn(0x7fcde4070dc0 0x7fcde40731e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:49.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:49.297+0000 7fce02d9d640 1 --2- 172.21.15.92:0/689580555 >> [v2:172.21.15.118:6817/48122059,v1:172.21.15.118:6818/48122059] conn(0x7fcde4070dc0 0x7fcde40731e0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:49.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:49.299+0000 7fcdfaffd640 1 -- 172.21.15.92:0/689580555 <== osd.7 v2:172.21.15.118:6817/48122059 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7fcde4073890 con 0x7fcde4070dc0 2023-11-24T06:13:49.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:49.312+0000 7fce08fe5640 1 -- 172.21.15.92:0/689580555 --> [v2:172.21.15.118:6817/48122059,v1:172.21.15.118:6818/48122059] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fce041f4a60 con 0x7fcde4070dc0 2023-11-24T06:13:49.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:49.313+0000 7fcdfaffd640 1 -- 172.21.15.92:0/689580555 <== osd.7 v2:172.21.15.118:6817/48122059 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fcdfc02c900 con 0x7fcde4070dc0 2023-11-24T06:13:49.317 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-24T06:13:49.317 INFO:teuthology.orchestra.run.smithi092.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2023-11-24T06:13:49.317 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-24T06:13:49.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:49.314+0000 7fce08fe5640 1 -- 172.21.15.92:0/689580555 >> [v2:172.21.15.118:6817/48122059,v1:172.21.15.118:6818/48122059] conn(0x7fcde4070dc0 msgr2=0x7fcde40731e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:49.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:49.314+0000 7fce08fe5640 1 --2- 172.21.15.92:0/689580555 >> [v2:172.21.15.118:6817/48122059,v1:172.21.15.118:6818/48122059] conn(0x7fcde4070dc0 0x7fcde40731e0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:49.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:49.314+0000 7fce08fe5640 1 -- 172.21.15.92:0/689580555 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fcde406af60 msgr2=0x7fcde406d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:49.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:49.314+0000 7fce08fe5640 1 --2- 172.21.15.92:0/689580555 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fcde406af60 0x7fcde406d420 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fce040b3580 tx=0x7fcdec008040 comp rx=0 tx=0).stop 2023-11-24T06:13:49.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:49.314+0000 7fce08fe5640 1 -- 172.21.15.92:0/689580555 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fce0413df80 msgr2=0x7fce0413cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:49.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:49.314+0000 7fce08fe5640 1 --2- 172.21.15.92:0/689580555 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fce0413df80 0x7fce0413cc00 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7fcdf400c820 tx=0x7fcdf400cce0 comp rx=0 tx=0).stop 2023-11-24T06:13:49.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:49.314+0000 7fce02d9d640 1 -- 172.21.15.92:0/689580555 reap_dead start 2023-11-24T06:13:49.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:49.314+0000 7fce08fe5640 1 -- 172.21.15.92:0/689580555 shutdown_connections 2023-11-24T06:13:49.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:49.314+0000 7fce08fe5640 1 -- 172.21.15.92:0/689580555 >> 172.21.15.92:0/689580555 conn(0x7fce0406a800 msgr2=0x7fce0406abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:13:49.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:49.314+0000 7fce08fe5640 1 -- 172.21.15.92:0/689580555 shutdown_connections 2023-11-24T06:13:49.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:49.315+0000 7fce08fe5640 1 -- 172.21.15.92:0/689580555 wait complete. 2023-11-24T06:13:49.333 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2023-11-24T06:13:49.539 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:49.535+0000 7f6cfeabb640 1 Processor -- start 2023-11-24T06:13:49.539 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:49.536+0000 7f6cfeabb640 1 -- start start 2023-11-24T06:13:49.544 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:49.537+0000 7f6cfeabb640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f6cf80d0100 0x7f6cf80b3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:49.544 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:49.537+0000 7f6cfeabb640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6cf80d0a10 con 0x7f6cf8160920 2023-11-24T06:13:49.544 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:49.537+0000 7f6cfeabb640 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6cf80cf080 con 0x7f6cf80d0f40 2023-11-24T06:13:49.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:49.537+0000 7f6cfeabb640 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6cf80c6be0 con 0x7f6cf80d0100 2023-11-24T06:13:49.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:49.537+0000 7f6cfdab9640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f6cf8160920 0x7f6cf80cf9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:58164/0 (socket says 172.21.15.92:58164) 2023-11-24T06:13:49.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:49.537+0000 7f6cfdab9640 1 -- 172.21.15.92:0/3390713677 learned_addr learned my addr 172.21.15.92:0/3390713677 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-24T06:13:49.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:49.538+0000 7f6cfcab7640 1 -- 172.21.15.92:0/3390713677 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3751563476 0 0) 0x7f6cf80d0a10 con 0x7f6cf8160920 2023-11-24T06:13:49.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:49.538+0000 7f6cfcab7640 1 -- 172.21.15.92:0/3390713677 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6ce4003520 con 0x7f6cf8160920 2023-11-24T06:13:49.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:49.538+0000 7f6cfd2b8640 1 --2- 172.21.15.92:0/3390713677 >> v2:172.21.15.118:3300/0 conn(0x7f6cf80d0100 0x7f6cf80b3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:49.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:49.538+0000 7f6cfcab7640 1 -- 172.21.15.92:0/3390713677 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1326114089 0 0) 0x7f6ce4003520 con 0x7f6cf8160920 2023-11-24T06:13:49.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:49.538+0000 7f6cfcab7640 1 -- 172.21.15.92:0/3390713677 >> v2:172.21.15.118:3300/0 conn(0x7f6cf80d0100 msgr2=0x7f6cf80b3f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:13:49.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:49.538+0000 7f6cfcab7640 1 --2- 172.21.15.92:0/3390713677 >> v2:172.21.15.118:3300/0 conn(0x7f6cf80d0100 0x7f6cf80b3f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:49.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:49.538+0000 7f6cfcab7640 1 -- 172.21.15.92:0/3390713677 >> v1:172.21.15.118:6789/0 conn(0x7f6cf80d0f40 legacy=0x7f6cf80d1390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:13:49.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:49.538+0000 7f6cfcab7640 1 -- 172.21.15.92:0/3390713677 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cf80ce7e0 con 0x7f6cf8160920 2023-11-24T06:13:49.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:49.538+0000 7f6cfcab7640 1 -- 172.21.15.92:0/3390713677 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2607661350 0 0) 0x7f6cf4001670 con 0x7f6cf8160920 2023-11-24T06:13:49.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:49.538+0000 7f6cfcab7640 1 -- 172.21.15.92:0/3390713677 >> v1:172.21.15.92:6789/0 conn(0x7f6cf8160920 legacy=0x7f6cf80cf9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:49.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:49.538+0000 7f6cfcab7640 1 --2- 172.21.15.92:0/3390713677 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6ce4003970 0x7f6ce4003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:49.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:49.538+0000 7f6cfcab7640 1 --2- 172.21.15.92:0/3390713677 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6ce40047c0 0x7f6ce4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:49.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:49.538+0000 7f6cfcab7640 1 --2- 172.21.15.92:0/3390713677 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6ce4005180 0x7f6ce400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:49.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:49.538+0000 7f6cfcab7640 1 -- 172.21.15.92:0/3390713677 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cf80c6be0 con 0x7f6ce40047c0 2023-11-24T06:13:49.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:49.538+0000 7f6cfcab7640 1 -- 172.21.15.92:0/3390713677 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ce400c0d0 con 0x7f6ce4003970 2023-11-24T06:13:49.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:49.538+0000 7f6cfcab7640 1 -- 172.21.15.92:0/3390713677 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ce400c290 con 0x7f6ce4005180 2023-11-24T06:13:49.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:49.539+0000 7f6cfd2b8640 1 --2- 172.21.15.92:0/3390713677 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6ce40047c0 0x7f6ce4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:49.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:49.539+0000 7f6cfdab9640 1 --2- 172.21.15.92:0/3390713677 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6ce4003970 0x7f6ce4003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:49.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:49.539+0000 7f6cfd2b8640 1 -- 172.21.15.92:0/3390713677 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6ce4005180 msgr2=0x7f6ce400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:13:49.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:49.539+0000 7f6cfd2b8640 1 --2- 172.21.15.92:0/3390713677 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6ce4005180 0x7f6ce400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:49.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:49.539+0000 7f6cfd2b8640 1 -- 172.21.15.92:0/3390713677 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6ce4003970 msgr2=0x7f6ce4003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:13:49.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:49.539+0000 7f6cfd2b8640 1 --2- 172.21.15.92:0/3390713677 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6ce4003970 0x7f6ce4003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:49.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:49.539+0000 7f6cfd2b8640 1 -- 172.21.15.92:0/3390713677 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ce4003520 con 0x7f6ce40047c0 2023-11-24T06:13:49.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:49.539+0000 7f6cfe2ba640 1 -- 172.21.15.92:0/3390713677 reap_dead start 2023-11-24T06:13:49.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:49.540+0000 7f6cfd2b8640 1 --2- 172.21.15.92:0/3390713677 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6ce40047c0 0x7f6ce4004c40 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f6ce800ba30 tx=0x7f6ce40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbde3b5ba6a05c8e server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:49.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:49.540+0000 7f6cfcab7640 1 -- 172.21.15.92:0/3390713677 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ce8032640 con 0x7f6ce40047c0 2023-11-24T06:13:49.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:49.540+0000 7f6cfcab7640 1 -- 172.21.15.92:0/3390713677 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ce80327a0 con 0x7f6ce40047c0 2023-11-24T06:13:49.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:49.540+0000 7f6cfcab7640 1 -- 172.21.15.92:0/3390713677 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ce8032b30 con 0x7f6ce40047c0 2023-11-24T06:13:49.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:49.541+0000 7f6cfeabb640 1 -- 172.21.15.92:0/3390713677 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6ce40047c0 msgr2=0x7f6ce4004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:49.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:49.541+0000 7f6cfeabb640 1 --2- 172.21.15.92:0/3390713677 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6ce40047c0 0x7f6ce4004c40 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f6ce800ba30 tx=0x7f6ce40057f0 comp rx=0 tx=0).stop 2023-11-24T06:13:49.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:49.541+0000 7f6cfeabb640 1 -- 172.21.15.92:0/3390713677 shutdown_connections 2023-11-24T06:13:49.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:49.541+0000 7f6cfeabb640 1 --2- 172.21.15.92:0/3390713677 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6ce40047c0 0x7f6ce4004c40 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:49.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:49.541+0000 7f6cfeabb640 1 -- 172.21.15.92:0/3390713677 >> 172.21.15.92:0/3390713677 conn(0x7f6cf80d3090 msgr2=0x7f6cf80d34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:13:49.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:49.541+0000 7f6cfeabb640 1 -- 172.21.15.92:0/3390713677 shutdown_connections 2023-11-24T06:13:49.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:49.541+0000 7f6cfeabb640 1 -- 172.21.15.92:0/3390713677 wait complete. 2023-11-24T06:13:49.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:49.543+0000 7f6cfeabb640 1 Processor -- start 2023-11-24T06:13:49.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:49.543+0000 7f6cfeabb640 1 -- start start 2023-11-24T06:13:49.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:49.544+0000 7f6cfeabb640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6ce40047c0 0x7f6cf81f36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:49.556 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:49.544+0000 7f6cfeabb640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6cf81f0160 0x7f6cf80b7b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:49.556 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:49.544+0000 7f6cfeabb640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6cf80b80a0 0x7f6cf81f6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:49.556 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:49.544+0000 7f6cfeabb640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cf80b8520 con 0x7f6cf80b80a0 2023-11-24T06:13:49.557 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:49.544+0000 7f6cfeabb640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cf80b6c00 con 0x7f6cf81f0160 2023-11-24T06:13:49.557 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:49.544+0000 7f6cfeabb640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cf81f3c30 con 0x7f6ce40047c0 2023-11-24T06:13:49.557 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:49.544+0000 7f6cfe2ba640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6cf80b80a0 0x7f6cf81f6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:49.557 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:49.544+0000 7f6cfe2ba640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6cf80b80a0 0x7f6cf81f6260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:54318/0 (socket says 172.21.15.92:54318) 2023-11-24T06:13:49.558 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:49.544+0000 7f6cfe2ba640 1 -- 172.21.15.92:0/769377980 learned_addr learned my addr 172.21.15.92:0/769377980 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:13:49.558 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:49.545+0000 7f6cfe2ba640 1 -- 172.21.15.92:0/769377980 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6ce40047c0 msgr2=0x7f6cf81f36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:49.558 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:49.545+0000 7f6cfe2ba640 1 --2- 172.21.15.92:0/769377980 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6ce40047c0 0x7f6cf81f36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:49.558 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:49.545+0000 7f6cfe2ba640 1 -- 172.21.15.92:0/769377980 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6cf81f0160 msgr2=0x7f6cf80b7b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:49.559 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:49.545+0000 7f6cfe2ba640 1 --2- 172.21.15.92:0/769377980 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6cf81f0160 0x7f6cf80b7b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:49.559 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:49.545+0000 7f6cfe2ba640 1 -- 172.21.15.92:0/769377980 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cf80b5ff0 con 0x7f6cf80b80a0 2023-11-24T06:13:49.559 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:49.545+0000 7f6cfe2ba640 1 --2- 172.21.15.92:0/769377980 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6cf80b80a0 0x7f6cf81f6260 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f6cf808d590 tx=0x7f6cf000cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:49.559 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:49.545+0000 7f6cee7fc640 1 -- 172.21.15.92:0/769377980 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cf0005d40 con 0x7f6cf80b80a0 2023-11-24T06:13:49.560 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:49.545+0000 7f6cee7fc640 1 -- 172.21.15.92:0/769377980 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cf000ced0 con 0x7f6cf80b80a0 2023-11-24T06:13:49.560 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:49.545+0000 7f6cee7fc640 1 -- 172.21.15.92:0/769377980 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cf0005ea0 con 0x7f6cf80b80a0 2023-11-24T06:13:49.560 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:49.545+0000 7f6cfeabb640 1 -- 172.21.15.92:0/769377980 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6cf80b62d0 con 0x7f6cf80b80a0 2023-11-24T06:13:49.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:49.545+0000 7f6cfeabb640 1 -- 172.21.15.92:0/769377980 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cf80b6890 con 0x7f6cf80b80a0 2023-11-24T06:13:49.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:49.547+0000 7f6cfeabb640 1 -- 172.21.15.92:0/769377980 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6cf8067450 con 0x7f6cf80b80a0 2023-11-24T06:13:49.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:49.547+0000 7f6cee7fc640 1 -- 172.21.15.92:0/769377980 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 7) v1 ==== 89333+0+0 (secure 0 0 0) 0x7f6cf000a850 con 0x7f6cf80b80a0 2023-11-24T06:13:49.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:49.548+0000 7f6cee7fc640 1 --2- 172.21.15.92:0/769377980 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f6cdc06af90 0x7f6cdc06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:49.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:49.548+0000 7f6cee7fc640 1 -- 172.21.15.92:0/769377980 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f6cf00883b0 con 0x7f6cf80b80a0 2023-11-24T06:13:49.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:49.549+0000 7f6cfdab9640 1 --2- 172.21.15.92:0/769377980 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f6cdc06af90 0x7f6cdc06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:49.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:49.549+0000 7f6cfdab9640 1 --2- 172.21.15.92:0/769377980 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f6cdc06af90 0x7f6cdc06d450 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f6cf4001eb0 tx=0x7f6cf4001dc0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:49.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:49.553+0000 7f6cee7fc640 1 -- 172.21.15.92:0/769377980 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6cf00536c0 con 0x7f6cf80b80a0 2023-11-24T06:13:49.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:49.804+0000 7f6cfeabb640 1 -- 172.21.15.92:0/769377980 --> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) v1 -- 0x7f6cf8160d00 con 0x7f6cdc06af90 2023-11-24T06:13:49.808 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:49.805+0000 7f6cee7fc640 1 -- 172.21.15.92:0/769377980 <== mgr.4101 v2:172.21.15.92:6800/784027401 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f6cf400e810 con 0x7f6cdc06af90 2023-11-24T06:13:49.808 INFO:teuthology.orchestra.run.smithi092.stderr:instructed osd(s) 7 to deep-scrub 2023-11-24T06:13:49.811 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:49.807+0000 7f6cfeabb640 1 -- 172.21.15.92:0/769377980 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f6cdc06af90 msgr2=0x7f6cdc06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:49.811 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:49.807+0000 7f6cfeabb640 1 --2- 172.21.15.92:0/769377980 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f6cdc06af90 0x7f6cdc06d450 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f6cf4001eb0 tx=0x7f6cf4001dc0 comp rx=0 tx=0).stop 2023-11-24T06:13:49.811 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:49.807+0000 7f6cfeabb640 1 -- 172.21.15.92:0/769377980 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6cf80b80a0 msgr2=0x7f6cf81f6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:49.811 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:49.807+0000 7f6cfeabb640 1 --2- 172.21.15.92:0/769377980 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6cf80b80a0 0x7f6cf81f6260 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f6cf808d590 tx=0x7f6cf000cb90 comp rx=0 tx=0).stop 2023-11-24T06:13:49.812 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:49.808+0000 7f6cfeabb640 1 -- 172.21.15.92:0/769377980 shutdown_connections 2023-11-24T06:13:49.812 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:49.808+0000 7f6cfeabb640 1 --2- 172.21.15.92:0/769377980 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f6cdc06af90 0x7f6cdc06d450 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:49.812 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:49.808+0000 7f6cfeabb640 1 --2- 172.21.15.92:0/769377980 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6cf80b80a0 0x7f6cf81f6260 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:49.812 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:49.808+0000 7f6cfeabb640 1 --2- 172.21.15.92:0/769377980 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6cf81f0160 0x7f6cf80b7b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:49.812 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:49.808+0000 7f6cfeabb640 1 --2- 172.21.15.92:0/769377980 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f6ce40047c0 0x7f6cf81f36f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:49.812 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:49.808+0000 7f6cfeabb640 1 -- 172.21.15.92:0/769377980 >> 172.21.15.92:0/769377980 conn(0x7f6cf80d3090 msgr2=0x7f6cf80c6d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:13:49.812 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:49.808+0000 7f6cfeabb640 1 -- 172.21.15.92:0/769377980 shutdown_connections 2023-11-24T06:13:49.812 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:49.808+0000 7f6cfeabb640 1 -- 172.21.15.92:0/769377980 wait complete. 2023-11-24T06:13:49.830 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-24T06:13:50.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:50.025+0000 7fb5f0f53640 1 Processor -- start 2023-11-24T06:13:50.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:50.025+0000 7fb5f0f53640 1 -- start start 2023-11-24T06:13:50.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:50.027+0000 7fb5f0f53640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fb5ec13e360 0x7fb5ec0cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:50.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:50.027+0000 7fb5f0f53640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb5ec15f9c0 0x7fb5ec0cf7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:50.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:50.027+0000 7fb5f0f53640 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5ec15c5f0 con 0x7fb5ec13df80 2023-11-24T06:13:50.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:50.027+0000 7fb5f0f53640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5ec06c470 con 0x7fb5ec15f9c0 2023-11-24T06:13:50.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:50.027+0000 7fb5f0f53640 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5ec06c750 con 0x7fb5ec13e360 2023-11-24T06:13:50.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:50.027+0000 7fb5ead9d640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb5ec15f9c0 0x7fb5ec0cf7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:50.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:50.027+0000 7fb5ead9d640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb5ec15f9c0 0x7fb5ec0cf7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:54334/0 (socket says 172.21.15.92:54334) 2023-11-24T06:13:50.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:50.027+0000 7fb5ead9d640 1 -- 172.21.15.92:0/2764788046 learned_addr learned my addr 172.21.15.92:0/2764788046 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:13:50.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:50.028+0000 7fb5ead9d640 1 -- 172.21.15.92:0/2764788046 >> v2:172.21.15.118:3300/0 conn(0x7fb5ec13e360 msgr2=0x7fb5ec0cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:13:50.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:50.028+0000 7fb5ead9d640 1 --2- 172.21.15.92:0/2764788046 >> v2:172.21.15.118:3300/0 conn(0x7fb5ec13e360 0x7fb5ec0cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:50.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:50.028+0000 7fb5ead9d640 1 -- 172.21.15.92:0/2764788046 >> v1:172.21.15.118:6789/0 conn(0x7fb5ec13df80 legacy=0x7fb5ec15bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:13:50.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:50.028+0000 7fb5ead9d640 1 -- 172.21.15.92:0/2764788046 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5ec06ca30 con 0x7fb5ec15f9c0 2023-11-24T06:13:50.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:50.029+0000 7fb5ead9d640 1 --2- 172.21.15.92:0/2764788046 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb5ec15f9c0 0x7fb5ec0cf7d0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7fb5ec0ccae0 tx=0x7fb5e402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5d48898515953e1 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:50.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:50.029+0000 7fb5e9d9b640 1 -- 172.21.15.92:0/2764788046 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5e402ca70 con 0x7fb5ec15f9c0 2023-11-24T06:13:50.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:50.029+0000 7fb5e9d9b640 1 -- 172.21.15.92:0/2764788046 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5e402cbd0 con 0x7fb5ec15f9c0 2023-11-24T06:13:50.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:50.029+0000 7fb5e9d9b640 1 -- 172.21.15.92:0/2764788046 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5e40343f0 con 0x7fb5ec15f9c0 2023-11-24T06:13:50.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:50.030+0000 7fb5f0f53640 1 -- 172.21.15.92:0/2764788046 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb5ec15f9c0 msgr2=0x7fb5ec0cf7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:50.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:50.030+0000 7fb5f0f53640 1 --2- 172.21.15.92:0/2764788046 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb5ec15f9c0 0x7fb5ec0cf7d0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7fb5ec0ccae0 tx=0x7fb5e402bed0 comp rx=0 tx=0).stop 2023-11-24T06:13:50.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:50.030+0000 7fb5f0f53640 1 -- 172.21.15.92:0/2764788046 shutdown_connections 2023-11-24T06:13:50.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:50.030+0000 7fb5f0f53640 1 --2- 172.21.15.92:0/2764788046 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb5ec15f9c0 0x7fb5ec0cf7d0 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:50.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:50.030+0000 7fb5f0f53640 1 --2- 172.21.15.92:0/2764788046 >> v2:172.21.15.118:3300/0 conn(0x7fb5ec13e360 0x7fb5ec0cf290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:50.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:50.030+0000 7fb5f0f53640 1 -- 172.21.15.92:0/2764788046 >> 172.21.15.92:0/2764788046 conn(0x7fb5ec06a800 msgr2=0x7fb5ec06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:13:50.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:50.030+0000 7fb5f0f53640 1 -- 172.21.15.92:0/2764788046 shutdown_connections 2023-11-24T06:13:50.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:50.030+0000 7fb5f0f53640 1 -- 172.21.15.92:0/2764788046 wait complete. 2023-11-24T06:13:50.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:50.032+0000 7fb5f0f53640 1 Processor -- start 2023-11-24T06:13:50.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:50.032+0000 7fb5f0f53640 1 -- start start 2023-11-24T06:13:50.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:50.034+0000 7fb5f0f53640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb5ec13df80 0x7fb5ec136c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:50.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:50.034+0000 7fb5f0f53640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb5ec13e360 0x7fb5ec14b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:50.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:50.034+0000 7fb5f0f53640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb5ec15f9c0 0x7fb5ec0b3290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:50.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:50.034+0000 7fb5f0f53640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5ec14df50 con 0x7fb5ec13e360 2023-11-24T06:13:50.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:50.034+0000 7fb5f0f53640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5ec14e230 con 0x7fb5ec13df80 2023-11-24T06:13:50.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:50.034+0000 7fb5f0f53640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5ec14e510 con 0x7fb5ec15f9c0 2023-11-24T06:13:50.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:50.034+0000 7fb5e1d9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb5ec13e360 0x7fb5ec14b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:50.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:50.034+0000 7fb5e1d9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb5ec13e360 0x7fb5ec14b340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:54340/0 (socket says 172.21.15.92:54340) 2023-11-24T06:13:50.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:50.034+0000 7fb5e1d9b640 1 -- 172.21.15.92:0/3948939212 learned_addr learned my addr 172.21.15.92:0/3948939212 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:13:50.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:50.034+0000 7fb5ea59c640 1 --2- 172.21.15.92:0/3948939212 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb5ec13df80 0x7fb5ec136c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:50.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:50.034+0000 7fb5e1d9b640 1 -- 172.21.15.92:0/3948939212 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb5ec15f9c0 msgr2=0x7fb5ec0b3290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:50.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:50.034+0000 7fb5ead9d640 1 --2- 172.21.15.92:0/3948939212 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb5ec15f9c0 0x7fb5ec0b3290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:50.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:50.034+0000 7fb5e1d9b640 1 --2- 172.21.15.92:0/3948939212 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb5ec15f9c0 0x7fb5ec0b3290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:50.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:50.034+0000 7fb5e1d9b640 1 -- 172.21.15.92:0/3948939212 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb5ec13df80 msgr2=0x7fb5ec136c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:50.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:50.034+0000 7fb5e1d9b640 1 --2- 172.21.15.92:0/3948939212 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb5ec13df80 0x7fb5ec136c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:50.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:50.034+0000 7fb5e1d9b640 1 -- 172.21.15.92:0/3948939212 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5e4009b30 con 0x7fb5ec13e360 2023-11-24T06:13:50.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:50.034+0000 7fb5ea59c640 1 --2- 172.21.15.92:0/3948939212 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb5ec13df80 0x7fb5ec136c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:13:50.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:50.035+0000 7fb5e1d9b640 1 --2- 172.21.15.92:0/3948939212 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb5ec13e360 0x7fb5ec14b340 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7fb5d400b450 tx=0x7fb5d400b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:50.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:50.035+0000 7fb5e37fe640 1 -- 172.21.15.92:0/3948939212 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5d4011890 con 0x7fb5ec13e360 2023-11-24T06:13:50.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:50.035+0000 7fb5e37fe640 1 -- 172.21.15.92:0/3948939212 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5d40119f0 con 0x7fb5ec13e360 2023-11-24T06:13:50.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:50.035+0000 7fb5e37fe640 1 -- 172.21.15.92:0/3948939212 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5d4011da0 con 0x7fb5ec13e360 2023-11-24T06:13:50.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:50.035+0000 7fb5f0f53640 1 -- 172.21.15.92:0/3948939212 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5ec14e7f0 con 0x7fb5ec13e360 2023-11-24T06:13:50.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:50.035+0000 7fb5f0f53640 1 -- 172.21.15.92:0/3948939212 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5ec137150 con 0x7fb5ec13e360 2023-11-24T06:13:50.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:50.037+0000 7fb5f0f53640 1 -- 172.21.15.92:0/3948939212 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5ec14ed10 con 0x7fb5ec13e360 2023-11-24T06:13:50.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:50.037+0000 7fb5e37fe640 1 -- 172.21.15.92:0/3948939212 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 7) v1 ==== 89333+0+0 (secure 0 0 0) 0x7fb5d401a400 con 0x7fb5ec13e360 2023-11-24T06:13:50.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:50.038+0000 7fb5e37fe640 1 --2- 172.21.15.92:0/3948939212 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fb5cc06af60 0x7fb5cc06d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:13:50.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:50.038+0000 7fb5e37fe640 1 -- 172.21.15.92:0/3948939212 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fb5d4089860 con 0x7fb5ec13e360 2023-11-24T06:13:50.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:50.038+0000 7fb5ea59c640 1 --2- 172.21.15.92:0/3948939212 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fb5cc06af60 0x7fb5cc06d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:13:50.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:50.039+0000 7fb5ea59c640 1 --2- 172.21.15.92:0/3948939212 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fb5cc06af60 0x7fb5cc06d420 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fb5dc00a810 tx=0x7fb5dc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:13:50.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:50.042+0000 7fb5e37fe640 1 -- 172.21.15.92:0/3948939212 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb5d4054b00 con 0x7fb5ec13e360 2023-11-24T06:13:50.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:50.229+0000 7fb5f0f53640 1 -- 172.21.15.92:0/3948939212 --> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb5ec06cff0 con 0x7fb5cc06af60 2023-11-24T06:13:50.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:50.234+0000 7fb5e37fe640 1 -- 172.21.15.92:0/3948939212 <== mgr.4101 v2:172.21.15.92:6800/784027401 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+48558 (secure 0 0 0) 0x7fb5dc00ad90 con 0x7fb5cc06af60 2023-11-24T06:13:50.238 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-24T06:13:50.239 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2023-11-24T06:13:50.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:50.237+0000 7fb5f0f53640 1 -- 172.21.15.92:0/3948939212 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fb5cc06af60 msgr2=0x7fb5cc06d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:50.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:50.237+0000 7fb5f0f53640 1 --2- 172.21.15.92:0/3948939212 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fb5cc06af60 0x7fb5cc06d420 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fb5dc00a810 tx=0x7fb5dc008040 comp rx=0 tx=0).stop 2023-11-24T06:13:50.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:50.237+0000 7fb5f0f53640 1 -- 172.21.15.92:0/3948939212 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb5ec13e360 msgr2=0x7fb5ec14b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:13:50.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:50.237+0000 7fb5f0f53640 1 --2- 172.21.15.92:0/3948939212 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb5ec13e360 0x7fb5ec14b340 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7fb5d400b450 tx=0x7fb5d400b910 comp rx=0 tx=0).stop 2023-11-24T06:13:50.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:50.237+0000 7fb5f0f53640 1 -- 172.21.15.92:0/3948939212 shutdown_connections 2023-11-24T06:13:50.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:50.237+0000 7fb5f0f53640 1 --2- 172.21.15.92:0/3948939212 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7fb5cc06af60 0x7fb5cc06d420 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:50.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:50.237+0000 7fb5f0f53640 1 --2- 172.21.15.92:0/3948939212 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fb5ec15f9c0 0x7fb5ec0b3290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:50.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:50.237+0000 7fb5f0f53640 1 --2- 172.21.15.92:0/3948939212 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb5ec13e360 0x7fb5ec14b340 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:50.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:50.237+0000 7fb5f0f53640 1 --2- 172.21.15.92:0/3948939212 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb5ec13df80 0x7fb5ec136c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:13:50.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:50.237+0000 7fb5f0f53640 1 -- 172.21.15.92:0/3948939212 >> 172.21.15.92:0/3948939212 conn(0x7fb5ec06a800 msgr2=0x7fb5ec15dd70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:13:50.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:50.238+0000 7fb5f0f53640 1 -- 172.21.15.92:0/3948939212 shutdown_connections 2023-11-24T06:13:50.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:13:50.238+0000 7fb5f0f53640 1 -- 172.21.15.92:0/3948939212 wait complete. 2023-11-24T06:13:50.259 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":165,"stamp":"2023-11-24T06:13:49.729403+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":248632,"kb_used_data":7488,"kb_used_omap":12,"kb_used_meta":241011,"kb_avail":754726088,"statfs":{"total":773094113280,"available":772839514112,"internally_reserved":0,"allocated":7667712,"data_stored":3946676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":246795825},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.502096"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":205,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-24T06:13:38.632118+0000","last_change":"2023-11-24T06:10:20.082677+0000","last_active":"2023-11-24T06:13:38.632118+0000","last_peered":"2023-11-24T06:13:38.632118+0000","last_clean":"2023-11-24T06:13:38.632118+0000","last_became_active":"2023-11-24T06:10:20.080551+0000","last_became_peered":"2023-11-24T06:10:20.080551+0000","last_unstale":"2023-11-24T06:13:38.632118+0000","last_undegraded":"2023-11-24T06:13:38.632118+0000","last_fullsized":"2023-11-24T06:13:38.632118+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:18.602209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:18.602209+0000","last_clean_scrub_stamp":"2023-11-24T06:10:18.602209+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:47:24.216235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574892,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32552,"kb_used_data":1372,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339288,"statfs":{"total":96636764160,"available":96603430912,"internally_reserved":0,"allocated":1404928,"data_stored":936264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:13:18 2023","interfaces":[{"interface":"back","average":{"1min":0.99199999999999999,"5min":1.0669999999999999,"15min":1.095},"min":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":1.581,"5min":1.9630000000000001,"15min":1.9630000000000001},"last":0.94299999999999995},{"interface":"front","average":{"1min":1.0640000000000001,"5min":1.0569999999999999,"15min":1.0569999999999999},"min":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.413,"5min":1.621,"15min":1.621},"last":0.871}]},{"osd":1,"last update":"Fri Nov 24 06:13:18 2023","interfaces":[{"interface":"back","average":{"1min":1.0620000000000001,"5min":1.113,"15min":1.121},"min":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":1.569,"5min":2.0259999999999998,"15min":2.0259999999999998},"last":1.0700000000000001},{"interface":"front","average":{"1min":1.274,"5min":1.1779999999999999,"15min":1.1459999999999999},"min":{"1min":0.72799999999999998,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.6579999999999999,"5min":2.1480000000000001,"15min":2.1480000000000001},"last":0.73099999999999998}]},{"osd":2,"last update":"Fri Nov 24 06:13:18 2023","interfaces":[{"interface":"back","average":{"1min":1.1439999999999999,"5min":1.1339999999999999,"15min":1.1399999999999999},"min":{"1min":0.72399999999999998,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.4770000000000001,"5min":1.7809999999999999,"15min":1.7809999999999999},"last":1.036},{"interface":"front","average":{"1min":1.222,"5min":1.1950000000000001,"15min":1.179},"min":{"1min":0.88100000000000001,"5min":0.86599999999999999,"15min":0.86599999999999999},"max":{"1min":1.599,"5min":2.0619999999999998,"15min":2.0619999999999998},"last":1.155}]},{"osd":3,"last update":"Fri Nov 24 06:13:18 2023","interfaces":[{"interface":"back","average":{"1min":1.077,"5min":1.1779999999999999,"15min":1.196},"min":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":1.4339999999999999,"5min":1.615,"15min":1.615},"last":1.1859999999999999},{"interface":"front","average":{"1min":1.28,"5min":1.3,"15min":1.3069999999999999},"min":{"1min":0.82699999999999996,"5min":0.81999999999999995,"15min":0.81999999999999995},"max":{"1min":1.778,"5min":1.9570000000000001,"15min":1.9570000000000001},"last":1.2549999999999999}]},{"osd":4,"last update":"Fri Nov 24 06:13:18 2023","interfaces":[{"interface":"back","average":{"1min":1.379,"5min":1.3220000000000001,"15min":1.3},"min":{"1min":0.60399999999999998,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":2.1739999999999999,"5min":2.1739999999999999,"15min":2.1739999999999999},"last":1.113},{"interface":"front","average":{"1min":1.2829999999999999,"5min":1.3200000000000001,"15min":1.3260000000000001},"min":{"1min":0.621,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.7809999999999999,"5min":2.1680000000000001,"15min":2.1680000000000001},"last":1.3009999999999999}]},{"osd":5,"last update":"Fri Nov 24 06:13:18 2023","interfaces":[{"interface":"back","average":{"1min":1.502,"5min":1.4770000000000001,"15min":1.456},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.964,"5min":2.0379999999999998,"15min":2.0379999999999998},"last":1.2330000000000001},{"interface":"front","average":{"1min":1.4730000000000001,"5min":1.4810000000000001,"15min":1.4630000000000001},"min":{"1min":0.69699999999999995,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.915,"5min":2.0539999999999998,"15min":2.0539999999999998},"last":1.4179999999999999}]},{"osd":6,"last update":"Fri Nov 24 06:13:18 2023","interfaces":[{"interface":"back","average":{"1min":1.647,"5min":1.573,"15min":1.538},"min":{"1min":0.66000000000000003,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":2.1320000000000001,"5min":2.1320000000000001,"15min":2.1320000000000001},"last":1.2849999999999999},{"interface":"front","average":{"1min":1.5649999999999999,"5min":1.5649999999999999,"15min":1.5409999999999999},"min":{"1min":0.75800000000000001,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":2.0310000000000001,"5min":2.0680000000000001,"15min":2.0680000000000001},"last":1.476}]}]},{"osd":6,"up_from":13,"seq":55834574893,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31976,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339864,"statfs":{"total":96636764160,"available":96604020736,"internally_reserved":0,"allocated":811008,"data_stored":345896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:13:24 2023","interfaces":[{"interface":"back","average":{"1min":1.0580000000000001,"5min":1.1619999999999999,"15min":1.2},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.45,"5min":2.032,"15min":2.032},"last":1.0009999999999999},{"interface":"front","average":{"1min":1.0860000000000001,"5min":1.1850000000000001,"15min":1.2110000000000001},"min":{"1min":0.65700000000000003,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.637,"5min":2.0230000000000001,"15min":2.0230000000000001},"last":0.63100000000000001}]},{"osd":1,"last update":"Fri Nov 24 06:13:24 2023","interfaces":[{"interface":"back","average":{"1min":0.97799999999999998,"5min":1.125,"15min":1.153},"min":{"1min":0.60099999999999998,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.571,"5min":2.1499999999999999,"15min":2.1499999999999999},"last":1.605},{"interface":"front","average":{"1min":1.1659999999999999,"5min":1.2130000000000001,"15min":1.2270000000000001},"min":{"1min":0.872,"5min":0.77500000000000002,"15min":0.77500000000000002},"max":{"1min":1.653,"5min":1.8899999999999999,"15min":1.8899999999999999},"last":1.29}]},{"osd":2,"last update":"Fri Nov 24 06:13:24 2023","interfaces":[{"interface":"back","average":{"1min":1.05,"5min":1.1839999999999999,"15min":1.1619999999999999},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.5009999999999999,"5min":4.7359999999999998,"15min":4.7359999999999998},"last":1.056},{"interface":"front","average":{"1min":1.252,"5min":1.413,"15min":1.4299999999999999},"min":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":1.8380000000000001,"5min":3.9700000000000002,"15min":3.9700000000000002},"last":1.121}]},{"osd":3,"last update":"Fri Nov 24 06:13:24 2023","interfaces":[{"interface":"back","average":{"1min":1.274,"5min":1.2989999999999999,"15min":1.3089999999999999},"min":{"1min":0.94999999999999996,"5min":0.78500000000000003,"15min":0.78500000000000003},"max":{"1min":1.6120000000000001,"5min":1.919,"15min":1.919},"last":0.96799999999999997},{"interface":"front","average":{"1min":1.2609999999999999,"5min":1.2849999999999999,"15min":1.292},"min":{"1min":0.77200000000000002,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":1.605,"5min":2.3929999999999998,"15min":2.3929999999999998},"last":1.1919999999999999}]},{"osd":4,"last update":"Fri Nov 24 06:13:24 2023","interfaces":[{"interface":"back","average":{"1min":1.264,"5min":1.417,"15min":1.444},"min":{"1min":0.70699999999999996,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.9610000000000001,"5min":2.4039999999999999,"15min":2.4039999999999999},"last":1.7430000000000001},{"interface":"front","average":{"1min":1.335,"5min":1.4670000000000001,"15min":1.4750000000000001},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.8029999999999999,"5min":2.5609999999999999,"15min":2.5609999999999999},"last":1.5529999999999999}]},{"osd":5,"last update":"Fri Nov 24 06:13:24 2023","interfaces":[{"interface":"back","average":{"1min":1.306,"5min":1.4390000000000001,"15min":1.464},"min":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"max":{"1min":1.9159999999999999,"5min":2.8290000000000002,"15min":2.8290000000000002},"last":1.512},{"interface":"front","average":{"1min":1.4590000000000001,"5min":1.613,"15min":1.6419999999999999},"min":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"max":{"1min":2.0259999999999998,"5min":2.9420000000000002,"15min":2.9420000000000002},"last":1.8959999999999999}]},{"osd":7,"last update":"Fri Nov 24 06:13:24 2023","interfaces":[{"interface":"back","average":{"1min":1.403,"5min":1.5580000000000001,"15min":1.5800000000000001},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.772,"5min":2.5139999999999998,"15min":2.5139999999999998},"last":1.829},{"interface":"front","average":{"1min":1.421,"5min":1.5629999999999999,"15min":1.5529999999999999},"min":{"1min":0.79300000000000004,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":1.992,"5min":4.7859999999999996,"15min":4.7859999999999996},"last":1.6890000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574893,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31972,"kb_used_data":788,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339868,"statfs":{"total":96636764160,"available":96604024832,"internally_reserved":0,"allocated":806912,"data_stored":345282,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:13:25 2023","interfaces":[{"interface":"back","average":{"1min":0.97999999999999998,"5min":1.1339999999999999,"15min":1.212},"min":{"1min":0.62,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.526,"5min":3.0030000000000001,"15min":3.0030000000000001},"last":0.88400000000000001},{"interface":"front","average":{"1min":0.998,"5min":1.1339999999999999,"15min":1.196},"min":{"1min":0.623,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.504,"5min":2.0979999999999999,"15min":2.0979999999999999},"last":1.0529999999999999}]},{"osd":1,"last update":"Fri Nov 24 06:13:25 2023","interfaces":[{"interface":"back","average":{"1min":1.0720000000000001,"5min":1.212,"15min":1.254},"min":{"1min":0.63300000000000001,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.665,"5min":5.0499999999999998,"15min":5.0499999999999998},"last":1.3720000000000001},{"interface":"front","average":{"1min":1.101,"5min":1.3180000000000001,"15min":1.4259999999999999},"min":{"1min":0.72599999999999998,"5min":0.435,"15min":0.435},"max":{"1min":1.5449999999999999,"5min":3.6930000000000001,"15min":3.6930000000000001},"last":1.23}]},{"osd":2,"last update":"Fri Nov 24 06:13:25 2023","interfaces":[{"interface":"back","average":{"1min":1.103,"5min":1.2589999999999999,"15min":1.3160000000000001},"min":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.5960000000000001,"5min":3.085,"15min":3.085},"last":1.298},{"interface":"front","average":{"1min":1.2230000000000001,"5min":1.4510000000000001,"15min":1.5609999999999999},"min":{"1min":0.746,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.5980000000000001,"5min":4.7359999999999998,"15min":4.7359999999999998},"last":1.151}]},{"osd":3,"last update":"Fri Nov 24 06:13:25 2023","interfaces":[{"interface":"back","average":{"1min":1.246,"5min":1.3500000000000001,"15min":1.397},"min":{"1min":0.747,"5min":0.73999999999999999,"15min":0.73999999999999999},"max":{"1min":1.6579999999999999,"5min":3.2229999999999999,"15min":3.2229999999999999},"last":1.4430000000000001},{"interface":"front","average":{"1min":1.333,"5min":1.3919999999999999,"15min":1.427},"min":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"max":{"1min":1.7789999999999999,"5min":2.3260000000000001,"15min":2.3260000000000001},"last":1.498}]},{"osd":4,"last update":"Fri Nov 24 06:13:25 2023","interfaces":[{"interface":"back","average":{"1min":1.333,"5min":1.482,"15min":1.538},"min":{"1min":0.95399999999999996,"5min":0.84299999999999997,"15min":0.84299999999999997},"max":{"1min":1.762,"5min":3.1230000000000002,"15min":3.1230000000000002},"last":1},{"interface":"front","average":{"1min":1.3400000000000001,"5min":1.478,"15min":1.532},"min":{"1min":0.84399999999999997,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.7509999999999999,"5min":2.4809999999999999,"15min":2.4809999999999999},"last":1.4750000000000001}]},{"osd":6,"last update":"Fri Nov 24 06:13:25 2023","interfaces":[{"interface":"back","average":{"1min":1.4630000000000001,"5min":1.6319999999999999,"15min":1.7150000000000001},"min":{"1min":1.2010000000000001,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.887,"5min":4.6529999999999996,"15min":4.6529999999999996},"last":1.2729999999999999},{"interface":"front","average":{"1min":1.5049999999999999,"5min":1.6479999999999999,"15min":1.716},"min":{"1min":1.226,"5min":0.90400000000000003,"15min":0.90400000000000003},"max":{"1min":1.863,"5min":4.3529999999999998,"15min":4.3529999999999998},"last":1.5609999999999999}]},{"osd":7,"last update":"Fri Nov 24 06:13:25 2023","interfaces":[{"interface":"back","average":{"1min":1.4490000000000001,"5min":1.615,"15min":1.679},"min":{"1min":0.98499999999999999,"5min":0.874,"15min":0.874},"max":{"1min":1.847,"5min":2.8839999999999999,"15min":2.8839999999999999},"last":1.5349999999999999},{"interface":"front","average":{"1min":1.579,"5min":1.7509999999999999,"15min":1.8280000000000001},"min":{"1min":1.2709999999999999,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.9770000000000001,"5min":4.8739999999999997,"15min":4.8739999999999997},"last":1.595}]}]},{"osd":4,"up_from":13,"seq":55834574893,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31976,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339864,"statfs":{"total":96636764160,"available":96604020736,"internally_reserved":0,"allocated":811008,"data_stored":345896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:13:24 2023","interfaces":[{"interface":"back","average":{"1min":1.01,"5min":1.052,"15min":1.0449999999999999},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.907,"5min":1.907,"15min":1.907},"last":0.96499999999999997},{"interface":"front","average":{"1min":1.046,"5min":1.1479999999999999,"15min":1.194},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.732,"5min":1.8220000000000001,"15min":1.8220000000000001},"last":1.353}]},{"osd":1,"last update":"Fri Nov 24 06:13:24 2023","interfaces":[{"interface":"back","average":{"1min":1.1990000000000001,"5min":1.141,"15min":1.127},"min":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.655,"5min":1.97,"15min":1.97},"last":0.85799999999999998},{"interface":"front","average":{"1min":1.0920000000000001,"5min":1.161,"15min":1.1719999999999999},"min":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.831,"5min":1.917,"15min":1.917},"last":1.161}]},{"osd":2,"last update":"Fri Nov 24 06:13:24 2023","interfaces":[{"interface":"back","average":{"1min":1.119,"5min":1.194,"15min":1.232},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.744,"5min":2.319,"15min":2.319},"last":0.80100000000000005},{"interface":"front","average":{"1min":1.2809999999999999,"5min":1.3080000000000001,"15min":1.3080000000000001},"min":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":1.956,"5min":2.0990000000000002,"15min":2.0990000000000002},"last":1.419}]},{"osd":3,"last update":"Fri Nov 24 06:13:24 2023","interfaces":[{"interface":"back","average":{"1min":1.1839999999999999,"5min":1.325,"15min":1.383},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":2.2269999999999999,"5min":2.2269999999999999,"15min":2.2269999999999999},"last":1.4399999999999999},{"interface":"front","average":{"1min":1.3180000000000001,"5min":1.395,"15min":1.4259999999999999},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":2.0990000000000002,"5min":2.1720000000000002,"15min":2.1720000000000002},"last":1.2350000000000001}]},{"osd":5,"last update":"Fri Nov 24 06:13:24 2023","interfaces":[{"interface":"back","average":{"1min":1.274,"5min":1.3140000000000001,"15min":1.3300000000000001},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":1.9870000000000001,"5min":1.9870000000000001,"15min":1.9870000000000001},"last":1.0720000000000001},{"interface":"front","average":{"1min":1.3979999999999999,"5min":1.498,"15min":1.534},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":2.2890000000000001,"5min":2.5059999999999998,"15min":2.5059999999999998},"last":1.4770000000000001}]},{"osd":6,"last update":"Fri Nov 24 06:13:24 2023","interfaces":[{"interface":"back","average":{"1min":1.2569999999999999,"5min":1.464,"15min":1.5189999999999999},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.7729999999999999,"5min":2.2069999999999999,"15min":2.2069999999999999},"last":1.504},{"interface":"front","average":{"1min":1.5109999999999999,"5min":1.6299999999999999,"15min":1.6659999999999999},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":2.3679999999999999,"5min":2.4510000000000001,"15min":2.4510000000000001},"last":1.5229999999999999}]},{"osd":7,"last update":"Fri Nov 24 06:13:24 2023","interfaces":[{"interface":"back","average":{"1min":1.369,"5min":1.573,"15min":1.6200000000000001},"min":{"1min":0.61299999999999999,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.8480000000000001,"5min":2.4350000000000001,"15min":2.4350000000000001},"last":1.6850000000000001},{"interface":"front","average":{"1min":1.454,"5min":1.581,"15min":1.615},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":2.173,"5min":3.1120000000000001,"15min":3.1120000000000001},"last":1.288}]}]},{"osd":3,"up_from":13,"seq":55834574893,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27816,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344024,"statfs":{"total":96636764160,"available":96608280576,"internally_reserved":0,"allocated":811008,"data_stored":345896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:13:30 2023","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.95599999999999996,"15min":0.98999999999999999},"min":{"1min":0.41799999999999998,"5min":0.314,"15min":0.314},"max":{"1min":1.3300000000000001,"5min":2.766,"15min":2.766},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.872,"5min":0.93999999999999995,"15min":0.92400000000000004},"min":{"1min":0.503,"5min":0.441,"15min":0.441},"max":{"1min":1.4850000000000001,"5min":1.873,"15min":1.873},"last":1.353}]},{"osd":1,"last update":"Fri Nov 24 06:13:30 2023","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.86499999999999999,"15min":0.878},"min":{"1min":0.434,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.3129999999999999,"5min":2.5779999999999998,"15min":2.5779999999999998},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.96599999999999997,"5min":1.107,"15min":1.1579999999999999},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.49,"5min":3.6899999999999999,"15min":3.6899999999999999},"last":1.1970000000000001}]},{"osd":2,"last update":"Fri Nov 24 06:13:30 2023","interfaces":[{"interface":"back","average":{"1min":1.012,"5min":1.107,"15min":1.153},"min":{"1min":0.53100000000000003,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.431,"5min":4.0620000000000003,"15min":4.0620000000000003},"last":1.264},{"interface":"front","average":{"1min":1.131,"5min":1.1850000000000001,"15min":1.2130000000000001},"min":{"1min":0.58499999999999996,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.659,"5min":2.9529999999999998,"15min":2.9529999999999998},"last":1.2929999999999999}]},{"osd":4,"last update":"Fri Nov 24 06:13:30 2023","interfaces":[{"interface":"back","average":{"1min":1.117,"5min":1.3320000000000001,"15min":1.3959999999999999},"min":{"1min":0.71999999999999997,"5min":0.70999999999999996,"15min":0.70999999999999996},"max":{"1min":1.51,"5min":4.125,"15min":4.125},"last":1.133},{"interface":"front","average":{"1min":1.206,"5min":1.379,"15min":1.4239999999999999},"min":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"max":{"1min":1.633,"5min":3.9409999999999998,"15min":3.9409999999999998},"last":1.4410000000000001}]},{"osd":5,"last update":"Fri Nov 24 06:13:30 2023","interfaces":[{"interface":"back","average":{"1min":1.335,"5min":1.399,"15min":1.431},"min":{"1min":0.79500000000000004,"5min":0.749,"15min":0.749},"max":{"1min":2.0609999999999999,"5min":3.3359999999999999,"15min":3.3359999999999999},"last":1.403},{"interface":"front","average":{"1min":1.1910000000000001,"5min":1.357,"15min":1.425},"min":{"1min":0.78200000000000003,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.7929999999999999,"5min":4.1669999999999998,"15min":4.1669999999999998},"last":1.5049999999999999}]},{"osd":6,"last update":"Fri Nov 24 06:13:30 2023","interfaces":[{"interface":"back","average":{"1min":1.3260000000000001,"5min":1.3720000000000001,"15min":1.415},"min":{"1min":0.74199999999999999,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":2.169,"5min":4.0190000000000001,"15min":4.0190000000000001},"last":1.1719999999999999},{"interface":"front","average":{"1min":1.3759999999999999,"5min":1.48,"15min":1.5149999999999999},"min":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"max":{"1min":2.004,"5min":3.2480000000000002,"15min":3.2480000000000002},"last":1.6040000000000001}]},{"osd":7,"last update":"Fri Nov 24 06:13:30 2023","interfaces":[{"interface":"back","average":{"1min":1.258,"5min":1.4870000000000001,"15min":1.571},"min":{"1min":0.79700000000000004,"5min":0.76600000000000001,"15min":0.76600000000000001},"max":{"1min":1.831,"5min":4.202,"15min":4.202},"last":1.7150000000000001},{"interface":"front","average":{"1min":1.4099999999999999,"5min":1.524,"15min":1.589},"min":{"1min":1.0149999999999999,"5min":0.89700000000000002,"15min":0.89700000000000002},"max":{"1min":2.2989999999999999,"5min":3.9849999999999999,"15min":3.9849999999999999},"last":1.669}]}]},{"osd":2,"up_from":13,"seq":55834574893,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27812,"kb_used_data":788,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344028,"statfs":{"total":96636764160,"available":96608284672,"internally_reserved":0,"allocated":806912,"data_stored":345282,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:13:25 2023","interfaces":[{"interface":"back","average":{"1min":0.88800000000000001,"5min":0.88,"15min":0.88600000000000001},"min":{"1min":0.34000000000000002,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":1.1850000000000001,"5min":1.6259999999999999,"15min":1.6259999999999999},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.90700000000000003,"5min":1.0349999999999999,"15min":1.079},"min":{"1min":0.51300000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.1799999999999999,"5min":1.7669999999999999,"15min":1.7669999999999999},"last":0.45700000000000002}]},{"osd":1,"last update":"Fri Nov 24 06:13:25 2023","interfaces":[{"interface":"back","average":{"1min":0.90700000000000003,"5min":0.94299999999999995,"15min":0.93999999999999995},"min":{"1min":0.38200000000000001,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.294,"5min":1.635,"15min":1.635},"last":0.501},{"interface":"front","average":{"1min":0.91400000000000003,"5min":0.96299999999999997,"15min":0.97499999999999998},"min":{"1min":0.51600000000000001,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.579,"5min":1.8140000000000001,"15min":1.8140000000000001},"last":0.35299999999999998}]},{"osd":3,"last update":"Fri Nov 24 06:13:25 2023","interfaces":[{"interface":"back","average":{"1min":0.88700000000000001,"5min":1.014,"15min":1.0580000000000001},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.3400000000000001,"5min":2.3559999999999999,"15min":2.3559999999999999},"last":0.74299999999999999},{"interface":"front","average":{"1min":1.0489999999999999,"5min":1.1180000000000001,"15min":1.121},"min":{"1min":0.47899999999999998,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.339,"5min":2.1840000000000002,"15min":2.1840000000000002},"last":0.68799999999999994}]},{"osd":4,"last update":"Fri Nov 24 06:13:25 2023","interfaces":[{"interface":"back","average":{"1min":1.228,"5min":1.379,"15min":1.405},"min":{"1min":0.73999999999999999,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.488,"5min":3.411,"15min":3.411},"last":0.88},{"interface":"front","average":{"1min":1.179,"5min":1.4059999999999999,"15min":1.472},"min":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"max":{"1min":1.5409999999999999,"5min":3.7109999999999999,"15min":3.7109999999999999},"last":0.85499999999999998}]},{"osd":5,"last update":"Fri Nov 24 06:13:25 2023","interfaces":[{"interface":"back","average":{"1min":1.1970000000000001,"5min":1.2829999999999999,"15min":1.3109999999999999},"min":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"max":{"1min":1.498,"5min":2.5259999999999998,"15min":2.5259999999999998},"last":1.1499999999999999},{"interface":"front","average":{"1min":1.3100000000000001,"5min":1.3320000000000001,"15min":1.3120000000000001},"min":{"1min":0.999,"5min":0.90400000000000003,"15min":0.90400000000000003},"max":{"1min":1.5409999999999999,"5min":2.1640000000000001,"15min":2.1640000000000001},"last":0.80600000000000005}]},{"osd":6,"last update":"Fri Nov 24 06:13:25 2023","interfaces":[{"interface":"back","average":{"1min":1.2529999999999999,"5min":1.373,"15min":1.429},"min":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"max":{"1min":1.7170000000000001,"5min":2.3039999999999998,"15min":2.3039999999999998},"last":0.72199999999999998},{"interface":"front","average":{"1min":1.218,"5min":1.3939999999999999,"15min":1.444},"min":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":1.474,"5min":2.605,"15min":2.605},"last":0.82699999999999996}]},{"osd":7,"last update":"Fri Nov 24 06:13:25 2023","interfaces":[{"interface":"back","average":{"1min":1.2889999999999999,"5min":1.4870000000000001,"15min":1.5589999999999999},"min":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"max":{"1min":1.7829999999999999,"5min":2.3740000000000001,"15min":2.3740000000000001},"last":0.90300000000000002},{"interface":"front","average":{"1min":1.2689999999999999,"5min":1.3939999999999999,"15min":1.4419999999999999},"min":{"1min":0.89200000000000002,"5min":0.877,"15min":0.877},"max":{"1min":1.6299999999999999,"5min":2.5699999999999998,"15min":2.5699999999999998},"last":1.0189999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574892,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31976,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339864,"statfs":{"total":96636764160,"available":96604020736,"internally_reserved":0,"allocated":811008,"data_stored":345896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:13:21 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.85099999999999998,"15min":0.89900000000000002},"min":{"1min":0.36799999999999999,"5min":0.311,"15min":0.311},"max":{"1min":1.212,"5min":1.9299999999999999,"15min":1.9299999999999999},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.873,"5min":0.89500000000000002,"15min":0.92100000000000004},"min":{"1min":0.35099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":1.339,"5min":1.7669999999999999,"15min":1.7669999999999999},"last":0.97199999999999998}]},{"osd":2,"last update":"Fri Nov 24 06:13:21 2023","interfaces":[{"interface":"back","average":{"1min":0.97499999999999998,"5min":0.95599999999999996,"15min":0.998},"min":{"1min":0.496,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.605,"5min":2.5329999999999999,"15min":2.5329999999999999},"last":0.69999999999999996},{"interface":"front","average":{"1min":1.079,"5min":1.0009999999999999,"15min":1.02},"min":{"1min":0.502,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.7190000000000001,"5min":1.7190000000000001,"15min":1.7190000000000001},"last":1.1020000000000001}]},{"osd":3,"last update":"Fri Nov 24 06:13:21 2023","interfaces":[{"interface":"back","average":{"1min":0.91700000000000004,"5min":0.96399999999999997,"15min":1.0129999999999999},"min":{"1min":0.46800000000000003,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.4119999999999999,"5min":1.841,"15min":1.841},"last":0.84899999999999998},{"interface":"front","average":{"1min":1.1479999999999999,"5min":1.1200000000000001,"15min":1.1439999999999999},"min":{"1min":0.61899999999999999,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.643,"5min":2.8940000000000001,"15min":2.8940000000000001},"last":0.65400000000000003}]},{"osd":4,"last update":"Fri Nov 24 06:13:21 2023","interfaces":[{"interface":"back","average":{"1min":1.3149999999999999,"5min":1.2609999999999999,"15min":1.2749999999999999},"min":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.728,"5min":2.1930000000000001,"15min":2.1930000000000001},"last":1.425},{"interface":"front","average":{"1min":1.1899999999999999,"5min":1.1819999999999999,"15min":1.194},"min":{"1min":0.86799999999999999,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":1.5640000000000001,"5min":1.784,"15min":1.784},"last":1.4059999999999999}]},{"osd":5,"last update":"Fri Nov 24 06:13:21 2023","interfaces":[{"interface":"back","average":{"1min":1.2430000000000001,"5min":1.3959999999999999,"15min":1.4850000000000001},"min":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.732,"5min":5.5449999999999999,"15min":5.5449999999999999},"last":1.385},{"interface":"front","average":{"1min":1.361,"5min":1.444,"15min":1.524},"min":{"1min":0.90900000000000003,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":1.7270000000000001,"5min":6.0030000000000001,"15min":6.0030000000000001},"last":1.1899999999999999}]},{"osd":6,"last update":"Fri Nov 24 06:13:21 2023","interfaces":[{"interface":"back","average":{"1min":1.3220000000000001,"5min":1.446,"15min":1.528},"min":{"1min":0.81100000000000005,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":1.8779999999999999,"5min":5.2809999999999997,"15min":5.2809999999999997},"last":1.292},{"interface":"front","average":{"1min":1.4079999999999999,"5min":1.423,"15min":1.4570000000000001},"min":{"1min":0.82999999999999996,"5min":0.749,"15min":0.749},"max":{"1min":1.819,"5min":4.327,"15min":4.327},"last":1.522}]},{"osd":7,"last update":"Fri Nov 24 06:13:21 2023","interfaces":[{"interface":"back","average":{"1min":1.264,"5min":1.302,"15min":1.3400000000000001},"min":{"1min":0.85099999999999998,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":1.7849999999999999,"5min":2.3069999999999999,"15min":2.3069999999999999},"last":1.0760000000000001},{"interface":"front","average":{"1min":1.415,"5min":1.492,"15min":1.5620000000000001},"min":{"1min":0.91700000000000004,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":2.0179999999999998,"5min":5.4500000000000002,"15min":5.4500000000000002},"last":1.339}]}]},{"osd":0,"up_from":13,"seq":55834574892,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32552,"kb_used_data":1372,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339288,"statfs":{"total":96636764160,"available":96603430912,"internally_reserved":0,"allocated":1404928,"data_stored":936264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Nov 24 06:13:25 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.84799999999999998,"15min":0.90800000000000003},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.145,"5min":2.3999999999999999,"15min":2.3999999999999999},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.75800000000000001,"5min":1.024,"15min":1.133},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":1.54,"5min":4.8289999999999997,"15min":4.8289999999999997},"last":1.0189999999999999}]},{"osd":2,"last update":"Fri Nov 24 06:13:25 2023","interfaces":[{"interface":"back","average":{"1min":0.83799999999999997,"5min":0.97999999999999998,"15min":1.0349999999999999},"min":{"1min":0.42999999999999999,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.4019999999999999,"5min":2.5139999999999998,"15min":2.5139999999999998},"last":1.234},{"interface":"front","average":{"1min":0.79800000000000004,"5min":1.0229999999999999,"15min":1.0820000000000001},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.26,"5min":2.3599999999999999,"15min":2.3599999999999999},"last":0.85899999999999999}]},{"osd":3,"last update":"Fri Nov 24 06:13:25 2023","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":1.056,"15min":1.1020000000000001},"min":{"1min":0.48699999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.5369999999999999,"5min":2.5840000000000001,"15min":2.5840000000000001},"last":1.1459999999999999},{"interface":"front","average":{"1min":0.91600000000000004,"5min":1.246,"15min":1.3580000000000001},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.4630000000000001,"5min":5.1230000000000002,"15min":5.1230000000000002},"last":0.47999999999999998}]},{"osd":4,"last update":"Fri Nov 24 06:13:25 2023","interfaces":[{"interface":"back","average":{"1min":1.0329999999999999,"5min":1.1799999999999999,"15min":1.248},"min":{"1min":0.70499999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.488,"5min":2.887,"15min":2.887},"last":0.77700000000000002},{"interface":"front","average":{"1min":1.0720000000000001,"5min":1.2310000000000001,"15min":1.3009999999999999},"min":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.581,"5min":2.8530000000000002,"15min":2.8530000000000002},"last":0.70199999999999996}]},{"osd":5,"last update":"Fri Nov 24 06:13:25 2023","interfaces":[{"interface":"back","average":{"1min":1.149,"5min":1.4219999999999999,"15min":1.524},"min":{"1min":0.72199999999999998,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.978,"5min":4.6299999999999999,"15min":4.6299999999999999},"last":0.92800000000000005},{"interface":"front","average":{"1min":1.1060000000000001,"5min":1.232,"15min":1.2989999999999999},"min":{"1min":0.85399999999999998,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.7010000000000001,"5min":2.794,"15min":2.794},"last":0.80900000000000005}]},{"osd":6,"last update":"Fri Nov 24 06:13:25 2023","interfaces":[{"interface":"back","average":{"1min":1.244,"5min":1.508,"15min":1.617},"min":{"1min":0.73899999999999999,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.8620000000000001,"5min":4.798,"15min":4.798},"last":1.3300000000000001},{"interface":"front","average":{"1min":1.3100000000000001,"5min":1.5189999999999999,"15min":1.597},"min":{"1min":0.97099999999999997,"5min":0.77800000000000002,"15min":0.77800000000000002},"max":{"1min":1.841,"5min":4.7149999999999999,"15min":4.7149999999999999},"last":0.996}]},{"osd":7,"last update":"Fri Nov 24 06:13:25 2023","interfaces":[{"interface":"back","average":{"1min":1.2629999999999999,"5min":1.544,"15min":1.6579999999999999},"min":{"1min":0.85099999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.9770000000000001,"5min":4.7649999999999997,"15min":4.7649999999999997},"last":1.4079999999999999},{"interface":"front","average":{"1min":1.3,"5min":1.5609999999999999,"15min":1.6499999999999999},"min":{"1min":0.89900000000000002,"5min":0.84599999999999997,"15min":0.84599999999999997},"max":{"1min":1.9159999999999999,"5min":4.6820000000000004,"15min":4.6820000000000004},"last":0.95999999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-24T06:13:50.263 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2023-11-24T06:10:18.602209+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=24, tm_hour=6, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=328, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=24, tm_hour=6, tm_min=13, tm_sec=43, tm_wday=4, tm_yday=328, tm_isdst=0) 2023-11-24T06:13:50.263 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-11-24T06:14:10.264 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-24T06:14:10.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:10.486+0000 7f4ec65ff640 1 Processor -- start 2023-11-24T06:14:10.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:10.486+0000 7f4ec65ff640 1 -- start start 2023-11-24T06:14:10.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:10.487+0000 7f4ec65ff640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4ec015afb0 0x7f4ec013f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:14:10.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:10.487+0000 7f4ec65ff640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f4ec013fae0 0x7f4ec00b2040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:14:10.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:10.487+0000 7f4ec65ff640 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ec013ff00 con 0x7f4ec013cfa0 2023-11-24T06:14:10.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:10.487+0000 7f4ec65ff640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ec00cdc40 con 0x7f4ec013fae0 2023-11-24T06:14:10.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:10.487+0000 7f4ec65ff640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ec00cdec0 con 0x7f4ec015afb0 2023-11-24T06:14:10.492 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:10.488+0000 7f4ec5dfe640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f4ec013fae0 0x7f4ec00b2040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:14:10.492 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:10.488+0000 7f4ec5dfe640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f4ec013fae0 0x7f4ec00b2040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39278/0 (socket says 172.21.15.92:39278) 2023-11-24T06:14:10.492 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:10.488+0000 7f4ec5dfe640 1 -- 172.21.15.92:0/1825958857 learned_addr learned my addr 172.21.15.92:0/1825958857 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:14:10.492 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:10.488+0000 7f4ec5dfe640 1 -- 172.21.15.92:0/1825958857 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4ec015afb0 msgr2=0x7f4ec013f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:14:10.492 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:10.488+0000 7f4ec5dfe640 1 --2- 172.21.15.92:0/1825958857 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4ec015afb0 0x7f4ec013f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:14:10.492 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:10.488+0000 7f4ec5dfe640 1 -- 172.21.15.92:0/1825958857 >> v1:172.21.15.118:6789/0 conn(0x7f4ec013cfa0 legacy=0x7f4ec00cd530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:14:10.492 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:10.488+0000 7f4ec5dfe640 1 -- 172.21.15.92:0/1825958857 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ec00ce1a0 con 0x7f4ec013fae0 2023-11-24T06:14:10.493 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:10.489+0000 7f4ec5dfe640 1 --2- 172.21.15.92:0/1825958857 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4ec013fae0 0x7f4ec00b2040 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f4eb8009e50 tx=0x7f4eb802e050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72a62b462c2f1214 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:14:10.494 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:10.490+0000 7f4eb7fff640 1 -- 172.21.15.92:0/1825958857 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4eb802ebf0 con 0x7f4ec013fae0 2023-11-24T06:14:10.494 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:10.490+0000 7f4eb7fff640 1 -- 172.21.15.92:0/1825958857 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4eb802ed50 con 0x7f4ec013fae0 2023-11-24T06:14:10.494 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:10.490+0000 7f4eb7fff640 1 -- 172.21.15.92:0/1825958857 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4eb8036610 con 0x7f4ec013fae0 2023-11-24T06:14:10.494 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:10.490+0000 7f4ec65ff640 1 -- 172.21.15.92:0/1825958857 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4ec013fae0 msgr2=0x7f4ec00b2040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:14:10.494 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:10.490+0000 7f4ec65ff640 1 --2- 172.21.15.92:0/1825958857 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4ec013fae0 0x7f4ec00b2040 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f4eb8009e50 tx=0x7f4eb802e050 comp rx=0 tx=0).stop 2023-11-24T06:14:10.494 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:10.490+0000 7f4ec65ff640 1 -- 172.21.15.92:0/1825958857 shutdown_connections 2023-11-24T06:14:10.494 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:10.490+0000 7f4ec65ff640 1 --2- 172.21.15.92:0/1825958857 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4ec013fae0 0x7f4ec00b2040 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:14:10.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:10.490+0000 7f4ec65ff640 1 --2- 172.21.15.92:0/1825958857 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4ec015afb0 0x7f4ec013f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:14:10.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:10.490+0000 7f4ec65ff640 1 -- 172.21.15.92:0/1825958857 >> 172.21.15.92:0/1825958857 conn(0x7f4ec006a800 msgr2=0x7f4ec006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:14:10.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:10.491+0000 7f4ec65ff640 1 -- 172.21.15.92:0/1825958857 shutdown_connections 2023-11-24T06:14:10.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:10.491+0000 7f4ec65ff640 1 -- 172.21.15.92:0/1825958857 wait complete. 2023-11-24T06:14:10.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:10.492+0000 7f4ec65ff640 1 Processor -- start 2023-11-24T06:14:10.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:10.493+0000 7f4ec65ff640 1 -- start start 2023-11-24T06:14:10.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:10.494+0000 7f4ec65ff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4ec013cfa0 0x7f4ec00ae040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:14:10.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:10.494+0000 7f4ec65ff640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4ec013fae0 0x7f4ec0153130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:14:10.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:10.494+0000 7f4ec65ff640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4ec015afb0 0x7f4ec00712e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:14:10.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:10.494+0000 7f4ec65ff640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ec01566f0 con 0x7f4ec013cfa0 2023-11-24T06:14:10.498 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:10.494+0000 7f4ec65ff640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ec01569d0 con 0x7f4ec015afb0 2023-11-24T06:14:10.498 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:10.494+0000 7f4ec65ff640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ec0156cb0 con 0x7f4ec013fae0 2023-11-24T06:14:10.498 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:10.494+0000 7f4ec55fd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4ec013cfa0 0x7f4ec00ae040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:14:10.498 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:10.494+0000 7f4ec55fd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4ec013cfa0 0x7f4ec00ae040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39284/0 (socket says 172.21.15.92:39284) 2023-11-24T06:14:10.498 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:10.494+0000 7f4ec55fd640 1 -- 172.21.15.92:0/4037022501 learned_addr learned my addr 172.21.15.92:0/4037022501 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:14:10.498 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:10.494+0000 7f4ec4dfc640 1 --2- 172.21.15.92:0/4037022501 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4ec013fae0 0x7f4ec0153130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:14:10.498 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:10.494+0000 7f4ec55fd640 1 -- 172.21.15.92:0/4037022501 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4ec013fae0 msgr2=0x7f4ec0153130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:14:10.498 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:10.494+0000 7f4ec55fd640 1 --2- 172.21.15.92:0/4037022501 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4ec013fae0 0x7f4ec0153130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:14:10.499 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:10.494+0000 7f4ec55fd640 1 -- 172.21.15.92:0/4037022501 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4ec015afb0 msgr2=0x7f4ec00712e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:14:10.499 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:10.494+0000 7f4ec55fd640 1 --2- 172.21.15.92:0/4037022501 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4ec015afb0 0x7f4ec00712e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:14:10.499 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:10.494+0000 7f4ec55fd640 1 -- 172.21.15.92:0/4037022501 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4eb8009b30 con 0x7f4ec013cfa0 2023-11-24T06:14:10.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:10.495+0000 7f4ec55fd640 1 --2- 172.21.15.92:0/4037022501 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4ec013cfa0 0x7f4ec00ae040 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f4ebc00c410 tx=0x7f4ebc00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:14:10.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:10.495+0000 7f4eb5ffb640 1 -- 172.21.15.92:0/4037022501 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ebc0035f0 con 0x7f4ec013cfa0 2023-11-24T06:14:10.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:10.495+0000 7f4eb5ffb640 1 -- 172.21.15.92:0/4037022501 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ebc003750 con 0x7f4ec013cfa0 2023-11-24T06:14:10.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:10.495+0000 7f4eb5ffb640 1 -- 172.21.15.92:0/4037022501 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ebc003b40 con 0x7f4ec013cfa0 2023-11-24T06:14:10.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:10.495+0000 7f4ec65ff640 1 -- 172.21.15.92:0/4037022501 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ec0156f90 con 0x7f4ec013cfa0 2023-11-24T06:14:10.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:10.495+0000 7f4ec65ff640 1 -- 172.21.15.92:0/4037022501 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ec0153670 con 0x7f4ec013cfa0 2023-11-24T06:14:10.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:10.496+0000 7f4ec65ff640 1 -- 172.21.15.92:0/4037022501 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ec01574b0 con 0x7f4ec013cfa0 2023-11-24T06:14:10.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:10.497+0000 7f4eb5ffb640 1 -- 172.21.15.92:0/4037022501 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 7) v1 ==== 89333+0+0 (secure 0 0 0) 0x7f4ebc003ca0 con 0x7f4ec013cfa0 2023-11-24T06:14:10.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:10.498+0000 7f4eb5ffb640 1 --2- 172.21.15.92:0/4037022501 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f4ea406af60 0x7f4ea406d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:14:10.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:10.498+0000 7f4eb5ffb640 1 -- 172.21.15.92:0/4037022501 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f4ebc0886b0 con 0x7f4ec013cfa0 2023-11-24T06:14:10.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:10.499+0000 7f4ec4dfc640 1 --2- 172.21.15.92:0/4037022501 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f4ea406af60 0x7f4ea406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:14:10.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:10.499+0000 7f4ec4dfc640 1 --2- 172.21.15.92:0/4037022501 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f4ea406af60 0x7f4ea406d420 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f4ec013cd00 tx=0x7f4eb0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:14:10.506 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:10.503+0000 7f4eb5ffb640 1 -- 172.21.15.92:0/4037022501 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4ec01574b0 con 0x7f4ec013cfa0 2023-11-24T06:14:10.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:10.695+0000 7f4ec65ff640 1 -- 172.21.15.92:0/4037022501 --> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f4ec00cea40 con 0x7f4ea406af60 2023-11-24T06:14:10.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:10.700+0000 7f4eb5ffb640 1 -- 172.21.15.92:0/4037022501 <== mgr.4101 v2:172.21.15.92:6800/784027401 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+48412 (secure 0 0 0) 0x7f4eb000c4f0 con 0x7f4ea406af60 2023-11-24T06:14:10.704 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-24T06:14:10.704 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2023-11-24T06:14:10.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:10.703+0000 7f4ec65ff640 1 -- 172.21.15.92:0/4037022501 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f4ea406af60 msgr2=0x7f4ea406d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:14:10.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:10.703+0000 7f4ec65ff640 1 --2- 172.21.15.92:0/4037022501 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f4ea406af60 0x7f4ea406d420 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f4ec013cd00 tx=0x7f4eb0008040 comp rx=0 tx=0).stop 2023-11-24T06:14:10.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:10.703+0000 7f4ec65ff640 1 -- 172.21.15.92:0/4037022501 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4ec013cfa0 msgr2=0x7f4ec00ae040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:14:10.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:10.703+0000 7f4ec65ff640 1 --2- 172.21.15.92:0/4037022501 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4ec013cfa0 0x7f4ec00ae040 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f4ebc00c410 tx=0x7f4ebc00c8d0 comp rx=0 tx=0).stop 2023-11-24T06:14:10.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:10.703+0000 7f4ec65ff640 1 -- 172.21.15.92:0/4037022501 shutdown_connections 2023-11-24T06:14:10.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:10.703+0000 7f4ec65ff640 1 --2- 172.21.15.92:0/4037022501 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f4ea406af60 0x7f4ea406d420 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:14:10.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:10.703+0000 7f4ec65ff640 1 --2- 172.21.15.92:0/4037022501 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4ec015afb0 0x7f4ec00712e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:14:10.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:10.703+0000 7f4ec65ff640 1 --2- 172.21.15.92:0/4037022501 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4ec013fae0 0x7f4ec0153130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:14:10.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:10.703+0000 7f4ec65ff640 1 --2- 172.21.15.92:0/4037022501 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4ec013cfa0 0x7f4ec00ae040 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:14:10.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:10.703+0000 7f4ec65ff640 1 -- 172.21.15.92:0/4037022501 >> 172.21.15.92:0/4037022501 conn(0x7f4ec006a800 msgr2=0x7f4ec006abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:14:10.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:10.703+0000 7f4ec65ff640 1 -- 172.21.15.92:0/4037022501 shutdown_connections 2023-11-24T06:14:10.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:10.703+0000 7f4ec65ff640 1 -- 172.21.15.92:0/4037022501 wait complete. 2023-11-24T06:14:10.726 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":175,"stamp":"2023-11-24T06:14:09.735357+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":248624,"kb_used_data":7488,"kb_used_omap":12,"kb_used_meta":241011,"kb_avail":754726096,"statfs":{"total":773094113280,"available":772839522304,"internally_reserved":0,"allocated":7667712,"data_stored":3946676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":246795825},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003505"},"pg_stats":[{"pgid":"1.0","version":"66'119","reported_seq":216,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-24T06:13:53.279779+0000","last_change":"2023-11-24T06:13:53.279855+0000","last_active":"2023-11-24T06:13:53.279779+0000","last_peered":"2023-11-24T06:13:53.279779+0000","last_clean":"2023-11-24T06:13:53.279779+0000","last_became_active":"2023-11-24T06:10:20.080551+0000","last_became_peered":"2023-11-24T06:10:20.080551+0000","last_unstale":"2023-11-24T06:13:53.279779+0000","last_undegraded":"2023-11-24T06:13:53.279779+0000","last_fullsized":"2023-11-24T06:13:53.279779+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'119","last_scrub_stamp":"2023-11-24T06:13:53.279603+0000","last_deep_scrub":"66'119","last_deep_scrub_stamp":"2023-11-24T06:13:53.279603+0000","last_clean_scrub_stamp":"2023-11-24T06:13:53.279603+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:25:23.520852+0000","scrub_duration":0.013615082000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574896,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32548,"kb_used_data":1372,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339292,"statfs":{"total":96636764160,"available":96603435008,"internally_reserved":0,"allocated":1404928,"data_stored":936264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:13:18 2023","interfaces":[{"interface":"back","average":{"1min":0.99199999999999999,"5min":1.0669999999999999,"15min":1.095},"min":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":1.581,"5min":1.9630000000000001,"15min":1.9630000000000001},"last":0.85999999999999999},{"interface":"front","average":{"1min":1.0640000000000001,"5min":1.0569999999999999,"15min":1.0569999999999999},"min":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.413,"5min":1.621,"15min":1.621},"last":1.2070000000000001}]},{"osd":1,"last update":"Fri Nov 24 06:13:18 2023","interfaces":[{"interface":"back","average":{"1min":1.0620000000000001,"5min":1.113,"15min":1.121},"min":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":1.569,"5min":2.0259999999999998,"15min":2.0259999999999998},"last":1.038},{"interface":"front","average":{"1min":1.274,"5min":1.1779999999999999,"15min":1.1459999999999999},"min":{"1min":0.72799999999999998,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.6579999999999999,"5min":2.1480000000000001,"15min":2.1480000000000001},"last":0.94799999999999995}]},{"osd":2,"last update":"Fri Nov 24 06:13:18 2023","interfaces":[{"interface":"back","average":{"1min":1.1439999999999999,"5min":1.1339999999999999,"15min":1.1399999999999999},"min":{"1min":0.72399999999999998,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.4770000000000001,"5min":1.7809999999999999,"15min":1.7809999999999999},"last":1.296},{"interface":"front","average":{"1min":1.222,"5min":1.1950000000000001,"15min":1.179},"min":{"1min":0.88100000000000001,"5min":0.86599999999999999,"15min":0.86599999999999999},"max":{"1min":1.599,"5min":2.0619999999999998,"15min":2.0619999999999998},"last":1.151}]},{"osd":3,"last update":"Fri Nov 24 06:13:18 2023","interfaces":[{"interface":"back","average":{"1min":1.077,"5min":1.1779999999999999,"15min":1.196},"min":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":1.4339999999999999,"5min":1.615,"15min":1.615},"last":1.4159999999999999},{"interface":"front","average":{"1min":1.28,"5min":1.3,"15min":1.3069999999999999},"min":{"1min":0.82699999999999996,"5min":0.81999999999999995,"15min":0.81999999999999995},"max":{"1min":1.778,"5min":1.9570000000000001,"15min":1.9570000000000001},"last":1.258}]},{"osd":4,"last update":"Fri Nov 24 06:13:18 2023","interfaces":[{"interface":"back","average":{"1min":1.379,"5min":1.3220000000000001,"15min":1.3},"min":{"1min":0.60399999999999998,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":2.1739999999999999,"5min":2.1739999999999999,"15min":2.1739999999999999},"last":0.75700000000000001},{"interface":"front","average":{"1min":1.2829999999999999,"5min":1.3200000000000001,"15min":1.3260000000000001},"min":{"1min":0.621,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.7809999999999999,"5min":2.1680000000000001,"15min":2.1680000000000001},"last":1.4850000000000001}]},{"osd":5,"last update":"Fri Nov 24 06:13:18 2023","interfaces":[{"interface":"back","average":{"1min":1.502,"5min":1.4770000000000001,"15min":1.456},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.964,"5min":2.0379999999999998,"15min":2.0379999999999998},"last":1.4510000000000001},{"interface":"front","average":{"1min":1.4730000000000001,"5min":1.4810000000000001,"15min":1.4630000000000001},"min":{"1min":0.69699999999999995,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.915,"5min":2.0539999999999998,"15min":2.0539999999999998},"last":1.605}]},{"osd":6,"last update":"Fri Nov 24 06:13:18 2023","interfaces":[{"interface":"back","average":{"1min":1.647,"5min":1.573,"15min":1.538},"min":{"1min":0.66000000000000003,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":2.1320000000000001,"5min":2.1320000000000001,"15min":2.1320000000000001},"last":1.3560000000000001},{"interface":"front","average":{"1min":1.5649999999999999,"5min":1.5649999999999999,"15min":1.5409999999999999},"min":{"1min":0.75800000000000001,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":2.0310000000000001,"5min":2.0680000000000001,"15min":2.0680000000000001},"last":1.5549999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574897,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31976,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339864,"statfs":{"total":96636764160,"available":96604020736,"internally_reserved":0,"allocated":811008,"data_stored":345896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:13:24 2023","interfaces":[{"interface":"back","average":{"1min":1.0580000000000001,"5min":1.1619999999999999,"15min":1.2},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.45,"5min":2.032,"15min":2.032},"last":1.482},{"interface":"front","average":{"1min":1.0860000000000001,"5min":1.1850000000000001,"15min":1.2110000000000001},"min":{"1min":0.65700000000000003,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.637,"5min":2.0230000000000001,"15min":2.0230000000000001},"last":1.3089999999999999}]},{"osd":1,"last update":"Fri Nov 24 06:13:24 2023","interfaces":[{"interface":"back","average":{"1min":0.97799999999999998,"5min":1.125,"15min":1.153},"min":{"1min":0.60099999999999998,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.571,"5min":2.1499999999999999,"15min":2.1499999999999999},"last":1.3480000000000001},{"interface":"front","average":{"1min":1.1659999999999999,"5min":1.2130000000000001,"15min":1.2270000000000001},"min":{"1min":0.872,"5min":0.77500000000000002,"15min":0.77500000000000002},"max":{"1min":1.653,"5min":1.8899999999999999,"15min":1.8899999999999999},"last":0.97399999999999998}]},{"osd":2,"last update":"Fri Nov 24 06:13:24 2023","interfaces":[{"interface":"back","average":{"1min":1.05,"5min":1.1839999999999999,"15min":1.1619999999999999},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.5009999999999999,"5min":4.7359999999999998,"15min":4.7359999999999998},"last":1.272},{"interface":"front","average":{"1min":1.252,"5min":1.413,"15min":1.4299999999999999},"min":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":1.8380000000000001,"5min":3.9700000000000002,"15min":3.9700000000000002},"last":1.583}]},{"osd":3,"last update":"Fri Nov 24 06:13:24 2023","interfaces":[{"interface":"back","average":{"1min":1.274,"5min":1.2989999999999999,"15min":1.3089999999999999},"min":{"1min":0.94999999999999996,"5min":0.78500000000000003,"15min":0.78500000000000003},"max":{"1min":1.6120000000000001,"5min":1.919,"15min":1.919},"last":1.2170000000000001},{"interface":"front","average":{"1min":1.2609999999999999,"5min":1.2849999999999999,"15min":1.292},"min":{"1min":0.77200000000000002,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":1.605,"5min":2.3929999999999998,"15min":2.3929999999999998},"last":1.1140000000000001}]},{"osd":4,"last update":"Fri Nov 24 06:13:24 2023","interfaces":[{"interface":"back","average":{"1min":1.264,"5min":1.417,"15min":1.444},"min":{"1min":0.70699999999999996,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.9610000000000001,"5min":2.4039999999999999,"15min":2.4039999999999999},"last":1.0389999999999999},{"interface":"front","average":{"1min":1.335,"5min":1.4670000000000001,"15min":1.4750000000000001},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.8029999999999999,"5min":2.5609999999999999,"15min":2.5609999999999999},"last":1.0780000000000001}]},{"osd":5,"last update":"Fri Nov 24 06:13:24 2023","interfaces":[{"interface":"back","average":{"1min":1.306,"5min":1.4390000000000001,"15min":1.464},"min":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"max":{"1min":1.9159999999999999,"5min":2.8290000000000002,"15min":2.8290000000000002},"last":1.4099999999999999},{"interface":"front","average":{"1min":1.4590000000000001,"5min":1.613,"15min":1.6419999999999999},"min":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"max":{"1min":2.0259999999999998,"5min":2.9420000000000002,"15min":2.9420000000000002},"last":1.169}]},{"osd":7,"last update":"Fri Nov 24 06:13:24 2023","interfaces":[{"interface":"back","average":{"1min":1.403,"5min":1.5580000000000001,"15min":1.5800000000000001},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.772,"5min":2.5139999999999998,"15min":2.5139999999999998},"last":1.387},{"interface":"front","average":{"1min":1.421,"5min":1.5629999999999999,"15min":1.5529999999999999},"min":{"1min":0.79300000000000004,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":1.992,"5min":4.7859999999999996,"15min":4.7859999999999996},"last":1.532}]}]},{"osd":5,"up_from":13,"seq":55834574897,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31972,"kb_used_data":788,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339868,"statfs":{"total":96636764160,"available":96604024832,"internally_reserved":0,"allocated":806912,"data_stored":345282,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:13:25 2023","interfaces":[{"interface":"back","average":{"1min":0.97999999999999998,"5min":1.1339999999999999,"15min":1.212},"min":{"1min":0.62,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.526,"5min":3.0030000000000001,"15min":3.0030000000000001},"last":1.359},{"interface":"front","average":{"1min":0.998,"5min":1.1339999999999999,"15min":1.196},"min":{"1min":0.623,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.504,"5min":2.0979999999999999,"15min":2.0979999999999999},"last":1.109}]},{"osd":1,"last update":"Fri Nov 24 06:13:25 2023","interfaces":[{"interface":"back","average":{"1min":1.0720000000000001,"5min":1.212,"15min":1.254},"min":{"1min":0.63300000000000001,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.665,"5min":5.0499999999999998,"15min":5.0499999999999998},"last":0.88100000000000001},{"interface":"front","average":{"1min":1.101,"5min":1.3180000000000001,"15min":1.4259999999999999},"min":{"1min":0.72599999999999998,"5min":0.435,"15min":0.435},"max":{"1min":1.5449999999999999,"5min":3.6930000000000001,"15min":3.6930000000000001},"last":1.194}]},{"osd":2,"last update":"Fri Nov 24 06:13:25 2023","interfaces":[{"interface":"back","average":{"1min":1.103,"5min":1.2589999999999999,"15min":1.3160000000000001},"min":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.5960000000000001,"5min":3.085,"15min":3.085},"last":1.274},{"interface":"front","average":{"1min":1.2230000000000001,"5min":1.4510000000000001,"15min":1.5609999999999999},"min":{"1min":0.746,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.5980000000000001,"5min":4.7359999999999998,"15min":4.7359999999999998},"last":1.446}]},{"osd":3,"last update":"Fri Nov 24 06:13:25 2023","interfaces":[{"interface":"back","average":{"1min":1.246,"5min":1.3500000000000001,"15min":1.397},"min":{"1min":0.747,"5min":0.73999999999999999,"15min":0.73999999999999999},"max":{"1min":1.6579999999999999,"5min":3.2229999999999999,"15min":3.2229999999999999},"last":1.7290000000000001},{"interface":"front","average":{"1min":1.333,"5min":1.3919999999999999,"15min":1.427},"min":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"max":{"1min":1.7789999999999999,"5min":2.3260000000000001,"15min":2.3260000000000001},"last":1.0269999999999999}]},{"osd":4,"last update":"Fri Nov 24 06:13:25 2023","interfaces":[{"interface":"back","average":{"1min":1.333,"5min":1.482,"15min":1.538},"min":{"1min":0.95399999999999996,"5min":0.84299999999999997,"15min":0.84299999999999997},"max":{"1min":1.762,"5min":3.1230000000000002,"15min":3.1230000000000002},"last":1.548},{"interface":"front","average":{"1min":1.3400000000000001,"5min":1.478,"15min":1.532},"min":{"1min":0.84399999999999997,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.7509999999999999,"5min":2.4809999999999999,"15min":2.4809999999999999},"last":1.4950000000000001}]},{"osd":6,"last update":"Fri Nov 24 06:13:25 2023","interfaces":[{"interface":"back","average":{"1min":1.4630000000000001,"5min":1.6319999999999999,"15min":1.7150000000000001},"min":{"1min":1.2010000000000001,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.887,"5min":4.6529999999999996,"15min":4.6529999999999996},"last":1.3120000000000001},{"interface":"front","average":{"1min":1.5049999999999999,"5min":1.6479999999999999,"15min":1.716},"min":{"1min":1.226,"5min":0.90400000000000003,"15min":0.90400000000000003},"max":{"1min":1.863,"5min":4.3529999999999998,"15min":4.3529999999999998},"last":1.647}]},{"osd":7,"last update":"Fri Nov 24 06:13:25 2023","interfaces":[{"interface":"back","average":{"1min":1.4490000000000001,"5min":1.615,"15min":1.679},"min":{"1min":0.98499999999999999,"5min":0.874,"15min":0.874},"max":{"1min":1.847,"5min":2.8839999999999999,"15min":2.8839999999999999},"last":1.595},{"interface":"front","average":{"1min":1.579,"5min":1.7509999999999999,"15min":1.8280000000000001},"min":{"1min":1.2709999999999999,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.9770000000000001,"5min":4.8739999999999997,"15min":4.8739999999999997},"last":1.1659999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574897,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31976,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339864,"statfs":{"total":96636764160,"available":96604020736,"internally_reserved":0,"allocated":811008,"data_stored":345896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:13:24 2023","interfaces":[{"interface":"back","average":{"1min":1.01,"5min":1.052,"15min":1.0449999999999999},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.907,"5min":1.907,"15min":1.907},"last":1.179},{"interface":"front","average":{"1min":1.046,"5min":1.1479999999999999,"15min":1.194},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.732,"5min":1.8220000000000001,"15min":1.8220000000000001},"last":1.5669999999999999}]},{"osd":1,"last update":"Fri Nov 24 06:13:24 2023","interfaces":[{"interface":"back","average":{"1min":1.1990000000000001,"5min":1.141,"15min":1.127},"min":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.655,"5min":1.97,"15min":1.97},"last":1.621},{"interface":"front","average":{"1min":1.0920000000000001,"5min":1.161,"15min":1.1719999999999999},"min":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.831,"5min":1.917,"15min":1.917},"last":1.2929999999999999}]},{"osd":2,"last update":"Fri Nov 24 06:13:24 2023","interfaces":[{"interface":"back","average":{"1min":1.119,"5min":1.194,"15min":1.232},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.744,"5min":2.319,"15min":2.319},"last":1.4530000000000001},{"interface":"front","average":{"1min":1.2809999999999999,"5min":1.3080000000000001,"15min":1.3080000000000001},"min":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":1.956,"5min":2.0990000000000002,"15min":2.0990000000000002},"last":1.395}]},{"osd":3,"last update":"Fri Nov 24 06:13:24 2023","interfaces":[{"interface":"back","average":{"1min":1.1839999999999999,"5min":1.325,"15min":1.383},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":2.2269999999999999,"5min":2.2269999999999999,"15min":2.2269999999999999},"last":1.429},{"interface":"front","average":{"1min":1.3180000000000001,"5min":1.395,"15min":1.4259999999999999},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":2.0990000000000002,"5min":2.1720000000000002,"15min":2.1720000000000002},"last":1.6679999999999999}]},{"osd":5,"last update":"Fri Nov 24 06:13:24 2023","interfaces":[{"interface":"back","average":{"1min":1.274,"5min":1.3140000000000001,"15min":1.3300000000000001},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":1.9870000000000001,"5min":1.9870000000000001,"15min":1.9870000000000001},"last":1.0589999999999999},{"interface":"front","average":{"1min":1.3979999999999999,"5min":1.498,"15min":1.534},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":2.2890000000000001,"5min":2.5059999999999998,"15min":2.5059999999999998},"last":1.2310000000000001}]},{"osd":6,"last update":"Fri Nov 24 06:13:24 2023","interfaces":[{"interface":"back","average":{"1min":1.2569999999999999,"5min":1.464,"15min":1.5189999999999999},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.7729999999999999,"5min":2.2069999999999999,"15min":2.2069999999999999},"last":0.95999999999999996},{"interface":"front","average":{"1min":1.5109999999999999,"5min":1.6299999999999999,"15min":1.6659999999999999},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":2.3679999999999999,"5min":2.4510000000000001,"15min":2.4510000000000001},"last":1.4970000000000001}]},{"osd":7,"last update":"Fri Nov 24 06:13:24 2023","interfaces":[{"interface":"back","average":{"1min":1.369,"5min":1.573,"15min":1.6200000000000001},"min":{"1min":0.61299999999999999,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.8480000000000001,"5min":2.4350000000000001,"15min":2.4350000000000001},"last":1.3600000000000001},{"interface":"front","average":{"1min":1.454,"5min":1.581,"15min":1.615},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":2.173,"5min":3.1120000000000001,"15min":3.1120000000000001},"last":1.333}]}]},{"osd":3,"up_from":13,"seq":55834574897,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27816,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344024,"statfs":{"total":96636764160,"available":96608280576,"internally_reserved":0,"allocated":811008,"data_stored":345896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:13:30 2023","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.95599999999999996,"15min":0.98999999999999999},"min":{"1min":0.41799999999999998,"5min":0.314,"15min":0.314},"max":{"1min":1.3300000000000001,"5min":2.766,"15min":2.766},"last":1.034},{"interface":"front","average":{"1min":0.872,"5min":0.93999999999999995,"15min":0.92400000000000004},"min":{"1min":0.503,"5min":0.441,"15min":0.441},"max":{"1min":1.4850000000000001,"5min":1.873,"15min":1.873},"last":1.0089999999999999}]},{"osd":1,"last update":"Fri Nov 24 06:13:30 2023","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.86499999999999999,"15min":0.878},"min":{"1min":0.434,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.3129999999999999,"5min":2.5779999999999998,"15min":2.5779999999999998},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.96599999999999997,"5min":1.107,"15min":1.1579999999999999},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.49,"5min":3.6899999999999999,"15min":3.6899999999999999},"last":1.107}]},{"osd":2,"last update":"Fri Nov 24 06:13:30 2023","interfaces":[{"interface":"back","average":{"1min":1.012,"5min":1.107,"15min":1.153},"min":{"1min":0.53100000000000003,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.431,"5min":4.0620000000000003,"15min":4.0620000000000003},"last":0.96999999999999997},{"interface":"front","average":{"1min":1.131,"5min":1.1850000000000001,"15min":1.2130000000000001},"min":{"1min":0.58499999999999996,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.659,"5min":2.9529999999999998,"15min":2.9529999999999998},"last":1.163}]},{"osd":4,"last update":"Fri Nov 24 06:13:30 2023","interfaces":[{"interface":"back","average":{"1min":1.117,"5min":1.3320000000000001,"15min":1.3959999999999999},"min":{"1min":0.71999999999999997,"5min":0.70999999999999996,"15min":0.70999999999999996},"max":{"1min":1.51,"5min":4.125,"15min":4.125},"last":0.94499999999999995},{"interface":"front","average":{"1min":1.206,"5min":1.379,"15min":1.4239999999999999},"min":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"max":{"1min":1.633,"5min":3.9409999999999998,"15min":3.9409999999999998},"last":1.1870000000000001}]},{"osd":5,"last update":"Fri Nov 24 06:13:30 2023","interfaces":[{"interface":"back","average":{"1min":1.335,"5min":1.399,"15min":1.431},"min":{"1min":0.79500000000000004,"5min":0.749,"15min":0.749},"max":{"1min":2.0609999999999999,"5min":3.3359999999999999,"15min":3.3359999999999999},"last":1.242},{"interface":"front","average":{"1min":1.1910000000000001,"5min":1.357,"15min":1.425},"min":{"1min":0.78200000000000003,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.7929999999999999,"5min":4.1669999999999998,"15min":4.1669999999999998},"last":1.0740000000000001}]},{"osd":6,"last update":"Fri Nov 24 06:13:30 2023","interfaces":[{"interface":"back","average":{"1min":1.3260000000000001,"5min":1.3720000000000001,"15min":1.415},"min":{"1min":0.74199999999999999,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":2.169,"5min":4.0190000000000001,"15min":4.0190000000000001},"last":1.3140000000000001},{"interface":"front","average":{"1min":1.3759999999999999,"5min":1.48,"15min":1.5149999999999999},"min":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"max":{"1min":2.004,"5min":3.2480000000000002,"15min":3.2480000000000002},"last":1.3009999999999999}]},{"osd":7,"last update":"Fri Nov 24 06:13:30 2023","interfaces":[{"interface":"back","average":{"1min":1.258,"5min":1.4870000000000001,"15min":1.571},"min":{"1min":0.79700000000000004,"5min":0.76600000000000001,"15min":0.76600000000000001},"max":{"1min":1.831,"5min":4.202,"15min":4.202},"last":1.145},{"interface":"front","average":{"1min":1.4099999999999999,"5min":1.524,"15min":1.589},"min":{"1min":1.0149999999999999,"5min":0.89700000000000002,"15min":0.89700000000000002},"max":{"1min":2.2989999999999999,"5min":3.9849999999999999,"15min":3.9849999999999999},"last":1.3999999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574897,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27812,"kb_used_data":788,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344028,"statfs":{"total":96636764160,"available":96608284672,"internally_reserved":0,"allocated":806912,"data_stored":345282,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:13:25 2023","interfaces":[{"interface":"back","average":{"1min":0.88800000000000001,"5min":0.88,"15min":0.88600000000000001},"min":{"1min":0.34000000000000002,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":1.1850000000000001,"5min":1.6259999999999999,"15min":1.6259999999999999},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.90700000000000003,"5min":1.0349999999999999,"15min":1.079},"min":{"1min":0.51300000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.1799999999999999,"5min":1.7669999999999999,"15min":1.7669999999999999},"last":1.25}]},{"osd":1,"last update":"Fri Nov 24 06:13:25 2023","interfaces":[{"interface":"back","average":{"1min":0.90700000000000003,"5min":0.94299999999999995,"15min":0.93999999999999995},"min":{"1min":0.38200000000000001,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.294,"5min":1.635,"15min":1.635},"last":1.006},{"interface":"front","average":{"1min":0.91400000000000003,"5min":0.96299999999999997,"15min":0.97499999999999998},"min":{"1min":0.51600000000000001,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.579,"5min":1.8140000000000001,"15min":1.8140000000000001},"last":0.95099999999999996}]},{"osd":3,"last update":"Fri Nov 24 06:13:25 2023","interfaces":[{"interface":"back","average":{"1min":0.88700000000000001,"5min":1.014,"15min":1.0580000000000001},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.3400000000000001,"5min":2.3559999999999999,"15min":2.3559999999999999},"last":1.056},{"interface":"front","average":{"1min":1.0489999999999999,"5min":1.1180000000000001,"15min":1.121},"min":{"1min":0.47899999999999998,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.339,"5min":2.1840000000000002,"15min":2.1840000000000002},"last":1.3919999999999999}]},{"osd":4,"last update":"Fri Nov 24 06:13:25 2023","interfaces":[{"interface":"back","average":{"1min":1.228,"5min":1.379,"15min":1.405},"min":{"1min":0.73999999999999999,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.488,"5min":3.411,"15min":3.411},"last":1.51},{"interface":"front","average":{"1min":1.179,"5min":1.4059999999999999,"15min":1.472},"min":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"max":{"1min":1.5409999999999999,"5min":3.7109999999999999,"15min":3.7109999999999999},"last":1.6579999999999999}]},{"osd":5,"last update":"Fri Nov 24 06:13:25 2023","interfaces":[{"interface":"back","average":{"1min":1.1970000000000001,"5min":1.2829999999999999,"15min":1.3109999999999999},"min":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"max":{"1min":1.498,"5min":2.5259999999999998,"15min":2.5259999999999998},"last":1.3049999999999999},{"interface":"front","average":{"1min":1.3100000000000001,"5min":1.3320000000000001,"15min":1.3120000000000001},"min":{"1min":0.999,"5min":0.90400000000000003,"15min":0.90400000000000003},"max":{"1min":1.5409999999999999,"5min":2.1640000000000001,"15min":2.1640000000000001},"last":1.681}]},{"osd":6,"last update":"Fri Nov 24 06:13:25 2023","interfaces":[{"interface":"back","average":{"1min":1.2529999999999999,"5min":1.373,"15min":1.429},"min":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"max":{"1min":1.7170000000000001,"5min":2.3039999999999998,"15min":2.3039999999999998},"last":1.4810000000000001},{"interface":"front","average":{"1min":1.218,"5min":1.3939999999999999,"15min":1.444},"min":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":1.474,"5min":2.605,"15min":2.605},"last":1.589}]},{"osd":7,"last update":"Fri Nov 24 06:13:25 2023","interfaces":[{"interface":"back","average":{"1min":1.2889999999999999,"5min":1.4870000000000001,"15min":1.5589999999999999},"min":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"max":{"1min":1.7829999999999999,"5min":2.3740000000000001,"15min":2.3740000000000001},"last":1.5509999999999999},{"interface":"front","average":{"1min":1.2689999999999999,"5min":1.3939999999999999,"15min":1.4419999999999999},"min":{"1min":0.89200000000000002,"5min":0.877,"15min":0.877},"max":{"1min":1.6299999999999999,"5min":2.5699999999999998,"15min":2.5699999999999998},"last":1.4410000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574896,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31976,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339864,"statfs":{"total":96636764160,"available":96604020736,"internally_reserved":0,"allocated":811008,"data_stored":345896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:13:21 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.85099999999999998,"15min":0.89900000000000002},"min":{"1min":0.36799999999999999,"5min":0.311,"15min":0.311},"max":{"1min":1.212,"5min":1.9299999999999999,"15min":1.9299999999999999},"last":0.97999999999999998},{"interface":"front","average":{"1min":0.873,"5min":0.89500000000000002,"15min":0.92100000000000004},"min":{"1min":0.35099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":1.339,"5min":1.7669999999999999,"15min":1.7669999999999999},"last":0.747}]},{"osd":2,"last update":"Fri Nov 24 06:13:21 2023","interfaces":[{"interface":"back","average":{"1min":0.97499999999999998,"5min":0.95599999999999996,"15min":0.998},"min":{"1min":0.496,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.605,"5min":2.5329999999999999,"15min":2.5329999999999999},"last":1.24},{"interface":"front","average":{"1min":1.079,"5min":1.0009999999999999,"15min":1.02},"min":{"1min":0.502,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.7190000000000001,"5min":1.7190000000000001,"15min":1.7190000000000001},"last":1.208}]},{"osd":3,"last update":"Fri Nov 24 06:13:21 2023","interfaces":[{"interface":"back","average":{"1min":0.91700000000000004,"5min":0.96399999999999997,"15min":1.0129999999999999},"min":{"1min":0.46800000000000003,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.4119999999999999,"5min":1.841,"15min":1.841},"last":0.91800000000000004},{"interface":"front","average":{"1min":1.1479999999999999,"5min":1.1200000000000001,"15min":1.1439999999999999},"min":{"1min":0.61899999999999999,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.643,"5min":2.8940000000000001,"15min":2.8940000000000001},"last":1.482}]},{"osd":4,"last update":"Fri Nov 24 06:13:21 2023","interfaces":[{"interface":"back","average":{"1min":1.3149999999999999,"5min":1.2609999999999999,"15min":1.2749999999999999},"min":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.728,"5min":2.1930000000000001,"15min":2.1930000000000001},"last":1.3049999999999999},{"interface":"front","average":{"1min":1.1899999999999999,"5min":1.1819999999999999,"15min":1.194},"min":{"1min":0.86799999999999999,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":1.5640000000000001,"5min":1.784,"15min":1.784},"last":1.119}]},{"osd":5,"last update":"Fri Nov 24 06:13:21 2023","interfaces":[{"interface":"back","average":{"1min":1.2430000000000001,"5min":1.3959999999999999,"15min":1.4850000000000001},"min":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.732,"5min":5.5449999999999999,"15min":5.5449999999999999},"last":1.423},{"interface":"front","average":{"1min":1.361,"5min":1.444,"15min":1.524},"min":{"1min":0.90900000000000003,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":1.7270000000000001,"5min":6.0030000000000001,"15min":6.0030000000000001},"last":1.3999999999999999}]},{"osd":6,"last update":"Fri Nov 24 06:13:21 2023","interfaces":[{"interface":"back","average":{"1min":1.3220000000000001,"5min":1.446,"15min":1.528},"min":{"1min":0.81100000000000005,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":1.8779999999999999,"5min":5.2809999999999997,"15min":5.2809999999999997},"last":1.367},{"interface":"front","average":{"1min":1.4079999999999999,"5min":1.423,"15min":1.4570000000000001},"min":{"1min":0.82999999999999996,"5min":0.749,"15min":0.749},"max":{"1min":1.819,"5min":4.327,"15min":4.327},"last":1.4570000000000001}]},{"osd":7,"last update":"Fri Nov 24 06:13:21 2023","interfaces":[{"interface":"back","average":{"1min":1.264,"5min":1.302,"15min":1.3400000000000001},"min":{"1min":0.85099999999999998,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":1.7849999999999999,"5min":2.3069999999999999,"15min":2.3069999999999999},"last":1.331},{"interface":"front","average":{"1min":1.415,"5min":1.492,"15min":1.5620000000000001},"min":{"1min":0.91700000000000004,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":2.0179999999999998,"5min":5.4500000000000002,"15min":5.4500000000000002},"last":1.536}]}]},{"osd":0,"up_from":13,"seq":55834574896,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32548,"kb_used_data":1372,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339292,"statfs":{"total":96636764160,"available":96603435008,"internally_reserved":0,"allocated":1404928,"data_stored":936264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Nov 24 06:13:25 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.84799999999999998,"15min":0.90800000000000003},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.145,"5min":2.3999999999999999,"15min":2.3999999999999999},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.75800000000000001,"5min":1.024,"15min":1.133},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":1.54,"5min":4.8289999999999997,"15min":4.8289999999999997},"last":1.0269999999999999}]},{"osd":2,"last update":"Fri Nov 24 06:13:25 2023","interfaces":[{"interface":"back","average":{"1min":0.83799999999999997,"5min":0.97999999999999998,"15min":1.0349999999999999},"min":{"1min":0.42999999999999999,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.4019999999999999,"5min":2.5139999999999998,"15min":2.5139999999999998},"last":1.3340000000000001},{"interface":"front","average":{"1min":0.79800000000000004,"5min":1.0229999999999999,"15min":1.0820000000000001},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.26,"5min":2.3599999999999999,"15min":2.3599999999999999},"last":1.081}]},{"osd":3,"last update":"Fri Nov 24 06:13:25 2023","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":1.056,"15min":1.1020000000000001},"min":{"1min":0.48699999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.5369999999999999,"5min":2.5840000000000001,"15min":2.5840000000000001},"last":0.91500000000000004},{"interface":"front","average":{"1min":0.91600000000000004,"5min":1.246,"15min":1.3580000000000001},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.4630000000000001,"5min":5.1230000000000002,"15min":5.1230000000000002},"last":1.1899999999999999}]},{"osd":4,"last update":"Fri Nov 24 06:13:25 2023","interfaces":[{"interface":"back","average":{"1min":1.0329999999999999,"5min":1.1799999999999999,"15min":1.248},"min":{"1min":0.70499999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.488,"5min":2.887,"15min":2.887},"last":1.1399999999999999},{"interface":"front","average":{"1min":1.0720000000000001,"5min":1.2310000000000001,"15min":1.3009999999999999},"min":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.581,"5min":2.8530000000000002,"15min":2.8530000000000002},"last":1.24}]},{"osd":5,"last update":"Fri Nov 24 06:13:25 2023","interfaces":[{"interface":"back","average":{"1min":1.149,"5min":1.4219999999999999,"15min":1.524},"min":{"1min":0.72199999999999998,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.978,"5min":4.6299999999999999,"15min":4.6299999999999999},"last":1.2849999999999999},{"interface":"front","average":{"1min":1.1060000000000001,"5min":1.232,"15min":1.2989999999999999},"min":{"1min":0.85399999999999998,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.7010000000000001,"5min":2.794,"15min":2.794},"last":1.3109999999999999}]},{"osd":6,"last update":"Fri Nov 24 06:13:25 2023","interfaces":[{"interface":"back","average":{"1min":1.244,"5min":1.508,"15min":1.617},"min":{"1min":0.73899999999999999,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.8620000000000001,"5min":4.798,"15min":4.798},"last":1.4830000000000001},{"interface":"front","average":{"1min":1.3100000000000001,"5min":1.5189999999999999,"15min":1.597},"min":{"1min":0.97099999999999997,"5min":0.77800000000000002,"15min":0.77800000000000002},"max":{"1min":1.841,"5min":4.7149999999999999,"15min":4.7149999999999999},"last":1.389}]},{"osd":7,"last update":"Fri Nov 24 06:13:25 2023","interfaces":[{"interface":"back","average":{"1min":1.2629999999999999,"5min":1.544,"15min":1.6579999999999999},"min":{"1min":0.85099999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.9770000000000001,"5min":4.7649999999999997,"15min":4.7649999999999997},"last":1.579},{"interface":"front","average":{"1min":1.3,"5min":1.5609999999999999,"15min":1.6499999999999999},"min":{"1min":0.89900000000000002,"5min":0.84599999999999997,"15min":0.84599999999999997},"max":{"1min":1.9159999999999999,"5min":4.6820000000000004,"15min":4.6820000000000004},"last":1.4470000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-24T06:14:10.727 DEBUG:teuthology.orchestra.run.smithi092:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2023-11-24T06:14:10.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:10.921+0000 7f4477fff640 1 Processor -- start 2023-11-24T06:14:10.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:10.921+0000 7f4477fff640 1 -- start start 2023-11-24T06:14:10.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:10.923+0000 7f4477fff640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f44780cdfa0 0x7f44780cc5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:14:10.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:10.923+0000 7f4477fff640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44780ce4b0 con 0x7f44780ccaf0 2023-11-24T06:14:10.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:10.923+0000 7f4477fff640 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f447815b540 con 0x7f447815b160 2023-11-24T06:14:10.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:10.923+0000 7f4477fff640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44780ce750 con 0x7f44780cdfa0 2023-11-24T06:14:10.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:10.924+0000 7f44777fe640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f44780ccaf0 0x7f44780b0880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:39864/0 (socket says 172.21.15.92:39864) 2023-11-24T06:14:10.928 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:10.924+0000 7f44777fe640 1 -- 172.21.15.92:0/1259620449 learned_addr learned my addr 172.21.15.92:0/1259620449 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-24T06:14:10.928 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:10.924+0000 7f446e7fc640 1 --2- 172.21.15.92:0/1259620449 >> v2:172.21.15.92:3300/0 conn(0x7f44780cdfa0 0x7f44780cc5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:14:10.928 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:10.924+0000 7f44767fc640 1 -- 172.21.15.92:0/1259620449 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 941645752 0 0) 0x7f44780ce4b0 con 0x7f44780ccaf0 2023-11-24T06:14:10.928 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:10.924+0000 7f44767fc640 1 -- 172.21.15.92:0/1259620449 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4464003520 con 0x7f44780ccaf0 2023-11-24T06:14:10.928 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:10.924+0000 7f44767fc640 1 -- 172.21.15.92:0/1259620449 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 166879397 0 0) 0x7f4464003520 con 0x7f44780ccaf0 2023-11-24T06:14:10.928 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:10.924+0000 7f44767fc640 1 -- 172.21.15.92:0/1259620449 >> v2:172.21.15.92:3300/0 conn(0x7f44780cdfa0 msgr2=0x7f44780cc5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:14:10.928 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:10.924+0000 7f44767fc640 1 --2- 172.21.15.92:0/1259620449 >> v2:172.21.15.92:3300/0 conn(0x7f44780cdfa0 0x7f44780cc5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:14:10.928 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:10.924+0000 7f44767fc640 1 -- 172.21.15.92:0/1259620449 >> v1:172.21.15.118:6789/0 conn(0x7f447815b160 legacy=0x7f44780ceb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:14:10.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:10.924+0000 7f44767fc640 1 -- 172.21.15.92:0/1259620449 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44780cbdb0 con 0x7f44780ccaf0 2023-11-24T06:14:10.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:10.924+0000 7f44767fc640 1 -- 172.21.15.92:0/1259620449 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2607661350 0 0) 0x7f4470001670 con 0x7f44780ccaf0 2023-11-24T06:14:10.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:10.924+0000 7f44767fc640 1 -- 172.21.15.92:0/1259620449 >> v1:172.21.15.92:6789/0 conn(0x7f44780ccaf0 legacy=0x7f44780b0880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:14:10.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:10.925+0000 7f44767fc640 1 --2- 172.21.15.92:0/1259620449 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4464003970 0x7f4464003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:14:10.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:10.925+0000 7f44767fc640 1 --2- 172.21.15.92:0/1259620449 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f44640047c0 0x7f4464004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:14:10.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:10.925+0000 7f44767fc640 1 --2- 172.21.15.92:0/1259620449 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4464005180 0x7f446400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:14:10.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:10.925+0000 7f44767fc640 1 -- 172.21.15.92:0/1259620449 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44780ce750 con 0x7f4464003970 2023-11-24T06:14:10.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:10.925+0000 7f44767fc640 1 -- 172.21.15.92:0/1259620449 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4464010430 con 0x7f4464005180 2023-11-24T06:14:10.930 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:10.925+0000 7f44767fc640 1 -- 172.21.15.92:0/1259620449 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44640105f0 con 0x7f44640047c0 2023-11-24T06:14:10.931 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:10.925+0000 7f4476ffd640 1 --2- 172.21.15.92:0/1259620449 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4464003970 0x7f4464003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:14:10.931 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:10.925+0000 7f44777fe640 1 --2- 172.21.15.92:0/1259620449 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4464005180 0x7f446400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:14:10.931 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:10.925+0000 7f4476ffd640 1 -- 172.21.15.92:0/1259620449 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f44640047c0 msgr2=0x7f4464004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:14:10.931 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:10.925+0000 7f4476ffd640 1 --2- 172.21.15.92:0/1259620449 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f44640047c0 0x7f4464004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:14:10.932 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:10.925+0000 7f4476ffd640 1 -- 172.21.15.92:0/1259620449 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4464005180 msgr2=0x7f446400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:14:10.932 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:10.925+0000 7f4476ffd640 1 --2- 172.21.15.92:0/1259620449 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4464005180 0x7f446400b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:14:10.932 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:10.925+0000 7f4476ffd640 1 -- 172.21.15.92:0/1259620449 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4464003520 con 0x7f4464003970 2023-11-24T06:14:10.932 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:10.925+0000 7f44777fe640 1 --2- 172.21.15.92:0/1259620449 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4464005180 0x7f446400b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:14:10.932 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:10.925+0000 7f44777fe640 1 -- 172.21.15.92:0/1259620449 reap_dead start 2023-11-24T06:14:10.932 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:10.926+0000 7f4476ffd640 1 --2- 172.21.15.92:0/1259620449 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4464003970 0x7f4464003d90 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f4468001e50 tx=0x7f44640057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c30a5d06ae9eded server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:14:10.932 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:10.927+0000 7f44767fc640 1 -- 172.21.15.92:0/1259620449 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4468003b70 con 0x7f4464003970 2023-11-24T06:14:10.932 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:10.927+0000 7f44767fc640 1 -- 172.21.15.92:0/1259620449 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4468003cd0 con 0x7f4464003970 2023-11-24T06:14:10.933 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:10.927+0000 7f44767fc640 1 -- 172.21.15.92:0/1259620449 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4468005310 con 0x7f4464003970 2023-11-24T06:14:10.933 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:10.927+0000 7f4477fff640 1 -- 172.21.15.92:0/1259620449 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4464003970 msgr2=0x7f4464003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:14:10.933 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:10.927+0000 7f4477fff640 1 --2- 172.21.15.92:0/1259620449 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4464003970 0x7f4464003d90 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f4468001e50 tx=0x7f44640057f0 comp rx=0 tx=0).stop 2023-11-24T06:14:10.933 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:10.928+0000 7f4477fff640 1 -- 172.21.15.92:0/1259620449 shutdown_connections 2023-11-24T06:14:10.933 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:10.928+0000 7f4477fff640 1 --2- 172.21.15.92:0/1259620449 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4464003970 0x7f4464003d90 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:14:10.933 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:10.928+0000 7f4477fff640 1 -- 172.21.15.92:0/1259620449 >> 172.21.15.92:0/1259620449 conn(0x7f44780b2a30 msgr2=0x7f44780b2e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:14:10.933 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:10.928+0000 7f4477fff640 1 -- 172.21.15.92:0/1259620449 shutdown_connections 2023-11-24T06:14:10.933 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:10.928+0000 7f4477fff640 1 -- 172.21.15.92:0/1259620449 wait complete. 2023-11-24T06:14:10.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:10.930+0000 7f4477fff640 1 Processor -- start 2023-11-24T06:14:10.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:10.930+0000 7f4477fff640 1 -- start start 2023-11-24T06:14:10.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:10.931+0000 7f4477fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4464003970 0x7f447815d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:14:10.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:10.931+0000 7f4477fff640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f44780b5d00 0x7f44780d1110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:14:10.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:10.931+0000 7f4477fff640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4478157040 0x7f44781574c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:14:10.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:10.931+0000 7f4477fff640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4478157a00 con 0x7f4464003970 2023-11-24T06:14:10.936 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:10.931+0000 7f4477fff640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f447815d9a0 con 0x7f44780b5d00 2023-11-24T06:14:10.936 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:10.931+0000 7f4477fff640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f447815dc80 con 0x7f4478157040 2023-11-24T06:14:10.936 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:10.931+0000 7f4476ffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4464003970 0x7f447815d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:14:10.936 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:10.931+0000 7f4476ffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4464003970 0x7f447815d2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39316/0 (socket says 172.21.15.92:39316) 2023-11-24T06:14:10.936 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:10.931+0000 7f4476ffd640 1 -- 172.21.15.92:0/1452698571 learned_addr learned my addr 172.21.15.92:0/1452698571 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:14:10.936 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:10.932+0000 7f4476ffd640 1 -- 172.21.15.92:0/1452698571 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4478157040 msgr2=0x7f44781574c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:14:10.936 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:10.932+0000 7f4476ffd640 1 --2- 172.21.15.92:0/1452698571 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4478157040 0x7f44781574c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:14:10.937 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:10.932+0000 7f4476ffd640 1 -- 172.21.15.92:0/1452698571 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f44780b5d00 msgr2=0x7f44780d1110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:14:10.937 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:10.932+0000 7f4476ffd640 1 --2- 172.21.15.92:0/1452698571 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f44780b5d00 0x7f44780d1110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:14:10.937 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:10.932+0000 7f4476ffd640 1 -- 172.21.15.92:0/1452698571 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f447815df60 con 0x7f4464003970 2023-11-24T06:14:10.937 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:10.932+0000 7f4476ffd640 1 --2- 172.21.15.92:0/1452698571 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4464003970 0x7f447815d2c0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f446802bf70 tx=0x7f44680014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:14:10.938 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:10.932+0000 7f4474ff9640 1 -- 172.21.15.92:0/1452698571 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f446803e070 con 0x7f4464003970 2023-11-24T06:14:10.938 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:10.932+0000 7f4474ff9640 1 -- 172.21.15.92:0/1452698571 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4468003e40 con 0x7f4464003970 2023-11-24T06:14:10.938 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:10.932+0000 7f4477fff640 1 -- 172.21.15.92:0/1452698571 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f447815e240 con 0x7f4464003970 2023-11-24T06:14:10.938 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:10.932+0000 7f4474ff9640 1 -- 172.21.15.92:0/1452698571 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f446802f890 con 0x7f4464003970 2023-11-24T06:14:10.938 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:10.933+0000 7f4477fff640 1 -- 172.21.15.92:0/1452698571 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44780b6180 con 0x7f4464003970 2023-11-24T06:14:10.938 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:10.934+0000 7f4477fff640 1 -- 172.21.15.92:0/1452698571 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4478067450 con 0x7f4464003970 2023-11-24T06:14:10.939 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:10.935+0000 7f4474ff9640 1 -- 172.21.15.92:0/1452698571 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 7) v1 ==== 89333+0+0 (secure 0 0 0) 0x7f446802f9f0 con 0x7f4464003970 2023-11-24T06:14:10.939 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:10.935+0000 7f4474ff9640 1 --2- 172.21.15.92:0/1452698571 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f445806af90 0x7f445806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:14:10.939 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:10.936+0000 7f4474ff9640 1 -- 172.21.15.92:0/1452698571 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f44680aad30 con 0x7f4464003970 2023-11-24T06:14:10.940 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:10.936+0000 7f446e7fc640 1 --2- 172.21.15.92:0/1452698571 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f445806af90 0x7f445806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:14:10.940 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:10.936+0000 7f446e7fc640 1 --2- 172.21.15.92:0/1452698571 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f445806af90 0x7f445806d450 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f4460006e80 tx=0x7f4460006dd0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:14:10.943 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:10.940+0000 7f4474ff9640 1 -- 172.21.15.92:0/1452698571 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4468075eb0 con 0x7f4464003970 2023-11-24T06:14:11.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:11.126+0000 7f4477fff640 1 -- 172.21.15.92:0/1452698571 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) v1 -- 0x7f44780c8850 con 0x7f4464003970 2023-11-24T06:14:11.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:11.131+0000 7f4474ff9640 1 -- 172.21.15.92:0/1452698571 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v16)=0 v16) v1 ==== 123+0+0 (secure 0 0 0) 0x7f44680470c0 con 0x7f4464003970 2023-11-24T06:14:11.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:11.133+0000 7f4474ff9640 1 -- 172.21.15.92:0/1452698571 <== mon.0 v2:172.21.15.92:3300/0 8 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f4468075eb0 con 0x7f4464003970 2023-11-24T06:14:11.138 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:11.134+0000 7f4477fff640 1 -- 172.21.15.92:0/1452698571 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f445806af90 msgr2=0x7f445806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:14:11.138 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:11.134+0000 7f4477fff640 1 --2- 172.21.15.92:0/1452698571 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f445806af90 0x7f445806d450 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f4460006e80 tx=0x7f4460006dd0 comp rx=0 tx=0).stop 2023-11-24T06:14:11.138 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:11.134+0000 7f4477fff640 1 -- 172.21.15.92:0/1452698571 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4464003970 msgr2=0x7f447815d2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:14:11.138 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:11.134+0000 7f4477fff640 1 --2- 172.21.15.92:0/1452698571 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4464003970 0x7f447815d2c0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f446802bf70 tx=0x7f44680014a0 comp rx=0 tx=0).stop 2023-11-24T06:14:11.138 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:11.134+0000 7f4477fff640 1 -- 172.21.15.92:0/1452698571 shutdown_connections 2023-11-24T06:14:11.138 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:11.134+0000 7f4477fff640 1 --2- 172.21.15.92:0/1452698571 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f445806af90 0x7f445806d450 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:14:11.138 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:11.134+0000 7f4477fff640 1 --2- 172.21.15.92:0/1452698571 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f4478157040 0x7f44781574c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:14:11.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:11.134+0000 7f4477fff640 1 --2- 172.21.15.92:0/1452698571 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f44780b5d00 0x7f44780d1110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:14:11.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:11.134+0000 7f4477fff640 1 --2- 172.21.15.92:0/1452698571 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4464003970 0x7f447815d2c0 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:14:11.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:11.134+0000 7f4477fff640 1 -- 172.21.15.92:0/1452698571 >> 172.21.15.92:0/1452698571 conn(0x7f44780b2a30 msgr2=0x7f447806b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:14:11.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:11.135+0000 7f4477fff640 1 -- 172.21.15.92:0/1452698571 shutdown_connections 2023-11-24T06:14:11.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:11.135+0000 7f4477fff640 1 -- 172.21.15.92:0/1452698571 wait complete. 2023-11-24T06:14:11.154 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-24T06:14:11.154 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T06:14:11.354 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:11.351+0000 7f24da743640 1 Processor -- start 2023-11-24T06:14:11.355 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:11.351+0000 7f24da743640 1 -- start start 2023-11-24T06:14:11.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:11.353+0000 7f24da743640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f24d4157c30 0x7f24d40cd600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:14:11.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:11.353+0000 7f24da743640 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f24d41618f0 0x7f24d4070af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:14:11.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:11.353+0000 7f24da743640 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24d4071460 con 0x7f24d4071030 2023-11-24T06:14:11.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:11.353+0000 7f24da743640 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24d415ea30 con 0x7f24d41618f0 2023-11-24T06:14:11.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:11.353+0000 7f24da743640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24d406d860 con 0x7f24d4157c30 2023-11-24T06:14:11.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:11.354+0000 7f24d3fff640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f24d4157c30 0x7f24d40cd600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:14:11.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:11.354+0000 7f24d3fff640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f24d4157c30 0x7f24d40cd600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3301/0 says I am v2:172.21.15.92:52826/0 (socket says 172.21.15.92:52826) 2023-11-24T06:14:11.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:11.354+0000 7f24d3fff640 1 -- 172.21.15.92:0/830204607 learned_addr learned my addr 172.21.15.92:0/830204607 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:14:11.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:11.354+0000 7f24d37fe640 1 --2- 172.21.15.92:0/830204607 >> v2:172.21.15.118:3300/0 conn(0x7f24d41618f0 0x7f24d4070af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:14:11.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:11.354+0000 7f24d3fff640 1 -- 172.21.15.92:0/830204607 >> v2:172.21.15.118:3300/0 conn(0x7f24d41618f0 msgr2=0x7f24d4070af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:14:11.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:11.354+0000 7f24d3fff640 1 --2- 172.21.15.92:0/830204607 >> v2:172.21.15.118:3300/0 conn(0x7f24d41618f0 0x7f24d4070af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:14:11.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:11.354+0000 7f24d3fff640 1 -- 172.21.15.92:0/830204607 >> v1:172.21.15.118:6789/0 conn(0x7f24d4071030 legacy=0x7f24d406dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:14:11.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:11.354+0000 7f24d3fff640 1 -- 172.21.15.92:0/830204607 --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24d415deb0 con 0x7f24d4157c30 2023-11-24T06:14:11.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:11.356+0000 7f24d3fff640 1 --2- 172.21.15.92:0/830204607 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f24d4157c30 0x7f24d40cd600 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f24c4009870 tx=0x7f24c402b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4d570ad9e5ba143 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:14:11.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:11.356+0000 7f24d2ffd640 1 -- 172.21.15.92:0/830204607 <== mon.2 v2:172.21.15.118:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24c4030840 con 0x7f24d4157c30 2023-11-24T06:14:11.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:11.356+0000 7f24d2ffd640 1 -- 172.21.15.92:0/830204607 <== mon.2 v2:172.21.15.118:3301/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f24c40309a0 con 0x7f24d4157c30 2023-11-24T06:14:11.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:11.356+0000 7f24d2ffd640 1 -- 172.21.15.92:0/830204607 <== mon.2 v2:172.21.15.118:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24c4030d50 con 0x7f24d4157c30 2023-11-24T06:14:11.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:11.357+0000 7f24da743640 1 -- 172.21.15.92:0/830204607 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f24d4157c30 msgr2=0x7f24d40cd600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:14:11.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:11.357+0000 7f24da743640 1 --2- 172.21.15.92:0/830204607 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f24d4157c30 0x7f24d40cd600 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f24c4009870 tx=0x7f24c402b980 comp rx=0 tx=0).stop 2023-11-24T06:14:11.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:11.357+0000 7f24da743640 1 -- 172.21.15.92:0/830204607 shutdown_connections 2023-11-24T06:14:11.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:11.357+0000 7f24da743640 1 --2- 172.21.15.92:0/830204607 >> v2:172.21.15.118:3300/0 conn(0x7f24d41618f0 0x7f24d4070af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:14:11.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:11.357+0000 7f24da743640 1 --2- 172.21.15.92:0/830204607 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f24d4157c30 0x7f24d40cd600 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:14:11.362 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:11.357+0000 7f24da743640 1 -- 172.21.15.92:0/830204607 >> 172.21.15.92:0/830204607 conn(0x7f24d40cf700 msgr2=0x7f24d40cfae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:14:11.362 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:11.357+0000 7f24da743640 1 -- 172.21.15.92:0/830204607 shutdown_connections 2023-11-24T06:14:11.362 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:11.357+0000 7f24da743640 1 -- 172.21.15.92:0/830204607 wait complete. 2023-11-24T06:14:11.362 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:11.359+0000 7f24da743640 1 Processor -- start 2023-11-24T06:14:11.363 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:11.359+0000 7f24da743640 1 -- start start 2023-11-24T06:14:11.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:11.360+0000 7f24da743640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f24d4071030 0x7f24d40adb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:14:11.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:11.360+0000 7f24da743640 1 --2- >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f24d4157c30 0x7f24d41468b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:14:11.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:11.360+0000 7f24da743640 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f24d41618f0 0x7f24d4146df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:14:11.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:11.360+0000 7f24da743640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24d414af70 con 0x7f24d4071030 2023-11-24T06:14:11.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:11.360+0000 7f24da743640 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24d414d3a0 con 0x7f24d41618f0 2023-11-24T06:14:11.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:11.360+0000 7f24da743640 1 -- --> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24d4148e00 con 0x7f24d4157c30 2023-11-24T06:14:11.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:11.360+0000 7f24d3fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f24d4071030 0x7f24d40adb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:14:11.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:11.361+0000 7f24d3fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f24d4071030 0x7f24d40adb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39328/0 (socket says 172.21.15.92:39328) 2023-11-24T06:14:11.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:11.361+0000 7f24d3fff640 1 -- 172.21.15.92:0/2107887124 learned_addr learned my addr 172.21.15.92:0/2107887124 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-24T06:14:11.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:11.361+0000 7f24d37fe640 1 --2- 172.21.15.92:0/2107887124 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f24d4157c30 0x7f24d41468b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:14:11.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:11.361+0000 7f24d8ce0640 1 --2- 172.21.15.92:0/2107887124 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f24d41618f0 0x7f24d4146df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:14:11.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:11.361+0000 7f24d3fff640 1 -- 172.21.15.92:0/2107887124 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f24d4157c30 msgr2=0x7f24d41468b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:14:11.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:11.361+0000 7f24d3fff640 1 --2- 172.21.15.92:0/2107887124 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f24d4157c30 0x7f24d41468b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:14:11.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:11.361+0000 7f24d3fff640 1 -- 172.21.15.92:0/2107887124 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f24d41618f0 msgr2=0x7f24d4146df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:14:11.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:11.361+0000 7f24d3fff640 1 --2- 172.21.15.92:0/2107887124 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f24d41618f0 0x7f24d4146df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:14:11.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:11.361+0000 7f24d3fff640 1 -- 172.21.15.92:0/2107887124 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24c40094d0 con 0x7f24d4071030 2023-11-24T06:14:11.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:11.361+0000 7f24d8ce0640 1 --2- 172.21.15.92:0/2107887124 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f24d41618f0 0x7f24d4146df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:14:11.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:11.361+0000 7f24d37fe640 1 --2- 172.21.15.92:0/2107887124 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f24d4157c30 0x7f24d41468b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:14:11.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:11.361+0000 7f24d3fff640 1 --2- 172.21.15.92:0/2107887124 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f24d4071030 0x7f24d40adb90 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f24c402b9c0 tx=0x7f24c4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:14:11.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:11.362+0000 7f24d0ff9640 1 -- 172.21.15.92:0/2107887124 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24c4039c90 con 0x7f24d4071030 2023-11-24T06:14:11.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:11.362+0000 7f24d0ff9640 1 -- 172.21.15.92:0/2107887124 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f24c4039df0 con 0x7f24d4071030 2023-11-24T06:14:11.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:11.362+0000 7f24d0ff9640 1 -- 172.21.15.92:0/2107887124 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24c40446e0 con 0x7f24d4071030 2023-11-24T06:14:11.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:11.362+0000 7f24da743640 1 -- 172.21.15.92:0/2107887124 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24d41490e0 con 0x7f24d4071030 2023-11-24T06:14:11.369 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:11.362+0000 7f24da743640 1 -- 172.21.15.92:0/2107887124 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24d40ae0d0 con 0x7f24d4071030 2023-11-24T06:14:11.369 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:11.363+0000 7f24da743640 1 -- 172.21.15.92:0/2107887124 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24d414a9f0 con 0x7f24d4071030 2023-11-24T06:14:11.370 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:11.364+0000 7f24d0ff9640 1 -- 172.21.15.92:0/2107887124 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 7) v1 ==== 89333+0+0 (secure 0 0 0) 0x7f24c402f390 con 0x7f24d4071030 2023-11-24T06:14:11.370 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:11.365+0000 7f24d0ff9640 1 --2- 172.21.15.92:0/2107887124 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f24c006af90 0x7f24c006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:14:11.370 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:11.365+0000 7f24d0ff9640 1 -- 172.21.15.92:0/2107887124 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f24c403e070 con 0x7f24d4071030 2023-11-24T06:14:11.370 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:11.365+0000 7f24d37fe640 1 --2- 172.21.15.92:0/2107887124 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f24c006af90 0x7f24c006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:14:11.370 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:11.366+0000 7f24d37fe640 1 --2- 172.21.15.92:0/2107887124 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f24c006af90 0x7f24c006d450 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f24d40ce850 tx=0x7f24c8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:14:11.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:11.369+0000 7f24d0ff9640 1 -- 172.21.15.92:0/2107887124 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f24d414a9f0 con 0x7f24d4071030 2023-11-24T06:14:11.601 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:11.598+0000 7f24da743640 1 -- 172.21.15.92:0/2107887124 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f24d414a9f0 con 0x7f24d4071030 2023-11-24T06:14:11.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:11.599+0000 7f24d0ff9640 1 -- 172.21.15.92:0/2107887124 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) v1 ==== 94+0+3705 (secure 0 0 0) 0x7f24c4049e90 con 0x7f24d4071030 2023-11-24T06:14:11.602 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-24T06:14:11.603 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":43,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5143,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6833/2094733752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":2094733752},{"type":"v1","addr":"172.21.15.118:6833","nonce":2094733752}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":36},{"gid":5146,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6835/2873546779","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":2873546779},{"type":"v1","addr":"172.21.15.118:6835","nonce":2873546779}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":36},{"gid":6141,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6836/3335501795","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":3335501795},{"type":"v1","addr":"172.21.15.92:6836","nonce":3335501795}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":35},{"gid":6522,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/1187279115","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":1187279115},{"type":"v1","addr":"172.21.15.92:6837","nonce":1187279115}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":43}],"filesystems":[]} 2023-11-24T06:14:11.603 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 43 2023-11-24T06:14:11.605 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:11.602+0000 7f24da743640 1 -- 172.21.15.92:0/2107887124 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f24c006af90 msgr2=0x7f24c006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:14:11.605 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:11.602+0000 7f24da743640 1 --2- 172.21.15.92:0/2107887124 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f24c006af90 0x7f24c006d450 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f24d40ce850 tx=0x7f24c8008040 comp rx=0 tx=0).stop 2023-11-24T06:14:11.605 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:11.602+0000 7f24da743640 1 -- 172.21.15.92:0/2107887124 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f24d4071030 msgr2=0x7f24d40adb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:14:11.605 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:11.602+0000 7f24da743640 1 --2- 172.21.15.92:0/2107887124 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f24d4071030 0x7f24d40adb90 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f24c402b9c0 tx=0x7f24c4004730 comp rx=0 tx=0).stop 2023-11-24T06:14:11.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:11.602+0000 7f24da743640 1 -- 172.21.15.92:0/2107887124 shutdown_connections 2023-11-24T06:14:11.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:11.602+0000 7f24da743640 1 --2- 172.21.15.92:0/2107887124 >> [v2:172.21.15.92:6800/784027401,v1:172.21.15.92:6801/784027401] conn(0x7f24c006af90 0x7f24c006d450 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:14:11.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:11.602+0000 7f24da743640 1 --2- 172.21.15.92:0/2107887124 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f24d41618f0 0x7f24d4146df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:14:11.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:11.602+0000 7f24da743640 1 --2- 172.21.15.92:0/2107887124 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f24d4157c30 0x7f24d41468b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:14:11.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:11.602+0000 7f24da743640 1 --2- 172.21.15.92:0/2107887124 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f24d4071030 0x7f24d40adb90 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:14:11.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:11.602+0000 7f24da743640 1 -- 172.21.15.92:0/2107887124 >> 172.21.15.92:0/2107887124 conn(0x7f24d40cf700 msgr2=0x7f24d406e370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:14:11.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:11.602+0000 7f24da743640 1 -- 172.21.15.92:0/2107887124 shutdown_connections 2023-11-24T06:14:11.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-24T06:14:11.602+0000 7f24da743640 1 -- 172.21.15.92:0/2107887124 wait complete. 2023-11-24T06:14:11.624 DEBUG:tasks.cephfs.filesystem:already dead... 2023-11-24T06:14:11.624 INFO:teuthology.misc:Shutting down mds daemons... 2023-11-24T06:14:11.625 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-24T06:14:11.625 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T06:14:11.651 INFO:tasks.ceph.mds.a:Stopped 2023-11-24T06:14:11.651 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-11-24T06:14:11.651 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T06:14:11.712 INFO:tasks.ceph.mds.c:Stopped 2023-11-24T06:14:11.712 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-24T06:14:11.713 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T06:14:11.743 INFO:tasks.ceph.mds.b:Stopped 2023-11-24T06:14:11.744 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-11-24T06:14:11.744 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T06:14:11.809 INFO:tasks.ceph.mds.d:Stopped 2023-11-24T06:14:11.809 INFO:teuthology.misc:Shutting down osd daemons... 2023-11-24T06:14:11.810 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2023-11-24T06:14:11.810 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T06:14:11.863 INFO:tasks.ceph.osd.0:Stopped 2023-11-24T06:14:11.864 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2023-11-24T06:14:11.864 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T06:14:11.936 INFO:tasks.ceph.osd.1:Stopped 2023-11-24T06:14:11.936 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2023-11-24T06:14:11.937 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T06:14:11.988 INFO:tasks.ceph.osd.2:Stopped 2023-11-24T06:14:11.988 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2023-11-24T06:14:11.988 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T06:14:12.036 INFO:tasks.ceph.osd.3:Stopped 2023-11-24T06:14:12.037 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2023-11-24T06:14:12.037 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T06:14:12.088 INFO:tasks.ceph.osd.4:Stopped 2023-11-24T06:14:12.088 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2023-11-24T06:14:12.088 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T06:14:12.136 INFO:tasks.ceph.osd.5:Stopped 2023-11-24T06:14:12.136 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2023-11-24T06:14:12.137 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T06:14:12.183 INFO:tasks.ceph.osd.6:Stopped 2023-11-24T06:14:12.184 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2023-11-24T06:14:12.184 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T06:14:12.240 INFO:tasks.ceph.osd.7:Stopped 2023-11-24T06:14:12.240 INFO:teuthology.misc:Shutting down mgr daemons... 2023-11-24T06:14:12.241 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2023-11-24T06:14:12.241 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T06:14:12.320 INFO:tasks.ceph.mgr.y:Stopped 2023-11-24T06:14:12.320 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2023-11-24T06:14:12.321 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T06:14:12.412 INFO:tasks.ceph.mgr.x:Stopped 2023-11-24T06:14:12.412 INFO:teuthology.misc:Shutting down mon daemons... 2023-11-24T06:14:12.412 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2023-11-24T06:14:12.413 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T06:14:12.464 INFO:tasks.ceph.mon.a:Stopped 2023-11-24T06:14:12.465 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2023-11-24T06:14:12.465 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T06:14:12.511 INFO:tasks.ceph.mon.b:Stopped 2023-11-24T06:14:12.511 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2023-11-24T06:14:12.511 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T06:14:12.556 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~0s 2023-11-24T06:14:12.556 INFO:tasks.ceph.mon.c:Stopped 2023-11-24T06:14:12.556 INFO:tasks.ceph:Checking cluster log for badness... 2023-11-24T06:14:12.557 DEBUG:teuthology.orchestra.run.smithi092:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'OSD full dropping all updates' | egrep -v 'OSD near full' | egrep -v 'pausewr flag' | egrep -v 'failsafe engaged, dropping updates' | egrep -v 'failsafe disengaged, no longer dropping' | egrep -v 'is full \(reached quota' | egrep -v POOL_FULL | egrep -v POOL_BACKFILLFULL | head -n 1 2023-11-24T06:14:12.672 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi092.front.sepia.ceph.com 2023-11-24T06:14:12.672 DEBUG:teuthology.orchestra.run.smithi092:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2023-11-24T06:14:12.921 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi092.front.sepia.ceph.com 2023-11-24T06:14:12.922 DEBUG:teuthology.orchestra.run.smithi092:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2023-11-24T06:14:13.077 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi092.front.sepia.ceph.com 2023-11-24T06:14:13.078 DEBUG:teuthology.orchestra.run.smithi092:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2023-11-24T06:14:13.234 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi092.front.sepia.ceph.com 2023-11-24T06:14:13.235 DEBUG:teuthology.orchestra.run.smithi092:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2023-11-24T06:14:13.402 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi118.front.sepia.ceph.com 2023-11-24T06:14:13.402 DEBUG:teuthology.orchestra.run.smithi118:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2023-11-24T06:14:13.656 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi118.front.sepia.ceph.com 2023-11-24T06:14:13.657 DEBUG:teuthology.orchestra.run.smithi118:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2023-11-24T06:14:13.826 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi118.front.sepia.ceph.com 2023-11-24T06:14:13.827 DEBUG:teuthology.orchestra.run.smithi118:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2023-11-24T06:14:13.995 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi118.front.sepia.ceph.com 2023-11-24T06:14:13.995 DEBUG:teuthology.orchestra.run.smithi118:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2023-11-24T06:14:14.169 INFO:tasks.ceph:Archiving mon data... 2023-11-24T06:14:14.171 DEBUG:teuthology.misc:Transferring archived files from smithi092:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/rishabh-2023-11-24_05:41:33-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7465546/data/mon.a.tgz 2023-11-24T06:14:14.171 DEBUG:teuthology.orchestra.run.smithi092:> mktemp 2023-11-24T06:14:14.231 INFO:teuthology.orchestra.run.smithi092.stdout:/tmp/tmp.gdcFVjjKOv 2023-11-24T06:14:14.231 DEBUG:teuthology.orchestra.run.smithi092:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.gdcFVjjKOv 2023-11-24T06:14:14.440 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod 0666 /tmp/tmp.gdcFVjjKOv 2023-11-24T06:14:14.593 DEBUG:teuthology.orchestra.remote:smithi092:/tmp/tmp.gdcFVjjKOv is 459KB 2023-11-24T06:14:14.705 DEBUG:teuthology.orchestra.run.smithi092:> rm -fr /tmp/tmp.gdcFVjjKOv 2023-11-24T06:14:14.766 DEBUG:teuthology.misc:Transferring archived files from smithi118:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/rishabh-2023-11-24_05:41:33-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7465546/data/mon.b.tgz 2023-11-24T06:14:14.766 DEBUG:teuthology.orchestra.run.smithi118:> mktemp 2023-11-24T06:14:14.827 INFO:teuthology.orchestra.run.smithi118.stdout:/tmp/tmp.Pr3hG0U4Lv 2023-11-24T06:14:14.827 DEBUG:teuthology.orchestra.run.smithi118:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.Pr3hG0U4Lv 2023-11-24T06:14:15.064 DEBUG:teuthology.orchestra.run.smithi118:> sudo chmod 0666 /tmp/tmp.Pr3hG0U4Lv 2023-11-24T06:14:15.216 DEBUG:teuthology.orchestra.remote:smithi118:/tmp/tmp.Pr3hG0U4Lv is 609KB 2023-11-24T06:14:15.321 DEBUG:teuthology.orchestra.run.smithi118:> rm -fr /tmp/tmp.Pr3hG0U4Lv 2023-11-24T06:14:15.378 DEBUG:teuthology.misc:Transferring archived files from smithi118:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/rishabh-2023-11-24_05:41:33-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7465546/data/mon.c.tgz 2023-11-24T06:14:15.379 DEBUG:teuthology.orchestra.run.smithi118:> mktemp 2023-11-24T06:14:15.431 INFO:teuthology.orchestra.run.smithi118.stdout:/tmp/tmp.Du3xOIIyOK 2023-11-24T06:14:15.431 DEBUG:teuthology.orchestra.run.smithi118:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.Du3xOIIyOK 2023-11-24T06:14:15.656 DEBUG:teuthology.orchestra.run.smithi118:> sudo chmod 0666 /tmp/tmp.Du3xOIIyOK 2023-11-24T06:14:15.807 DEBUG:teuthology.orchestra.remote:smithi118:/tmp/tmp.Du3xOIIyOK is 609KB 2023-11-24T06:14:15.918 DEBUG:teuthology.orchestra.run.smithi118:> rm -fr /tmp/tmp.Du3xOIIyOK 2023-11-24T06:14:15.978 INFO:tasks.ceph:Cleaning ceph cluster... 2023-11-24T06:14:15.978 DEBUG:teuthology.orchestra.run.smithi092:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-11-24T06:14:15.987 DEBUG:teuthology.orchestra.run.smithi118:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-11-24T06:14:16.081 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2023-11-24T06:14:16.082 DEBUG:teuthology.orchestra.run.smithi092:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-11-24T06:14:16.090 DEBUG:teuthology.orchestra.run.smithi118:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-11-24T06:14:16.189 INFO:teuthology.orchestra.run.smithi092.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-11-24T06:14:16.195 INFO:teuthology.orchestra.run.smithi118.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-11-24T06:14:16.210 INFO:tasks.ceph:Archiving crash dumps... 2023-11-24T06:14:16.211 DEBUG:teuthology.misc:Transferring archived files from smithi092:/var/lib/ceph/crash to /home/teuthworker/archive/rishabh-2023-11-24_05:41:33-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7465546/remote/smithi092/crash 2023-11-24T06:14:16.212 DEBUG:teuthology.orchestra.run.smithi092:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-11-24T06:14:16.298 DEBUG:teuthology.misc:Transferring archived files from smithi118:/var/lib/ceph/crash to /home/teuthworker/archive/rishabh-2023-11-24_05:41:33-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7465546/remote/smithi118/crash 2023-11-24T06:14:16.299 DEBUG:teuthology.orchestra.run.smithi118:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-11-24T06:14:16.394 INFO:tasks.ceph:Shutting down logrotate 2023-11-24T06:14:16.395 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T06:14:16.404 DEBUG:teuthology.orchestra.run.smithi118:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T06:14:16.536 DEBUG:teuthology.orchestra.run.smithi092:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-11-24T06:14:16.636 DEBUG:teuthology.orchestra.run.smithi118:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-11-24T06:14:16.738 INFO:tasks.ceph:Compressing logs... 2023-11-24T06:14:16.738 DEBUG:teuthology.orchestra.run.smithi092:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-11-24T06:14:16.748 DEBUG:teuthology.orchestra.run.smithi118:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-11-24T06:14:16.833 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110903.log 2023-11-24T06:14:16.834 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103931.log 2023-11-24T06:14:16.835 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.110903.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103461.log 2023-11-24T06:14:16.836 INFO:teuthology.orchestra.run.smithi092.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.110903.log.gz 2023-11-24T06:14:16.836 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.103931.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97948.log 2023-11-24T06:14:16.837 INFO:teuthology.orchestra.run.smithi092.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.103931.log.gz 2023-11-24T06:14:16.838 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.103461.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100425.log 2023-11-24T06:14:16.838 INFO:teuthology.orchestra.run.smithi092.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.103461.log.gz 2023-11-24T06:14:16.839 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.97948.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.97948.log.gz 2023-11-24T06:14:16.839 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2023-11-24T06:14:16.840 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.100425.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105530.log 2023-11-24T06:14:16.840 INFO:teuthology.orchestra.run.smithi092.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.100425.log.gz 2023-11-24T06:14:16.841 INFO:teuthology.orchestra.run.smithi118.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2023-11-24T06:14:16.841 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101630.log 2023-11-24T06:14:16.842 INFO:teuthology.orchestra.run.smithi118.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88379.log 2023-11-24T06:14:16.843 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.105530.log: gzip 88.0% -- replaced with /var/log/ceph/ceph-client.admin.105530.log.gz -5 2023-11-24T06:14:16.843 INFO:teuthology.orchestra.run.smithi092.stderr: --verbose -- /var/log/ceph/ceph-client.admin.99441.log 2023-11-24T06:14:16.844 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88579.log 2023-11-24T06:14:16.845 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.88379.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.95623.log 2023-11-24T06:14:16.845 INFO:teuthology.orchestra.run.smithi118.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.88379.log.gz 2023-11-24T06:14:16.845 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.101630.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.101630.log.gz 2023-11-24T06:14:16.845 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96226.log 2023-11-24T06:14:16.845 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.99441.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108584.log 2023-11-24T06:14:16.846 INFO:teuthology.orchestra.run.smithi092.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.99441.log.gz 2023-11-24T06:14:16.847 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.96226.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111225.log 2023-11-24T06:14:16.847 INFO:teuthology.orchestra.run.smithi092.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.96226.log.gz 2023-11-24T06:14:16.847 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.88579.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2023-11-24T06:14:16.847 INFO:teuthology.orchestra.run.smithi118.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.88579.log.gz 2023-11-24T06:14:16.848 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.1.95623.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2023-11-24T06:14:16.849 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.108584.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96688.log 2023-11-24T06:14:16.849 INFO:teuthology.orchestra.run.smithi092.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.108584.log.gz 2023-11-24T06:14:16.849 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.111225.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.111225.log.gz 2023-11-24T06:14:16.849 INFO:teuthology.orchestra.run.smithi118.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.1.95623.log.gz 2023-11-24T06:14:16.849 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.94335.log 2023-11-24T06:14:16.850 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2023-11-24T06:14:16.851 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102419.log 2023-11-24T06:14:16.851 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.96688.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102503.log 2023-11-24T06:14:16.851 INFO:teuthology.orchestra.run.smithi092.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.96688.log.gz 2023-11-24T06:14:16.852 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.102419.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101841.log 2023-11-24T06:14:16.852 INFO:teuthology.orchestra.run.smithi092.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.102419.log.gz 2023-11-24T06:14:16.852 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.1.94335.log: 93.7% -- replaced with /var/log/ceph/ceph.audit.log.gz 2023-11-24T06:14:16.853 INFO:teuthology.orchestra.run.smithi118.stderr: 92.0% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2023-11-24T06:14:16.853 INFO:teuthology.orchestra.run.smithi118.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88499.log 2023-11-24T06:14:16.853 INFO:teuthology.orchestra.run.smithi118.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.1.94335.log.gz 2023-11-24T06:14:16.853 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88459.log 2023-11-24T06:14:16.855 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.88499.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2023-11-24T06:14:16.855 INFO:teuthology.orchestra.run.smithi118.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.88499.log.gz 2023-11-24T06:14:16.855 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.88459.log: gzip -5 --verbose 88.0% -- -- replaced with /var/log/ceph/ceph-client.admin.88459.log.gz /var/log/ceph/ceph-osd.5.log 2023-11-24T06:14:16.855 INFO:teuthology.orchestra.run.smithi118.stderr: 2023-11-24T06:14:16.856 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.102503.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.102503.log.gz 2023-11-24T06:14:16.856 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100713.log 2023-11-24T06:14:16.856 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.101841.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.101841.log.gz 2023-11-24T06:14:16.856 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108836.log 2023-11-24T06:14:16.856 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.100713.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109546.log 2023-11-24T06:14:16.856 INFO:teuthology.orchestra.run.smithi092.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.100713.log.gz 2023-11-24T06:14:16.858 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88539.log 2023-11-24T06:14:16.858 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.3.93594.log 2023-11-24T06:14:16.858 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.108836.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100881.log 2023-11-24T06:14:16.858 INFO:teuthology.orchestra.run.smithi092.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.108836.log.gz 2023-11-24T06:14:16.858 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.109546.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103643.log 2023-11-24T06:14:16.859 INFO:teuthology.orchestra.run.smithi092.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.109546.log.gz 2023-11-24T06:14:16.860 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.88539.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88339.log 2023-11-24T06:14:16.860 INFO:teuthology.orchestra.run.smithi118.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.88539.log.gz 2023-11-24T06:14:16.861 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.100881.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105352.log 2023-11-24T06:14:16.861 INFO:teuthology.orchestra.run.smithi092.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.100881.log.gz 2023-11-24T06:14:16.861 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.103643.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100671.log 2023-11-24T06:14:16.861 INFO:teuthology.orchestra.run.smithi092.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.103643.log.gz 2023-11-24T06:14:16.861 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.105352.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94157.log 2023-11-24T06:14:16.861 INFO:teuthology.orchestra.run.smithi092.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.105352.log.gz 2023-11-24T06:14:16.862 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.3.93594.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2023-11-24T06:14:16.862 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.88339.log: 91.3% -- replaced with /var/log/ceph/ceph-client.3.93594.log.gz 2023-11-24T06:14:16.863 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.100671.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105446.log 2023-11-24T06:14:16.863 INFO:teuthology.orchestra.run.smithi092.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.100671.log.gz 2023-11-24T06:14:16.863 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.94157.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.94157.log.gz 2023-11-24T06:14:16.863 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110819.log 2023-11-24T06:14:16.864 INFO:teuthology.orchestra.run.smithi118.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.93142.log 2023-11-24T06:14:16.865 INFO:teuthology.orchestra.run.smithi118.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.88339.log.gz 2023-11-24T06:14:16.865 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2023-11-24T06:14:16.865 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.105446.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.105446.log.gz 2023-11-24T06:14:16.865 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111183.log 2023-11-24T06:14:16.866 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.1.93142.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.94964.log 2023-11-24T06:14:16.866 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.110819.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.110819.log.gz 2023-11-24T06:14:16.867 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100587.log 2023-11-24T06:14:16.867 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.111183.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102051.log 2023-11-24T06:14:16.867 INFO:teuthology.orchestra.run.smithi092.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.111183.log.gz 2023-11-24T06:14:16.868 INFO:teuthology.orchestra.run.smithi118.stderr: 90.4%/var/log/ceph/ceph.log: -- replaced with /var/log/ceph/ceph-client.1.93142.log.gz 2023-11-24T06:14:16.868 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.100587.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98658.log 2023-11-24T06:14:16.868 INFO:teuthology.orchestra.run.smithi092.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.100587.log.gz 2023-11-24T06:14:16.869 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.1.94964.log: gzip -5 92.1% --verbose -- replaced with /var/log/ceph/ceph.log.gz -- 2023-11-24T06:14:16.869 INFO:teuthology.orchestra.run.smithi118.stderr: /var/log/ceph/ceph-mon.b.log 2023-11-24T06:14:16.870 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.102051.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98364.log 2023-11-24T06:14:16.870 INFO:teuthology.orchestra.run.smithi092.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.102051.log.gz 2023-11-24T06:14:16.871 INFO:teuthology.orchestra.run.smithi118.stderr: 93.2% 88.6% -- replaced with /var/log/ceph/ceph-client.1.94964.log.gz 2023-11-24T06:14:16.871 INFO:teuthology.orchestra.run.smithi118.stderr: -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2023-11-24T06:14:16.871 INFO:teuthology.orchestra.run.smithi118.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88299.log 2023-11-24T06:14:16.872 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.98658.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.98658.log.gz 2023-11-24T06:14:16.872 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105742.log 2023-11-24T06:14:16.872 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.98364.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96832.log 2023-11-24T06:14:16.872 INFO:teuthology.orchestra.run.smithi092.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.98364.log.gz 2023-11-24T06:14:16.873 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2023-11-24T06:14:16.874 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.105742.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100965.log 2023-11-24T06:14:16.874 INFO:teuthology.orchestra.run.smithi092.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.105742.log.gz 2023-11-24T06:14:16.874 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.96832.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103847.log 2023-11-24T06:14:16.874 INFO:teuthology.orchestra.run.smithi092.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.96832.log.gz 2023-11-24T06:14:16.875 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.88299.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88419.log 2023-11-24T06:14:16.875 INFO:teuthology.orchestra.run.smithi118.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.88299.log.gz 2023-11-24T06:14:16.875 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.100965.log: gzip 89.4% -5 -- replaced with /var/log/ceph/ceph-client.admin.100965.log.gz --verbose 2023-11-24T06:14:16.876 INFO:teuthology.orchestra.run.smithi092.stderr: -- /var/log/ceph/ceph-client.admin.88501.log 2023-11-24T06:14:16.876 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2023-11-24T06:14:16.877 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.103847.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103559.log 2023-11-24T06:14:16.877 INFO:teuthology.orchestra.run.smithi092.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.103847.log.gz 2023-11-24T06:14:16.877 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.88419.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.88419.log.gz 2023-11-24T06:14:16.878 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.88501.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94672.log 2023-11-24T06:14:16.878 INFO:teuthology.orchestra.run.smithi092.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.88501.log.gz 2023-11-24T06:14:16.879 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.103559.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101091.log 2023-11-24T06:14:16.879 INFO:teuthology.orchestra.run.smithi092.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.103559.log.gz 2023-11-24T06:14:16.879 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.94672.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108626.log 2023-11-24T06:14:16.880 INFO:teuthology.orchestra.run.smithi092.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.94672.log.gz 2023-11-24T06:14:16.881 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.101091.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96268.log 2023-11-24T06:14:16.881 INFO:teuthology.orchestra.run.smithi092.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.101091.log.gz 2023-11-24T06:14:16.882 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.108626.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.108626.log.gz 2023-11-24T06:14:16.882 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103153.log 2023-11-24T06:14:16.883 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.96268.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111351.log 2023-11-24T06:14:16.883 INFO:teuthology.orchestra.run.smithi092.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.96268.log.gz 2023-11-24T06:14:16.884 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.103153.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103111.log 2023-11-24T06:14:16.884 INFO:teuthology.orchestra.run.smithi092.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.103153.log.gz 2023-11-24T06:14:16.885 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.111351.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109807.log 2023-11-24T06:14:16.886 INFO:teuthology.orchestra.run.smithi092.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.111351.log.gz 2023-11-24T06:14:16.887 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.103111.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102009.log 2023-11-24T06:14:16.887 INFO:teuthology.orchestra.run.smithi092.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.103111.log.gz 2023-11-24T06:14:16.888 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.109807.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.109807.log.gz 2023-11-24T06:14:16.888 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96478.log 2023-11-24T06:14:16.889 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.102009.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105266.log 2023-11-24T06:14:16.889 INFO:teuthology.orchestra.run.smithi092.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.102009.log.gz 2023-11-24T06:14:16.890 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.96478.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101883.log 2023-11-24T06:14:16.891 INFO:teuthology.orchestra.run.smithi092.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.96478.log.gz 2023-11-24T06:14:16.891 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.105266.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101427.log 2023-11-24T06:14:16.892 INFO:teuthology.orchestra.run.smithi092.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.105266.log.gz 2023-11-24T06:14:16.893 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.101883.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100299.log 2023-11-24T06:14:16.893 INFO:teuthology.orchestra.run.smithi092.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.101883.log.gz 2023-11-24T06:14:16.894 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.101427.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102189.log 2023-11-24T06:14:16.894 INFO:teuthology.orchestra.run.smithi092.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.101427.log.gz 2023-11-24T06:14:16.895 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.100299.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101967.log 2023-11-24T06:14:16.895 INFO:teuthology.orchestra.run.smithi092.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.100299.log.gz 2023-11-24T06:14:16.896 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.102189.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111519.log 2023-11-24T06:14:16.896 INFO:teuthology.orchestra.run.smithi092.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.102189.log.gz 2023-11-24T06:14:16.897 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.101967.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103727.log 2023-11-24T06:14:16.897 INFO:teuthology.orchestra.run.smithi092.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.101967.log.gz 2023-11-24T06:14:16.899 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.111519.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105140.log 2023-11-24T06:14:16.899 INFO:teuthology.orchestra.run.smithi092.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.111519.log.gz 2023-11-24T06:14:16.900 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.103727.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99357.log 2023-11-24T06:14:16.900 INFO:teuthology.orchestra.run.smithi092.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.103727.log.gz 2023-11-24T06:14:16.901 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.105140.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110506.log 2023-11-24T06:14:16.901 INFO:teuthology.orchestra.run.smithi092.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.105140.log.gz 2023-11-24T06:14:16.902 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.99357.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99873.log 2023-11-24T06:14:16.902 INFO:teuthology.orchestra.run.smithi092.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.99357.log.gz 2023-11-24T06:14:16.903 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.110506.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102139.log 2023-11-24T06:14:16.903 INFO:teuthology.orchestra.run.smithi092.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.110506.log.gz 2023-11-24T06:14:16.904 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.99873.log: gzip -5 --verbose -- 88.4% /var/log/ceph/ceph-client.admin.95222.log -- replaced with /var/log/ceph/ceph-client.admin.99873.log.gz 2023-11-24T06:14:16.905 INFO:teuthology.orchestra.run.smithi092.stderr: 2023-11-24T06:14:16.905 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-mon.c.log: 91.6% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2023-11-24T06:14:16.906 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.102139.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.102139.log.gz 2023-11-24T06:14:16.906 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96604.log 2023-11-24T06:14:16.907 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.95222.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.95222.log.gz 2023-11-24T06:14:16.907 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111663.log 2023-11-24T06:14:16.908 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.96604.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98826.log 2023-11-24T06:14:16.908 INFO:teuthology.orchestra.run.smithi092.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.96604.log.gz 2023-11-24T06:14:16.909 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.111663.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98227.log 2023-11-24T06:14:16.909 INFO:teuthology.orchestra.run.smithi092.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.111663.log.gz 2023-11-24T06:14:16.911 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.98826.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103889.log 2023-11-24T06:14:16.911 INFO:teuthology.orchestra.run.smithi092.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.98826.log.gz 2023-11-24T06:14:16.912 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.98227.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.98227.log.gz 2023-11-24T06:14:16.912 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111477.log 2023-11-24T06:14:16.913 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.103889.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.103889.log.gz 2023-11-24T06:14:16.913 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95556.log 2023-11-24T06:14:16.915 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.111477.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108710.log 2023-11-24T06:14:16.915 INFO:teuthology.orchestra.run.smithi092.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.111477.log.gz 2023-11-24T06:14:16.916 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.95556.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99603.log 2023-11-24T06:14:16.916 INFO:teuthology.orchestra.run.smithi092.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.95556.log.gz 2023-11-24T06:14:16.917 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.108710.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96184.log 2023-11-24T06:14:16.917 INFO:teuthology.orchestra.run.smithi092.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.108710.log.gz 2023-11-24T06:14:16.918 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.99603.log: gzip 88.3% -- replaced with /var/log/ceph/ceph-client.admin.99603.log.gz -5 2023-11-24T06:14:16.918 INFO:teuthology.orchestra.run.smithi092.stderr: --verbose -- /var/log/ceph/ceph-client.admin.99957.log 2023-11-24T06:14:16.919 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.96184.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105784.log 2023-11-24T06:14:16.919 INFO:teuthology.orchestra.run.smithi092.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.96184.log.gz 2023-11-24T06:14:16.920 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.99957.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.99957.log.gz 2023-11-24T06:14:16.920 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107286.log 2023-11-24T06:14:16.921 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.105784.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98952.log 2023-11-24T06:14:16.921 INFO:teuthology.orchestra.run.smithi092.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.105784.log.gz 2023-11-24T06:14:16.922 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.107286.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95897.log 2023-11-24T06:14:16.922 INFO:teuthology.orchestra.run.smithi092.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.107286.log.gz 2023-11-24T06:14:16.924 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.98952.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104673.log 2023-11-24T06:14:16.924 INFO:teuthology.orchestra.run.smithi092.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.98952.log.gz 2023-11-24T06:14:16.925 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.95897.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.95897.log.gz 2023-11-24T06:14:16.925 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107328.log 2023-11-24T06:14:16.926 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.104673.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.104673.log.gz 2023-11-24T06:14:16.926 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107683.log 2023-11-24T06:14:16.927 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.107328.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101301.log 2023-11-24T06:14:16.928 INFO:teuthology.orchestra.run.smithi092.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.107328.log.gz 2023-11-24T06:14:16.928 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.107683.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98868.log 2023-11-24T06:14:16.928 INFO:teuthology.orchestra.run.smithi092.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.107683.log.gz 2023-11-24T06:14:16.930 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.101301.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94882.log 2023-11-24T06:14:16.930 INFO:teuthology.orchestra.run.smithi092.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.101301.log.gz 2023-11-24T06:14:16.931 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.98868.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.98868.log.gz 2023-11-24T06:14:16.931 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108540.log 2023-11-24T06:14:16.932 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.94882.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110380.log 2023-11-24T06:14:16.932 INFO:teuthology.orchestra.run.smithi092.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.94882.log.gz 2023-11-24T06:14:16.933 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.108540.log: gzip 89.1% -- replaced with /var/log/ceph/ceph-client.admin.108540.log.gz -5 2023-11-24T06:14:16.934 INFO:teuthology.orchestra.run.smithi092.stderr: --verbose -- /var/log/ceph/ceph-client.admin.104426.log 2023-11-24T06:14:16.934 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.110380.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95562.log 2023-11-24T06:14:16.935 INFO:teuthology.orchestra.run.smithi092.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.110380.log.gz 2023-11-24T06:14:16.935 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.104426.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107851.log 2023-11-24T06:14:16.936 INFO:teuthology.orchestra.run.smithi092.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.104426.log.gz 2023-11-24T06:14:16.937 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.95562.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107538.log 2023-11-24T06:14:16.938 INFO:teuthology.orchestra.run.smithi092.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.95562.log.gz 2023-11-24T06:14:16.938 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.107851.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.107851.log.gz 2023-11-24T06:14:16.938 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100629.log 2023-11-24T06:14:16.939 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.107538.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102377.log 2023-11-24T06:14:16.940 INFO:teuthology.orchestra.run.smithi092.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.107538.log.gz 2023-11-24T06:14:16.940 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.100629.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94966.log 2023-11-24T06:14:16.941 INFO:teuthology.orchestra.run.smithi092.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.100629.log.gz 2023-11-24T06:14:16.942 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.102377.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.106211.log 2023-11-24T06:14:16.942 INFO:teuthology.orchestra.run.smithi092.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.102377.log.gz 2023-11-24T06:14:16.943 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.94966.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104468.log 2023-11-24T06:14:16.943 INFO:teuthology.orchestra.run.smithi092.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.94966.log.gz 2023-11-24T06:14:16.944 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.0.106211.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98784.log 2023-11-24T06:14:16.945 INFO:teuthology.orchestra.run.smithi092.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.0.106211.log.gz 2023-11-24T06:14:16.946 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.104468.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111267.log 2023-11-24T06:14:16.946 INFO:teuthology.orchestra.run.smithi092.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.104468.log.gz 2023-11-24T06:14:16.947 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.98784.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100031.log 2023-11-24T06:14:16.947 INFO:teuthology.orchestra.run.smithi092.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.98784.log.gz 2023-11-24T06:14:16.948 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.111267.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.111267.log.gz 2023-11-24T06:14:16.948 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106518.log 2023-11-24T06:14:16.949 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.100031.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94059.log 2023-11-24T06:14:16.949 INFO:teuthology.orchestra.run.smithi092.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.100031.log.gz 2023-11-24T06:14:16.950 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.106518.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107496.log 2023-11-24T06:14:16.950 INFO:teuthology.orchestra.run.smithi092.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.106518.log.gz 2023-11-24T06:14:16.951 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.94059.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106959.log 2023-11-24T06:14:16.952 INFO:teuthology.orchestra.run.smithi092.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.94059.log.gz 2023-11-24T06:14:16.953 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.107496.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98616.log 2023-11-24T06:14:16.953 INFO:teuthology.orchestra.run.smithi092.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.107496.log.gz 2023-11-24T06:14:16.954 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.106959.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104757.log 2023-11-24T06:14:16.954 INFO:teuthology.orchestra.run.smithi092.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.106959.log.gz 2023-11-24T06:14:16.955 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.98616.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.98616.log.gz 2023-11-24T06:14:16.955 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106574.log 2023-11-24T06:14:16.956 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.104757.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106350.log 2023-11-24T06:14:16.957 INFO:teuthology.orchestra.run.smithi092.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.104757.log.gz 2023-11-24T06:14:16.957 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.106574.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110254.log 2023-11-24T06:14:16.958 INFO:teuthology.orchestra.run.smithi092.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.106574.log.gz 2023-11-24T06:14:16.959 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.106350.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97438.log 2023-11-24T06:14:16.959 INFO:teuthology.orchestra.run.smithi092.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.106350.log.gz 2023-11-24T06:14:16.960 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.110254.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.110254.log.gz 2023-11-24T06:14:16.960 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98100.log 2023-11-24T06:14:16.961 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.97438.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95564.log 2023-11-24T06:14:16.961 INFO:teuthology.orchestra.run.smithi092.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.97438.log.gz 2023-11-24T06:14:16.962 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.98100.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101757.log 2023-11-24T06:14:16.962 INFO:teuthology.orchestra.run.smithi092.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.98100.log.gz 2023-11-24T06:14:16.963 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.95564.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95215.log 2023-11-24T06:14:16.963 INFO:teuthology.orchestra.run.smithi092.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.95564.log.gz 2023-11-24T06:14:16.964 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.101757.log: 88.7%gzip -- replaced with /var/log/ceph/ceph-client.admin.101757.log.gz 2023-11-24T06:14:16.964 INFO:teuthology.orchestra.run.smithi092.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.107244.log 2023-11-24T06:14:16.965 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.95215.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103237.log 2023-11-24T06:14:16.965 INFO:teuthology.orchestra.run.smithi092.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.95215.log.gz 2023-11-24T06:14:16.966 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.107244.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105934.log 2023-11-24T06:14:16.966 INFO:teuthology.orchestra.run.smithi092.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.107244.log.gz 2023-11-24T06:14:16.967 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.103237.log: 88.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.103237.log.gz 2023-11-24T06:14:16.968 INFO:teuthology.orchestra.run.smithi092.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.101385.log 2023-11-24T06:14:16.968 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.105934.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94756.log 2023-11-24T06:14:16.969 INFO:teuthology.orchestra.run.smithi092.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.105934.log.gz 2023-11-24T06:14:16.969 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.101385.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110632.log 2023-11-24T06:14:16.970 INFO:teuthology.orchestra.run.smithi092.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.101385.log.gz 2023-11-24T06:14:16.970 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.94756.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111057.log 2023-11-24T06:14:16.971 INFO:teuthology.orchestra.run.smithi092.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.94756.log.gz 2023-11-24T06:14:16.971 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.110632.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94798.log 2023-11-24T06:14:16.972 INFO:teuthology.orchestra.run.smithi092.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.110632.log.gz 2023-11-24T06:14:16.973 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.111057.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109849.log 2023-11-24T06:14:16.973 INFO:teuthology.orchestra.run.smithi092.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.111057.log.gz 2023-11-24T06:14:16.974 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.94798.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94450.log 2023-11-24T06:14:16.974 INFO:teuthology.orchestra.run.smithi092.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.94798.log.gz 2023-11-24T06:14:16.975 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.109849.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.109849.log.gz 2023-11-24T06:14:16.975 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2023-11-24T06:14:16.976 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.94450.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106476.log 2023-11-24T06:14:16.976 INFO:teuthology.orchestra.run.smithi092.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.94450.log.gz 2023-11-24T06:14:16.977 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104978.log 2023-11-24T06:14:16.978 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.106476.log: 89.0% -- replaced with /var/log/ceph/ceph.audit.log.gz 2023-11-24T06:14:16.979 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103375.log 2023-11-24T06:14:16.979 INFO:teuthology.orchestra.run.smithi092.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.106476.log.gz 2023-11-24T06:14:16.980 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.104978.log: 87.3%gzip -- replaced with /var/log/ceph/ceph-client.admin.104978.log.gz -5 2023-11-24T06:14:16.980 INFO:teuthology.orchestra.run.smithi092.stderr: --verbose -- /var/log/ceph/ceph-client.admin.105826.log 2023-11-24T06:14:16.981 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.103375.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.103375.log.gz 2023-11-24T06:14:16.981 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98277.log 2023-11-24T06:14:16.982 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.105826.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104132.log 2023-11-24T06:14:16.982 INFO:teuthology.orchestra.run.smithi092.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.105826.log.gz 2023-11-24T06:14:16.983 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.98277.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95211.log 2023-11-24T06:14:16.983 INFO:teuthology.orchestra.run.smithi092.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.98277.log.gz 2023-11-24T06:14:16.984 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.104132.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.104132.log.gz 2023-11-24T06:14:16.984 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104090.log 2023-11-24T06:14:16.985 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.95211.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107725.log 2023-11-24T06:14:16.985 INFO:teuthology.orchestra.run.smithi092.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.95211.log.gz 2023-11-24T06:14:16.986 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.104090.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95230.log 2023-11-24T06:14:16.986 INFO:teuthology.orchestra.run.smithi092.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.104090.log.gz 2023-11-24T06:14:16.987 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.107725.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111393.log 2023-11-24T06:14:16.987 INFO:teuthology.orchestra.run.smithi092.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.107725.log.gz 2023-11-24T06:14:16.988 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.95230.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111873.log 2023-11-24T06:14:16.988 INFO:teuthology.orchestra.run.smithi092.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.95230.log.gz 2023-11-24T06:14:16.989 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.111393.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98490.log 2023-11-24T06:14:16.989 INFO:teuthology.orchestra.run.smithi092.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.111393.log.gz 2023-11-24T06:14:16.990 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.111873.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109765.log 2023-11-24T06:14:16.990 INFO:teuthology.orchestra.run.smithi092.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.111873.log.gz 2023-11-24T06:14:16.991 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.98490.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105616.log 2023-11-24T06:14:16.991 INFO:teuthology.orchestra.run.smithi092.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.98490.log.gz 2023-11-24T06:14:16.993 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.109765.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.102972.log 2023-11-24T06:14:16.993 INFO:teuthology.orchestra.run.smithi092.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.109765.log.gz 2023-11-24T06:14:16.994 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.105616.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105868.log 2023-11-24T06:14:16.994 INFO:teuthology.orchestra.run.smithi092.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.105616.log.gz 2023-11-24T06:14:16.995 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.0.102972.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99399.log 2023-11-24T06:14:16.996 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.105868.log: 90.4% -- replaced with /var/log/ceph/ceph-client.0.102972.log.gz 2023-11-24T06:14:16.996 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108318.log 2023-11-24T06:14:16.996 INFO:teuthology.orchestra.run.smithi092.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.105868.log.gz 2023-11-24T06:14:16.997 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.99399.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107935.log 2023-11-24T06:14:16.997 INFO:teuthology.orchestra.run.smithi092.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.99399.log.gz 2023-11-24T06:14:16.998 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.108318.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.108318.log.gz 2023-11-24T06:14:16.998 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108962.log 2023-11-24T06:14:16.999 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.107935.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111099.log 2023-11-24T06:14:16.999 INFO:teuthology.orchestra.run.smithi092.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.107935.log.gz 2023-11-24T06:14:17.000 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.108962.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98700.log 2023-11-24T06:14:17.001 INFO:teuthology.orchestra.run.smithi092.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.108962.log.gz 2023-11-24T06:14:17.001 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.111099.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95050.log 2023-11-24T06:14:17.002 INFO:teuthology.orchestra.run.smithi092.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.111099.log.gz 2023-11-24T06:14:17.003 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.98700.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107809.log 2023-11-24T06:14:17.003 INFO:teuthology.orchestra.run.smithi092.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.98700.log.gz 2023-11-24T06:14:17.005 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.95050.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.95050.log.gz 2023-11-24T06:14:17.006 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108794.log 2023-11-24T06:14:17.006 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.107809.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100115.log 2023-11-24T06:14:17.006 INFO:teuthology.orchestra.run.smithi092.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.107809.log.gz 2023-11-24T06:14:17.007 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.108794.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111915.log 2023-11-24T06:14:17.007 INFO:teuthology.orchestra.run.smithi092.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.108794.log.gz 2023-11-24T06:14:17.008 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.100115.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107043.log 2023-11-24T06:14:17.008 INFO:teuthology.orchestra.run.smithi092.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.100115.log.gz 2023-11-24T06:14:17.008 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.111915.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105700.log 2023-11-24T06:14:17.009 INFO:teuthology.orchestra.run.smithi092.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.111915.log.gz 2023-11-24T06:14:17.009 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.107043.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100839.log 2023-11-24T06:14:17.009 INFO:teuthology.orchestra.run.smithi092.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.107043.log.gz 2023-11-24T06:14:17.010 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.105700.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100157.log 2023-11-24T06:14:17.010 INFO:teuthology.orchestra.run.smithi092.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.105700.log.gz 2023-11-24T06:14:17.011 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.100839.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108233.log 2023-11-24T06:14:17.011 INFO:teuthology.orchestra.run.smithi092.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.100839.log.gz 2023-11-24T06:14:17.013 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.100157.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100545.log 2023-11-24T06:14:17.013 INFO:teuthology.orchestra.run.smithi092.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.100157.log.gz 2023-11-24T06:14:17.014 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.108233.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.108233.log.gz 2023-11-24T06:14:17.014 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110338.log 2023-11-24T06:14:17.015 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.100545.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111747.log 2023-11-24T06:14:17.015 INFO:teuthology.orchestra.run.smithi092.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.100545.log.gz 2023-11-24T06:14:17.016 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.110338.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103973.log 2023-11-24T06:14:17.016 INFO:teuthology.orchestra.run.smithi092.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.110338.log.gz 2023-11-24T06:14:17.017 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.111747.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98532.log 2023-11-24T06:14:17.017 INFO:teuthology.orchestra.run.smithi092.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.111747.log.gz 2023-11-24T06:14:17.018 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.103973.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95548.log 2023-11-24T06:14:17.019 INFO:teuthology.orchestra.run.smithi092.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.103973.log.gz 2023-11-24T06:14:17.019 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.98532.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.98532.log.gz 2023-11-24T06:14:17.019 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110861.log 2023-11-24T06:14:17.020 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.95548.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109933.log 2023-11-24T06:14:17.020 INFO:teuthology.orchestra.run.smithi092.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.95548.log.gz 2023-11-24T06:14:17.021 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.110861.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102335.log 2023-11-24T06:14:17.021 INFO:teuthology.orchestra.run.smithi092.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.110861.log.gz 2023-11-24T06:14:17.023 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.109933.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103417.log 2023-11-24T06:14:17.023 INFO:teuthology.orchestra.run.smithi092.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.109933.log.gz 2023-11-24T06:14:17.024 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.102335.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110777.log 2023-11-24T06:14:17.024 INFO:teuthology.orchestra.run.smithi092.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.102335.log.gz 2023-11-24T06:14:17.025 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.103417.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98321.log 2023-11-24T06:14:17.025 INFO:teuthology.orchestra.run.smithi092.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.103417.log.gz 2023-11-24T06:14:17.026 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.110777.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.110777.log.gz 2023-11-24T06:14:17.026 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101715.log 2023-11-24T06:14:17.027 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.98321.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111831.log 2023-11-24T06:14:17.027 INFO:teuthology.orchestra.run.smithi092.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.98321.log.gz 2023-11-24T06:14:17.028 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.101715.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104631.log 2023-11-24T06:14:17.028 INFO:teuthology.orchestra.run.smithi092.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.101715.log.gz 2023-11-24T06:14:17.029 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.111831.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96352.log 2023-11-24T06:14:17.030 INFO:teuthology.orchestra.run.smithi092.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.111831.log.gz 2023-11-24T06:14:17.030 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.104631.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102695.log 2023-11-24T06:14:17.031 INFO:teuthology.orchestra.run.smithi092.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.104631.log.gz 2023-11-24T06:14:17.032 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.96352.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104589.log 2023-11-24T06:14:17.032 INFO:teuthology.orchestra.run.smithi092.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.96352.log.gz 2023-11-24T06:14:17.033 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.102695.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101549.log 2023-11-24T06:14:17.033 INFO:teuthology.orchestra.run.smithi092.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.102695.log.gz 2023-11-24T06:14:17.034 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.104589.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94840.log 2023-11-24T06:14:17.034 INFO:teuthology.orchestra.run.smithi092.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.104589.log.gz 2023-11-24T06:14:17.035 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.101549.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2023-11-24T06:14:17.035 INFO:teuthology.orchestra.run.smithi092.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.101549.log.gz 2023-11-24T06:14:17.036 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.94840.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.94840.log.gz 2023-11-24T06:14:17.036 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99131.log 2023-11-24T06:14:17.037 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101217.log 2023-11-24T06:14:17.038 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.99131.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88233.log 2023-11-24T06:14:17.038 INFO:teuthology.orchestra.run.smithi092.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.99131.log.gz 2023-11-24T06:14:17.039 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.101217.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101469.log 2023-11-24T06:14:17.040 INFO:teuthology.orchestra.run.smithi092.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.101217.log.gz 2023-11-24T06:14:17.040 INFO:teuthology.orchestra.run.smithi118.stderr: 91.2% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2023-11-24T06:14:17.041 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.88233.log: gzip -5 --verbose 92.1% -- -- replaced with /var/log/ceph/ceph-client.admin.88233.log.gz /var/log/ceph/ceph-client.admin.95228.log 2023-11-24T06:14:17.041 INFO:teuthology.orchestra.run.smithi092.stderr: 2023-11-24T06:14:17.042 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.101469.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.101469.log.gz 2023-11-24T06:14:17.042 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102545.log 2023-11-24T06:14:17.043 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.95228.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.95228.log.gz 2023-11-24T06:14:17.043 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104258.log 2023-11-24T06:14:17.044 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.102545.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.102545.log.gz 2023-11-24T06:14:17.045 INFO:teuthology.orchestra.run.smithi118.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2023-11-24T06:14:17.045 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105574.log 2023-11-24T06:14:17.046 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.104258.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.104258.log.gz 2023-11-24T06:14:17.046 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110011.log 2023-11-24T06:14:17.047 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.105574.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.105574.log.gz 2023-11-24T06:14:17.047 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104048.log 2023-11-24T06:14:17.048 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.110011.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.110011.log.gz 2023-11-24T06:14:17.048 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94492.log 2023-11-24T06:14:17.049 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.104048.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.104048.log.gz 2023-11-24T06:14:17.050 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111957.log 2023-11-24T06:14:17.051 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.94492.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.94492.log.gz 2023-11-24T06:14:17.051 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95565.log 2023-11-24T06:14:17.052 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.111957.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.111957.log.gz 2023-11-24T06:14:17.052 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95896.log 2023-11-24T06:14:17.053 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.95565.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.95565.log.gz 2023-11-24T06:14:17.053 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95902.log 2023-11-24T06:14:17.054 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.95896.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.95896.log.gz 2023-11-24T06:14:17.055 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98184.log 2023-11-24T06:14:17.056 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.95902.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.95902.log.gz 2023-11-24T06:14:17.056 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109891.log 2023-11-24T06:14:17.057 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.98184.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.98184.log.gz 2023-11-24T06:14:17.057 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98574.log 2023-11-24T06:14:17.058 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.109891.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.109891.log.gz 2023-11-24T06:14:17.058 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94544.log 2023-11-24T06:14:17.059 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.98574.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.98574.log.gz 2023-11-24T06:14:17.060 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110590.log 2023-11-24T06:14:17.061 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.94544.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.94544.log.gz 2023-11-24T06:14:17.061 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107767.log 2023-11-24T06:14:17.062 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.110590.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.110590.log.gz 2023-11-24T06:14:17.062 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111619.log 2023-11-24T06:14:17.063 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.107767.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.107767.log.gz 2023-11-24T06:14:17.063 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110053.log 2023-11-24T06:14:17.064 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.111619.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.111619.log.gz 2023-11-24T06:14:17.065 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105182.log 2023-11-24T06:14:17.065 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.110053.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104216.log 2023-11-24T06:14:17.066 INFO:teuthology.orchestra.run.smithi092.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.110053.log.gz 2023-11-24T06:14:17.067 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.105182.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.105182.log.gz 2023-11-24T06:14:17.067 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108275.log 2023-11-24T06:14:17.068 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.104216.log: gzip -5 88.9% --verbose -- replaced with /var/log/ceph/ceph-client.admin.104216.log.gz -- 2023-11-24T06:14:17.068 INFO:teuthology.orchestra.run.smithi092.stderr: /var/log/ceph/ceph-client.admin.98994.log 2023-11-24T06:14:17.069 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.108275.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.108275.log.gz 2023-11-24T06:14:17.069 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106653.log 2023-11-24T06:14:17.070 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.98994.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.98994.log.gz 2023-11-24T06:14:17.070 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105658.log 2023-11-24T06:14:17.071 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.106653.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.106653.log.gz 2023-11-24T06:14:17.072 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103601.log 2023-11-24T06:14:17.073 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.105658.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104342.log 2023-11-24T06:14:17.073 INFO:teuthology.orchestra.run.smithi092.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.105658.log.gz 2023-11-24T06:14:17.073 INFO:teuthology.orchestra.run.smithi118.stderr: 91.5% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2023-11-24T06:14:17.074 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.103601.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.103601.log.gz 2023-11-24T06:14:17.074 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101259.log 2023-11-24T06:14:17.075 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.104342.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94238.log 2023-11-24T06:14:17.075 INFO:teuthology.orchestra.run.smithi092.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.104342.log.gz 2023-11-24T06:14:17.076 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.101259.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.101259.log.gz 2023-11-24T06:14:17.076 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110422.log 2023-11-24T06:14:17.077 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.94238.log: gzip -5 --verbose -- 88.7% /var/log/ceph/ceph-client.admin.98142.log 2023-11-24T06:14:17.078 INFO:teuthology.orchestra.run.smithi092.stderr: -- replaced with /var/log/ceph/ceph-client.admin.94238.log.gz 2023-11-24T06:14:17.079 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.110422.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.110422.log.gz 2023-11-24T06:14:17.079 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95840.log 2023-11-24T06:14:17.080 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.98142.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95231.log 2023-11-24T06:14:17.080 INFO:teuthology.orchestra.run.smithi092.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.98142.log.gz 2023-11-24T06:14:17.081 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.95840.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.95840.log.gz 2023-11-24T06:14:17.081 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95863.log 2023-11-24T06:14:17.082 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.95231.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.95231.log.gz 2023-11-24T06:14:17.082 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106917.log 2023-11-24T06:14:17.083 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.95863.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.95863.log.gz 2023-11-24T06:14:17.083 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101175.log 2023-11-24T06:14:17.084 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.106917.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102461.log 2023-11-24T06:14:17.085 INFO:teuthology.orchestra.run.smithi092.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.106917.log.gz 2023-11-24T06:14:17.085 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.101175.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.101175.log.gz 2023-11-24T06:14:17.086 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2023-11-24T06:14:17.087 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.102461.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106434.log 2023-11-24T06:14:17.087 INFO:teuthology.orchestra.run.smithi092.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.102461.log.gz 2023-11-24T06:14:17.088 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99215.log 2023-11-24T06:14:17.090 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.106434.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.106434.log.gz 2023-11-24T06:14:17.090 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95893.log 2023-11-24T06:14:17.092 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.99215.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104546.log 2023-11-24T06:14:17.092 INFO:teuthology.orchestra.run.smithi092.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.99215.log.gz 2023-11-24T06:14:17.093 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.95893.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106839.log 2023-11-24T06:14:17.093 INFO:teuthology.orchestra.run.smithi092.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.95893.log.gz 2023-11-24T06:14:17.094 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.104546.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102231.log 2023-11-24T06:14:17.095 INFO:teuthology.orchestra.run.smithi092.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.104546.log.gz 2023-11-24T06:14:17.096 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.106839.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97990.log 2023-11-24T06:14:17.096 INFO:teuthology.orchestra.run.smithi092.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.106839.log.gz 2023-11-24T06:14:17.098 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.102231.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109462.log 2023-11-24T06:14:17.098 INFO:teuthology.orchestra.run.smithi092.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.102231.log.gz 2023-11-24T06:14:17.099 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.97990.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100073.log 2023-11-24T06:14:17.100 INFO:teuthology.orchestra.run.smithi092.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.97990.log.gz 2023-11-24T06:14:17.100 INFO:teuthology.orchestra.run.smithi118.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2023-11-24T06:14:17.101 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.109462.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104883.log 2023-11-24T06:14:17.101 INFO:teuthology.orchestra.run.smithi092.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.109462.log.gz 2023-11-24T06:14:17.102 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.100073.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.100073.log.gz 2023-11-24T06:14:17.103 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108402.log 2023-11-24T06:14:17.104 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.104883.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.104883.log.gz 2023-11-24T06:14:17.104 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95567.log 2023-11-24T06:14:17.105 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.108402.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102273.log 2023-11-24T06:14:17.105 INFO:teuthology.orchestra.run.smithi092.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.108402.log.gz 2023-11-24T06:14:17.107 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.95567.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.95567.log.gz 2023-11-24T06:14:17.107 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98406.log 2023-11-24T06:14:17.109 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.102273.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95903.log 2023-11-24T06:14:17.109 INFO:teuthology.orchestra.run.smithi092.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.102273.log.gz 2023-11-24T06:14:17.110 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.98406.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103805.log 2023-11-24T06:14:17.110 INFO:teuthology.orchestra.run.smithi092.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.98406.log.gz 2023-11-24T06:14:17.111 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.95903.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99561.log 2023-11-24T06:14:17.112 INFO:teuthology.orchestra.run.smithi092.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.95903.log.gz 2023-11-24T06:14:17.113 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.103805.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.103805.log.gz 2023-11-24T06:14:17.113 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102095.log 2023-11-24T06:14:17.115 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.99561.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94630.log 2023-11-24T06:14:17.115 INFO:teuthology.orchestra.run.smithi092.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.99561.log.gz 2023-11-24T06:14:17.116 INFO:teuthology.orchestra.run.smithi118.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2023-11-24T06:14:17.117 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.102095.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100503.log 2023-11-24T06:14:17.117 INFO:teuthology.orchestra.run.smithi092.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.102095.log.gz 2023-11-24T06:14:17.118 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.94630.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.109323.log 2023-11-24T06:14:17.118 INFO:teuthology.orchestra.run.smithi092.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.94630.log.gz 2023-11-24T06:14:17.118 INFO:teuthology.orchestra.run.smithi118.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2023-11-24T06:14:17.119 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.100503.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.100503.log.gz 2023-11-24T06:14:17.119 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111561.log 2023-11-24T06:14:17.121 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.0.109323.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110095.log 2023-11-24T06:14:17.121 INFO:teuthology.orchestra.run.smithi118.stderr: 2023-11-24T06:14:17.121 INFO:teuthology.orchestra.run.smithi118.stderr:real 0m0.321s 2023-11-24T06:14:17.122 INFO:teuthology.orchestra.run.smithi118.stderr:user 0m1.298s 2023-11-24T06:14:17.122 INFO:teuthology.orchestra.run.smithi118.stderr:sys 0m0.244s 2023-11-24T06:14:17.123 INFO:teuthology.orchestra.run.smithi092.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.0.109323.log.gz 2023-11-24T06:14:17.123 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.111561.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99089.log 2023-11-24T06:14:17.123 INFO:teuthology.orchestra.run.smithi092.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.111561.log.gz 2023-11-24T06:14:17.124 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.110095.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95552.log 2023-11-24T06:14:17.124 INFO:teuthology.orchestra.run.smithi092.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.110095.log.gz 2023-11-24T06:14:17.125 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.99089.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.99089.log.gz 2023-11-24T06:14:17.126 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104174.log 2023-11-24T06:14:17.127 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.95552.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.95552.log.gz 2023-11-24T06:14:17.127 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96100.log 2023-11-24T06:14:17.128 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.104174.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110464.log 2023-11-24T06:14:17.129 INFO:teuthology.orchestra.run.smithi092.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.104174.log.gz 2023-11-24T06:14:17.130 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.96100.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.96100.log.gz 2023-11-24T06:14:17.130 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110548.log 2023-11-24T06:14:17.132 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.110464.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95232.log 2023-11-24T06:14:17.132 INFO:teuthology.orchestra.run.smithi092.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.110464.log.gz 2023-11-24T06:14:17.133 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.110548.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101007.log 2023-11-24T06:14:17.133 INFO:teuthology.orchestra.run.smithi092.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.110548.log.gz 2023-11-24T06:14:17.134 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.95232.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95219.log 2023-11-24T06:14:17.135 INFO:teuthology.orchestra.run.smithi092.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.95232.log.gz 2023-11-24T06:14:17.136 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.101007.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.101007.log.gz 2023-11-24T06:14:17.136 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95008.log 2023-11-24T06:14:17.138 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.95219.log: 87.4%gzip -- replaced with /var/log/ceph/ceph-client.admin.95219.log.gz -5 2023-11-24T06:14:17.138 INFO:teuthology.orchestra.run.smithi092.stderr: --verbose -- /var/log/ceph/ceph-client.admin.99483.log 2023-11-24T06:14:17.140 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.95008.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99173.log 2023-11-24T06:14:17.140 INFO:teuthology.orchestra.run.smithi092.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.95008.log.gz 2023-11-24T06:14:17.141 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.99483.log: gzip -5 88.7% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.99483.log.gz 2023-11-24T06:14:17.141 INFO:teuthology.orchestra.run.smithi092.stderr: /var/log/ceph/ceph-client.admin.107160.log 2023-11-24T06:14:17.142 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.99173.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95904.log 2023-11-24T06:14:17.142 INFO:teuthology.orchestra.run.smithi092.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.99173.log.gz 2023-11-24T06:14:17.144 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.107160.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.107160.log.gz 2023-11-24T06:14:17.144 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110674.log 2023-11-24T06:14:17.145 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.95904.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.95904.log.gz 2023-11-24T06:14:17.146 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96142.log 2023-11-24T06:14:17.147 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.110674.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107202.log 2023-11-24T06:14:17.147 INFO:teuthology.orchestra.run.smithi092.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.110674.log.gz 2023-11-24T06:14:17.148 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.96142.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.96142.log.gz 2023-11-24T06:14:17.148 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100797.log 2023-11-24T06:14:17.150 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.107202.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.107202.log.gz 2023-11-24T06:14:17.150 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2023-11-24T06:14:17.151 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.100797.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96436.log 2023-11-24T06:14:17.151 INFO:teuthology.orchestra.run.smithi092.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.100797.log.gz 2023-11-24T06:14:17.153 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95568.log 2023-11-24T06:14:17.154 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.96436.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101925.log 2023-11-24T06:14:17.155 INFO:teuthology.orchestra.run.smithi092.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.96436.log.gz 2023-11-24T06:14:17.156 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.95568.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99315.log 2023-11-24T06:14:17.156 INFO:teuthology.orchestra.run.smithi092.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.95568.log.gz 2023-11-24T06:14:17.157 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.101925.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99257.log 2023-11-24T06:14:17.157 INFO:teuthology.orchestra.run.smithi092.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.101925.log.gz 2023-11-24T06:14:17.159 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.99315.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99771.log 2023-11-24T06:14:17.159 INFO:teuthology.orchestra.run.smithi092.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.99315.log.gz 2023-11-24T06:14:17.160 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.99257.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111789.log 2023-11-24T06:14:17.160 INFO:teuthology.orchestra.run.smithi092.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.99257.log.gz 2023-11-24T06:14:17.161 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.99771.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96520.log 2023-11-24T06:14:17.162 INFO:teuthology.orchestra.run.smithi092.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.99771.log.gz 2023-11-24T06:14:17.163 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.111789.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111435.log 2023-11-24T06:14:17.163 INFO:teuthology.orchestra.run.smithi092.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.111789.log.gz 2023-11-24T06:14:17.164 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.96520.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111309.log 2023-11-24T06:14:17.165 INFO:teuthology.orchestra.run.smithi092.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.96520.log.gz 2023-11-24T06:14:17.166 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.111435.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103333.log 2023-11-24T06:14:17.166 INFO:teuthology.orchestra.run.smithi092.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.111435.log.gz 2023-11-24T06:14:17.167 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.111309.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105396.log 2023-11-24T06:14:17.167 INFO:teuthology.orchestra.run.smithi092.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.111309.log.gz 2023-11-24T06:14:17.169 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.103333.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108360.log 2023-11-24T06:14:17.169 INFO:teuthology.orchestra.run.smithi092.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.103333.log.gz 2023-11-24T06:14:17.171 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.105396.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94588.log 2023-11-24T06:14:17.171 INFO:teuthology.orchestra.run.smithi092.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.105396.log.gz 2023-11-24T06:14:17.172 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.108360.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94714.log 2023-11-24T06:14:17.172 INFO:teuthology.orchestra.run.smithi092.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.108360.log.gz 2023-11-24T06:14:17.173 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.94588.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106392.log 2023-11-24T06:14:17.173 INFO:teuthology.orchestra.run.smithi092.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.94588.log.gz 2023-11-24T06:14:17.175 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.94714.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111141.log 2023-11-24T06:14:17.175 INFO:teuthology.orchestra.run.smithi092.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.94714.log.gz 2023-11-24T06:14:17.176 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.106392.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108490.log 2023-11-24T06:14:17.176 INFO:teuthology.orchestra.run.smithi092.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.106392.log.gz 2023-11-24T06:14:17.178 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.111141.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98742.log 2023-11-24T06:14:17.178 INFO:teuthology.orchestra.run.smithi092.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.111141.log.gz 2023-11-24T06:14:17.179 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.108490.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.108490.log.gzgzip 2023-11-24T06:14:17.180 INFO:teuthology.orchestra.run.smithi092.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.107370.log 2023-11-24T06:14:17.181 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.98742.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104799.log 2023-11-24T06:14:17.181 INFO:teuthology.orchestra.run.smithi092.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.98742.log.gz 2023-11-24T06:14:17.183 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.107370.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104384.log 2023-11-24T06:14:17.183 INFO:teuthology.orchestra.run.smithi092.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.107370.log.gz 2023-11-24T06:14:17.184 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.104799.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2023-11-24T06:14:17.184 INFO:teuthology.orchestra.run.smithi092.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.104799.log.gz 2023-11-24T06:14:17.185 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.104384.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106695.log 2023-11-24T06:14:17.185 INFO:teuthology.orchestra.run.smithi092.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.104384.log.gz 2023-11-24T06:14:17.187 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110179.log 2023-11-24T06:14:17.188 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.106695.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99813.log 2023-11-24T06:14:17.189 INFO:teuthology.orchestra.run.smithi092.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.106695.log.gz 2023-11-24T06:14:17.190 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.110179.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108878.log 2023-11-24T06:14:17.190 INFO:teuthology.orchestra.run.smithi092.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.110179.log.gz 2023-11-24T06:14:17.191 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.99813.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107001.log 2023-11-24T06:14:17.192 INFO:teuthology.orchestra.run.smithi092.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.99813.log.gz 2023-11-24T06:14:17.193 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.108878.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96730.log 2023-11-24T06:14:17.193 INFO:teuthology.orchestra.run.smithi092.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.108878.log.gz 2023-11-24T06:14:17.193 INFO:teuthology.orchestra.run.smithi092.stderr: 90.6% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2023-11-24T06:14:17.195 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.107001.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96646.log 2023-11-24T06:14:17.195 INFO:teuthology.orchestra.run.smithi092.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.107001.log.gz 2023-11-24T06:14:17.196 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.96730.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.96730.log.gz 2023-11-24T06:14:17.197 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100257.log 2023-11-24T06:14:17.198 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.96646.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96310.log 2023-11-24T06:14:17.198 INFO:teuthology.orchestra.run.smithi092.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.96646.log.gz 2023-11-24T06:14:17.200 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.100257.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110734.log 2023-11-24T06:14:17.200 INFO:teuthology.orchestra.run.smithi092.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.100257.log.gz 2023-11-24T06:14:17.201 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.96310.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111705.log 2023-11-24T06:14:17.201 INFO:teuthology.orchestra.run.smithi092.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.96310.log.gz 2023-11-24T06:14:17.203 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.110734.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109686.log 2023-11-24T06:14:17.203 INFO:teuthology.orchestra.run.smithi092.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.110734.log.gz 2023-11-24T06:14:17.204 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.111705.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107454.log 2023-11-24T06:14:17.204 INFO:teuthology.orchestra.run.smithi092.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.111705.log.gz 2023-11-24T06:14:17.206 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.109686.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102587.log 2023-11-24T06:14:17.206 INFO:teuthology.orchestra.run.smithi092.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.109686.log.gz 2023-11-24T06:14:17.207 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.107454.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94924.log 2023-11-24T06:14:17.207 INFO:teuthology.orchestra.run.smithi092.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.107454.log.gz 2023-11-24T06:14:17.209 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.102587.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107412.log 2023-11-24T06:14:17.209 INFO:teuthology.orchestra.run.smithi092.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.102587.log.gz 2023-11-24T06:14:17.210 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.94924.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99687.log 2023-11-24T06:14:17.210 INFO:teuthology.orchestra.run.smithi092.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.94924.log.gz 2023-11-24T06:14:17.212 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.107412.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100755.log 2023-11-24T06:14:17.212 INFO:teuthology.orchestra.run.smithi092.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.107412.log.gz 2023-11-24T06:14:17.213 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.99687.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2023-11-24T06:14:17.213 INFO:teuthology.orchestra.run.smithi092.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.99687.log.gz 2023-11-24T06:14:17.214 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.100755.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96394.log 2023-11-24T06:14:17.215 INFO:teuthology.orchestra.run.smithi092.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.100755.log.gz 2023-11-24T06:14:17.216 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108446.log 2023-11-24T06:14:17.217 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.96394.log: 85.5% -- replaced with /var/log/ceph/ceph.log.gz 2023-11-24T06:14:17.218 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100199.log 2023-11-24T06:14:17.218 INFO:teuthology.orchestra.run.smithi092.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.96394.log.gz 2023-11-24T06:14:17.219 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.108446.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101049.log 2023-11-24T06:14:17.219 INFO:teuthology.orchestra.run.smithi092.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.108446.log.gz 2023-11-24T06:14:17.220 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.100199.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101343.log 2023-11-24T06:14:17.220 INFO:teuthology.orchestra.run.smithi092.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.100199.log.gz 2023-11-24T06:14:17.222 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.101049.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2023-11-24T06:14:17.222 INFO:teuthology.orchestra.run.smithi092.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.101049.log.gz 2023-11-24T06:14:17.223 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.101343.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111997.log 2023-11-24T06:14:17.224 INFO:teuthology.orchestra.run.smithi092.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.101343.log.gz 2023-11-24T06:14:17.224 INFO:teuthology.orchestra.run.smithi092.stderr: 91.5% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2023-11-24T06:14:17.225 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106797.log 2023-11-24T06:14:17.226 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.111997.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103685.log 2023-11-24T06:14:17.227 INFO:teuthology.orchestra.run.smithi092.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.111997.log.gz 2023-11-24T06:14:17.228 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.106797.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.106797.log.gz 2023-11-24T06:14:17.228 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2023-11-24T06:14:17.229 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.103685.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106737.log 2023-11-24T06:14:17.229 INFO:teuthology.orchestra.run.smithi092.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.103685.log.gz 2023-11-24T06:14:17.231 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109504.log 2023-11-24T06:14:17.232 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.106737.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108152.log 2023-11-24T06:14:17.232 INFO:teuthology.orchestra.run.smithi092.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.106737.log.gz 2023-11-24T06:14:17.233 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.109504.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105488.log 2023-11-24T06:14:17.234 INFO:teuthology.orchestra.run.smithi092.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.109504.log.gz 2023-11-24T06:14:17.235 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.108152.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.108152.log.gz 2023-11-24T06:14:17.235 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107893.log 2023-11-24T06:14:17.236 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.105488.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.105488.log.gz 2023-11-24T06:14:17.237 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104300.log 2023-11-24T06:14:17.239 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.107893.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112039.log 2023-11-24T06:14:17.239 INFO:teuthology.orchestra.run.smithi092.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.107893.log.gz 2023-11-24T06:14:17.240 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.104300.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108920.log 2023-11-24T06:14:17.240 INFO:teuthology.orchestra.run.smithi092.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.104300.log.gz 2023-11-24T06:14:17.242 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.112039.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94357.log 2023-11-24T06:14:17.242 INFO:teuthology.orchestra.run.smithi092.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.112039.log.gz 2023-11-24T06:14:17.243 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.108920.log: 89.1%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100383.log 2023-11-24T06:14:17.243 INFO:teuthology.orchestra.run.smithi092.stderr: -- replaced with /var/log/ceph/ceph-client.admin.108920.log.gz 2023-11-24T06:14:17.244 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.94357.log: gzip -5 --verbose -- 88.4% /var/log/ceph/ceph.tmp-client.admin.82162.log 2023-11-24T06:14:17.245 INFO:teuthology.orchestra.run.smithi092.stderr: -- replaced with /var/log/ceph/ceph-client.admin.94357.log.gz 2023-11-24T06:14:17.246 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.100383.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105308.log 2023-11-24T06:14:17.246 INFO:teuthology.orchestra.run.smithi092.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.100383.log.gz 2023-11-24T06:14:17.247 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph.tmp-client.admin.82162.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.82162.log.gz 2023-11-24T06:14:17.247 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107977.log 2023-11-24T06:14:17.249 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.105308.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104715.log 2023-11-24T06:14:17.249 INFO:teuthology.orchestra.run.smithi092.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.105308.log.gz 2023-11-24T06:14:17.251 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.107977.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110137.log 2023-11-24T06:14:17.251 INFO:teuthology.orchestra.run.smithi092.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.107977.log.gz 2023-11-24T06:14:17.252 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.104715.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103195.log 2023-11-24T06:14:17.252 INFO:teuthology.orchestra.run.smithi092.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.104715.log.gz 2023-11-24T06:14:17.253 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.110137.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.110137.log.gz 2023-11-24T06:14:17.254 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100341.log 2023-11-24T06:14:17.255 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.103195.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105058.log 2023-11-24T06:14:17.255 INFO:teuthology.orchestra.run.smithi092.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.103195.log.gz 2023-11-24T06:14:17.257 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.100341.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108072.log 2023-11-24T06:14:17.257 INFO:teuthology.orchestra.run.smithi092.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.100341.log.gz 2023-11-24T06:14:17.258 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.105058.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110296.log 2023-11-24T06:14:17.259 INFO:teuthology.orchestra.run.smithi092.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.105058.log.gz 2023-11-24T06:14:17.260 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.108072.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108668.log 2023-11-24T06:14:17.260 INFO:teuthology.orchestra.run.smithi092.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.108072.log.gz 2023-11-24T06:14:17.261 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.110296.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103279.log 2023-11-24T06:14:17.262 INFO:teuthology.orchestra.run.smithi092.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.110296.log.gz 2023-11-24T06:14:17.263 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.108668.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99645.log 2023-11-24T06:14:17.263 INFO:teuthology.orchestra.run.smithi092.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.108668.log.gz 2023-11-24T06:14:17.264 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.103279.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99729.log 2023-11-24T06:14:17.264 INFO:teuthology.orchestra.run.smithi092.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.103279.log.gz 2023-11-24T06:14:17.266 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.99645.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.2.97844.log 2023-11-24T06:14:17.266 INFO:teuthology.orchestra.run.smithi092.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.99645.log.gz 2023-11-24T06:14:17.267 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.99729.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101672.log 2023-11-24T06:14:17.267 INFO:teuthology.orchestra.run.smithi092.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.99729.log.gz 2023-11-24T06:14:17.269 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.2.97844.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.97264.log 2023-11-24T06:14:17.270 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.101672.log: 91.7% -- replaced with /var/log/ceph/ceph-client.2.97844.log.gz 2023-11-24T06:14:17.270 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94300.log 2023-11-24T06:14:17.271 INFO:teuthology.orchestra.run.smithi092.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.101672.log.gz 2023-11-24T06:14:17.272 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.0.97264.log: 90.6%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109028.log 2023-11-24T06:14:17.272 INFO:teuthology.orchestra.run.smithi092.stderr: -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2023-11-24T06:14:17.273 INFO:teuthology.orchestra.run.smithi092.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.0.97264.log.gz 2023-11-24T06:14:17.273 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.94300.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99915.log 2023-11-24T06:14:17.273 INFO:teuthology.orchestra.run.smithi092.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.94300.log.gz 2023-11-24T06:14:17.275 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.109028.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94408.log 2023-11-24T06:14:17.275 INFO:teuthology.orchestra.run.smithi092.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.109028.log.gz 2023-11-24T06:14:17.276 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.99915.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101133.log 2023-11-24T06:14:17.276 INFO:teuthology.orchestra.run.smithi092.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.99915.log.gz 2023-11-24T06:14:17.277 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.94408.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.94408.log.gz 2023-11-24T06:14:17.278 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100923.log 2023-11-24T06:14:17.279 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.101133.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109630.log 2023-11-24T06:14:17.279 INFO:teuthology.orchestra.run.smithi092.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.101133.log.gz 2023-11-24T06:14:17.280 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.100923.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.100923.log.gz 2023-11-24T06:14:17.280 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98910.log 2023-11-24T06:14:17.281 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.109630.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101799.log 2023-11-24T06:14:17.282 INFO:teuthology.orchestra.run.smithi092.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.109630.log.gz 2023-11-24T06:14:17.282 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.98910.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.98910.log.gz 2023-11-24T06:14:17.283 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108752.log 2023-11-24T06:14:17.284 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.101799.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104841.log 2023-11-24T06:14:17.284 INFO:teuthology.orchestra.run.smithi092.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.101799.log.gz 2023-11-24T06:14:17.286 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.108752.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.108752.log.gz 2023-11-24T06:14:17.286 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107580.log 2023-11-24T06:14:17.287 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.104841.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102629.log 2023-11-24T06:14:17.287 INFO:teuthology.orchestra.run.smithi092.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.104841.log.gz 2023-11-24T06:14:17.288 INFO:teuthology.orchestra.run.smithi092.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2023-11-24T06:14:17.288 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.107580.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.107580.log.gz 2023-11-24T06:14:17.289 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2023-11-24T06:14:17.290 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.102629.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88697.log 2023-11-24T06:14:17.290 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-mds.a.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.102629.log.gz 2023-11-24T06:14:17.291 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107085.log 2023-11-24T06:14:17.292 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.88697.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.88697.log.gz 2023-11-24T06:14:17.293 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107638.log 2023-11-24T06:14:17.294 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.107085.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96562.log 2023-11-24T06:14:17.294 INFO:teuthology.orchestra.run.smithi092.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.107085.log.gz 2023-11-24T06:14:17.295 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.107638.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.107638.log.gz 2023-11-24T06:14:17.295 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109588.log 2023-11-24T06:14:17.297 INFO:teuthology.orchestra.run.smithi092.stderr:gzip/var/log/ceph/ceph-client.admin.96562.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.98448.log 2023-11-24T06:14:17.297 INFO:teuthology.orchestra.run.smithi092.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.96562.log.gz 2023-11-24T06:14:17.298 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.109588.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.109588.log.gz 2023-11-24T06:14:17.298 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105224.log 2023-11-24T06:14:17.299 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.98448.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.98448.log.gz 2023-11-24T06:14:17.300 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.105224.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.105224.log.gz 2023-11-24T06:14:17.332 INFO:teuthology.orchestra.run.smithi092.stderr: 91.4% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2023-11-24T06:14:17.367 INFO:teuthology.orchestra.run.smithi092.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2023-11-24T06:14:17.404 INFO:teuthology.orchestra.run.smithi092.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2023-11-24T06:14:17.417 INFO:teuthology.orchestra.run.smithi092.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2023-11-24T06:14:17.420 INFO:teuthology.orchestra.run.smithi092.stderr: 2023-11-24T06:14:17.421 INFO:teuthology.orchestra.run.smithi092.stderr:real 0m0.628s 2023-11-24T06:14:17.421 INFO:teuthology.orchestra.run.smithi092.stderr:user 0m1.511s 2023-11-24T06:14:17.421 INFO:teuthology.orchestra.run.smithi092.stderr:sys 0m1.088s 2023-11-24T06:14:17.422 INFO:tasks.ceph:Archiving logs... 2023-11-24T06:14:17.422 DEBUG:teuthology.misc:Transferring archived files from smithi092:/var/log/ceph to /home/teuthworker/archive/rishabh-2023-11-24_05:41:33-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7465546/remote/smithi092/log 2023-11-24T06:14:17.423 DEBUG:teuthology.orchestra.run.smithi092:> sudo tar c -f - -C /var/log/ceph -- . 2023-11-24T06:14:17.559 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c has been restored 2023-11-24T06:14:17.951 DEBUG:teuthology.misc:Transferring archived files from smithi118:/var/log/ceph to /home/teuthworker/archive/rishabh-2023-11-24_05:41:33-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7465546/remote/smithi118/log 2023-11-24T06:14:17.952 DEBUG:teuthology.orchestra.run.smithi118:> sudo tar c -f - -C /var/log/ceph -- . 2023-11-24T06:14:18.216 DEBUG:teuthology.run_tasks:Unwinding manager install 2023-11-24T06:14:18.241 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2023-11-24T06:14:18.241 DEBUG:teuthology.orchestra.run.smithi092:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-11-24T06:14:18.251 DEBUG:teuthology.orchestra.run.smithi118:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-11-24T06:14:18.405 DEBUG:teuthology.run_tasks:Unwinding manager clock 2023-11-24T06:14:18.414 INFO:teuthology.task.clock:Checking final clock skew... 2023-11-24T06:14:18.415 DEBUG:teuthology.orchestra.run.smithi092:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-11-24T06:14:18.424 DEBUG:teuthology.orchestra.run.smithi118:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-11-24T06:14:18.474 INFO:teuthology.orchestra.run.smithi092.stderr:bash: line 1: ntpq: command not found 2023-11-24T06:14:18.481 INFO:teuthology.orchestra.run.smithi118.stderr:bash: line 1: ntpq: command not found 2023-11-24T06:14:18.483 INFO:teuthology.orchestra.run.smithi092.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-11-24T06:14:18.484 INFO:teuthology.orchestra.run.smithi092.stdout:=============================================================================== 2023-11-24T06:14:18.484 INFO:teuthology.orchestra.run.smithi092.stdout:^+ hv01.front.sepia.ceph.com 3 6 377 54 +2151us[+2151us] +/- 70ms 2023-11-24T06:14:18.484 INFO:teuthology.orchestra.run.smithi092.stdout:^* hv02.front.sepia.ceph.com 3 6 377 54 -1679us[-1689us] +/- 58ms 2023-11-24T06:14:18.484 INFO:teuthology.orchestra.run.smithi092.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 52 -124us[ -124us] +/- 90ms 2023-11-24T06:14:18.484 INFO:teuthology.orchestra.run.smithi092.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2023-11-24T06:14:18.491 INFO:teuthology.orchestra.run.smithi118.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-11-24T06:14:18.492 INFO:teuthology.orchestra.run.smithi118.stdout:=============================================================================== 2023-11-24T06:14:18.492 INFO:teuthology.orchestra.run.smithi118.stdout:^+ hv01.front.sepia.ceph.com 3 6 377 54 +2177us[+2171us] +/- 70ms 2023-11-24T06:14:18.492 INFO:teuthology.orchestra.run.smithi118.stdout:^* hv02.front.sepia.ceph.com 3 6 377 54 -1657us[-1664us] +/- 58ms 2023-11-24T06:14:18.492 INFO:teuthology.orchestra.run.smithi118.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 52 -107us[ -107us] +/- 90ms 2023-11-24T06:14:18.492 INFO:teuthology.orchestra.run.smithi118.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2023-11-24T06:14:18.493 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2023-11-24T06:14:18.503 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2023-11-24T06:14:18.504 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2023-11-24T06:14:18.573 DEBUG:teuthology.orchestra.run.smithi092:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-11-24T06:14:18.843 DEBUG:teuthology.orchestra.run.smithi118:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-11-24T06:14:19.106 DEBUG:teuthology.orchestra.run.smithi092:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-11-24T06:14:19.212 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-24T06:14:19.212 DEBUG:teuthology.orchestra.run.smithi118:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-11-24T06:14:19.322 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-24T06:14:19.323 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2023-11-24T06:14:19.332 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2023-11-24T06:14:19.341 INFO:teuthology.task.internal:Duration was 1176.709115 seconds 2023-11-24T06:14:19.342 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2023-11-24T06:14:19.351 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2023-11-24T06:14:19.351 DEBUG:teuthology.orchestra.run.smithi092:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-11-24T06:14:19.360 DEBUG:teuthology.orchestra.run.smithi118:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-11-24T06:14:19.517 INFO:teuthology.orchestra.run.smithi092.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-11-24T06:14:19.529 INFO:teuthology.orchestra.run.smithi118.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-11-24T06:14:19.880 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2023-11-24T06:14:19.880 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi092.front.sepia.ceph.com 2023-11-24T06:14:19.881 DEBUG:teuthology.orchestra.run.smithi092:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-11-24T06:14:19.946 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi118.front.sepia.ceph.com 2023-11-24T06:14:19.947 DEBUG:teuthology.orchestra.run.smithi118:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-11-24T06:14:20.026 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2023-11-24T06:14:20.026 DEBUG:teuthology.orchestra.run.smithi092:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-11-24T06:14:20.034 DEBUG:teuthology.orchestra.run.smithi118:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-11-24T06:14:20.133 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2023-11-24T06:14:20.134 DEBUG:teuthology.orchestra.run.smithi092:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-11-24T06:14:20.139 DEBUG:teuthology.orchestra.run.smithi118:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-11-24T06:14:20.325 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2023-11-24T06:14:20.335 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2023-11-24T06:14:20.335 DEBUG:teuthology.orchestra.run.smithi092:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-11-24T06:14:20.425 DEBUG:teuthology.orchestra.run.smithi118:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-11-24T06:14:20.527 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2023-11-24T06:14:20.537 DEBUG:teuthology.orchestra.run.smithi092:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-11-24T06:14:20.546 DEBUG:teuthology.orchestra.run.smithi118:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-11-24T06:14:20.651 INFO:teuthology.orchestra.run.smithi092.stdout:kernel.core_pattern = core 2023-11-24T06:14:20.657 INFO:teuthology.orchestra.run.smithi118.stdout:kernel.core_pattern = core 2023-11-24T06:14:20.711 DEBUG:teuthology.orchestra.run.smithi092:> test -e /home/ubuntu/cephtest/archive/coredump 2023-11-24T06:14:20.761 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-24T06:14:20.761 DEBUG:teuthology.orchestra.run.smithi118:> test -e /home/ubuntu/cephtest/archive/coredump 2023-11-24T06:14:20.817 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-24T06:14:21.068 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2023-11-24T06:14:21.083 INFO:teuthology.task.internal:Transferring archived files... 2023-11-24T06:14:21.084 DEBUG:teuthology.misc:Transferring archived files from smithi092:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2023-11-24_05:41:33-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7465546/remote/smithi092 2023-11-24T06:14:21.084 DEBUG:teuthology.orchestra.run.smithi092:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-11-24T06:14:21.194 DEBUG:teuthology.misc:Transferring archived files from smithi118:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2023-11-24_05:41:33-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7465546/remote/smithi118 2023-11-24T06:14:21.194 DEBUG:teuthology.orchestra.run.smithi118:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-11-24T06:14:21.305 INFO:teuthology.task.internal:Removing archive directory... 2023-11-24T06:14:21.306 DEBUG:teuthology.orchestra.run.smithi092:> rm -rf -- /home/ubuntu/cephtest/archive 2023-11-24T06:14:21.315 DEBUG:teuthology.orchestra.run.smithi118:> rm -rf -- /home/ubuntu/cephtest/archive 2023-11-24T06:14:21.370 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2023-11-24T06:14:21.382 INFO:teuthology.task.internal:Not uploading archives. 2023-11-24T06:14:21.382 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2023-11-24T06:14:21.392 INFO:teuthology.task.internal:Tidying up after the test... 2023-11-24T06:14:21.392 DEBUG:teuthology.orchestra.run.smithi092:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-11-24T06:14:21.401 DEBUG:teuthology.orchestra.run.smithi118:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-11-24T06:14:21.452 INFO:teuthology.orchestra.run.smithi092.stdout: 524301 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Nov 24 06:14 /home/ubuntu/cephtest 2023-11-24T06:14:21.457 INFO:teuthology.orchestra.run.smithi118.stdout: 393229 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Nov 24 06:14 /home/ubuntu/cephtest 2023-11-24T06:14:21.461 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2023-11-24T06:14:21.471 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2023-11-24T06:14:21.496 INFO:teuthology.nuke:Checking targets against current locks 2023-11-24T06:14:21.520 DEBUG:teuthology.nuke:shortname: smithi092 2023-11-24T06:14:21.521 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-11-24T06:14:21.543 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi092.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-11-24_05:41:33-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7465546', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-24 05:45:04.588550', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBM/6AgknJyX69rVjCUaIgHxhqMmdlXcpfMRywOSR//a3BdC53SvAjaxmI9LSZboTEgD8EzutTJWZfDUDZG21qko='} 2023-11-24T06:14:21.546 DEBUG:teuthology.nuke:shortname: smithi118 2023-11-24T06:14:21.547 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-11-24T06:14:21.572 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi118.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-11-24_05:41:33-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7465546', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-24 05:45:04.590485', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDtNpRTYzJZfvzdUULaqo1CEoH8cnZH6jMFDwICFAvv6nLJJyaEH9q71cRsl+4OgPQO1/cgIH3Ch5W2Ttk4w7N8='} 2023-11-24T06:14:21.600 INFO:teuthology.orchestra.console.smithi092:Power off 2023-11-24T06:14:21.600 DEBUG:teuthology.orchestra.console.smithi092:pexpect command: ipmitool -H smithi092.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-11-24T06:14:21.618 INFO:teuthology.orchestra.console.smithi118:Power off 2023-11-24T06:14:21.619 DEBUG:teuthology.orchestra.console.smithi118:pexpect command: ipmitool -H smithi118.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-11-24T06:14:21.629 DEBUG:teuthology.orchestra.console.smithi092:power off output: Chassis Power Control: Down/Off 2023-11-24T06:14:21.630 DEBUG:teuthology.orchestra.console.smithi092:pexpect command: ipmitool -H smithi092.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-24T06:14:21.641 DEBUG:teuthology.orchestra.console.smithi118:power off output: Chassis Power Control: Down/Off 2023-11-24T06:14:21.641 DEBUG:teuthology.orchestra.console.smithi118:pexpect command: ipmitool -H smithi118.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-24T06:14:21.651 DEBUG:teuthology.orchestra.console.smithi092:check power output: Chassis Power is on 2023-11-24T06:14:21.662 DEBUG:teuthology.orchestra.console.smithi118:check power output: Chassis Power is on 2023-11-24T06:14:25.653 DEBUG:teuthology.orchestra.console.smithi092:pexpect command: ipmitool -H smithi092.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-24T06:14:25.663 DEBUG:teuthology.orchestra.console.smithi118:pexpect command: ipmitool -H smithi118.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-24T06:14:25.764 DEBUG:teuthology.orchestra.console.smithi092:check power output: Chassis Power is on 2023-11-24T06:14:25.773 DEBUG:teuthology.orchestra.console.smithi118:check power output: Chassis Power is on 2023-11-24T06:14:29.766 DEBUG:teuthology.orchestra.console.smithi092:pexpect command: ipmitool -H smithi092.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-24T06:14:29.774 DEBUG:teuthology.orchestra.console.smithi118:pexpect command: ipmitool -H smithi118.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-24T06:14:29.877 DEBUG:teuthology.orchestra.console.smithi092:check power output: Chassis Power is off 2023-11-24T06:14:29.883 DEBUG:teuthology.orchestra.console.smithi118:check power output: Chassis Power is off 2023-11-24T06:14:29.978 INFO:teuthology.orchestra.console.smithi092:Power off completed 2023-11-24T06:14:29.984 INFO:teuthology.orchestra.console.smithi118:Power off completed 2023-11-24T06:14:30.089 INFO:teuthology.run:Summary data: description: fs:volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/basic}} duration: 1176.7091152668 failure_reason: 'Test failure: test_volume_info_pending_subvol_deletions (tasks.cephfs.test_volumes.TestVolumes)' flavor: default mon.a-kernel-sha1: 8247bbb5b470f7e2a1f3eefcc16e24cf1338f606 mon.b-kernel-sha1: 8247bbb5b470f7e2a1f3eefcc16e24cf1338f606 owner: scheduled_rishabh@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=79a1606a3ac048d792f56a773cfae9f0 status: fail success: false 2023-11-24T06:14:30.089 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-24T06:14:30.167 INFO:teuthology.run:FAIL